Outerra forum

Outerra Engine => Off Topic => Topic started by: theFrenchDutch on August 28, 2016, 06:30:35 am

Title: Terrain engine project I’m working on, inspired by Outerra
Post by: theFrenchDutch on August 28, 2016, 06:30:35 am
Hi :)

You might have seen a year and a half ago, I asked a few questions on this forum about how Outerra works and Cameni helped us a lot back then by answering them, when we were working on a school project about procedural planet generation using Unity.

After the six months duration of the school project, we dropped the planetary aspect to focus on large scale flat terrains for a game project we had, and I’ve kept working on it since then (for almost two years now). I’m creating this thread to share our results and to thank Cameni for helping us and for his posts on the outerra blog, as well as to thank the Outerra team for making Outerra as it has been a huge source of inspiration for us during the entire project. Here’s a link to a presentation page with a few screenshots and details on the methods we used.
http://newheadstudio.com/DevBlog/Procedural_Generation/Articles/article1.html (http://newheadstudio.com/DevBlog/Procedural_Generation/Articles/article1.html)

I’ve learnt a huge amount while working on this as we started this project as students with almost no prior knowledge in the field of computer graphics, and I still have a lot to learn, but Outerra has been one of my consistent learning and motivation source. So I really wanted to thank you for this :) Thank you for reading and have a nice day !
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: 2eyed on August 28, 2016, 07:37:19 am
Man, looks like a real competitor :). Good work!
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: aWac9 on August 28, 2016, 11:04:18 am
I do not see images of rivers .. physics applied in rivers?
seems an excellent job ... you have a video?
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: theFrenchDutch on August 28, 2016, 12:04:45 pm
Thanks for the feedback :)

And sorry about the confusion, the "(In development) Terrain editing, roads, dirt paths, rivers" part in the page wasn't saying that we have already worked on rendering rivers and lakes, but we are working on the system which will be used to place them (the same system used to place the roads in some of the screenshots). This system is still also very much a work in progress. In the future, we plan on trying to implement water rendering with lakes and rivers, but we haven't started anything on this. I've corrected the phrasing, thanks :)

We will soon upload a video when we find the time, mainly to show how it runs, I'll link to it here when we've done it !
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: Lewin on August 28, 2016, 12:10:06 pm
It seems the students will soon become teachers :)
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: HiFlyer on August 28, 2016, 04:51:10 pm
I've always said outerra couldn't behave as if the rest of the world was standing still on this........  =|

Tick tock, tick tock.
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: cameni on August 28, 2016, 05:38:15 pm
After the six months duration of the school project, we dropped the planetary aspect to focus on large scale flat terrains for a game project we had, and I’ve kept working on it since then (for almost two years now). I’m creating this thread to share our results and to thank Cameni for helping us and for his posts on the outerra blog, as well as to thank the Outerra team for making Outerra as it has been a huge source of inspiration for us during the entire project. Here’s a link to a presentation page with a few screenshots and details on the methods we used.
http://newheadstudio.com/DevBlog/Procedural_Generation/Articles/article1.html (http://newheadstudio.com/DevBlog/Procedural_Generation/Articles/article1.html)

Good job! Glad to be the inspiration for you. We have been asked several times if it was possible to make a terrain generator usable for other engines like Unity, but the planetary scale and all the issues with it make it rather hard even if we wanted to go that route. But it may be a good way for you, I noticed people on Reddit were interested too, though I'm not sure what other obstacles are there ...
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: theFrenchDutch on August 29, 2016, 09:04:05 am
Thanks :) Yes I agree, but the scope of our project is much smaller than Outerra's, we only aim to provide flat environments for a video game with this. There's no dynamic download of DEM data to allow the player to go around the entire earth, etc, (which seem to me very complicated to implement and maintain), and all the other stuff Outerra offers as a software. I don't know if it would be feasible to get it all working inside Unity.

About issues with Unity, we ourselves have faced some limitations which we had to work around, some others that we can't do anything about... The Unity API not being thread safe is the biggest drawback, there is some stuff I would much prefer to be doing in my worker thread instead of having to let the main thread do it (creating and filling up meshes with data, creating mesh colliders). There is also no way for us to have the terrain data existing only on VRAM and being rendered from there (which is I believe what Outerra does ?) we are forced to get the generated terrain data arrays back from the GPU to create Unity Mesh objects to be able to render the terrain, which also uses a lot of RAM. We had some trouble with Unity's old implementation of Mono which uses an old, inefficient garbage collector, and we'd much prefer to be handling the data with pointers ourselves at this point. Unity also uses by default a traditional depth buffer which mostly give us unstable shadows with their cascaded shadow map system with huge distances (but implementing a reverse-float depth buffer is on their road map so fingers crossed, or we could do it ourselves as I've seen others do)

But it is still an incredbly flexible engine, I'm amazed by what it let us do with it to this day. ShaderLab also makes for very easy shader learning and programming. But I'd say that something entirely like Outerra within Unity would be a difficult thing to achieve.

Edit: I've also uploaded a video (which hardly survived youtube compression even though it looked good before uploading...) if anyone wants to see it in movement :
https://www.youtube.com/watch?v=AoinEU3iQ64 (https://www.youtube.com/watch?v=AoinEU3iQ64)
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: HiFlyer on August 29, 2016, 10:11:54 am
If shown this without being told beforehand that this was not Outerra, I would swear it was Outerra.
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: 2eyed on August 29, 2016, 11:23:02 am
I've always said outerra couldn't behave as if the rest of the world was standing still on this........  =|

Tick tock, tick tock.
Calm down, OT is still ahead and it's nice to see others taking the route as well.
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: theFrenchDutch on August 29, 2016, 11:26:52 am
I've always said outerra couldn't behave as if the rest of the world was standing still on this........  =|

Tick tock, tick tock.
Calm down, OT is still ahead and it's nice to see others taking the route as well.

Yes, Outerra is way ahead of what we have achieved so far, we hope to one day have something running as great as it. Besides, what we have is only a flat terrain generator for Unity, the Outerra team are building an entire engine for entire Earth simulation ! I've seen nothing else come close to what they have achieved yet.
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: HiFlyer on August 29, 2016, 11:38:39 am
I've always said outerra couldn't behave as if the rest of the world was standing still on this........  =|

Tick tock, tick tock.
Calm down, OT is still ahead and it's nice to see others taking the route as well.

My point stands. Technology doesn't stand still, and the gaming scene is full of engines that were amazing when they first appeared, but where just one of the crowd by the time they were finally released.

A good example might be Infinity Battlescape, that was once astonishing, but ended up making its final push for recognition just as Elite Dangerous and Star Citizen sucked all of the air out the "Look! you can actually seamlessly land on planets!" room.

No interesting technology exists in a vacuum, and no amount of well wishing (and I am a fan) should fail to keep that in mind.
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: 2eyed on August 29, 2016, 03:15:10 pm
Don't get me wrong, I too wished a much faster development, but OT schooled me in patience.
They probably would need a few more talented coders sharing the same vision to speed things up.
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: bomber on August 30, 2016, 01:36:32 pm
it's a bit off topic discussing Outerra's development yet strangely correct in a way.

Quote
They probably would need a few more talented coders sharing the same vision to speed things up.

If I was to offer one single bit of advice to the Outerra team, and that would be to think 24hours... that's the number of development hours in a day they should be aiming for.

At present we naturally think of getting up and going to work doing our 8 hours or so and then coming home... I'm a firm believer in work life balance. Yes you can work like a maniac, been there, done that.... but I've also had a nervous breakdown. So I know you can work full on in your 20's and early 30's but real burn out is a real issue and once you've thrown the jigsaw puzzle onto the floor, you'll never put it back perfectly.

So I don't advise working harder, pushing the limits I advise working smarter....

Build a Design House that acts like a relay team going around the athletics track (no not taking steroids), but working full pelt and then handing the baton on... A design house that chases the sun around the planet, don't put all your eggs into a European based design house working 8 hours and then stopping as you go home.. but instead build one in which design momentum is constantly rolling.

take it or leave it, it's just advise.

Simon
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: CrazyMemer on August 30, 2016, 02:11:22 pm
If I was to offer one single bit of advice to the Outerra team, and that would be to think 24hours... that's the number of development hours in a day they should be aiming for.

At present we naturally think of getting up and going to work doing our 8 hours or so and then coming home... I'm a firm believer in work life balance. Yes you can work like a maniac, been there, done that.... but I've also had a nervous breakdown. So I know you can work full on in your 20's and early 30's but real burn out is a real issue and once you've thrown the jigsaw puzzle onto the floor, you'll never put it back perfectly.

So I don't advise working harder, pushing the limits I advise working smarter....

Build a Design House that acts like a relay team going around the athletics track (no not taking steroids), but working full pelt and then handing the baton on... A design house that chases the sun around the planet, don't put all your eggs into a European based design house working 8 hours and then stopping as you go home.. but instead build one in which design momentum is constantly rolling.

take it or leave it, it's just advise.

Simon

http://i.imgur.com/inVz20S.mp4
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: Occams Razer on September 02, 2016, 12:28:21 am
Don't get me wrong, I too wished a much faster development, but OT schooled me in patience.
They probably would need a few more talented coders sharing the same vision to speed things up.

I've had half a mind in the past to offer 'donations' of my skills and time to various projects that I wanted to see succeed or succeed more quickly, volunteering to help with development. Though the list does include Outerra, I only really know 3D and art, which isn't particularly in demand among the Outerra devs at present.
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: theFrenchDutch on November 06, 2016, 01:37:22 pm
Quick update to show something I've been working on on and off since a few months : a try at a different rendering technique for vegetation with ray-casting. I was previously using a geometry shader, but it was very limiting in this project in terms of vegetation distance and draw distance when wanting realistic sceneries. This new method makes it possible but is still very heavy, depending on the screen render resolution : http://thomasdeliot.wixsite.com/blog/single-post/2016/11/05/Rendering-infinite-forests-with-raycasting (http://thomasdeliot.wixsite.com/blog/single-post/2016/11/05/Rendering-infinite-forests-with-raycasting)

I think it has potential though. Unfortunately my studies currently leave me very little time to work on it as much as I'd like too, for a few months more still. Outerra does feature an impressive draw distance with trees already, and I'm wondering if the solution you use is something similar to the "standard" technique  with a geometry shader generating quads facing the camera ?
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: cameni on November 09, 2016, 04:05:39 pm
It's an interesting technique, I planned to use it to extend the distance of tree cover. As you point out in the blog, this would be less advantageous for sparser vegetation, so I guess maybe a compromise or a mixed technique could be better. We know the number of trees produced in tiles and can draw just the non-empty ones. Hmm, maybe the grid could be generated by geometry shader as well, and only for cells with trees (nearby). How are you handling camera view from below the tree line?

Yes, OT currently uses a geometry shader generating quads.
Title: Re: Terrain engine project I’m working on, inspired by Outerra
Post by: theFrenchDutch on November 19, 2016, 01:41:27 pm
It would probably be best suited to a mix of different techniques, with maybe a switch to our previous geometry shader technique before switching to 3D objects up close. Currently I don't handle the camera being inside the shell mesh of the forests, as there is no real case where this is problematic  (the method isn't usable for trees close to the player).

I have been able to get a lot more performance out of it recently with optimizations that I don't understand fully, but probably due to how the GPU works. Mainly with the loop that traverses the simulated vegetation grid, I had to set a constant iteration amount instead of using the amount of cells traversed that I compute before the loop (as compiler doesn't want a variable amount of iterations), but I break out of the loop once it exceeds that computed amount so I thought performance-wise it would be the same. I had it set to a safe amount that I was sure would be higher than the computed amount of loop iterations. However reducing the constant iteration amount much closer to what the computed amount is on average, got me a huge performance boost, with no degradation to the quality of the tree detection.

I still have a loooot to learn on working with GPUs I guess  =D I can now run it in full screen at very good framerates for the computer that I use (gtx 850m laptop) depending on the scene , between 30-50fps without shadows, 15-20 with. It still varies wildly though and frametime is very inconsistent. But it seems like a viable method, and it's interesting to see that you were thinking about something similar too :) I would be very interested in seeing what you could achieve yourself with this idea, and seeing that in Outerra.

I have also began trying some ideas to get 3D vegetation instead of flat billboards using the same method, by replacing the quad-intersection test with a full raymarching procedure inside a cube representing the bounds of a single tree. I was very surprised to see that it actually runs at the same performance level than the billboard-only method... Which I don't really understand yet. But with this, I can try two different things:
- raymarch a voxelized representation of the tree, stored in a 3D texture. The rendering method isn't the same but I think it could end up looking a lot like this : http://phildec.users.sourceforge.net/Research/VolumetricBillboards.php (http://phildec.users.sourceforge.net/Research/VolumetricBillboards.php). However up until now I used 2D texture arrays to store the different trees' billboard textures, and I would need 3D texture arrays to be able to do this, and these don't exist in either DirectX or Unity's API... So I can't really do that at all at the moment.
- raymarch a standard billboard tree but with a displacement map, to give volume to the tree. I don't know how good the results could be, but it would give the volume that is lacking with the flat billboards. Don't know when I'll have the time to code something to generate the displacement maps (speedtree's billboards only come with color and normal maps)