It would probably be best suited to a mix of different techniques, with maybe a switch to our previous geometry shader technique before switching to 3D objects up close. Currently I don't handle the camera being inside the shell mesh of the forests, as there is no real case where this is problematic (the method isn't usable for trees close to the player).
I have been able to get a lot more performance out of it recently with optimizations that I don't understand fully, but probably due to how the GPU works. Mainly with the loop that traverses the simulated vegetation grid, I had to set a constant iteration amount instead of using the amount of cells traversed that I compute before the loop (as compiler doesn't want a variable amount of iterations), but I break out of the loop once it exceeds that computed amount so I thought performance-wise it would be the same. I had it set to a safe amount that I was sure would be higher than the computed amount of loop iterations. However reducing the constant iteration amount much closer to what the computed amount is on average, got me a huge performance boost, with no degradation to the quality of the tree detection.
I still have a loooot to learn on working with GPUs I guess
I can now run it in full screen at very good framerates for the computer that I use (gtx 850m laptop) depending on the scene , between 30-50fps without shadows, 15-20 with. It still varies wildly though and frametime is very inconsistent. But it seems like a viable method, and it's interesting to see that you were thinking about something similar too
I would be very interested in seeing what you could achieve yourself with this idea, and seeing that in Outerra.
I have also began trying some ideas to get 3D vegetation instead of flat billboards using the same method, by replacing the quad-intersection test with a full raymarching procedure inside a cube representing the bounds of a single tree. I was very surprised to see that it actually runs at the same performance level than the billboard-only method... Which I don't really understand yet. But with this, I can try two different things:
- raymarch a voxelized representation of the tree, stored in a 3D texture. The rendering method isn't the same but I think it could end up looking a lot like this :
http://phildec.users.sourceforge.net/Research/VolumetricBillboards.php. However up until now I used 2D texture arrays to store the different trees' billboard textures, and I would need 3D texture arrays to be able to do this, and these don't exist in either DirectX or Unity's API... So I can't really do that at all at the moment.
- raymarch a standard billboard tree but with a displacement map, to give volume to the tree. I don't know how good the results could be, but it would give the volume that is lacking with the flat billboards. Don't know when I'll have the time to code something to generate the displacement maps (speedtree's billboards only come with color and normal maps)