Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[cameni]

Those rivers look good indeed.
In preparation of a new, fixed Earth dataset, I'm currently reworking the tool that maps and compresses external raster data into our internal format suitable for streaming.
Also in the light of this thread I decided to integrate the functionality directly into the engine code, as it will be also necessary for real-time terrain editing and computation.

I'm not sure though how to run the terrain computation algorithms on that thing. You guys talk about arrays of 4096² or 8192² size, but the planet here has got 6 faces of 524288² data points, that's ≈3.3TB of raw data (with 2 bytes per sample, which may not be enough). Even though the continents will be smaller, it's quite unimaginable to run multiple passes of incremental algorithms here and expect it to terminate in reasonable time. I guess an algorithm that first partitions the world into separate drainage regions - much like what Monks posted but not per ocean but per rivers - will be necessary. And that's why I am also more attracted to algorithms that try to generate the resulting appearance, not to emulate the process that led there. Though in the end it will probably be a combination, like you both said.

[monks]

I think I'm understanding what your intentions are more are now. Emulation rather than simulation. I was watching some of your videos and I can see what is obviously Earth data but integrated into different shaped continents- they seemed to be anyway.
 You are going to use Earth data and superimpose your design map on that and change the data where necessary. Then you'll blend in the area using fractal code that will emulate the appearance of the Earth data. Is that what you're going to do Cameni?

 I was considering using Earth data myself. The most difficult feature to emulate is the structure- especially with the present tools. You really have to manually do it. Even the most obvious structures like mountain ranges are very time consuming to create. Most artists either use real world data and integrate that or more or less place fractal masses and erode it. The less
obvious structures are easier to get away with. But in a planetary renderer you can just see the difference between a procedural engine mush, and the real world.
 In terms of laying down the design map stage, I think the contour tool that I suggested where the tool operates on vectors would save oodles of time AND it would guarantee predictable flow, even over the large surfaces (with the help of interpolation algorithms which you would only have to run at bake time). I'm sure that if someone coded it on a gpu, it would run very well in rt, even with larger point sets. Wilbur has a contour shader as does Leveller, that updates with the modelling, but it's the underlying mechanics of painting with pixels and the lighting in 3D, that slows it down. You wouldn't even need a 3D interface, I think 2D (with the occasional bake to 3D) would be easier to work with.
 I've asked Mike at Global Mapper if he'll implement it. He's put it on his to do list but he reckons it won't be that easy to do. I asked for brush pressure sensitivity- and perhaps that's out of his area of expertise.

 World Machine's spline tools are very useful as well but when you're considering modelling a real looking mountain range you need all the help you can get. That's where the watersheds representation could be applied. If you could procedurally generate an L-system (for want of a better description) of ridges/basins, it could potentially make the creation of very large mountain ranges with correct drainage, a possibility. A future version of World Machine may include this tool. You would define your primary backbone rigde using a lofting curve for heights, (possibly define the boundary or footprint of the range- much like a reverse watershed) and then hit generate and grow the network of smaller, lower ridges off that.
 This could be applied to all of the terrain data, not just mountain ranges, Define the watershed area, define the principle river (which would serve as the backbone ridge), hit generate, and you grow from that a network of basins. The program would always have to fill in the other part though to save time- ie the basin networks for the mountains, the ridge networks for the watersheds.  Maybe there are some subdivision routines out there that would fly on the gpu.
 Different terrain morphologies could be created much like different trees are created with L-Systems. Then you get into the idea that you could sample r-w terrain and if you could repesent it internally on this scheme, develop libraries of morphologies. If the sampling was good You wouldn't even need erosion! GeoControl has this feature that lets you generate infinite levels of detail, by zooming in and re-generating the terrain. It breaks down a little bit a after a while (you lose the initial looks of the terrain), but it is really excellent. The program represents terrain in a heirarchical form- well it uses the noise octaves, and stores terrain morphologies as library presets under this scheme. It doesn;t use the watershed
ordering though, but I reckon  he's got the right setup to take it in that direction.


  I think if we're going to have very large terrain creation, usefully predictable hydrology, good intuitive artist tools, we're going to need both contours and the networks idea. To do that we'll probably need the software to be able to sample input terrain and internally represent it as vector networks within an overarching framwework of watershed subdivision.

Apologies for banging on, but this stuff really interests me  

monks

[cameni]

I think I'm understanding what your intentions are more are now. Emulation rather than simulation. I was watching some of your videos and I can see what is obviously Earth data but integrated into different shaped continents- they seemed to be anyway.
 You are going to use Earth data and superimpose your design map on that and change the data where necessary. Then you'll blend in the area using fractal code that will emulate the appearance of the Earth data. Is that what you're going to do Cameni?

No, the continents are shaped alright, those are real Earth data.

And no, I didn't mean to reshape Earth - but it might not be such a bad idea
You are right about creation of all the structures being time consuming, so some reusing and mixing of existing "templates" might be something that can be used in manual mode.

But I meant something different - emulating the resulting appearance of erosion on some artistically sketched terrain by means of fractal generation. Much like how you describe the reverse watershed area generation and refinement. Watersheds for deeper levels (of river branching) will be also creating additional mountain ridges. Original idea was to sketch the contours of continents, place the estuaries for major rivers and have the generator generate the rivers upstream while also modifying terrain elevation along the way. With many more possible additions, like river branching depending on land type it's passing through, ability to place mountain range attractors so that the rivers are drawn towards it, carving the mountain valleys and peaks with the last river levels and so on.

[monks]

Sounds like a good plan. Contours are so useful for obvious reasons. You are only defining a tiny subset of the surface. Interpolation and gridding them is the big time saver. You are guaranteed gradients that you couldn't paint in greyscale or indexed colour very easily if at all- certainly not in 16 bit greys. It's much better than the paint and blur approach. You always get stepping with painting.
 You'll be able to set up good conditions for your main flows as well.

 If you're coding an interface for the contour drawing stage You might want to code automatic z tagging of drawn contours. Tagging vectors can be very time consuming given enough of them. You could set up a dialog that automatically tagged all contours with the same height until told otherwise or tagged them incrememntally via a hot key. Tagging mutiple contours of the same height with selction and ctrl + mouse but also speed things up.

 The traversal upstream sounds pretty cool if you can pull it off. You might want to program in more meandering in the floodplains. You aso might want to observe exponential fall offs as well with curves wthin the terrain- rather than a linear fall-off. Like you say, there are a number of parameters you might want to include.

 Apprarently there was a SIGGRAPH paper on creating terrain by the calcualting rivers first. It focussed on a single watershed, which is at least half the problem solved if you divide things up on the watershed scheme. I've never read it though and unfortunately I don't know who the author author was.
 After a quick look I found this:
 http://www.springerlink.com/content/kxw0uwt010p80331/

 http://www.slidefinder.net/R/RiverLand_Efficient_Procedural_Modeling_System/29957735

 ppt available here with some references:
 http://www.cs.sjsu.edu/~teoh/research/presentations/

 I see Howard Zhou gets a citation. I tested his Terrain Synthesis plugin for World Machine.  

 One idea is to allow the user to input morphology maps- atlas- that could apply to both simulated erosion algorithms and also your own approach. Any synthesis done within the map areas would influence the paramaters and therefore the terrain appearance. L3DT uses a Design Map which incorporates this idea for texture creation. As far as I know it doesn't use it for the actual terrain creation.
 Concentrating on the rivers will help to provide the large scale structures. This approach might help with the smaller scale variations too.

 Btw, Brano's news was fantastic!    Cheers guys.

monks

[cameni]

Cool, thanks for the links.
Here's also the pdf: http://www.cs.sjsu.edu/~teoh/research/papers/isvc09.pdf

And hi, I am Brano

[monks]

haha  ...ah you've spoilt my illusion of this huge Dev Team with its different departments.  

monks

[cameni]

Btw today I updated our plugin for XnView image viewer allowing to view raw *.hgt or *.srtm terrain files, with autodetection of image dimensions using neighboring row correlation computation, and more.

Might be handy for some following this thread

[Redrobes]

Thats cool. So if we supply some raw height tiles that render correctly using this plugin then your good to go. Im a bit tied up for a short while but just starting to ease off other projects so I will look at a medem height tile conversion pretty shortly. I have written a few little apps which convert our HF2 or RAW data into PNG and saves them out so its kinda similar.

This is the usual link I get my SRTM data from: http://dds.cr.usgs.gov/srtm/version2_1/  Do you know of a link that has it at any better res than these downloads ?

[cameni]

There are also Viewfinder Panorama DEMs, with links to other sources. A global coverage 3" compilation at rmw.recordist.com.

[SpaceFlight]

There is a satellite mission (TerraSAR-X/TanDEM-X) running right now, to make a very precise 3D map of earth´s surface:

http://www.bbc.co.uk/news/10422511

The seamless DEM of the Earth's surface will be built up over three years of joint operations. Ultimately, it should have a vertical resolution of 1-2m and a spatial resolution of 12m - far superior to any previous global data set.

I suppose the complete DEM will be available in about 3-4 years, but one can check out samples here (I dont know if the format they use is compatible with OT though):

http://www.infoterra.de/free-sample-data

Also, http://www.infoterra.de/ sells the data, so right now, except for the samples, it is not for free.

More info on the mission here:

http://en.wikipedia.org/wiki/TerraSAR-X

http://en.wikipedia.org/wiki/TanDEM-X

edit:
Hmm, 17€ per square kilometer for a basic DSM, I guess that is not an option then.  

[monks]

That is going to be some DEM, but they're never going to give that away. Prices for other dems should drop though.   I thought there was already 10m dems available? Or is that just US coverage?

monks

[Redrobes]

Cameni, just an update and a question. Update first.

We made the 40K map and textured it out but found in the result the light maps were bad so were redoing them. Its off processing again.

The question is this, since you don't take in any textures and generate all the lighting then you are only interested in the height maps. On middle earth tho, we absolutely must have Lorien, Fangorn, and Mirkwood as a thick wooded area in a specific delineated region and places like Mordor as volcanic hot, inhospitable land.

Are you going to have any kind of input masks or control over the climate, temperature, vegetation and so on. We are generating the masks from GeoTerSys but we also have input bias maps for vegetation which force or prevent veg growth. We have a base temperature map we put in and let GTS generate the final adjusted one. But that allows us to dictate where there will be snow or where its artificially hot.

I was looking at your pics of Sahara and its textured with some trees on it. You have great auto generation of trees but are you expecting to input some bias into that system at some point ? A programmable climate model or masked set of climate properties ?

[cameni]

It will use climate/land class input together with the heights. The exact format is not yet determined though.
For Earth there exists a 500m dataset with some 15 discrete values determining land type, like the desert, savanna, woods of different types, up to polar regions.

Tree cover will use the input from land class data to bias the probability up or down for given type, so forests will be forests but the probability computation and fractal behavior will still apply, creating occasional clearings and not placing trees on steep rock walls, and also creating natural transitions to the surrounding land types.

The problem is that those values just tell about the dominant land type. A transition from desert to savanna takes place on some tens of kilometers, with changing ratio between sand/grass surface, but the dataset doesn't tell any of it - it's like if there's more sand than grass, it's a desert.
One possibility is to postprocess these discrete datasets into ones that encode probabilities, but looking at the Earth's dataset shows it won't be that easy - for the transitions between large continuous areas it may work, but at the places where land type changes quickly between grass/woods/swamps it would inappropriately smooth out the details.

Although, with artificial maps this problem usually doesn't occur - they don't have the detail in them in the first place

In addition to land class maps we'll be also processing color maps that bring additional colorization for the same land class, like different shades of sand color.

[Redrobes]

OK thats cool. We can generate those maps I think. We have masks for all sorts of different terrain types and we use those to generate the veg bias map but we could generate something else just as easily.

Have you thought about snow as well. Originally we used to map snow to any height > X but found that it does not look like snow at all. Its especially noticeable when your at altitude X looking across at the mountains and there's a ruler straight snow line. GTS now models temperature flow like water so snow can pour down a mountain side and go below the Height > X line and maintain it being snow since it carries coldness with it. In this way we get glaciers and a wobbly snow line. We extended this idea so that the direction of the sun melts snow on southerly faces too which looks quite nice as well.

We can provide a snow mask or the post processed temperature map showing where snow would be or bake it in as part of the climate / terrain type map. In your engine, calculating that snow fall line is probably going to be hard as is much like a fluid flow problem that needs to be precomputed.

Since we have the colour map we can do a fade transition from deserts to vegetated areas no problem. What we cant do, which your engine does, it to put in 3D trees so your engine produces such good results looking across terrain. Our terrain only looks good in our 3D tools when viewed from a distance where trees and micro detail is lost to the overall terrain shape. Why we like the look of Outerra so much is that you can get close to the ground and let the fractal engine take over.

Since your engine knows about the probability of putting in trees, why not adjust the tree height (and tree style when you get that bit done) as well as the spacing for them. We have seen pics with these lone trees in deserts because presumably there is a non zero chance of a tree. If that tree height were similarly scaled by the probability then you might get the odd stumpy tree instead of one full sized one all on its own. In this way as the probability of going from one terrain type into another changes the trees would turn to scrub and it would appear more natural.

[cameni]

Snow is subjected to a heuristics that basically says this: snow can hold on places that are cold enough and flat enough. So it computes the temperature for a given place, and biases it with the degree of steepness of terrain (the steeper terrain, the higher the computed virtual temperature is). In effect this allows the snow to hold on flatter places in lower elevations, and on gradually steeper parts as the elevation rises, and it almost completely breaks the line. It's also kind of logical, when you think about it.
It looks like this (more at http://outerra.blogspot.com/2010/07/collection-of-screenshots-from.html):



Additionally there's a fractal bias added to break it even more.
Snow this way naturally holds onto the valleys, but we want to help it even more by accounting for terrain curvature as well - "V" shapes will lower the temperature so that the snow can pour down.

Technically, the equation should additionally include precipitation and temperature should be computed also from latitude and be locally adjustable, to account for other effects like warming/cooling from ocean currents and shielding by mountain ranges and such.
But just using the altitude and latitude together with the equation should be good enough for many cases.

As for the fading between land types - what we will ultimately need is a format that carries probabilities of occurrence of land types. A format that says: this is a mixed type, with 70% of dry ground and 30% grass (eventually in a particular period of year). We need this to feed the fractal generator that would produce a fractal mix of land that obeys the probabilities, and looks natural - clumps of grass in a sandy land, in this case.

Since your engine knows about the probability of putting in trees, why not adjust the tree height (and tree style when you get that bit done) as well as the spacing for them. We have seen pics with these lone trees in deserts because presumably there is a non zero chance of a tree. If that tree height were similarly scaled by the probability then you might get the odd stumpy tree instead of one full sized one all on its own. In this way as the probability of going from one terrain type into another changes the trees would turn to scrub and it would appear more natural.

The height of trees is already determined by the probability, there are lower trees at forest borders, for example. You'd immediately notice it if it wasn't the case. Of course, in different biomes the trees will be generated appropriately, so there will be palms in oases and such. Or the stumps here and there, but that will be probably done differently - a low probability generator placing these, I think.