Terrain Texturing

A quieter space for design discussion of long-term projects
FluffyFreak
Posts: 1343
Joined: Tue Jul 02, 2013 1:49 pm
Location: Beeston, Nottinghamshire, GB
Contact:

Terrain Texturing

Post by FluffyFreak »

Well apparently I let myself get goaded into trying to implement terrain texturing despite knowing that the height normalisation would bite me on the arse.

To quote myself from the other thread:
FluffyFreak wrote:Ah ha! Starting to get there now!

Image

Image

Looks like my height and slope calculations were wrong for a start, also the indexing LUT texture will need to be customised / rebuilt.

Ah but sweeet sweet progress! :)
FluffyFreak
Posts: 1343
Joined: Tue Jul 02, 2013 1:49 pm
Location: Beeston, Nottinghamshire, GB
Contact:

Re: Terrain Texturing

Post by FluffyFreak »

Still to do:

* Repeat the texture for lower (/higher?) LODs - basically the highest detail LOD needs to show the texture 1:1, the next highest at 2:1 and so forth.
* Build a new LUT, or a way of automatically building LUTs, that includes water where needed and doesn't where not.
* Build a way of building the texture atlases, not all terrain has plant life and green grass!
- Create a library of texture patches to go into the generated Atlas textures.
* Encode the specular into the alpha channel so that water, ice, rocks, dirt, sand etc can all have different specular values.
* Decide if we need or want normal maps - it would let me get rid of the normal from the terrain vertex and reduce the vertex size to 32bytes again as well as possibly looking much nicer.
* What else can we encode into texture data that would be useful?
* ...
* Profit!

Brain's gone blank, stupid day job taking all of my thinkening powers!
impaktor
Posts: 994
Joined: Fri Dec 20, 2013 9:54 am
Location: Tellus
Contact:

Re: Terrain Texturing

Post by impaktor »

FluffyFreak wrote:Still to do:
* Build a new LUT, or a way of automatically building LUTs, that includes water where needed and doesn't where not.
Water specifically or any kind of fluid?
FluffyFreak
Posts: 1343
Joined: Tue Jul 02, 2013 1:49 pm
Location: Beeston, Nottinghamshire, GB
Contact:

Re: Terrain Texturing

Post by FluffyFreak »

impaktor wrote:Water specifically or any kind of fluid?
Actual proper water, or liquid surface, that's animated and stuff is beyond the scope of this bit of R&D.
This will just be a coloured texture that get's applied to places that have no slope and are at the lowest height - I'll try to make the colour match whatever the liquid on the planet would be, blue water, blue/green water, etc.

In reality though it'll look a lot like our current system does.

In future I hope we can do more with water, allowing heights below 0 (or shifting the base land height upwards or something) so that we can have underwater areas and then having an proper animated water mesh like in Outerra or something.
However that's completely outside the scope of this.
FluffyFreak
Posts: 1343
Joined: Tue Jul 02, 2013 1:49 pm
Location: Beeston, Nottinghamshire, GB
Contact:

Re: Terrain Texturing

Post by FluffyFreak »

Thinking about terrain texturing (https://github.com/pioneerspacesim/pioneer/pull/4554), I've been hacking away at it slowly, I needed textures and remembered this vast resource of them

https://www.deviantart.com/yughues/gall ... 86308/Free

Lots and LOTS of free textures and models!

He also releases some stuff for Unity (https://assetstore.unity.com/publishers/4986) and there's an equivalent page for UE4 stuff somewhere too.
FluffyFreak
Posts: 1343
Joined: Tue Jul 02, 2013 1:49 pm
Location: Beeston, Nottinghamshire, GB
Contact:

Re: Terrain Texturing

Post by FluffyFreak »

Coming up on that yearly update again...

<sturnclaw> fluffyfreak: I honestly think we should do the texturing on the generation thread and make the shader only handle display
<sturnclaw> there's a lot more logic which should go into texturing, e.g. mineral deposits, hot springs, biomes, etc.
<sturnclaw> I'd rather do a virtual-texturing approach, although that requires sparse textures and GL4.5
<sturnclaw> the problem is of course storing material indexes in vertex data in a way that naturally blends
<sturnclaw> fluffyfreak: https://www.gamedev.net/articles/progra ... ain-r3278/
<sturnclaw> I need to figure out exactly what megatexturing is
<sturnclaw> but this looks like a step in the right direction
<sturnclaw> possible cons: time taken to re-generate the megatextures, doesn't cope well with fast flight, possibly high CPU cost
<sturnclaw> fluffyfreak: might have found something worthwhile
<sturnclaw> https://kosmonautblog.wordpress.com/201 ... nd-tricks/
<sturnclaw> Links at the bottom of the article go to Frostbite presentations, still working on reading them
<sturnclaw> https://developer.amd.com/wordpress/med ... ssion).pdf
<sturnclaw> https://media.contentapi.ea.com/content ... field3.pdf
<sturnclaw> better links
<sturnclaw> fluffyfreak: this guy seems to have gotten as far as we have, on SM4.0 hardware to boot! https://www.gamedev.net/blogs/entry/168 ... texturing/
<sturnclaw> We can probably only blend once, and probably only tap the detail texture thrice
<sturnclaw> Calculating the offset might be a problem, but with some clever use of derivatives it should be achievable
<sturnclaw> unfortunately I don't have the shader-fu required to implement it all right now
<sturnclaw> also to add to the stack of papers to read: http://twvideo01.ubm-us.net/o1/vault/gd ... exture.pdf
<sturnclaw> The in-depth slides from GDC-07: https://developer.amd.com/wordpress/med ... aph07).pdf
<sturnclaw> That in particular isn't really useful to our use case, as it involves a lot of offline rendering to generate shader combinations based on terrain tiles
<sturnclaw> , it does raise an interesting point - we may want some
<sturnclaw> ...and that's what I get for messing with line editing shortcuts
<sturnclaw> fluffyfreak: ctrl+F11 also opens the ship debug spawner menu, because I didn't see the shader reload command in Pi.cpp :D
<sturnclaw> fluffyfreak: there's something weird going on with the mipmaps for array textures
<sturnclaw> I'm sampling with textureLod(tex, uv, 9.0) - aka mipmap 10 - and still getting moire' patterns ... even though that should only ever return a single color
<sturnclaw> ...er, 10.0 for mipmap level 10
<sturnclaw> ah
<sturnclaw> MIN_COMPRESSED_TEXTURE_DIMENSION
<sturnclaw> FINALLY
<sturnclaw> FINALLY
<sturnclaw> who in their right mind calls glGenerateMipmaps() on a texture with all of its mipmaps filled out
<sturnclaw> more to the point, who STOPS GENERATING MIPMAP LEVELS WHILE THEY'RE BIGGER THAN 1x1!?
<sturnclaw> fluffyfreak: I apologize if I'm accidentally casting blame on you
<sturnclaw> texture creation absolutely should not be stopping while there's mipmap levels left and the size is non-zero
<sturnclaw> also min_compressed_texture_dimension should be bigger too
<sturnclaw> *smaller
<sturnclaw> 4x4 block is what DXT* et al standardize on
<sturnclaw> testing improvements to TextureGL now, hopefully we can drop MIN_COMPRESSED_TEXTURE_DIMENSION
<sturnclaw> because mipmap levels are required to pad to 4x4 from what I remember of the MS spec
<sturnclaw> maybe it'll break horribly, who knows!
<sturnclaw> found and fixed the moire' pattern bug
<sturnclaw> turns out if you don't upload mipmaps smaller than 16x16, OpenGL can't actually sample from a smaller mipmap when you're using less than 16x16 pixels of screen space
<sturnclaw> hmm... is 0.279 earth radii == 0.279 earth grav accurate? I feel like it isn't
<sturnclaw> among other things, having 0.022 earth masses seems to indicate there's an inaccuracy there
<ecraven> should not be, as volume is not linear with radius
<ecraven> but it all depends on the densities
<sturnclaw> looking at some other things in the system inspector, we're definitely calculating surface gravity as == earth masses
<sturnclaw> that's a bug!
<sturnclaw> fluffyfreak: I think we can get rid of MIN_COMPRESSED_TEXTURE_DIMENSION, unless there's a reason it's there in the first place. Anything coming in a compressed format will have a viable minimum block size down to 1x1, and anything uncompressed won't have mipmaps anyways
<sturnclaw> I think we're actually fairly close to being able to merge this thing, even as a basic first implementation. Blending might be a bit of an issue, but I'm sure I can work my way around it in one way or another
<sturnclaw> GL's MAX_ARRAY_TEXTURE_LAYERS value is required to be 64, so we have plenty of texture slots available for a planet
<sturnclaw> we could even pass uv1 as a vec3, encoding biome index in the third parameter, and use it to index into a 3D lookup texture
<sturnclaw> I'd imagine we won't use more than three or four textures for each biome, so that gives us plenty of wiggle room
<sturnclaw> anyways, I'm heading off early tonight.
<sturnclaw> heh, early. It's 2:30am here :P
* sturnclaw -> Zzzzzz
Papers:
There's a lot to reply to in there so let start with the links:
Some good links I hadn't seen before, the big problem with the Frostbite stuff is that they spend days-per-level baking their data whereas ours must not only be runtime, it must be generated at runtime. Some great ideas in their though, evne if not 100% directly applicable they can still be a source of ideas and methods.

The Ysaneya blog is what became Infinity -> https://inovaestudios.com/ and is where Kimmo (kko in Git Blame) went to work for a while after being with us on Pioneer :) They eventually abandoned the terrain texturing system described in that blog post and switched to texture arrays I think. Ysaneya and his blog was a huge inspiration to me many years ago, especially when I first joined Pioneer and started improving the terrain system.

I'll try to get through some of those papers in the evening.

Texturing bit:
Yes I think generating and figuring out a lot more of the biome and thus blending stuff in the terrain generation jobs would be a good idea.
There's a series of blog posts on http://spaceengine.org/news/blog171016/ that are really good and I think we could benefit from really trying to understand their system as it produces very good results.
part 2 http://spaceengine.org/news/blog171102/
part 3 http://spaceengine.org/news/blog171120/
part 4 http://spaceengine.org/news/blog171230/
part 5 http://spaceengine.org/news/blog180120/
part 6 http://spaceengine.org/news/blog180323/

This is still a splatting based approach, and each patch is basically 5 textures of base texture + 4 splats, that's quite approachable I think.

MipMapping Limitations:
Improvements to the mipmapping are welcome, there's a lot of hacks to keep things working on Intel iGPUs and some for OSX when generating/supplying the lowest(/highest?) mipmap levels which is the cause of some of that shitty code.

It can probably be ignored for now, and we'll just have to test it with recent Linux distro's at some point, then try to fix any problems a new way.
FluffyFreak
Posts: 1343
Joined: Tue Jul 02, 2013 1:49 pm
Location: Beeston, Nottinghamshire, GB
Contact:

Re: Terrain Texturing

Post by FluffyFreak »

FluffyFreak
Posts: 1343
Joined: Tue Jul 02, 2013 1:49 pm
Location: Beeston, Nottinghamshire, GB
Contact:

Re: Terrain Texturing

Post by FluffyFreak »

Splatting ideas
https://kosmonautblog.wordpress.com/201 ... nd-tricks/
The classic reference from 10 years ago http://www.catalinzima.com/xna/tutorial ... eightmaps/
Advanced splatting that I implemented for asteroids generation (unused but as usual, there's a branch)
https://jadkhoury.github.io/terrain_blog.html

Modeling real world Terrain by Ian Parberry
http://ianparberry.com/research/tobler/
Keeper
Posts: 97
Joined: Thu Jan 04, 2018 9:23 pm
Location: Indian Hills, Nevada, USA
Contact:

Re: Terrain Texturing

Post by Keeper »

In the mean time I'd like to increase the LOD distance of the existing terrain generation system on my own installation, so more polygonal detail is drawn at farther distances. Especially as I use a more realistic FOV, the terrain "pop-up" effect is huge and unpleasant. I have the computing power to handle the increased polygons, but I have not been able to find what in the code determines the distance at which terrain detail gets reduced. Any idea?
FluffyFreak
Posts: 1343
Joined: Tue Jul 02, 2013 1:49 pm
Location: Beeston, Nottinghamshire, GB
Contact:

Re: Terrain Texturing

Post by FluffyFreak »

You need to look into where it decides to split or not and adjust that to use a different heuristic
Post Reply