Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Uriah]

Hello fellow Outerrians!

With everyone asking about mipmaps and the performance problems they are causing, I thought I would share a method to batch compress all the dds textures in your Anteworld folder.

Download the latest version of TheCompressonator ( http://developer.amd.com/tools-and-sdks/archive/legacy-cpu-gpu-tools/the-compressonator/ )

Assuming you only need to add mipmaps to the diffuse textures (albedo), than you will need to:

Copy both packages folders, /Anteworld/packages where outerra packages are located and /Anteworld_Data/packages where mods are located. Copy both packages folders to some arbitrary location, I used C:/MIPMAP/Anteworld/packages and C:/MIPMAP/Anteworld_Data/packages, to keep the folder names the same, as these are the folders you will batch process and re-merge with the originals. We need to maintain the 'directory structure', of each package folder, so that after batch processing the folders can be merged together, maintaining the directory structure of all the files. This process is necessary to target only the diffuse textures, and not to alter the normal, opacity, roughness, reflection, environ, etc... maps.

Search the directory you copied the package folders to for the 'dds' extension. The result is a full list of all the dds files in both packages folders. I went through the list by hand and deleted all of them that were not diffuse, since we only want to process the diffuse ones remaining. Tips: you can search for different types of maps using common names such as norm.dds, nrm.dds, normal.dds, etc.. to help expedite the process of deleting these. Different artists uses different naming conventions, so you'll have to go through the entire list manually for a final check to make sure ONLY THE DIFFUSE MAPS REMAIN!

Open Compressonator, navigate to and select 'File' tab ---> 'Batch Compression' tool. For processing diffuse maps, I used the following settings. Note: I do not know if Box-Filter is the most optimal, but it worked for me.



NOTE: It is very important that the format of the compression algorithm matches the type of map, as specified by Outerra. Since we are dealing with only diffuse/albedo, the DXT1 format is used.

Code: [Select]

    "name" : "example",                          // material name
    "color" : "1.0,1.0,1.0,1.0",                 // albedo multiplier (in linear space not sRGB)
    "f0" : ".04",                                // F0 reflection coefficient for fresnel equation, range 0-1
    "roughness" : "1.0",                         // roughness parameter range 0-1, 0 - glossy, 1 - matte
    "no_light" : false,                          // if false lights are not computed for this material
    "alpha_masked" : false,                      // true means it's alpha masked geometry pixels below 0.5 are discarded
    "tex_albedo" : "m210_body_color.dds",        // albedo texture must be DXT1 format
    "tex_normal" : "m210_body_normal.dds",       // normal texture must be DXT5/3dc2/ATI2 format
    "tex_roughness" : "m210_body_specular.dds",  // roughness texture (usually negative specular map) must be in BC4/ATI1 format
    "tex_opacity" : "",                          // opacity texture must be in BC4/ATI1 format
    "tex_reflectance" : "",                      // F0 reflection map must be in BC4/ATI1 format
    "tex_environment" : ""                       // local environment map (not used yet) must be in DXT1 format
Batch compression can take anywhere from a few minutes to several hours depending on the size, for me it took nearly 13 hours for 15 GB of data.

Upon completion, back up your original packages folders in both the Anteworld/ and Data/ directories. I like to zip up the folders (download 7zip) and name them packages-backup.zip. This way, if I mess up the original, I can delete the entire directory and copy the backup over from the zip file, which also saves room. It is a convenient and safe method to make sure you don't do something wrong.

Once you've backed up the original packages folders, copy ONLY the new packages folders from MIPMAPS/Anteworld/ and MIPMAPS/Anteworld_Data into the corresponding original directories. packages/outerra/ should go to /Anteworld/, don't confuse them. If the names have not been changed the packages will ask if you want to Merge the Folders, select Yes, and Replace all the files.

If done correctly, all the diffuse/albedo textures should be replaced with mipmapped ones, and none of the other textures should have been modified. You can check the Date Modified to make sure.

IN CONCLUSION: You can always simply open the textures one by one in Photoshop or TheCompressonator, or Gimp, etc..., add mipmaps and save them one by one. For small volumes, I suggest that simple manual method. However, if you are dealing with large numbers of files, the method I have described is the only way I have found to do it. While it took 13 hours to process all the data, it would have taken me many weeks or even months to do that manually.

NOTE: If there is a better tool to search through the copied folders by what format the DDS files have, it would be much easier to delete all the ones that are not DXT1.

Hope this helps!

Best regards,
Uriah

[Acetone]

Seems like a good solution. I've used HiFlyer method with paint.net described here : http://forum.outerra.com/index.php?topic=3003.msg34357#msg34357.
It's pretty quick (a minute for my 1.5 package file) but a few texture files were not properly converted.

[HiFlyer]

Apparently some textures are in an unrecognized format. I suspect some of these user contributed files are converted from Unity.....?

[Uriah]

I tried that, for smaller datasets it works perfectly. However, the features are more limited for massive datasets. And as you just noted HiFlyer, there is no discrimination between DDS file format in either TheCompressonator or Paint.net method. So the following method must be used for large batches.

This method is very reliable but requires considerably more manual labor.

Since albedo uses DXT1, normal uses DXT5, and the rest use ATI1, I batch compress the entire package folder with DXT5, for normals. I copied the package folder, deleted all the normal map files from it manually, batch compressed that with ATI1 for roughness, reflectance and opacity, and marged it with the original packages folder. Then I merged the DXT1 diffuse packages folder with the original packages folder and it worked perfectly.

The process:
Step 1: Original packages folder, un-mipmapped
Step 2: Original packages folder, batch compress DXT5 format for normals
Step 3: Copy 1 of packages folder, deleted all normal files
Step 4: Copy 1 of packages folder, batch compress ATI1 format for rough, opac, refl
Step 5: Copy 2 of packages folder, delete all norm, rough, opac, refl files
Step 6: Copy 2 of packages folder, batch compress DXT1 format for diffuse
Step 7: Merge Copy 1 with Original packages folder, Replace All
Step 8: Merge Copy 2 with Original packages folder, Replace All

Results:

Maintains perfect directory structure
Texture types have correct format (DXT1, DXT5 or ATI1)
Mipmaps can be optimized, and advanced parameters adjusted

By the way, to batch compress an entire directory using TheCompressonator like I'm doing, you must hit 'Compress Tree'.

Regards,
Uriah

[HiFlyer]

I'm just a bit concerned a bit that all of this do-it-yourself mip correction will lead to a mishmash of versions of various files floating around out there, some correctly mipped, some not, leading to general confusion, pink-panther-syndrome, and degraded performance for new/inexperienced users.

Hopefully the 2 model collection sites will:

1) Be looking for files to be in OTX format from now on

2) Be looking for correctly mipped submissions from now on.

To start getting this back under control.

It should be noted though that only a very few packages are causing issues with unrecognized formats. Specifically Morikubos buildings, Deathreavers sign, And the famous Outerra balloon (can't remember who made that one)

The vast majority of files converted well in paint.net, so it remains a viable option for those seeking a very quick fix.

Good to have alternatives for getting this done.

[Uriah]

 

[aWac9]

there were some models that I could not until I tried to "mipmap manager", but this solved.
I have been helpful .. thanks