0

I’m trying to use gdalwarp to merge and reproject many elevation tif files (DEM).

To do so I first create a VRT file from all the tif files and then convert and reproject to one big tif file.

This is an example to the command I use: gdalwarp.exe --config GDAL_CACHEMAX 90% -wm 90% -multi -co NUM_THREADS=ALL_CPUS -wo NUM_THREADS=ALL_CPUS -oo NUM_THREADS=ALL_CPUS -doo NUM_THREADS=ALL_CPUS -of GTiff "input.vrt" "output.tif" -co COMPRESS=DEFLATE -co ZLEVEL=9 -co PREDICTOR=3 -co BIGTIFF=YES -t_srs "EPSG:4326" -wo SKIP_NOSOURCE=YES

This works well since it handles cases there are not enough memory. My issue is that it first prints percentage of work and at the end “DONE”. After that the process is still not completed and there’s memory release. All the memory that was built up is being released VERY slowly and this takes too much time. I’m trying to understand if there’s a way around that and also what does it mean because gdalwarp returns DONE before this happens.

PolyGeo
  • 65,136
  • 29
  • 109
  • 338
Itay
  • 1
  • Often the work that continues after 100% is reached is spend for writing the final result to disk. Have you compared the speed without using any of the advanced parameters? The defaults at least are never very bad. Absolutely create the output as tiled. – user30184 Nov 30 '21 at 06:52
  • Also, I don't know if it is still an issue, but gdalwarp did not used to handlle compression well and the advice was to not warp to GeoTiff with compression, instead warp to another VRT then gdal_translate to GeoTiff with compression. https://gis.stackexchange.com/a/89549/2856 – user2856 Nov 30 '21 at 07:02
  • Adding compressed data incrementally into a huge and growing TIFF file may be difficult for any program but with GDAL that is a known issue https://trac.osgeo.org/gdal/wiki/UserDocs/GdalWarp#GeoTIFFoutput-coCOMPRESSisbroken. – user30184 Nov 30 '21 at 07:24
  • Seems like it is the compression. Removing the compression solves this issue. I did try to use gdal_translate next to compress but it takes a lot of time as well. – Itay Nov 30 '21 at 08:28
  • Changing the compression from DEFLATE to LSZ reduced the processing time significantly. – Itay Nov 30 '21 at 14:55

0 Answers0