17

I would like to speed up the process corresponding to this command:

gdalwarp -t_srs EPSG:4326 -overwrite input.ntf output.tif

Is there a way to use parallel processing in GDAL? Semi-offtopic: If not, do you recommend a non-GDAL solution to speeding up a gdalwarp-like process?

Here are the web pages I've looked at:

Edit: I asked this question because I thought I saw my CPU% stay below 100% when processing gdalwarp. However, on a second look, it reached 555.5%.

Matt Kleinsmith
  • 421
  • 3
  • 7

1 Answers1

15

Yes, GDAL supports parallel processing, and this support applies to gdalwarp by default.

Use the -multi option with gdalwarp to enable multithreading, as opposed to only multiple cores.

Details:

Without -multi: 33.849s, and the CPU reached 555%. (multiple cores)

With -multi: 23.377s, and the CPU reached 700%. (multiple cores and multiple threads)

Raster size: 34721 x 20453, OS: Ubuntu 16.04, # Cores: 6, # Threads: 12

Credit: user30184 mentioned the -multi option in a comment.

Documentation: gdalwarp

Matt Kleinsmith
  • 421
  • 3
  • 7
  • 2
    See the improved documentation (by https://trac.osgeo.org/gdal/changeset/38196) of gdalwarp -multi: Use multithreaded warping implementation. Two threads will be used to process chunks of image and perform input/output operation simultaneously. Note that computation is not multithreaded itself. To do that, you can use the -wo NUM_THREADS=val/ALL_CPUS option, which can be combined with -multi. See also http://www.gdal.org/structGDALWarpOptions.html#a0ed77f9917bb96c7a9aabd73d4d06e08 – user30184 May 07 '17 at 09:24
  • Does this work with gdal_rasterize? – jzadra Jan 05 '21 at 02:38