3

Is it possible to run a standalone PyQGIS script using multiprocessing for better performance?

I'm creating 1,800 rendered map images and would like to run it across 4 processes to speed it up.

If I try to instantiate 4 QgsApplication objects using the following code (one in each process) - they fail silently (i.e. nothing happens and no CPU/hard drive/memory is being used).

QgsApplication.setPrefixPath("/Applications/QGIS3.8.app/Contents/MacOS", True)
app = QgsApplication([], False)
app.initQgis()

If I instantiate it once and then run the 4 processes - I get a "QgsApplication couldn't be pickled" error.

The code uses Python's multiprocessing module to create the parallel processes; using pool.imap_unordered

I'm using QGIS 3.8.2 and Python 3.7.3 on a Mac with MacOS 10.14.6 (Mojave).

minus34
  • 757
  • 9
  • 20
  • 1
    Perhaps this post might help: https://gis.stackexchange.com/questions/330255/no-output-in-pyqgis-standalone-script-running-distance-matrix/330801#330801 – Joseph Sep 10 '19 at 10:03
  • Good to know it can be done using bash scripts, however I want to keep in all in a single Python script for simplicity. – minus34 Sep 18 '19 at 03:42

0 Answers0