3

My goal is to create a standalone PyQGIS script that is able to run QgsProcessingAlgorithm scripts without GUI. My setup runs on Windows and my IDE of choice is VS Code.

I've followed the Windows instructions from the official documentation and added those environment variables to my system.

I'd like to be able to simply run the following two lines from VS Code:

from qgis.core import *
import processing

without getting into problems. This looks easy, right? Wrong. First thing you run into when running this code is that it is missing PyQt.

No module named 'PyQt5.QtCore'

There is no mention of this in the documentation, so naturally you might be inclined to think: okay, let's install that using pip.

Now the first line from qgis.core import * runs fine, but the second doesn't.

No module named 'processing'

To let this work, apparently a lot more environment paths (again undocumented) need to set. So with the help of @Kadir's answer I created the following batch file:

SET OSGEO4W_ROOT=C:\OSGeo4W
call %OSGEO4W_ROOT%\bin\o4w_env.bat

path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\Qt5\bin
path %PATH%;%OSGEO4W_ROOT%\apps\Python39\Scripts
set QGIS_PREFIX=%OSGEO4W_ROOT%\apps\qgis
set PYTHONHOME=%OSGEO4W_ROOT%\apps\Python39
set PYTHONPATH=%OSGEO4W_ROOT%\apps\qgis\python;%OSGEO4W_ROOT%\apps\qgis\python\plugins;%PYTHONPATH%
set QT_QPA_PLATFORM_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\Qt5\plugins\platforms
set QT_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\qgis\qtplugins;%OSGEO4W_ROOT%\apps\qt5\plugins


set GDAL_FILENAME_IS_UTF8=YES
set VSI_CACHE=TRUE
set VSI_CACHE_SIZE=1000000

Next I restarted VS Code and executed the initial two lines again. But immediately got into problems. This time PyQT complains:

qgis._gui cannot import type '����' from PyQt5.QtCore

After looking at various posts here and reported issues on GitHub, it turns out that that error represents a mismatch between the just installed PyQt5, and the one that is apparently shipped with QGIS itself.

Long story short: don't use pip to install PyQt5. So I uninstalled it and retried. Unfortunately I once again got the error:

No module named 'PyQt5.QtCore'

even after running that batch file. So I found another batch file, but that one didn't help either. I ended up with the same No module named 'PyQt.QtCore' error and after fiddling with combining the two batch files I even managed to get this error:

No module named 'osgeo'

somewhere deep in the import processing line. And that's when I concluded that solving this problem is way beyond my capabilities.

Would someone be able to write a step-by-step instruction on how to let those two lines of code run just fine in VS Code, from a fresh installation of QGIS?

Mar Tijn
  • 503
  • 3
  • 10
  • 1
    Are you even using OSGeo4W and are all those paths correct in regards to your local setup? Did you launch VSCode or your other interpreters using such a bat file? Where exactly did you install what? I'd recommend starting from scratch and go in really tiny steps, not installing anything but trying to understand what happens and what the bat lines do. – bugmenot123 Jan 18 '22 at 12:28
  • Did https://gis.stackexchange.com/questions/292024/how-to-setup-visual-studio-code-windows-for-pyqgis-2-or-3 not work? – bugmenot123 Jan 18 '22 at 12:28
  • 1
    Using the original setup, as described in the official documentation + the pip install of PyQt5, I was able to execute scripts, but only partially. Once I tried to call import processing, all went downhill as described above. Yes I tested my paths, and verified them by calling 'set' from the commandline, before opening VS Code. I reinstalled OSGeo4W using the default settings. Are there perhaps any settings in the installer that I forgot to enable? I'll uninstall all again and investigate the settings.json solution offered in that post, thanks for this. – Mar Tijn Jan 18 '22 at 12:37
  • Did you open VSCode in the same commandline? – bugmenot123 Jan 18 '22 at 12:41
  • 1
    Of course I did :) I'll focus on the auto-installer + config solution now. Hopefully it's working! – Mar Tijn Jan 18 '22 at 12:44
  • 1
    So far the settings solution doesn't seem to be compatible with the latest OSGeo4W release, as I'm experiencing a number of problems with that configuration. I'll fiddle a bit further. – Mar Tijn Jan 18 '22 at 18:54

2 Answers2

2

Thanks to @bugmenot123 I found an outdated PowerShell script and VS code settings file but after a bit of fiddling, I was able to get it working!

And to ensure future versions of me, and other people who would love to not go through all these loops and hoops and instantly test their scripts in PyQGIS without GUI, I've created a GitHub repository with a simple example.

https://github.com/MarByteBeep/pyqgis-standalone

It doesn't rely on a batch file to launch VS Code and I modified the settings in such a way that it would be relatively easy to update it in the future.

TL; DR;

Should the repo ever get offline, here is a step-by-step version of the answer:

  1. Install the full version of OSGeo4W LTR and put it in C:/OSGeo4W

  2. Create a .env file with the following contents and put it in the root of your VS Code project folder:

# Configure these
OSGEO4W_ROOT=C:/OSGeo4W
QGIS_CONFIG=qgis-ltr
PYTHON_VERSION=Python39

Variables below will be generated from config above


Path


PATH=${OSGEO4W_ROOT}/apps/${QGIS_CONFIG}/bin;${OSGEO4W_ROOT}/apps/${PYTHON_VERSION};${OSGEO4W_ROOT}/apps/${PYTHON_VERSION}/Scripts;${OSGEO4W_ROOT}/apps/qt5/bin;${OSGEO4W_ROOT}/apps/Python27/Scripts;${OSGEO4W_ROOT}/bin;C:/WINDOWS/system32;C:/WINDOWS;C:/WINDOWS/system32/WBem;${OSGEO4W_ROOT}/apps/${PYTHON_VERSION}/lib/site-packages/pywin32_system32;${OSGEO4W_ROOT}/apps/${PYTHON_VERSION}/lib/site-packages/numpy/.libs


Python


PYTHONHOME=${OSGEO4W_ROOT}/apps/${PYTHON_VERSION}
PYTHONPATH=${OSGEO4W_ROOT}/apps/${QGIS_CONFIG}/python;${OSGEO4W_ROOT}/apps/${QGIS_CONFIG}/python/plugins;${PYTHONPATH}


GDAL


GDAL_DATA=${OSGEO4W_ROOT}/share/gdal
GDAL_DRIVER_PATH=${OSGEO4W_ROOT}/bin/gdalplugins
GDAL_FILENAME_IS_UTF8=YES


GeoTIFF


GEOTIFF_CSV=${OSGEO4W_ROOT}/share/epsg_csv


Qt


O4W_QT_BINARIES=${OSGEO4W_ROOT}/apps/Qt5/bin
O4W_QT_DOC=${OSGEO4W_ROOT}/apps/Qt5/doc
O4W_QT_HEADERS=${OSGEO4W_ROOT}/apps/Qt5/include
O4W_QT_LIBRARIES=${OSGEO4W_ROOT}/apps/Qt5/lib
O4W_QT_PLUGINS=${OSGEO4W_ROOT}/apps/Qt5/plugins
O4W_QT_PREFIX=${OSGEO4W_ROOT}/apps/Qt5
O4W_QT_TRANSLATIONS=${OSGEO4W_ROOT}/apps/Qt5/translations
QT_PLUGIN_PATH=${OSGEO4W_ROOT}/apps/${QGIS_CONFIG}/qtplugins;${OSGEO4W_ROOT}/apps/qt5/plugins


QGIS


QGIS_PREFIX_PATH=${OSGEO4W_ROOT}/apps/${QGIS_CONFIG}


Cache


VSI_CACHE=TRUE
VSI_CACHE_SIZE=1000000"
  1. Create a .vscode/settings.json with the following content:
{
    "python.envFile": "${workspaceFolder}/.env",
    "python.defaultInterpreterPath": "C:/OSGeo4W/apps/Python39/python.exe"
}
  1. Create main.py like this and feel free to replace Proximity.py with whatever script you are testing.
from qgis.core import *

qgs = QgsApplication([], False)
qgs.initQgis()


import processing after initializing the application


import processing
from processing.core.Processing import Processing
Processing.initialize()


from scripts.proximity import Proximity


I load the VectorLayer in code and pass it as an argument; obviously you could also just pass the


file name to 'INPUT'. This is just a demonstration that you can have full control over the input


layer before you send it off to the processing script


inLayer = QgsVectorLayer('test/nl_airports.osm|layername=multipolygons')
if not inLayer.isValid():
    raise Exception("Layer failed to load!")


Create the Proximity algorithm


alg = Proximity()


Set the params needed for this algorithm


params = {
    'INPUT': inLayer,
    'DISTANCE': 10000, # in meters
    'OUTPUT': 'test/rasterized.tif'
}


Run the algorithm as you would from inside the QGIS GUI


alg.initAlgorithm()
ctx = QgsProcessingContext()
feedback = QgsProcessingFeedback()
alg.prepareAlgorithm(params, ctx, feedback)
alg.processAlgorithm(params, ctx, feedback)


All done


print("Done")


qgs.exitQgis()
  1. And now launch this main.py from VS Code and off you go.

Enjoy!

Mar Tijn
  • 503
  • 3
  • 10
  • yay! Would be interesting to try if that works as .bat file as well if you would put the launch of vscode in the last line after VSI_CACHE_SIZE=1000000" :D – bugmenot123 Jan 18 '22 at 21:57
  • Thanks for all of this I learned a lot about VSCODE configuration I previously didn't. That said VSCODE is not reading my .env file, so none of the environment variables are ultimately registered and available to use. Even when i side load the env variables and run from osgeo import gdal I still get an error about finding gdalXXX.dll. The only way I found this to work is to run OSGEO4W terminal and edit in VSCODE – Michael Wallace Jun 12 '22 at 20:04
0

Finally, I got a solution to be able to run the "standalone PyQGIS"* example "Proximity" (above mentioned by "Mar Tijn" and provided by "MarByteBeep).

This solution was possible without needing to launch the configuration file "launch.json" as described in the "README.md" file (under the heading: "run example").
I just first added the following two code-lines (see here line 2 and 3) in the python file "main.py" so as to be able to use the plugin "PROCESSING" (initially line 8 of the "main.py" file), then I store it and finally I directly ran it.

Line 1: from qgis.core import

Line 2: import sys
Line 3: sys.path.append('C:\Program Files\QGIS 3.24.1\apps\qgis\python\plugins')

Line 4: qgs = QgsApplication([], False)
Line 5: ...

. * By "Standalone PyQGIS" I refer to code/scripts that can be run outside the QGIS-GUI (=> QGIS-Desktop/Server Application). In my case under the external Editor VS Code

Rick Atb
  • 1