1

I have a Python application that runs some QGIS processing tools. The tools are called in a loop and I am looking to clean up the tool outputs (shapefiles) in between each iteration. I have found that only a call to qgs.exitQgis() releases the locks on the shapefile (the output of the snap process). But if I try to do further processing or try to initialize QGIS again, the Python execution stops, no exception. I have tried permutations of this but it seems any calls to QGIS or processing after qgs.exitQgis() stops the execution.

I assume it is the layer registry that has a lock on the shapefile and that exitQgis clearing of the layer registry is releasing the lock. I also suspect clearing the provider registry breaks everything else...

How can I release the lock so I can delete the intermediate files. (Some intermediate layers are inputs to OGR processes, so I don't want to create them in memory, I want to be able to open them as OGR layers).

I got this far looking at Using QGIS Processing algorithms from PyQGIS standalone scripts (outside of GUI) and other posts on StackExchange.

import os
import sys
import stat
import shutil
from osgeo import gdal #used for other things not in this sample
from qgis.core import (
     QgsApplication, 
     QgsProcessingFeedback, 
     QgsVectorLayer
)
from qgis.analysis import QgsNativeAlgorithms
QgsApplication.setPrefixPath("C:\Program Files\QGIS 3.30.3\apps\qgis\bin", True)

#sys.path.append('C:/Program Files/QGIS 3.30.3/apps/qgis/python/plugins') #this is handled in a shell or batch script calling this py
import processing 
from processing.core.Processing import Processing


def delFolder(folder):
     def on_rm_error(_action, path, _exc):
              os.chmod(path, stat.S_IWRITE)
              os.remove(path)


 if os.path.exists(folder) and os.path.isdir(folder):
      shutil.rmtree(folder, onerror=on_rm_error)
      print(f'removed directory {folder}')




def main():
     try:


      qgs = QgsApplication([], False)
      qgs.initQgis()
      Processing.initialize()
      inputlayer =r'C:/temp/temp/inputPoly.shp'
      outputPath = r'C:/temp/temp/tmp'
      snaplayer=r'C:/temp/temp/tmp/polysnap.shp'
      dissolvelayer=r'C:/temp/temp/tmp/polydiss.shp'

      os.makedirs(outputPath, exist_ok = True)

      print("snap")
      processing.run("native:snapgeometries", {'INPUT':inputlayer,'REFERENCE_LAYER':inputlayer,'TOLERANCE':1,'BEHAVIOR':0,'OUTPUT':snaplayer})
      print('snapped')

      print('dissolve')
      processing.run("native:dissolve", {'INPUT':snaplayer,'FIELD':['POLY_TYPE'],'SEPARATE_DISJOINT':True,'OUTPUT':dissolvelayer})
      print('dissolved')

      Processing.deinitialize() #tried with and without this
      qgs.exitQgis() #releases the snap lock, the dissolved shapefile doesn't seem locked at all
      #qgs.exit() #doesn't seem to make a difference for the lock

      print(f'try to delete temp folder {outputPath}')
      delFolder(outputPath)

      #do the same processing for the sake of the test  
      os.makedirs(outputPath, exist_ok = True)
      print(f"{outputPath} made")

      qgs.initQgis()  #script ends here, tried creating a new qgs but I think it is singleton
      print('qgsApp initialized')
      Processing.initialize() #tried with and without
      print('processing initialized')

      #do again for test
      print("snap")
      processing.run("native:snapgeometries", {'INPUT':inputlayer,'REFERENCE_LAYER':inputlayer,'TOLERANCE':1,'BEHAVIOR':0,'OUTPUT':snaplayer})
      print('snapped')

      print('dissolve')
      processing.run("native:dissolve", {'INPUT':snaplayer,'FIELD':['POLY_TYPE'],'SEPARATE_DISJOINT':True,'OUTPUT':dissolvelayer})
      print('dissolved')

      Processing.deinitialize()
      qgs.exitQgis()
      qgs.exit()

      print(f'try to delete temp folder {outputPath}')
      delFolder(outputPath)

 except Exception as e:
      print(e)




if name == "main":
     main()
Vince
  • 20,017
  • 15
  • 45
  • 64
Alex Gray
  • 69
  • 4

1 Answers1

2

Ok more reading and tinkering I found a solution so I will post it for others. I had to create the QgsVectorLayers and add them to the QgsProject and remove all the layers from the QgsProject. I also had to get the ProcessingContext from the process and temporaryLayerStore().removeAllMapLayers(). Not sure it was necessary to do all the layers and all the processing contexts but I see no downside to doing each one. This avoided using exitQgis() before I wanted to.

print("snap")
          snapcontext  = QgsProcessingContext()
          processing.run("native:snapgeometries", {'INPUT':inputlayer,'REFERENCE_LAYER':inputlayer,'TOLERANCE':1,'BEHAVIOR':0,'OUTPUT':snaplayer}, context=snapcontext)
          print('snapped')

      disscontext = QgsProcessingContext()
      print('dissolve')
      processing.run("native:dissolve", {'INPUT':snaplayer,'FIELD':['POLY_TYPE'],'SEPARATE_DISJOINT':True,'OUTPUT':dissolvelayer}, context=disscontext)
      print('dissolved')


      lyrSnap = QgsVectorLayer(snaplayer, 'snaplayer', 'ogr' )
      lyrDiss = QgsVectorLayer(dissolvelayer, 'disslayer', 'ogr' )
      QgsProject.instance().addMapLayer(lyrSnap) #this is the layer that had the lock
      QgsProject.instance().addMapLayer(lyrDiss) #found this was unnecessary but I think if the dissolve layer was an input to something else it would

      QgsProject.instance().removeAllMapLayers()
      QgsProject.instance().clear()
      del lyrSnap, lyrDiss #works without this but examples showed doing this...

      snapcontext.temporaryLayerStore().removeAllMapLayers() #this didn't seem to fix it, I think the lock is caused by being an input to the dissolve, if I was trying to remove the input to snap, this might do it
      disscontext.temporaryLayerStore().removeAllMapLayers() # this removed the lock

      print(f'try to delete temp folder {outputPath}')
      delFolder(outputPath)
Alex Gray
  • 69
  • 4