Selecting only N records from layer using PyQGIS

Question

I would like to join attributes by location of 2 layers. My INPUT layer has 446k records and it is too many. I would like to divide my layer and select for example 10000 records.

For example in Python if I have a list I can select by data[:10000]. How can I select x records from layer?

My code:

import sys
from qgis.core import *
from qgis.utils import *
from qgis.analysis import QgsNativeAlgorithms
from PyQt5.QtCore import QVariant
from qgis.core import QgsApplication, QgsProcessingFeedback, QgsRasterLayer
sys.path.append('/usr/lib/qgis')
sys.path.append('/usr/share/qgis/python/plugins')
os.environ["QT_QPA_PLATFORM"] = "offscreen"
QgsApplication.setPrefixPath(r'/usr/bin/qgis', True)
qgs = QgsApplication([], False)
qgs.initQgis()
import processing
from processing.core.Processing import Processing
Processing.initialize()
QgsApplication.processingRegistry().addProvider(QgsNativeAlgorithms())
feedback = QgsProcessingFeedback()
processing.run("native:joinattributesbylocation", {
                'INPUT': r'/home/gis/budynki_opolskie/budynki_opolskie/bdot10k_bubd_a_opolskie.gpkg|layername=bubd_a',
                'JOIN': r'/home/gis/adresypolska/PRG_PunktyAdresowe_POLSKA.shp',
                'PREDICATE': [1,5],
                'JOIN_FIELDS': ['PNA','SIMC_id','SIMC_nazwa','ULIC_nazwa','Numer'],
                'METHOD': 0,
                'DISCARD_NONMATCHING': True,
                'PREFIX': '',
                'OUTPUT': r'/home/gis/zlaczoneopolskie.shp'
                })

The code joins attributes of 2 layers, but first layer has 446k records and it takes too much time. I would like to select 10000 records from my first layer and then run code. — datasciencebegginer, Nov 18 '20 at 11:17
Please, do not forget about "What should I do when someone answers my question?" — Taras, Mar 14 '22 at 06:47

score 4 · Answer 1 · edited Jun 22 '22 at 06:40

There doesn't seem to be a direct way of doing that, but PyQGIS API let's us to achieve it in an easy way:

def selectLimitedByCount(layer, count):
    request = QgsFeatureRequest().setFlags(QgsFeatureRequest.NoGeometry)
    request.setNoAttributes()  # We only go for feature ids
    request.setLimit(count)
    layer.select([f.id() for f in layer.getFeatures(request)])
selectLimitedByCount(iface.activeLayer(), 10000)

By and large, QgsFeatureRequest allows us to configure queries to vector data providers, just like if we were creating SELECT statements in SQL. In this case, the SQL LIMIT keyword can be set via PyQGIS in this way:

request.setLimit(count)

score 1 · Answer 2 · answered Nov 18 '20 at 20:07

1

Here are some general steps using pyqgis:

Create vector layer of your bubd_a layer
Create an memory vector layer (q/a)
Loop through the features in the vector layer object using .getFeatures() (help doc) method with a counter and add each feature to the memory layer
Once you hit 10K interval, call the processing.run() statement using the 10K memory layer
Repeat step 2-4

answered Nov 18 '20 at 20:07

artwork21

35,114
8
66
134

Point 2 is too old. Could you show me how will looks the code? – datasciencebegginer Nov 19 '20 at 10:25

Taras · Answer 3 · 2022-11-20T12:23:14.533

Another option is to use the "Random selection" tool for selecting N random features.

import processing
from qgis.utils import iface
layer = iface.activeLayer()
processing.run("qgis:randomselection", {
                    'INPUT' : layer,
                    'METHOD' : 0,
                    'NUMBER' : 5 -- specify a number here
                    })

To get more information about this algorithm, please run the following command processing.algorithmHelp("qgis:randomselection") in the Python Console (Ctrl+Alt+P).

References:

PyQGIS Developer Cookbook » 21. Cheat sheet for PyQGIS

score 0 · Answer 4 · answered Nov 19 '20 at 12:48

I solved that in another way, in my opinion easier way. I recommend to use Geopandas and add something like that:

opolskie = geopandas.read_file(r'/home/gis/budynki_opolskie/budynki_opolskie/bdot10k_bubd_a_opolskie.gpkg', layer ='bubd_a')
maleopolskie = opolskie[:10001]
maleopolskie.to_file("maleopolskie.shp")
processing.run("native:joinattributesbylocation", {'INPUT':r'/home/gis/maleopolskie.shp|layername=maleopolskie',
                                                   'JOIN':r'/home/gis/adresypolska/PRG_PunktyAdresowe_POLSKA.shp',
                                                   'PREDICATE':[1,5],
                                                   'JOIN_FIELDS':['PNA','SIMC_id','SIMC_nazwa','ULIC_nazwa','Numer'],
                                                   'METHOD':0,
                                                   'DISCARD_NONMATCHING':True,
                                                   'PREFIX':'',
                                                   'OUTPUT':r'/home/gis/opolskiegeo.shp'})

Selecting only N records from layer using PyQGIS

4 Answers4