Creating virtual layers using Python and export them without using QGIS UI

Asked Apr 29 '21 at 03:32

Active May 01 '21 at 12:17

Viewed 280 times

I am trying to create a flow map and have 5000 nodes and millions of edges. I've used

SELECT StartID, DestID, Weight, 
       make_line(a.geometry, b.geometry)
FROM edges
JOIN nodes a ON edges.StartID = a.ID
JOIN nodes b ON edges.DestID = b.ID
WHERE a.ID != b.ID

which I found on https://anitagraser.com/2019/05/04/flow-maps-in-qgis-no-plugins-needed/.

Not surprisingly with known performance issues with virtual layers in QGIS, QGIS keeps crashing as it can't handle the large dataset.

I'm looking for ways I can create and export this using just python.

How can I do this?

edited May 01 '21 at 12:17

PolyGeo

65,136
29
109
338

asked Apr 29 '21 at 03:32

sjp_1989

1

Are you looking for this? Another part of your question concerns the use of 'PyQGIS in standalone scripts' – Taras Apr 29 '21 at 04:54
From a performance perspective, this still seems quite slow. Would an alternative workflow using shapely be faster? – sjp_1989 Apr 29 '21 at 11:47
IMHO, I would say yes. How many points do you have? – Taras Apr 29 '21 at 11:52
What is your main aim ? Doing it quickly or doing it programmatically ? – Snaileater Apr 29 '21 at 13:32
@Taras i have 5,500 nodes and 180 million edges. – sjp_1989 Apr 29 '21 at 14:22
@Snaileater I'd like to achieve both. It's something that I need to re-run every month so I'd like it to be done as efficiently as possible. The data also grows by 200MB every month. – sjp_1989 Apr 29 '21 at 14:23

Creating virtual layers using Python and export them without using QGIS UI

0 Answers0

Linked