19

For my own data analysis pipeline, I am running a split screen setup with IPython notebooks on one side, generating variables from source data, pushing them to a CSV file which QGIS then reads/joins to an appropriate geography, instantly visible on the other side of the screen.

I would like to push the control of the running instance of QGIS to the ipython notebook, in effect allowing a super quick calculation/visualization cycle.

Right now I can do this in the QGIS python console with something like this:

canvas = qgis.utils.iface.mapCanvas()
cLayer = canvas.currentLayer()
canvas.refresh()

but I would like to:

  1. connect to an existing instance of qgis from an external Python environment
  2. at a minimum refresh the map, after each re-write to the csv from ipython notebook, but preferably
  3. re-calculate breaks (using quantiles, natural breaks) based on the newly joined values

How can I achieve this?

Taras
  • 32,823
  • 4
  • 66
  • 137
ako
  • 1,974
  • 1
  • 17
  • 24
  • 3
    This is going to be a lot tricker than you might think. First of all, process security will ensure that a process that doesn't want to be controlled can't be controlled by another process. QGIS would have to provide a well-defined IPC interface, or use something like D-Bus to allow other processes to interact with it. The next question is, how would that tie into the QGIS user interface? If you're in edit mode for a layer in QGIS, and trigger an update from your notebook, what happens in QGIS? – Lukas Graf Nov 17 '13 at 20:00
  • 3
    What I imagine could work is writing a QGIS plugin that reads the CSV, and does all the refreshing and recalculating you need it to do, and bind that to a shortcut like F5. On the other side in the IPython notebook, write a function that saves your data to the CSV and make sure that function will always get imported into your global namespace, by using the PYTHONSTARTUP environment variable for example. – Lukas Graf Nov 17 '13 at 20:05
  • Indeed, I had not thought about all the conflicts that could entail. So this approach would 'drive' the ipython stuff from qgis, not the other way around, but that is clearly a way to Rome. – ako Nov 17 '13 at 20:33
  • Also, I meant to post this to gis.SE, not SE :) – ako Nov 17 '13 at 20:34
  • 1
    No, my suggestion was to not make the update fully automatic. Bind reload to a quick shortcut like F5 in QGIS, and save to a simple function in notebook. Still two actions, but should nevertheless speed up your cycle. A one button solution would be possible I guess, but considerably more complex, and you'd need to be willing to invest quite some time into it. – Lukas Graf Nov 17 '13 at 21:36
  • 2
    Turns out, IPython provides some very useful features for IPC: Communication between an IPython Kernel and an interactive interpreter like the notebook follows a decoupled two-process model. Also, messaging between kernel and client is well specified and documented, and uses ZeroMQ. So you could probably leverage that, and talk to the IPython kernel from a QGIS plugin (via ZMQ). Still a lot of work though. – Lukas Graf Nov 17 '13 at 21:47
  • Ipython QT console tha can be embedded in qgis does not share kernel with an external notebook... it's a qtconsole limitarion (AFAIK). There are some custom solution (eg. plugins) that allow remote control creating similar mechanism (RTC o MQTT). Try "QGIS remote control" or "Telemetry layer" plugins. The latter as MQTT node bridge inside qgis. – Luigi Pirelli Feb 08 '18 at 16:32
  • 4
    Throw away your CSV and use instead PostgreSQL/PostGIS. There is a mecanism to notify change in the database to QGIS that could help synchronize content on both side. See this blog post https://oslandia.com/en/2017/10/07/refresh-your-maps-from-postgresql/ for QGIS. On the notebook side (pure Python, you may look at https://tapoueh.org/blog/2018/07/postgresql-listen-notify/) – ThomasG77 Jan 30 '20 at 23:31

2 Answers2

1

I don't know if this will help but in the ArcGIS land you can attach a debugger to an existing python process with Visual Studio. I know you are running on QGIS and I haven't tried, but you may be able to follow the same steps to attach Visual Studio to your QGIS Session.

In this documentation it describes how to hook up Visual Studio and the Pro Version of PyCharm. Also the Visual Studio link for how to attach to a python debugging process.

You basically start Visual Studio (not Vs Code) and make sure the Python Development is installed then you attach your debug to hopefully your QGIS instance. It may not work but it could be worth a shot.

From ArcGIS Docs Attach Python Debugger

Ben
  • 651
  • 3
  • 5
-1

QGIS conda-forge package should be handy

Enock4seth
  • 127
  • 1
  • 6