31

With ArcGIS 10, Python has become the de-facto scripting language for ESRI. While leading Open source software like QGIS, GeoServer, PostGIS all supports Python. Thus it became important to know/learn Python for GIS developers a well as users.

Can anybody make a list of such tools/modules/add-ins in Python that are crucial in GIS?

PolyGeo
  • 65,136
  • 29
  • 109
  • 338
thelastray
  • 2,864
  • 3
  • 24
  • 43
  • 10
    I think this question is too broad and should be limited to python packages that can be used in any GIS with Python support. – underdark Apr 16 '12 at 21:15
  • 4
    Is ArcPy a true Python module? – Mapperz Apr 16 '12 at 21:32
  • 2
    @Mapperz it's actually a site package. The predecessor arcgisscripting is a module. See http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//000v000000v7000000 – blah238 Apr 16 '12 at 21:37
  • 3
    @underdark - I think having a nice broad question can be useful. In this case it gives a central resource that search engines will find when someone starts googling for which python toolset to use. – GIS-Jonathan Apr 17 '12 at 10:47
  • 2
    +1 for too broad. The most relevant answers to the question will be found at the end of a [tag:python] + [tag:my-software] search and sorted via the votes and faq tabs. The answers here can never hope to match that. – matt wilkie Apr 17 '12 at 19:41
  • 1
    @mattwilkie - Disagree. I just tried searching for [python] [qgis] - the results were all highly specific use-cases where the user was trying to do something highly specialised. This question is useful because it gives a nice list of all packages. The fact 8 other people have favourited it suggests they also find it useful. – GIS-Jonathan Apr 18 '12 at 16:31
  • I remain unconvinced this is the best formula to get there but stand down willingly as obviously the votes go otherwise ;-) – matt wilkie Apr 18 '12 at 19:55
  • Im trying to use SciPy, NumPy and PIL (Python Imaging Library) for the FME python interpreter. My main goal is to do som kernel filtering on some rasterfiles. I have copied the sitepackages directory for PIL, numpy and scipy into the sitepackages directory under fmepython\lib\sitepackages. When I try to import my new modules into FME I get a runtime Error, where it says my modules tried to attempt the C runtime Library incorrectly. Does anyone know if this is possible to do correctly? –  Jan 24 '14 at 16:21
  • Good morning everyone, Many thanks for the above. I am just now learning Python and find myself overwhelmed as I am trying to put script writing to memory. Again, many thanks. Tony –  Feb 25 '15 at 14:34

10 Answers10

40

  • NumPy: NumPy is the fundamental package for scientific computing with Python. It contains among other things:

    • a powerful N-dimensional array object
    • sophisticated (broadcasting) functions
    • tools for integrating C/C++ and Fortran code
    • useful linear algebra, Fourier transform, and random number capabilities

    Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a wide variety of databases.

  • SciPy: SciPy (pronounced "Sigh Pie") is open-source software for mathematics, science, and engineering. It is also the name of a very popular conference on scientific programming with Python. The SciPy library depends on NumPy, which provides convenient and fast N-dimensional array manipulation. The SciPy library is built to work with NumPy arrays, and provides many user-friendly and efficient numerical routines such as routines for numerical integration and optimization. Together, they run on all popular operating systems, are quick to install, and are free of charge. NumPy and SciPy are easy to use, but powerful enough to be depended upon by some of the world's leading scientists and engineers. If you need to manipulate numbers on a computer and display or publish the results, give SciPy a try!

  • Shapely: Shapely is a BSD-licensed Python package for manipulation and analysis of planar geometric objects. It is based on the widely deployed GEOS (the engine of PostGIS) and JTS (from which GEOS is ported) libraries. This C dependency is traded for the ability to execute with blazing speed. Shapely is not concerned with data formats or coordinate systems, but can be readily integrated with packages that are.

  • GDAL Python bindings: This Python package and extensions are a number of tools for programming and manipulating the GDAL Geospatial Data Abstraction Library.

  • GeoDjango: GeoDjango intends to be a world-class geographic Web framework. Its goal is to make it as easy as possible to build GIS Web applications and harness the power of spatially enabled data.

  • PyProj

  • SpatialPython: Nicely documented github repository.

Glorfindel
  • 1,096
  • 2
  • 9
  • 14
blah238
  • 35,793
  • 7
  • 94
  • 195
  • 1
    I would add the actively developed http://geopandas.org/ library for anyone looking at this older answer! – clifgray Oct 05 '17 at 16:25
12

To answer my own question I have just found this Pythons module. Although I have not used it yet, it looks exciting.

NetworkX is a Python language software package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.

And

rtree - spatial index for Python GIS

Glorfindel
  • 1,096
  • 2
  • 9
  • 14
thelastray
  • 2,864
  • 3
  • 24
  • 43
11

Go to Topic :: Scientific/Engineering :: GIS dnd you have all the Python Modules for GIS (for working with shapefiles, rasters, KML, GML,GPX geocoding, etc.)

The most important have already been cited but I recommend also Fiona "Fiona provides a minimal, uncomplicated Python interface to the open source GIS community's most trusted geodata access library and integrates readily with other Python GIS packages such as pyproj, Rtree, and Shapely."

and for networking with shapefiles or Esri Feature Class with Networkx module Geometric Network Geoprocessing: "As far as I can tell, ESRI has not released any geoprocessing tools for their Geometric Network" or Seeking alternatives to pgRouting for open source routing / network analysis? or Python: how to transform a shapefile (or feature class ESRI) in a topological network (graph) (in French)

import networkx as nx
G = nx.read_shp('pointshapefile.shp')
print(G.nodes())
# result [(1.0, 2.0), (3.0, 2.0), (0.0, 0.0), (3.0, 1.0), (4.0, 4.0), (2.0, 1.0), (2.0, 4.0), (1.0, 3.0), (2.0, 3.0), (1.0, 4.0), (4.0, 3.0), (4.0, 2.0), (3.0, 4.0), (1.0, 1.0)]
print(G.edges())
# result [((1.0, 2.0), (1.0, 1.0)), ((3.0, 2.0), (2.0, 1.0)), ((3.0, 1.0), (2.0, 1.0)), ((4.0, 4.0), (3.0, 4.0)), ((2.0, 1.0), (1.0, 1.0)), ((2.0, 4.0), (2.0, 3.0)), ((1.0, 3.0), (1.0, 2.0)), ((2.0, 3.0), (1.0, 2.0)), ((1.0, 4.0), (1.0, 3.0)), ((4.0, 3.0), (4.0, 2.0)), ((4.0, 2.0), (3.0, 2.0)), ((3.0, 4.0), (2.0, 3.0)), ((1.0, 1.0), (0.0, 0.0))]

shortest path


print(nx.astar_path(H,(1.0, 4.0),(4.0, 2.0),dist))


result [(1.0, 4.0), (1.0, 3.0), (1.0, 2.0), (2.0, 3.0), (3.0, 2.0), (4.0, 2.0)]


and so with all the algorithms of Networkx module


you can also export the results in shapefile format

I use Shapely, Fiona, GDAL/OGR, Pyshp, Networkx and others in QGIS and GRASS GIS without problem (and with matplotlib or descartes for interactive graphing to). They often have algorithms easier to use for treatment.

Some of these modules can also be used in ArcPy with problems because ArcPy uses version 1.3 of Numpy, outdated (now version 1.6.1...) and you can not update it without breaking the ArcPy module.

Glorfindel
  • 1,096
  • 2
  • 9
  • 14
gene
  • 54,868
  • 3
  • 110
  • 187
8

For the cartographic rendering of GIS data:

  1. Mapnik Python Bindings
  2. MapServer MapScript
user890
  • 5,733
  • 1
  • 42
  • 71
7

Adding to the list:

PySAL - "open source cross-platform library of spatial analysis functions"

available from: http://code.google.com/p/pysal/

pyshp - a python shapefile reader and writer in pure python

available from: http://code.google.com/p/pyshp/

Edit:

Someone showed me this module today, may be of some intrest to people. Sample GIS vector and raster data for python usage:

gisdata - http://pypi.python.org/pypi/gisdata/0.3.3

James Milner
  • 1,932
  • 18
  • 22
5

I use and recommend ReportLab Toolkit, the Open Source PDF library for programatically creating documents in PDF format. As advertised on its pages, it is a robust, flexible, time-proven, industry-strength solution. It's free, open-source software written in Python but its syntax is not the easiest that I've ever dealt with :-)

At ArcGIS 10.0 it is invaluable for writing reports in PDF format from feature classes and tables although some of this functionality may not be so necessary at 10.1 when access to PDF reports from the ArcGIS report writer becomes available to ArcPy.

Glorfindel
  • 1,096
  • 2
  • 9
  • 14
PolyGeo
  • 65,136
  • 29
  • 109
  • 338
4

In the Django world:

  • vectorformats (for sppiting our geojson as geodjango does not support it natively);
  • Geraldo (for reporting, uses ReportLab);
Glorfindel
  • 1,096
  • 2
  • 9
  • 14
George Silva
  • 6,298
  • 3
  • 36
  • 71
2

There are yet more:

GRASS - You can call GRASS with Python.

FMEObjects - If you have an FME license they have a python module too which allows you to call some of their cool transformers.

GIS-Jonathan
  • 6,775
  • 5
  • 29
  • 58
  • Really? FMEObjects to my knowledge does not have Python bindings for calling transformers. I really wish it did. If you know that it does and know how to access that API, please let us know. It would make my life a whole lot easier at the present time. – celticflute Nov 06 '13 at 00:51
1

The Enthought Package Distribution has a lot of the packages listed above bundled into a cohesive platform. They've even ensured that it can be easily configured to work from ArcGIS's python prompt and that arcpy can be used from it's python prompt. We use this in our offices. From our internal wiki:

The best way connect ArcGIS and EPD is to install both and link them using .pth files so the Python sys.path includes the other system's modules. The file "zzEPD.pth" allows ArcGIS Python to access EPD modules, and "zzArcGIS.pth" allows EPD Python to access arcpy. (The "zz" prefix is there to ensure that the "foreign" packages are last in the sys.path to avoid potential conflicts for modules that exist in both Python installations.) If you run into ArcGIS conflicts, simply rename the file to .txt ("zzEPD.pth.txt") and restart ArcGIS and ArcGIS will no longer "see" EPD modules (they will not be in the sys.path).

*zzEPD.pth - place in folder \Python27\Desktop10.1\lib\site-packages*

# zzEPD.pth 
# Path to Enthought modules
C:\Python27\epd32\lib\site-packages

*zzArcGIS.pth - place in folder \Python27\epd32\lib\site-packages*

# zzArcGIS.pth 
# copy of \Python27\Desktop10.1\lib\site-packages\ArcGIS.pth
C:\ArcGIS\Desktop10.1\bin
C:\ArcGIS\Desktop10.1\arcpy
C:\ArcGIS\Desktop10.1\ArcToolbox\Scripts
Roland
  • 1,290
  • 9
  • 21
1

Not specific to GIS, but for debugging python, IPDB is amazing. https://pypi.python.org/pypi/ipdb

To use it, just place the following lines in your code:

import ipdb
ipdb.set_trace()

Then in your Python shell you can drop into any piece of code and interact with all of the variables at that current state.

djq
  • 16,297
  • 31
  • 110
  • 182