7

I hope this question is appropriate here and I apologize if it is more of a discussion question than a specific technical problem.

I have to buffer 150,000 polygons for a real estate project. While it's not impossibly slow, it got me wondering about 'the fastest way to geoprocess'. What packages, tips, or tricks could be used to make this faster? Does anyone have an idea how arcmap, arcpy, shapely, postgres compare in performance time?

EDIT: After asking around, I heard 'arcpy is slow!' a lot.

PolyGeo
  • 65,136
  • 29
  • 109
  • 338
EmdyP
  • 300
  • 3
  • 8
  • 8
    Let the database do the work, ArcGIS is quite slow for large amounts of geo-processing. See ST_Buffer (PostGIS) http://postgis.refractions.net/documentation/manual-2.0/ST_Buffer.html – Mapperz Mar 05 '13 at 18:06
  • 16
    Think of ways to avoid buffering. IMHO, people overuse it because it's a conceptually simple, direct way to perform some kinds of fixed-distance-based analyses. In many cases buffering can be avoided altogether by structuring the analyses appropriately and exploiting other available methods of proximity-based calculation. The disadvantage is that buffering each polygon produces on average around several dozen more vertices surrounding each vertex, which in your case could produce billions of new vertices: that's worth avoiding. – whuber Mar 05 '13 at 18:14
  • Thanks for the advice. I'll try to keep that in mind in the future. I was asked to create the buffers because my supervisor wanted to create a map with these buffers but had forgotten how to buffer things. – EmdyP Mar 05 '13 at 18:25
  • 2
    Perhaps you could use this as a "learning experience" for your supervisor? Digging deeper into a simple request often reveals the "real reason" behind the request, and often you can not only demonstrate faster turnaround time, but also more accurate analysis. – RyanKDalton Mar 05 '13 at 19:11
  • 1
    While this blog (which references this article) does not answer faster alternatives for buffering, it may provide you some ideas on speed of geoprocessing operations for many software. It could at least give you some direction as to where you may want to start. – RyanKDalton Mar 05 '13 at 19:15
  • She wants a buffer layer to display on a map. I agree with you but in this case there isn't analysis, just cartography. – EmdyP Mar 05 '13 at 19:17
  • Have you asked "Why?" she just wants to display a buffer layer on a map? If nothing else, I'm betting she is doing some sort of analysis, even if it's only a mental, not software, exercise. – RyanKDalton Mar 05 '13 at 19:23
  • Thanks for your help Ryan, that is the kind of discussion I was looking for. – EmdyP Mar 05 '13 at 19:25
  • Some company did a site suitability analysis for us a few years ago and produced maps of these 'zones'. She wants to show our actual infrastructure on top of their maps but needed to recreate the buffers in order to match their map.

    Maybe a 'select by location' with a distance could be used to identify what's in their 'zones' without buffering.

     – EmdyP Mar 05 '13 at 19:29
  • 1
    If you just need the cartography--which is a perfectly valid reason to create buffers--it at least opens up the prospect of computing the buffers with a raster analysis rather than a vector one. With so many polygons, the raster calculations ought to be much faster than the vector calculations, even at extremely high resolutions. Moreover, once this possibility is entertained, cartographic improvements become available. – whuber Mar 05 '13 at 19:57

1 Answers1

2

You could also look at using the OGR library, it has python bindings so you could create an ArcGIS buffer tool pretty easily.

I find ArcGIS/ArcPy slow in general, so try and avoid it as much as possible.

I would also endorse using PostGIS, it eats this kind of processing for breakfast and isn't difficult to setup

Andrew Dennison
  • 49
  • 1