3

The problem I'm encountering is simple. I have a shape file of points (about 8 million). They have three XY pairs that I am attempting to create XY lines from for purposes of measuring distance. However, the tool repeatedly has crashed, bringing down whatever program was running it (Catalog or ArcMap). I am assuming this is not related to the issue of XY To Line crashing if it encounters a record that does not have both pairs of XY co-ordinates, as I cleaned those out. Is there a size limit issue, here? Of the 7.9 million records input, it gets to 7.3 before the entire program goes kaput.

For the purpose of clarity, let me describe the data setup. For each record in the table there is a point with an XY and two geocoded locations with XYs. So Point A Geocodes to Point B, and Point A geocodes to Point C. I have the XY locations of A, B, and C all on the same record, and am trying to glean the distance between their coordinates (A -> B), (A -> C), (B -> C).

Two questions:

1) Is there a better method in ArcMap 10 to measure the distance between two points than snatching the length from an XY to Line feature class?

2) Why would XY to Line be crashing at about 7.3 million records?

whuber
  • 69,783
  • 15
  • 186
  • 281
Nathanus
  • 4,318
  • 1
  • 32
  • 47
  • 1
    You may want to break your shp up into smaller ones then import those into a file geodatabase to run whatever tool you use. – artwork21 Jun 02 '11 at 18:34
  • Yes, that is my alternative (and much less efficient) method. But I'd like to know why it splatters so magnificently. – Nathanus Jun 02 '11 at 18:37

1 Answers1

2

Are you saying you want the distances from each of the 8 million points to each of three fixed points? If so, then a fast workaround might be to perform three spatial joins--if that doesn't also crash ArcGIS :-).

Another workaround is to compute the distances with a field calculation (using the Pythagorean Theorem for projected coordinates or spherical geometry for geographic coordinates). The field calculation doesn't even need a GIS: you can use your database, write a little script or program, or even (shudder) use a spreadsheet. With a compiled program the only size limit is imposed by the OS on the output file and the speed will be so fast it's I/O limited.

In light of the clarified question, the first workaround is out but the field calculation would do fine.

whuber
  • 69,783
  • 15
  • 186
  • 281
  • I wouldn't even know where to begin for creating a proper calculation that would do what I wish. Any suggestions on some good reading on the subject? – Nathanus Jun 06 '11 at 14:56
  • @Nathanus It depends on what coordinates the points are in, how close they tend to be, and the accuracy you need. If they are scattered all over the globe and either tend to be far apart or your accuracy requirements are not super high, use the Haversine formula – whuber Jun 06 '11 at 15:44
  • In this particular example, a set of measurements will always be within a given state, and their distance should never exceed more than a couple of hundred meters. Accuracy... would be nice to 1 meter if possible. The coordinates are in WGS84. – Nathanus Jun 06 '11 at 15:46
  • @Nathanus Haversine will be ok. If you prefer, you can group sets of points within (say) 6-degree longitude bands, project all points in each group to (say) UTM, and apply the Pythagorean formula. That will still get you +-1 meter accuracy (easily). Doing the projection is a heavy price to pay, though: the little bit of trig involved in the Haversine is probably the least painful solution. Note, too, that there are decent approximations that are even simpler; for example, multiply the longitudes by cos(latitude), use Pythagoras, and convert the answer from degrees to meters. – whuber Jun 06 '11 at 15:52
  • All right, thanks for the help! It's plenty to get started with. – Nathanus Jun 06 '11 at 15:57