-2

After reviewing various information regarding accuracy of latitude and longitude for specific decimal places, I have a question regarding the information that many have posted, such as in Measuring accuracy of latitude and longitude?.

If my analysis and calculations are correct, it seems accuracy values that people reference for 6 decimal places upwards are not scaled correctly. Is this a correct statement, or am I looking at this wrong?

For instance, many are saying 9 decimal places would be 111 μm, but from my math:
((111,111 m) / 10^9) * (1000 m / cm) * (1000 mm / cm) = 111 mm.

This would be 1,000x larger than what many others are saying. Where is the difference?

Vince
  • 20,017
  • 15
  • 45
  • 64
Colby Burns
  • 5
  • 2

1 Answers1

4

There is 100 cm in a meter, not 1,000 and only 10mm in a cm, not 1,000. So you are 1,000 times out in your calculation.

Leigh Bettenay
  • 1,401
  • 8
  • 16