Calculating Geographic Distance

Any geographical analysis almost invariably involves calculating distance between two points on the surface of the Earth (or two geographic coordinates). Below are four different options/formulas to calculate geographic distance. Each formula makes different assumption about the Earth’s shape and thereby has different accuracy and computational complexity.

Formula 1: Pythagorus’ distance formula

distance = ( (lat2 – lat1) ** 2 + (lng2 – lng1) ** 2) ** 0.5

lat1,lat2, lng1, lng2 – lat/lng in decimal degrees

Assumption: Earth is flat.
Notes about error: Although, the Earth is not flat, over a very small distance it can be reasonably assumed to be flat. Thus, Pythagorus formula can be used to calculate geographic distance between two geographic coordinates when two geographic coordinates are close to each other. However, the definition of ‘close’ remains ambiguous. I haven’t found any numerical value to define ‘close’ and identifying that value is an aspect of my next post.

Formula 2: Modified Pythagorus’ distance formula

distance = (
[69.1 * (lat2 – lat1)] ** 2 +
[53.0 * (lng2 – lng1)] ** 2
)** 0.5

Notes: This formula is same as Pythagorus’ formula but includes correction for the spherical surface of the Earth. Again, this is not very accurate formula and should be used only for small distances

Formula 3: Great Circle Distance Formula

distance = R * arccos( [sin(lat1) * sin(lat2)] + [cos(lat1) * cos(lat2) * cos(lon2-lon1)] )

R = radius of the Earth (equatorial radius)
lat1, lat2, lon1, lon2 – latitude/longitude in radians.

Assumption: Earth is spherical
Notes about error: One of the problem with Great Circle formula is that cos(x) value tends to be unreliable. For example, see below (Ref)

cos (5 degrees) = 0.996194698
cos (1 degree) = 0.999847695
cos (1 minute) = 0.9999999577
cos (1 second) = 0.9999999999882
cos (0.05 sec) = 0.999999999999971

As can be seen from above values, the difference between cos(1 minute) and cos (1 sec) starts to appear only after 8 decimal places. Thus, make sure to use datatype that has sufficient long to capture these minute differences. In Java, always use “double” (and not float) to perform these calculations.

Formula 4: Haversine Formula

d = R * 2 arcsin ( sqrt[ sin_square(dlat/2) + cos(lat1)*cos(lat2)*sin_square(dlon/2)] )

R = radius of the Earth
dlat = lat2 – lat1
dlon = lon2 – lon1
lat1, lat1, lon1, lon2 are in radians

Assumption: Earth is spherical, but includes some correction for flattening around poles
Notes about error: Haversine formula is also based on spherical model of the Earth but is considered more accurate than the Great Circle Formula.

Formula 5: Vincety’s Algorithm

Reference to Vincety’s algorithm

Assumption: Earth is ellipsoidal.
Notes about error: Vincety’s algorithm is the most accurate formula to calculate distance between two geographic coordinates. Based on empirical evidences, it has been shown to be accurate within 0.5 mm. However, Vincety’s algorithm is an iterative distance calculation algorithm and thereby, as compared to any of the above formula, it takes longer time.

1. Vincenty’s Algorithm –
2. Modified Pythagorus Formula –

Reblog this post [with Zemanta]

About Ritesh Agrawal

I am a applied researcher who enjoys anything related to statistics, large data analysis, data mining, machine learning and data visualization.
This entry was posted in Map and tagged , , , , . Bookmark the permalink.

5 Responses to Calculating Geographic Distance

  1. UK thesis says:

    This is extremely helpful info!!! Especially since you guys are offering it for free!! Very good listing. Everything is true. Thanx.

  2. Khayer says:

    Great Post

  3. davidgs says:

    This is all great, and I have actually been using the Haversine formula for calculating distances between points. I simply didn’t need the overhead for the increased accuracy of Vincenty’s. But what I’m now looking for is a way to calculate the distance of a point from a line between 2 geographic points. I have to (lat,lon) coordinates and I need to determine if a given point (late,lonx) is within a given distance of that line. So far, I’m stumped and can find no real answers on the google machine.🙂

    Any ideas?

  4. Pingback: Calculating Distance in Miles via SQL Query - Coding Times of a .Net Developer from Yard - Site Home - MSDN Blogs

  5. Pingback: Calculating Distance in Miles via SQL Query | CmdExec Technology News

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s