Go Down

Topic: floating point precision: is true double possible? (Read 11175 times) previous topic - next topic

mowcius

Quote
Earth's diameter in m (6371000)

That might be your issue

Quote
Earth's Diameter at the Equator: 7,926.28 miles (12,756.1 km)

Earth's Diameter at the Poles: 7,899.80 miles (12,713.5 km)


Mowcius

borref

Quote
Is there any other conversion required?

Arguments for latitude and longitude need to be passed as radians (not degrees). Could this be what you have missed?

The return value (also in radians) must then be multiplied with the earth radius. For this I use the the IUGG mean radius defined as 6371009 meters.

http://en.wikipedia.org/wiki/Earth_radius

AWOL

Quote
though am not getting a valid result (getting 50~300m

Have you checked your datum?
For instance, the UK OS datum produces errors of about this magnitude, compared to the WGS datum used by GPS.

Go Up