Go Down

Topic: Precision of calculations in Arduino? (Read 2120 times) previous topic - next topic

lemming

I want to perform some geodetic (GPS type) calculations in Arduino (using Haversine or Vincentys formula).

However I want precision out to around 12 decimal places or more.

Does Arduino math support precision to this many decimal places? 

WizenedEE

I think the long long type is available, which would be 64 bits. 12 decimal digits is about 36 binary digits, so you could almost do it with a normal long (32 bits).

There is also a "big number" library someone ported that is floating around that can compute numbers to hundreds of digits (limited by the arduino's SRAM)

mrdovey

No. It supports only 'float' (no 'double' or 'long double'). For more precision, you'll need to use an extension library or roll your own.

nickgammon

There is indeed "long long" but it uses up program memory at a pretty fast rate.

If you really need that much precision you could use the "big number" library I ported:

http://gammon.com.au/forum/?id=11519
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

lemming


wildbill

Twelve decimal places sounds like more precision that you'd get even out of a milspec GPS. What are you trying to do?

John_S

Longitude to 12 decimal places will get you precision to 0.00001mm (that's right, millimeters). The only reason I could see this being useful would be for blasting microbes from space ;)

tomperdarwin


Longitude to 12 decimal places will get you precision to 0.00001mm (that's right, millimeters). The only reason I could see this being useful would be for blasting microbes from space ;)


An Orbital Surgical Death Ray based on an Arduino..  I don't know wether to be proud or very, very sad.

el_supremo

Quote
However I want precision out to around 12 decimal places or more.

Why?

This web page http://www.movable-type.co.uk/scripts/latlong-vincenty.html says of Vincenty's and Haversin:
Quote
Vincenty's formula is accurate to within 0.5mm, or 0.000015? (!), on the ellipsoid being used. Calculations based on a spherical model, such as the (much simpler) Haversine, are accurate to around 0.3%

John_S pointed out that 12 digits
Quote
will get you precision to 0.00001mm
which means you can't even come close to a precision of 12 decimal places using either formula.

@OP. Why would you want to compute Vincenty's formula on an Arduino anyway even if it was to a reasonable number of digits precision?

Pete
Don't send me technical questions via Private Message.

Go Up