Arduino Forum

Using Arduino => Programming Questions => Topic started by: lemming on Feb 06, 2012, 07:01 am

Title: Precision of calculations in Arduino?
Post by: lemming on Feb 06, 2012, 07:01 am
I want to perform some geodetic (GPS type) calculations in Arduino (using Haversine or Vincentys formula).

However I want precision out to around 12 decimal places or more.

Does Arduino math support precision to this many decimal places? 
Title: Re: Precision of calculations in Arduino?
Post by: WizenedEE on Feb 06, 2012, 07:05 am
I think the long long type is available, which would be 64 bits. 12 decimal digits is about 36 binary digits, so you could almost do it with a normal long (32 bits).

There is also a "big number" library someone ported that is floating around that can compute numbers to hundreds of digits (limited by the arduino's SRAM)
Title: Re: Precision of calculations in Arduino?
Post by: mrdovey on Feb 06, 2012, 07:08 am
No. It supports only 'float' (no 'double' or 'long double'). For more precision, you'll need to use an extension library or roll your own.
Title: Re: Precision of calculations in Arduino?
Post by: nickgammon on Feb 06, 2012, 07:58 am
There is indeed "long long" but it uses up program memory at a pretty fast rate.

If you really need that much precision you could use the "big number" library I ported:

http://gammon.com.au/forum/?id=11519
Title: Re: Precision of calculations in Arduino?
Post by: lemming on Feb 06, 2012, 01:43 pm
Thanks guys.
Title: Re: Precision of calculations in Arduino?
Post by: wildbill on Feb 07, 2012, 12:45 pm
Twelve decimal places sounds like more precision that you'd get even out of a milspec GPS. What are you trying to do?
Title: Re: Precision of calculations in Arduino?
Post by: John_S on Feb 07, 2012, 05:57 pm
Longitude to 12 decimal places will get you precision to 0.00001mm (that's right, millimeters). The only reason I could see this being useful would be for blasting microbes from space ;)
Title: Re: Precision of calculations in Arduino?
Post by: tomperdarwin on Feb 07, 2012, 07:43 pm

Longitude to 12 decimal places will get you precision to 0.00001mm (that's right, millimeters). The only reason I could see this being useful would be for blasting microbes from space ;)


An Orbital Surgical Death Ray based on an Arduino..  I don't know wether to be proud or very, very sad.
Title: Re: Precision of calculations in Arduino?
Post by: el_supremo on Feb 07, 2012, 08:21 pm
Quote
However I want precision out to around 12 decimal places or more.

Why?

This web page http://www.movable-type.co.uk/scripts/latlong-vincenty.html says of Vincenty's and Haversin:
Quote
Vincenty's formula is accurate to within 0.5mm, or 0.000015? (!), on the ellipsoid being used. Calculations based on a spherical model, such as the (much simpler) Haversine, are accurate to around 0.3%

John_S pointed out that 12 digits
Quote
will get you precision to 0.00001mm
which means you can't even come close to a precision of 12 decimal places using either formula.

@OP. Why would you want to compute Vincenty's formula on an Arduino anyway even if it was to a reasonable number of digits precision?

Pete