Go Down

### Topic: Precision of calculations in Arduino? (Read 2113 times)previous topic - next topic

#### lemming

##### Feb 06, 2012, 07:01 am
I want to perform some geodetic (GPS type) calculations in Arduino (using Haversine or Vincentys formula).

However I want precision out to around 12 decimal places or more.

Does Arduino math support precision to this many decimal places?

#### WizenedEE

#1
##### Feb 06, 2012, 07:05 am
I think the long long type is available, which would be 64 bits. 12 decimal digits is about 36 binary digits, so you could almost do it with a normal long (32 bits).

There is also a "big number" library someone ported that is floating around that can compute numbers to hundreds of digits (limited by the arduino's SRAM)

#### mrdovey

#2
##### Feb 06, 2012, 07:08 am
No. It supports only 'float' (no 'double' or 'long double'). For more precision, you'll need to use an extension library or roll your own.

#### nickgammon

#3
##### Feb 06, 2012, 07:58 am
There is indeed "long long" but it uses up program memory at a pretty fast rate.

If you really need that much precision you could use the "big number" library I ported:

http://gammon.com.au/forum/?id=11519
Please post technical questions on the forum, not by personal message. Thanks!

#4
Thanks guys.

#### wildbill

#5
##### Feb 07, 2012, 12:45 pm
Twelve decimal places sounds like more precision that you'd get even out of a milspec GPS. What are you trying to do?

#### John_S

#6
##### Feb 07, 2012, 05:57 pm
Longitude to 12 decimal places will get you precision to 0.00001mm (that's right, millimeters). The only reason I could see this being useful would be for blasting microbes from space

#### tomperdarwin

#7
##### Feb 07, 2012, 07:43 pm

Longitude to 12 decimal places will get you precision to 0.00001mm (that's right, millimeters). The only reason I could see this being useful would be for blasting microbes from space

An Orbital Surgical Death Ray based on an Arduino..  I don't know wether to be proud or very, very sad.

#### el_supremo

#8
##### Feb 07, 2012, 08:21 pm
Quote
However I want precision out to around 12 decimal places or more.

Why?

This web page http://www.movable-type.co.uk/scripts/latlong-vincenty.html says of Vincenty's and Haversin:
Quote
Vincenty's formula is accurate to within 0.5mm, or 0.000015? (!), on the ellipsoid being used. Calculations based on a spherical model, such as the (much simpler) Haversine, are accurate to around 0.3%

John_S pointed out that 12 digits
Quote
will get you precision to 0.00001mm
which means you can't even come close to a precision of 12 decimal places using either formula.

@OP. Why would you want to compute Vincenty's formula on an Arduino anyway even if it was to a reasonable number of digits precision?

Pete
Don't send me technical questions via Private Message.

Go Up