Title: **Precision of calculations in Arduino?**

Post by:**lemming** on **Feb 06, 2012, 07:01 am**

Post by:

I want to perform some geodetic (GPS type) calculations in Arduino (using Haversine or Vincentys formula).

However I want precision out to around 12 decimal places or more.

Does Arduino math support precision to this many decimal places?

However I want precision out to around 12 decimal places or more.

Does Arduino math support precision to this many decimal places?

Title: **Re: Precision of calculations in Arduino?**

Post by:**WizenedEE** on **Feb 06, 2012, 07:05 am**

Post by:

I think the long long type is available, which would be 64 bits. 12 decimal digits is about 36 binary digits, so you could almost do it with a normal long (32 bits).

There is also a "big number" library someone ported that is floating around that can compute numbers to hundreds of digits (limited by the arduino's SRAM)

There is also a "big number" library someone ported that is floating around that can compute numbers to hundreds of digits (limited by the arduino's SRAM)

Title: **Re: Precision of calculations in Arduino?**

Post by:**mrdovey** on **Feb 06, 2012, 07:08 am**

Post by:

No. It supports only 'float' (no 'double' or 'long double'). For more precision, you'll need to use an extension library or roll your own.

Title: **Re: Precision of calculations in Arduino?**

Post by:**nickgammon** on **Feb 06, 2012, 07:58 am**

Post by:

There is indeed "long long" but it uses up program memory at a pretty fast rate.

If you really need that much precision you could use the "big number" library I ported:

http://gammon.com.au/forum/?id=11519

If you really need that much precision you could use the "big number" library I ported:

http://gammon.com.au/forum/?id=11519

Title: **Re: Precision of calculations in Arduino?**

Post by:**lemming** on **Feb 06, 2012, 01:43 pm**

Post by:

Thanks guys.

Title: **Re: Precision of calculations in Arduino?**

Post by:**wildbill** on **Feb 07, 2012, 12:45 pm**

Post by:

Twelve decimal places sounds like more precision that you'd get even out of a milspec GPS. What are you trying to do?

Title: **Re: Precision of calculations in Arduino?**

Post by:**John_S** on **Feb 07, 2012, 05:57 pm**

Post by:

Longitude to 12 decimal places will get you precision to 0.00001mm (that's right, millimeters). The only reason I could see this being useful would be for blasting microbes from space ;)

Title: **Re: Precision of calculations in Arduino?**

Post by:**tomperdarwin** on **Feb 07, 2012, 07:43 pm**

Post by:

Longitude to 12 decimal places will get you precision to 0.00001mm (that's right, millimeters). The only reason I could see this being useful would be for blasting microbes from space ;)

An Orbital Surgical Death Ray based on an Arduino.. I don't know wether to be proud or very, very sad.

Title: **Re: Precision of calculations in Arduino?**

Post by:**el_supremo** on **Feb 07, 2012, 08:21 pm**

Post by:

Quote

However I want precision out to around 12 decimal places or more.

Why?

This web page http://www.movable-type.co.uk/scripts/latlong-vincenty.html says of Vincenty's and Haversin:

Quote

Vincenty's formula is accurate to within 0.5mm, or 0.000015? (!), on the ellipsoid being used. Calculations based on a spherical model, such as the (much simpler) Haversine, are accurate to around 0.3%

John_S pointed out that 12 digits

Quote

will get you precision to 0.00001mmwhich means you can't even come close to a precision of 12 decimal places using either formula.

@OP. Why would you want to compute Vincenty's formula on an Arduino anyway even if it was to a reasonable number of digits precision?

Pete