Trying to understand floating point accuracy

rogerneed · January 18, 2022, 12:56am

I am a new Arduino user and am developing a frequency counter. I want to measure in fractions of a Hertz so I need to display results in floating point. However I find the accuracy of the calculations is quite poor and it gets worse when operating on larger numbers. I wrote a simple piece of code to illustrate my point. The output from the Serial Console is shown below the code.

Am I do something wrong or is this just the way it is with the Arduino floating point implementation?

Thanks - Roger

// Test floating point calculations and division

float num1 =10000.0;
float num2 =3333.0;
float num3 =100000.0;
float num4 =33333.0;
float num5 =1000000.0;
float num6 =333333.0;
int x = 3;

void setup() {

Serial.begin (9600);
Serial.println ("Start Calculating");
Serial.println ("--------");
Serial.println (num1,5);  // should print 1000000.00000
num1 = (num1/3.0);
Serial.println (num1,5);  // should print 33333.33333
num1 = num1-num2;
Serial.println (num1,5);  // should print 0.33333
num1 = num1*3.0;
Serial.println (num1,5);  // should print 0.99999

Serial.println ("--------");
Serial.println (num3,5);  // should print 100000.00000
num3 = (num3/3.0);
Serial.println (num3,5);  // should print 33333.33333
num3 = num3-num4;
Serial.println (num3,5);  // should print 0.33333
num3 = num3*3.0;
Serial.println (num3,5);  // should print 0.99999

Serial.println ("--------");
Serial.println (num5,5);  // should print 1000000.00000
num5 = (num5/3.0);
Serial.println (num5,5);  // should print 333333.33333
num5 = num5-num6;
Serial.println (num5,5);  // should print 0.33333
num5 = num5*3.0;
Serial.println (num5,5);  // should print 0.99999
}

void loop() {
  // put your main code here, to run repeatedly:
while (x == 3);
}

Output from Serial Console

`Start Calculating
--------
10000.00000
3333.33325
0.33325
0.99976
--------
100000.00000
33333.33203
0.33203
0.99609
--------
1000000.00000
333333.34375
0.34375
1.03125
`

anon57585045 · January 18, 2022, 1:11am

You guessed it. It's the way it is with AVR GCC, so any Uno or Mega. It only supports 32 bit floating point, not a "double" type which is 64 bits.

Most 32 bit Arduinos do support the double type. So, which one do you have, and is this important enough to you, to select a different board?

anon57585045 · January 18, 2022, 1:13am

or alternately,

void loop() {
}

lastchancename · January 18, 2022, 1:14am

It is confusing, but quite simple.
Look at the math library you’re using, and somewhere you’ll see a reference to ‘n significant digits’

The n is not negotiable.
You can have n digits total including left & right if the decimal.

For more precision, use a different library, or a different method to rumble your big numbers.

david_2018 · January 18, 2022, 1:16am

You are going to need a very accurate reference oscillator in order to measure frequency with the 6 to 7 digits of precision that a float will allow.

anon57585045 · January 18, 2022, 1:20am

However those do exist... I do think you should share more details of the application. I think the point david_2018 is making, is that if you use software for the measurements, not only do you need an accurate reference oscillator, but it has to be connected to the CPU clock somehow.

rogerneed · January 18, 2022, 1:41am

I am using a Uno R3 and the final implementation will be on a Nano

jremington · January 18, 2022, 1:44am

The Arduino 16 MHz clock frequency can quite easily be measured to 8 decimal digits of accuracy, using the GPS 1PPS signal as the reference and Timer 1 input capture mode (Uno R3 et al). The CPU clock can then be used to measure other signals (warning: the CPU clock is temperature dependent).

Nick Gammon's frequency counter code on the timer tutorial page shows the basic idea: https://www.gammon.com.au/timers

rogerneed · January 18, 2022, 1:47am

Yes I have a very accurate reference oscillator. I am using a GPS locked signal generator with 1x e-10 accuracy. I count the frequency over a long time period of 100 or 1000 seconds and that gives me 0.01 or 0.001 Hz respectively. I have it working OK using 64 bit integers but wanted floating point for certain calculations I am doing with the measured frequency.

rogerneed · January 18, 2022, 1:49am

Yes I am using a variation of Gammon's implementation. I will probably end up with an external gating circuit because of interrupt latency and tricky code timing.

rogerneed · January 18, 2022, 1:52am

I am just using the standard library that links by default to my sketch. Any idea what the n digits are for this floating point implementation?

Have you got a suggestion for something better?

rogerneed · January 18, 2022, 1:57am

There are several ways to do this and using a stable frequency source for the CPU clock is one method. Another way is to use a precision external gate circuit and only use the Arduino as a counter (with overflow counting as well).

gfvalvo · January 18, 2022, 1:57am

As already stated ... 6, maybe 7 significant figures for the 32-bit IEEE floating point implementation.

And, as already stated, get an ARM or ESP-based board and you can use 'double' .... 64-bit floating point.

westfw · January 18, 2022, 2:09am

Remember learning about "Scientific Notation" in school?
Numbers are expressed as M x 10^E, where you call M the Mantissa and E the Exponent.
When you get fussy, M is always supposed to be between 1 and 10, so that 1234 becomes 1.234x10³, and Avagardo's number is 6.022x10²³ instead of 6022x10²⁰

Floating point numbers inside a computer work like that, more or less (and sometimes pretty much exactly like that^*.) So you re-write your number as N.FFFFFFx10^e, and the total number of digits (number of F's plus 1 for the N) has to fit within the "6 digits of precision" available from the AVR's 32bit floating point format.) Notice that this is NOT the same as the number of digits AFTER the decimal point, in the original number. 10000.2 gets to be 1.00002x10⁴ requires six digits of precision.

As you've noticed, the errors introduced by this have a nasty way of propagating if you're not careful. 1000.1 - 1000.0 ought to be exactly 0.1, but you've "wasted" 4 digits of precision in there.
(This USED to occupy a good portion of a semester-long class in "numerical methods", and people would carefully pick and/or design algorithms that minimized such propagation. Nowadays most people just increase the size of the datatype - got a mainframe? Quad-precision with 30+ digits is common, and octal-precision with 60+ digits might be available too.)

^* The main wait that the computer representation differs from scientific notation is that the base of the exponent will probably be 2, rather than 10, and the mantissa will be a binary number. That's why you see "between 6 and 7 digits of precision" - it's actually "exactly 24 bits", but that doesn't work out to a exact number of decimal digits...

Koepel · January 18, 2022, 3:00am

The answer of your sketch is:

0.9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999

I got that result with BigNumber:

// Sketch from here:
//   https://forum.arduino.cc/t/trying-to-understand-floating-point-accuracy/948268

// Test floating point calculations and division

#include "BigNumber.h"

// The compiler uses int and long and so on for the constants.
// With large constant values, the BigNumber has trouble
// to convert them.
// So the constants are written down as text string.
BigNumber num1 = "10000";
BigNumber num2 = "3333";
BigNumber num3 = "100000";
BigNumber num4 = "33333";
BigNumber num5 = "1000000";
BigNumber num6 = "333333";

void setup() {

  BigNumber::begin(100);   // 100 digits after the dot

  Serial.begin (9600);
  Serial.println ("Start Calculating");
  Serial.println ("--------");
  Serial.println (num1);  // should print 1000000.00000
  num1 = (num1 / BigNumber(3));
  Serial.println (num1);  // should print 33333.33333
  num1 = num1 - num2;
  Serial.println (num1);  // should print 0.33333
  num1 = num1 * BigNumber(3);
  Serial.println (num1);  // should print 0.99999

  Serial.println ("--------");
  Serial.println (num3);  // should print 100000.00000
  num3 = (num3 / BigNumber(3));
  Serial.println (num3);  // should print 33333.33333
  num3 = num3 - num4;
  Serial.println (num3);  // should print 0.33333
  num3 = num3 * BigNumber(3);
  Serial.println (num3);  // should print 0.99999

  Serial.println ("--------");
  Serial.println (num5);  // should print 1000000.00000
  num5 = (num5 / BigNumber(3));
  Serial.println (num5);  // should print 333333.33333
  num5 = num5 - num6;
  Serial.println (num5);  // should print 0.33333
  num5 = num5 * BigNumber(3);
  Serial.println (num5);  // should print 0.99999
}

void loop() {
}

The sketch in Wokwi:

You should be very careful with the BigNumber library and check every calculation. In a other topic I am completely lost how to use BigNumber.
A Arduino board or compatible that can do double precision floating point calculations is way better than messing around with BigNumber.

GolamMostafa · January 18, 2022, 6:37am

https://forum.arduino.cc/t/complex-math-in-arduino/946934/1

Topic		Replies	Views
Strange float behaviour on Arduino Mega Programming	28	527	June 28, 2025
Complex Math in arduino Programming	55	8002	January 17, 2022
Basic Math Error I can't resolve Programming	7	2011	December 20, 2013
Why Arduino can't deal with high precision numbers? Programming	16	3495	June 11, 2018
Pow accuracy and data types Programming	20	728	October 16, 2023

Trying to understand floating point accuracy

Related topics