Accuracy loss while converting from int to float

Projects Programming
1

I am reading a value of 9375 from EEPROM stored as an INT.

Then I am casting it to a float variable and dividing it with 1000000. The value I expect to get is 0,009375. But what i get is 0.01 after a round off. My process that uses this value gets messed up due to the rounding off … What am i doing wrong ??

EEPROM.get ( eeAddress, eepromVal) ;
Serial.println(eepromVal);              // Reads 9375. The correct value
voltPerScan = float(eepromVal) / 1000000 ; 
Serial.println(voltPerScan);            // Reads 0.01 . Why is it rounding it off ?
2

When you print a float using Serial.print() you can specify how many decimal places are printed. If you don't do that then the function defaults to printing 2 decimal places

Try this

Serial.println(voltPerScan, 7);  //print 7 decimal places
3

Great !! Thanks !!

4

I am glad that it worked

Is there any reason why you don't save the float value in EEPROM to avoid converting the integer ?

As a matter of interest, what data type is eepromVal ?

5

Footnote:
If you have an integer of the type int32_t or uint32_t and it is rather large (uses 24-31 bits),
it cannot be represented by a float exactly as a float has only 23 bits mantissa.

6

You have wanted to see six digits after the decimal point.

But, I have got: 0.0093750 (seven digit after decimal point) after executing:
Serial.println(voltPerScan, 7); which is right.

So, if the 2nd argument of this code: Serial.println(float number, arg2) indicates the number of digits after the decimla point, then the actual code should be:

Serial.println(voltPerScan, 6);

The default is 2 digts.

7

have a look at - GitHub - RobTillaart/printHelpers: Arduino library to help formatting data for printing - which is a collection of formatters for numbers.e.g. print any float in scientific or engineering format.

e.g. scientific Serial.print(sci(0.009375, 3)) ==> 9.375E-3

The standard Serial.print() cannot print floats larger than ~4 billion while the range of floats goes up to 10^38. On the lower side the standard print round to 0.00 by default. To print very small numbers you need up to 45 decimals (subnormal floats).

8

Good doubt !

I simply wanted to handle only INT types. It is easy to trasnmit these over serial from my LabVIEW application . Nothing else !!

9

I just work with Voltage values between 0- 5V and the maximum precision is the value 0.009999

10

1. You want 6-digit precision (number of digits after the decimal point). Using Serila.print(float, n) command, you can print as many digits as you wish.

2. How many accurate digits you want after the decimal point?

3. If using Arduino UNO R3's 10-bit ADC at Full Scale (FS) = 5V, how much accuracy can you achieve?

11

I am using a MCP 4725 DAC which is 12 bits.

So for a 5V span I should get a per bit resolution of 1.22mv . Which is 0.00122 V.

Accuracy hence cannot be better than this.

12

check - GitHub - RobTillaart/MCP4725: Arduino library for 12 bit I2C DAC - MCP4725

It is easy to add an float getMilliVolts()to the library if needed. That will give you good readable numbers in the low range without using 6 decimals. Just another scale.

Created an issue so it is on my backlog - Add getMilliVolts() wrapper function · Issue #36 · RobTillaart/MCP4725 · GitHub

13

You mean 5-digit accuracy after decimal point?

Which Arduino are you using?

14

@Ardubit
I have created a develop branch in my MCP4725 library, implementing milliVolts wrappers.
It might (or not) solve your needs by handling the device with mV instead of Volts.

Expect to merge and release 0.4.3 later today

update: merged and released