# Computational error with Uno R3 **URL:** \ **Category:** Programming\ **Created:** [July 5, 2023, 11:31pm UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068 "2023-07-05T23:31:27Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![damirk1944](https://avatars.discourse-cdn.com/v4/letter/d/4bbf92/32.png) [@damirk1944](https://forum.arduino.cc/u/damirk1944)\ **Post date:** [July 5, 2023, 11:31pm UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/1 "2023-07-05T23:31:28Z")
Using IDE2.1.0 on Linux Mint 21.1 and Arduino Uno R3. Result of calculation is incorrect for some but not all values. Here is a sample code: ```arduino void setup() { // put your setup code here, to run once: Serial.begin(9600); float value1 = 32.757; float value2 = 32.758; float value3 = 32.759; float value4 = 32.760; float value5 = 32.761; float value6 = 32.762; long result1 = (long)(value1 * 2000.0); long result2 = (long)(value2 * 2000.0); long result3 = (long)(value3 * 2000.0); long result4 = (long)(value4 * 2000.0); long result5 = (long)(value5 * 2000.0); long result6 = (long)(value6 * 2000.0); Serial.print(result1); Serial.print("\t"); Serial.print(result2); Serial.print("\t"); Serial.print(result3); Serial.print("\t"); Serial.print(result4); Serial.print("\t"); Serial.print(result5); Serial.print("\t"); Serial.print(result6); Serial.print("\t"); Serial.println(""); } void loop() { // put your main code here, to run repeatedly: } ``` it prints results as follows: 65514 65516 65517 65519 65522 65524 Clearly, the results 3 and 4 are wrong. Unfortunately, I do not have another Uno to see if the problem is caused by bad Atmega chip or bad software. Your help would be much appreciated. Best regards ---
**Author:** ![anon94801460](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@anon94801460](https://forum.arduino.cc/u/anon94801460)\ **Post date:** [July 5, 2023, 11:36pm UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/2 "2023-07-05T23:36:31Z")
> [@damirk1944](#): > > Clearly, the results 3 and 4 are wrong. That is perfectly typical float to integer truncation. Float values are not exact. Try this code. It should make clear what is happening. ```arduino void setup() { Serial.begin(115200); while (!Serial); float x= 32.759; Serial.println(x,6); Serial.println(x*2000.0,6); } void loop() {} ``` ---
**Author:** ![GolamMostafa](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/golammostafa/32/239011_2.png) [@GolamMostafa](https://forum.arduino.cc/u/GolamMostafa)\ **Post date:** [July 6, 2023, 12:18am UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/3 "2023-07-06T00:18:39Z")
> [@damirk1944](#): > > 65514 65516 65517 65519 65522 65524 > Clearly, the results 3 and 4 are wrong. **1.** What are your expected umbers? **2.** If you use paper-pencil to calculate 32.758 x 2000.0, you will get 65,516.0000. **3.** To see the figure/number of Section-2 on the Serial Monitor, execute the following sketch: ```arduino void setup() { Serial.begin(9600); float y = 32.758*2000.0; Serial.println(y, 4);//4-digit after decimal point } void loop() {} ``` **4. Output:** ```arduino 65516.0000 ``` **5.** > [@damirk1944](#): > > `long result1 = (long)(value2 * 2000.0);` In the above code, you have told the MCU to perform 32.758 x 2000.0; the MCU has indeed calculated 65516.0000. But, you have told the Compiler to discard the fractional part by appending **(long)** (called casting) at RHS; as a result, you have got 65516 which is what you have asked for. ---
**Author:** ![camsysca](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/camsysca/32/1015228_2.png) [@camsysca](https://forum.arduino.cc/u/camsysca)\ **Post date:** [July 6, 2023, 12:31am UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/4 "2023-07-06T00:31:53Z")
To see the problem with a bit more clarity, change `long result1 = (long)(value1 * 2000.0);` etc. to `float result1 = (float)(value1 * 2000.0);` What does this reveal? Remember, a float has approximately six(IIRC) decimal digits of resolution, due to the limits of the 4-byte implementation of a single-precision floating number. When you see the results, then google rounding vs truncation, and see what you can infer. ---
**Author:** ![qubits-us](https://avatars.discourse-cdn.com/v4/letter/q/5daacb/32.png) [@qubits-us](https://forum.arduino.cc/u/qubits-us)\ **Post date:** [July 6, 2023, 12:58am UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/5 "2023-07-06T00:58:51Z")
seems like some rounding is in order.. try.. ```arduino long result3 = round(value3 * 2000.0); long result4 = round(value4 * 2000.0); ``` good luck.. ~q ---
**Author:** ![damirk1944](https://avatars.discourse-cdn.com/v4/letter/d/4bbf92/32.png) [@damirk1944](https://forum.arduino.cc/u/damirk1944)\ **Post date:** [July 6, 2023, 1:16pm UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/6 "2023-07-06T13:16:00Z")
Thank you all for quick replies. What confused me was that 'value' looks like scaled 15-bit number and the 'result' is a 16-bit number. Of course, the scaling factor 1/1000 is not a power of two... Best regards. P.S. How do I mark this topic as 'solved'? ---
**Author:** ![anon94801460](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@anon94801460](https://forum.arduino.cc/u/anon94801460)\ **Post date:** [July 6, 2023, 1:33pm UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/7 "2023-07-06T13:33:11Z")
> [@damirk1944](#): > > the 'result' is a 16-bit number Type "long" is a 32 bit signed integer on typical Arduinos. ---
**Author:** ![GolamMostafa](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/golammostafa/32/239011_2.png) [@GolamMostafa](https://forum.arduino.cc/u/GolamMostafa)\ **Post date:** [July 6, 2023, 4:40pm UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/8 "2023-07-06T16:40:38Z")
> [@damirk1944](#): > > `float value1 = 32.757;` With reference to the above variable (the valu1), can you please explain what have you wanted to mean by "scaled 15-bit number"? When a float number is declared, it (32.757) is saved in memory as 32-bit number into four consecutive memory locations in _binary32 (IEEE-754)_ format (Fig-1). ![floatmem32-757](https://europe1.discourse-cdn.com/arduino/original/4X/e/9/e/e9e52c6273d035e59d7524e448fa8b4620b96352.png) Figure-1: ---
**Author:** ![almytom](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/almytom/32/345495_2.png) [@almytom](https://forum.arduino.cc/u/almytom)\ **Post date:** [July 6, 2023, 5:03pm UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/9 "2023-07-06T17:03:54Z")
Totally agree with the precision analysis but want to point out that if you believe computation is wrong in the Arduino you can always verify by running the same calculations on your computer to see if it's the same. I compiled and ran the following on my Intel-based Mac. Note that I explicitly forced the lengths to be completely compatible: ```arduino #include int main(void) { float value1 = 32.757F; float value2 = 32.758F; float value3 = 32.759F; float value4 = 32.760F; float value5 = 32.761F; float value6 = 32.762F; int32_t result1 = (int32_t)(value1 * 2000.0F); int32_t result2 = (int32_t)(value2 * 2000.0F); int32_t result3 = (int32_t)(value3 * 2000.0F); int32_t result4 = (int32_t)(value4 * 2000.0F); int32_t result5 = (int32_t)(value5 * 2000.0F); int32_t result6 = (int32_t)(value6 * 2000.0F); printf("%d\t%d\t%d\t%d\t%d\t%d\n", result1, result2, result3, result4, result5, result6); } ``` The program printed: ```arduino 65514 65516 65517 65519 65522 65524 ``` exactly the same as the Arduino. ---
**Author:** ![GolamMostafa](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/golammostafa/32/239011_2.png) [@GolamMostafa](https://forum.arduino.cc/u/GolamMostafa)\ **Post date:** [July 6, 2023, 5:09pm UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/10 "2023-07-06T17:09:52Z")
> [@almytom](#): > > ` float value1 = 32.757F;` 32.757 is already a float number and yet you have appended **F** -- why? ---
**Author:** ![almytom](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/almytom/32/345495_2.png) [@almytom](https://forum.arduino.cc/u/almytom)\ **Post date:** [July 6, 2023, 7:16pm UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/11 "2023-07-06T19:16:14Z")
> [@GolamMostafa](#): > > 32.757 is already a float number and yet you have appended **F** -- why? 32.757 is a double precision (64-bit) floating point number. The "F" makes it single precision (32-bit). Since the compiler used for the Arduino doesn't support double precision, when you use 32.757 there it is a single precision constant. So I wanted to make sure the comparison was as close as possible. If I leave off all the "F"s I get a different result, but then the calculations aren't the same as in the Arduino: ```arduino 65513 65515 65517 65519 65522 65524 ``` ---
**Author:** ![westfw](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/westfw/32/213443_2.png) [@westfw](https://forum.arduino.cc/u/westfw)\ **Post date:** [July 6, 2023, 7:58pm UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/12 "2023-07-06T19:58:04Z")
> [@GolamMostafa](#): > > > [@almytom](#): > > > > `float value1 = 32.757F;` > > 32.757 is already a float number and yet you have appended **F** -- why? On most desktops, 32.757 is a double. In this case it doesn't matter, but you can/will get different code for: ```arduino float result, input; result = input * 1000.0; // input, constant, and calc will be promoted to "double." // vs result = input * 1000.0F; // everything stays as "float." ``` (which can be somewhat important on ARM chips with single-precision floating point hardware. Being careful to use float constants instead of the default double constants can result in much smaller and faster code.) ---
**Author:** ![GolamMostafa](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/golammostafa/32/239011_2.png) [@GolamMostafa](https://forum.arduino.cc/u/GolamMostafa)\ **Post date:** [July 7, 2023, 2:09am UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/13 "2023-07-07T02:09:30Z")
> [@westfw](#): > > ```arduino > float result, input; > result = input * 1000.0; // input, constant, and calc will be promoted to "double." > > ``` **1.** That means that the declaration _float input = 32.757;_ or _double input = 32.757;_ in 32-bit architecture like ESP8266 will store the number as 64-bit number (_0x404060E560418937_). The data type _float_ will be promoted to _double_. **2.** If we declare like _float input = 32.757F_ in 32-bit arch, then the number 32.757 will be stored as _0x4203072B_ and the data type will remain _float_. **3.** In 8-bit architecture (ATmega328P), the declaration _float input = 32.757;_ or _float input = 32.757F;_ will always store the number as _0x4003072B_. ---
**Author:** ![westfw](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/westfw/32/213443_2.png) [@westfw](https://forum.arduino.cc/u/westfw)\ **Post date:** [July 7, 2023, 5:47am UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/14 "2023-07-07T05:47:54Z")
> [@GolamMostafa](#): > > _float input = 32.757;_ or _double input = 32.757;_ in 32-bit architecture like ESP8266 will store the number as 64-bit number (_0x404060E560418937_). The data type _float_ will be promoted to _double_. Theoretically, the right hand side is calculated as a double, but then it is silently converted to a float (32bit) when stored in the float variable (which is also 32bits.) In reality, the compiler will be smart enough NOT to do this for constants (and even for some expressions.) But consider the common example of adjusting an analogRead to a floating point value: ```arduino float sensorValue = (analogRead(sensorPin)*5.0)/1024.0; ``` On an Uno R4 (32bit ARM), this compiles to the following invocations of (expensive) double-precision math: ```arduino 411e: f001 fcf7 bl 5b10 4122: f005 f813 bl 914c <__aeabi_i2d> ;; convert int to double. 4126: 2200 movs r2, #0 4128: 4b18 ldr r3, [pc, #96] ; (418c ) 412a: f005 f879 bl 9220 <__aeabi_dmul> ;; double multiply 412e: 2200 movs r2, #0 4130: 4b17 ldr r3, [pc, #92] ; (4190 ) 4132: f005 f875 bl 9220 <__aeabi_dmul> ;; another double multiply (the divide?) 4136: f005 fb43 bl 97c0 <__aeabi_d2f> ;; convert the double result back to float ``` But if I do: ```arduino float sensorValue = (analogRead(sensorPin)*5.0f)/1024.0f; ``` I get this nice short sequence of the native ARM floating point operations: ```arduino 411e: f001 fcf3 bl 5b08 4122: ee07 0a90 vmov s15, r0 4126: eeb8 8ae7 vcvt.f32.s32 s16, s15 ;; convert int to float. 412a: eef1 7a04 vmov.f32 s15, #20 412e: ee28 8a27 vmul.f32 s16, s16, s15 ;; float multiply instruction 4132: eddf 7a15 vldr s15, [pc, #84] ; 4188 4136: 7820 ldrb r0, [r4, #0] 4138: ee28 8a27 vmul.f32 s16, s16, s15 ;; second float mutiply ``` > In 8-bit architecture (ATmega328P)... Note that the use of 32bits for "double" on AVR is a compiler _choice_ that may not apply to all 8bit CPUs. It would not be totally unexpected for some other compiler to do the intermediate operations with 64bit doubles as well. ---
**Author:** ![system](https://dub1.discourse-cdn.com/arduino/user_avatar/forum.arduino.cc/system/32/1140315_2.png) [@system](https://forum.arduino.cc/u/system)\ **Post date:** [January 3, 2024, 5:48am UTC](https://forum.arduino.cc/t/computational-error-with-uno-r3/1145068/15 "2024-01-03T05:48:46Z")
This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.