Gang, was doing some simple calculations involving a double and an unsigned int. Here is the routine as a code snippet:
void say_decimal(double num, int precision)
{
//say the left side of the number first
unsigned int leftPart = num;
say_left(leftPart);
speak("point.wav");
num -= leftPart;
for (int i = 0; i< precision; i++)
{
num = num * 10; //note line #1
Serial.println(num); //note line #2
Serial.println((unsigned int)num); //note line #3
speak((unsigned int)num); //note line #4
num -= (unsigned int)num; //note line #5 }
}
The problem is that I am processing a number that ends in .23, but when the code executes, the following happens. On the second time through the loop, num starts out equal to 0.3, after "note line #1",
num then contains 3.0. It is printed out on line #2 as that. On line #3, what is printed out is 2. I do not understand why when the double 3.0 is converted to an unsigned int the result is not a 3. Am I missing something>
While you say that the number ends in .23, it is probably stored internally as something more like 0.2299997. When the number is displayed by Serial.println, the default is to show the value to 6 decimal places, so it is rounded to 0.23 for display purposes. That has no effect on the stored value.
When you multiply by 10, the number becomes 2.2999997, which gets truncated to 2 as an int.
The value ending in 0.97 probably actually ends in 0.9700002, which does what you expect.
Intresting and it maybe the source of the issue, but when I print out the number after it is multiplied by 10. it prints as 3.00. In your scenario I would expect at least 2.99. I am going to create a little scetch to test this out in a more compact way and then maybe others can run it and test too.
OK, here is a complete sketch that show what is happening.
void setup() {
Serial.begin(9600); // set up Serial library at 9600 bps for debugging
}
void loop() {
test_truncation(234.230,2);
test_truncation(234.790,2);
test_truncation(234.970,2);
test_truncation(234.870,2);
test_truncation(234.820,2);
delay(200000);
}
void test_truncation(double num, int precision)
{
//say the left side of the number first
unsigned int leftPart = num;
Serial.print("num=");
Serial.print(num);
Serial.print(" leftPart=");
Serial.print(leftPart);
//say_left(leftPart);
//speak("point.wav");
num -= leftPart;
//Serial.print(" num - leftPart=");
//Serial.print(num);
Serial.print(" . ");
for (int i = 0; i< precision; i++)
{
num = num * 10; //note line #1
Serial.print(" num*10.0=");
Serial.print(num);
Serial.print(" digit=");
Serial.print((unsigned int)num); //note line #3
//speak((unsigned int)num); //note line #4
num -= (unsigned int)num; //note line #5
//Serial.print(" num-(unsigned int)num=");
//Serial.print(num);
}
Serial.println();
}
So I am concluding that what you are saying is exactly right. The 3.00 is really a 2.9999999, but when I do the truncation I am left with only the 2.
I have confirmed this by using a different method. I do all the multiplication by 10s at the start and that moves the correct digits to the left of the decimal. I think round and move it into a long and I have my digits.
Another thing to do, to illustrate what is going on, is to use sprintf to convert the number to a string. Using "%.12f" as a format statement will show the value to 12 decimal places. This will let you see that the Serial.print function is rounding the number it prints, when it shows two decimal places.