Go Down

### Topic: Why won't this convert properly?! It is driving me maddddd (Read 2621 times)previous topic - next topic

#### PeterMHull

##### Apr 12, 2013, 07:29 amLast Edit: Apr 12, 2013, 07:50 am by PeterMHull Reason: 1
So I've tried converting the data in the following code both using casting and conversion methods.
i.e.  int(variable) and (int) Variable

The offending code has been isolated to this portion, which i stripped down from the rest of the project:

void loop()
{
double total = 0;
int y = 0;

for(int x = 0; x <= 7; x++)
{
while(y<=x)
{
total += pow(2,x);
y++;
}
Serial.print("Unconverted total (double Type): ");
Serial.println(total);
int intTotalCast = (int)total;
int intTotalConversion = int(total);

Serial.print("Converted using casting (int Type): ");
Serial.println(intTotalCast);
Serial.print("Converted using Converter (int Type): ");
Serial.println(intTotalConversion);
delay(1000);
Serial.println("Loop Starts Over");
}
Here's the output:

Unconverted total (double Type): 1.00
Converted using casting (int Type): 1
Converted using Converter (int Type): 1
Loop Starts Over
Unconverted total (double Type): 3.00
Converted using casting (int Type): 3
Converted using Converter (int Type): 3
Loop Starts Over
Unconverted total (double Type): 7.00
Converted using casting (int Type): 6
Converted using Converter (int Type): 6
Loop Starts Over
Unconverted total (double Type): 15.00
Converted using casting (int Type): 14
Converted using Converter (int Type): 14
Loop Starts Over
Unconverted total (double Type): 31.00
Converted using casting (int Type): 30
Converted using Converter (int Type): 30
Loop Starts Over
Unconverted total (double Type): 63.00
Converted using casting (int Type): 62
Converted using Converter (int Type): 62
Loop Starts Over
Unconverted total (double Type): 127.00
Converted using casting (int Type): 126
Converted using Converter (int Type): 126
Loop Starts Over
Unconverted total (double Type): 255.00
Converted using casting (int Type): 254
Converted using Converter (int Type): 254

As you can see, the converted values start off correct, 1:1, 3:3, then it goes weird... 7:6, 15:14, 31:30, etc...

every single value after 3 is off by one, i don't get it!!!! Please help, I'm out of ideas.

BTW, what I'm trying to do basically... is count up in binary... to get this pattern:
00000001
00000011
00000111
00001111
00011111
00111111
01111111
11111111

Edit:

#### WizenedEE

#1
##### Apr 12, 2013, 07:57 am
Because of the way doubles are stored, they can't be perfectly precise. So when you type 5 it might hear 5.00000012 or 4.9999984. So while 5.000000012 casted to an int (which is always rounded down) may be 5, 4.99999984 casted to an int is 4, which is obviously incorrect.

So instead of rounding down, you want to round to nearest. The easiest way to do this is to add 0.5 and then round down:
Code: [Select]
` int intTotalCast = (int)(total+.5);`

See if that helps you

#### PeterMHull

#2
##### Apr 12, 2013, 08:22 am
Thanks, that worked.  Even with your explanation I don't understand how that could happen.  I mean, i would think a data type of double would imply even  more precision, rather then less.  I would expect it to store 5.00000000000000000000. Also, I assumed it would simply truncate the value, so that everything to the right of the decimal point would be chopped off and discarded.   It seems silly... but it worked! thanks again.

#### WizenedEE

#3
##### Apr 12, 2013, 08:28 am
Thanks, that worked.  Even with your explanation I don't understand how that could happen.  I mean, i would think a data type of double would imply even  more precision, rather then less.

Nope. Fixed point types (int, long, byte) always have more precision than floating point (float, double) of the same length since they operate over a much smaller range of numbers. But even with ten thousand digits after the decimal point of precision, you can't store precisely 5 the way arduino does, just like you can't specify 1/3 precisely in decimal (it's ABOUT 0.3333333). And even if it's 0.00000000000000000001 below 5, it just chops off the stuff after the decimal point leaving you with four.
Quote
Also, I assumed it would simply truncate the value, so that everything to the right of the decimal point would be chopped off and discarded.

it does. That's why 4.99999999999 makes 4.

#### AWOL

#4
##### Apr 12, 2013, 08:32 am
If you want powers of two, why are you using floating point types at all?
The processor is perfectly happy working in binary.
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.
I speak for myself, not Arduino.

#### PeterMHull

#5
##### Apr 12, 2013, 08:45 am
Quote
it does. That's why 4.99999999999 makes 4.

ok, but why does it return 5.00 when displaying it's double form?  It rounds for println purposes?

Quote
If you want powers of two, why are you using floating point types at all?
The processor is perfectly happy working in binary.

I'm playing with someone elses code and that's the datatype they have the class setup to accept.

#### WizenedEE

#6
##### Apr 12, 2013, 08:48 amLast Edit: Apr 12, 2013, 08:50 am by WizenedEE Reason: 1

Quote
it does. That's why 4.99999999999 makes 4.

ok, but why does it return 5.00 when displaying it's double form?  It rounds for println purposes?

Yes, the implementation of print(double) uses the same trick I showed you. From Print.cpp:
Code: [Select]
`size_t Print::printFloat(double number, uint8_t digits) {  size_t n = 0;    if (isnan(number)) return print("nan");  if (isinf(number)) return print("inf");  if (number > 4294967040.0) return print ("ovf");  // constant determined empirically  if (number <-4294967040.0) return print ("ovf");  // constant determined empirically    // Handle negative numbers  if (number < 0.0)  {     n += print('-');     number = -number;  }  // Round correctly so that print(1.999, 2) prints as "2.00"  double rounding = 0.5;  for (uint8_t i=0; i<digits; ++i)    rounding /= 10.0;    number += rounding;  // Extract the integer part of the number and print it  unsigned long int_part = (unsigned long)number;  double remainder = number - (double)int_part;  n += print(int_part);  // Print the decimal point, but only if there are digits beyond  if (digits > 0) {    n += print(".");  }  // Extract digits from the remainder one at a time  while (digits-- > 0)  {    remainder *= 10.0;    int toPrint = int(remainder);    n += print(toPrint);    remainder -= toPrint;  }    return n;}`

if you're still curious:
http://en.wikipedia.org/wiki/Double-precision_floating-point_format

#### PeterMHull

#7
##### Apr 12, 2013, 10:36 am
Alright, I see what your saying.  I'll give that link a look tomorrow, thanks.

#8
##### Apr 12, 2013, 10:55 am

And if you're really curious...
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

#### Fletcher_Chr

#9
##### Apr 12, 2013, 05:48 pm
Hi

Quote
BTW, what I'm trying to do basically... is count up in binary... to get this pattern:
00000001
00000011
00000111
00001111
00011111
00111111
01111111
11111111

Code: [Select]
`void setup(){  Serial.begin(115200);  int lines=8;  int number=1;    for (int s=lines-1;s>=0;s--){    for (int t=lines-1;t>=0;t--){      Serial.print(bitRead(number,t));    }    Serial.println();    number<<=1;  // bitshift number one to the left    number+=1;   // add 1 to the right most bit  }}void loop(){}`

Output:
00000001
00000011
00000111
00001111
00011111
00111111
01111111
11111111

-Fletcher

#### AWOL

#10
##### Apr 12, 2013, 06:43 pm
Quote
what I'm trying to do basically... is count up in binary... to get this pattern:
00000001
00000011
00000111
00001111
00011111
00111111
01111111
11111111
which of course is unary, not binary.
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.
I speak for myself, not Arduino.

#### Delta_G

#11
##### Apr 12, 2013, 09:22 pm

Quote
what I'm trying to do basically... is count up in binary... to get this pattern:
00000001
00000011
00000111
00001111
00011111
00111111
01111111
11111111
which of course is unary, not binary.

Can you explain what you mean by that?  They look like binary numbers to me...
If at first you don't succeed, up - home - sudo - enter.

#### AWOL

#12
##### Apr 12, 2013, 09:44 pm
Well, of course they're binary numbers.
Just not consecutive binary numbers.
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.
I speak for myself, not Arduino.

Go Up