Go Down

### Topic: Simple math doing my head in - when is an int treated like an int? (Read 1 time)previous topic - next topic

#### davetapson

##### Aug 18, 2012, 11:00 pm
Hi there

I'm trying to do something simple - convert analogue pin input (0 - 1023) to byte equivalent (0 - 255).

All variables declared as int.

I do this (i.e. input value x 255/1023) and all starts going strange at about value 130.

Serial.print("Pin Value: ");
Serial.println(potValue);
// pot pin value will be from 0 - 1023
// convert this to 0 - 255
result = potValue * 255/1023;
Serial.print("Result: ");
Serial.println(result);

Pin Value: 128
Result: 31

Pin Value: 131
Result: -31

If I replace the "255 / 1023" with 0.25, then all works fine

result = potValue * 0.25; //255 / 1023;

Pin Value: 215
Result: 53

Now I understand that intermidiate int calcs are treated as ints, which may cause a problem with 255/1023, but 0.25 is no more an int than 255/1023?

Can anyone tell me what's up?

#### majenko

#1
##### Aug 18, 2012, 11:07 pm
An int can represent any integer value between -32768 and +32767

Anything outside this range wraps around.

130*255/1023.  Think about how that sum works.

130 * 255 = 33150

That's outside the amount that an int will store.

You should either use an "unsigned int" that can store between 0 and 65535, or a "long" which can store considerably more (we're talking billions here).

#### MarkT

#2
##### Aug 18, 2012, 11:39 pm
Is there a reason you are not dividing by 4 (in other words why 255 and not 256, 1023 and not 1024?)

Code: [Select]
`result = potValue / 4 ;`
or with shifting right:
Code: [Select]
`result = potValue >> 2 ;`

Or if the slight difference matters, there is a built-in function specifically for this task:

Code: [Select]
`result := map (potValue, 0, 1023, 0, 255) ;`
http://arduino.cc/en/Reference/map
[ I will NOT respond to personal messages, I WILL delete them, use the forum please ]

#### AWOL

#3
##### Aug 18, 2012, 11:39 pm
Quote
I'm trying to do something simple - convert analogue pin input (0 - 1023) to byte equivalent (0 - 255).
So why are you going about it in such a complicated way?
0..1023 represents a span of ten bits.
0..255  represents a span of eight bits.

1023 / 22 = ?
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

#### AWOL

#4
##### Aug 18, 2012, 11:40 pm
Code: [Select]
`result := map (potValue, 0, 1023, 0, 255) ;`
Have we got our Pascal head on ?
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

#### MarkT

#5
##### Aug 19, 2012, 12:44 am
Sorry, = not := (and its Spin, not Pascal, in fact!)
[ I will NOT respond to personal messages, I WILL delete them, use the forum please ]

#### davetapson

#6
##### Aug 19, 2012, 08:35 am

130*255/1023.  Think about how that sum works.

130 * 255 = 33150

That's outside the amount that an int will store.

Ah. Ok.    Thanks.

Thanks for the input guys - as for the why not divide by 4, I saw the 1023 and decided it would not quite be 4.  doh.

#### davetapson

#7
##### Aug 19, 2012, 01:21 pm
Is this behaviour (intermediate calcs being influenced by data type of variable) just Arduino, or is it C?  Damn sure this has never happened to me in years of Java..?

#### AWOL

#8
##### Aug 19, 2012, 01:56 pm
Mostly C, I'd say, with a smidgin of AVR - it is all to do with the platform and the default size of an "int".
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

#### westfw

#9
##### Aug 19, 2012, 10:07 pm
Mostly AVR.  it's one of the few platforms where an int is only 16 bits.

#### drone

#10
##### Aug 19, 2012, 10:39 pm

Is this behaviour (intermediate calcs being influenced by data type of variable) just Arduino, or is it C?  Damn sure this has never happened to me in years of Java..?

C/C++ have that issue everywhere... Just try this anywhere =)

Code: [Select]
`int foo = 20;int bar = 35;float baz = 0.0;baz = foo / bar;`

Go Up

Please enter a valid email to subscribe

To complete the subscription, please click the link in the email we just sent you.

Thank you for subscribing!

Arduino
via Egeo 16
Torino, 10131
Italy