Go Down

Topic: Simple math doing my head in - when is an int treated like an int? (Read 915 times) previous topic - next topic

davetapson

Hi there

I'm trying to do something simple - convert analogue pin input (0 - 1023) to byte equivalent (0 - 255).

All variables declared as int.

I do this (i.e. input value x 255/1023) and all starts going strange at about value 130.

  // read potentiometer pin
  potValue = analogRead(potPin);
  Serial.print("Pin Value: ");
  Serial.println(potValue);
  // pot pin value will be from 0 - 1023
  // convert this to 0 - 255
  result = potValue * 255/1023;
  Serial.print("Result: ");
  Serial.println(result);


Pin Value: 128
Result: 31

Pin Value: 131
Result: -31

If I replace the "255 / 1023" with 0.25, then all works fine
 
result = potValue * 0.25; //255 / 1023;

Pin Value: 215
Result: 53

Now I understand that intermidiate int calcs are treated as ints, which may cause a problem with 255/1023, but 0.25 is no more an int than 255/1023?

Can anyone tell me what's up?

majenko

An int can represent any integer value between -32768 and +32767

Anything outside this range wraps around.


130*255/1023.  Think about how that sum works.

130 * 255 = 33150

That's outside the amount that an int will store.

You should either use an "unsigned int" that can store between 0 and 65535, or a "long" which can store considerably more (we're talking billions here).
Get 10% off all 4D Systems TFT screens this month: use discount code MAJENKO10

MarkT

Is there a reason you are not dividing by 4 (in other words why 255 and not 256, 1023 and not 1024?)

Code: [Select]
result = potValue / 4 ;
or with shifting right:
Code: [Select]
result = potValue >> 2 ;

Or if the slight difference matters, there is a built-in function specifically for this task:

Code: [Select]
result := map (potValue, 0, 1023, 0, 255) ;
http://arduino.cc/en/Reference/map
[ I won't respond to messages, use the forum please ]

AWOL

Quote
I'm trying to do something simple - convert analogue pin input (0 - 1023) to byte equivalent (0 - 255).
So why are you going about it in such a complicated way?
0..1023 represents a span of ten bits.
0..255  represents a span of eight bits.

1023 / 22 = ?
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

AWOL

Code: [Select]
result := map (potValue, 0, 1023, 0, 255) ;
Have we got our Pascal head on ?   ;)
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

MarkT

[ I won't respond to messages, use the forum please ]

davetapson



130*255/1023.  Think about how that sum works.

130 * 255 = 33150

That's outside the amount that an int will store.



Ah. Ok.  :smiley-red:  Thanks.

Thanks for the input guys - as for the why not divide by 4, I saw the 1023 and decided it would not quite be 4.  doh.

davetapson

Is this behaviour (intermediate calcs being influenced by data type of variable) just Arduino, or is it C?  Damn sure this has never happened to me in years of Java..?

AWOL

Mostly C, I'd say, with a smidgin of AVR - it is all to do with the platform and the default size of an "int".
"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

westfw

Mostly AVR.  it's one of the few platforms where an int is only 16 bits.

drone


Is this behaviour (intermediate calcs being influenced by data type of variable) just Arduino, or is it C?  Damn sure this has never happened to me in years of Java..?


C/C++ have that issue everywhere... Just try this anywhere =)

Code: [Select]

int foo = 20;
int bar = 35;
float baz = 0.0;

baz = foo / bar;

Go Up