hopefully you will understand in what I'm trying to do, I'm just been playing around doing some maths and sending it to serial port. The maths is all correct but I need to only have the last 3 digits past the decimal point like this 0.001, the code below works out the Shunt_mv / Shunt_am = 0.001071429 which is correct. Anon is calculated to 48.15 so when I carry out this formula OP_gain = (1 / (48.15 * 0.001071429 )) = 19.38 which is correct but gives me wrong readings in my other program if I do it like this OP_gain = (1 / (48.15 * 0.001 )) = 20.76 (this gives correct readings in my other program. I need to ignore the (X)'s 0.001XXXXXX this is possible, I can manually enter the value 0.001 but would like to do it automatically as it's part of a set up procedure
Anon is calculated to 48.15 so when I carry out this formula OP_gain = (1 / (48.15 * 0.001071429 )) = 19.38 which is correct but gives me wrong readings in my other program if I do it like this OP_gain = (1 / (48.15 * 0.001 )) = 20.76 (this gives correct readings in my other program.
This does not sound like a math problem, and I think we need to see the "other program".
cattledog:
This does not sound like a math problem, and I think we need to see the "other program".
I don't think it's the a math problem and it's not the other program if you enter on the calculator 0.075/70 = the result is 0.001071429 and then (1 / (48.15 * 0.001071429 )) the result is 19.38 where if you enter (1 / (48.15 * 0.001 )) the result is 20.76 which is minus the 5 digits, So Arduino is working out correct.
Even if I print
Serial.println(shunt_gain, 3);
The result is the same which it does the same in excel result is 0.001071429, If I multiply it by 1000 I get 1.071428571
Delta_g I don't quite understand what you mean, store it as an integer or long integer and use fixed point math with it.
If I manually enter 0.001 in my equation the results are correct
So you seem to be saying that your other program is truncating that second number to 0.001, and you need your arduino to match what that other program is doing.
You can easily truncate the number to the nearest thousanth like so:
but there's a good chance that this won't match your other program in all cases. Maybe the other program is truncating, maybe it's rounding; maybe rounding up, maybe rounding down; maybe it's truncating to a nearest decimal digit, maybe a binary one; maybe to a specific place, maybe to a specific number of places.
It all depends.
Your real problem is that you are trying to compare floating-point numbers for exact equality. This, well, never works out at all. You need to think about treating floating point values as approximations with a tolerance.
The other program is extctly the same, in excel if I do this =(round(0.075/70,3))=0.001 then this works correct =(1/(48.10.001))= 20.79 where as in excel if you do this 0.075/70=0.00171428571 which it is the default calculation which is the same answer arduino but when you do this (1/(48.10.00174142))= 19.40
So how do you do a round function in arduino