Python-Arduino USB communication floats problem

Hello!!!

i have made a project that a python script communicates with arduino sending various data types.
Well everything works great except when arduino sends back floats in some cases.

For e.g:
When arduino sends numbers 4112.5, -7631.5 python receive them correct
In case of 4112.112, -7631.23 python receives 4112.11181641, -7631.22998047

What is causing this??

Python code:

import os 
import struct 
import serial
import time
print('HELLO WORLD!!!!\nI AM PYTHON READY TO TALK WITH ARDUINO\nINSERT PASSWORD PLEASE.')
ser=serial.Serial("COM5", 9600) #Serial port COM5, baudrate=9600
ser.close()
ser.open() #open Serial Port
a = int(raw_input("Enter number: ")) #integer object
b = int(raw_input("Enter number: ")) #integer object
c = float(raw_input("Enter number: ")) #float object
d = float(raw_input("Enter number: ")) #float object
time.sleep(2) #wait 
ser.write(struct.pack("2i2f",a,b,c,d)) #write to port all all number bytes
if a == 22 :
 if b == -22 :
  if c == 2212.113 :
   if d == -3131.111 :
    print("Congratulations!!! Check the ledpin should be ON!!!")
    receivedbytes=ser.read(16) #read from Serial port 16 bytes=2 int32_t + 2 floats from arduino
    (number1,number2,number3,number4,)=struct.unpack("2i2f",receivedbytes) #convert bytes to numbers
    print "Arduino also send me back ",str(number1),",",str(number2),",",str(number3),",",str(number4)
   else :
      print("WRONG PASSWORD")
os.system("pause") #wait for user to press enter

Arduino code:

struct sendata { //data to send
  volatile int32_t a=53;
  volatile int32_t b=-2121;
  volatile float c=4112.5;
  volatile float d=-7631.5;
};

struct receive { //data to receive
  volatile int32_t a; //it will not work with int 
  volatile int32_t b;
  volatile float c;
  volatile float d;
};

struct receive bytes;
struct sendata values;


const int total_bytes=16; //total bytes to send
int i;
byte buf[total_bytes]; //each received Serial byte saved into byte array


void setup() {
  Serial.begin(9600);
  pinMode(13,OUTPUT); //Arduino Mega ledpin
  }

  
void loop() {
  
}

void serialEvent() { //Called each time Serial data is received
    if (Serial.available()==total_bytes){ //Receive data first saved to Serial buffer,Serial.available return how many bytes are saved.The Serial buffer space is limited.
       while(i<=total_bytes-1){
           buf[i] = Serial.read(); //Save each byte from Serial buffer to byte array
           i++;
       }
       memmove(&bytes,buf,sizeof(bytes)); //Move each single byte memory location of array to memory field of the struct,the numbers are reconstructed from bytes.
       if (bytes.a==22){ //Access each struct number.
         if (bytes.b==-22){
          if (bytes.c==2212.113){
            if (bytes.d==-3131.111){ //If the password is right
                Serial.write((const uint8_t*)&values,sizeof(values)); //Write struct to Serial port.
                delay(100);
                digitalWrite(13,HIGH);//Turn ON LED.
            }
          }
        }
      }
   }
}

For further information you can also check my video:

What is causing this??

Your misunderstanding about how floats are stored on each end of the pipe.

You also do not understand what the volatile keyword is for, or you wouldn't be splattering it all over the place.

You should have ONE struct for sending AND receiving. You create an instance of the struct, and populate it, to send data.

You receive data into another instance of the same struct, and extract the data of interest from that instance.

const int total_bytes=16; //total bytes to send

That is the WRONG way to declare the size of the struct.

PaulS:
Your misunderstanding about how floats are stored on each end of the pipe.

You also do not understand what the volatile keyword is for, or you wouldn't be splattering it all over the place.

You should have ONE struct for sending AND receiving. You create an instance of the struct, and populate it, to send data.

You receive data into another instance of the same struct, and extract the data of interest from that instance.

const int total_bytes=16; //total bytes to send

That is the WRONG way to declare the size of the struct.

Thank's for the answer!!!
So if i correct the code some way, will it work that way with structs?

So if i correct the code some way, will it work that way with structs?

The "problem" has nothing to do with structs. You are sending 2 longs and 2 floats from a python script, and assuming that the Arduino will use the bytes you send to create exactly the same longs and floats as existed on the PC. How many bytes is a float, on the PC? Which endian-ness is the Python script using to send data? Which endian-ness does the Arduino use?

Comparing floats using == is generally not a good idea. 0.5 can be stored exactly in a float. 0.1 can not.

Converting the float you get back to a string, and expecting the printed representation to be exactly the keyed in value is foolish. Print the stored representation of the keyed in value exactly the same way, if you expect two strings to match.

You need to examine the bytes that the Arduino receives when Python sends a float so you can check if Python has encoded the float in the same way that an Arduino does.

Sending integers is probably easier and to preserve decimals the value could be multiplied by 100 or 1000.

And if the extra performance from sending binary data is not actually needed things will be much easier to debug if you send the data as human readable text.

...R
Serial Input Basics - simple reliable ways to receive data.

After some tests i find out that 8-bit arduino and Python can send floats with maximum 3 decimal places with accuracy!!!

I also wrote a non struct code, here are the results:

NON_STRUCT

Arduino side: https://drive.google.com/file/d/1lvgT-LqQa7DxDorFF0MTe7UMpBfHn6LA/view?usp=sharing

Python side: https://drive.google.com/file/d/1gPKfhTvbd4vp4L4VrZuns95yQoekg-vn/view?usp=sharing

WITH_STRUCT better performance

Arduino side: https://drive.google.com/file/d/153fuSVeMz2apI-JbDNjdkw9PQKHfGDGI/view?usp=sharing

Python side: https://drive.google.com/file/d/1M6iWnluXdNzTKO1hfcsk3qi9omzMiYeh/view?usp=sharing

Youtube video: - YouTube

panagiotis96:
After some tests i find out that 8-bit arduino and Python can send floats with maximum 3 decimal places with accuracy!!!

I would be very suspicious of that. It may fail sometime. The concept of "float" is not limited to 3 decimal places.

And pleas post code in the Forum so we don't need to go to another website.

...R

Robin2:
I would be very suspicious of that. It may fail sometime. The concept of "float" is not limited to 3 decimal places.

And pleas post code in the Forum so we don't need to go to another website.

...R

As PaulS said in a previous post it's not a good idea to compare floats with "==".
In fact something like 'if (fabs(float1-float2)<epsilon (epsilon could be=0.0001...)'which means float1=float2 is better.
So when you limit (or round) the float only to 3 decimal places its like compare the difference with epsilon. I also tested many numbers that i had problem and worked properly.

The big disadvantage is that you are limited to 3 decimal.

NON_STRUCT

arduino

//values to send//
int32_t aa=53; 
int32_t bb=-2121;
float cc=4112.3; //each float must have max 3 decimal places else it will rounded to 3!!
float dd=-7631.23;
////***/////

///values to receive////
int32_t a; //it will not work with int 
int32_t b;
float c;
float d;
int i,e;
/////****////

void setup() {
  Serial.begin(9600);
  pinMode(13,OUTPUT); //Arduino Mega ledpin
  }

  
void loop() {
  
}

void serialEvent() { //Called each time Serial data is received
   a=Serial.parseInt();
   b=Serial.parseInt();
   c=Serial.parseFloat();
   d=Serial.parseFloat();
   if (a==22){ //Access each struct number.
      if (b==-22){
         if (c==2212.113){
            if (d==-3131.111){ //If the password is right
                Serial.println(aa);
                Serial.println(bb);
                Serial.println(cc,3); //must be <=3 decimal places else it will rounded
                Serial.println(dd,3); //must be <=3 decimal places else it will rounded
                delay(100);
                digitalWrite(13,HIGH);//Turn ON LED.
            }
         }
      }
   }
}

python

import os 
import struct 
import serial
import time
print('HELLO WORLD!!!!\nI AM PYTHON READY TO TALK WITH ARDUINO\nINSERT PASSWORD PLEASE.')
ser=serial.Serial("COM5", 9600) #Serial port COM5, baudrate=9600
ser.close()
ser.open() #open Serial Port
a = int(raw_input("Enter number: ")) #integer object
b = int(raw_input("Enter number: ")) #integer object
c = float(format(float(raw_input("Enter number: ")), '.3f'))#float object <=3 decimal places
d = float(format(float(raw_input("Enter number: ")), '.3f'))
time.sleep(2) #wait 
ser.write(str(a).encode()) #convert int to string and write it to port 
ser.write('\n'.encode())
ser.write(str(b).encode())
ser.write('\n'.encode())
ser.write(str(c).encode())
ser.write('\n'.encode())
ser.write(str(d).encode())
ser.write('\n'.encode())
if str(a) == "22" :
 if str(b) == "-22" :
  if str(c) == "2212.113" :
   if str(d) == "-3131.111" :
    print("Congratulations!!! Check the ledpin should be ON!!!")
    number1=int(ser.readline()) #read from Serial port convert to int
    number2=int(ser.readline())
    number3=float(ser.readline()) ##read from Serial port convert to float (3 decimal places from arduino)
    number4=float(ser.readline())
    print "Arduino also send me back ",str(number1),",",str(number2),",",str(number3),",",str(number4)
   else :
      print("WRONG PASSWORD")
os.system("pause") #wait for user to press enter

STRUCT

arduino

struct sendata { //data to send
  volatile int32_t a=53;
  volatile int32_t b=-2121;
  volatile float c=4112.3;
  volatile float d=-7631.4;
};

struct receive { //data to receive
  volatile int32_t a; //it will not work with int 
  volatile int32_t b;
  volatile float c;
  volatile float d;
};

struct receive bytes;
struct sendata values;


const int total_bytes=16; //total bytes to send
int i;
byte buf[total_bytes]; //each received Serial byte saved into byte array


void setup() {
  Serial.begin(9600);
  pinMode(13,OUTPUT); //Arduino Mega ledpin
  }

  
void loop() {
  
}

void serialEvent() { //Called each time Serial data is received
    if (Serial.available()==total_bytes){ //Receive data first saved to Serial buffer,Serial.available return how many bytes are saved.The Serial buffer space is limited.
       while(i<=total_bytes-1){
           buf[i] = Serial.read(); //Save each byte from Serial buffer to byte array
           i++;
       }
       memmove(&bytes,buf,sizeof(bytes)); //Move each single byte memory location of array to memory field of the struct,the numbers are reconstructed from bytes.
       if (bytes.a==22){ //Access each struct number.
         if (bytes.b==-22){
          if (bytes.c==2212.113){
            if (bytes.d==-3131.111){ //If the password is right
                Serial.write((const uint8_t*)&values,sizeof(values)); //Write struct to Serial port.
                delay(100);
                digitalWrite(13,HIGH);//Turn ON LED.
            }
          }
        }
      }
   }
}

python

import os 
import struct 
import serial
import time
print('HELLO WORLD!!!!\nI AM PYTHON READY TO TALK WITH ARDUINO\nINSERT PASSWORD PLEASE.')
ser=serial.Serial("COM5", 9600) #Serial port COM5, baudrate=9600
ser.close()
ser.open() #open Serial Port
a = int(raw_input("Enter number: ")) #integer object
b = int(raw_input("Enter number: ")) #integer object
c = float(format(float(raw_input("Enter number: ")), '.3f'))#float object <=3 decimal places
d = float(format(float(raw_input("Enter number: ")), '.3f'))
time.sleep(2) #wait 
ser.write(struct.pack("2i2f",a,b,c,d)) #write to port all all number bytes
if a == 22 :
 if b == -22 :
  if c == 2212.113 :
   if d == -3131.111 :
    print("Congratulations!!! Check the ledpin should be ON!!!")
    receivedbytes=ser.read(16) #read from Serial port 16 bytes=2 int32_t + 2 floats from arduino
    (number1,number2,number3,number4,)=struct.unpack("2i2f",receivedbytes) #convert bytes to numbers
    number3=float(format(number3, '.3f')) #floats must be under 3 decimal points else will be rounded
    number4=float(format(number4, '.3f'))
    print "Arduino also send me back ",str(number1),",",str(number2),",",str(number3),",",str(number4)
   else :
      print("WRONG PASSWORD")
os.system("pause") #wait for user to press enter

panagiotis96:
The big disadvantage is that you are limited to 3 decimal.

It seems to me that what you really mean is that if you use more than 3 decimal places you can't reliably use == to compare two floats.

That is a limitation you are imposing on yourself and has nothing to do with the way Python and the Arduino communicate.

...R

How does the Arduino find the start of the data packet? That delay(100) is going to get the buffer filled up with unidentifiable crap while you are not reading it.

I think that the problem is on arduino.

For e.g this prints 4411.29980 as python receives (so no serial crap) instead of 4411.3. If you round the data to 3 decimal places the result is 4411.300.

float data=4411.3;
void setup(){
  Serial.begin(9600); //Begin Serial
}

void loop()
{
  Serial.print(data,5);  // Print test variable as string
  Serial.println();
}

panagiotis96:
I think that the problem is on arduino.

The problem is that you mis-understand how floating point variables are stored. They are NOT precise.

To give you a practical example, the banking system has never used floating point maths to record dollars and cents (and that's only 2 decimal places) because it would constantly give rounding errors.

If you need precision store the number as an integer and if you want "decimal places" then store (for example) 1234.5678 as 12345678

...R