Serial data transmission - weird alteration of data

[EDIT] Clarification of question. [/EDIT]

I have connected my Arduino Nano with a Raspberry Pi 3 using Serial transmission. The Pi uses a Python script which sends a string to the Arduino. Once this is received, the Arduino increments the first two digits (representing an index), overwriting the original char array in which the string is stored. This is then sent back to the Pi.

The weird thing is this; while the index (first two digits) is successfully incremented and over-written, other random two digits are also getting incremented. This is very weird since as you can see in my code below, the function altering these digits (buildIndex()) is specifically calling characters at locations 0 and 1 in my array.

Any idea what could be causing this? I have also, at one point, included a counter with the serialEvent function to check how many times it is being called. The counter increments twice for each string sent, while it should be just once obviously.

Code:

#include <LiquidCrystal.h>
#include <stdio.h>
#include <stdlib.h>

LiquidCrystal lcd(12, 11, 5, 4, 3, 2);
char sign[300]="";
int init_index = 0;
int rgb_values = 0;
int sensor_location = 0;
bool chk = false;

void setup(){
  Serial.begin(115200);
  lcd.begin(16,2);
  lcd.print("Hello World");
}

void refreshSign() {
  for(int i = 0; i < sizeof(sign); i++) {
    sign[i] = '\0';
  }
  Serial.flush();
}

void parseSign() {  //parse the sign into a string
  
  char tmp[2];
  //bool chk = false;
  
  tmp[0] = sign[0];
  tmp[1] = sign[1];
  init_index = atoi(tmp);

}

void buildIndex() { //building of index into new sign
  //chk = true;
  int incr_index = init_index+1;
  //Serial.write(incr_index);
  char tmp[2];
  itoa(incr_index, tmp, 10);
  if(incr_index <= 9) {
    sign[0] = '0';
    sign[1] = tmp[0];
  } else {
    sign[0] = tmp[0];
    sign[1] = tmp[1];
  }
  //Serial.write(sign);
}

void serialEvent() { //to trigger upon receiving sign
  int index = 0;
  lcd.clear();

  while(Serial.available() > 0) { 
    if(index < sizeof(sign)) {
      sign[index] = Serial.read();
      index++;
    } else {
      char ext = Serial.read(); 
    }
  }
  sign[index]='\0';
  parseSign();
  buildIndex();
  for(int i=0; i < index; i++) { 
    Serial.print(sign[i]);
    lcd.print(sign[i]);
  }
  refreshSign();
}

void loop() {
  }

Python Code on Pi:

if __name__ == "__main__":
    t0 = datetime.datetime.now()
    t2=datetime.datetime.now()
    t1=datetime.datetime.now()
    s=""
    s1=""
    #testStr = "(233,43,12):(13,13,13):(24,5,67)%(3.05,4.06,12.3)%3"
    try:
        with serial.Serial('/dev/ttyAMA0', baudrate=115200,rtscts=True, dsrdtr=True, timeout=0.05) as port:
            s1 = "03abcdefghijkl%112233445566778899001122\0"
            print s1       
            t1=datetime.datetime.now()
            print("Pre IO: "+ str((t1-t0).total_seconds()*1000)+"ms")
            port.write(s1)
            print("write IO: "+ str((t1-t0).total_seconds()*1000)+"ms")
            
            s = port.readline()
            
    except serial.SerialException:
        port.close()
        port.open()
    except Exception as e:
        print(e)
        pass  
    t2=datetime.datetime.now()
    print (s)

    print("read IO: "+ str((t2-t1).total_seconds()*1000)+"ms")

if port.open():
    port.close()
print("End: "+ str((datetime.datetime.now()-t2).total_seconds()*1000)+"ms")
print("Total: "+ str((datetime.datetime.now()-t0).total_seconds()*1000)+"ms")

What are you "up to"? Why this strange string embedded counter?

The idea of the string is, to later on, be interpreted by the Arduino. Depending on the values held in the string, the Arduino decides on what action to take. That, however, is beside the point.

darnegar:
The idea of the string is, to later on, be interpreted by the Arduino. Depending on the values held in the string, the Arduino decides on what action to take. That, however, is beside the point.

I would say, hardly. If it just increments every time you get a new string, why not just increment an integer variable?

Edit - I see that you've edited your original post. Be prudent with that, it can make the thread replies appear confusing.

Does the PI not know how many strings it has sent? Why does it depend on feedback from the Arduino to tell it that? It seems to me that both the PI and the Arduino can easily independently keep count of how many strings have been sent/received.

I'm trying to avoid doing that because I want to only transmit one signal containing all the relevant information to the Arduino. Appending an integer means having to send two signals, which increases time overhead which has to be avoided at all costs. Regardless, I just want to know where the error in the string parsing lies due to these limitations. Even if it changes later.

darnegar:
I'm trying to avoid doing that because I want to only transmit one signal containing all the relevant information to the Arduino. Appending an integer means having to send two signals, which increases time overhead which has to be avoided at all costs. Regardless, I just want to know where the error in the string parsing lies due to these limitations. Even if it changes later.

But why would you need to send an integer? If I send "foo" and "bar", they are #1 and #2, and so on... right? I don't have to send "#1 - foo" and "#2 - bar".

void parseSign() {  //parse the sign into a string
  
  char tmp[2];
  
  tmp[0] = sign[0];
  tmp[1] = sign[1];
  init_index = atoi(tmp);

}

What if the memory location after tmp[1] contains '3' ? :slight_smile:

darnegar:
...which increases time overhead which has to be avoided at all costs...

Sending a datetime as a string isn't a very good design then.

I don't think you're actually understanding the issue here. The string is designed in such a way as to make it ax compact and small as possible. It is divided into pre-determined groups of characters, the index being one of them, consisting of strictly two digits (00 - 99). No date-time data is being sent.

darnegar:
I don't think you're actually understanding the issue here. The string is designed in such a way as to make it ax compact and small as possible. It is divided into pre-determined groups of characters, the index being one of them, consisting of strictly two digits (00 - 99). No date-time data is being sent.

Okay, it's your party. Have you fixed parseSign() yet?

I'm not sure I follow. Would you kindly specify what is wrong with my function?

See reply 6. You are missing a null on the end of your string.

I don't believe I need that though since I am referring to the elements in my array using their specific index.

I guess it hast be changed in this way:

void parseSign() {

   char tmp[2] = "";

   tmp[0] = sign[0];
   tmp[1] = sign[1];
   init_index = atoi(tmp);

}

it is possible that the SerialEvent is not reading all the data from the pi in one go
i.e. it reads block of the incomming data incrementing the first two digits then reads another block of data inrementing the first two digits etc
try putting a delay in the loop

   if(index < sizeof(sign)) {
      sign[index] = Serial.read();
      delay(1);
      index++;
    } else {

"I don't believe I need that though since I am referring to the elements in my array using their specific index."

The atoi function may feel a bit differently.

Actually Horace that indeed seems to be my problem.

I have tried adding the delay just as you have suggested and it returned the value perfectly, fixing the problem.

However, given the time constraints on the system I am building (multiple Arduinos connected to each other increase the time taken significantly), this does seem a bit sketchy.

Would there be any other fix I can apply to bypass this problem and strike a balance between the time taken and the validity of data?

The delay is a kludge.

the delay was just a test to identify the problem!
are the frames a fixed length?
if so fill an array with data until you have the require number of bytes then process the frame

if the frames are variable length send the first byte to specify the length (0 to 255) then the data

an alternative is to have a frame start (say STX) and end (say ETX) bytes and use byte stuffing if the data contains the end frame indicator (ETX)

Have a look at this Python - Arduino demo and at the examples in Serial Input Basics - simple reliable ways to receive data. There is also a parse example.

...R