Robust Serial read with delimiters

Hi,

I am struggling to robustly read integers that I send over a serial port.

Objective;
I want to read x,y and z values without errors/glitches.

If the format/string doesn't match the format that I expect, I want to reject the result and read the new serial input.

Possible solution;
I want to check the format of the Serial.read value. Only if it matches "start,x,y,z,end" I want to save the integers x,y and z as individual integers.

Problem;
I cannot seem to find a solution that suits my needs. Can someone help me out?

I cannot seem to find a solution that suits my needs.

What have you tried? How did that “not suit your needs”?

SuperR:
Objective;
I want to read x,y and z values without errors/glitches.

If the format/string doesn't match the format that I expect, I want to reject the result and read the new serial input.

Possible solution;
I want to check the format of the Serial.read value. Only if it matches "start,x,y,z,end" I want to save the integers x,y and z as individual integers.

What about a format sent like that:

"x,y,z,crc\n"

With x,y,z ==> your int values, ASCII-formatted
crc ==> checksum to test for validity
\n ==> newline character

In that case you could use the CRC value to check if all the values had been received without any errors/glitches.

What do you think?

The examples in serial input basics are simple and reliable. There is also a parse demo to show how to convert the received data to integers.

...R

This is the code I am running to try the speed and robustness of a checksum method. I want to throw values at the arduino >10 times per second.

If I send a single; 1,2,3,11,
It responds with 1. Good.

However, if I the send command repeaditly, the results are slow/bad. It does not respond quickly and I need to wait for >1s to send a new data command. Only then will it respond with a 1. It seems that there is a timeout 'thingy' going on somewhere.

"Fix";
I want to see if a command is received, then get that command, analyze it, and when it suits my criteria, spit out the value.

int d1 = 0;
int d2 = 0;
int d3 = 0;
int d4 = 0;

void setup() {
  // initialize serial:
  Serial.begin(9600);
}

void loop() {
  // if there's any serial available, read it:
  while (Serial.available() > 0) {
    d1 = Serial.parseInt();
    d2 = Serial.parseInt();
    d3 = Serial.parseInt();
    d4 = Serial.parseInt();
    if (d1+d2+d3+5 == d4){
     Serial.println(d1);
    }}
  }

Okay, Serial.available() returns something greater than 1, which you are reading at 9600 baud. In your code:

  while (Serial.available() > 0) {
    d1 = Serial.parseInt();
    d2 = Serial.parseInt();
    d3 = Serial.parseInt();
    d4 = Serial.parseInt();

d1 captures the byte and the next series of parseInt() calls are going to zip along at some speed determined by the instruction set and a 16MHz clock while the Serial object is lobbing things at you at 9600 baud. Think that might be a problem?

SuperR:
"Fix";
I want to see if a command is received, then get that command, analyze it, and when it suits my criteria, spit out the value.

Here is a “line receive” function and a loop that checks for 4 values in the line, seperated by comma:
x as first value
y as second value
z as third value
crc as fourth value

CRC is in this case just the sum of the 3 values x,y,z.

So if you send:

1,2,3,6

you get a “CRC OK”

And if the last value is not there or if the sum is not correct, you get a “CRC fail”.

#define SEPERATING_CHAR 13  // ASCII-13 is '\r' carriage return character that seperates two lines
char* receiveLine(char c)
{
  static char lineBuffer[41];  // define maximum buffer length here (max string length +1)
  static byte counter=0;
  if (counter==0) memset(lineBuffer,0,sizeof(lineBuffer)); // clear buffer before using it
  if (c==SEPERATING_CHAR)
  {
    counter=0;
    return lineBuffer;
  }
  else if (c>=32 && counter<sizeof(lineBuffer)-1) 
  { // if is it a printable ASCII-character and fits into the lineBuffer
    lineBuffer[counter]=c; // insert character into lineBuffer
    counter++;
  }
  return NULL;
}


void setup() {
  Serial.begin(9600);
}

void loop() {
  char *str, *substr;
  int x,y,z,crc;
  if (Serial.available())
  {
    str=receiveLine(Serial.read());
    if (str!=NULL) // we have received a complete line
    {
      Serial.print("Input: ");
      substr=strtok(str,",");
      x=atoi(substr);
      Serial.print(x);Serial.print(" - ");
      substr=strtok(NULL,",");
      y=atoi(substr);
      Serial.print(y);Serial.print(" - ");
      substr=strtok(NULL,",");
      z=atoi(substr);
      Serial.print(z);Serial.print(" - ");
      substr=strtok(NULL,",");
      crc=atoi(substr);
      if (substr!=NULL && crc==x+y+z) Serial.println("CRC OK");
      else Serial.println("CRC fail");
    }
  }
}

Please be sure to set the line endings option in the serial monitor to “Carriage Return” or “Both”!

If you want to transmit many lines per second, you should increase the baudrate to possibly 115200, 250000 or 500000.

SuperR:
However, if I the send command repeaditly, the results are slow/bad.

So try the examples in serial input basics. Serial.parseInt() is a blocking function. My code does not block.

…R

I’ve never used the Serial.parseInt() and not sure what it does. The below serial delimited parsing code might be a better approach to do what you want.

//zoomkat 11-12-13 String capture and parsing  
//from serial port input (via serial monitor)
//and print result out serial port
//copy test strings and use ctrl/v to paste in
//serial monitor if desired
// * is used as the data string delimiter
// , is used to delimit individual data 

String readString; //main captured String 
String angle; //data String
String fuel;
String speed1;
String altidude;

int ind1; // , locations
int ind2;
int ind3;
int ind4;
 
void setup() {
  Serial.begin(9600);
  Serial.println("serial delimit test 11-12-13"); // so I can keep track of what is loaded
}

void loop() {

  //expect a string like 90,low,15.6,125*
  //or 130,hi,7.2,389*

  if (Serial.available())  {
    char c = Serial.read();  //gets one byte from serial buffer
    if (c == '*') {
      //do stuff
      
      Serial.println();
      Serial.print("captured String is : "); 
      Serial.println(readString); //prints string to serial port out
      
      ind1 = readString.indexOf(',');  //finds location of first ,
      angle = readString.substring(0, ind1);   //captures first data String
      ind2 = readString.indexOf(',', ind1+1 );   //finds location of second ,
      fuel = readString.substring(ind1+1, ind2+1);   //captures second data String
      ind3 = readString.indexOf(',', ind2+1 );
      speed1 = readString.substring(ind2+1, ind3+1);
      ind4 = readString.indexOf(',', ind3+1 );
      altidude = readString.substring(ind3+1); //captures remain part of data after last ,

      Serial.print("angle = ");
      Serial.println(angle); 
      Serial.print("fuel = ");
      Serial.println(fuel);
      Serial.print("speed = ");
      Serial.println(speed1);
      Serial.print("altidude = ");
      Serial.println(altidude);
      Serial.println();
      Serial.println();
      
      readString=""; //clears variable for new input
      angle="";
      fuel="";
      speed1="";
      altidude="";
    }  
    else {     
      readString += c; //makes the string readString
    }
  }
}

@zoomkat: would strtok() be easier and use less memory?

econjack:
@zoomkat: would strtok() be easier and use less memory?

I can't say as I don't don't know the differences in methods/operation at the machine language level. The locating and indexing a byte in a memory location may actually be the same.

I was curious, so I tried using strtok(). I don't know about speed, but your version used 4502 bytes of program memory and 450 bytes of SRAM. The code below uses 2502 bytes of program memory and 338 bytes of SRAM. This kinda reconfirms my bias against using a String object in programs.

/*
String readString; //main captured String
String angle; //data String
String fuel;
String speed1;
String altidude;
*/
char readString[20];   // Too big for all of these??
char angle[4];
char fuel[5];
char speed1[6];
char altitude[5];

int ind1; // , locations
int ind2;
int ind3;
int ind4;

void setup() {
  Serial.begin(9600);
  Serial.println("serial delimit test 11-12-13"); // so I can keep track of what is loaded
}

void loop() {
  char *ptr;
  int charsRead;

  //expect a string like     90,low,15.6,125*
  //                  or    130, hi, 7.2,389*

  if (Serial.available())  {
    charsRead = Serial.readBytesUntil('*', readString, sizeof(readString) - 1);
    readString[charsRead] = '\0';

    Serial.println();
    Serial.print("captured String is : ");
    Serial.println(readString); //prints string to serial port out

    ptr = strtok(readString, ",");
    if (ptr != '\0') {
      strcpy(angle, ptr);
    }
    ptr = strtok('\0', ",");
    if (ptr != '\0') {
      strcpy(fuel, ptr);
    }
    ptr = strtok('\0', ",");
    if (ptr != '\0') {
      strcpy(speed1, ptr);
    }
    ptr = strtok('\0', ",");
    if (ptr != '\0') {
      strcpy(altitude, ptr);
    }



    Serial.print("angle = ");
    Serial.println(angle);
    Serial.print("fuel = ");
    Serial.println(fuel);
    Serial.print("speed = ");
    Serial.println(speed1);
    Serial.print("altitude = ");
    Serial.println(altitude);
    Serial.println();
    Serial.println();

    readString[0] = '\0'; //clears variable for new input

  }
}

This kinda reconfirms my bias against using a String object in programs.

I can see more programming space being taken for the code to dynamically manage the bytes in the SRAM memory location. Can you describe how the SRAM memory space is used by the two memory management methods? Dynamic memory management may keep the bytes in the SRAM memory locations consolidated together (like defragging the SRAM memory registers each time memory locations become vacated) vs. using static memory allocations that will always keep the SRAM memory registers fragmented. Can you explain just how the SRAM memory useage values are calculated/derived? A long time back I got a book about assembly language programming for the x86 chips, and found moving bits/bytes in and out of memory locations interesting. The same basic operations are called many different things depending on the language being used. Kind of like some arduino types refer to "sketches", and I just say code.

@zoomkat: Not too long after Gutenberg's breakthrough, I was teaching assembler classes at a midwest university. Alas, I haven't done much with assembler since. I do believe that there have been a number of posts here that could answer at least some of your questions. Part of the SRAM difference is that the String class brings in all of the static elements of the class, including elements that any given sketch doesn't use, which ends up bloating the SRAM usage higher than need be.

I've seen enough of your posts to know that you understand what's going on with Arduino code. Beginners, however, use the String class because it's easy to understand and, often, they are not writing complex code that "bumps into" SRAM limits. However, as they gain experience, many will drop the crutch that the String class offers them and discover the many str*() functions the Standard C Library makes available. I think the same arguments apply to the use of sprintf(), too.