serial communication with 2 bytes terminator

ansami · April 21, 2020, 4:57am

I am tring to realize communication with a self-defined protocol device using serial port. The frame length is not fixed but the terminator is fixed. So i want to use the readBytesUntil(char terminator, char *buffer, size_t length) function to get the frame which i want.
However, the terminator in the self-defined protocol is a 2 bytes terminator which is 0x0D0A, and the parameter char terminator is 1 byte. So how can i realize the fixed 2 bytes terminator communication?

And more ：what is the better choice when i meet some problem like this?(the standard library can not satisfy my need). is it a good idea to creat a derived class to extend the standard library?

sterretje · April 21, 2020, 5:11am

Read till 0x0A. Next strip of the 0x0D.

Note that readBytesUntil can block; to prevent that, you can use approaches as described in Robin's updated Serial Input Basics thread. It also might answer your second question.

GolamMostafa · April 21, 2020, 7:10am

sterretje:
Read till 0x0A. Next strip of the 0x0D.

@OP
Try this sketch (tested on NANO):

char myData[50] = "";

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  byte n = Serial.available();
  if (n != 0)
  {
    byte m = Serial.readBytesUntil('\n', myData, 50);//0x0A(\n) is not saved
    byte x1 = myData[m-1];
    if(x1 == '\r')//0x0D
    {
      Serial.println("Message received with 0x0D0A as terminator.");
      Serial.println(myData);
    }
  }
}

sterretje · April 21, 2020, 7:25am

@GolamMostafa

It might be better to replace the '\r' by '\0'; that way you get rid of it completely as OP wants.

GolamMostafa · April 21, 2020, 7:32am

sterretje:
It might be better to replace the '\r' by '\0'; that way you get rid of it completely as OP wants.

Yes!

The modified codes:

char myData[50] = "";
void setup()
{
  Serial.begin(9600);
}

void loop()
{
  byte n = Serial.available();
  if (n != 0)
  {
    byte m = Serial.readBytesUntil('\n', myData, 50);//0x0A(\n) is not saved
    byte x1 = myData[m-1];
    if(x1 == '\r')//0x0D
    {
      myData[m-1] = '\0' ; //inserted null character
      Serial.println("Message received with 0x0D0A as terminator.");
      Serial.println(myData);
    }
  }
}

Power_Broker · April 21, 2020, 5:35pm

ansami:
And more ：what is the better choice when i meet some problem like this?(the standard library can not satisfy my need). is it a good idea to creat a derived class to extend the standard library?

What "standard" library are you referencing?

ansami · April 23, 2020, 3:21am

sterretje:
Read till 0x0A. Next strip of the 0x0D.

Note that readBytesUntil can block; to prevent that, you can use approaches as described in Robin's updated Serial Input Basics thread. It also might answer your second question.

thanks a lot. It is really a good and clear tutorial for serial and helps me a lot.

ansami · April 23, 2020, 4:57am

GolamMostafa:
Yes!

The modified codes:
char myData[50] = "";

void setup()
{
Serial.begin(9600);
}

void loop()
{
byte n = Serial.available();
if (n != 0)
{
byte m = Serial.readBytesUntil('\n', myData, 50);//0x0A(\n) is not saved
byte x1 = myData[m-1];
if(x1 == '\r')//0x0D
{
myData[m-1] = '\0' ; //inserted null character
Serial.println("Message received with 0x0D0A as terminator.");
Serial.println(myData);
}
}
}

Yep, it is a method to solve this problem. And your code suits the situation of my question well. However, after reading the tutorialSerial Input Basics which sterretje suggested, I think it can be better in my situation for two reason:1. The funtion readBytesUntil() is blocking function. 2. Two-bytes terminator is used because one of 0x0A and 0x0D may show in data(sorry for my defective expounding).

const byte numChars = 32;
char receivedChars[numChars];

boolean newData = false;

void setup() {
    Serial.begin(9600);
    Serial.println("<Arduino is ready>");
}

void loop() {
    recvWithStartEndMarkers();
    showNewData();
}

void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = 0x3e;
    char endMarker[2] = {0x0d, 0x0a};
    char rc[2];
 
    while (Serial.available() > 0 && newData == false) {
        rc[0] = Serial.read();
        rc[1] = rc[0];

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc[1];
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc[1] == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
        Serial.print("This just in ... ");
        Serial.println(receivedChars);
        newData = false;
    }
}

And this is the code i briefly changed from "example 3" in that tutorial.

Power_Broker · April 23, 2020, 5:02am

Power_Broker:
What "standard" library are you referencing?

ansami · April 23, 2020, 5:33am

Power_Broker:
What "standard" library are you referencing?

I mean the functions in the resources->reference page of the arduino official site.

Robin2 · April 23, 2020, 7:24am

ansami:
I am tring to realize communication with a self-defined protocol device using serial port.

If it's a self-defined protocol why add the complexity of a 2-byte terminator/

I don't think this will work to compare two arrays

if (rc != endMarker) {

If you insist on using two end-markers then the correct procedure is to read an extra byte when the first marker is detected and then check that extra byte for the second end-marker.

Something like this might work

void recvWithTwpEndMarkers() {
    static byte ndx = 0;
    char endMarker = '\r';
    char endMarker2 = '\n';
    char rc;
    static bool waitingForSecondMarker = false;
    
    while (Serial.available() > 0 && newData == false) {
        rc = Serial.read();
        
        if (rc == endMarker) {
            waitingForSecondMarker = true;
        }

        if (waitingForSecondMarker == true {
            if (rc == endMarker2) {
                receivedChars[ndx] = '\0'; // terminate the string
                ndx = 0;
                newData = true;
                waitingForSecondMarker = false;
                return;
            }
            waitingForSecondMarker = false;
        }

        if (waitingForSeondMarker == false) {
            receivedChars[ndx] = rc;
            ndx++;
            if (ndx >= numChars) {
                ndx = numChars - 1;
            }
        }
    }
}

...R

sterretje · April 23, 2020, 7:30am

Actually it's simpler

Just use '\r' as the endMarker in the code. The '\n' will be ignored when reading a next 'packet'. It will stay in the buffer, though, till such time.

Robin2 · April 23, 2020, 8:39am

sterretje:
Actually it's simpler

Just use '\r' as the endMarker in the code.

I did think of that but I assumed that a '\r' on its own might legitimately occur within the data - if not there is no need for a 2-byte terminator.

...R

sterretje · April 23, 2020, 9:08am

Robin2:
I did think of that but I assumed that a '\r' on its own might legitimately occur within the data - if not there is no need for a 2-byte terminator.

...R

Fair enough.

GolamMostafa · April 23, 2020, 10:23am

When numerical data are transmitted in ASCII, there are only frames for 0 - F (0x30 - 0x39, 0x41 - 0x46). How, \r (0x0D) or \n (0x0A) could appear there as an element of the data?

When text data are transmitted in ASCII, there are only frames for the printable charterers of the English Language (A- Z, a -z, numerals, punctuation marks, special charcaters). How, \r or \n could appear there as an element of the data?

Robin2 · April 23, 2020, 10:37am

GolamMostafa:
When text data are transmitted in ASCII, there are only frames for the printable charterers of the English Language (A- Z, a -z, numerals, punctuation marks, special charcaters). How, \r or \n could appear there as an element of the data?

Maybe the data is not exclusively human readable text.

I assume the OP has some reason for the complication of a 2-byte end marker.

...R

Power_Broker · April 24, 2020, 4:10am

ansami:
I mean the functions in the resources->reference page of the arduino official site.

That doesn't answer my question, but ok. Are you referring to an actual ".h" library or the Arduino API?

ansami · April 26, 2020, 2:07am

Power_Broker:
That doesn't answer my question, but ok. Are you referring to an actual ".h" library or the Arduino API?

no I just mean the functions on that reference page.

ansami · April 26, 2020, 2:21am

Robin2:
I did think of that but I assumed that a '\r' on its own might legitimately occur within the data - if not there is no need for a 2-byte terminator.

...R

yep, it is a self protocol of a device i got which consist largely of ASCII. But there is one byte in the protocol for slave machine device address which can be data in 0~255 theoretically. I think this is the reason of using 2 bytes terminator.

I fully agree that your code is good for this purpose. however i was puzzled why 2 arrays can not be campared . Is there any good material for learning?thanks a lot.

Robin2:
I don't think this will work to compare two arrays
if (rc != endMarker) {

Power_Broker · April 26, 2020, 3:25am

Ok, that makes sense.

If you want to use a real library, you can use SerialTransfer.h to automatically packetize and parse your data for inter-Arduino communication without the headace. The library is installable through the Arduino IDE and includes many examples.

Here are the library's features:

This library:

can be downloaded via the Arduino IDE's Libraries Manager (search "SerialTransfer.h")

works with "software-serial" libraries

is non blocking

uses packet delimiters

uses consistent overhead byte stuffing

uses CRC-8 (Polynomial 0x9B with lookup table)

allows the use of dynamically sized packets (packets can have payload lengths anywhere from 1 to 254 bytes)

can transfer bytes, ints, floats, and even structs!!

Example TX Arduino Sketch:

#include "SerialTransfer.h"

SerialTransfer myTransfer;

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

void loop()
{
  char buff[] = "hi";

  myTransfer.txObj(buff, sizeof(buff));
  myTransfer.sendData(sizeof(buff));
  delay(100);
}

Example RX Arduino Sketch:

#include "SerialTransfer.h"

SerialTransfer myTransfer;

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

void loop()
{
  if(myTransfer.available())
  {
    char buff[40];
    
    myTransfer.rxObj(buff, sizeof(buff));
    
    Serial.println("New Data: ");
    Serial.write(buff, sizeof(buff));
    Serial.println();
  }
  else if(myTransfer.status < 0)
  {
    Serial.print("ERROR: ");

    if(myTransfer.status == -1)
      Serial.println(F("CRC_ERROR"));
    else if(myTransfer.status == -2)
      Serial.println(F("PAYLOAD_ERROR"));
    else if(myTransfer.status == -3)
      Serial.println(F("STOP_BYTE_ERROR"));
  }
}

For theory behind robust serial communication, check out the tutorials Serial Input Basics and Serial Input Advanced.

Topic		Replies	Views
Reading serial data without 'terminator' Programming	34	9214	May 5, 2021
Serial.readBytesUntil() terminator char, signed vs unsigned Programming	20	1489	September 7, 2022
serial readbytes until Programming	3	2278	May 5, 2021
Understanding ==> Serial.read[Bytes\|String][Until] Programming	13	8370	May 5, 2021
Does readBytesUntil INCLUDE the terminator you specify? Programming	6	1618	May 5, 2021

serial communication with 2 bytes terminator

Related topics