Serial send and receive - Hopefully useful to someone

kpg · July 25, 2020, 12:04pm

As a newbie, this started out an exercise to send serial using lowByte, highByte in the program questions forum but on the good advice of others and guidance from Robin2's excellent 'Serial Input Basics', resulted in an exercise to generate send and receive serial with Char buffers and convert from and to an Int

Robin2's post is great to understand the receive end but I was getting stuck with the send end. So I experimented and had good results so I thought I would post this to hopefully help others

After lots of playing I have a system that seems to work although the receive is a bit unstable if Serial.prints are used in the sketch (not sure why yet).

Send code is;

#include <string.h>
char sentchar[7];
int rpm = 12341;//test variable to send
char endbuff [2];
char startbuff [8];//plus null plus end marker plus start marker
char endmarker = '>';
char startmarker = '<';
void setup() {
  Serial.begin(9600);
}

void loop() {
  memset(startbuff, 0x00, 8);
  memset(endbuff, 0x00, 2);
  memset(startbuff, 0x00, 2);
  endbuff[0] = endmarker;
  startbuff[0] = startmarker;
  itoa (rpm, sentchar, 10);
  strcat(sentchar, endbuff);
  strcat(startbuff, sentchar);
  Serial.write (startbuff);//reports <12341> on serial monitor
}

Receive code is;

//Receive with an start and end-marker
#include <string.h>

const byte numBin = 6;//do not write start and end markers
char receivedBin[numBin];
boolean newData = false;

void setup() {
  Serial.begin(9600);//on Nano
  Serial1.begin(9600);//on Mega
  Serial.println("<Arduino is ready>");
}

void loop() {
  recvWithStartEndMarker();
  showNewData();
}
void recvWithStartEndMarker() {
  static boolean recvInProgress = false;
  static byte ndx = 0;
  char startMarker = '<';
  char endMarker = '>';
  char rc;
  while (Serial1.available() > 0 && newData == false) {
    rc = Serial1.read();
    if (recvInProgress == true) {
      if (rc != endMarker) {
        Serial.println("no endMarker");
        receivedBin[ndx] = rc;//keep populating the array until the end marker is found
        ndx++;

        if (ndx >= numBin) {//if last segment in array is found
          ndx = numBin - 1;// move marker to position 5
        }
      }
      else {
        Serial.println("endMarker found");
        receivedBin[ndx] = '\0'; // if end marker is found then terminate
        recvInProgress = false;
        ndx = 0;//reset array position
        newData = true;
      }
    }
    else if (rc == startMarker) {
      recvInProgress = true;
    }
  }
rpm = atoi(receivedBin);//convert back to Int
Serial.println (rpm);
}

void showNewData() {
  if (newData == true) {//if array is polulated with good data
    Serial.println("This just in ...should report 12341 ");

    Serial.println(receivedBin[0]);
    Serial.println(receivedBin[1]);
    Serial.println(receivedBin[2]);
    Serial.println(receivedBin[3]);
    Serial.println(receivedBin[4]);
    Serial.println(receivedBin[5]);

    newData = false;
  }
}

Hopefully I have understood everything correctly. The comms seems stable.

anon73444976 · July 25, 2020, 12:07pm

Why are you using strcat?
Why not just send the partial strings sequentially?

int rpm = 12341;//test variable to send
char endmarker = '>';
char startmarker = '<';
void setup() 
{
  Serial.begin(9600);
}

void loop() 
{
  Serial.print (startmarker);
  Serial.print (rpm);
  Serial.print (endmarker);
}

Robin2 · July 25, 2020, 1:41pm

Why not use my recvWithStartEndMarkers() without any modifications? After all, you are just sending text.

The lines

rpm = atoi(receivedBin);//convert back to Int
Serial.println (rpm);

should not be in recvWithStartEndMarkers() as they are for parsing the data rather than for receiving it. Restricting the work of a function to a single task makes program development and debugging much easier.

Won't this do the sending job in a simpler way?

void loop() {
    itoa (rpm, sentchar, 10);
    Serial.print(startMarker);
    Serial.print(sentChar);
    Serial.print(endMarker);
    delay(1000);
}

...R

kpg · July 25, 2020, 2:21pm

Appreciate the feedback.

Most of my code was about trying to learn some new functions and how to use them all be it I could have done it simpler as you say

Also because my Int RPM can be between 0 and 12000 I thought I needed to write it to a buffer?

Was I wrong?

anon73444976 · July 25, 2020, 2:25pm

kpg:
Also because my Int RPM can be between 0 and 12000 I thought I needed to write it to a buffer?

Was I wrong?

All you're doing is replicating what "Serial.print(rpm);" can already do.

kpg · July 25, 2020, 2:39pm

Of course you are right, Brain fart when I posted that.

Still, learnt some more stuff all be it probably not best deployed.....

Can you please explain what you mean by....

rpm = atoi(receivedBin);//convert back to Int
Serial.println (rpm);

should not be in recvWithStartEndMarkers() as they are for parsing the data rather than for receiving it. Restricting the work of a function to a single task makes program development and debugging much easier.

Thanks again.

sterretje · July 25, 2020, 3:37pm

Can you please explain what you mean by...

You have reduced Robin's code to something that now can only handle integers.

See, the idea is that the receive function just does that, receive. Printing is not part of receive, neither is conversion.

Think in layers

[size=small]
+---------------------+
| use processed data  |
+---------------------+
| process recv'd data |
+---------------------+
| receive data        |
+---------------------+
[/size]

kpg · July 29, 2020, 10:39am

On the advice I rebuilt the receive code to just be Robin2's version with start and end markers and receive characters and also now the send code is;

int rpm;
char startmarker = ('<');//error corrected was "<" in original
char endmarker = ('>');
char sentChar[5];

void setup() {
  Serial.begin(9600);
  Serial.println("<Arduino is ready>");
}

void loop() {
  rpm = 12341;
  
  itoa(rpm, sentChar, 10);
  Serial.print(startmarker);
  Serial.print(sentChar);
  Serial.print(endmarker);
}

I assume I need to send the null?

When monitoring the receive it looses characters.

What am I missing?

Robin2 · July 29, 2020, 10:57am

kpg:
I assume I need to send the null?

No, don't.

You can send data in a compatible format with code like this (or the equivalent in any other programming language)

Serial.print('<'); // start marker
Serial.print(value1);
Serial.print(','); // comma separator
Serial.print(value2);
Serial.println('>'); // end marker

...R

kpg · July 29, 2020, 11:34am

Robin

I just saw the error in my send. I was using "<" instead of '<'.

The send looks good on the serial monitor but the receive still isnt right and misses characters (I am not trying to convert back to Int yet on receive, it is just your code).

const byte numChars = 32;
char receivedChars[numChars];//receive char version
//byte receivedChars[numChars];//receive byte version


boolean newData = false;

void setup() {
    Serial.begin(9600);
     Serial1.begin(9600);
    Serial.println("<Arduino is ready>");
}

void loop() {
    recvWithStartEndMarkers();
    showNewData();
}

void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = '<';
    char endMarker = '>';
    char rc;//receive char version
  //  byte rc;//receive byte version
 
 // if (Serial.available() > 0) {
    while (Serial1.available() > 0 && newData == false) {
        rc = Serial1.read();
        Serial.println (rc);

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
        Serial.print("This just in ... ");
        Serial.println(receivedChars);// char version
       // Serial.println((char*)receivedChars);//byte version
        newData = false;
    }
}

Serial monitor is;

This just in ... 12341
This just in ... 12341
This just in ... 12341
This just in ... 12341
This just in ... 12347
This just in ... 12341
This just in ... 123412341
This just in ... 1341
This just in ... 141
This just in ... 1231
This just in ... 1234a
This just in ... 12341
This just in ... 12341<12341
This just in ... 12341
This just in ... 12341

sterretje · July 29, 2020, 12:21pm

Your sentChar variable in reply #7 needs to be one byte bigger; there needs to be space for a terminating nul-character.

kpg · July 29, 2020, 1:20pm

I have made sentChar 7 now.

Serial monitor is;

This just in ... 11
This just in ... 1234123412341
This just in ... 41
This just in ... 123<12341
This just in ... 123341
This just in ... 41
This just in ... 123<12341
This just in ... 41
This just in ... 11
This just in ... 11
This just in ... 12341
This just in ... 1234123412341
This just in ... 123412341
This just in ... 12
This just in ... 12
This just in ... 12341
This just in ... 12
This just in ... 12

Could it just be line noise?

I have very short wires (jumpers) and everything is powered from PC USB. Just a nano and mega connected (5v, 0v, Tx/Rx) with no other wires or code going on.

sterretje · July 29, 2020, 2:19pm

Put a short delay in the sender (at the end of loop()); I suspect that you're overflowing the input buffer.

kpg · July 29, 2020, 3:27pm

Thanks very much.

It just needed a 100ms delay. I would have never worked that out. Why does it occur? What does the delay do?

anon73444976 · July 29, 2020, 3:37pm

You've got both interfaces running at the same speed, and one of them is outputting more than the other is receiving; something's got to give...

sterretje · July 29, 2020, 4:28pm

You're continously pumping data from the transmitter (7 bytes); it's the only thing the transmitter has to do so is 'fast'.

The receiver receives this in a 64 byte buffer from where you read it using Serial.read(); however, the receiver has more to do like keeping track of the index, checking for startmarker and endmarker and other stuff in your code; that takes a little more time and as a result it can e.g. only process 6 bytes of the 7 and 1 is still in the buffer when the next bunch of 7 arrives (so you now have eight bytes). Eventually the 64 byte buffer fills up and when another character arrives, it's simply discarded.

The delay prevents that you continously pump data and gives the receiver time to catch up. At 9600 baud, one character takes rougly 1 millisecond to transmit, so 100 milliseconds delay is more than likely overkill (trial and error to find a reasonable value).

Under normal situations, the transmitter will only send something when it has to; e.g. after reading an analog input (this takes about 100 milliseconds) or when a button becomes pressed (this can take anywhere from N milliseconds to hours).

If you really need to send data as fast as possible, you will need to implement a protocol where the receiver tells the transmitter when it can send. E.g. after the receiver has received and processed the information, it can send a single byte back to the transmitter. The transmitter will wait for this byte before sending the next bunch of data. It involves a little more but this should give you the idea.

kpg · July 29, 2020, 4:37pm

Thanks all,

So for everyones benefit here is the send and receive code now which starts with an Int, converts to Char over serial then back to Int.

Send.

int rpm;
char startmarker = ('<');
char endmarker = ('>');
char sentChar[6];

void setup() {
  Serial.begin(9600);
  Serial.println("<Arduino is ready>");
}

void loop() {
  rpm = 12341;
  //Serial.println(rpm);
  itoa(rpm, sentChar, 10);
  Serial.print(startmarker);
  Serial.print(sentChar);
  Serial.print(endmarker);
  delay (100);
}

Receive.

const byte numChars = 32;
char receivedChars[numChars];//receive char version
//byte receivedChars[numChars];//receive byte version
int rpm_val;


boolean newData = false;

void setup() {
  Serial.begin(9600);
  Serial1.begin(9600);
  Serial.println("<Arduino is ready>");
}

void loop() {
  recvWithStartEndMarkers();
  showNewData();
  rpm_val = atoi(receivedChars);
  Serial.print("Converted back to RPM ");
  Serial.println(rpm_val);
}

void recvWithStartEndMarkers() {
  static boolean recvInProgress = false;
  static byte ndx = 0;
  char startMarker = '<';
  char endMarker = '>';
  char rc;//receive char version
  //  byte rc;//receive byte version

  // if (Serial.available() > 0) {
  while (Serial1.available() > 0 && newData == false) {
    rc = Serial1.read();
    //Serial.println (rc);

    if (recvInProgress == true) {
      if (rc != endMarker) {
        receivedChars[ndx] = rc;
        ndx++;
        if (ndx >= numChars) {
          ndx = numChars - 1;
        }
      }
      else {
        receivedChars[ndx] = '\0'; // terminate the string
        recvInProgress = false;
        ndx = 0;
        newData = true;
      }
    }

    else if (rc == startMarker) {
      recvInProgress = true;
    }
  }
}

void showNewData() {
  if (newData == true) {
    Serial.print("This just in ... ");
    Serial.println(receivedChars);// char version
    // Serial.println((char*)receivedChars);//byte version
    newData = false;
  }
}

Output is;

This just in ... 12341
Converted back to RPM 12341
This just in ... 12341
Converted back to RPM 12341
This just in ... 12341
Converted back to RPM 12341
This just in ... 12341
Converted back to RPM 12341
This just in ... 12341
Converted back to RPM 12341
This just in ... 12341

cattledog · July 29, 2020, 5:07pm

void loop() {
  rpm = 12341;
  //Serial.println(rpm);
  itoa(rpm, sentChar, 10);
  Serial.print(startmarker);
  Serial.print(sentChar);
  Serial.print(endmarker);
  delay (100);
}

Why are you using itoa?
In the Arduino environment with the Print class, it is unnecessary.

Serial.print(rpm) without the conversion using itoa will send the characters of the integer.

Your loop code could be simplified to

void loop() {
  rpm = 12341;
  Serial.print(startmarker);
  Serial.print(rpm);
  Serial.print(endmarker);
  delay (100);
}

This was given in reply #1 and #8.

kpg · July 30, 2020, 10:25am

Obviously missed something somewhere. I thought I needed to place rpm in a character array (with a space for a null) and in doing so needed ITOA? It was suggested in one of Robin2's, but I probably mis understood the concept.

But as you say the print class sends the characters of the integer anyway. What about the terminator?

I tested it by just running Serial.print (rpm); and I seem to get more received errors.

My next problem is that the Mega code has so much going on that the reporting of RPM is very slow and by the time it gets round to noticing a change at the receive end display its probably 20 seconds before the receive buffer empties after stopping the send from the nano.

Robin2 · July 30, 2020, 11:27am

kpg:
My next problem is that the Mega code has so much going on that the reporting of RPM is very slow and by the time it gets round to noticing a change at the receive end display its probably 20 seconds behind the send end from the nano.

Sounds like the Mega code needs a big re-write

...R

Topic		Replies	Views
joinging Uno and Mega with serial General Guidance	42	2041	May 5, 2021
SOLVED! highByte, lowByte and serial send/receive Programming	41	7701	May 6, 2021
Speeding up my Slow Serial!? Programming	40	2074	May 6, 2021
Serial Input Basics Programming	81	580932	May 5, 2021
Sending more than one integer between 2 Arduinos via the same serial channel Programming	34	2637	May 6, 2021

Serial send and receive - Hopefully useful to someone

Related topics