issues parsing Serial.read() data

Using Arduino Programming Questions
1

I'm using 2 Unos each with an XBee radio to communicate in one direction. However, I am not happy with the results I am getting. It is working to the extent that I am receiving data from the transmitter to the receiver, but the data is not always how I would expect it.

I have read lots of posts here and other places Google has led me, and I have borrowed very heavily from those posts attempting to get this to work.

Here's the code for my transmitter:

String outgoingString = "AA0123 BB0456 CC0789 Z\n";

void setup() {
  Serial.begin(9600);
}

void loop() {
  Serial.print(outgoingString);
}

In the Serial Monitor, this results in the expected:

...
AA0123 BB0456 CC0789 Z
AA0123 BB0456 CC0789 Z
AA0123 BB0456 CC0789 Z
AA0123 BB0456 CC0789 Z
AA0123 BB0456 CC0789 Z
AA0123 BB0456 CC0789 Z
AA0123 BB0456 CC0789 Z
...

Here's the code for my receiver:

char incomingByte;
String incomingString = "";

void setup() {
  Serial.begin(9600);
}

void loop() {
  if (Serial.available() > 0) {
    incomingByte = Serial.read();
    while (incomingByte != '\n') {
      incomingString += incomingByte;
      incomingByte = Serial.read();
    }
    Serial.print(incomingString.length());
    Serial.print("\t"); Serial.println(incomingString);
    incomingString = "";
  }
}

The output of this code isn't exactly what I would expect:

22	AA0123 BB0456 CC0789 Z
19	AA6 CC0CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
19	AA6 CC0CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
19	AA237	CÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿCÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ7ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿZÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
26	AA0123 BB0456 CC0 CC0789 Z
19	AA0123 BB04560789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
25	AA0123 23 BB0456 CC0789 Z
21	AA0123 BB0456 C0789 Z
18	AA0123 BB0456 C9 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
486	AA0123 BB0456ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿCÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿCÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ7ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ8ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ9ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿZÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
10	AA0C0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
10	AA012389 Z
18	AA0123456 CC0789 Z
18	AA0123 BB0CC0789 Z
18	AA0123 BB0456 89 Z
19	AA01230456 CC0789 Z
24	AA0123 BBBB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
286	AA0123 BB0456 CC078ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ9ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿZÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
32	AA0123 BB0456 23 BB0456 CC0789 Z
22	AA0123 BB0456 CC0789 Z
29	AA0123 AA0123 BB0456 CC0789 Z
18	AA0123 BB0CC0789 Z
18	AA0123 BB0456 89 Z
22	AA0123 BB0456 CC0789 Z
23	AA0123 BB0456  CC0789 Z
22	AA0123 BB0456 CC0789 Z
...

As you can see, I'm getting random data from the Serial buffer. Eventually, the bad data continues to fill incomingString to a point where the Uno locks up. I'm assuming that buffers are overflowing at that point.

If I added coded that validated the data, I could just ignore the bad data and empty the value of incomingString to hopefully avoid buffer overflows. Eventually, I will have the incoming data constantly changing with values read from various input methods on the transmitter. I already have code to parse the values into useable variables, but right now, I'm trying to keep it as simple as possible to improve the quality of the read data.

If I'm just "doing it wrong, bro", then fine. Any suggestions on how to do it correctly or improve what is there would be very much appreciated.

2 
void setup() {
  Serial.begin(9600);
}

void loop() {
  if (Serial.available() > 0) {
    incomingByte = Serial.read();
    while (incomingByte != '\n') {
      incomingString += incomingByte;
      incomingByte = Serial.read();
    }
    Serial.print(incomingString.length());
    Serial.print("\t"); Serial.println(incomingString);
    incomingString = "";
  }
}

In your loop you see if there is at least one byte available and then keep reading until you get a '/n'. You may be reading data faster than it is arriving. When you call Serial.read() and there's nothing there is returns -1. That's the funny Y looking character you see long strings of there.

You can do one of several things here. You can sit and block until the next byte arrives with a while(Serial.avalable() == 0); or you could just let loop turn over and over checking to see if there's a character available and then reading it in and IF that character happens to be a '/n' then you know you got the whole thing and you go and do the other thing and print them. That means ditching the while loop.

3

Delta_G,

Thank you very much!! That totally makes sense. I will give that a shot.

4

Your sender is spewing data faster than the receiver can handle it. You are leaving the receiver NO time to process the data it is reading.

5

PaulS:
Your sender is spewing data faster than the receiver can handle it. You are leaving the receiver NO time to process the data it is reading.

I was just returning to post that the suggestion that Delta_G made did improve things. My original code had about a 5:1 corrupt data ratio, but the modified code got it down to about 3:1 bad data.

Adding simple delay(50) to the transmitter now results in 100% accurate data on the receiver.

Thank you very much PaulS!!!!!!

6

With regard to your solution of adding a delay in your transmitter....

If your receiving code was written differently it would have plenty of time to recieve and process all the incoming data with no need for any restriction at the transmitter. At 9600 baud you only get about 960 characters per second or about 1 every millisecond. An Arduino can do a lot in that intervening millisecond.

...R

7

What about something like the following:

char incomingMessage[30];
int messageLength;

void setup() {
  Serial.begin(9600);
}

void loop() {
  if (Serial.available() > 0) {
    messageLength = Serial.readBytesUntil('\n', incomingMessage, 29);
  }
  Serial.println(incomingMessage);
}

By getting rid of the String object, the compiled code size drops from 3862 bytes to 2164.