Receiving strings from Serial not coming synchonously

protomor · February 11, 2017, 1:57am

I have an arduino uno hooked up as a slave to an arduino mega. The uno’s job is to collect data from sensors and send it in csv string values. The format is something like “A01,706”. Here is the tutorial I used as a basis.

Literally, this is the code for the slave arduino currently

 void loop() {
    char buf[32];
    val = analogRead(potPin);    
    sprintf(buf, "<A01,%d>", val);
    Serial.println(buf);   
  }

The master logs from an OBD2 system which can be slow so the main loop has an idle subset of code for 20ms

	while (millis() - start < OBD_MIN_INTERVAL) {
		obd.dataIdleLoop();
	}

The idle loop calls code to log an accelerometer and GPS unit. I added in the following code (from the tutorial mentioned above) to read from the slave

recvWithStartEndMarkers();
    if (newData == true) {
        lcd.setXY(5, 12);
        lcd.print(receivedChars);
        //Serial.println(receivedChars);
        newData = false;
    }

void recvWithStartEndMarkers() {
static boolean recvInProgress = false;
static byte ndx = 0;
char startMarker = '<';
char endMarker = '>';
char rc;

while (Serial2.available() > 0 && newData == false) {
    rc = Serial2.read();

    if (recvInProgress == true) {
        if (rc != endMarker) {
            receivedChars[ndx] = rc;
            ndx++;
            if (ndx >= numChars) {
                ndx = numChars - 1;
            }
        }
        else {
            receivedChars[ndx] = '\0'; // terminate the string
            recvInProgress = false;
            ndx = 0;
            newData = true;
        }
    }

    else if (rc == startMarker && ndx == 0) {
        recvInProgress = true;
    }
    else if (rc == startMarker && ndx > 0)
    {
        return;
    }
}
}

If I put this code in an isolated loop outside the rest of my code, I receive the data from the slave properly. But put in my code posted above I get something like “A01,93A0<93A3”

I’m getting that something is missing characters and it’s just reading things out of order, but I’m not sure why. Does anyone have insight that could help me?

Paul_KD7HB · February 11, 2017, 3:21am

Perhaps the problem is in the code you aren't showing us.

Paul

Robin2 · February 11, 2017, 11:09am

Receiving strings from Serial not coming synchonously

That's not very surprising as the receiving part of the Arduino is a USART and the A stands for Asynchronous.

As @Paul_KD7HB implied, post your complete program.

...R

protomor · February 11, 2017, 12:29pm

The code is pretty badly organized and things are very messy. I’ve attached it below. It’s based on the freematics.com obd2 logger that I’ve added extra functionality. I’m trying to get this second arduino to externally grab data and send it to be logged with the main arduino mega.

GinzoLogger.ino (25.4 KB)

Robin2 · February 11, 2017, 2:41pm

There is a lot of code and, in particular, your loop() function is very long which makes it hard to figure out.

You have not told us where, in that program, this gets produced

But put in my code posted above I get something like “A01,93A0<93A3”

It looks to me like the recvWithStartEndMarkers() function missed a closing > and that could just as easily be due to an error in the program that is sending the data.

If the problem is within the posted program it is likely due to the fact that the recvWith… function is not called often enough to empty the Serial Input buffer and data has been lost by the USART.

Edit to add …
Using a much higher baud rate might help (assuming an individual message is less than 64 bytes) because a single call to recvWith… would be likely to find more data waiting for it. 500,000 baud should work fine if you are using HardwareSerial on the Uno. If using SoftwareSerial on the Uno I believe 38400 baud should work.

…R

protomor · February 11, 2017, 3:01pm

I know. I really need to clean stuff up >_<

Where the bad data is produced? you mean the slave or the bad string is displayed? The first code snippet in my original post is literally the code in the slave arduino. The only other code is the 9600 baud rate in the setup.

The bad data is displayed via “lcd.print(receivedChars);” which is the print to a UTFT display.

I could definitely see the possibility of the receive function not being called enough. But wouldn’t a high baud rate cause more issues not fewer? I think I’m missing something very basic about this concept. Wouldn’t the higher rate cause the buffer to fill faster? My code wouldn’t get it to it any faster.

Paul_KD7HB · February 11, 2017, 3:19pm

protomor:
I know. I really need to clean stuff up >_<

Where the bad data is produced? you mean the slave or the bad string is displayed? The first code snippet in my original post is literally the code in the slave arduino. The only other code is the 9600 baud rate in the setup.

The bad data is displayed via “lcd.print(receivedChars);” which is the print to a UTFT display.

I could definitely see the possibility of the receive function not being called enough. But wouldn’t a high baud rate cause more issues not fewer? I think I’m missing something very basic about this concept. Wouldn’t the higher rate cause the buffer to fill faster? My code wouldn’t get it to it any faster.

The “baud rate” is the bit rate, not the character rate. When using async, the character rate is controlled by the sender. The sender may use all the available time to send characters, or may use only part of the available time.

Paul

protomor · February 11, 2017, 3:23pm

Although, now that I look at it, I'm just tossing the whole character array into the print function with no thought of the index for the terminating character. But I'm not entirely sure that would be an issue.

Robin2 · February 11, 2017, 3:59pm

protomor:
The first code snippet in my original post is literally the code in the slave arduino.

That code will send so much data that it will completely overwhelm the Serial system - it will probably block its own output as well as the input on the Mega. At least for testing add delay(500); into loop().

And make that change before you start experimenting with any other part of the problem.

Also I am not familiar with sprintf(). Will it put a terminating ‘\0’ after the ‘>’ ?
I just make output with a series of prints like

Serial.print("<A01,");
Serial.print(val);
Serial.println('>');

…R

protomor · February 14, 2017, 3:48pm

I think it makes more sense now. The serial buffer gets filled independent of any of my code. So if too much gets pushed to it, it’ll overflow and start spitting out junk. I raised the baud rate to 38400 and put in a 50ms delay between loops and things seem to be mostly resolved.

system · February 14, 2017, 3:52pm

it'll overflow and start spitting out junk.

The incoming buffer will fill, causing incoming data to be discarded. That does not mean that anything is "spitting out junk". It means that the data read from the buffer will be incomplete, which could look like junk.

I raised the baud rate to 38400

That means that data will come in faster.

and put in a 50ms delay between loops

That means that it will take longer to get around to reading data.

It appears that you have made the situation worse, not better.

protomor · February 14, 2017, 4:07pm

But then… why does it work now?

Robin2 · February 14, 2017, 4:12pm

protomor:
But then... why does it work now?

You have not posted your latest versions of your programs.

Have you put the 50msec delay in your sending program or in your receiving program?

...R

protomor · February 14, 2017, 4:26pm

Here are the only changes I made to the master.

void processExternalIO()
{
	logger.dataTime = millis();

	recvWithStartEndMarkers();

	if (newData == true) {
		lcd.setXY(5, 12);
		lcd.print(receivedChars);
		logger.logData(receivedChars);
		newData = false;
	}
}


void recvWithStartEndMarkers() {
	static boolean recvInProgress = false;
	static byte ndx = 0;

	char startMarker = '<';
	char endMarker = '>';
	char rc;

	while (Serial2.available() > 0 && newData == false) {
		rc = Serial2.read();

		if (recvInProgress == true) {
			if (rc != endMarker) {
				receivedChars[ndx] = rc;
				ndx++;
				if (ndx >= numChars) {
					ndx = numChars - 1;
				}
			}
			else {
				receivedChars[ndx] = '\0'; // terminate the string
				recvInProgress = false;
				ndx = 0;
				newData = true;
			}
		}
		else if (rc == startMarker && ndx == 0) {
			recvInProgress = true;
		}
	}
	return;
}

And the slave code is

const int buttonPin = 2;     // the number of the pushbutton pin
const int ledPin =  13;      // the number of the LED pin
const int potPin = 2;    // select the input pin for the potentiometer

// variables will change:
int buttonState = 0;         // variable for reading the pushbutton status

void setup() {
  //Serial.begin(38400);
  Serial.begin(9600);
  pinMode(ledPin, OUTPUT);
}

int val = 0;       // variable to store the value coming from the sensor

void loop() {
char buf[32];
  val = analogRead(potPin);    // read the value from the sensor
    //sprintf(buf, "<A01,%d>", val);
    //Serial.println(buf);
Serial.print("<A01,");
Serial.print(val);
Serial.println('>');

Serial.println("<B03,100>");
  
delay(50);
}

Interestingly enough, the log file looks perfect except for one blip on first connection. Then the rest are fine. (the first number is ms since last update)

1018 A01 805
33 B03 100
43 A01 805
54 B03 100
64 A01 804
74 B03 10<A01 805
93 A01 805
105 B03 100
115 A01 805
…

protomor · February 14, 2017, 6:14pm

For the record I did both baud rates and it really makes no difference.

Robin2 · February 14, 2017, 6:56pm

protomor:
Interestingly enough, the log file looks perfect except for one blip on first connection.

If that is a problem then you need to send a special startup message and ignore any junk that comes before it.

...R

protomor · February 14, 2017, 7:05pm

But it's not exactly on startup. It's a few lines in. I just added an exception for anything with a duplicate start character.

But if I'm still doing this wrong, I'd like to fix it and do it properly. I thought I had it figured out too.

system · February 14, 2017, 7:56pm

But if I'm still doing this wrong, I'd like to fix it and do it properly.

Well, I read back through the whole thread. I couldn't find anywhere where anyone said "Post some of your code". Everyone said post ALL of your code.

It looks to me like you are doing something early on that takes a bunch of time, and the serial buffer has overflowed. After that, you appear to be reading serial data fast enough.

protomor · February 14, 2017, 9:48pm

I did post all of my code. It's in post #3 for the master. All of my slave code is in post #13. Do you want all the library code?

I don't get why all of the code would be slow 2 seconds after load up and then be fine. If it was just slow loading at first, that would make sense and would easily be fixable. I'm literally piggy backing on the GPS logging logic since I know that works.

Robin2 · February 14, 2017, 10:04pm

protomor:
But it's not exactly on startup. It's a few lines in. I just added an exception for anything with a duplicate start character.

I don't know exactly what you mean by the first 2 sentences. Perhaps it depends on which unit starts first? I meant the start of communication rather than the start of the program.

But from the last sentence it sounds like you have a satisfactory solution.

...R