Serial Input Basics

Why not simply send clear, unambiguous commands like: “x=1,y=5,z=3”? Parse that using strtok(), and you can do whatever you want.

Regards,
Ray L.

@mmunic194,

Please continue the discussion about your project in your own Thread so as not to confuse readers of this one.

...R

EDIT 05 Feb 2015
Following the very helpful comment in Reply #23 I have updated the code and posted it in Reply #25

The examples in my original Post and in Reply #1 take a single character from the input buffer on each iteration of loop(). However in this Thread the problem arose that loop() did not repeat fast enough to keep up with the incoming characters, the buffer overflowed and data was lost.

This following version which has two extra lines in the function recvWithStartEndMarkers() deals with that problem by taking all the available bytes from the buffer on every iteration.

To illustrate this I have added a demonstration function doOtherStuff() which simply has a short 20 millisecond delay() as a crude way to simulate other calculations (or whatever) that might be taking place in a bigger program and preventing loop() from repeating fast enough.

I think the two extra lines I have added to recvWithStartEndMarkers() could be added to any of the examples.

const byte numChars = 90;
char receivedChars[numChars];

boolean newData = false;

void setup() {
	Serial.begin(9600);
	Serial.println("<Arduino is ready -->");
}

void loop() {
	recvWithStartEndMarkers();
	showNewData();
	doOtherStuff(); // <<=== NEW  --- This just simulates time spent elsewhere in a big program
}

void recvWithStartEndMarkers() {
	static boolean recvInProgress = false;
	static byte ndx = 0;
	char startMarker = '<';
	char endMarker = '>';
	char rc;
	
	if (Serial.available() > 0) {
		while (Serial.available() > 0 && newData == false) { // <<== NEW - get all bytes from buffer
			rc = Serial.read();

			if (recvInProgress == true) {
				if (rc != endMarker) {
					receivedChars[ndx] = rc;
					ndx++;
					if (ndx >= numChars) {
						ndx = numChars - 1;
					}
				}
				else {
					receivedChars[ndx] = '\0'; // terminate the string
					recvInProgress = false;
					ndx = 0;
					newData = true;
				}
			}

			else if (rc == startMarker) {
				recvInProgress = true;
			}
		} // <<=== NEW
	}
}

void showNewData() {
	if (newData == true) {
		Serial.print("This just in ... ");
		Serial.println(receivedChars);
		newData = false;
	}
}

void doOtherStuff() {
	delay(20); // <<===NEW - This just simulates time spent elsewhere in a big program
}

…R

That is a good addition!

One possible optimization:

Robin2:

 if (Serial.available() > 0) {

while (Serial.available() > 0 && newData == false) { // <<== NEW - get all bytes from buffer

I think the IF test is redundant here, and can be eliminated without affecting the operation, as you will be covered by the new while loop:

  • If there IS data available when you enter, the while loop will spin as necessary to receive all data
  • If there is NO data available when you enter, the while condition will not be satisfied and the loop will not execute

It’s a minor nit, but it saves a few bytes of precious PROGMEM without affecting operation and readability. If you feel the IF statement helps to show that nothing is done if no data is available, then put it in a comment?

ShapeShifter: One possible optimization:I think the IF test is redundant here,

Thanks, I have not tested but I think you are correct and I had overlooked that.

I will test it and probably change it - but I need to change the text as well so it may be tomorrow. It may actually make the explanation simpler.

...R

Having considered @ShapeShifter’s comment in Reply #23 I have decided that it would be most useful to users of the original examples to incorporate his change in them. That way people who only read the original post and Reply #1 will get the benefit.

The updated version of the code I posted in Reply #22 is as follows.

const byte numChars = 90;
char receivedChars[numChars];

boolean newData = false;

void setup() {
	Serial.begin(9600);
	Serial.println("<Arduino is ready -->");
}

void loop() {
	recvWithStartEndMarkers();
	showNewData();
	doOtherStuff(); // <<=== NEW  --- This just simulates time spent elsewhere in a big program
}

void recvWithStartEndMarkers() {
	static boolean recvInProgress = false;
	static byte ndx = 0;
	char startMarker = '<';
	char endMarker = '>';
	char rc;
	
	// if (Serial.available() > 0) {
	while (Serial.available() > 0 && newData == false) { // <<== NEW - get all bytes from buffer
		rc = Serial.read();

		if (recvInProgress == true) {
			if (rc != endMarker) {
				receivedChars[ndx] = rc;
				ndx++;
				if (ndx >= numChars) {
					ndx = numChars - 1;
				}
			}
			else {
				receivedChars[ndx] = '\0'; // terminate the string
				recvInProgress = false;
				ndx = 0;
				newData = true;
			}
		}

		else if (rc == startMarker) {
			recvInProgress = true;
		}
	}
}

void showNewData() {
	if (newData == true) {
		Serial.print("This just in ... ");
		Serial.println(receivedChars);
		newData = false;
	}
}

void doOtherStuff() {
	delay(20); // <<===NEW - This just simulates time spent elsewhere in a big program
}

I have edited the earlier posts so that, hopefully, the whole Thread makes sense.

A big thank-you to @ShapeShifter.

…R

Good work as usual. Thanks for giving your time to this.

<EDIT 15.02.2015> Previously I made a statement in this post about saving quite a bit of memory by using the void serialEvent() function instead of having the watch_serial_port() function inside the main loop.

Unfortunately that observation was all based on a typo in my program code :frowning:

Apart from saving maybe a few bytes by omitting some duplicate “if (Serial.available())” statements, no further savings can be made.

halfdome: void SerialEvent(){     while(Serial.available(){         do_something(); }

Interesting. But keep in mind the note on the SerialEvent() Reference page:

NB : Currently, serialEvent() is not compatible with the Esplora, Leonardo, or Micro

I'm sure this caveat also applies to the Yun, which has the same processor as a Leonardo.

Also, I think you have a typo in your code sample: Shouldn't serialEvent() start with a lower case 's'?

halfdome: In case one runs out of (flash) memory:

I have deliberately not used serialEvent() in order to keep my examples simple and tranparent.

I would be very interested to see the two code examples you used to determine the saving of 400 bytes.

...R

Unfortunately, my observation of saving memory was based on a typo - using "SerialEvent() " instead of "serialEvent()". Since this new function SerialEvent was never called, the compiler optimized it away...

halfdome: Unfortunately, my observation of saving memory was based on a typo - using "SerialEvent() " instead of "serialEvent()". Since this new function SerialEvent was never called, the compiler optimized it away...

Thank you very much for your honesty.

It could happen to anyone.

...R

Hello Robin,

I am struggling with a wireless serial transmission issue and I was wondering if you could please assist me with it? I have started a thread here.

Thanks very much for your time, it is much appreciated!

Dylan

I don't quite get recvWithStartEndMarkers(). What if a single "while Serial.available()" loop iteration finishes before the counterside has send a new character? as far as I can see, the index for the buffer is reset to 0 in each call of the function, so the buffer doesn't seem to be preservered over multiple loop() iterations?

ElCaron: as far as I can see, the index for the buffer is reset to 0 in each call of the function, so the buffer doesn't seem to be preservered over multiple loop() iterations?

ndx, as well as recvInProgress, are both marked static. This means that they are allocated once (static, as in fixed) and are not dynamically allocated on the function's stack frame. This means that they do not get created and destroyed on each call, and their values are preserved from call to call. An assignment made in a static variable's declaration only actually happens once, at system startup time, in order to give it an initial value.

Static variables are allocated at fixed addresses and behave just as if they were globals at the top of the file. But by putting them in the function, their visibility is limited to just that function. So it's a way to have the best of both worlds: the lifespan and persistence of global variables so that values are preserved from call to call, and the limited visibility of local variables so other functions can't mess with the values.

Ooooooooh, I overlooked that. Thanks!

Nice code! Though one problem with using

while (Serial.available()) {
  Serial.read();
}

to clear the input buffer (your last advice) is that it happens very fast, which means that we can easily be in a situation when the next character has not yet been received, so the loop will exit and fail to 'clear' the buffer (unless I'm doing something wrong, in which case please correct me :) ). One can easily test this by introducing

while (Serial.available()) {
  Serial.read();
  delayMicroseconds(10);
}

which even at 115200 baud rate will still be insufficient to keep up with the input; if you change 10 to 100, it starts to work.

So to alleviate that, we either have to introduce this (somewhat artificial) delay, or wait for the terminating character using something like

Serial.readStringUntil(endMarker);

which also has a nice feature of timing out after 1s (that can be changed with Serial.setTimeout()) if no ending marker is provided.

denvlad: to clear the input buffer (your last advice) is that it happens very fast, which means that we can easily be in a situation when the next character has not yet been received, so the loop will exit and fail to 'clear' the buffer

I think you are over-analyzing it and making it harder than it needs to be. The goal is to clear the input buffer: if the next character has not been received, it's not in the input buffer, so how can it be cleared?

This is not a function to ignore characters until some arbitrary end of message marker, or until there is no more data to be received. It is simply to clear out the characters that have already been received. The code was proposed to counter the lack of a flush() function for the serial input. A flush() function typically has two purposes: for an output stream, it makes sure all output has actually been written; for an input stream it discards any input data already available (and not any future input data.) The Serial class implements a flush() function, but only on the output stream. The proposed code is to provide a similar functionality for the input stream.

The major point of this approach to serial processing is to not block processing under any situations. By arbitrarily redefining the semantics of flush() to discard data until some end of message indication subverts that goal by introducing a blocking scenario. If you do want to introduce a function to skip until a certain marker, it would take a little more effort to do it in a non-blocking way. But that was not the intent of the subject code.

@ShapeShifter, thank you for your help here.

…R

While trying to help someone else I discovered that the Parse example in Reply #1 will only work once on the same data because the strtok() function changes the array it is working on. In this case it replaces the commas with \0.

The following slightly modified version uses a temporary copy of the array for parsing.

// simple parse demo with repeating output
char receivedChars[] = "This is a test, 1234, 45.3" ;
char tempChars[32];        // temporary array for use by strtok() function

char messageFromPC[32] = {0};
int integerFromPC = 0;
float floatFromPC = 0.0;

char recvChar;
char endMarker = '>';
boolean newData = false;
byte dLen = 0;


void setup() {
    Serial.begin(9600);
    Serial.println("<Arduino is ready>");
}


void loop() {

        // this is necessary because strtok() alters the array
        //   in this case replacing commas with \0
    strcpy(tempChars, receivedChars);
    parseData();
    showParsedData();
    delay(2000);
}

    
void parseData() {

    // split the data into its parts
    
  char * strtokIndx; // this is used by strtok() as an index

  strtokIndx = strtok(tempChars,",");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
  
  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  integerFromPC = atoi(strtokIndx);     // convert this part to an integer

  strtokIndx = strtok(NULL, ","); 
  floatFromPC = atof(strtokIndx);     // convert this part to a float

}


void showParsedData() {
    Serial.print("Message ");
    Serial.println(messageFromPC);
    Serial.print("Integer ");
    Serial.println(integerFromPC);
    Serial.print("Float ");
    Serial.println(floatFromPC);
}

…R