Serial Input Basics

EDIT 05 May 2016 - please see the updated and shorter version of this Tutorial

Introduction

Newcomers often seem to have difficulty with the process of receiving Serial data on the Arduino - especially when they need to receive more than a single character. The fact that there are 18 different functions listed on the Serial reference page probably does not help

You could write a small book and still not cover every possible situation for data reception. Rather than write pages and pages that few would read I thought it would be more useful to present a few examples which will probably cover all of a newcomer's needs. And when you understand these examples you should be able to figure out solutions for other strange cases.

Almost all serial input data can be covered by three simple situations

A - when only a single character is required
B - when only simple manual input from the Serial Monitor is required
C - other

Please note that this text continues into the next Post

Serial data is slow by Arduino standards

When anything sends serial data to the Arduino it arrives into the Arduino input buffer at a speed set by the baud rate. At 9600 baud about 960 characters arrive per second which means there is a gap of just over 1 millisecond between characters. The Arduino can do a lot in 1 millisecond so the code that follows is designed not to waste time waiting when there is nothing in the input buffer even if all of the data has not yet arrived. Even at 115200 baud there is still 86 microseconds or 1376 Arduino instructions between characters.

Receiving single characters

In very many cases all that is needed is to send a single character to the Arduino. Between the upper and lower case letters and the numeric characters there are 62 options.

Code to receive a single character is as simple as this

char receivedChar;
boolean newData = false;

void setup() {
 Serial.begin(9600);
 Serial.println("<Arduino is ready>");
}

void loop() {
 recvOneChar();
 showNewData();
}

void recvOneChar() {
 if (Serial.available() > 0) {
 receivedChar = Serial.read();
 newData = true;
 }
}

void showNewData() {
 if (newData == true) {
 Serial.print("This just in ... ");
 Serial.println(receivedChar);
 newData = false;
 }
}

Organizing code into functions

Even though this example is short and simple I have deliberately put the code to receive the character into a separate function called recvOneChar() as that makes it simple to add it into any other program. I also have the code for showing the character in the function showNewData() because you can change that to do whatever you want without upsetting the rest of the code.

Receiving several characters from the Serial Monitor

EDIT 05 Feb 2015
Updated to work better - see Replies 22-25 for explanation
If you need to receive more than a single character from the Serial Monitor (perhaps you want to input people's names) you will need some method of letting the Arduino know when it has received the full message. The simplest way to do this is to set the line-ending to newline.

This is done with the box at the bottom of the Serial Monitor window. You can choose between "No line ending", "Newline", "Carriage return" and "Both NL and CR". When you select the "Newline" option a new-line character ('\n') is added at the end of everything you send.

const byte numChars = 32;
char receivedChars[numChars]; // an array to store the received data

boolean newData = false;

void setup() {
 Serial.begin(9600);
 Serial.println("<Arduino is ready>");
}

void loop() {
 recvWithEndMarker();
 showNewData();
}

void recvWithEndMarker() {
 static byte ndx = 0;
 char endMarker = '\n';
 char rc;
 
 // if (Serial.available() > 0) {
           while (Serial.available() > 0 && newData == false) {
 rc = Serial.read();

 if (rc != endMarker) {
 receivedChars[ndx] = rc;
 ndx++;
 if (ndx >= numChars) {
 ndx = numChars - 1;
 }
 }
 else {
 receivedChars[ndx] = '\0'; // terminate the string
 ndx = 0;
 newData = true;
 }
 }
}

void showNewData() {
 if (newData == true) {
 Serial.print("This just in ... ");
 Serial.println(receivedChars);
 newData = false;
 }
}

This version of the program reads all the characters into an array until it detects the Newline character as an end marker.

It is important to notice that each time the function recvWithEndMarker() is called it reads at most one character from the input buffer. This assumes that loop() repeats very frequently - perhaps hundreds or thousands of times per second. If there is nothing in the buffer recvWithEndMarker() does not waste time waiting.

continued in next Post
...R

2 Likes

...continued from previous post

A more complete system

EDIT 05 Feb 2015
Updated to work better - see Replies 22-25 for explanation
EDIT 23 Sep 2015
In Reply #69 have included a revised version that works with bytes rather than chars. For most uses the char version in this Reply will be sufficient
The simple system in the previous section will work well with a sympathetic human who does not try to mess it up. But if the computer or person sending the data cannot know when the Arduino is ready to receive there is a real risk that the Arduino will not know where the data starts.

If you would like to explore this, change the end marker in the previous program from '\n' to '>'. This is so that you can include the end marker in your text for illustration purposes. (You can't manually enter a Newline character in the text you are sending from the Serial Monitor). And put the line ending back to "No line ending"

Now, with the revised code send "qwert>" and you will see that it behaves exactly the same as when you were using Newline as the end marker.

But if you try this "asdfg>zxcvb" you will only see the first part "asdfg". And then if you send "qwert>" you will see "zxcvbqwert" because the Arduino has become confused and cannot know that it should have ignored "zxcvb".

The answer to this problem is to include a start marker as well as an end marker.

const byte numChars = 32;
char receivedChars[numChars];

boolean newData = false;

void setup() {
    Serial.begin(9600);
    Serial.println("<Arduino is ready>");
}

void loop() {
    recvWithStartEndMarkers();
    showNewData();
}

void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = '<';
    char endMarker = '>';
    char rc;
 
 // if (Serial.available() > 0) {
    while (Serial.available() > 0 && newData == false) {
        rc = Serial.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
        Serial.print("This just in ... ");
        Serial.println(receivedChars);
        newData = false;
    }
}

Edited 24 Oct 2015 to correct the indentation

To see how it works try sending "qwertyzxcvb" and you will see that it ignores everything except "asdfg".

In this program you will notice that there is a new variable called recvInProgress. This is necessary because a distinction needs to be made between unwanted characters that arrive before the start marker and the valid characters that arrive after the start marker.

This version of the program is very similar to the Arduino code in this demo.

Binary data

So far we have been receiving character data - for example the number 137 is represented by the characters '1', '3' and '7'. It is also possible to send that value as binary data in a single byte.

If you need to receive binary data rather than ascii characters just change the line
char receivedChars[numChars];
to
byte receivedChars[numChars];

and the line
char rc;
to
byte rc;

and if you want to continue using the showNewData() function you will also need to change
Serial.println(receivedChars);
to
Serial.println((char*)receivedChars);

(Of course it would also make more sense to change the names of the variables to reflect the fact that they contain bytes rather than chars - but I will leave that as a homework exercise).

Parsing the received data

Edit 02 May 2015 ... In Reply #43 I have added a simpler example for cases where only a single number is input.

So far there has been no attempt to parse the received data, but that is easy to do once all of the data has been received. The code in this short demo assumes that the data "This is a test, 1234, 45.3" has been received and placed in the array receivedChars. It uses the function strtok() to spilt the data at the commas and the functions atoi() and atof() to convert the ascii data into an integer and a float respectively.

// simple parse demo
char receivedChars[] = "This is a test, 1234, 45.3" ;

char messageFromPC[32] = {0};
int integerFromPC = 0;
float floatFromPC = 0.0;

char recvChar;
char endMarker = '>';
boolean newData = false;


void setup() {
 Serial.begin(9600);
 Serial.println("<Arduino is ready>");
 
 parseData();
 showParsedData();
}


void loop() {

}

 
void parseData() {

    // split the data into its parts
    
  char * strtokIndx; // this is used by strtok() as an index
  
  strtokIndx = strtok(receivedChars,",");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
  
  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  integerFromPC = atoi(strtokIndx);     // convert this part to an integer
  
  strtokIndx = strtok(NULL, ","); 
  floatFromPC = atof(strtokIndx);     // convert this part to a float

}


void showParsedData() {
 Serial.print("Message ");
 Serial.println(messageFromPC);
 Serial.print("Integer ");
 Serial.println(integerFromPC);
 Serial.print("Float ");
 Serial.println(floatFromPC);
}

This code is as close as possible to the equivalent Arduino code in this demo
Edit 06 Apr 2015 ... this code only works once because strtok() changes the array it is parsing. See Reply #39 for more

Edit 01 Nov 2015 ... See Reply #72 for an example that combines receiving and parsing

If the data must be able to including the markers

The demo here shows how to extend this concept so that the data stream can include bytes that have the same values as the start or end markers.

Things that are not used

You will notice that the examples here do not use any of the Arduino functions
Serial.parseInt()
Serial.parseFloat()
Serial.readBytes()
Serial.readBytesUntil()

All of these are blocking functions that prevent the Arduino from doing something else until they are satisfied, or until the timeout expires. The examples here do exactly the same job without blocking.

serialEvent() Added 15 Jan 2016

I don't recommend using this function - I prefer to deal with the Serial data when it suits me. It behaves just as if you had this code as the last thing in loop()

if (Serial.available > 0) {
  mySerialEvent();
}

Clearing the input buffer

It is probably worth mentioning that the poorly named Serial.flush() function does not empty the input buffer. Its purpose is to block the Arduino until all outgoing the data has been sent.

If you need to ensure the Serial input buffer is empty you can do so like this
while (Serial.available() > 0) {
Serial.read();
}

END

...R

2 Likes

As usual, good job, Robin. One question: When you discuss getting characters and looking for an end character, why not use the Serial object's readBytesUntil() method?

Edit: Never mind! I must have read over your blocking comment!

Excellent work and I appreciate the effort. I picked quite a bit of insight into how the Arduino OS works.

All of these are blocking functions that prevent the Arduino from doing something else until they are satisfied, or until the timeout expires.

The readBytes() function may, or may not, block. It depends on whether or not it is called with a number of bytes to read that is less than, equal to, or more than what Serial.available() reports.

PaulS:
The readBytes() function may, or may not, block. It depends on whether or not it is called with a number of bytes to read that is less than, equal to, or more than what Serial.available() reports.

Thanks Paul, I appreciate your taking the trouble to read it.

I agree with your comment. My objective was to present general purpose advice without confusing newcomers with the peculiarities of each possible function.

Robin-- Thank you so much for putting this tutorial together. Your efforts are very much appreciated.

There is a small error in the simple parse demo.

There is no third comma in the parsed string and strtok is actually stopping at the terminating NULL character when it parses that final float.

Below, I have modified parseData() to demonstrate this. The strtok function clearly works with either the specified NULL or the default behaviour when looking for a nonexistant comma.

void parseData() {

    // split the data into its parts
    
  char * strtokIndx; // this is used by strtok() as an index
  
  strtokIndx = strtok(receivedChars,",");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
  
  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  integerFromPC = atoi(strtokIndx);     // convert this part to an integer
  
  //strtokIndx = strtok(NULL, ",");
  strtokIndx = strtok(NULL, NULL); 
  floatFromPC = atof(strtokIndx);     // convert this part to a float

}

Thank you very much for taking the time to study the demo.

I have tested my code again. It seems to work properly and exactly the same with my original code and with your suggested change.

I referred to this webpage about the strtok() function when I was writing my examples and it does not suggest the need for a different form for the final test.

Unless you (or someone else) can show a definite problem I do not propose to change my code as it is easier for newcomers if they do not have to bother with treating the last test as a special case.

...R

Using strtok(NULL, ",") and strtok(NULL, NULL) will give exactly the same result. There is nothing at all wrong with letting strtok run off the end of the string, if that satisfies the parsing requirements, which, in this case, it does. I would leave it as it.

Regarrds,
Ray L.

@Robin,
You used recvWithEndMarker() to receive more then one character at a time.But if i declare receivedChar as int and then use receivedChar = Serial.parseInt() i am able to use both the single character and multiple character.Is there some reason why you wrote your code in that way?
Thanks
Bikash

sorry i missed one of your comments.Serial.parseInt is a blocking function and but recvWithEndMarker() provides total control over how the data is to be sent.Very informative posts,Thanks for your effort.
Regards,
Bikash

for Robin2 , how I can receive and store the wave file into the micro sd ?

NRU28:
for Robin2 , how I can receive and store the wave file into the micro sd ?

Just the same way as any other data - it's just that there will be a lot of bytes. You may want to receive it in (say) 32 byte chunks. When the Arduino has stored the 32 bytes it can ask the PC to send the next 32 bytes.

I have no experience of writing to an SD card.
have you considered writing the SD card on your PC.

...R

When you use while (Serial.available() > 0) {Serial.read();} to clear the input buffer, do you put everything in the while loop, or put the while loop after everything else? Also, when I try to parse binary data, it gives me an error here strtokIndx = strtok(receivedBytes,","); saying that it's an invalid conversion from byte to char. What should I do?

strtok returns a pointer to a character array, not an index.

char *s = strtok(receivedBytes,",");
Serial.println(s);

Regards,
Ray L.

Qiking:

strtokIndx = strtok(receivedBytes,",");

saying that it's an invalid conversion from byte to char. What should I do?

In my examples the data is put into a char array - receivedChars[]. I'm guessing that is what strtok() is intended for. I don't know whether a cast would be suitable.

I don't understand your question about clearing the input buffer - what exactly do you not understand.

...R

Robin2:
In my examples the data is put into a char array - receivedChars[]. I'm guessing that is what strtok() is intended for. I don't know whether a cast would be suitable.

I don't understand your question about clearing the input buffer - what exactly do you not understand.

...R

I don't understand where to put the while loop. Do I put it after all my other functions, before all my other functions, or put all my functions inside the while loop?

Qiking:
I don't understand where to put the while loop. Do I put it after all my other functions, before all my other functions, or put all my functions inside the while loop?

Put it at the place in your code where you want to empty the input buffer. You could put the code into a function and just call the function from the appropriate location,

void clearInputBuffer() {
   while (Serial.available() > 0) {
        Serial.read();
   }
}

...R

Hi,
first of all this was very usefull for me. Now I have one question. Is it possible to use markers to separate string into variables? For example: if I send "<1><2><3>" over serial monitor, how could I store that in to variables like x=1, y=2, z=3? I need that to controll the pins on my arduino so I can send instruction using PLC over the serialc conection. My plan is tu create swich statement wich will use that data to go trough. I want to make it in the way that when the instruction is over (after i send <1><2><3>, it should store it like x=1, y=2, z=3, go trough few swich statements that use variable x,y and z and than stop and wait for next instruction) I hope I made my question clear anough and I'm really looking foward to your reply.
Regards!

mmunic194:
For example: if I send "<1><2><3>" over serial monitor, how could I store that in to variables like x=1, y=2, z=3?

It would be possible, but not so easy because the code will treat each value as the start of a new set of data.
But the demo code does the same thing using <1,2,3> so why not use that system?

Perhaps I don't understand what you want to do?

...R