Go Down

Topic: Serial Input Basics (Read 114254 times) previous topic - next topic

Robin2

Dec 26, 2014, 12:45 pm Last Edit: May 05, 2016, 11:24 am by Robin2
EDIT 05 May 2016 - please see the updated and shorter version of this Tutorial

Introduction
============
Newcomers often seem to have difficulty with the process of receiving Serial data on the Arduino - especially when they need to receive more than a single character. The fact that there are 18 different functions listed on the Serial reference page probably does not help

You could write a small book and still not cover every possible situation for data reception. Rather than write pages and pages that few would read I thought it would be more useful to present a few examples which will probably cover all of a newcomer's needs. And when you understand these examples you should be able to figure out solutions for other strange cases.

Almost all serial input data can be covered by three simple situations

A - when only a single character is required
B - when only simple manual input from the Serial Monitor is required
C - other

Please note that this text continues into the next Post


Serial data is slow by Arduino standards
========================================
When anything sends serial data to the Arduino it arrives into the Arduino input buffer at a speed set by the baud rate. At 9600 baud about 960 characters arrive per second which means there is a gap of just over 1 millisecond between characters. The Arduino can do a lot in 1 millisecond so the code that follows is designed not to waste time waiting when there is nothing in the input buffer even if all of the data has not yet arrived. Even at 115200 baud there is still 86 microseconds or 1376 Arduino instructions between characters.


Receiving single characters
===========================
In very many cases all that is needed is to send a single character to the Arduino. Between the upper and lower case letters and the numeric characters there are 62 options.

Code to receive a single character is as simple as this

Code: [Select]

char receivedChar;
boolean newData = false;

void setup() {
 Serial.begin(9600);
 Serial.println("<Arduino is ready>");
}

void loop() {
 recvOneChar();
 showNewData();
}

void recvOneChar() {
 if (Serial.available() > 0) {
 receivedChar = Serial.read();
 newData = true;
 }
}

void showNewData() {
 if (newData == true) {
 Serial.print("This just in ... ");
 Serial.println(receivedChar);
 newData = false;
 }
}



Organizing code into functions
==============================
Even though this example is short and simple I have deliberately put the code to receive the character into a separate function called recvOneChar() as that makes it simple to add it into any other program. I also have the code for showing the character in the function showNewData() because you can change that to do whatever you want without upsetting the rest of the code.


Receiving several characters from the Serial Monitor
====================================================
EDIT 05 Feb 2015
Updated to work better - see Replies 22-25 for explanation

If you need to receive more than a single character from the Serial Monitor (perhaps you want to input people's names) you will need some method of letting the Arduino know when it has received the full message. The simplest way to do this is to set the line-ending to newline.

This is done with the box at the bottom of the Serial Monitor window. You can choose between "No line ending", "Newline", "Carriage return" and "Both NL and CR". When you select the "Newline" option a new-line character ('\n') is added at the end of everything you send.

Code: [Select]
const byte numChars = 32;
char receivedChars[numChars]; // an array to store the received data

boolean newData = false;

void setup() {
 Serial.begin(9600);
 Serial.println("<Arduino is ready>");
}

void loop() {
 recvWithEndMarker();
 showNewData();
}

void recvWithEndMarker() {
 static byte ndx = 0;
 char endMarker = '\n';
 char rc;
 
 // if (Serial.available() > 0) {
           while (Serial.available() > 0 && newData == false) {
 rc = Serial.read();

 if (rc != endMarker) {
 receivedChars[ndx] = rc;
 ndx++;
 if (ndx >= numChars) {
 ndx = numChars - 1;
 }
 }
 else {
 receivedChars[ndx] = '\0'; // terminate the string
 ndx = 0;
 newData = true;
 }
 }
}

void showNewData() {
 if (newData == true) {
 Serial.print("This just in ... ");
 Serial.println(receivedChars);
 newData = false;
 }
}


This version of the program reads all the characters into an array until it detects the Newline character as an end marker.

It is important to notice that each time the function recvWithEndMarker() is called it reads at most one character from the input buffer. This assumes that loop() repeats very frequently - perhaps hundreds or thousands of times per second.  If there is nothing in the buffer recvWithEndMarker() does not waste time waiting.

continued in next Post
...R
Two or three hours spent thinking and reading documentation solves most programming problems.

Robin2

#1
Dec 26, 2014, 12:45 pm Last Edit: Jan 15, 2016, 01:30 pm by Robin2
...continued from previous post

A more complete system
======================
EDIT 05 Feb 2015
Updated to work better - see Replies 22-25 for explanation

EDIT 23 Sep 2015
In Reply #69 have included a revised version that works with bytes rather than chars. For most uses the char version in this Reply will be sufficient

The simple system in the previous section will work well with a sympathetic human who does not try to mess it up. But if the computer or person sending the data cannot know when the Arduino is ready to receive there is a real risk that the Arduino will not know where the data starts.

If you would like to explore this, change the end marker in the previous program from '\n' to '>'. This is so that you can include the end marker in your text for illustration purposes. (You can't manually enter a Newline character in the text you are sending from the Serial Monitor). And put the line ending back to "No line ending"

Now, with the revised code send "qwert>" and you will see that it behaves exactly the same as when you were using Newline as the end marker.

But if you try this "asdfg>zxcvb" you will only see the first part "asdfg". And then if you send "qwert>" you will see "zxcvbqwert" because the Arduino has become confused and cannot know that it should have ignored "zxcvb".

The answer to this problem is to include a start marker as well as an end marker.

Code: [Select]
const byte numChars = 32;
char receivedChars[numChars];

boolean newData = false;

void setup() {
    Serial.begin(9600);
    Serial.println("<Arduino is ready>");
}

void loop() {
    recvWithStartEndMarkers();
    showNewData();
}

void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = '<';
    char endMarker = '>';
    char rc;
 
 // if (Serial.available() > 0) {
    while (Serial.available() > 0 && newData == false) {
        rc = Serial.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
        Serial.print("This just in ... ");
        Serial.println(receivedChars);
        newData = false;
    }
}


Edited 24 Oct 2015 to correct the indentation

To see how it works try sending "qwerty<asdfg>zxcvb" and you will see that it ignores everything except "asdfg".

In this program you will notice that there is a new variable called recvInProgress. This is necessary because a distinction needs to be made between unwanted characters that arrive before the start marker and the valid characters that arrive after the start marker.

This version of the program is very similar to the Arduino code in this demo.


Binary data
===========
So far we have been receiving character data - for example the number 137 is represented by the characters '1', '3' and '7'. It is also possible to send that value as binary data in a single byte.

If you need to receive binary data rather than ascii characters just change the line
char receivedChars[numChars];
to
byte receivedChars[numChars];

and the line
char rc;
to
byte rc;

and if you want to continue using the showNewData() function you will also need to change
Serial.println(receivedChars);
to
Serial.println((char*)receivedChars);

(Of course it would also make more sense to change the names of the variables to reflect the fact that they contain bytes rather than chars - but I will leave that as a homework exercise).


Parsing the received data
=========================
Edit 02 May 2015 ... In Reply #43  I have added a simpler example for cases where only a single number is input.

So far there has been no attempt to parse the received data, but that is easy to do once all of the data has been received. The code in this short demo assumes that the data "This is a test, 1234, 45.3" has been received and placed in the array receivedChars. It uses the function strtok() to spilt the data at the commas and the functions atoi() and atof() to convert the ascii data into an integer and a float respectively.

Code: [Select]
// simple parse demo
char receivedChars[] = "This is a test, 1234, 45.3" ;

char messageFromPC[32] = {0};
int integerFromPC = 0;
float floatFromPC = 0.0;

char recvChar;
char endMarker = '>';
boolean newData = false;


void setup() {
 Serial.begin(9600);
 Serial.println("<Arduino is ready>");
 
 parseData();
 showParsedData();
}


void loop() {

}

 
void parseData() {

    // split the data into its parts
    
  char * strtokIndx; // this is used by strtok() as an index
  
  strtokIndx = strtok(receivedChars,",");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
  
  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  integerFromPC = atoi(strtokIndx);     // convert this part to an integer
  
  strtokIndx = strtok(NULL, ",");
  floatFromPC = atof(strtokIndx);     // convert this part to a float

}


void showParsedData() {
 Serial.print("Message ");
 Serial.println(messageFromPC);
 Serial.print("Integer ");
 Serial.println(integerFromPC);
 Serial.print("Float ");
 Serial.println(floatFromPC);
}


This code is as close as possible to the equivalent Arduino code in this demo
Edit 06 Apr 2015 ... this code only works once because strtok() changes the array it is parsing. See Reply #39 for more

Edit 01 Nov 2015 ... See Reply #72 for an example that combines receiving and parsing


If the data must be able to including the markers
=================================================
The demo here shows how to extend this concept so that the data stream can include bytes that have the same values as the start or end markers.


Things that are not used
========================
You will notice that the examples here do not use any of the Arduino functions
Serial.parseInt()
Serial.parseFloat()
Serial.readBytes()
Serial.readBytesUntil()

All of these are blocking functions that prevent the Arduino from doing something else until they are satisfied, or until the timeout expires. The examples here do exactly the same job without blocking.


serialEvent() Added 15 Jan 2016
=========================
I don't recommend using this function - I prefer to deal with the Serial data when it suits me. It behaves just as if you had this code as the last thing in loop()
Code: [Select]
if (Serial.available > 0) {
  mySerialEvent();
}


Clearing the input buffer
=========================
It is probably worth mentioning that the poorly named Serial.flush() function does not empty the input buffer. Its purpose is to block the Arduino until all outgoing the data has been sent.

If you need to ensure the Serial input buffer is empty you can do so like this
while (Serial.available() > 0) {
Serial.read();
}

END

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

econjack

#2
Dec 26, 2014, 01:51 pm Last Edit: Dec 26, 2014, 01:52 pm by econjack
As usual, good job, Robin. One question: When you discuss getting characters and looking for an end character, why not use the Serial object's readBytesUntil() method?

Edit: Never mind! I must have read over your blocking comment!

quasidor-dux

Excellent work and I appreciate the effort.  I picked quite a bit of insight into how the Arduino OS works.

PaulS

Quote
All of these are blocking functions that prevent the Arduino from doing something else until they are satisfied, or until the timeout expires.
The readBytes() function may, or may not, block. It depends on whether or not it is called with a number of bytes to read that is less than, equal to, or more than what Serial.available() reports.
The art of getting good answers lies in asking good questions.

Robin2

The readBytes() function may, or may not, block. It depends on whether or not it is called with a number of bytes to read that is less than, equal to, or more than what Serial.available() reports.
Thanks Paul, I appreciate your taking the trouble to read it.

I agree with your comment. My objective was to present general purpose advice without confusing newcomers with the peculiarities of each possible function.
Two or three hours spent thinking and reading documentation solves most programming problems.

cattledog

#6
Jan 07, 2015, 11:40 pm Last Edit: Jan 07, 2015, 11:41 pm by cattledog
Robin-- Thank you so much for putting this tutorial together. Your efforts are very much appreciated.

There is a small error in the simple parse demo.

There is no third comma in the parsed string and strtok is actually stopping at the terminating NULL character when it parses that final float. 

Below, I have modified parseData() to demonstrate this. The strtok function clearly works with either the specified NULL or the default behaviour when looking for a nonexistant comma.
Code: [Select]
void parseData() {

    // split the data into its parts
   
  char * strtokIndx; // this is used by strtok() as an index
 
  strtokIndx = strtok(receivedChars,",");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
 
  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  integerFromPC = atoi(strtokIndx);     // convert this part to an integer
 
  //strtokIndx = strtok(NULL, ",");
  strtokIndx = strtok(NULL, NULL);
  floatFromPC = atof(strtokIndx);     // convert this part to a float

}

Robin2

Thank you very much for taking the time to study the demo.

I have tested my code again. It seems to work properly and exactly the same with my original code and with your suggested change.

I referred to this webpage about the strtok() function when I was writing my examples and it does not suggest the need for a different form for the final test.

Unless you (or someone else) can show a definite problem I do not propose to change my code as it is easier for newcomers if they do not have to bother with treating the last test as a special case.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

RayLivingston

Using strtok(NULL, ",") and strtok(NULL, NULL) will give exactly the same result.  There is nothing at all wrong with letting strtok run off the end of the string, if that satisfies the parsing requirements, which, in this case, it does.  I would leave it as it.

Regarrds,
Ray L.

bikas

@Robin,
You used recvWithEndMarker() to receive more then one character at a time.But if i declare receivedChar as int and then use receivedChar = Serial.parseInt() i am able to use both the single character and multiple character.Is there some reason why you wrote your code in that way?
Thanks
Bikash


bikas

#10
Jan 13, 2015, 04:26 am Last Edit: Jan 13, 2015, 04:28 am by bikas
sorry i missed one of your comments.Serial.parseInt is a blocking function and but recvWithEndMarker() provides total control over how the data is to be sent.Very informative posts,Thanks for your effort.
Regards,
Bikash

NRU28

for Robin2 , how I can receive and store the wave file into the micro sd ?

Robin2

for Robin2 , how I can receive and store the wave file into the micro sd ?
Just the same way as any other data  - it's just that there will be a lot of bytes. You may want to receive it in (say) 32 byte chunks. When the Arduino has stored the 32 bytes it can ask the PC to send the next 32 bytes.

I have no experience of writing to an SD card.
have you considered writing the SD card on your PC.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

Qiking

#13
Jan 17, 2015, 09:00 pm Last Edit: Jan 17, 2015, 09:33 pm by Qiking
When you use
Code: [Select]
while (Serial.available() > 0) {Serial.read();} to clear the input buffer, do you put everything in the while loop, or put the while loop after everything else? Also, when I try to parse binary data, it gives me an error here
Code: [Select]
strtokIndx = strtok(receivedBytes,","); saying that it's an invalid conversion from byte to char. What should I do?

RayLivingston

strtok returns a pointer to a character array, not an index.

char *s = strtok(receivedBytes,",");
Serial.println(s);

Regards,
Ray L.

Go Up