Arduino Forum

Using Arduino => Programming Questions => Topic started by: Robin2 on Dec 26, 2014, 12:45 pm

Title: Serial Input Basics
Post by: Robin2 on Dec 26, 2014, 12:45 pm
EDIT 05 May 2016 - please see the updated and shorter version of this Tutorial (http://forum.arduino.cc/index.php?topic=396450)

Introduction
============
Newcomers often seem to have difficulty with the process of receiving Serial data on the Arduino - especially when they need to receive more than a single character. The fact that there are 18 different functions listed on the Serial reference page (http://arduino.cc/en/Reference/Serial) probably does not help

You could write a small book and still not cover every possible situation for data reception. Rather than write pages and pages that few would read I thought it would be more useful to present a few examples which will probably cover all of a newcomer's needs. And when you understand these examples you should be able to figure out solutions for other strange cases.

Almost all serial input data can be covered by three simple situations

A - when only a single character is required
B - when only simple manual input from the Serial Monitor is required
C - other

Please note that this text continues into the next Post


Serial data is slow by Arduino standards
========================================
When anything sends serial data to the Arduino it arrives into the Arduino input buffer at a speed set by the baud rate. At 9600 baud about 960 characters arrive per second which means there is a gap of just over 1 millisecond between characters. The Arduino can do a lot in 1 millisecond so the code that follows is designed not to waste time waiting when there is nothing in the input buffer even if all of the data has not yet arrived. Even at 115200 baud there is still 86 microseconds or 1376 Arduino instructions between characters.


Receiving single characters
===========================
In very many cases all that is needed is to send a single character to the Arduino. Between the upper and lower case letters and the numeric characters there are 62 options.

Code to receive a single character is as simple as this

Code: [Select]

char receivedChar;
boolean newData = false;

void setup() {
 Serial.begin(9600);
 Serial.println("<Arduino is ready>");
}

void loop() {
 recvOneChar();
 showNewData();
}

void recvOneChar() {
 if (Serial.available() > 0) {
 receivedChar = Serial.read();
 newData = true;
 }
}

void showNewData() {
 if (newData == true) {
 Serial.print("This just in ... ");
 Serial.println(receivedChar);
 newData = false;
 }
}



Organizing code into functions
==============================
Even though this example is short and simple I have deliberately put the code to receive the character into a separate function called recvOneChar() as that makes it simple to add it into any other program. I also have the code for showing the character in the function showNewData() because you can change that to do whatever you want without upsetting the rest of the code.


Receiving several characters from the Serial Monitor
====================================================
EDIT 05 Feb 2015
Updated to work better - see Replies 22-25 for explanation

If you need to receive more than a single character from the Serial Monitor (perhaps you want to input people's names) you will need some method of letting the Arduino know when it has received the full message. The simplest way to do this is to set the line-ending to newline.

This is done with the box at the bottom of the Serial Monitor window. You can choose between "No line ending", "Newline", "Carriage return" and "Both NL and CR". When you select the "Newline" option a new-line character ('\n') is added at the end of everything you send.

Code: [Select]
const byte numChars = 32;
char receivedChars[numChars]; // an array to store the received data

boolean newData = false;

void setup() {
 Serial.begin(9600);
 Serial.println("<Arduino is ready>");
}

void loop() {
 recvWithEndMarker();
 showNewData();
}

void recvWithEndMarker() {
 static byte ndx = 0;
 char endMarker = '\n';
 char rc;
 
 // if (Serial.available() > 0) {
           while (Serial.available() > 0 && newData == false) {
 rc = Serial.read();

 if (rc != endMarker) {
 receivedChars[ndx] = rc;
 ndx++;
 if (ndx >= numChars) {
 ndx = numChars - 1;
 }
 }
 else {
 receivedChars[ndx] = '\0'; // terminate the string
 ndx = 0;
 newData = true;
 }
 }
}

void showNewData() {
 if (newData == true) {
 Serial.print("This just in ... ");
 Serial.println(receivedChars);
 newData = false;
 }
}


This version of the program reads all the characters into an array until it detects the Newline character as an end marker.

It is important to notice that each time the function recvWithEndMarker() is called it reads at most one character from the input buffer. This assumes that loop() repeats very frequently - perhaps hundreds or thousands of times per second.  If there is nothing in the buffer recvWithEndMarker() does not waste time waiting.

continued in next Post
...R
Title: Serial Input Basics contd
Post by: Robin2 on Dec 26, 2014, 12:45 pm
...continued from previous post

A more complete system
======================
EDIT 05 Feb 2015
Updated to work better - see Replies 22-25 for explanation

EDIT 23 Sep 2015
In Reply #69 have included a revised version that works with bytes rather than chars. For most uses the char version in this Reply will be sufficient

The simple system in the previous section will work well with a sympathetic human who does not try to mess it up. But if the computer or person sending the data cannot know when the Arduino is ready to receive there is a real risk that the Arduino will not know where the data starts.

If you would like to explore this, change the end marker in the previous program from '\n' to '>'. This is so that you can include the end marker in your text for illustration purposes. (You can't manually enter a Newline character in the text you are sending from the Serial Monitor). And put the line ending back to "No line ending"

Now, with the revised code send "qwert>" and you will see that it behaves exactly the same as when you were using Newline as the end marker.

But if you try this "asdfg>zxcvb" you will only see the first part "asdfg". And then if you send "qwert>" you will see "zxcvbqwert" because the Arduino has become confused and cannot know that it should have ignored "zxcvb".

The answer to this problem is to include a start marker as well as an end marker.

Code: [Select]
const byte numChars = 32;
char receivedChars[numChars];

boolean newData = false;

void setup() {
    Serial.begin(9600);
    Serial.println("<Arduino is ready>");
}

void loop() {
    recvWithStartEndMarkers();
    showNewData();
}

void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = '<';
    char endMarker = '>';
    char rc;
 
 // if (Serial.available() > 0) {
    while (Serial.available() > 0 && newData == false) {
        rc = Serial.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
        Serial.print("This just in ... ");
        Serial.println(receivedChars);
        newData = false;
    }
}


Edited 24 Oct 2015 to correct the indentation

To see how it works try sending "qwerty<asdfg>zxcvb" and you will see that it ignores everything except "asdfg".

In this program you will notice that there is a new variable called recvInProgress. This is necessary because a distinction needs to be made between unwanted characters that arrive before the start marker and the valid characters that arrive after the start marker.

This version of the program is very similar to the Arduino code in this demo (http://forum.arduino.cc/index.php?topic=225329.msg1810764#msg1810764).


Binary data
===========
So far we have been receiving character data - for example the number 137 is represented by the characters '1', '3' and '7'. It is also possible to send that value as binary data in a single byte.

If you need to receive binary data rather than ascii characters just change the line
char receivedChars[numChars];
to
byte receivedChars[numChars];

and the line
char rc;
to
byte rc;

and if you want to continue using the showNewData() function you will also need to change
Serial.println(receivedChars);
to
Serial.println((char*)receivedChars);

(Of course it would also make more sense to change the names of the variables to reflect the fact that they contain bytes rather than chars - but I will leave that as a homework exercise).


Parsing the received data
=========================
Edit 02 May 2015 ... In Reply #43 (http://forum.arduino.cc/index.php?topic=288234.msg2214119#msg2214119)  I have added a simpler example for cases where only a single number is input.

So far there has been no attempt to parse the received data, but that is easy to do once all of the data has been received. The code in this short demo assumes that the data "This is a test, 1234, 45.3" has been received and placed in the array receivedChars. It uses the function strtok() to spilt the data at the commas and the functions atoi() and atof() to convert the ascii data into an integer and a float respectively.

Code: [Select]
// simple parse demo
char receivedChars[] = "This is a test, 1234, 45.3" ;

char messageFromPC[32] = {0};
int integerFromPC = 0;
float floatFromPC = 0.0;

char recvChar;
char endMarker = '>';
boolean newData = false;


void setup() {
 Serial.begin(9600);
 Serial.println("<Arduino is ready>");
 
 parseData();
 showParsedData();
}


void loop() {

}

 
void parseData() {

    // split the data into its parts
    
  char * strtokIndx; // this is used by strtok() as an index
  
  strtokIndx = strtok(receivedChars,",");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
  
  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  integerFromPC = atoi(strtokIndx);     // convert this part to an integer
  
  strtokIndx = strtok(NULL, ",");
  floatFromPC = atof(strtokIndx);     // convert this part to a float

}


void showParsedData() {
 Serial.print("Message ");
 Serial.println(messageFromPC);
 Serial.print("Integer ");
 Serial.println(integerFromPC);
 Serial.print("Float ");
 Serial.println(floatFromPC);
}


This code is as close as possible to the equivalent Arduino code in this demo (http://forum.arduino.cc/index.php?topic=225329.msg1810764#msg1810764)
Edit 06 Apr 2015 ... this code only works once because strtok() changes the array it is parsing. See Reply #39 for more

Edit 01 Nov 2015 ... See Reply #72 for an example that combines receiving and parsing


If the data must be able to including the markers
=================================================
The demo here (http://forum.arduino.cc/index.php?topic=225329) shows how to extend this concept so that the data stream can include bytes that have the same values as the start or end markers.


Things that are not used
========================
You will notice that the examples here do not use any of the Arduino functions
Serial.parseInt()
Serial.parseFloat()
Serial.readBytes()
Serial.readBytesUntil()

All of these are blocking functions that prevent the Arduino from doing something else until they are satisfied, or until the timeout expires. The examples here do exactly the same job without blocking.


serialEvent() Added 15 Jan 2016
=========================
I don't recommend using this function - I prefer to deal with the Serial data when it suits me. It behaves just as if you had this code as the last thing in loop()
Code: [Select]
if (Serial.available > 0) {
  mySerialEvent();
}


Clearing the input buffer
=========================
It is probably worth mentioning that the poorly named Serial.flush() function does not empty the input buffer. Its purpose is to block the Arduino until all outgoing the data has been sent.

If you need to ensure the Serial input buffer is empty you can do so like this
while (Serial.available() > 0) {
Serial.read();
}

END

...R
Title: Re: Serial Input Basics
Post by: econjack on Dec 26, 2014, 01:51 pm
As usual, good job, Robin. One question: When you discuss getting characters and looking for an end character, why not use the Serial object's readBytesUntil() method?

Edit: Never mind! I must have read over your blocking comment!
Title: Re: Serial Input Basics
Post by: quasidor-dux on Jan 05, 2015, 01:20 am
Excellent work and I appreciate the effort.  I picked quite a bit of insight into how the Arduino OS works.
Title: Re: Serial Input Basics
Post by: PaulS on Jan 05, 2015, 01:50 am
Quote
All of these are blocking functions that prevent the Arduino from doing something else until they are satisfied, or until the timeout expires.
The readBytes() function may, or may not, block. It depends on whether or not it is called with a number of bytes to read that is less than, equal to, or more than what Serial.available() reports.
Title: Re: Serial Input Basics
Post by: Robin2 on Jan 05, 2015, 12:57 pm
The readBytes() function may, or may not, block. It depends on whether or not it is called with a number of bytes to read that is less than, equal to, or more than what Serial.available() reports.
Thanks Paul, I appreciate your taking the trouble to read it.

I agree with your comment. My objective was to present general purpose advice without confusing newcomers with the peculiarities of each possible function.
Title: Re: Serial Input Basics
Post by: cattledog on Jan 07, 2015, 11:40 pm
Robin-- Thank you so much for putting this tutorial together. Your efforts are very much appreciated.

There is a small error in the simple parse demo.

There is no third comma in the parsed string and strtok is actually stopping at the terminating NULL character when it parses that final float. 

Below, I have modified parseData() to demonstrate this. The strtok function clearly works with either the specified NULL or the default behaviour when looking for a nonexistant comma.
Code: [Select]
void parseData() {

    // split the data into its parts
   
  char * strtokIndx; // this is used by strtok() as an index
 
  strtokIndx = strtok(receivedChars,",");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
 
  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  integerFromPC = atoi(strtokIndx);     // convert this part to an integer
 
  //strtokIndx = strtok(NULL, ",");
  strtokIndx = strtok(NULL, NULL);
  floatFromPC = atof(strtokIndx);     // convert this part to a float

}
Title: Re: Serial Input Basics
Post by: Robin2 on Jan 08, 2015, 10:38 am
Thank you very much for taking the time to study the demo.

I have tested my code again. It seems to work properly and exactly the same with my original code and with your suggested change.

I referred to this webpage about the strtok() function (http://www.cplusplus.com/reference/cstring/strtok/) when I was writing my examples and it does not suggest the need for a different form for the final test.

Unless you (or someone else) can show a definite problem I do not propose to change my code as it is easier for newcomers if they do not have to bother with treating the last test as a special case.

...R
Title: Re: Serial Input Basics
Post by: RayLivingston on Jan 08, 2015, 04:24 pm
Using strtok(NULL, ",") and strtok(NULL, NULL) will give exactly the same result.  There is nothing at all wrong with letting strtok run off the end of the string, if that satisfies the parsing requirements, which, in this case, it does.  I would leave it as it.

Regarrds,
Ray L.
Title: Re: Serial Input Basics
Post by: bikas on Jan 13, 2015, 04:21 am
@Robin,
You used recvWithEndMarker() to receive more then one character at a time.But if i declare receivedChar as int and then use receivedChar = Serial.parseInt() i am able to use both the single character and multiple character.Is there some reason why you wrote your code in that way?
Thanks
Bikash

Title: Re: Serial Input Basics
Post by: bikas on Jan 13, 2015, 04:26 am
sorry i missed one of your comments.Serial.parseInt is a blocking function and but recvWithEndMarker() provides total control over how the data is to be sent.Very informative posts,Thanks for your effort.
Regards,
Bikash
Title: Re: Serial Input Basics
Post by: NRU28 on Jan 16, 2015, 03:25 pm
for Robin2 , how I can receive and store the wave file into the micro sd ?
Title: Re: Serial Input Basics
Post by: Robin2 on Jan 16, 2015, 09:56 pm
for Robin2 , how I can receive and store the wave file into the micro sd ?
Just the same way as any other data  - it's just that there will be a lot of bytes. You may want to receive it in (say) 32 byte chunks. When the Arduino has stored the 32 bytes it can ask the PC to send the next 32 bytes.

I have no experience of writing to an SD card.
have you considered writing the SD card on your PC.

...R
Title: Re: Serial Input Basics
Post by: Qiking on Jan 17, 2015, 09:00 pm
When you use
Code: [Select]
while (Serial.available() > 0) {Serial.read();} to clear the input buffer, do you put everything in the while loop, or put the while loop after everything else? Also, when I try to parse binary data, it gives me an error here
Code: [Select]
strtokIndx = strtok(receivedBytes,","); saying that it's an invalid conversion from byte to char. What should I do?
Title: Re: Serial Input Basics
Post by: RayLivingston on Jan 17, 2015, 09:55 pm
strtok returns a pointer to a character array, not an index.

char *s = strtok(receivedBytes,",");
Serial.println(s);

Regards,
Ray L.
Title: Re: Serial Input Basics
Post by: Robin2 on Jan 17, 2015, 11:03 pm
Code: [Select]
strtokIndx = strtok(receivedBytes,",");
 saying that it's an invalid conversion from byte to char. What should I do?
In my examples the data is put into a char array - receivedChars[]. I'm guessing that is what strtok() is intended for. I don't know whether a cast would be suitable.

I don't understand your question about clearing the input buffer - what exactly do you not understand.

...R
Title: Re: Serial Input Basics
Post by: Qiking on Jan 18, 2015, 12:24 am
In my examples the data is put into a char array - receivedChars[]. I'm guessing that is what strtok() is intended for. I don't know whether a cast would be suitable.

I don't understand your question about clearing the input buffer - what exactly do you not understand.

...R
I don't understand where to put the while loop. Do I put it after all my other functions, before all my other functions, or put all my functions inside the while loop?
Title: Re: Serial Input Basics
Post by: Robin2 on Jan 18, 2015, 11:49 am
I don't understand where to put the while loop. Do I put it after all my other functions, before all my other functions, or put all my functions inside the while loop?
Put it at the place in your code where you want to empty the input buffer. You could put the code into a function and just call the function from the appropriate location,

Code: [Select]
void clearInputBuffer() {
   while (Serial.available() > 0) {
        Serial.read();
   }
}

...R
Title: Re: Serial Input Basics
Post by: mmunic194 on Jan 29, 2015, 02:58 pm
Hi,
first of all this was very usefull for me. Now I have one question. Is it possible to use markers to separate string into variables? For example: if I send "<1><2><3>" over serial monitor, how could I store that in to variables like x=1, y=2, z=3? I need that to controll the pins on my arduino so I can send instruction using PLC over the serialc conection. My plan is tu create swich statement wich will use that data to go trough. I want to make it in the way that when the instruction is over (after i send <1><2><3>, it should store it like x=1, y=2, z=3, go trough few swich statements that use variable x,y and z and than stop and wait for next instruction) I hope I made my question clear anough and I'm really looking foward to your reply.
Regards!
Title: Re: Serial Input Basics
Post by: Robin2 on Jan 29, 2015, 04:26 pm
For example: if I send "<1><2><3>" over serial monitor, how could I store that in to variables like x=1, y=2, z=3?
It would be possible, but not so easy because the code will treat each value as the start of a new set of data.
But the demo code does the same thing using <1,2,3> so why not use that system?

Perhaps I don't understand what you want to do?

...R
Title: Re: Serial Input Basics
Post by: RayLivingston on Jan 29, 2015, 05:19 pm
Why not simply send clear, unambiguous commands like: "x=1,y=5,z=3"?  Parse that using strtok(), and you can do whatever you want.

Regards,
Ray L.
Title: Re: Serial Input Basics
Post by: Robin2 on Jan 29, 2015, 10:11 pm
@mmunic194,

Please continue the discussion about your project in your own Thread (http://forum.arduino.cc/index.php?topic=296081) so as not to confuse readers of this one.

...R
Title: Re: Serial Input Basics
Post by: Robin2 on Feb 04, 2015, 04:40 pm
EDIT 05 Feb 2015
Following the very helpful comment in Reply #23 I have updated the code and posted it in Reply #25

The examples in my original Post and in Reply #1 take a single character from the input buffer on each iteration of loop(). However in this Thread (http://forum.arduino.cc/index.php?topic=296333.msg2069157#msg2069157) the problem arose that loop() did not repeat fast enough to keep up with the incoming characters, the buffer overflowed and data was lost.

This following version which has two extra lines in the function recvWithStartEndMarkers() deals with that problem by taking all the available bytes from the buffer on every iteration.

To illustrate this I have added a demonstration function doOtherStuff() which simply has a short 20 millisecond delay() as a crude way to simulate other calculations (or whatever) that might be taking place in a bigger program and preventing loop() from repeating fast enough.

I think the two extra lines I have added to recvWithStartEndMarkers() could be added to any of the examples.

Code: [Select]

const byte numChars = 90;
char receivedChars[numChars];

boolean newData = false;

void setup() {
Serial.begin(9600);
Serial.println("<Arduino is ready -->");
}

void loop() {
recvWithStartEndMarkers();
showNewData();
doOtherStuff(); // <<=== NEW  --- This just simulates time spent elsewhere in a big program
}

void recvWithStartEndMarkers() {
static boolean recvInProgress = false;
static byte ndx = 0;
char startMarker = '<';
char endMarker = '>';
char rc;

if (Serial.available() > 0) {
while (Serial.available() > 0 && newData == false) { // <<== NEW - get all bytes from buffer
rc = Serial.read();

if (recvInProgress == true) {
if (rc != endMarker) {
receivedChars[ndx] = rc;
ndx++;
if (ndx >= numChars) {
ndx = numChars - 1;
}
}
else {
receivedChars[ndx] = '\0'; // terminate the string
recvInProgress = false;
ndx = 0;
newData = true;
}
}

else if (rc == startMarker) {
recvInProgress = true;
}
} // <<=== NEW
}
}

void showNewData() {
if (newData == true) {
Serial.print("This just in ... ");
Serial.println(receivedChars);
newData = false;
}
}

void doOtherStuff() {
delay(20); // <<===NEW - This just simulates time spent elsewhere in a big program
}


...R
Title: Re: Serial Input Basics
Post by: ShapeShifter on Feb 04, 2015, 05:08 pm
That is a good addition!

One possible optimization:
Code: [Select]
if (Serial.available() > 0) {
 while (Serial.available() > 0 && newData == false) { // <<== NEW - get all bytes from buffer
I think the IF test is redundant here, and can be eliminated without affecting the operation, as you will be covered by the new while loop:

It's a minor nit, but it saves a few bytes of precious PROGMEM without affecting operation and readability. If you feel the IF statement helps to show that nothing is done if no data is available, then put it in a comment?
Title: Re: Serial Input Basics
Post by: Robin2 on Feb 04, 2015, 08:02 pm
One possible optimization:I think the IF test is redundant here,
Thanks, I have not tested but I think you are correct and I had overlooked that.

I will test it and probably change it - but I need to change the text as well so it may be tomorrow. It may actually make the explanation simpler.

...R
Title: Re: Serial Input Basics
Post by: Robin2 on Feb 05, 2015, 11:30 am
Having considered @ShapeShifter's comment in Reply #23 I have decided that it would be most useful to users of the original examples to incorporate his change in them. That way people who only read the original post and Reply #1 will get the benefit.

The updated version of the code I posted in Reply #22 is as follows.
Code: [Select]

const byte numChars = 90;
char receivedChars[numChars];

boolean newData = false;

void setup() {
Serial.begin(9600);
Serial.println("<Arduino is ready -->");
}

void loop() {
recvWithStartEndMarkers();
showNewData();
doOtherStuff(); // <<=== NEW  --- This just simulates time spent elsewhere in a big program
}

void recvWithStartEndMarkers() {
static boolean recvInProgress = false;
static byte ndx = 0;
char startMarker = '<';
char endMarker = '>';
char rc;

// if (Serial.available() > 0) {
while (Serial.available() > 0 && newData == false) { // <<== NEW - get all bytes from buffer
rc = Serial.read();

if (recvInProgress == true) {
if (rc != endMarker) {
receivedChars[ndx] = rc;
ndx++;
if (ndx >= numChars) {
ndx = numChars - 1;
}
}
else {
receivedChars[ndx] = '\0'; // terminate the string
recvInProgress = false;
ndx = 0;
newData = true;
}
}

else if (rc == startMarker) {
recvInProgress = true;
}
}
}

void showNewData() {
if (newData == true) {
Serial.print("This just in ... ");
Serial.println(receivedChars);
newData = false;
}
}

void doOtherStuff() {
delay(20); // <<===NEW - This just simulates time spent elsewhere in a big program
}


I have edited the earlier posts so that, hopefully, the whole Thread makes sense.

A big thank-you to @ShapeShifter.

...R
Title: Re: Serial Input Basics
Post by: larryd on Feb 10, 2015, 10:34 pm
Good work as usual.
Thanks for giving your time to this.
Title: Re: Serial Input Basics
Post by: halfdome on Feb 14, 2015, 09:10 pm
<EDIT 15.02.2015> Previously I made a statement in this post about saving quite a bit of memory by using the void serialEvent() function instead of having the watch_serial_port() function inside the main loop.

Unfortunately that observation was all based on a typo in my program code :-(

Apart from saving maybe a few bytes by omitting some duplicate "if (Serial.available())" statements, no further savings can be made.

Title: Re: Serial Input Basics
Post by: ShapeShifter on Feb 14, 2015, 09:31 pm
Code: [Select]
void SerialEvent(){
    while(Serial.available(){
         do_something();
}
Interesting. But keep in mind the note on the SerialEvent() Reference (http://arduino.cc/en/Reference/SerialEvent) page:
Quote
NB : Currently, serialEvent() is not compatible with the Esplora, Leonardo, or Micro
I'm sure this caveat also applies to the Yun, which has the same processor as a Leonardo.

Also, I think you have a typo in your code sample: Shouldn't serialEvent() start with a lower case 's'?
Title: Re: Serial Input Basics
Post by: Robin2 on Feb 15, 2015, 10:40 am
In case one runs out of (flash) memory:
I have deliberately not used serialEvent() in order to keep my examples simple and tranparent.

I would be very interested to see the two code examples you used to determine the saving of 400 bytes.

...R
Title: Re: Serial Input Basics
Post by: halfdome on Feb 15, 2015, 02:19 pm
Unfortunately, my observation of saving memory was based on a typo - using "SerialEvent() " instead of "serialEvent()". Since this new function SerialEvent was never called, the compiler optimized it away...
Title: Re: Serial Input Basics
Post by: Robin2 on Feb 15, 2015, 05:32 pm
Unfortunately, my observation of saving memory was based on a typo - using "SerialEvent() " instead of "serialEvent()". Since this new function SerialEvent was never called, the compiler optimized it away...
Thank you very much for your honesty.

It could happen to anyone.

...R
Title: Re: Serial Input Basics
Post by: Dylan144GT on Mar 13, 2015, 05:15 pm
Hello Robin,

I am struggling with a wireless serial transmission issue and I was wondering if you could please assist me with it? I have started a thread (http://forum.arduino.cc/index.php?topic=307172.0) here.

Thanks very much for your time, it is much appreciated!

Dylan
Title: Re: Serial Input Basics
Post by: ElCaron on Mar 30, 2015, 02:30 pm

I don't quite get recvWithStartEndMarkers(). What if a single  "while Serial.available()" loop iteration finishes before the counterside has send a new character? as far as I can see, the index for the buffer is reset to 0 in each call of the function, so the buffer doesn't seem to be preservered over multiple loop() iterations?
Title: Re: Serial Input Basics
Post by: ShapeShifter on Mar 30, 2015, 03:17 pm
as far as I can see, the index for the buffer is reset to 0 in each call of the function, so the buffer doesn't seem to be preservered over multiple loop() iterations?
ndx, as well as recvInProgress, are both marked static. This means that they are allocated once (static, as in fixed) and are not dynamically allocated on the function's stack frame. This means that they do not get created and destroyed on each call, and their values are preserved from call to call. An assignment made in a static variable's declaration only actually happens once, at system startup time, in order to give it an initial value.

Static variables are allocated at fixed addresses and behave just as if they were globals at the top of the file. But by putting them in the function, their visibility is limited to just that function. So it's a way to have the best of both worlds: the lifespan and persistence of global variables so that values are preserved from call to call, and the limited visibility of local variables so other functions can't mess with the values.
Title: Re: Serial Input Basics
Post by: ElCaron on Mar 30, 2015, 03:30 pm
Ooooooooh, I overlooked that. Thanks!
Title: Re: Serial Input Basics
Post by: denvlad on Apr 03, 2015, 02:57 am
Nice code! Though one problem with using
Code: [Select]

while (Serial.available()) {
  Serial.read();
}

to clear the input buffer (your last advice) is that it happens very fast, which means that we can easily be in a situation when the next character has not yet been received, so the loop will exit and fail to 'clear' the buffer (unless I'm doing something wrong, in which case please correct me :) ). One can easily test this by introducing
Code: [Select]

while (Serial.available()) {
  Serial.read();
  delayMicroseconds(10);
}

which even at 115200 baud rate will still be insufficient to keep up with the input; if you change 10 to 100, it starts to work.

So to alleviate that, we either have to introduce this (somewhat artificial) delay, or wait for the terminating character using something like
Code: [Select]

Serial.readStringUntil(endMarker);

which also has a nice feature of timing out after 1s (that can be changed with Serial.setTimeout())  if no ending marker is provided.
Title: Re: Serial Input Basics
Post by: ShapeShifter on Apr 03, 2015, 09:05 am
to clear the input buffer (your last advice) is that it happens very fast, which means that we can easily be in a situation when the next character has not yet been received, so the loop will exit and fail to 'clear' the buffer
I think you are over-analyzing it and making it harder than it needs to be. The goal is to clear the input buffer: if the next character has not been received, it's not in the input buffer, so how can it be cleared?

This is not a function to ignore characters until some arbitrary end of message marker, or until there is no more data to be received. It is simply to clear out the characters that have already been received. The code was proposed to counter the lack of a flush() function for the serial input. A flush() function typically has two purposes: for an output stream, it makes sure all output has actually been written; for an input stream it discards any input data already available (and not any future input data.) The Serial class implements a flush() function, but only on the output stream. The proposed code is to provide a similar functionality for the input stream.

The major point of this approach to serial processing is to not block processing under any situations. By arbitrarily redefining the semantics of flush() to discard data until some end of message indication subverts that goal by introducing a blocking scenario. If you do want to introduce a function to skip until a certain marker, it would take a little more effort to do it in a non-blocking way. But that was not the intent of the subject code.
Title: Re: Serial Input Basics
Post by: Robin2 on Apr 03, 2015, 10:47 am
@ShapeShifter, thank you for your help here.

...R
Title: Re: Serial Input Basics
Post by: Robin2 on Apr 06, 2015, 10:51 am
While trying to help someone else I discovered that the Parse example in Reply #1 will only work once on the same data because the strtok() function changes the array it is working on. In this case it replaces the commas with \0.

The following slightly modified version uses a temporary copy of the array for parsing.

Code: [Select]
// simple parse demo with repeating output
char receivedChars[] = "This is a test, 1234, 45.3" ;
char tempChars[32];        // temporary array for use by strtok() function

char messageFromPC[32] = {0};
int integerFromPC = 0;
float floatFromPC = 0.0;

char recvChar;
char endMarker = '>';
boolean newData = false;
byte dLen = 0;


void setup() {
    Serial.begin(9600);
    Serial.println("<Arduino is ready>");
}


void loop() {

        // this is necessary because strtok() alters the array
        //   in this case replacing commas with \0
    strcpy(tempChars, receivedChars);
    parseData();
    showParsedData();
    delay(2000);
}

    
void parseData() {

    // split the data into its parts
    
  char * strtokIndx; // this is used by strtok() as an index

  strtokIndx = strtok(tempChars,",");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
  
  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  integerFromPC = atoi(strtokIndx);     // convert this part to an integer

  strtokIndx = strtok(NULL, ",");
  floatFromPC = atof(strtokIndx);     // convert this part to a float

}


void showParsedData() {
    Serial.print("Message ");
    Serial.println(messageFromPC);
    Serial.print("Integer ");
    Serial.println(integerFromPC);
    Serial.print("Float ");
    Serial.println(floatFromPC);
}


...R
Title: Re: Serial Input Basics
Post by: ShapeShifter on Apr 06, 2015, 01:04 pm
The way I usually handle this is once a complete line is received, immediately use strtok() to step through the line finding each token. Each time strtok() returns a token pointer, stick it an an array of pointers, keeping track of the number of tokens. When done, you have an array of tokens (each token is NULL terminated) and a count, just like argc/argv[] passed to a C main function. It is this array that is repeatedly accessed while determining the current command and it's semantics, it is no longer necessary to parse the original string.

This has a few advantages, like knowing the number of tokens right from the start (makes it easier to validate a complete correct command sequence as soon as you decode the command ID in the first token), plus it eliminates the need to parse the string more than once, and does not require an extra copy of the string or special storage of each token.

Of course, the downside is that it changes the original string, and is a bit more complicated of a concept which starts to move it away from a "basics" discussion.
Title: Re: Serial Input Basics
Post by: colin1 on Apr 17, 2015, 05:52 pm
Many thanks,

I can see that I am seriously out of my depth with this and need to start from some other point! Basically, I don't have a picture of where the various bits of code relate to one another: takes information, gives information etc. But still, thanks for the various bits of code. Beginning to know what one does not know is a good place to start on a project!

Best wishes,

Colin.
Title: Re: Serial Input Basics
Post by: Robin2 on Apr 17, 2015, 06:50 pm
I can see that I am seriously out of my depth with this and need to start from some other point! Basically, I don't have a picture of where the various bits of code relate to one another: takes information, gives information etc. But still, thanks for the various bits of code. Beginning to know what one does not know is a good place to start on a project!
Is this your first post in this Thread?
I would be happy to help if you explain in more detail what you are having trouble with.
If your question is connected to another Thread about your project it may be better to deal with your question there - if you provide a link to it.

...R
Title: Re: Serial Input Basics
Post by: Robin2 on May 02, 2015, 12:58 pm
It has occurred to me that there are many cases where a user only wishes to input a single number and the Parse example is unnecessarily complex for that.

The following code is a small extension of the version that receives several characters with an end marker to include code to convert the receivedChars to an int.  I have marked all the new lines of code with // new for this version

Code: [Select]
const byte numChars = 32;
char receivedChars[numChars]; // an array to store the received data

boolean newData = false;

int dataNumber = 0; // new for this version

void setup() {
Serial.begin(9600);
Serial.println("<Arduino is ready>");
}

void loop() {
recvWithEndMarker();
showNewData();
}

void recvWithEndMarker() {
static byte ndx = 0;
char endMarker = '\n';
char rc;

if (Serial.available() > 0) {
rc = Serial.read();

if (rc != endMarker) {
receivedChars[ndx] = rc;
ndx++;
if (ndx >= numChars) {
ndx = numChars - 1;
}
}
else {
receivedChars[ndx] = '\0'; // terminate the string
ndx = 0;
newData = true;
}
}
}

void showNewData() {
if (newData == true) {
dataNumber = 0; // new for this version
dataNumber = atoi(receivedChars); // new for this version
Serial.print("This just in ... ");
Serial.println(receivedChars);
Serial.print("Data as Number ... "); // new for this version
Serial.println(dataNumber); // new for this version
newData = false;
}
}


...R
Title: Re: Serial Input Basics
Post by: Geneticus0 on May 26, 2015, 05:45 am
If I understand this thread completely, this method allows for a high utilization of the serial port for two way communication. Doesn't that come at a cost of processing ability on the Arduino? While not using blocking methods keeps the line "open", collecting everything in character arrays and casting them to other types seems more taxing on the Arduino, and could cause a bottleneck in processing with large data streams and potentially lead to data loss if the buffer backs up too much waiting for data to be processed. Is that a fair assessment?
Title: Re: Serial Input Basics
Post by: michinyon on May 26, 2015, 06:11 am
No,  that's a poor assessment.

Putting things into character arrays and casting them to other types requires very little time and resources compared to actually processing the serial communications.

It is difficult for humans to grasp the difference in scale between milliseconds and microseconds.
Title: Re: Serial Input Basics
Post by: ShapeShifter on May 26, 2015, 12:51 pm
While not using blocking methods keeps the line "open", collecting everything in character arrays and casting them to other types seems more taxing on the Arduino
The non-blocking calls are not intended to keep the line "open" but rather to make the maximum number of processing cycles available for other processing.

If you use blocking calls to read the serial data (for example readStringUntil()), the processor will be spending all it's time servicing the serial port. It will be spending most of its time just looping around waiting for data, burning up precious processor cycles for no benefit.

By using the non-blocking calls, those wasted cycles are available to your code to be doing something useful, allowing additional processing to be performed. So, in reality, it's the blocking calls that are taxing the Arduino with extra serial port processing, leaving less time for you to do other things.

Yes, a little bit mire processing is being done on the actual serial data. But this pales in comparison to the amount of processing that is wasted by the traditional blocking serial processing methods.
Title: Re: Serial Input Basics
Post by: LMI1 on Jun 09, 2015, 03:10 pm
This is an usefull and important discussion.

One thing which which confused me when beginning Arduino, was Serial.print(78, HEX) BIN, OCT and so on. The data which goes via serial is bytes and it is programmers job to interprete those bytes. C is confusing enough itself.
Title: Re: Serial Input Basics
Post by: Robin2 on Jun 09, 2015, 03:26 pm
This is an usefull and important discussion.
Thank you for your kind words

Quote
One thing which which confused me when beginning Arduino, was Serial.print(78, HEX) BIN, OCT and so on. The data which goes via serial is bytes and it is programmers job to interprete those bytes. C is confusing enough itself.
I don't understand what, specifically, you are having a problem with or how you think it may be improved, or better explained. Perhaps you can provide a short program as an example.

I would not normally use HEX, BIN or OCT when sending data to/from a PC. But they can be useful for displaying data in the Serial Monitor.

...R
Title: Re: Serial Input Basics
Post by: ShapeShifter on Jun 09, 2015, 05:08 pm
One thing which which confused me when beginning Arduino, was Serial.print(78, HEX) BIN, OCT and so on. The data which goes via serial is bytes and it is programmers job to interprete those bytes. C is confusing enough itself.
Yes, the data is a series of bytes. However, many serial port implementations put special meaning on certain byte values, and may even do translations of some of those values (for example, a CR may become LF, or LF may become CR and LF.) Also, some control characters can be interpreted by the code on certain serial inputs, for example there may be line editing features where a backspace deletes a character, and control-X clears the input buffer.

If you have absolute control over the serial port drivers and know that absolutely no special meaning or translation is performed on any of the byte values, then you can go ahead and send the raw byte values over the link. But in many cases it can be not worth the risk.

The solution is to make sure that none of those special byte values are used in the communications stream, and the easiest way to do that is to convert the raw binary bytes into text representations. Using HEX is an efficient way to do so as it uses less characters to transfer data than decimal, octal, or binary, it divides easily into all of the commonly used data element sizes, and there are built in conversion mechanisms in most languages to format and parse hex values. Base64 is an even more compact way to encode binary data into safe printable values, but there aren't the built-in mechanisms in place to generate or decode such data streams.

Yes, hexadecimal notation is a little strange to grasp at first. But it's a valuable skill to learn. Depending on what you're doing with the data, hexadecimal can actually make the math easier! For example, if you want to define a particular bit pattern, with experience it's actually easier to figure out the value in hex than it is in decimal. For example, if you want to set the low bit of a byte, it's 0x1. If you want to set the low bit of the second byte of a word, it's 0x100. If you want to set the low bit of the high byte of a long word, it's 0x1000000. The corresponding decimal values are 1, 256, and 16777216. (I had to get out a calculator to figure out that last value, while all of the hexadecimal values I could do without thinking about it.)

There is a reason for the different number bases (although the reasons for octal are far fewer these days, but it made a lot of sense for older 12 bit computers.) It's worth taking the time to learn them, as they really can make your life easier in many situations - especially when dealing with low level hardware, which is so common in embedded electronics like Arduino projects.
Title: Re: Serial Input Basics
Post by: LMI1 on Jun 09, 2015, 06:23 pm
To Robin2:
The Serial.print and Serial. write are confusing, because it is not clear what is actually sent. There should be only one instruction which sends data exactly as it is.

To ShapeShifter: I have very little to add. A sure way to problems is to mix data and code. Serial bus itself does not help there. But there are protocols for transferring raw binary data. I think Xmodem or Kermit or something similar exists for Arduino.

For displaying numbers, libraries or functions translating variables like integers to Ascii/Unicode/ascii.hex, and so on, are important. But when those are combined with serial data it just confuses. Serial.print(78, HEX) and so on, were probably made for absolutely beginners, but I think they were more of a problem.
Title: Re: Serial Input Basics
Post by: Robin2 on Jun 09, 2015, 08:48 pm
To Robin2:
The Serial.print and Serial. write are confusing, because it is not clear what is actually sent.
Thankfully I had nothing to do with that  :)

I suspect part of the problem is your own lack of familiarity with data formats. There are times when Serial.print() is appropriate and times when Serial.write() is necessary.

Have a look at a table of Ascii codes and the difference may become clearer. Each character is represented by an Ascii code (for example 'M' is 77 decimal) but not all of the 256 possible byte values have printable characters associated with them - especially those from 128 to 255.

DEC, HEX, BIN and OCT are just different ways of presenting the same data.

...R
Title: Re: Serial Input Basics
Post by: Dave_vo on Jun 18, 2015, 05:31 am
Thank goodness Google finds this post early.

I really appreciate the effort to explain these functions.

Now I just have to work out how to get Excel to read and write to the comm port effortlessly (using default Win7 tools). Wish me luck.
 
Thanks. Dave

Title: Re: Serial Input Basics
Post by: Robin2 on Jun 18, 2015, 09:17 am
Now I just have to work out how to get Excel to read and write to the comm port effortlessly (using default Win7 tools). Wish me luck.
 
Thank you for your kind words.

I don't use Windows myself but you will find lots of info in the Forum section on Interfacing w/Software on the Computer

...R
Title: Re: Serial Input Basics
Post by: Coding Badly on Jun 18, 2015, 09:21 am
Now I just have to work out how to get Excel to read and write to the comm port effortlessly (using default Win7 tools).
https://www.google.com/search?q=gobetwino+excel (https://www.google.com/search?q=gobetwino+excel)

Title: Re: Serial Input Basics
Post by: Dave_vo on Jun 19, 2015, 04:06 am
https://www.google.com/search?q=gobetwino+excel (https://www.google.com/search?q=gobetwino+excel)


OK and thank you for the link. - I could not find it so I might post a complimentary post on Gobetwtino linking this thread.
Title: Re: Serial Input Basics
Post by: chameeradp on Jul 09, 2015, 02:55 pm
Does the Serial.Available function works fine with Uno board.Because example said it Only works with Arduino Mega
Title: Re: Serial Input Basics
Post by: Robin2 on Jul 09, 2015, 05:22 pm
Does the Serial.Available function works fine with Uno board.Because example said it Only works with Arduino Mega
It works with any Arduino.

Please post a link to the Post where it says it only works with the Mega? - if I have made an error or if my language is not clear I will correct it.

...R
Title: Re: Serial Input Basics
Post by: ShapeShifter on Jul 09, 2015, 10:53 pm
Does the Serial.Available function works fine with Uno board.Because example said it Only works with Arduino Mega
The Serial.Available() reference  (https://www.arduino.cc/en/Serial/Available)says this:

Quote
Syntax

Serial.available()

Arduino Mega only:
Serial1.available()
Serial2.available()
Serial3.available()
It's saying that Serial1, Serial2, and Serial3 is only available on the Arduino Mega (It's the only board that has the three additional serial ports.)

But Serial (no number after it) is available (no pun intended! ;) ) is on all Arduinos.
Title: Re: Serial Input Basics
Post by: Robin2 on Jul 10, 2015, 09:35 am
Thanks @Shapeshifter - that may well be the source of confusion.


@chameeradp, if the confusion does arise within something I have written please let us know where it is.

...R
Title: Re: Serial Input Basics
Post by: Stefano2800 on Aug 13, 2015, 11:32 am
Thank you for the very instructive post and the comments.
It is really worth reading and much clearer than some textbooks I have read.
I have all the answers (that I need :) ) now.
 
Title: Re: Serial Input Basics
Post by: Stefano2800 on Aug 13, 2015, 02:39 pm
One question,
in Reply #39, the parsing example is written assuming that the content of the serial message has a preset structure: "text, integer, float", so the parsing is done according to this scheme.
I understand that communication must follow a protocol, so this is the normal way of doing it.
But what if I wanted to analize the string to find out the content?
Maybe I could run a loop to find how many dividers and their position in the string can be saved in an array, but how to determine if the different tokens contain char representing text or numbers?
It is definitely too difficult for me, but I'm curious to know how you guys would deal with this  :)
Well, just matter of curiosity anyway.
Thank you again for the useful post.
Title: Re: Serial Input Basics
Post by: Robin2 on Aug 13, 2015, 03:32 pm
But what if I wanted to analize the string to find out the content?
It would require a very long reply to answer that question. At one extreme you have the complexities of parsing speech. At the other end you could use the first character to identify which of several protocols the rest of the message contains.

The trick is to design a scheme that is as simple as possible for the task at hand. Speech evolved. By and large computer systems are designed.

If you want to pursue this question please start your own Thread as it is well beyond the scope of this one and will just confuse beginners.

...R
Title: Re: Serial Input Basics
Post by: Stefano2800 on Aug 15, 2015, 05:35 am
Well I expected it would be complicated, I better study more before starting a thread I may not undertand, thank you anyway.
Title: Re: Serial Input Basics
Post by: blownupp on Aug 20, 2015, 06:20 pm
I finally created an account just to reply to this thread, but I am thankful and appreciate the work you put into this. I am lucky that I can start with absolutely no knowledge on a subject, spend a couple minutes on google and come out with new skills!

This was exactly what I needed for having my ESP8266 send data to my Pro Mini to parse and display on an LED matrix and 7-seg display. Thanks again!
Title: Re: Serial Input Basics
Post by: Robin2 on Aug 20, 2015, 07:38 pm
I appreciate the kind comments even if I don't reply separately to each one.

...R
Title: Re: Serial Input Basics
Post by: absbrain on Sep 09, 2015, 09:57 pm
Yes, a little bit mire processing is being done on the actual serial data. But this pales in comparison to the amount of processing that is wasted by the traditional blocking serial processing methods.
First of all, thanx for a very very useful thread. Some questions, (sort of newbie here, long time ago forgotten C programming, used to java, php, bash, etc high level stuff :( )
I found this thread googling for proper way to read serial, put it into String(or char array) and parse it. There are a lot of tutorials which use String class, and read the Serial in a couple of lines of code. I understand the benefits of non-blocking  great example here. But for the sake of argument, what is the "traditional" albeit blocking and ineffiecient way to read serial ?
Question nb 2: There's a thread here somewhere to completely ban the String class. But there are really nice functions like readStringUntil etc, so why not use them ?
Title: Re: Serial Input Basics
Post by: Robin2 on Sep 09, 2015, 10:23 pm
But for the sake of argument, what is the "traditional" albeit blocking and ineffiecient way to read serial ?
I don't know that "traditional" is an appropriate word. I don't think there is anything new or non-traditional in my examples.  Serial.parseInt() is a blocking function.

Quote
Question nb 2: There's a thread here somewhere to completely ban the String class. But there are really nice functions like readStringUntil etc, so why not use them ?
Because the Arduino has very little RAM the String class can cause corruption. There are plenty of useful string functions (http://www.cplusplus.com/reference/cstring/).

...R
Title: Re: Serial Input Basics
Post by: ShapeShifter on Sep 10, 2015, 01:55 pm
To add to Robin2's good comments, the classic blocking serial function is the readStringUntil() function you mention: it's convenient and powerful, but until the specified character is received, the sketch will block and do absolutely nothing else. Some simple sketches can tolerate this, like the ones that illustrate using that function, but most embedded systems must still go on doing the things they do while waiting for serial input. Therefore, the non-blocking techniques that Robin demonstrates are needed.

As for the String class, the issue is that it frequently allocates and frees chunks of dynamic memory as string contents change. Do a search on the terms heap fragmentation and memory leak to see why this could be a problem in the very limited RAM that is typically available to sketches. The string functions that Robin2 mentions work on fixed character arrays, and while they may not be as glamorous, they do not have these memory issues (but you do need to be careful about overrunning your buffer and trying to stuff too much into it.)
Title: Re: Serial Input Basics
Post by: Robin2 on Sep 23, 2015, 09:37 am
In response to a question in another Thread (http://forum.arduino.cc/index.php?topic=347632.msg2404169#msg2404169) I created a version of recvWithStartEndMarkers() that works with bytes rather than chars. This enables it to work with all byte values.

This short program (written for an Uno) sends some test data when it is restarted.
Code: [Select]
byte dataToSend[] = {0x80 ,0x00 ,0x00 ,0x00 ,0x02 ,0x20 ,0x00 ,0x03 ,0x00 ,0x05 ,0x15 ,0x00 ,0x04 ,0x00 ,0x00 ,0x00 ,0x3F ,0x04 ,0x00 ,0x00 ,0x00 ,0x3F ,0x00 ,0x00 ,0x00 ,0x00 ,0x40};

void setup() {
    Serial.begin(9600);
    for (byte n = 0; n < 27; n++) {
        Serial.write(dataToSend[n]);
        //~ Serial.print(dataToSend[n], HEX);
        //~ Serial.print(' ');
    }
}

void loop() {
   
}

and this is the revised version - now called recvBytesWithStartEndMarkers(); It is written to work on a Mega for the convenience of using Serial1 to receive the test data from the Uno while using Serial to show the results on the Serial Monitor.

The Uno and Mega are connected GND to GND and Uno TX to Mega RX1

Code: [Select]
// adapted from Serial Input Basics
// char data type replaced by byte
const byte numBytes = 32;
byte receivedBytes[numBytes];
byte numReceived = 0;

boolean newData = false;

void setup() {
    Serial.begin(9600);
    Serial.println("<Arduino is ready>");
    Serial1.begin(9600);
}

void loop() {
    recvBytesWithStartEndMarkers();
    showNewData();
}

void recvBytesWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    byte startMarker = 0x80;
    byte endMarker = 0x40;
    byte rb;
   

    while (Serial1.available() > 0 && newData == false) {
        rb = Serial1.read();

        if (recvInProgress == true) {
            if (rb != endMarker) {
                receivedBytes[ndx] = rb;
                ndx++;
                if (ndx >= numBytes) {
                    ndx = numBytes - 1;
                }
            }
            else {
                receivedBytes[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                numReceived = ndx;  // save the number for use when printing
                ndx = 0;
                newData = true;
            }
        }

        else if (rb == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
        Serial.print("This just in ... ");
        for (byte n = 0; n < numReceived; n++) {
            Serial.print(receivedBytes[n], HEX);
            Serial.print(' ');
        }
        Serial.println();
        showGroupsOfBytes();
        newData = false;
    }
}

void showGroupsOfBytes() {
    for (byte n = 0; n < numReceived; n++) {
        Serial.print(receivedBytes[n], HEX);
        Serial.print(' ');
        if ((n + 1) % 5 == 0) {
            Serial.println();
        }
    }
    Serial.println();
}


...R
Title: Re: Serial Input Basics
Post by: 3dprinter on Oct 26, 2015, 09:52 am
Ohh.. I haven't read the entire post, but here I offer for your enjoyment and possible help An input routine for simple number input (http://forum.arduino.cc/index.php?topic=115174.0) (small code size)
Title: Re: Serial Input Basics
Post by: Robin2 on Oct 26, 2015, 10:03 am
From what I have read of the Link in Reply #70 its code is fundamentally different to the approach in the tutorials which started this Thread because the code in that link does not receive all the characters in a message before trying to interpret them.

I prefer the concept of receiving all the data and using the standard atoi() and atof() functions to convert ascii to numeric values.

...R

PS, @MSquare, the essential parts of this Thread are in the first two posts.
Title: Re: Serial Input Basics
Post by: Robin2 on Nov 01, 2015, 03:58 pm
In response to another Thread I have created the following example that combines the receive-with-start-and-end-markers example from Reply #1 and the parse-data example from Reply #39
Code: [Select]
// Receive with start- and end-markers combined with parsing

const byte numChars = 32;
char receivedChars[numChars];
char tempChars[numChars];        // temporary array for use by strtok() function

      // variables to hold the parsed data
char messageFromPC[numChars] = {0};
int integerFromPC = 0;
float floatFromPC = 0.0;

boolean newData = false;

//============

void setup() {
    Serial.begin(9600);
    Serial.println("This demo expects 3 pieces of data - text, an integer and a floating point value");
    Serial.println("Enter data in this style <text,12,24.7>  ");
    Serial.println();
}

//============

void loop() {
    recvWithStartEndMarkers();
    if (newData == true) {
        strcpy(tempChars, receivedChars);
            // this temporary copy is necessary to protect the original data
            //   because strtok() replaces the commas with \0
        parseData();
        showParsedData();
        newData = false;
    }
}

//============

void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = '<';
    char endMarker = '>';
    char rc;

    while (Serial.available() > 0 && newData == false) {
        rc = Serial.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

//============

void parseData() {

      // split the data into its parts
    char * strtokIndx; // this is used by strtok() as an index

    strtokIndx = strtok(tempChars,",");      // get the first part - the string
    strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
 
    strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
    integerFromPC = atoi(strtokIndx);     // convert this part to an integer

    strtokIndx = strtok(NULL, ",");
    floatFromPC = atof(strtokIndx);     // convert this part to a float

}

//============

void showParsedData() {
    Serial.print("Message ");
    Serial.println(messageFromPC);
    Serial.print("Integer ");
    Serial.println(integerFromPC);
    Serial.print("Float ");
    Serial.println(floatFromPC);
}


...R
Title: Re: Serial Input Basics
Post by: hello_folks on Dec 02, 2015, 05:05 am
Thank you so much @Robin2,

I was troubling with this for more than 3 weeks to latch multiple characters in an order. Your post simply answered the problem.

And Thanks @/dev for suggesting this.

Cheers...
Have a nice day :)
Title: Re: Serial Input Basics
Post by: ardxb on Jan 10, 2016, 01:39 pm
Thanks, Robin2. I have spent all my last week in just to keep received readings straight and put them onto the specific location on their designated TFT LCD boxes. Everything seemed alright but in the middle somewhere a wave of junk comes onto LCD. I was wondering because so far I was thinking that all Serial receive functions are "Blocking" functions, and were focusing only on the strtok and related areas of the code.

The info on Arduino pages are too short and sometimes misleading. As they do not talk things in required depth. I agree no one is paid to do so, but I believe professionals do the professional jobs. No matter they are paid or not, because the work a professional does, does not reflect how much he/she is paid, but it shows how much he loves his work.

I think your posts should be added on to Arduino official pages.

Thanks again.
Title: Re: Serial Input Basics
Post by: ghlawrence2000 on Feb 25, 2016, 11:54 am
@Robin2 Thanks for a brilliant tutorial! I am not exactly a newbie but I still found it very useful.

Developing the parsedata() function a little further I can now send commands like <getInt32()> <writeImage(0xbb800, 800, 480)> <sendBytes(0x1fe400, 256)> with up to 3 parameters and the parameters can be Hex or decimal.

Code: [Select]
void parseData() {
  char * strtokIndx; // this is used by strtok() as an index
  strtokIndx = strtok(tempChars, "(");     // get the first part - the string
  strcpy(COMMANDFromPC, strtokIndx); // copy it to COMMANDFromPC
  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  if (!(strncmp(strtokIndx, "0x", 2))) {
    sscanf(strtokIndx, "%2s%lx", NULL, &VAL1FromPC);
  } else {
    VAL1FromPC =  atoi(strtokIndx);  // convert this part to an integer
  }
  strtokIndx = strtok(NULL, ",");
  if (!(strncmp(strtokIndx, "0x", 2))) {
    sscanf(strtokIndx, "%2s%lx", NULL, &VAL2FromPC);
  } else {
    VAL2FromPC = atoi(strtokIndx);  // convert this part to an integer
  }
  strtokIndx = strtok(NULL, ",");
  if (!(strncmp(strtokIndx, "0x", 2))) {
    sscanf(strtokIndx, "%2s%lx", NULL, &VAL3FromPC);
  } else {
    VAL3FromPC = atoi(strtokIndx);  // convert this part to an integer
  }
}


Continuing with the theme, I then implemented a bool to allow switching between 'command' and 'data' mode.

Partial 'loop' function.
Code: [Select]
void loop() {
  serialEvent();
  recvWithStartEndMarkers();
  if (newData == true) {
    strcpy(tempChars, receivedChars);
    parseData();
    if (!(strcmp(COMMANDFromPC, "ERASE"))) {
      erase();
    }
    if (!(strcmp(COMMANDFromPC, "getInt32"))) {
      readInt32(VAL1FromPC);
    }
    if (!(strcmp(COMMANDFromPC, "getString"))) {
      readString(VAL1FromPC);
    }
    if (!(strcmp(COMMANDFromPC, "getInt16"))) {
      readInt16(VAL1FromPC);
    }


The only changes to serialEvent and recvWithStartEndMarkers are the switch to check which mode we are in.

Code: [Select]
void serialEvent() {
  if (datamode) {


and

Code: [Select]
void recvWithStartEndMarkers() {
  if (!datamode) {


Again, brilliant article, thank you.

Regards,

Graham
Title: Re: Serial Input Basics
Post by: PaulS on Feb 25, 2016, 12:44 pm
Code: [Select]
  if (!(strncmp(strtokIndx, "0x", 2))) {
This is FAR easier to read if you accept the fact that strcmp() returns -1 or +1 if the strings don't match, depending on the order of the strings in the dictionary, and returns 0 if they do match.

Code: [Select]
  if (strncmp(strtokIndx, "0x", 2) != 0)
  {


Quote
Again, brilliant article, thank you.
Agreed.
Title: Re: Serial Input Basics
Post by: LMI1 on Feb 25, 2016, 08:48 pm
These functions should be included into the IDE. Transferring larger than byte variables is a common subject in the forum, in my opinion
Title: Re: Serial Input Basics
Post by: Coding Badly on Feb 25, 2016, 10:05 pm
These functions should be included into the IDE.
https://github.com/arduino/Arduino/ (https://github.com/arduino/Arduino/)

Create an issue.  Attach a pull request to the issue.

Title: Re: Serial Input Basics
Post by: LMI1 on Mar 02, 2016, 06:08 pm
https://github.com/arduino/Arduino/ (https://github.com/arduino/Arduino/)

Create an issue.
I can write to that forum.

Attach a pull request to the issue.
What does that mean? How to do it?
Title: Re: Serial Input Basics
Post by: Robin2 on Mar 02, 2016, 07:14 pm
I seem to have missed the Posts to this Thread in February 2016 (apologies, if required).

@ghlawrence2000, it seems as if your developments are particular to your needs rather than appropriate for a general tutorial.

You should not call serialEvent() from within loop(). It is called automatically by the hidden main() function. However, as I mentioned in my tutorial I don't think serialEvent() adds anything useful to the Arduino system and I never use it.

Because my code does not use serialEvent() I don't understand the statement
Quote
The only changes to serialEvent and recvWithStartEndMarkers are
@LMI, I don't agree that the functions I have written should be in the Arduino IDE. My functions were written for demonstration purposed and it is unlikely that anyone will want to use them without making some changes. It is easier to see the code and change it when it is not in the IDE.

...R