Go Down

Topic: Serial Input Basics (Read 358731 times) previous topic - next topic

Stefano2800

#60
Aug 13, 2015, 11:32 am Last Edit: Aug 13, 2015, 11:32 am by Stefano2800
Thank you for the very instructive post and the comments.
It is really worth reading and much clearer than some textbooks I have read.
I have all the answers (that I need :) ) now.
 

Stefano2800

One question,
in Reply #39, the parsing example is written assuming that the content of the serial message has a preset structure: "text, integer, float", so the parsing is done according to this scheme.
I understand that communication must follow a protocol, so this is the normal way of doing it.
But what if I wanted to analize the string to find out the content?
Maybe I could run a loop to find how many dividers and their position in the string can be saved in an array, but how to determine if the different tokens contain char representing text or numbers?
It is definitely too difficult for me, but I'm curious to know how you guys would deal with this  :)
Well, just matter of curiosity anyway.
Thank you again for the useful post.

Robin2

#62
Aug 13, 2015, 03:32 pm Last Edit: Aug 13, 2015, 03:33 pm by Robin2
But what if I wanted to analize the string to find out the content?
It would require a very long reply to answer that question. At one extreme you have the complexities of parsing speech. At the other end you could use the first character to identify which of several protocols the rest of the message contains.

The trick is to design a scheme that is as simple as possible for the task at hand. Speech evolved. By and large computer systems are designed.

If you want to pursue this question please start your own Thread as it is well beyond the scope of this one and will just confuse beginners.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

Stefano2800

Well I expected it would be complicated, I better study more before starting a thread I may not undertand, thank you anyway.

blownupp

I finally created an account just to reply to this thread, but I am thankful and appreciate the work you put into this. I am lucky that I can start with absolutely no knowledge on a subject, spend a couple minutes on google and come out with new skills!

This was exactly what I needed for having my ESP8266 send data to my Pro Mini to parse and display on an LED matrix and 7-seg display. Thanks again!

Robin2

I appreciate the kind comments even if I don't reply separately to each one.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

absbrain

Yes, a little bit mire processing is being done on the actual serial data. But this pales in comparison to the amount of processing that is wasted by the traditional blocking serial processing methods.
First of all, thanx for a very very useful thread. Some questions, (sort of newbie here, long time ago forgotten C programming, used to java, php, bash, etc high level stuff :( )
I found this thread googling for proper way to read serial, put it into String(or char array) and parse it. There are a lot of tutorials which use String class, and read the Serial in a couple of lines of code. I understand the benefits of non-blocking  great example here. But for the sake of argument, what is the "traditional" albeit blocking and ineffiecient way to read serial ?
Question nb 2: There's a thread here somewhere to completely ban the String class. But there are really nice functions like readStringUntil etc, so why not use them ?

Robin2

#67
Sep 09, 2015, 10:23 pm Last Edit: Sep 09, 2015, 10:26 pm by Robin2
But for the sake of argument, what is the "traditional" albeit blocking and ineffiecient way to read serial ?
I don't know that "traditional" is an appropriate word. I don't think there is anything new or non-traditional in my examples.  Serial.parseInt() is a blocking function.

Quote
Question nb 2: There's a thread here somewhere to completely ban the String class. But there are really nice functions like readStringUntil etc, so why not use them ?
Because the Arduino has very little RAM the String class can cause corruption. There are plenty of useful string functions.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

ShapeShifter

To add to Robin2's good comments, the classic blocking serial function is the readStringUntil() function you mention: it's convenient and powerful, but until the specified character is received, the sketch will block and do absolutely nothing else. Some simple sketches can tolerate this, like the ones that illustrate using that function, but most embedded systems must still go on doing the things they do while waiting for serial input. Therefore, the non-blocking techniques that Robin demonstrates are needed.

As for the String class, the issue is that it frequently allocates and frees chunks of dynamic memory as string contents change. Do a search on the terms heap fragmentation and memory leak to see why this could be a problem in the very limited RAM that is typically available to sketches. The string functions that Robin2 mentions work on fixed character arrays, and while they may not be as glamorous, they do not have these memory issues (but you do need to be careful about overrunning your buffer and trying to stuff too much into it.)

Robin2

In response to a question in another Thread I created a version of recvWithStartEndMarkers() that works with bytes rather than chars. This enables it to work with all byte values.

This short program (written for an Uno) sends some test data when it is restarted.
Code: [Select]
byte dataToSend[] = {0x80 ,0x00 ,0x00 ,0x00 ,0x02 ,0x20 ,0x00 ,0x03 ,0x00 ,0x05 ,0x15 ,0x00 ,0x04 ,0x00 ,0x00 ,0x00 ,0x3F ,0x04 ,0x00 ,0x00 ,0x00 ,0x3F ,0x00 ,0x00 ,0x00 ,0x00 ,0x40};

void setup() {
    Serial.begin(9600);
    for (byte n = 0; n < 27; n++) {
        Serial.write(dataToSend[n]);
        //~ Serial.print(dataToSend[n], HEX);
        //~ Serial.print(' ');
    }
}

void loop() {
   
}

and this is the revised version - now called recvBytesWithStartEndMarkers(); It is written to work on a Mega for the convenience of using Serial1 to receive the test data from the Uno while using Serial to show the results on the Serial Monitor.

The Uno and Mega are connected GND to GND and Uno TX to Mega RX1

Code: [Select]
// adapted from Serial Input Basics
// char data type replaced by byte
const byte numBytes = 32;
byte receivedBytes[numBytes];
byte numReceived = 0;

boolean newData = false;

void setup() {
    Serial.begin(9600);
    Serial.println("<Arduino is ready>");
    Serial1.begin(9600);
}

void loop() {
    recvBytesWithStartEndMarkers();
    showNewData();
}

void recvBytesWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    byte startMarker = 0x80;
    byte endMarker = 0x40;
    byte rb;
   

    while (Serial1.available() > 0 && newData == false) {
        rb = Serial1.read();

        if (recvInProgress == true) {
            if (rb != endMarker) {
                receivedBytes[ndx] = rb;
                ndx++;
                if (ndx >= numBytes) {
                    ndx = numBytes - 1;
                }
            }
            else {
                receivedBytes[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                numReceived = ndx;  // save the number for use when printing
                ndx = 0;
                newData = true;
            }
        }

        else if (rb == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
        Serial.print("This just in ... ");
        for (byte n = 0; n < numReceived; n++) {
            Serial.print(receivedBytes[n], HEX);
            Serial.print(' ');
        }
        Serial.println();
        showGroupsOfBytes();
        newData = false;
    }
}

void showGroupsOfBytes() {
    for (byte n = 0; n < numReceived; n++) {
        Serial.print(receivedBytes[n], HEX);
        Serial.print(' ');
        if ((n + 1) % 5 == 0) {
            Serial.println();
        }
    }
    Serial.println();
}


...R
Two or three hours spent thinking and reading documentation solves most programming problems.

3dprinter

#70
Oct 26, 2015, 09:52 am Last Edit: Oct 26, 2015, 09:54 am by Msquare
Ohh.. I haven't read the entire post, but here I offer for your enjoyment and possible help An input routine for simple number input (small code size)

Robin2

From what I have read of the Link in Reply #70 its code is fundamentally different to the approach in the tutorials which started this Thread because the code in that link does not receive all the characters in a message before trying to interpret them.

I prefer the concept of receiving all the data and using the standard atoi() and atof() functions to convert ascii to numeric values.

...R

PS, @MSquare, the essential parts of this Thread are in the first two posts.
Two or three hours spent thinking and reading documentation solves most programming problems.

Robin2

In response to another Thread I have created the following example that combines the receive-with-start-and-end-markers example from Reply #1 and the parse-data example from Reply #39
Code: [Select]
// Receive with start- and end-markers combined with parsing

const byte numChars = 32;
char receivedChars[numChars];
char tempChars[numChars];        // temporary array for use by strtok() function

      // variables to hold the parsed data
char messageFromPC[numChars] = {0};
int integerFromPC = 0;
float floatFromPC = 0.0;

boolean newData = false;

//============

void setup() {
    Serial.begin(9600);
    Serial.println("This demo expects 3 pieces of data - text, an integer and a floating point value");
    Serial.println("Enter data in this style <text,12,24.7>  ");
    Serial.println();
}

//============

void loop() {
    recvWithStartEndMarkers();
    if (newData == true) {
        strcpy(tempChars, receivedChars);
            // this temporary copy is necessary to protect the original data
            //   because strtok() replaces the commas with \0
        parseData();
        showParsedData();
        newData = false;
    }
}

//============

void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = '<';
    char endMarker = '>';
    char rc;

    while (Serial.available() > 0 && newData == false) {
        rc = Serial.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

//============

void parseData() {

      // split the data into its parts
    char * strtokIndx; // this is used by strtok() as an index

    strtokIndx = strtok(tempChars,",");      // get the first part - the string
    strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
 
    strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
    integerFromPC = atoi(strtokIndx);     // convert this part to an integer

    strtokIndx = strtok(NULL, ",");
    floatFromPC = atof(strtokIndx);     // convert this part to a float

}

//============

void showParsedData() {
    Serial.print("Message ");
    Serial.println(messageFromPC);
    Serial.print("Integer ");
    Serial.println(integerFromPC);
    Serial.print("Float ");
    Serial.println(floatFromPC);
}


...R
Two or three hours spent thinking and reading documentation solves most programming problems.

hello_folks

Thank you so much @Robin2,

I was troubling with this for more than 3 weeks to latch multiple characters in an order. Your post simply answered the problem.

And Thanks @/dev for suggesting this.

Cheers...
Have a nice day :)

ardxb

Thanks, Robin2. I have spent all my last week in just to keep received readings straight and put them onto the specific location on their designated TFT LCD boxes. Everything seemed alright but in the middle somewhere a wave of junk comes onto LCD. I was wondering because so far I was thinking that all Serial receive functions are "Blocking" functions, and were focusing only on the strtok and related areas of the code.

The info on Arduino pages are too short and sometimes misleading. As they do not talk things in required depth. I agree no one is paid to do so, but I believe professionals do the professional jobs. No matter they are paid or not, because the work a professional does, does not reflect how much he/she is paid, but it shows how much he loves his work.

I think your posts should be added on to Arduino official pages.

Thanks again.

Go Up