Serial Input Basics

While trying to help someone else I discovered that the Parse example in Reply #1 will only work once on the same data because the strtok() function changes the array it is working on. In this case it replaces the commas with \0.

The following slightly modified version uses a temporary copy of the array for parsing.

// simple parse demo with repeating output
char receivedChars[] = "This is a test, 1234, 45.3" ;
char tempChars[32];        // temporary array for use by strtok() function

char messageFromPC[32] = {0};
int integerFromPC = 0;
float floatFromPC = 0.0;

char recvChar;
char endMarker = '>';
boolean newData = false;
byte dLen = 0;


void setup() {
    Serial.begin(9600);
    Serial.println("");
}


void loop() {

        // this is necessary because strtok() alters the array
        //   in this case replacing commas with \0
    strcpy(tempChars, receivedChars);
    parseData();
    showParsedData();
    delay(2000);
}

    
void parseData() {

    // split the data into its parts
    
  char * strtokIndx; // this is used by strtok() as an index

  strtokIndx = strtok(tempChars,",");      // get the first part - the string
  strcpy(messageFromPC, strtokIndx); // copy it to messageFromPC
  
  strtokIndx = strtok(NULL, ","); // this continues where the previous call left off
  integerFromPC = atoi(strtokIndx);     // convert this part to an integer

  strtokIndx = strtok(NULL, ","); 
  floatFromPC = atof(strtokIndx);     // convert this part to a float

}


void showParsedData() {
    Serial.print("Message ");
    Serial.println(messageFromPC);
    Serial.print("Integer ");
    Serial.println(integerFromPC);
    Serial.print("Float ");
    Serial.println(floatFromPC);
}

...R

The way I usually handle this is once a complete line is received, immediately use strtok() to step through the line finding each token. Each time strtok() returns a token pointer, stick it an an array of pointers, keeping track of the number of tokens. When done, you have an array of tokens (each token is NULL terminated) and a count, just like argc/argv[] passed to a C main function. It is this array that is repeatedly accessed while determining the current command and it's semantics, it is no longer necessary to parse the original string.

This has a few advantages, like knowing the number of tokens right from the start (makes it easier to validate a complete correct command sequence as soon as you decode the command ID in the first token), plus it eliminates the need to parse the string more than once, and does not require an extra copy of the string or special storage of each token.

Of course, the downside is that it changes the original string, and is a bit more complicated of a concept which starts to move it away from a "basics" discussion.

Many thanks,

I can see that I am seriously out of my depth with this and need to start from some other point! Basically, I don't have a picture of where the various bits of code relate to one another: takes information, gives information etc. But still, thanks for the various bits of code. Beginning to know what one does not know is a good place to start on a project!

Best wishes,

Colin.

colin1: I can see that I am seriously out of my depth with this and need to start from some other point! Basically, I don't have a picture of where the various bits of code relate to one another: takes information, gives information etc. But still, thanks for the various bits of code. Beginning to know what one does not know is a good place to start on a project!

Is this your first post in this Thread? I would be happy to help if you explain in more detail what you are having trouble with. If your question is connected to another Thread about your project it may be better to deal with your question there - if you provide a link to it.

...R

It has occurred to me that there are many cases where a user only wishes to input a single number and the Parse example is unnecessarily complex for that.

The following code is a small extension of the version that receives several characters with an end marker to include code to convert the receivedChars to an int. I have marked all the new lines of code with // new for this version

const byte numChars = 32;
char receivedChars[numChars];   // an array to store the received data

boolean newData = false;

int dataNumber = 0;             // new for this version

void setup() {
    Serial.begin(9600);
    Serial.println("");
}

void loop() {
    recvWithEndMarker();
    showNewData();
}

void recvWithEndMarker() {
    static byte ndx = 0;
    char endMarker = '\n';
    char rc;

    if (Serial.available() > 0) {
        rc = Serial.read();

        if (rc != endMarker) {
            receivedChars[ndx] = rc;
            ndx++;
            if (ndx >= numChars) {
                ndx = numChars - 1;
            }
        }
        else {
            receivedChars[ndx] = '\0'; // terminate the string
            ndx = 0;
            newData = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
        dataNumber = 0;             // new for this version
        dataNumber = atoi(receivedChars);   // new for this version
        Serial.print("This just in ... ");
        Serial.println(receivedChars);
        Serial.print("Data as Number ... ");    // new for this version
        Serial.println(dataNumber);     // new for this version
        newData = false;
    }
}

...R

If I understand this thread completely, this method allows for a high utilization of the serial port for two way communication. Doesn't that come at a cost of processing ability on the Arduino? While not using blocking methods keeps the line "open", collecting everything in character arrays and casting them to other types seems more taxing on the Arduino, and could cause a bottleneck in processing with large data streams and potentially lead to data loss if the buffer backs up too much waiting for data to be processed. Is that a fair assessment?

No, that's a poor assessment.

Putting things into character arrays and casting them to other types requires very little time and resources compared to actually processing the serial communications.

It is difficult for humans to grasp the difference in scale between milliseconds and microseconds.

Geneticus0: While not using blocking methods keeps the line "open", collecting everything in character arrays and casting them to other types seems more taxing on the Arduino

The non-blocking calls are not intended to keep the line "open" but rather to make the maximum number of processing cycles available for other processing.

If you use blocking calls to read the serial data (for example readStringUntil()), the processor will be spending all it's time servicing the serial port. It will be spending most of its time just looping around waiting for data, burning up precious processor cycles for no benefit.

By using the non-blocking calls, those wasted cycles are available to your code to be doing something useful, allowing additional processing to be performed. So, in reality, it's the blocking calls that are taxing the Arduino with extra serial port processing, leaving less time for you to do other things.

Yes, a little bit mire processing is being done on the actual serial data. But this pales in comparison to the amount of processing that is wasted by the traditional blocking serial processing methods.

This is an usefull and important discussion.

One thing which which confused me when beginning Arduino, was Serial.print(78, HEX) BIN, OCT and so on. The data which goes via serial is bytes and it is programmers job to interprete those bytes. C is confusing enough itself.

LMI: This is an usefull and important discussion.

Thank you for your kind words

One thing which which confused me when beginning Arduino, was Serial.print(78, HEX) BIN, OCT and so on. The data which goes via serial is bytes and it is programmers job to interprete those bytes. C is confusing enough itself.

I don't understand what, specifically, you are having a problem with or how you think it may be improved, or better explained. Perhaps you can provide a short program as an example.

I would not normally use HEX, BIN or OCT when sending data to/from a PC. But they can be useful for displaying data in the Serial Monitor.

...R

LMI: One thing which which confused me when beginning Arduino, was Serial.print(78, HEX) BIN, OCT and so on. The data which goes via serial is bytes and it is programmers job to interprete those bytes. C is confusing enough itself.

Yes, the data is a series of bytes. However, many serial port implementations put special meaning on certain byte values, and may even do translations of some of those values (for example, a CR may become LF, or LF may become CR and LF.) Also, some control characters can be interpreted by the code on certain serial inputs, for example there may be line editing features where a backspace deletes a character, and control-X clears the input buffer.

If you have absolute control over the serial port drivers and know that absolutely no special meaning or translation is performed on any of the byte values, then you can go ahead and send the raw byte values over the link. But in many cases it can be not worth the risk.

The solution is to make sure that none of those special byte values are used in the communications stream, and the easiest way to do that is to convert the raw binary bytes into text representations. Using HEX is an efficient way to do so as it uses less characters to transfer data than decimal, octal, or binary, it divides easily into all of the commonly used data element sizes, and there are built in conversion mechanisms in most languages to format and parse hex values. Base64 is an even more compact way to encode binary data into safe printable values, but there aren't the built-in mechanisms in place to generate or decode such data streams.

Yes, hexadecimal notation is a little strange to grasp at first. But it's a valuable skill to learn. Depending on what you're doing with the data, hexadecimal can actually make the math easier! For example, if you want to define a particular bit pattern, with experience it's actually easier to figure out the value in hex than it is in decimal. For example, if you want to set the low bit of a byte, it's 0x1. If you want to set the low bit of the second byte of a word, it's 0x100. If you want to set the low bit of the high byte of a long word, it's 0x1000000. The corresponding decimal values are 1, 256, and 16777216. (I had to get out a calculator to figure out that last value, while all of the hexadecimal values I could do without thinking about it.)

There is a reason for the different number bases (although the reasons for octal are far fewer these days, but it made a lot of sense for older 12 bit computers.) It's worth taking the time to learn them, as they really can make your life easier in many situations - especially when dealing with low level hardware, which is so common in embedded electronics like Arduino projects.

To Robin2: The Serial.print and Serial. write are confusing, because it is not clear what is actually sent. There should be only one instruction which sends data exactly as it is.

To ShapeShifter: I have very little to add. A sure way to problems is to mix data and code. Serial bus itself does not help there. But there are protocols for transferring raw binary data. I think Xmodem or Kermit or something similar exists for Arduino.

For displaying numbers, libraries or functions translating variables like integers to Ascii/Unicode/ascii.hex, and so on, are important. But when those are combined with serial data it just confuses. Serial.print(78, HEX) and so on, were probably made for absolutely beginners, but I think they were more of a problem.

LMI: To Robin2: The Serial.print and Serial. write are confusing, because it is not clear what is actually sent.

Thankfully I had nothing to do with that :)

I suspect part of the problem is your own lack of familiarity with data formats. There are times when Serial.print() is appropriate and times when Serial.write() is necessary.

Have a look at a table of Ascii codes and the difference may become clearer. Each character is represented by an Ascii code (for example 'M' is 77 decimal) but not all of the 256 possible byte values have printable characters associated with them - especially those from 128 to 255.

DEC, HEX, BIN and OCT are just different ways of presenting the same data.

...R

Thank goodness Google finds this post early.

I really appreciate the effort to explain these functions.

Now I just have to work out how to get Excel to read and write to the comm port effortlessly (using default Win7 tools). Wish me luck.

Thanks. Dave

Dave_vo: Now I just have to work out how to get Excel to read and write to the comm port effortlessly (using default Win7 tools). Wish me luck.

Thank you for your kind words.

I don't use Windows myself but you will find lots of info in the Forum section on Interfacing w/Software on the Computer

...R

Dave_vo: Now I just have to work out how to get Excel to read and write to the comm port effortlessly (using default Win7 tools).

https://www.google.com/search?q=gobetwino+excel

[quote author=Coding Badly link=msg=2281107 date=1434612102] https://www.google.com/search?q=gobetwino+excel

[/quote]

OK and thank you for the link. - I could not find it so I might post a complimentary post on Gobetwtino linking this thread.

Does the Serial.Available function works fine with Uno board.Because example said it Only works with Arduino Mega

chameeradp: Does the Serial.Available function works fine with Uno board.Because example said it Only works with Arduino Mega

It works with any Arduino.

Please post a link to the Post where it says it only works with the Mega? - if I have made an error or if my language is not clear I will correct it.

...R

chameeradp: Does the Serial.Available function works fine with Uno board.Because example said it Only works with Arduino Mega

The Serial.Available() reference says this:

Syntax

Serial.available()

Arduino Mega only: Serial1.available() Serial2.available() Serial3.available()

It's saying that Serial1, Serial2, and Serial3 is only available on the Arduino Mega (It's the only board that has the three additional serial ports.)

But Serial (no number after it) is available (no pun intended! ;) ) is on all Arduinos.