Understanding ==> Serial.read[Bytes|String][Until]

[Beavis4ever]

I need some 'splaining re: Reference > Language > Functions > Communication > Serial

• Is the buffer parameter in Serial.readBytes() and Serial.readBytesUntil() the same as array? So if I already declared char message[6] = "hello"; and had "imbecile\n" waiting in the serial port when Serial.readBytes(message); is called, what happens? Does message remain 6 bytes in size as it was declared, and so now it holds "imbec\n" (the first 5 characters from the serial port plus the null terminating character that is required when declaring an array of type char)? If Serial.readBytes(message); is called again, when the 4th character pulled off the serial port turns out to be the \n {which is then discarded}, would message then return with "ile\n"? Is message now only 4 bytes in size, or does it stay 6 bytes with the last 2 bytes left-over from before: "ile\nc\n"
• Is the terminator parameter in Serial.readStringUntil() the same as the character parameter in Serial.readBytesUntil() ? Both parameters say ,"the character to search for. Allowed data types: char." I confuse because I wouldn't give 2 different names to a parameter which does the same exact thing. And the use of the word "terminator" seems a poor choice when your char array (which is now called a buffer?) ALSO has a terminating character!

(So I don't commit an "XY question" faux pas:)
What I am trying to ultimately do is in dealing with the IRremote library. I took this code from the IRrecvDemo example:

  if (irrecv.decode(&results)) {
    Serial.println(results.value, HEX);
    irrecv.resume();

I want to take the signal sent by my TV remote, and which was just read by the module irrecv.enableIRIn() and stored in results.value (or is it &results ?) and NOT do a Serial.println, but instead store it in a variable of my choosing (to be checked against a list of codes which will correspond to some preset macros) and then send the data over the serial bus in binary form using **Serial.write() ** If my remote uses 32 bit codes, instead of the 8 character hex transmission I can just send 4 bytes in binary form. I suspect that all the extra code needed to save 4 bytes sent out the serial port may cost more than it saves, but I really want to do it this way because it's what I'm already familiar with. (kind of like a "hello world" example) Once I have my old way working fine, then I may try to stretch myself and learn new things. But for now I just want to straighten out what I have experience with.
Thanks in advance to anyone ambitious enough to explain this to my sick mind!

[sterretje]

Regarding the second part of your question, results.value is an uint32_t so you can save it in a uint32_t and you can Serial.write() the four bytes instead of using Serial.print().

Regarding the first part of your question, I'm reasonable sure of the below but you can easily test it yourself to verify using e.g.

for(byte cnt=0; cnt< sizeof(message); cnt++)
{
  Serial.println(message[cnt], HEX);
}

point (1)
For readBytesUntill
It depends what the length parameter is. If it's strlen(message) or 5, it will contain the first 5 bytes of "imbecile\n" and will still contain the terminating '\0'. If it's sizeof(message) or 6, it will contain the first 6 bytes of "imbecile\n" and will not contain the terminating '\0' so it's not a c-string.

You will have to test if the terminator is included or stripped.

As I don't use readStringUntil, I've never looked into that.

point (2)
Yes, they are the same.

[pert]

Beavis4ever:
Is the buffer parameter in Serial.readBytes() and Serial.readBytesUntil() the same as array?

The buffer parameter is an array of char or bytes.

Beavis4ever:
Does message remain 6 bytes in size as it was declared

Yes. An array will always stay the same size as it was declared.

Beavis4ever:
when Serial.readBytes(message); is called, what happens?

It's impossible to call Serial.readBytes(message); because a function of that signature was never declared and so the code won't compile. The length parameter is required.

Beavis4ever:
Is message now only 4 bytes in size, or does it stay 6 bytes with the last 2 bytes left-over from before: "ile\nc\n"

As I already explained, message will always be 6 bytes. Now if you called strlen(message) it would return 3 because that is the length of the string in message.

Beavis4ever:
Is messageor does it stay 6 bytes with the last 2 bytes left-over from before: "ile\nc\n"

Yes, but anything that might be left over after the terminator is irrelevant.

Beavis4ever:
I confuse because I wouldn't give 2 different names to a parameter which does the same exact thing.

Neither would the author of those functions. They named the parameter terminator in both functions. For some reason the author of the Serial.readBytesUntil() documentation decided to give the parameter a different name. I would prefer to assume the programmers chose good parameter names and use those in the documentation as well.

Beavis4ever:
And the use of the word "terminator" seems a poor choice when your char array (which is now called a buffer?) ALSO has a terminating character!

All strings have terminators. That's how strings work.

An array that is used to receive data is called a buffer. There are other uses for arrays as well. For this reason it is more accurate and descriptive to call the parameter "buffer" instead of "array".

[Beavis4ever]

OK, so results.value is the name of the variable (which is a data type uint32_t) !?! I'm still super shaky on class, method, object, instance, inheritance etc. not to mention & and * ...
So I can just ignore (in this situation) irrecv.decode(&results**)** since that refers to the library's code that runs outside of my sketch, but I can treat results.value like it was another global variable that I declared in my sketch?

Also, thanks so much for the heads-up on

strlen(message) or 5

and

sizeof(message) or 6

I didn't know/remember/realize that there are the two kinds of arrays. That saved me a lot of head scratching! Already it has helped me see that I've been confusing \n (newline) and **\0 ** (null terminator)

So thanks again for the help, it's much appreciated!

P.S. (micro-rant:) "zero" and "O" have become too similar, I remember when zero was obvious because it had a "/" through it, which then became a little dot in the middle of the "0", and now today I have to judge the ovalness and roundness to tell them apart. And don't get me started on I, l, and 1 ;+P

[Beavis4ever]

@ pert Thanks for going line by line and addressing all of my confusion issues. Sometimes when I think I've grasped something, and I actually am correct about whatever the case may be, it still is immensly helpful to get an explicit "yes" in response to my "Isn't ______ the same as ______?"

All strings have terminators. That's how strings work.

Much of what I posted is meaningless/pointless because I confused **\n ** (newline) and \0 (null terminator), not realizing they are two entirely different things. I also know to be mindful of the two types of arrays where one is terminated and the other isn't (right?!)

Thanks for setting me straight!

[pert]

Beavis4ever:
P.S. (micro-rant:) "zero" and "O" have become too similar, I remember when zero was obvious because it had a "/" through it, which then became a little dot in the middle of the "0", and now today I have to judge the ovalness and roundness to tell them apart. And don't get me started on I, l, and 1 ;+P

That annoys me big time also. It's annoying that the default font of the Arduino IDE (at least on Windows, I didn't bother to check on Linux) does that. I switched it to use the Consolas font because it makes it easy to distinguish all those characters from each other.

Beavis4ever:
I also know to be mindful of the two types of arrays where one is terminated and the other isn't (right?!)

I'm not sure what you mean by two types of arrays. An array can be any type so there are many more than two. A string is a null terminated array of chars. The terminator is needed so that functions that parse a string know where it ends without you always needing to specify the length of the string. If code tries to read a string with a missing terminator, it will just continue on past the bounds of the array into some random memory until it happens to find a terminator. Unless you're a security researcher or one of their less ethical counterparts, that is not the sort of behavior you want. In other uses of arrays (an array of numbers for example), a terminator is not used and the code will need to know the size of the array (or just a specific element of the array it should read, which is inside the array's bounds).

[Robin2]

Have a look at the examples in Serial Input Basics - simple reliable ways to receive data. There is also a parse example to illustrate how to extract numbers from the received text.

The examples don't use readBytes() or readBytesUntil() as those functions block the Arduino until all the data arrives. Serial data is slow and the Arduino could be doing other useful things instead of "hanging around waiting for the postman"

...R

[Beavis4ever]

When I said, "... one is terminated and the other isn't (right?!)" I was referring to

A string is a null terminated array of chars.

as being the terminated one, and

(an array of numbers for example), a terminator is not used

as being the one which isn't.

I think I wasn't clear when I wrote

... mindful of the two types of arrays ...

I think my use of the word "type" was sloppy. If I had said, "... two kinds of arrays where one is terminated and the other isn't ..." would that have been better/clear ?

[GolamMostafa]

@OP

We need to explore a little bit of the UART Communication before studying the functional mechanics of the following instruction:

Serial.readBytesUntil(character, buffer, length);

1. Data comes to the receiver from the sender one character (8-bit ASCII Code = 1-byte) at a time.

2. When the character/data byte arrives at the receiver port, the MCU gets interrupted; it goes to ISR and saves the data byte into the very first location (Location-0) of the 64-byte wide unseen FIFO (first-in first-out) type buffer (Fig-1). This process happens beyond the visible knowledge of the user.

Figure-1: Serial FIFO Buffer of Arduino

3. Assume that the sender has sent two charcaters (AB) to the receiver. The ASCII code of A(0x41) has arrived and entered into Location-0 of Fig-1; the Location Pointer has moved down, and now it is pointing Location-1; after a while, the ASCII code of B (0x41) has has arrived and entered into Location-1 of Fig-1; the Location Pointer has moved down, and it is now pointing Location-2 into which the next character/data byte would be saved.

4. Assume that the user has taken out character A from Location-0 by executing this code: char x = Serial.read(); as a result, the character B will automatically move to Location-0 and the Location Pointer will also move up to point Location-1.

The arrival time of a character from the sender to the receiver must be less than the time taken by the user to read a data byte from the buffer and process it; otherwise, the FIFO Buffer will be quickly filled up; there will be no more buffer locations to store the newly arrived data bytes and they will be lost.

5. Assume that the sender has sent this string "imbecile\n" to the receiver. In this string \n is the newline character whose ASCII Code is 0x0A, and it will be automatically sent when the user selects the newline option in the Line ending tab of the Serial Monitor (Fig-2). The newline character is added with the string to mark the end-of-message.

Figure-2: Serial Monitor of Arduino IDE

6. We may now execute the following codes at the receiver side to bring out the characters of the message (imbecile\n) from the FIFO Buffer as they arrive one-by-one. We will save all the charcaters in a null-byte terminated character type array (char myArray[20] = "" except the newline character.

char myArray[20] = "";  //null-byte terminated character type array; it can hold 20 data bytes
int i = 0;         //array index

void setup()
{
   Serial.begin(9600);
}

void loop()
{
    byte n = Serial.available();    //check if a character has arrived and saved in FIFO buffer
    if(n !=0)
    {
         char x = Serial.read();     //a character is in the buffer; bring it out and put into x
         if(x != '\n')
         {
             myArray[i] = x;           //the character is not a newline character; so, save it in the array
             i++;                           //increment the array index
         }
         i = 0;                             //reset array index
         Serial.println(myArray);   //Serial Monitor shows: imbecile     
    }
}

7. The following lines of the sketch of Step-6 can be expressed by a single line of code which is: Serial.readBytesUntil('\n', myArray, 20);

    char x = Serial.read();     //a character is in the buffer; bring it out and put into x
    if(x != '\n')
    {
        myArray[i] = x;           //the character is not a newline character; so, save it in the array
        i++;                           //increment the array index
    }

8. The final version of the sketch of Step-6 is:

char myArray[20] = "";  //null-byte terminated character type array; it can hold 20 data bytes
int i = 0;         //array index

void setup()
{
   Serial.begin(9600);
}

void loop()
{
    byte n = Serial.available();    //check if a character has arrived and saved in FIFO buffer
    if(n !=0)
    {
         Serial.readBytesUntil('\n', myArray, 20);
    }
    Serial.println(myArray);   //Serial Monitor shows: imbecile
    chat myArray[20] = "";   //reset array index
 }

[Beavis4ever]

@ Robin2 I did in fact actually read "Serial Input Basics" which I found to be most helpful and concise, and it was the biggest influence on my current endeavor.

Foolish or not, my current project it handled by 2 ProMinis connected via the UART. The first ProMini only uses the IRremote library to decode IR signals from various remote control units, and Tx the (always 32 bit) code out the serial port. The other ProMini has nothing else at all to do when listening to the serial port, so there is no problem with anything blocking.

In such a simple set-up, where UART data is always 4 bytes long and sent one at a time, I decided not to mark the beginning and ending of transmissions. The ProMini listening to the serial port ignores the buffer when it has less than 4 bytes. When 4 bytes are waiting, it pulls them off the buffer and checks to see if they match any codes held in memory. Any matches will have a corresponding preset code or sequence of codes which gets passed to the IRremote library's send function. The end result is to drive an IR blaster which has exclusive access to the IR receiver on the devices being controlled.
After the send completes, the ProMini goes back to monitoring the serial port. If the buffer has less than 4 bytes waiting, they are discarded after a time interval slightly longer than the longest gap defined by the codes' protocols. If the buffer accumulates 4 bytes before the timeout interval, they get pulled off the buffer and this process repeats until a discard happens or the buffer becomes empty. Then, it's back to the top of the loop.
Hopefully discards won't be so frequent as to be annoying. In that case, it's back to the drawing board!

[sterretje]

The ProMini listening to the serial port ignores the buffer when it has less than 4 bytes.

What happens when you first get the last 3 bytes of a 'packet' and a little later the first byte of another 'packet'. The receiver will not be able interprete the data correctly.

You will need a mechanism to sync; e.g. if you did not receive 4 bytes within a specified time, ignore and start over.

[Beavis4ever]

@GolamMostafa Thanks for the detailed description, it's good to know what's actually happening behind the scenes.

[Beavis4ever]

@ sterretje

What happens when you first get the last 3 bytes of a 'packet' and a little later the first byte of another 'packet'.

If the first byte is in the FIFO and then the last 3 arrive, that completes the packet which is then pulled and dealt with. The first byte of another packet coming in will then be in the same "wait" situation and possibly be followed by the rest of the packet.
But I think I see what you're getting at: my "mechanism to sync" (the part where I mention bytes being "discarded") should not initialize "After the send completes," (as I wrote in post #9), but instead, should start at the top of the loop.

I haven't actually done this yet, but I'm in the middle of working on it. I'm thinking it should go like this:
The top of the loop should do a tight polling loop:

while (!Serial.available()) {
}

Pull the first 4 bytes off the FIFO and deal with them. If there are less than 4 bytes, initialize the timeout handler:

initTO = millis()     // or perhaps:  initTO = micros()

The timeout handler is a loop which ends when 1 of 2 things happen: the FIFO has 4 or more bytes (deal with them); or the elapsed time since initialization has exceeded the threshold (clear the FIFO)

Thanks for calling attention to the part where I say:

The ProMini listening to the serial port ignores the buffer when it has less than 4 bytes.

because it made me realize that the point where my code "ignores" the incomplete packet should, instead, mark the beginning of the timeout interval.