Go Down

Topic: Unexpected behaviour of Serial.available after Serial.read (Read 902 times) previous topic - next topic

lekok

So I was experimenting with Serial to read large chunks of bytes and I came across this behavior with Serial I wasn't expecting.

Referring to the following code:

Code: [Select]

void setup() {
  Serial.begin(115200);
}

void loop() {
  if (Serial.available())
  {
    char ch = Serial.read();
    while (Serial.available())
    {
        char p = Serial.read();
        Serial.println(p);
    }
  }
}



By giving an input of say "abc", I was expecting the Serial monitor print "a", "b", "c" on each line. However, nothing were printed. Strangely, if I were to comment out the following line:
Quote
char ch = Serial.read();
It works as expected!

Looking at the way Serial.read() is implemented, it seems like the tail pointer gets incremented by 1. However, this shouldn't return false from Serial.available() in this case since reading "a" will leave with "b" and "c". I think I am missing something about understanding how Serial buffer works. Does anyone knows what is going on that explains this behavior?

TheMemberFormerlyKnownAsAWOL

Maybe you should re-read what available () and read () return.
Please don't PM technical questions - post them on the forum, then everyone benefits/suffers equally

Juraj

the CPU is much faster then Serial
 
nothing is available.
'a' arrives,
1 byte is available and is read.
nothing more is available.
'b' arrives,
1 byte is available, and is read.
nothing more is available.
'c' arrives,
1 byte is available, and is read.
nothing more is available.

without the first read():

nothing is available.
'a' arrives, 1 byte is available
still is 1 byte available.
'a' is read and printed
nothing more is available.
'b' arrives, 1 byte is available
still is 1 byte available.
'b' is read and printed
nothing more is available.
'c' arrives, 1 byte is available
still is 1 byte available.
'c' is read and printed
nothing more is available.

lekok

the CPU is much faster then Serial
 
nothing is available.
'a' arrives,
1 byte is available and is read.
nothing more is available.
'b' arrives,
1 byte is available, and is read.
nothing more is available.
'c' arrives,
1 byte is available, and is read.
nothing more is available.

without the first read():

nothing is available.
'a' arrives, 1 byte is available
still is 1 byte available.
'a' is read and printed
nothing more is available.
'b' arrives, 1 byte is available
still is 1 byte available.
'b' is read and printed
nothing more is available.
'c' arrives, 1 byte is available
still is 1 byte available.
'c' is read and printed
nothing more is available.

Ok that makes so much sense now. Thank you for pointing that out!

Robin2

Have a look at the examples in Serial Input Basics - simple reliable non-blocking ways to receive data. There is also a parse example to illustrate how to extract numbers from the received text.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

lekok

Have a look at the examples in Serial Input Basics - simple reliable non-blocking ways to receive data. There is also a parse example to illustrate how to extract numbers from the received text.

...R
Thanks for linking that! I should have pointed out earlier that I was using those examples but tweaking them to better suit my project needs and thus came across this behavior I wasn't expecting.

Robin2

using those examples but tweaking them to better suit my project needs
I don't see any evidence of the functions from my tutorial in the code in your Original Post

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

lekok

I don't see any evidence of the functions from my tutorial in the code in your Original Post

...R
I have extracted out the portions I thought it was irrelevant to original question. But for completeness, the following is what I had:

Code: [Select]


void setup() {
  Serial.begin(115200);
}

void loop() {
  if (Serial.available())
  {
    byte bite = Serial.read();

    if (bite == BEGIN_CHAR)
    {
      current_len = 0;
      clear(); // Zeroes the buffer array.
      while (Serial.available())
      {
         if (current_len >= SERIAL_READ_SIZE) // Maximum allowable size of one batch of data.
         {
           break;
         }
         byte next = Serial.read();
         buffer[current_len++] = (uint8_t)next;
      }

      // Process the buffer further down
    }
  }
}



Again, I have extracted out other parts, the entire sketch is pretty large with multiple components.

What I'm trying to achieve with the above is to look for the start marker and read a specific number of bytes and ignore any additional bytes until the next start marker is seen.

Basically what I am trying to achieve for my project:
  • Read large chunks of data from Serial.
  • Transmit these chunks using this Grove LoRa radio through a mesh network using the RadioHead library.

I have decent progress with 2. but 1. is where I am at.

Aside from that, I have also came across another one of your post where you shared a code that reads bytes that can consist of the start/end marker value, using a special marker.

However, your example assumes that the sender adheres to a certain rules when sending those bytes. That is marking payload bytes that is start/end marker by putting the special marker in front. However, in our use case, that may not be possible because the data is large (could be images) and has to be quickly processed on the sender's side. So I am looking for alternatives that is simpler and requires less computation.

So, I am wondering if there is any technique that can account for the following specifications:
  • The payload bytes can vary in sizes.
  • The payload data can be start or end markers and there could be multiple of them.


Example 3 in the link you gave will work well except for the concern of how would I know when the payload bytes has ended.

Thank you for your time so far.

TheMemberFormerlyKnownAsAWOL

You have a while loop inside the "if (Serial.available ()...," block.

Unless "clear()" takes a significant amount of time, that while will never even start.
Please don't PM technical questions - post them on the forum, then everyone benefits/suffers equally

lekok

You have a while loop inside the "if (Serial.available ()...," block.

Unless "clear()" takes a significant amount of time, that while will never even start.
Right, it brings back to what Juraj mentioned earlier right? If the next byte has yet to arrive at the buffer, the MCU would have reached the while loop and Serial.available() would return 0 since it so much faster than Serial.

In that case, would the general rule of thumb be, to not read() any before I do a available()?

TheMemberFormerlyKnownAsAWOL

The rule is never do a read unless you've checked availability.

Use a state machine.
Please don't PM technical questions - post them on the forum, then everyone benefits/suffers equally

lekok

The rule is never do a read unless you've checked availability.

Use a state machine.
Ok, I will keep this in mind. Thanks for these information, I very much appreciate them. :)

Robin2

What I'm trying to achieve with the above is to look for the start marker and read a specific number of bytes and ignore any additional bytes until the next start marker is seen.
This came up in another Thread recently and this is what I suggested.

...R

PS ... its much easier to help if you stay as close as possible to the original version of a piece of code - so the changes you have made are obvious within a body of familiar code
Two or three hours spent thinking and reading documentation solves most programming problems.

Power_Broker

You could also use a proven and efficient serial transfer library to send standardized byte packets between Arduinos. The library uses start/end markers plust cyclic redundancy checking, consistent overhead byte stuffing, and dynamic payload length handling. I use it for several of my own projects and should work perfectly for this one, too.

You can install it using the Arduino IDE's Libraries Manager (search "serialTransfer")


Example code:

TX Arduino:
Code: [Select]
#include "SerialTransfer.h"

SerialTransfer myTransfer;

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

void loop()
{
  float myFloat = 100.5;
 
  myTransfer.txBuff[0] = 'h';
  myTransfer.txBuff[1] = 'i';
  myTransfer.txBuff[2] = '\n';
  myTransfer.txFloat(myFloat, 3); //insert the float "myFloat" at index 3 since "hi\n" already takes up indicies 0-2
 
  myTransfer.sendData(7); //3 bytes for "hi\n" plus 4 bytes for the float "myFloat"
  delay(100);
}


RX Arduino:
Code: [Select]
#include "SerialTransfer.h"

SerialTransfer myTransfer;

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

void loop()
{
  float myFloat;
 
  if(myTransfer.available())
  {
    /////////////////////////////////////////////////////////////// Handle Entire Packet
    Serial.println("New Data");
    for(byte i = 0; i < myTransfer.bytesRead; i++)
      Serial.write(myTransfer.rxBuff[i]);
    Serial.println();

    /////////////////////////////////////////////////////////////// Parse Out Float From Packet
    myTransfer.rxFloat(myFloat, 3);
    Serial.print("Received float: "); Serial.println(myFloat);
    Serial.println();
  }
  else if(myTransfer.status < 0)
  {
    Serial.print("ERROR: ");
    Serial.println(myTransfer.status);
  }
}
"The desire that guides me in all I do is the desire to harness the forces of nature to the service of mankind."
   - Nikola Tesla

lekok

This came up in another Thread recently and this is what I suggested.

...R
Noted this.
What if I want to allow varied payload size? I thinking if the sender sends something like: <ABC>><123>
Where '<' and '>' are start/end markers respectively. I wouldn't know if the data ended after the 'C' or '3' since the markers themselves can appear in the data and there is no fixed length to read.

As for the special marker method you posted, I'm concerned with having too much processing done on the sender and receiver side.
I'm thinking of using headers to inform the receiver how many bytes to receive but this will incur some overhead. I guess this is the trade off I have to compromise for.


Quote
PS ... its much easier to help if you stay as close as possible to the original version of a piece of code - so the changes you have made are obvious within a body of familiar code
That makes sense, I will keep this in mind when I post in future.



You could also use a proven and efficient serial transfer library to send standardized byte packets between Arduinos. The library uses start/end markers plust cyclic redundancy checking, consistent overhead byte stuffing, and dynamic payload length handling. I use it for several of my own projects and should work perfectly for this one, too.

You can install it using the Arduino IDE's Libraries Manager (search "serialTransfer")


Example code:

TX Arduino:
Code: [Select]
#include "SerialTransfer.h"

SerialTransfer myTransfer;

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

void loop()
{
  float myFloat = 100.5;
  
  myTransfer.txBuff[0] = 'h';
  myTransfer.txBuff[1] = 'i';
  myTransfer.txBuff[2] = '\n';
  myTransfer.txFloat(myFloat, 3); //insert the float "myFloat" at index 3 since "hi\n" already takes up indicies 0-2
  
  myTransfer.sendData(7); //3 bytes for "hi\n" plus 4 bytes for the float "myFloat"
  delay(100);
}


RX Arduino:
Code: [Select]
#include "SerialTransfer.h"

SerialTransfer myTransfer;

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

void loop()
{
  float myFloat;
  
  if(myTransfer.available())
  {
    /////////////////////////////////////////////////////////////// Handle Entire Packet
    Serial.println("New Data");
    for(byte i = 0; i < myTransfer.bytesRead; i++)
      Serial.write(myTransfer.rxBuff[i]);
    Serial.println();

    /////////////////////////////////////////////////////////////// Parse Out Float From Packet
    myTransfer.rxFloat(myFloat, 3);
    Serial.print("Received float: "); Serial.println(myFloat);
    Serial.println();
  }
  else if(myTransfer.status < 0)
  {
    Serial.print("ERROR: ");
    Serial.println(myTransfer.status);
  }
}

Thanks for sharing that! Although the sender in my use case may not be using the Arduino framework, I will take a look at that library.

Go Up