Serial.available glitches.

Using Arduino Programming Questions
#1

I have a lot of experience with Serial Communication and I can work with it well but there is one thing that bothers me.

void setup() { Serial.begin(115200); }

void loop() { String x = "";

if (Serial.available()) {

while(Serial.available() > 0) { char y = Serial.read(); x += y; //delay(1); } Serial.println(x); } }

the idea is: I have an empty string x = "" and i add characters to it and I print this string. with the delay(1) active it works fine. but when I disable the delay it tends to jump out of the while loop when there are still bytes to be read from the buffer. I tested Serial.available and it does return the correct number of bytes stored in the buffer.

when I type in hello world I get back:

he ll o wo rl d

this means that it does jump out of the while loop, in other words, it ignores while(Serial.available() > 0) even without the > 0. it keeps jumping out.

The only thing I can think of is that Serial.available briefly returns a 0 after a byte is read.

does anybody knows this for sure? I am trying to get a flawless serial communication without having to call a delay.

#2

Serial comms are sloooooow.

#3

115200 baud means about 10000 chars /sec => 10 chars /millisec => 1 char /100 microsec

The loop (test for availability) is way faster and by adding the delay(1) you wait about 10 chars worth of time per character

you could try this variation which only adds the millisecond delay if there are no chars available.

void setup()
{
  Serial.begin(115200);
}

void loop()
{
  String x = "";

  while (Serial.available() > 0)
  {
    char y = Serial.read();
    x += y;
    if (Serial.available() == 0) delay(1); //<<<<<<<<<<<< changed line
  }
  Serial.println(x);
}

You can add timing code to see the difference (excluding the print(x) )

#4

robtillaart:
115200 baud means about 10000 chars /sec => 10 chars /millisec => 1 char /100 microsec

The loop (test for availability) is way faster and by adding the delay(1) you wait about 10 chars worth of time per character

you could try this variation which only adds the millisecond delay if there are no chars available.

void setup()

{
  Serial.begin(115200);
}

void loop()
{
  String x = “”;

while (Serial.available() > 0)
  {
    char y = Serial.read();
    x += y;
    if (Serial.available() == 0) delay(1); //<<<<<<<<<<<< changed line
  }
  Serial.println(x);
}




You can add timing code to see the difference (excluding the print(x) )

but but 115200 BitPerSecond / 8 = 14400 byte/second not 10000 :stuck_out_tongue:

#5

I have a lot of experience with Serial Communication and I can work with it well but there is one thing that bothers me.

but but 115200 BitPerSecond / 8 = 14400 byte/second not 10000 :P

If you are experienced with Serial Communication you should recall that Serial(*) adds a start and a stop bit for synchronization and that there might be up to one bit delay between these 10 bit packages that hold a byte. So you cannot simply divide the baud rate by 8 to get the byte rate.

10 bits per package makes 115200 bits between 10000 and 11500 bytes per second which - that's true - I rounded to 10KB to make the math easier, and probably closer to the effective speed

*) This is true for the Serial class and SoftwareSerial with default config but not for SPI, I2C, shiftin and shiftout which are also serial in nature

Did you test the timing of the alternative code?

#6

I get it now, I see the problem.

when I send "hello world" I send 11 bytes. the program is only doing this line: if (Serial.available() > 0) {

when it receives the string, it immediatly jumps to the while loop adds the character to the string and checks again here: while(Serial.available() > 0) { but at this point the 2nd byte hasn't reached the buffer yet, so it jumps out of the while loop. the 115200 baudrate is just fast enough to read 2 bytes in row, but it is to slow to read the 3rd byte on time before it jumps out of the while loop.

In order words, arduino runs (alot) faster then it can receive bytes. duh...

I am going for an attempt to calculate the delayMicroseconds time.

But for now I will solve it by putting the delay(1) between

If(serial.available()) {

delay(1);

while(serial..... etc

so in the 1ms there is enough time to receive the entire string before going in the while loop :)

#7

Ok i can receive 12 bytes (sometimes 13) per millisecond with the following lines of code

void setup() { Serial.begin(115200); }

void loop() { String x = "";

if (Serial.available()) {

delay(1); // here the serial buffer gets filled Serial.println(Serial.available()); // print out the number of availble bytes, this does not exceed 13!

while(Serial.available()) { char y = Serial.read(); x += y; } Serial.println(x); }

}

#8

if (Serial.available()) { <<<<<<< but how many are allready in here? at least 1

delay(1); // here the serial buffer gets filled
Serial.println(Serial.available()); // print out the number of availble bytes, this does not exceed 13!

try this

void setup()
{
  Serial.begin(115200);
}

void loop()
{
  String x = "";

  if (Serial.available()) {
    uint32_t start = micros();
    int a = Serial.available();
    delay(1);
    int b = Serial.available();
    uint32_t stop = micros();

    Serial.print("byte:\t");
    Serial.println(b - a);
    Serial.print("usec:\t");
    Serial.println(stop - start);
    Serial.print("Bps:\t");
    Serial.println( float(b - a) * 1000 / (stop - start), 2);


    while (Serial.available()) {
      char y = Serial.read();
      x += y;
    }
    Serial.println(x);
  }
}

my output IDE158, win7

byte: 12
usec: 1036
Bps: 11.58

That is very close to the max 11.52 Bps expected. Note that the timing has an accuracy of about 4 micros ~0.4% so the measurement is within valid limits.

lesson learned: the Arduino has except for the start and stop bit in this test almost no overhead in Serial.

#9

bask185: ...

But for now I will solve it by putting the delay(1) between

If(serial.available()) {

delay(1);

while(serial..... etc

so in the 1ms there is enough time to receive the entire string before going in the while loop :)

That is a dangerous assumption as next time your string might just exceed that limit.

#10

bask185: but when I disable the delay it tends to jump out of the while loop when there are still bytes to be read from the buffer

This is not a correct analysis. The reason it jumps out of the WHILE is because there are no bytes in the buffer because it is reading them faster than they arrive.

Don't use time to manage serial input. Look at this Thread serial input basics.

...R

#11

n order words, arduino runs (alot) faster then it can receive bytes. duh...

Probably better stated that the arduino runs a lot faster than the input buffer can be filled.

#12

Robin2: This is not a correct analysis. The reason it jumps out of the WHILE is because there are no bytes in the buffer because it is reading them faster than they arrive.

Don't use time to manage serial input. Look at this Thread serial input basics.

...R

if you would have read everything you also would have come across the following:

I get it now, I see the problem.

when I send "hello world" I send 11 bytes. the program is only doing this line: if (Serial.available() > 0) {

when it receives the string, it immediatly jumps to the while loop adds the character to the string and checks again here: while(Serial.available() > 0) { but at this point the 2nd byte hasn't reached the buffer yet, so it jumps out of the while loop. the 115200 baudrate is just fast enough to read 2 bytes in row, but it is to slow to read the 3rd byte on time before it jumps out of the while loop.

In order words, arduino runs (alot) faster then it can receive bytes. duh...

I am going for an attempt to calculate the delayMicroseconds time.

But for now I will solve it by putting the delay(1) between

If(serial.available()) {

delay(1);

while(serial..... etc

so in the 1ms there is enough time to receive the entire string before going in the while loop :)

I figured it out myself already

#13

robtillaart: That is a dangerous assumption as next time your string might just exceed that limit.

I just dont type strings longer then 12 chars :P

but I know I put a delay of 50 microseconds in the while loop.

void loop() { String x = "";

if (Serial.available()) {

while(Serial.available()) { char y = Serial.read(); x += y; delayMicroseconds(50); } Serial.println(x); } }

it is interesting that it starts bugging on me, when I put a delay of 48 us. But with 12 bytes/1036us I thought I would need a delay of arround the 86us 1Byte/86us yet 50us works flawless.

So apparerently I need a delay :(

#14

In your particular restricted test you might need a delay, but in general, you'd simply wait for the user to signal that the data was complete with, for example, a carriage return.

#15

bask185: I just dont type strings longer then 12 chars :P

Have you heard the expression, "re-inventing the wheel, badly." That is what you are doing here.

Asynchronous

adjective. not occurring at the same time. Computers, Telecommunications. of or relating to operation without the use of fixed time intervals (opposed to synchronous).

In other words, timing should not be the way to solve an asynchronous problem.

it is interesting that it starts bugging on me, when I put a delay of 48 us. But with 12 bytes/1036us I thought I would need a delay of arround the 86us 1Byte/86us yet 50us works flawless.

In telecomms, they call it a co-incidence.

Look at every other asynchronous serial protocol. They all provide some method for signalling message transmission is complete. Often the EndOfLine character, 'in band.' Sometimes hardware flow control, 'out of band.' Your protocol is using time to 'infer' the end of message, which is a poor and fragile, solution; because you are using asynchronous comms.

So apparerently I need a delay :(

No you don't. You need to make your protocol robust, which it isn't.

I am feeling sorry for Robin2 now. I guess some people just need to make their own mistakes before they start to listen.

#16

bask185: if you would have read everything you also would have come across the following:

I think the comments that have arrived after that Reply provide some justification for the code examples I linked to.

The real purpose behind my comment was to draw attention to the fact that your analysis seemed to assume the Arduino was making a mistake - rather than that there was an error in your program. Errors in microprocessors are not impossible but they are vanishingly rare - in complete contrast to the risk of errors by me and you. If you find yourself wondering if the computer is wrong tell yourself "stop wasting time".

...R

#17

MattS-UK: Have you heard the expression, "re-inventing the wheel, badly." That is what you are doing here.

Asynchronous

adjective. not occurring at the same time. Computers, Telecommunications. of or relating to operation without the use of fixed time intervals (opposed to synchronous).

In other words, timing should not be the way to solve an asynchronous problem. In telecomms, they call it a co-incidence.

Look at every other asynchronous serial protocol. They all provide some method for signalling message transmission is complete. Often the EndOfLine character, 'in band.' Sometimes hardware flow control, 'out of band.' Your protocol is using time to 'infer' the end of message, which is a poor and fragile, solution; because you are using asynchronous comms. No you don't. You need to make your protocol robust, which it isn't.

I am feeling sorry for Robin2 now. I guess some people just need to make their own mistakes before they start to listen.

that there is no clock line present does not mean you cant time. you said "in other words, timing should not be the way to solve an asynchronous problem." well there is the start bit, the byte and the stop bit, between the start and stop bit there is only timing. How do you think arduino can distinct two '0' from one '0'?? what do you think will happen if you let 2 arduinos communicate with eachother when one has Serial.begin(9600) and the other has Serial.begin(115200) in the setup. it wont work because they are not timing right.

and this "You need to make your protocol robust, which it isn't." WHY THE F*CK YOU THINK I POST THIS ON A FORUM. If you dont have anything usefull to post, then don't post at all. You only bitch how wrong I am and you haven't given a single "solution" or answer

I feel sorry for you because you are to stupid to understand that you are an idiot and that you are not nearly half as smart as you think you are.

so again, if you have nothing useful to add to my topic, will you please get the f*ck out of here.

#18

Robin2:
I think the comments that have arrived after that Reply provide some justification for the code examples I linked to.

The real purpose behind my comment was to draw attention to the fact that your analysis seemed to assume the Arduino was making a mistake - rather than that there was an error in your program. Errors in microprocessors are not impossible but they are vanishingly rare - in complete contrast to the risk of errors by me and you. If you find yourself wondering if the computer is wrong tell yourself “stop wasting time”.

…R

and do you even know what the serial buffer of arduino does? because it sounds like you don’t. every byte I send with the pc gets stored in the buffer and I can read from this buffer whenever I want. even when the buffer is full with 64 bytes I can read them out as fast the arduino runs with the 16MHz clock. It is the communication between arduino and pc which was lagging behind. my simple delay solution does not waste more then 2 microseconds. I find this fairly acceptable

#19

bask185:
I feel sorry for you because you are to stupid to understand that you are an idiot and that you are not nearly half as smart as you think you are.

First rule of learning is to listen rather than talk.

Insulting people who are trying to help (even if you don’t like the message) makes absolutely no sense and has no place in this Forum.

bask185:
and do you even know what the serial buffer of arduino does? … I find this fairly acceptable

I do … which is why I don’t

…R

#20

Robin2: First rule of learning is to listen rather than talk.

Insulting people who are trying to help (even if you don't like the message) makes absolutely no sense and has no place in this Forum.

I do ... which is why I don't

...R

I could not agree with you more. But matt5 is not helping at all, he only complains and he is not helping. That is why I may have slightly insulted him and suggested he should get his ass of me topic.

I am not insulting you because you are a good guy with good intentions, you might have posted something I already did which made have made me bang my head on the table (to hard obviously), but I could feel the love in the post. so thank you :)