Unexpected Serial.available() behaviour

You wrote that you thought your "do..while" "solution" was equivalent to the example you linked, when it should be obvious that a "do..while" will always loop at least once, unlike a "while" loop, which may not loop at all, depending on the condition tested at the top of the loop.

You still haven't posted any of your output to support your assertions.

t seems that Serial.available() is always > 0, though I know it isn't.

Apropos of nothing...

AWOL:
You wrote that you thought your "do..while" "solution" was equivalent to the example you linked, when it should be obvious that a "do..while" will always loop at least once, unlike a "while" loop, which may not loop at all, depending on the condition tested at the top of the loop.

'Ought to be obvious', ought to be easy to explain. And an explanation from such giddy heights as make such obvious errors so obvious ought to fall to Earth with more grace.

AWOL:
You still haven't posted any of your output to support your assertions.

What would you like? What form would this 'evidence' of my 'assertions' take?

I haven't made any assertions, other than to explain the disparity between my expectation and my programme's output. As I've explained, the result of the second Serial.available seems to return zero, though I know there are 35 bytes of data waiting.

though I know there are 35 bytes of data waiting.

More unsupported assertions.
How do you know this?

Are you aware that even at 3840 characters per second, serial comms are SLOW?

'Ought to be obvious', ought to be easy to explain.

And it is - do something while a condition is true. When the condition is no longer true, don't do something, BUT always do something at least once.

other than to explain the disparity between my expectation and my programme's output.

you have explained nothing, because all you have done is described a disparity, but haven't shown any output.

Go back to your reply #15, and explain carefully what it does, bearing in mind a processor running an
instruction every 62.5nanoseconds.

I'm sorry if I'm coming across as difficult, but just because you have a low post count doesn't necessarily mean you're not experienced, so it is very hard to pitch answers at the correct level.

AWOL:

'Ought to be obvious', ought to be easy to explain.

And it is [...]

Clearly it isn't so easy, or it would have taken you so many fewer attempts to make your point, your second offering being 'Does "should've gone to SpecSavers" mean anything to you?'. What this tone reflects is more obvious than the difference between a do... while and a while loop.

More unsupported assertions.
How do you know this?

I may not be sufficiently fluent in C type languages to know the difference between forms of loop structures; but I know the difference between invective and advice.

Perhaps you don't. I'm getting the distinct impression that I have exhausted this resource's potential for helping me. Some advice contra the spirit of your replies -- i.e. helpful -- in the hope that it will increase its potential...

This is not helpful:

More unsupported assertions.

This would have been just about sufficient...

How do you know this?

You seem to let a desire to browbeat get in the way of any intention to help. You would find your posts will reflect a much less combative and antagonistic tone, which seem to take pleasure in belittling others, if, rather than 'asserting' that what you are replying to is an 'unsupported assertion', you ask for more information explicitly, and take care to explain what it is you're looking for. You would also find yourself posting more efficiently; it took an entire page worth of exchanges to establish that my attempt to reduce the problem to something small enough to post here had created a new problem. You could have pointed it out in just one post. You waste your own time.

You 'assert', for example...

you have explained nothing, because all you have done is described a disparity, but haven't shown any output.

The disparity is a description of the problem: I was expecting the second instance of Serial.available() to produce an integer greater than zero. It produced a zero. That was its output.

If you want to help, but feel that this isn't sufficient, you need to explain what might make for a sufficiently accounted-for 'assertion'.

I'll admit I only skimmed the rest of this thread, but here's my try at a simple (perhaps already said) answer:

When loop() starts, it first reads the serial port, then prints out the value, then checks to see if there's anything more left in the buffer. This is unlikely because you've (1) just read one byte and (2) are reading bytes so quickly that it's unlikely that more than one byte arrives per pass through loop(). Thus, the one read clears the buffer and Serial.available() returns 0 because the buffer is no empty.

For an analogy:

Imagine that you work in an assembly line, and the person in front of you is rather slow (the serial line), while you are extremely fast (the arduino). Extremely quickly, you move your hand to where the person in front of you would put the object, even if it isn't there, try to pick up the potentially nonexistant object, and pass it to the next person (This is what it's like doing a Serial.read() before Serial.available()). After you've handed it off to the next person, you look to see if there's anything waiting for you to collect. Obviously, since you've just handed off one object, another one won't already be waiting.

You missed nothing in the skim-reading, WizenedEE! Putting a couple of delay() statements into my code has worked. Everything now works.

Sixteen hours of tearing my hair out -- an entire Saturday I thought I would never get back -- has come to a fruitful end. My code will work, and I've a priceless lesson in serial comms!

I am immensely grateful.

So is the serial buffer not fast enough to cope with the programme as it was? Just interested.

beepee:
do{
incomingByte = Serial.read();
// say what you got:
Serial.write(incomingByte);
} while (Serial.available() > 0);

That code does not check available until the end of the do-while, but reads before then.

If you think that learning to salt delays in is how to make serial code work then you learned a very bad lesson!

A big lesson that can learn is to take a break, drop your expectations, come back and look again.
Make that beotch work without delays and you will really have learned something about yourself.

You wrote this?

void setup() {
  Serial.begin(38400);
}

void loop() {
  if (Serial.available() > 0) {
    int inByte = Serial.read();
    if (inByte == 36) {
      // "$" - then get what processing has
      listening();
    }
  }
}

void listening() {
  int incomingByte;
  while(Serial.available() >0) {
   incomingByte = Serial.read();
    Serial.write(incomingByte);
  }
}

then you should be able to know what's wrong with the first code you posted.

BTW, the buffer is as fast as the 328P in the Arduino. You're grasping at excuses when the answer has been right in front of you the whole time.

GoForSmoke:
A big lesson that can learn is to take a break, drop your expectations, come back and look again.
Make that beotch work without delays and you will really have learned something about yourself.

Nobody, ever, has ever learned anything by dampening their expectations. Expectations are a necessary condition of learning.

So how can it be made to work without delays? I'm interested. But having read hundreds of pages about Serial.{xxxxx} all day, I don't think anything was as useful as WizenedEE's answer.

I'm sure you don't mean to turn the discovery into something joyless... but...

... then you should be able to know what's wrong with the first code you posted....

Yes. And it was discussed. In attempting to isolate the problem I was having so that I could ask for help here, I created another problem, by not thinking through the do..while loop, unrelated to my actual problem.

BTW, the buffer is as fast as the 328P in the Arduino. You're grasping at excuses when the answer has been right in front of you the whole time.

It didn't work without the delays. Now it does. What's the excuse?

Well to continue with the assembly line analogy...
There's two ways to fix the problem. One is to close your eyes for 10 seconds after every check. Hopefully, after ten seconds, the person in front of you is done, but you don't bother checking. That's what calling delay does. The other is to check for the item before trying to pick it up. With the second way, you can also do other things, like blinking leds and feeding your cat or whatever.

Thus, the common way to read serial is like so:

void loop() {
  if (Serial.available()) {
    char c = Serial.read();
    // blah blah blah
  }
  // Do other things! Just not delay
}

Also, it's not that the serial buffer is slow, but the actual writing the line high and low for the serial communication is rather slow.

WizenedEE:
Thus, the common way to read serial is like so:

void loop() {

if (Serial.available()) {
    char c = Serial.read();
    // blah blah blah
  }
  // Do other things! Just not delay
}

I was trying the if (Serial.available()) { approach, but without doing the other stuff -- there's nothing to do, since I'm just polling button presses on another device.

Can I understand from what you've said that what I should be doing is returning to the Serial buffer, every now and then, so to speak, rather than let the whole programme flow sit and wait for the operation to finish? Do something... check the serial buffer... do something else... check the buffer...? Maybe this would mean only getting one byte at a time, and then process the N bytes from the serial port when the entire bunch has been received?

void setup() {
  Serial.begin(38400);
}

void loop() {
  if (Serial.available() > 0) {
    int inByte = Serial.read();
    if (inByte == 36) {
      // "$" - then get what processing has
      listening();
    }
  }
}

void listening() {
  int incomingByte;
  while(Serial.available() >0) {
   incomingByte = Serial.read();
    Serial.write(incomingByte);
  }
}

In the above code you posted the uncontrolled arduino looping will be checking the serial buffer much faster than it will be filled. When the inbyte is emptied from the buffer and evaluated and the deired byte is found, the result is the immediate actuation of the function. the buffer was just emptied and another byte has not arrived yet, so there is nothing available in the buffer when checked, so the function is probably exited. I think of a "while" as a combination of an "if" and a "do" loop. Delays work because they slow down the arduino loop speed to the point that the serial buffer is filled faster than it is checked.

beepee:

GoForSmoke:
A big lesson that can learn is to take a break, drop your expectations, come back and look again.
Make that beotch work without delays and you will really have learned something about yourself.

Nobody, ever, has ever learned anything by dampening their expectations. Expectations are a necessary condition of learning.

In your case, the expectations are blinding you to the bleeding frikkin obvious.

So how can it be made to work without delays? I'm interested. But having read hundreds of pages about Serial.{xxxxx} all day, I don't think anything was as useful as WizenedEE's answer.

I'm sure you don't mean to turn the discovery into something joyless... but...

... then you should be able to know what's wrong with the first code you posted....

Yes. And it was discussed. In attempting to isolate the problem I was having so that I could ask for help here, I created another problem, by not thinking through the do..while loop, unrelated to my actual problem.

discussed != solved for you

BTW, the buffer is as fast as the 328P in the Arduino. You're grasping at excuses when the answer has been right in front of you the whole time.

It didn't work without the delays. Now it does. What's the excuse?

Oh geez.... this is like telling someone they've got their shoes on the wrong feet and having them tell you "they're my shoes and they're my feet".

What's the difference between

 do{
     incomingByte = Serial.read();
     // say what you got:
     Serial.write(incomingByte);
   } while (Serial.available() > 0);

and

 while (Serial.available() > 0) {
     incomingByte = Serial.read();
     // say what you got:
     Serial.write(incomingByte);
   };

Hmmmm?

But emptying the buffer doesn't necessarily mean that the complete message has been sent. So that's your next bug that AWOL dropped more than one clue about.

Some people enjoy helping people who are new to the things they enjoy, and sharing the benefit of their greater knowledge and competences.

Others seem motivated to use the deficit as leverage over others. But they never seem very happy. QED.

Perhaps you need to specifically restate what you are trying to accomplish in the end and start fresh.

beepee:

   do{ 

incomingByte = Serial.read();
    // say what you got:
    Serial.write(incomingByte);
  } while (Serial.available() > 0);
}

This code is obviously wrong. By that I mean that it is obvious to me that it is wrong, and it should be obvious to you. A 'do/while' loop always executes at least once; the condition determines whether the loop will be repeated. This means you will read (and process) a character on the serial port before checking that there is a character available. This is not a fault in the behaviour of Serial.available(); it is behaving correctly, but you are not calling it at the correct time.

I suggest that you replace the do/while loop with a while loop to correct this problem. I note that I am not the first to suggest this. The fact that the problem is obvious, and this has been pointed out repeatedly and then apparently not fixed, may explain why the tone of replies on this thread has been less constructive than you may have wished.

zoomkat:
Perhaps you need to specifically restate what you are trying to accomplish in the end and start fresh.

The issue was resolved. The issue was timing. WizenedEE spotted the likely problem I was having, in spite of my 'obvious' programming errors, and his advice worked, like a charm, in one hit. No invective. No insults. No browbeating. No patronising. No giggling about errors being 'obvious'. He went straight to the point.

PeterH:
This code is obviously wrong. By that I mean that it is obvious to me that it is wrong, and it should be obvious to you. A 'do/while' loop always executes at least once; the condition determines whether the loop will be repeated. This means you will read (and process) a character on the serial port before checking that there is a character available. This is not a fault in the behaviour of Serial.available(); it is behaving correctly, but you are not calling it at the correct time.

I suggest that you replace the do/while loop with a while loop to correct this problem. I note that I am not the first to suggest this. The fact that the problem is obvious, and this has been pointed out repeatedly and then apparently not fixed, may explain why the tone of replies on this thread has been less constructive than you may have wished.

The problem, as has been established, has nothing to do with the programming as such, but is a timing problem. The use of a while loop did not fix the problem, as I explained.

Errors in programming are not always obvious, especially where people are new to programming. Even if the obvious programming error was the problem -- and it wasn't -- then pointing and mocking about how it is so obvious -- HA HA HA! IT'S OBVIOUS/"should have gone to Specsavers" -- isn't going to make it any the more obvious. It's just going to make newcomers think that much less of you, this forum, and of the possibilities created by enterprises such as Arduino, and less inclined to post here. There are millions of forums across the internet where one can have flame wars, or be insulted.

Even if errors are obvious. It often takes more than one pair of eyes to spot them. This is true of all disciplines, at all levels. The point of open source development must surely be that possibilities are extended by the social aspect. When you introduce an anti-social tendency, so the possibilities of open source development diminish.

beepee:
The problem, as has been established, has nothing to do with the programming as such, but is a timing problem. The use of a while loop did not fix the problem, as I explained.

I have not seen any code posted in this thread which exhibited a timing problem. In your original post, you used a do/while loop instead of a while loop and this will cause exactly the problem you described. Are you saying that you have corrected that and still have a problem? If so, I missed that post. If you think the original problem was a timing problem, I think you are mistaken.

PeterH:

beepee:
The problem, as has been established, has nothing to do with the programming as such, but is a timing problem. The use of a while loop did not fix the problem, as I explained.

I have not seen any code posted in this thread which exhibited a timing problem. In your original post, you used a do/while loop instead of a while loop and this will cause exactly the problem you described. Are you saying that you have corrected that and still have a problem? If so, I missed that post. If you think the original problem was a timing problem, I think you are mistaken.

This is extraordinary. Why is there this forum, which purports to be a place to get help, a place where people seem only impatient to misunderstand?

Yes, the original code posted was an error. It was produced in an attempt to isolate the problem in a sufficiently small piece of code that would fit into a message here, but was one of countless dozens of attempts to do the same thing.

Because the first replies to the problem didn't directly address the coding problem, but instead sniggered about the problem, I didn't spot the problem until the top of page two. Have a look. The problem persisted.

Now, as you observe, the problem is one which might not necessarily emerge from a software-only perspective. But as you will know, the Arduino is not simply a software platform. and given that the problem relates to the use of the serial port, it ought to be obvious (ha ha ha) that there may be a hardware issue involved.

I hope everyone is learning something. Maybe it could be expanded in a new section: "The Psychology (or pathology) of Programming".
beepee: it's not funny anymore. I am sure there are other forums more suited to hosting your ego. Your post was solved long ago.
I am sure everyone would love to here your final post: "I've solved it by myself. Thank you all for the help"