I have a device that sends stream of data. Data includes stream of bytes ranging from 0 to 255. When stream of data contains zero (0x00), the available() does not account for it. For example, if the device sends five bytes (0xAA 0x00 0x4B 0x5F 0x36), then the available() returns only 4 instead of 5. Similarly, if another stream contains (0xAA 0x00 0x4B 0x00 0x36), then the available() returns only 3 instead of 5. How can I or What can I do to have available() account for 0x00s?
Thanks
Post your sketch - most likely the problem is in there.
Here is the simple schetch:
void setup()
{
Serial.begin(9600);
Serial1.begin(9600);
}
void loop()
{
// read from port 1, send to port 0:
delay(5000);
Serial.print("Available: ");
Serial.println(Serial1.available(), HEX);
if (Serial1.available()>0)
{
int inByte = Serial1.read();
Serial.println(char(inByte), HEX);
}
}
Board?
Sorry,
Arduino Mega Pro, Arduino 1.0 (also tried on Arduino 0022)
hanlee:
I have a device that sends stream of data.
What device?
Same board - Arduino Mega Pro. I've verified the sending device with HyperTerminal and other terminal programs. They all receive 0x00s in the stream.
Same board - Arduino Mega Pro.
It's like pulling teeth to get information from you. What code is on the sender?
I can't reproduce the reported problem. Setting up a Uno as the sender with this code:
void setup ()
{
Serial.begin (9600);
delay (500);
Serial.println ();
delay (500);
byte test [] = { 0xAA, 0x00, 0x4B, 0x5F, 0x36 };
Serial.write (test, sizeof test);
}
void loop () {}
And a Mega2560 with the posted code above:
void setup()
{
Serial.begin(9600);
Serial1.begin(9600);
}
void loop()
{
// read from port 1, send to port 0:
delay(5000);
Serial.print("Available: ");
Serial.println(Serial1.available(), HEX);
if (Serial1.available()>0)
{
int inByte = Serial1.read();
Serial.println(char(inByte), HEX);
}
}
Connecting Gnd, +5V, and pin D1 (Tx) on the Uno to D19 (Rx1) on the Mega2560, and then resetting the Uno, I get this:
Available: 7
D
Available: 6
A
Available: 5
FFFFFFAA
Available: 4
0
Available: 3
4B
Available: 2
5F
Available: 1
36
Available: 0
Available: 0
The first two bytes were the 0D 0A (carriage return, linefeed) from the initial Serial.println.
The rest were as expected, including the 0x00 byte.
They came in quite slowly, because of the delay(5000) in your loop. Perhaps post your sending sketch, your wiring, and the output, like I did above?
Thanks to all posters (especially Nick Gammon) who has responded to my inquiry. I've found the source of my problem, and I have remedied the issue temporary. The problem had to do with the the baud rate. Initially, I was communicating via Serial1 with 9600 Baud. Everything worked fine. On the final version I've raised the baud rate to 115200 baud. Under this baud rate I would loose 0x00s data in the stream. After lowering the baud rate to next down - 57600 baud, the problem had gone away. At this time I am happy to have the devices working. Can anyone explain why this is so?
Thanks
The higher the baud rate, the shorter the HIGH/LOW pulse time. Perhaps your crystal is at the edge of spec, and the clock on whatever you are talking to is at the other end of the spec, and there is not sufficient overlap. Lowering the baud rate from 115200 to 57600 doubles the pulse length, allowing sufficient time for the signal the be recognized.
hanlee:
... I've found the source of my problem, and I have remedied the issue temporary. The problem had to do with the the baud rate. Initially, I was communicating via Serial1 with 9600 Baud. Everything worked fine. On the final version I've raised the baud rate to 115200 baud. Under this baud rate I would loose 0x00s data in the stream. ... Can anyone explain why this is so?
Did you have both input and output at 115200? Or were you reading at 115200 and displaying debugging at 9600?
Certainly if you were outputting debugging at 9600 I imagine that the output would "block" because you were pumping data out too fast, thus you would lose some input.
Even with both at 115200, that is 1/11520 period per byte (10 bits) which is 86.8 uS. That isn't a heap of time to be doing things. Particularly as your demo program waited 5 seconds before gathering each byte, and then wasted time displaying "Available: ".
Even without displaying "Available: " attempting to echo out what you received is likely to introduce bottlenecks. Particularly when, in response to getting the single byte 0xAA you display "FFFFFFAA" plus carriage-return plus linefeed.
Of course I've paired same baud rate. For testing, it wasn't a continuous stream of data; instead I've been sending just five and ten bytes at a time. Anyway, the communication was between two identical boards - Mega2560. If the timing was an issue - very short pulse width, then perhaps these boards shouldn't allow baud rate of 115200. Unreliable performance should be excluded from its spec.
Thanks everyone.
hanlee:
Unreliable performance should be excluded from its spec.
Once again you haven't shown the sending code, and the information about sending 5 or 10 bytes at a time has only just been released.
It isn't unreliable, per se, however you can always write a sketch that pushes the hardware beyond its limits. For example, sending "FFFFFFAA\r\n" (10 bytes) out one serial port, in response to receiving one byte from another, and hoping that the receiver does not overflow its buffer, is one such case.
I don't believe there is an issue with sending/receiving 115200 baud as such, I do that all the time. However I am aware that, as I said before, I have only 86.8 uS to do something with an incoming byte before another one arrives.
Mr. Gammon,
Your point is well taken. Thank you.
I am aware at 115200 there isn't a whole lot of time in between the bits (or bytes) of an incoming data on a serial port. My application would only sends/expects short bursts of data - 5 to 25 bytes - a few times an hour. Hence, I was some what relying on the fact that each serial port has 128 bytes of buffer. With that buffer, I would wait until enough data has been received (serial1.available()) in the buffer before processing the data. Anyway the puzzlement was that even at 115200, the data was receiving okay except 0x00 value were not being received. For example, if one device sends 0xAA 0x00 0x4B 0x5F 0x36, the receiver only receives (or receives as) 0xAA 0x4B 0x5F 0x36. Likewise, 0xAA 0x00 0x4B 0x00 0x36 would be received as 0xAA 0x4B 0x36. Only the 0x00 values were dropped consistently while anything other than 0x00 would be received fine. Of course when I reduced the speed to 57600, the phenomenon when away.
Thanks for the discussion.
Interesting. I suppose the byte 0x00 is a boundary case as far as hardware goes (a consecutive stream of zeroes in effect).
If you were to post a sketch that reliably creates this problem I could analyze with the logic analyzer.
Also your hardware setup could be a cause. As PaulS said earlier:
PaulS:
It's like pulling teeth to get information from you. What code is on the sender?
Perhaps if you were to disclose your code, and the way you are connecting things, we could comment further.
Mr. Gammon and others interested,
Thank you for your interest in attacking this issue. I wasn’t trying to be difficult of disclosing my setup; Tentatively I thought I’d be asking a quick and simple question regarding an anomaly I had ran into. For the mean time, reducing the baud rate to 57600 remedied the problem; however, if you would like to investigate this matter further, I would be greatful to provide my intension and setup which lead to the mentioned anomaly.
The project: using an arduino (MEGA2560 Pro 3.3V from Sparkfun.com) to drive a display (5” LCD with Touch screen by DWIN DMG80480S050_01WT). This display is described as “intelligent display terminal”. It’s a serial interfaced device, and has intelligent (display processing/driver) built in. The display has default serial baud rate of 115200 – hence my initial choice to work at 115200 baud. This display is an extremely convenient display as most if not all lower lever controller/driver is built into the display. The driving the display is merely a “high level” command sent by a serial connection.
Before mating with Arduino, I have tested the display with a PC, I had obtained confidence of the display, and I was ready to mate it to Arduino.
When I first interfaced Arduino to the display, I noticed I was not receiving the full response (reply) from the display. When I was expecting ten bytes of reply from the display, I would always receive short. After matching what was expected to be received from the display, it is then that I noticed all 0x00 values were not being received by Arduino. This anomaly lead me to conduct a simple and quick experiment on the Arduino.
Using two Arduino (MEGA2560 Pro 3.3V from Sparkfun.com), I had set one Arduino to send stream of data - 18 bytes (exact data of what the display would send). I had set another Arduino to receive that stream. Both Arduino was set at 115200 baud.
Here is my sketch of the receiver:
void setup()
{
Serial.begin(9600);
Serial1.begin(115200);
}
void loop()
{
// read from port 1, send to port 0:
delay(5000);
Serial.print("Available: ");
Serial.println(Serial1.available(), HEX);
if (Serial1.available()>0)
{
int inByte = Serial1.read();
Serial.println(char(inByte), HEX);
}
}
Sketch of the Sender:
byte aBuf[18] = {0xAA, 0x00, 0x4F, 0x4B, 0x5F, 0x56, 0x36, 0x2E, 0x32, 0x02, 0x07, 0x00, 0x00, 0x00, 0xCC, 0x33, 0xC3, 0x3C} ;
void setup()
{
Serial1.begin(115200);
delay(1000);
for(int i=0; i<18; i++)
Serial1.write((byte) aBuf[i]);
}
void loop()
{
}
I ran the receiver sketch on the receiver Arduino, then I ran the sender sketch on the Sender Arduino.
The result on the receiver Arduino would show (on the Serial Monitor):
Available: E
AA
4F
4B
5F
56
36
2E
32
02
07
CC
33
C3
3C
As you can see all 0x00 values were missed? Filtered? Not detected? Not received? Or whatever…
After struggling with this anomaly for while I had decided to post the issue on the forum hoping for a quick answer.
After further struggle, I tried lowering the baud rate – to 57600. The sketches ran fine; no anomalies observed.
So there you have it. My journey with 115200 baud and 0x00 value data mystery. I hope I had described the intension and my experiment clearly.
With 57600 baud (and any lower baud) the Arduino interfaces with the display perfectly.
Thanks,
Did you notice how half of your post is in italics? If you put it into [ code ] tags that wouldn't happen, and we could see your exact code. Edit your post, select the code, and hit the "#" button above the input box.
Thanks again, Mr. Gammon
You probably could tell I'm a newbie at this...
Could you try adding 0x80, 0x01, 0xff, 0xfe and 0x7f into your test data @ 115200?
Iain