Hi all,
First post, long-time lurker.
I am tracking down some strange issues I came across when developing a serial protocol for communicating between an Arduino board (Uno in my case) and a Linux host (actually macOS but using the C Posix library that is the default on Linux and unix like systems).
I have added code examples and output to make this post easy to follow and to allow others to reproduce.
I have isolated the the problem into a very short sketch and a very short C program for debugging purposes.
- sketch on Arduino: The sketch continuously sends the same byte over Serial to the host as fast as it can in an endless loop. The byte remains the same during the execution of the sketch. However, the next time the sketch runs, the next byte is used. For example, during the first run the byte 'a' is continuously send, during the second run the byte 'b' is continuously send, during the third run the byte 'c' and so on. This is done by storing the byte in eeprom and updating it on every run.
- program on host: The program on the host opens the serial port (hence resetting the Arduino), sets the correct serial port options (8N1 raw mode), performs 4 reads dumping the data in the console, and closes the serial port.
One would (maybe naively) expect dat during the first run of the program on the host the byte 'a' is observed, during the second run the byte 'b', during the third run 'c', and so on. However, this is not the case. Although this behavior is frequently observed, I am also observing
1/ bytes in the current run that belong to the previous run, and
2/ bytes in the current run that have never been produced by the board.
Programs and output:
I first reset the sent byte in the eprom using the sketch below:
#include <EEPROM.h>
void setup() {
EEPROM.write(0, 'a');
}
void loop() {}
This is the sketch I use to send the bytes. It reads the byte to send from eeprom, updates it for the next run of the sketch, and then sends the byte continuously and as fast as it can checking Serial.availableForWrite().
#include <EEPROM.h>
byte b;
void setup() {
Serial.begin(9600);
b = EEPROM.read(0);
EEPROM.write(0, b + 1);
}
void loop() {
while (Serial.availableForWrite() > 0) {
Serial.write(b);
}
}
This is the C program that is used on the host:
// to compile:
// cc -Wall serial.c -o serial
//
// to run:
// ./serial /dev/cu.usbmodem1421 (replace /dev/cu.usbmodem1421 with your device)
#include <assert.h>
#include <stdio.h>
#include <ctype.h>
#include <fcntl.h>
#include <termios.h>
#include <unistd.h>
// hexdump
// used to print a hex and ascii dump of the read data, code not relevant in this context
void hexdump(const void * buffer, size_t size)
{
for (size_t o = 0; o < size; o += 16) {
printf ("%0*lx ", 4, o);
for (size_t i = o; i < o + 16; ++i) {
if (i < size) {
unsigned char uc = ((unsigned char *) buffer)[i];
printf(" %02x", uc);
}
else printf(" ");
}
printf(" ");
for (size_t i = o; i < o + 16; ++i) {
if (i < size) {
char c = ((char *) buffer)[i];
printf("%c", isprint(c) ? c : '?');
}
else printf(" ");
}
printf("\n");
}
}
int main(int argc, char* argv[])
{
assert(argc == 2);
int rv;
// open
int fd = open(argv[1], O_RDWR | O_NOCTTY);
assert(fd != -1);
//usleep(1000); // uncommenting "fixes" the problem, see below
struct termios termios;
rv = tcgetattr(fd, &termios);
assert(rv != -1);
cfmakeraw(&termios);
cfsetspeed(&termios, B9600);
rv = tcsetattr(fd, TCSANOW, &termios);
assert(rv != -1);
rv = tcflush(fd, TCIOFLUSH);
assert(rv != -1);
// 4 reads
char buf[256];
int nbyte;
printf("\n");
for (int j = 0; j < 4; ++j) {
nbyte = read(fd, buf, sizeof(buf));
assert(nbyte != -1);
printf("read %d bytes, hexdump: \n\n", nbyte);
hexdump(buf, nbyte);
printf("\n");
}
// close
rv = close(fd);
assert(rv != -1);
}
I am frequently observing the behavior decribed in 1/, for example below where 'u' was expected but first some 't' was received (due to the setup very likely from the previous run of the sketch).
XXX-MacBook-Pro:computer XXX$ ./serial /dev/cu.usbmodem1421
read 111 bytes, hexdump:
0000 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 tttttttttttttttt
0010 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 tttttttttttttttt
0020 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 tttttttttttttttt
0030 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 tttttttttttttttt
0040 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 tttttttttttttttt
0050 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 tttttttttttttttt
0060 74 74 74 74 74 74 74 74 74 74 74 74 74 74 74 ttttttttttttttt
read 2 bytes, hexdump:
0000 75 75 uu
read 4 bytes, hexdump:
0000 75 75 75 75 uuuu
read 4 bytes, hexdump:
0000 75 75 75 75 uuuu
And somewhat less frequently the behavior decribed in 2/ where bytes were received that were definitely never send by the Arduino (note the two f5 bytes in the first read that show as ? in the ascii part of the dump):
XXX-MacBook-Pro:computer XXX$ ./serial /dev/cu.usbmodem1421
read 212 bytes, hexdump:
0000 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
0010 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
0020 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
0030 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
0040 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
0050 75 75 75 f5 75 75 75 75 75 75 75 75 75 75 75 75 uuu?uuuuuuuuuuuu
0060 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
0070 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
0080 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
0090 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
00a0 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
00b0 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
00c0 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 75 uuuuuuuuuuuuuuuu
00d0 75 75 75 f5 uuu?
read 4 bytes, hexdump:
0000 76 76 76 76 vvvv
read 4 bytes, hexdump:
0000 76 76 76 76 vvvv
read 4 bytes, hexdump:
0000 76 76 76 76 vvvv
As for 1/ I assume that there can be bytes on the line and in the buffers while the Arduino is being reset. So this might be normal behavior.
I did some further digging with ioctl calls and the data is actually arriving after some time after the open() call. A short sleep of 1000 us (!) seems to resolve this issue. This allows the data to arrive and the subsequent tcflush makes it disappear. Withouth the sleep the tcflush flushes the buffer before the data has arrived and it shows up at the first read(). But sill the usleep() feels bad and the time to sleep is just a wild guess.
I have no explanation for the strange bytes though. So my question is if anyone could give me more insight into all of this, especially the reason for receiving the bytes that have not been send. What could cause this?
All of this does imply that making a robust serial protocol requires careful design (error detection, synchronization, ...) and is definitely not an "arduino style beginner level" task. But again, I am not looking for help in this direction, I am looking for an explanation of the strange bytes. I am also planning to test on different hosts and with other boards.
Maybe I should also look into the code for the Arduio GUI serial monitor.
