Problem with binary serial output. (jumping midstream, and losing bytes)

Greetings,

I am trying to send binary data through the serial port using a simple sketch and read the data in C on a linux box.

Here is my arduino sketch:

void setup() {
Serial.begin(115200);
}

byte i = 0;
void loop() {
Serial.write(i++);
}

And here is my C program:

#include <stdio.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

main() {
union {
uint8_t byte[256];
} buf;

int tty = open("/dev/ttyACM0", O_RDONLY);

int pos;

for(pos = 0; pos < sizeof(buf); pos++) buf.byte[pos] = 0;

pos = 0;
while(pos < sizeof(buf)) {
pos += read(tty, buf.byte + pos, sizeof(buf) - pos);
}

for(pos = 0; pos < sizeof(buf); pos++) {
printf("%4i: %4x\n", pos, buf.byte[pos]);
}

close(tty);
}

The issue that I am having is twofold:

  1. Running the C program seems to jump into the datastream at a random point. I would have thought that opening the serial port would reset the arduino (when DTR does its thing) and clear its buffers.

  2. I’m also losing bytes. I’m expecting a stream of bytes from 0x00 to 0xFF, so even if I jump in midstream, I should only get one of each. But instead, I seem to lose a few in between. Here for example is a snippet of the output:

207: 6b
208: 6c
209: 6d
210: ff
211: ef
212: f0
213: f1

For some reason I jump from 6d to ff to ef to f0.

If anyone has any experience sending binary data from the arduino to a C program on the host, I would be very grateful for some help.

Peter.

You are pumping data out pretty fast. It's possible the PC can't keep up.

Running the C program seems to jump into the datastream at a random point. I would have thought that opening the serial port would reset the arduino (when DTR does its thing) and clear its buffers.

Maybe. The IDE does that, I'm not sure if your program will. Build in a slow flash of the LED on pin 13 into setup, and then you will know if it reset or not.

Greetings Nick,

I thought of that, but even when I slowed things down to 9600 baud, I still had the same issue. And in any case, if I open the serial monitor, I get the sort of output I would expect (i.e. a series of identical lines of text including a swing through the printable ASCII character set). So it isn't a case of the host machine not keeping up.

And I have verified that my program does reset the arduino. You can watch the opening blinkiness as the bootloader looks for something to load...

Peter.

  while(pos < sizeof(buf)) {
                pos += read(tty, buf.byte + pos, sizeof(buf) - pos);
        }

Try reading one byte at a time.

For some reason I jump from 6d to ff to ef to f0.

0xFF is the IAC character. I wonder if the input is being interpreted somehow.

Greetings Nick,

I’ve tried your suggestion of reading one byte at a time:

#include <stdio.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

main() {
uint8_t buf;

int tty = open("/dev/ttyACM0", O_RDONLY);
int pos;

pos = 0;
while(pos < 256) {
while(read(tty, &buf, 1) <= 0);
printf("%4i: %4x\n", pos, buf);
pos++;
}

close(tty);
}

But I still get the same issues.

Also, if there was some kind of interpretation going on, I would expect the results to be fairly similar from run-to-run. I would expect that 6d would always jump to ff and so on. But it seems fairly random…

52: 5a
53: 5b
54: fc
55: dd
56: de

and

180: 5a
181: 5b
182: fc
183: 0
184: 1

I really appreciate your help BTW.

Cheers,

Peter

Hello Nick,

Both problems solved at once!

I needed to flush the local buffer (on the host machine) before I started reading. But I needed to do it after the bootloader sends its handshake, and before it gives up:

#include <stdio.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <termios.h>

main() {
uint8_t buf;
uint8_t pos;

// This is enough to trigger the bootloader
int tty = open("/dev/ttyACM0", O_RDONLY);
// Delay to avoid the bootloader’s handshake
sleep(1);
// Now flush the host’s buffers
tcflush(tty, TCIOFLUSH);

pos = -1;
do {
pos++;
while(read(tty, &buf, 1) <= 0);
printf("%4i: %4x\n", pos, buf);
} while(buf == pos);

close(tty);
}

Woot!

Thanks for all your help Nick!

Peter.