Go Down

Topic: Lost Bytes when reading Serial1 when receiving also data via native USB port (Read 27158 times) previous topic - next topic

Embed

I discovered a problem in the Arduino Due libraries (I am still at 1.5.4 due to the analogRead problems in 1.5.5). Using an Arduino Due board with following setup:
- Tx1 connected with Rx1
- native USB connected to a computer running a terminal (I will name it "USB Terminal" below)
- programming USB port connected to a computer running a terminal (I will name it "Monitoring Terminal" below)
- Demo Program using to the code below loaded.
Do the following. Open both Terminals with 115200baud. Reset the Arduino board. You will see some text on the Monitoring terminal, cyclically printing the current status. Without data to the native USB port, you will see no fails. Then with the USB Terminal send a string (I used ca. 150 characters) repeatedly to the Arduino Due native USB port. The Demo program shows also the amount of received USB data. Eventually you will see Fails counting up. In my setup, I had 5 Fails after sending 16kByte of data to the native USB port.
The nature of the error is that one of the transmitted bytes is not received on Serial1. The Demo program prints the received bytes to the terminal if an error occurs.
Using an oscilloscope and further debug code hints that the reception is the problem, not the transmission. In some way, the data reception on the native USB port influences the correct reception of bytes on Serial1.

Code: [Select]

#include <Arduino.h>

long serialcounter = 0;
long failcounter = 0;
long usbcounter = 0;
int cyclecounter = 0;

void setup() {
    Serial.begin(115200);
    Serial1.begin(115200);
    Serial.println("SerialErrorDemo");
}

void loop() {
    unsigned char testbytes[] = {
        1,2,3,4,5,6,7,8
    };
    char serial_buffer[1000];
    char usb_buffer[16000];
    int n;

    Serial1.write(testbytes, sizeof(testbytes));
    serialcounter += sizeof(testbytes);
    cyclecounter++;
    if (cyclecounter == 1000) {
        cyclecounter = 0;
        Serial.print("Tx Bytes: ");
        Serial.print(serialcounter, DEC);
        Serial.print("  Fails: ");
        Serial.print(failcounter, DEC);
                Serial.print("  USB received: ");
        Serial.println(usbcounter, DEC);
    }
    delayMicroseconds(6000);

    n = Serial1.available();
    if (n > sizeof(serial_buffer)) n = sizeof(serial_buffer);
    Serial1.readBytes(serial_buffer, n);
    if (n != sizeof(testbytes)) {
        for (int i = 0; i < n; i++) {
            Serial.print(serial_buffer[i], DEC);
            Serial.print(",");
        }
        Serial.println();
        failcounter++;
    }

    n = SerialUSB.available();
    if (n > sizeof(usb_buffer)) n = sizeof(usb_buffer);
    SerialUSB.readBytes(usb_buffer, n);
    usbcounter += n;
}

MarkT

That delayMicroseconds(6000) is the time it takes for 62 characters to transfer at
115200 baud, looks foolish to call delay in a sketch that's handling asynchronous
I/O.   BlnkWithoutDelay again...
[ I will NOT respond to personal messages, I WILL delete them, use the forum please ]

Embed

Regarding "foolish", this is only a demo program, solely written to demonstrate the problem with the interfering USB data, after it cost me one day of work to isolate the bug.
The delay is not the point here, it just guarantees that sufficient time has passed and it can be expected that all transmitted bytes can be read. I know that the delay could be less, and I chose another kind of checking for the transfer to be complete before reading the echo in my project.
What remains for here is the fact that there is a problem with the Arduino libraries that has to be fixed by an expert for the related interface driver code.

Embed

I refined my serial error demo program to send 500byte messages instead of only 50byte since I wanted to find out how long after reading characters from the native USB impacts the reliability of received serial bytes on serial ports. For that reason I also modified the output in the error case. Now the indices of the dropped bytes are provided. In addition, I wanted to be sure that also other serial inputs are affected, so I changed the receive serial port to serial 2 (the electrical connection on the DUE must be then between Tx1 and Rx2). See the code below for 115200baud.
In order to find out the dependency of the symptoms on the baud rate, I also checked with 57600baud and 28800baud (see logs below).
Results from testing for the serial input are:
- with some probability, there is a drop of one byte after the SerialUSB.readBytes statement
- with lower probability, there is a drop of two (mostly non-consecutive) bytes after the SerialUSB.readBytes statement
- the highest observed index of a dropped byte was 62 for 115200baud, 38 for 57600baud, and 17 for 28800baud; this translates to transmission times of 5.4ms, 6.6ms and 5.9ms, respectively.

Conclusion: for a time of up to ca. 6.6ms after a SerialUSB.readBytes statement (tested from 115200baud down to 28800baud),  the reception of serial bytes is corrupted by randomly dropped bytes - mostly one byte, sometimes two.

Now it is time for an expert to find the root cause and provide a fix.


Log with 115200baud(part)

Tx Bytes: 950000  Fails: 81  USB received: 583340
9
4
Tx Bytes: 1000000  Fails: 83  USB received: 616668
2,60
26
61
59
Tx Bytes: 1050000  Fails: 88  USB received: 649916
62
20
4
35
29


Log with 57600 baud

SerialErrorDemo
Tx Bytes: 50000  Fails: 0  USB received: 26292
38
Tx Bytes: 100000  Fails: 1  USB received: 77392
Tx Bytes: 150000  Fails: 1  USB received: 128492
4,10
Tx Bytes: 200000  Fails: 3  USB received: 179592
5
33
8
14
Tx Bytes: 250000  Fails: 7  USB received: 230692
26
Tx Bytes: 300000  Fails: 8  USB received: 281792
1,8
16
5
0,6
1,3
0
13
3
Tx Bytes: 350000  Fails: 19  USB received: 332892
0,4
4
3
26
Tx Bytes: 400000  Fails: 24  USB received: 383992


Log with 28800 baud

SerialErrorDemo
1
Tx Bytes: 50000  Fails: 1  USB received: 31551
Tx Bytes: 100000  Fails: 1  USB received: 82651
1
10
17
3
2,4
3
Tx Bytes: 150000  Fails: 8  USB received: 133751


Code: [Select]

#include <Arduino.h>

long serialcounter = 0;
long failcounter = 0;
long usbcounter = 0;
int cyclecounter = 0;
unsigned char testbytes[500];

void setup() {
    Serial.begin(115200);
    Serial1.begin(115200);
    Serial2.begin(115200);
    Serial.println("SerialErrorDemo");
    for (int i = 0; i < sizeof(testbytes); i++) {
        testbytes[i] = i;
    }
}

void loop() {

    char serial_buffer[1000];
    char usb_buffer[16000];
    int n;

    Serial1.write(testbytes, sizeof(testbytes));
    serialcounter += sizeof(testbytes);
    cyclecounter++;
    if (cyclecounter == 100) {
        cyclecounter = 0;
        Serial.print("Tx Bytes: ");
        Serial.print(serialcounter, DEC);
        Serial.print("  Fails: ");
        Serial.print(failcounter, DEC);
        Serial.print("  USB received: ");
        Serial.println(usbcounter, DEC);
    }
    delayMicroseconds(60000);

    n = Serial2.available();
    if (n > sizeof(serial_buffer)) n = sizeof(serial_buffer);
    Serial2.readBytes(serial_buffer, n);
    if (n != sizeof(testbytes)) {
        int j = 0;
        for (int i = 0; i < n; i++) {
            if (serial_buffer[i] != testbytes[i + j]) {
                if (j++) Serial.print(",");
                Serial.print(i, DEC);
                failcounter++;
            }
        }
        if (j) Serial.println();
    }

    n = SerialUSB.available();
    if (n > sizeof(usb_buffer)) n = sizeof(usb_buffer);
    SerialUSB.readBytes(usb_buffer, n);
    usbcounter += n;
}

ivanseidel

Erros on Transmissions are really hard to happen (not impossible).
Erros on receiving are even more likely to happen (they occur lots of time). That's the reason why a good protocol with CRC and others "packet checking system" are more reliable.

With what I have worked, erros that are most commonly:
* Output Buffer overflow. When sending, if the amount of bytes sent to the hardware exceed it's size, it's cropped (that's not YOUR case, but pay attention to it). Usually, you have to wait it to send things to continue flling the buffer.

* Input Buffer Overflow. When receiving, if the ammount of incomming data exceeds the input buffer size, you will have lost bytes and/or wrong byte data. In this case, ALWAYS reading the buffer resolves it, but, keep in mind that: Buffers overflows VERY fast! In the ATMEGA328 it's about 64bytes only! (Due is more, but not infinite).

* Errors on triggering/receipment. If you are dealing with lot's of concurrent operations (not "parallel" but things like interruptions or such things), this can slow make the hardware to crash while reading something (VERY hard to happen).

* Physical Errors. Data transmission problems... (Most common in my case, since electronics are really suseptible to noises and things that causes the message to receive with problem or, not receive.

In sum: If you want a ROBUST system, use/create a protocol that has a CRC system, to check if the content of the transmission is really correct...
Arduino DueTimer | https://github.com/ivanseidel/DueTimer
Arduino Thread | https://github.com/ivanseidel/ArduinoThread

Embed

Ivanseidel, thank you for your comments, however I was already aware of that general issues with serial communication.
In my project, I use 32Bit CRC, so I detect transmission errors with very high probability.  Most errors are not reported as CRC check errors since the incorrect size of the received message is detected before that. With a baud rate of 115200, I detect erroneous reception about once per second, and this is due to the described Arduino bug only. This error rate is far above the physical transmission error rate and in no way acceptable.

stimmer

Firstly you should call serialUSB.begin(0) in setup - this won't change anything because the method is currently empty, but that could change in future.

Please try your code with my alternative USB serial code here:
http://forum.arduino.cc/index.php?topic=143871.msg1280899#msg1280899
Note this code also suffers from data loss, but under different circumstances. It may work for you, it may not.
Due VGA library - http://arduino.cc/forum/index.php/topic,150517.0.html

ivanseidel


Ivanseidel, thank you for your comments, however I was already aware of that general issues with serial communication.
In my project, I use 32Bit CRC, so I detect transmission errors with very high probability.  Most errors are not reported as CRC check errors since the incorrect size of the received message is detected before that. With a baud rate of 115200, I detect erroneous reception about once per second, and this is due to the described Arduino bug only. This error rate is far above the physical transmission error rate and in no way acceptable.


Well done! 1 error per second is a lot of errors! It must be something wrong in the code/register dealing...

I cannot help you more than this... just with the hardware in my hands and a lemonade to start thinking, sorry =[
Arduino DueTimer | https://github.com/ivanseidel/DueTimer
Arduino Thread | https://github.com/ivanseidel/ArduinoThread

Embed

Stimmer, I tried your alternative USB serial code. While solving the problem on the Arduino side for the receive errors on other serial in ports, it causes another problem on the host side.
Using Linux on a virtual machine on a Mac OX 10.8 computer as development environment, there is now following issue.
My Linux program cyclically sends (with ca. 6ms period) a message to the Arduino board via USB, with typical message lengths of 24Bytes.
For that purpose I open a serial device and use statements like
size = write(fd, ByteBuffer, msgsize);
This has always worked so far, but now with the alternative USB serial code on the Arduino, I observed an unusual slow down of message rates. Adding some debugging code, I found out that the first 10-20 write calls take some microseconds (as I would always expect), and the following take mostly 0.1s - 0.2s and sometimes 1.2s.
It seems that the transfers are not closed correctly on the Arduino side, leaving the host waiting for confirmation and running into timeouts.

Go Up