High speed serial communication

Hi, I've got an Arduino with a USB connection. I'm trying pipe data via the USB port through the adruino to pump data into some LED drivers via the digital outs. The whole setup works flawlessly at low data rates. However, I'd like to send data to the Arduino as fast as it can send it out to the LED drivers.

The problem I'm finding is that I can pretty easily overflow the serial buffer on the Arduino. When this happens it starts losing data and the LEDs get all wacky. If this was a straight serial conection I could probably use hardware flow control to keep things under control. However I'm not sure how this would work via the USB connection.

Anyone have any tips for flow control via USB? Anyone know what the size of the serial buffer is? I know this must be a solved problem since the uploader (avr_dude) doesn't seem to have any problems sending about 3KB/s. I'm on an FC5 box btw.

Thanks!

There is no flow control between the USB<->Serial bridge and the Arduino.
The receive buffer is 128 bytes.
The bootloader copes with this by reading an entire command packet into a buffer before it goes off to do any work.

If you structured your protocol so it would write some number of bytes less than 128, then wait for an acknowledge before sending more data you could be fairly sure of not dropping data. Unless something blocks interrupts for longer than a character transfer time.

Thanks for the information! I did a test to confirm the 128 byte buffer to make sure my code and my particular arduino aren’t screwy and it checks out (well, it actually starts dropping at the 128th byte, but who’s counting). With that confidence I’ll try what you suggest about reading in larger chunks and waiting.

I’m not setting any explicit interrupts right now so I think I should be ok.

Thanks for your help!

I still seem to be having problems involving dropped data. Since the arduino serial port should be able to buffer 128 bytes, I decided to go with a chunk size of 72 bytes. I send that many, then I have the arduino code send back an acknowledge string. However, even with only 72 bytes I find that it still drops bytes. Here's the smallest arduino program that still demonstrates the problem I'm having:

void setup () {
  Serial.begin(115200);        // initialize the serial port
}
void loop () {
  int data[72];
  int idx = 0;

  while (idx != 72) {
    int avail = Serial.available();
    if (avail > 0) {
      data[idx++] = Serial.read();
      Serial.print("-- Avail: "); Serial.print(avail);
      Serial.print(" / Data: "); Serial.print(data[idx-1], DEC);
      Serial.print(" / Byte: ");Serial.print(idx);Serial.println("");
    }
  }
}

This code runs fine for a random number of iterations and then hangs, waiting for enough bytes to reach byte 72. Something else interesting about this is that the 'avail' variable regularly reports numbers larger than 128, often as high as 190 or 200. I haven't sent that many bytes and the arduino's serial buffer isn't supposed to be able to hold that many. This number will then suddenly drop down to a reasonable value. Here's a few loops of output as an example:

-- Avail: 140 / Data: 0 / Byte: 54
-- Avail: 139 / Data: 0 / Byte: 55
-- Avail: 138 / Data: 0 / Byte: 56
-- Avail: 9 / Data: 0 / Byte: 57
-- Avail: 8 / Data: 0 / Byte: 58
-- Avail: 7 / Data: 0 / Byte: 59

In the above loop, it lost 7 bytes of data and hung. Getting a number of bytes over 128 doesn't always mean trouble; I've had the problem with a weird number of available bytes and I've gotten it when I see a sane number of bytes.

The code driving this is Perl. Here's the shortest code that exhibits this problem:

#!/usr/bin/perl -w

use Device::SerialPort;

my $p = Device::SerialPort->new('/dev/ttyUSB0');
$p->baudrate(115200);

while (1) {
    foreach my $byte (0..71) {
        $p->write($byte == 50 ? chr(200) : chr(0));
    }

    my $line = '';
    while ($line !~ /Byte: 72$/) {
        my ($cnt, $str) = $p->read(1);
        if ($cnt) {
            print STDERR $str;
            $line .= $str;
        }
    }
}

The data being sent is meant to mimic the data I've been testing with. Mostly zeros and single value. I get the same behavior with random data too however.

I realize this is a lengthy post, but any help or ideas would be appreciated.

I'd also be interested to hear about the serial boundaries the author describes here. Has anyone seen detailed documentation on the issue?

thanks

In your test program you are calling serial.print 6 times for each byte you recieve, this might be (a part of ) the problem.

Does your erratic LED behaviour also occur if you do not call serial. print ?

I don't know if it is possible to test it without serial.print debugging.

Also, how can Avail be 140 if the buffer is only 128 bytes ????

Something is not right!

Also your array is 0 based so the last index is 71 not 72

From HardwareSerial.cpp...

#define RX_BUFFER_SIZE 128  // line 34
...
uint8_t HardwareSerial::available(void)  // line 168
{
  return (RX_BUFFER_SIZE + _rx_buffer->head - _rx_buffer->tail) % RX_BUFFER_SIZE;
}

Is it maybe your type mismatch between available (uint8_t) and avail (int)?