Data loss when sending to native USB port (SerialUSB)

The idea is to consume only as much data from the usb controller as you can store. As long as you don't have room for new data, the usb controller must be instructed to reply NACK or NYET to the host (it does that autonomously).

On the sam, the usb controller sends these NAKs/NYETs as long you don't "return" the fifo (or rather the memory bank(s) attached to it) to the the contoller. This is done by the udd_ack_fifocon(CDC_RX) function. There is an excellent diagram in the data sheet at pg. 1092 (40.5.2.13 Management of OUT Endpoints).

I gave it a try. In USBCore.cpp in USB_ISR():

 ...
#ifdef CDC_ENABLED
        if (Is_udd_endpoint_interrupt(CDC_RX))
        {
                udd_ack_out_received(CDC_RX);

                // Handle received bytes
                if (USBD_Available(CDC_RX))  // (this test is probably not needed)
                        SerialUSB.accept();

//              udd_ack_fifocon(CDC_RX);  // <-- don't return the fifo to the controller here, 
                                                               // do it from accept() if we could store all bytes
                  ...
        }

In CDC.cpp:

void Serial_::accept(void)
{
        ring_buffer *buffer = &cdc_rx_buffer;
        uint32_t i = (uint32_t)(buffer->head+1) % CDC_SERIAL_BUFFER_SIZE;

        // if we should be storing the received character into the location
        // just before the tail (meaning that the head would advance to the
        // current location of the tail), we're about to overflow the buffer
        // and so we don't write the character or advance the head.
        while (i != buffer->tail) {
                uint32_t c;
                if (!USBD_Available(CDC_RX)) {
                        udd_ack_fifocon(CDC_RX);
                        break;
                }
                c = USBD_Recv(CDC_RX);
                // c = UDD_Recv8(CDC_RX & 0xF);
                buffer->buffer[buffer->head] = c;
                buffer->head = i;

                i = (i + 1) % CDC_SERIAL_BUFFER_SIZE;
        }
}

In read(), after we consumed some data we must call accept again, thi might return the fifo to the controller:

int Serial_::read(void)
{
        ring_buffer *buffer = &cdc_rx_buffer;

        // if the head isn't ahead of the tail, we don't have any characters
        if (buffer->head == buffer->tail)
        {
                return -1;
        }
        else
        {
                unsigned char c = buffer->buffer[buffer->tail];
                buffer->tail = (unsigned int)(buffer->tail + 1) % CDC_SERIAL_BUFFER_SIZE;
                if (USBD_Available(CDC_RX))
                        accept();
                return c;
        }
}

It does not yet work correctly: it misses every 512th byte. The best test is to use Stimmer's loopback sketch Arduino Due - Serial speed? - #18 by stimmer - Arduino Due - Arduino Forum
Cat a file of e.g. 8KB to /dev/ttyACM1. Capture the reply also via cat:
cat /dev/ttyACM1 > txt. A diff shows you miss every 512th byte.

I don't think it is some off by one error in the circular buffer because if I define the CDC_SERIAL_BUFFER_SIZE as 64, I still loose each 512th byte.

I think it is somewhere in the code accessing the rx endpoint, but the low level api's are complex and there is some redundancy in them.

To be continued...
I thought I already share this as a starting point, maybe somebody else sees what is wrong...