next topic »

[on:]

I need to send about 120 bytes of data to the Arduino over the serial port (I'm using an older Arduino, a Duemilnoieve or whatever it's called  ). The data is coming from a Windows 8 (64 bit) C# program I've written.

I ran into a number of problems which I think are related to overflowing the Arduino serial buffer. So I thought I'd set up a handshaking protocol, where I send 30 bytes, wait for a character to come back from the Arduino, send another 30 bytes, repeat until done.

As part of the process I check Serial.available() to see if there's anything to read. If nothing's available, I just exit/re-enter loop(). If enough time passes with nothing available on the serial port I time out the data receiving function.

Here's a code fragment showing almost all of the algorithm:

Code:

boolean cmdMode = false;
byte serialAvail;
time_t lastRead;

void loop()
{
time_t t = now();

if( cmdMode )
{
byte curAvail = 0;

                // I thought this loop would "drain" the serial buffer, so the
                // next pass through loop() -- before the Windows program reloads
                // the Arduino's serial buffer -- curAvail would be 0. But that's not
                // happening, it's always 1.
while( Serial.read() >= 0 )
{
curAvail++;
}

serialAvail += curAvail;

if( curAvail > 0 )
{
// ask for more
Serial.write(109); // 109, m, for 'send more'
lastRead = t;
}
else
{
// timeout if nothing new received within
                        // 10 seconds of the last read
if( t - lastRead > 10 )
{
                                // some stuff to tell Windows program we're done
Serial.write(serialAvail);
Serial.print("okay");

cmdMode = false;
}
}

return;
}

...later on in loop() I set cmdMode = true when a specific byte code is read from the serial port.

Unfortunately, Serial.available() >>always<< shows 1 byte available. Which I don't understand, since I've only sent 30 bytes to the Arduino and I've presumably read them all.

Besides suggestions on fixing the algorithm I wouldn't mind some advice on better ways to handle transmitting large (i.e., compared to the serial buffer) amounts of data to the Arduino.

Moderator edit: [code] ... [/code] tags added. (Nick Gammon)

[Reply #1 on:]

Oops, sorry, I refer to Serial.available() in my post (which I used in an earlier version), but my problem is that Serial.read() is always returning a non-negative value. I thought it returned -1 when there was nothing in the buffer.

[Reply #2 on:]

That is a completely bizarre way to handle the serial port, and I have no idea what you hope to achieve with that.

A far simpler approach is just to use Serial.available() to test whether there is a character available, and read it if there is.

The Arduino runs massively faster than the serial port, and the only way you'd overflow the serial RX buffer is if you neglected to read from the serial port for longer than it takes to fill the buffer. You don't need a handshaking protocol to make the transfer reliable. The only benefit you'd get from the handshake is that it might stop the sending code from overflowing the transmit buffer on the sending side - but I have no idea whether that's likely to be a problem.

[Reply #3 on:]

I agree it's bizarre. But I ended up trying it because using the approach you suggested didn't work. I don't try to do things any more bizarre than they need to be.

[Reply #4 on:]

Does the string being received include an end marker?

[Reply #5 on:]

I'm not sending a string. I'm sending a 129 byte long array of bytes (the size is fixed, so the sketch knows it's received everything when it gets 129 bytes).

[Reply #6 on:]

I'm not sending a string. I'm sending a 129 byte long array of bytes (the size is fixed, so the sketch knows it's received everything when it gets 129 bytes).

bummer

[Reply #7 on:]

I ran into a number of problems which I think are related to overflowing the Arduino serial buffer. So I thought I'd set up a handshaking protocol, where I send 30 bytes, wait for a character to come back from the Arduino, send another 30 bytes, repeat until done.

A far simpler technique would be to copy the 129 bytes into a buffer in your program.

The nature of serial data is such that you really should have a "start" byte, so if you happen to connect in mid-stream you are not out of sync.

http://www.gammon.com.au/serial

the sketch knows it's received everything when it gets 129 bytes

Maybe. It knows it has received 129 bytes is about all.

Code:

                // I thought this loop would "drain" the serial buffer, so the
                // next pass through loop() -- before the Windows program reloads
                // the Arduino's serial buffer -- curAvail would be 0. But that's not
                // happening, it's always 1.
while( Serial.read() >= 0 )
{
curAvail++;
}

That won't really "drain" the buffer if data is currently incoming, because it is still arriving, and you will execute that loop faster than the data arrives.

Almost always, when we find people trying to "drain" or "flush" the buffer, they are going about it the wrong way ... no offence.

Since you are writing both ends, send a "start" byte (eg. 0x01), and then the data, and then an end marker. (eg. 0x02). The Arduino end saves between the start and end marker into memory. Preferably also have a CRC check. Also make sure you don't overflow the buffer (in case you miss the end marker).

I wrote a simple library to do exactly that, described as part of this page:

http://www.gammon.com.au/forum/?id=11428

[Reply #8 on:]

I don't understand what your code does.   You need to be reading the bytes and sticking them into some
kind of array,  and I don't see that happening.

[Reply #9 on:]

Try to send some fixed length TEXT string, like HELLO from your PC and see if you are receiving them correctly. If you still have problem, I think your PC program is doing something unexpected. If you got the text correctly with right length, then try to send some binary data (like 120 3s) with known length, if you still get problem, then I don't know what.

[Reply #10 on:]

You are making an assumption that sending 129 bytes of data means that the receiver will ALWAYS receive 129 bytes of data. This is not the case.

Serial data handling does not follow the UPS model, where they guarantee to deliver your package.

It follows the USPS model, where they guarantee to try to deliver your package.

Big difference, and one that you MUST take into account.

[Reply #11 on:]

I agree it's bizarre. But I ended up trying it because using the approach you suggested didn't work. I don't try to do things any more bizarre than they need to be.

A far simpler approach is just to use Serial.available() to test whether there is a character available, and read it if there is.

Lots of other people manage to get this approach working just fine, and if you're having trouble with it I suggest you address that rather than try to invent your own flow control mechanism.

Code:

// incomplete, untested
void loop()
{
    while(Serial.available())
    {
         byte input = Serial.read();
         if(bufferCount < BUFFER_LENGTH)
         {
             buffer[bufferCount++] = input;
         }

        ... test whether a complete message has been received, and if so process it and reset bufferCount to zero
    }
}

As long as you don't design your sketch to block for long periods when it should be listening to the serial port, this approach should work fine.

[Reply #12 on:]

I understand that there's no guarantee that a serial communication "session" will deliver all its data. That's why on the receiving end (i.e., in my Windows configuration app) there's a timeout period. As well as a variety of other integrity checks, including a checksum validation.

I was finally able to get the serial communications working reliably. Thanks for the suggestions.