Serial.read using USB connection

hey folks, i've got a sketch that records all sorts of data from CANBus, GPS, and some other sensors. All of the data is being written to a MicroSD card via the SdFat library.

I'm trying to read the data off of the SD card via the Serial port connected to a computer over USB.

I have my serial port set to 115200bps, but i'm having the same problem (though less often) even when i set the serial port to 9600bps.

Basically, i setup a simple protocol where i say "G " (obviously is replaced by the name of the file, and the format of the filename is something like "00023-LG.CSV".

My sketch code looks like this:

void setup() {    
    #define MAX_BUFSIZE 25
    char *buf_ptr;
    char buffer[MAX_BUFSIZE];

...

    Serial.begin ( 115200 );
}

void loop() {

...    

    buf_ptr = buffer;
    if ( Serial.available() > 0 ) {
        while ( Serial.available() ) {
            *buf_ptr = Serial.read();
            //Serial.print ( "read: " );
            //Serial.println ( *buf_ptr );
            if ( ( *buf_ptr == '\r' ) || ( *buf_ptr == '\n' ) ) {
                *buf_ptr = NULL;
            }
            buf_ptr++;
        }
        *buf_ptr = NULL; 
        Serial.print ( "COMMAND: " );
        Serial.println ( buffer );

        // more processing here...
    }
}

I commented out the Serial.print() and Serial.println() for every character read since i was concerned that affecting the issue. It may have some bearing on it, but i still see the problem when the code is commented out.

The problem i see is that when i print out buffer at the end of reading data, i always get the first character, and sometimes there are missing characters thereafter. However the last character is almost always present.

So for example, if i do "G 00023-LG.CSV", sometimes buffer will be correct at "G 00023-LG.CSV", but often times it will be "G SV", or "G 00V", or "G "

Originally, i thought it was that i was trying to read the data off of the serial port too quickly on the arduino, however, since most often the last character is in the buffer, i don't think that's the case.

Any ideas? Thanks!

It seems that the program needs some machine cycles to take the first character. I put a delay of 2ms and works well.
I change the "while" for "do while" just to ensure that the operation is performed at least once.
This is a program tailored to my needs (does not change much).

void loop()
{
buf_ptr = buffer;
if(Serial.available() > 0)
{
do
{
*buf_ptr = Serial.read();
//Serial.print ( "read: " );
//Serial.println ( buf_ptr );
delay(2);
/
if ( ( *buf_ptr == '\r' ) || ( *buf_ptr == '\n' ) ) {
*buf_ptr = NULL;
} */
buf_ptr++;
}while(Serial.available() > 0);
*buf_ptr = NULL;
//Serial.print ("COMMAND: ");
Serial.println (buffer);

Thank you very much for this code snippet, I needed it.

If your problem is not the missing characters, probably your problem is that the characters '\n' and '\r' is not recognized as line feed and carriage return, these are replaced by other characters.

My sketch code looks like this:

I seriously doubt that. That code won't even compile. buffer and buf_ptr are local variables in setup() that are referenced in loop(). That will never work. Post ALL of your code (properly, using the # button) is you really want help.

Serial data transmission is slow, like my typing. You are reading the data faster than it comes in. The only was to correct the problem is to use an end-of-packet marker, and store data until that end of packet marker arrives. You only use the stored data when that end-of-packet marker arrives.

Yeah, sorry about that compilation issue. My sketch is pretty big, so i was pulling out the salient parts. The loop where i'm reading the data off of the serial line is pretty much exactly as i have it coded up right now.

I've read that link to jhaskell's blog, that is an excellent and complete description of how to avoid problems, and i'll certainly put some of that into my sketch.

However, In your post, you say: "The only was to correct the problem is to use an end-of-packet marker, and store data until that end of packet marker arrives" (and the jhaskell blog also says the same). Effectively, the return character is my end of packet, and if you look at that loop, i'm storing the data in "buffer". Sure, i'm not doing as much error checking, but in this case it wouldn't help (i don't think)

The failure scenario you're describing (i think) is the loo? is terminated prematurely because "Serial.available()" returns -1 (because the arduino is reading data off of the serial line too quickly). In that scenario, you would expect that if the input data is "G 00023-LG.CSV", the buffer would contain a prefix substring of the data such as "G", or "G 0002", etc.

However, the failure scenario i'm seeing is that the data sometimes contains a hole in the middle, so things like "G 000V" (notice that "23-LG.CS" is missing from the middle of the string).

I'm trying to figure out how that problem occurs, and how to fix it. The method of storing data until the end-of-packet marker (even if i changed the end-of-packet marker)

Thanks!