Buffer overflow only when using Fat16 library

Using Arduino Storage
#1

Hello everyone,

I need your help.

My project is based on a raw NMEA sentence datalogger but so far I am stuck with SD library (I think).

I use greiman’s Fat16 library because I need something that is lightweight and yet reliable.

My microSD card is a 2Gb one, formatted using official SD Formatter.

When I use Serial.print() everything work fine.
When I ‘switch’ to file.print() every 150kb -in average- 1 to 120 characters are lost.

Here is my code:

unsigned int   r2w = 32;  // flag >> Ready to write to microSD
         char  com =  0;  // Buffer for serial data

#include <Fat16.h>
SdCard card;
Fat16 file;

void setup(){
  Serial.begin(9600);

  card.begin(SS);
  file.init(&card);

  file.writeError = false;
  file.open("nmea_raw.log", O_CREAT | O_APPEND | O_WRITE);
  file.println("----- restart  recording -----");

  while(!Serial.available());
}

void loop(){

  com = Serial.read();
  file.print(com);
  r2w++;

  if(r2w == 512){
    file.sync();
    r2w = 0;
  }

  while(!Serial.available()); //wait buffer to get some data

}

I tried using file.sync() in several schemes like every:

  • time a CRLF detected,
  • 1 byte,
  • 512 bytes.

I also tried running without any delay, with delay(1) and finally with a wait-until-data-arrived loop.

Every time with the same result.
Is there any kind of library’s limitation or my approach has flaws?

Azelo Gose

#2

Is there any kind of library's limitation or my approach has flaws?

Well, it has at least one flaw. You read data, and write it the the buffer. Then, you wait for data to be available to read.

You need to wait FIRST.

#3

Thank you PaulS.

PaulS:
You need to wait FIRST.

Can you be more specific; wait for: data to be written, data to become available or something else?

Azelo Gose

#4

Can you be more specific

Sure.

You must do this:

  while(!Serial.available()); //wait buffer to get some data

before this:

  com = Serial.read();

It is pointless to read from the serial buffer without knowing whether there was anything to read. It is pointless to use the value returned by read() without determining if the value meant "Oops, there was nothing to read...".

#5

PaulS:
It is pointless to read from the serial buffer without knowing whether there was anything to read. It is pointless to use the value returned by read() without determining if the value meant "Oops, there was nothing to read...".

I don't think this is the case.
Maybe with an unorthodox way but I do wait for a flag to be raised before I try to read data.

void setup(){
...
while(!Serial.available());  //Not leaving unless I'm ready for reading
}

void loop(){
com = Serial.read();         //Read only one byte
...
while(!Serial.available());  //Don't loop unless next byte is there waiting
}

PS It may looks dumb using two times the same while() but
I need to stay in setup() routine until GPS is connected

#6

PS It may looks dumb using two times the same while() but
I need to stay in setup() routine until GPS is connected

You can stay in setup() until data arrives, or you can do nothing in loop() until data arrives.

Or, you COULD do other things while there is no GPS data to read.

#7

For anyone that may concern: problem temporary fixed by using a 400-byte char array

Using this code with Serial.print() everything were fine.
After commenting and switching to file.print() I get relative many faulty/chopped NMEA sentences

void loop(){
char com = 0;
  while(!Serial.available()) delayMicrosedeonds(32);
  com = Serial.read();
  //Serial.print(com);
  file.print(com);
}

Now I’m using this code and everything are fine once again

void loop(){
char com[400];
uint8_t eol =  0;
uint16_t idx =  0;

  while(eol < 4){  //Read x4 NMEA sentences
    while(!Serial.available()) delayMicroseconds(32);
    com[idx] = Serial.read();
    if(com[idx] == 10) eol++;  //End of NMEA sentence counter
    idx++;
  }

  for(uint16_t i = 0; i < idx; i++){  //Print array to file
    file.print(com[i]);
    com[i] = 0;
  }
}

Any thoughts why does this happen?

#8