SD card writes corrupt incoming serial data

Hi,

I am using a Mega to log sensor data. One of the sensors gives data over serial, quite lengthy (say under 250 bytes in 1 sec) and in need of processing (parsing). All the data goes on an SD, in one or multiple files.

The problem is that sometimes the input buffer for the serial sensor is simply corrupted (I am sure the data is sent ok) - so I cannot parse it properly. I have linked this problem to SD file flushes. When there is only one file, the problem appears not as often. When I limit manual flushes (calling file.flush), performance is again better. However I cannot get a good steady reading for longer minutes.

Has anyone had such a problem before? I do not know all the intricacies of the Arduino, but does the flush prevent maybe serial receive interrupts? How long is the flush?

I am using a banale SD card on a Adafruit datalogging shield, and the Aduino SD lib. I saw the SDfat is not updated in ARduino, should I just skip the SD from Arduino, and go with SDfat?

Any ideas welcome.

Thanks

alkalin: Hi,

I am using a Mega to log sensor data. One of the sensors gives data over serial, quite lengthy (say under 250 bytes in 1 sec) and in need of processing (parsing). All the data goes on an SD, in one or multiple files.

The problem is that sometimes the input buffer for the serial sensor is simply corrupted (I am sure the data is sent ok) - so I cannot parse it properly. I have linked this problem to SD file flushes. When there is only one file, the problem appears not as often. When I limit manual flushes (calling file.flush), performance is again better. However I cannot get a good steady reading for longer minutes.

Has anyone had such a problem before? I do not know all the intricacies of the Arduino, but does the flush prevent maybe serial receive interrupts? How long is the flush?

I am using a banale SD card on a Adafruit datalogging shield, and the Aduino SD lib. I saw the SDfat is not updated in ARduino, should I just skip the SD from Arduino, and go with SDfat?

Any ideas welcome.

Thanks

you are overflowing the Arduino Receive buffer, The HardwareSerial (Serial) library uses 63 byte buffers. You say the input data is over 250bytes in length. When you call SD.write() or flush(), which can take up to a second to complete, the Serial input buffer is overflowing.

You have two options:

  • read all of the data from the serial data packet into a local buffer, process, store
  • read some data, process the data, TELL the SENDER to SHUTUP (Stop Sending New Data), Store Results. Tell the SENDER to Start Talking (Continue Sending more New Data) rinse, repeat

Chuck.

:) Truly that was the problem. Thank you. I was under the impression the buffer is 256 bytes...