SDfat.h Writing Speed

Hi guys,

i have already used the search but i am still strugeling with a problem of mine, so i´m hoping for some good hints.
I am using an Mega 2560 r3 to access three sensors via analogRead and logging the values via Adafruit SD shield on a normal Sandisk 1gb SD card (Fat16). I have to make a lot of mesuarements in a very short time (about 50 in 20 ms or as much as possible) and thus i changed from the SD.h to the SDfat.h library. I got it working after some minor problems but only to notice that there isn´t an inprovement in writing speed at all. Sd.h provided me with about one mesuarement every 1.1 ms and it hasn´t become any faster.
Even changing SPI_HALF_SPEED to FULL didn´t make any difference, so i am guessing that i did make a mistake while setting the whole thing up.

I would be really glad if someone would provide me with some help :slight_smile:

#include <SPI.h>
#include <SdFat.h>

SdFat sd;
SdFile myFile;

const int chipSelect = 10;

void setup() {
Serial.begin(9600);
while (!Serial){}
Serial.println (“Serial communication available”);

delay (1000);

if (!sd.begin(chipSelect, SPI_HALF_SPEED)){
sd.initErrorHalt();
}

if (!myFile.open(“test2.txt”, O_RDWR | O_CREAT | O_AT_END)){
sd.errorHalt (“Failed to open file”);
}

Serial.println(“File initialized”);
Serial.println(“Ready for logging”);

}

void loop(){

for (int i = 0 ; i < 10 ; i++){
myFile.print(micros());
myFile.print(" , “);
myFile.print(analogRead(A0));
myFile.print(” , “);
myFile.print(analogRead(A8));
myFile.print(” , ");
myFile.println(analogRead(A15));
}
myFile.flush();

}

SDfat_Log_3Sensor_Basis.ino (819 Bytes)

Your problem is the time required to format data and the slow speed of print.

You need to log binary data and write full 512 bytes blocks.

Look at these SdFat examples.

AnalogBinLogger - log analog pins at the fastest possible rate.

LowLatencyLogger - log data records from sensors at up to 500 Hz.

First try the unmodified examples. You will need to modify these examples for your application.

Ah, i see. I wasn´t aware of that fact but thank you. :slight_smile:
It took me some time but now i´ve got the basic setup working.

But at my current point another question occured to me. It is crutial for my whole setup to measure the times between every logged data. Since the sample only includes analog Pin´s, not commands like micros() and i´m not really skilled enough to modify it my logged data itself does not include any kind of time stamp.
However, by setting my sample rate i have a time stamp, e.g. 200 usec between each measurement. Is this value reliable for the time between each logging or does it vary due to writing latencys and stuff like that? If it´s something like +/- 10% i wouldn´t mind, but more could kill my whole experiment.

Best regards

However, by setting my sample rate i have a time stamp, e.g. 200 usec between each measurement. Is this value reliable for the time between each logging or does it vary due to writing latencys and stuff like that?

This won’t work unless you separate acquiring data from writing the SD like the SdFat AnalogBinLogger.

If it´s something like +/- 10% i wouldn´t mind, but more could kill my whole experiment.

If you can tolerate 10% time jitter you don’t need to log at that rate. There are many application notes about information content of ADC data with time jitter. If you want maximum information in a time series, time jitter may need to be as small as the error in amplitude as a percentage.

For 10 bit ADC data the lsb is 0.1% so you may need 0.1% time jitter for maximum information. You are measuring a curve in value-time space so both coordinates need similar accuracy.

The SdFat AnalogBinLogger triggers the ADC for the first analog pin using timer1. The start time will have a time jitter of less than one CPU cycle. The remaining pins in a sample are started as fast as possible in an interrupt routine. There will be time jitter and time skew depending on the ADC clock rate for these pins.

LowLatencyLogger uses this test and insures the jitter will not be larger than about 10 μs.

      do {
        diff = logTime - micros();
      } while(diff > 0);
      if (diff < -10) {
        error("LOG_INTERVAL_USEC too small");
      }
      if (curBlock == 0) {
        overrun++;
      } else {
        acquireData(&curBlock->data[curBlock->count++]);

LowLatencyLogger has a time jitter of about ± 4 μs when logging analog pins.

You need to do something like these examples to insure low jitter.

Wow, you gave a very comprehensive answer, thank you :).

But now i am still wondering if i could, besides logging my chosen analog pins also can add e.g. the micros() command in there, so there are there values plus the microseconds time in my logfile?
I´ve tried to comprehend the whole AnalogBinLogger sketch but its still challenging me quite a bit.

Best regards

If you use AnalogBinLogger the time between records is determined by timer1 and is more accurate than a call to micros. There is no way to call micros with AnalogBinLogger since timer1 starts the ADC without execution of code. Data is read from the ADC in an interrupt routine when the digitization is compete.

If you use LowLatencyLogger, the time is included as part of the record.

Be aware that the Arduino ADC takes a very long time to digitize each pin. The standard analogRead() takes over 100 microseconds to digitize a value.

I use a faster ADC clock but the time is still at least 25 microseconds to digitize a value. This means that the three pins are measured at different times since the single Arduino ADC is multiplexed.

What is the purpose of your measurements? How will you use the data?