Increase SD data logging speed

Using Arduino Storage
1

I'm trying to build a data logger with a sample rate of ~700-1000Hz.

My hardware is as follows:

Arduino UNO
5V to 3.3V logic level converter
Standard size SD card module
SanDisk SDHC 16GB class 4
LCD screen + assorted buttons, switches, and LEDs

The data logging portion of my code is below:

void LogData() {
  int sensorValue = analogRead(A0);
  RecordSwitch = digitalRead(7); 
 
  // open the file. note that only one file can be open at a time,
  // so you have to close this one before opening another.
  // SD.open will take either a string or a char* as its input. In this case, FileName is a string.
  File dataFile = SD.open(FileName, O_CREAT | O_WRITE);

  // if the file is available, write to it:
  // if file is not available, something has gone terribly wrong. Display an error to warn them!
  if (dataFile) {
    if ( q == 0 ) {
      digitalWrite(pin, HIGH); //turns on the LED
      lcd.clear();
      lcd.setCursor(0, 0);
      lcd.print("Recording..."); //prints "Recording" onto the lcd
      q = 1;

    }
    elapsedtime = millis() - Starttime; //Starts tracking the elapsed time
    dataFile.print(elapsedtime); //prints elapsed time in one column
    dataFile.print(", "); //seperates data by column
    dataFile.print(sensorValue); //prints sensor data in the second column
    dataFile.println();
    dataFile.close(); //closes the file
  }

  else {
    lcd.clear();
    lcd.setCursor(0, 0);
    lcd.print("Error Opening");
    lcd.setCursor(0, 1);
    lcd.print("File");
  }
}

The LogData function is called in a while loop in the main part of the code. While the record switch is high, log data. I could include my entire code. This is just what I thought was relevant.

When I run this code, I am getting a sample rate of around 80Hz. I want to identify my limiting factor.

The SD card should have a minimum write speed of 4mb/s. That seems like it should be more than enough.

I'm using the SD.h library, and I understand it is notoriously slow, but 80Hz seems VERY slow to me. Would switching to the sdfat library increase the speed that much?

Is the processor on the UNO not fast enough to crank through the SD formatting? Would switching to a DUE (or something else) be the best option?

Thanks for taking the time to read this. I appreciate any feedback!

2

michaelP13:
The SD card should have a minimum write speed of 4mb/s. That seems like it should be more than enough.

I think you mean "maximum" and not "minimum".

Nevertheless, 4 MB/s is most likely achievable on a USB SD card reader running the native SDIO protocol; SPI is slower (specially in low-clocked systems).
For an Arduino Uno, the top speed is 1 MB/s of raw I/O; but due to protocol and code overheads, the actual "low-level" write speed is somewhere around 80 KB/s. Handling the filesystem further decreases this number down to around 53 KB/s (in the best case).

michaelP13:
I'm using the SD.h library, and I understand it is notoriously slow, but 80Hz seems VERY slow to me. Would switching to the sdfat library increase the speed that much?

I'll say it's a minor improvement if any.

michaelP13:
Is the processor on the UNO not fast enough to crank through the SD formatting? Would switching to a DUE (or something else) be the best option?

Higher clock rates are usually better; however, an Uno would have more potential if the code is actually optimized for storage performance.

michaelP13:
The LogData function is called in a while loop in the main part of the code. While the record switch is high, log data. I could include my entire code. This is just what I thought was relevant.

When I run this code, I am getting a sample rate of around 80Hz. I want to identify my limiting factor.

As I said, an optimized code would have made the difference. Here some tips to log data in a looped manner:

  • Open the file before the loop, and close it after the loop. Every reopening attempt will cost you between 10 microseconds to a whole millisecond (it depends on how long it takes to find the file by name).
  • Avoid calling unrelated functions (e.g. printing to serial, doing stuff with the LCD screen), at least during the loop.
  • Avoid heavy calculations (like operating with float variables). Sometimes it's better to just log the raw data, and then do the calculations at the analysis stage.
  • Avoid unnecessary intermediate steps, like storing a readout in a local variable which value will be used only once in the loop.
  • If logging analog readouts is required, decrease the ADC's prescaler to its minimum.
  • If possible, write the data as raw binary. It's faster than printing it (but at a cost of human readability).

I hope these tips helped you in your quest.

Regards!

1 Like
3

I took your advice and moved the SD.open, dataFile.close, and the rest of the "junk" out of the loop; I also found an old DUE laying around. Between those two changes, my sample rate is now well over 1000.

Thanks for your help.

1 Like