SPI Speed Configurability for writing to SD card

Hello Everyone,

I am working with an ESP32 Dev Kit (using ESP32 WROOM 32E) to be exact. The goal is to write to an SD card at the rate of around 12MegaBITS per second. So far i've just been using the default SPI with the default SD.h example. The serial monitor says i'm writing 1048576 bytes in around 4000-5000 ms. If I did the math right, thats only around 2Mbps where I need 12 as stated earlier. I've seen on other forums that people recommend using the SDFat library instead of the arduino SD library. However, shouldn't SPI clock speed be the difference and not the library being used?

Because of that speculation, I tried altering the frequency in the SD.h file from 1400000 to 12000000 and it only made the sd write test faster by a second which translates to 3 Mbps instead of 2 Mbps. Still nowhere close to the speed i need. I've also tried increasing the frequency even higher than that but it doesnt get any faster. Since the ESP32 has a clock speed of 240MHz and my SD card has a speed of 70MB/s, I doubt its a hardware issue. Does anyone have any other ideas?

This is my altered SD.h file that made the sd write a second quicker. All other code is the default arduino SD example code.
#ifndef SD_H
#define SD_H

#include "FS.h"
#include "SPI.h"
#include "sd_defines.h"

namespace fs
{

class SDFS : public FS
{
protected:
uint8_t _pdrv;

public:
SDFS(FSImplPtr impl);
bool begin(uint8_t ssPin=SS, SPIClass &spi=SPI, uint32_t frequency=12000000, const char * mountpoint="/sd", uint8_t max_files=5);
void end();
sdcard_type_t cardType();
uint64_t cardSize();
uint64_t totalBytes();
uint64_t usedBytes();
};

It is very likely the speed of SD card needs to be changed dynamically and not statically. Because SD cards and devices that use them come from all kinds of manufacturer and the technology evolves over time there must be a mechanism to adapt the speed.

So, it is likely SD cards start with a lowest guaranteed speed and then the host and the SD card figure out how high each of them can go.

A SD card library might choose to just use the default that works in all cases and not implement support for higher speed. So, using another SD library could be the solution. But that is just a guess. Maybe it helps looking for differences in the libraries.

I am curious is the SD card rated at that speed or higher? Can you operate it at that speed in another device. I would suggest spending a few shekels and purchasing a 8 channel logic analyzer, rated at 24Mhz, from your favorite china supplier, they go for less then $10.00 ea and work via USB. This would allow you to validate the SP clock speed. A link for the SD card would be nice.

Klaus_K:
A SD card library might choose to just use the default that works in all cases and not implement support for higher speed. So, using another SD library could be the solution. But that is just a guess. Maybe it helps looking for differences in the libraries.

This is a valid idea that I will investigate more. As stated I already tried tampering with the Arduino library but its not very editor friendly, it took quite a while just to edit the .h files XD. Thank you for your response

gilshultz:
I am curious is the SD card rated at that speed or higher? Can you operate it at that speed in another device. I would suggest spending a few shekels and purchasing a 8 channel logic analyzer, rated at 24Mhz, from your favorite china supplier, they go for less then $10.00 ea and work via USB. This would allow you to validate the SP clock speed. A link for the SD card would be nice.

It probably wouldnt be a bad idea just to double check the SD card could handle the speeds, but the rating on the SD cards are always the minimum speeds that the manufacturer guarantees. In my case, thats 70MB/s which equates to 560Mbps. I only need 12Mbps XD, so I definitely think its a host issue. Thanks for your input

Is that 560Mbs the quoted speed of the card in SPI mode ?

One of the main limitations in how fast you write to an SD card is the way the code is structured. If, for example, you open and close the file after every few I/O operations then that will severely limit the performance (we keep seeing examples like that). Try posting up the code you are using (follow the forum guidelines and post it in a code block using </> from the menu for example).

srnet:
Is that 560Mbs the quoted speed of the card in SPI mode ?

I dont know about SPI mode, all I know is that the sd card says 70MB/s on it (Note that means Bytes not bits). That would make me think it could handle SPI clocks around 560MHz since one clock cycle equals one bit in SPI communication.

countrypaul:
One of the main limitations in how fast you write to an SD card is the way the code is structured. If, for example, you open and close the file after every few I/O operations then that will severely limit the performance (we keep seeing examples like that). Try posting up the code you are using (follow the forum guidelines and post it in a code block using </> from the menu for example).

#include "FS.h"
#include "SD.h"
#include "SPI.h"

void listDir(fs::FS &fs, const char * dirname, uint8_t levels){
    Serial.printf("Listing directory: %s\n", dirname);

    File root = fs.open(dirname);
    if(!root){
        Serial.println("Failed to open directory");
        return;
    }
    if(!root.isDirectory()){
        Serial.println("Not a directory");
        return;
    }

    File file = root.openNextFile();
    while(file){
        if(file.isDirectory()){
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if(levels){
                listDir(fs, file.name(), levels -1);
            }
        } else {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("  SIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void createDir(fs::FS &fs, const char * path){
    Serial.printf("Creating Dir: %s\n", path);
    if(fs.mkdir(path)){
        Serial.println("Dir created");
    } else {
        Serial.println("mkdir failed");
    }
}

void removeDir(fs::FS &fs, const char * path){
    Serial.printf("Removing Dir: %s\n", path);
    if(fs.rmdir(path)){
        Serial.println("Dir removed");
    } else {
        Serial.println("rmdir failed");
    }
}

void readFile(fs::FS &fs, const char * path){
    Serial.printf("Reading file: %s\n", path);

    File file = fs.open(path);
    if(!file){
        Serial.println("Failed to open file for reading");
        return;
    }

    Serial.print("Read from file: ");
    while(file.available()){
        Serial.write(file.read());
    }
    file.close();
}

void writeFile(fs::FS &fs, const char * path, const char * message){
    Serial.printf("Writing file: %s\n", path);

    File file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("Failed to open file for writing");
        return;
    }
    if(file.print(message)){
        Serial.println("File written");
    } else {
        Serial.println("Write failed");
    }
    file.close();
}

void appendFile(fs::FS &fs, const char * path, const char * message){
    Serial.printf("Appending to file: %s\n", path);

    File file = fs.open(path, FILE_APPEND);
    if(!file){
        Serial.println("Failed to open file for appending");
        return;
    }
    if(file.print(message)){
        Serial.println("Message appended");
    } else {
        Serial.println("Append failed");
    }
    file.close();
}

void renameFile(fs::FS &fs, const char * path1, const char * path2){
    Serial.printf("Renaming file %s to %s\n", path1, path2);
    if (fs.rename(path1, path2)) {
        Serial.println("File renamed");
    } else {
        Serial.println("Rename failed");
    }
}

void deleteFile(fs::FS &fs, const char * path){
    Serial.printf("Deleting file: %s\n", path);
    if(fs.remove(path)){
        Serial.println("File deleted");
    } else {
        Serial.println("Delete failed");
    }
}

void testFileIO(fs::FS &fs, const char * path){
    File file = fs.open(path);
    static uint8_t buf[512];
    size_t len = 0;
    uint32_t start = millis();
    uint32_t end = start;
    if(file){
        len = file.size();
        size_t flen = len;
        start = millis();
        while(len){
            size_t toRead = len;
            if(toRead > 512){
                toRead = 512;
            }
            file.read(buf, toRead);
            len -= toRead;
        }
        end = millis() - start;
        Serial.printf("%u bytes read for %u ms\n", flen, end);
        file.close();
    } else {
        Serial.println("Failed to open file for reading");
    }


    file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("Failed to open file for writing");
        return;
    }

    size_t i;
    start = millis();
    for(i=0; i<2048; i++){
        file.write(buf, 512);
    }
    end = millis() - start;
    Serial.printf("%u bytes written for %u ms\n", 2048 * 512, end);
    file.close();
}

void setup(){
    Serial.begin(115200);
    if(!SD.begin()){
        Serial.println("Card Mount Failed");
        return;
    }
    uint8_t cardType = SD.cardType();

    if(cardType == CARD_NONE){
        Serial.println("No SD card attached");
        return;
    }

    Serial.print("SD Card Type: ");
    if(cardType == CARD_MMC){
        Serial.println("MMC");
    } else if(cardType == CARD_SD){
        Serial.println("SDSC");
    } else if(cardType == CARD_SDHC){
        Serial.println("SDHC");
    } else {
        Serial.println("UNKNOWN");
    }

    uint64_t cardSize = SD.cardSize() / (1024 * 1024);
    Serial.printf("SD Card Size: %lluMB\n", cardSize);

    listDir(SD, "/", 0);
    createDir(SD, "/mydir");
    listDir(SD, "/", 0);
    removeDir(SD, "/mydir");
    listDir(SD, "/", 2);
    writeFile(SD, "/hello.txt", "Hello ");
    appendFile(SD, "/hello.txt", "World!\n");
    readFile(SD, "/hello.txt");
    deleteFile(SD, "/foo.txt");
    renameFile(SD, "/hello.txt", "/foo.txt");
    readFile(SD, "/foo.txt");
    testFileIO(SD, "/test.txt");
    Serial.printf("Total space: %lluMB\n", SD.totalBytes() / (1024 * 1024));
    Serial.printf("Used space: %lluMB\n", SD.usedBytes() / (1024 * 1024));
}

void loop(){

}

Hey countrypaul, thanks for your response. I am using the default SD code for the ESP32, I understand your concern about opening and closing files while writing, however in the “testFileIO” function, it tests the read/write speeds uninterrupted with open and close file.

We never know what is being used, so always easiest to assume the simplest option to start with.

What is the card you are writing to? Make, Size, Model

I know there was an sdFatEx which was supposed to be about 10x as fast as sdFat at one time, not sure if the 2 have been combined now or quite what the status is. I certainly don't know at present if you can make use of that through the SD wrapper. It may be worth you looking into though - if you can't find any other options.

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.