Hello everybody,
I use an ESP32 for the current project and must list files on the SD card in one place. However, that is very slow. I need about ~100 ms to ~120 ms "per file".
So about 12 seconds for 100 files.
Can this be normal? What do you have for times? I test with the following code:
...
#include <SPI.h>
#include <SD.h>
const uint8_t PIN_SD_CARD_CS = 5;
...
Serial.println(" - SD Card");
Serial.print (" init..."); while (!SD.begin(PIN_SD_CARD_CS, SPI, 10 * 1000 * 1000, "/sd")) { Serial.print("."); delay(1000); }
Serial.println("done");//SPI 16000000 Baud -> SD Card Read -> ~700kb/sec (5 MB File) / ~8000kb/sec (peeked 512 byte block)
...
File dir = SD.open("/mp3s");
uint16_t totalFileCount = 0;
if(dir){
dir.rewindDirectory();
File entry;
uint32_t enum_files_start = millis();
while((entry = dir.openNextFile())){
if(!entry.isDirectory()){ Serial.println(entry.name()); totalFileCount++; }
entry.close();
}
uint32_t enum_files_end = millis();
Serial.println("enum files duration " + (String)(enum_files_end - enum_files_start) + " ms");
Serial.println("total file count " + (String)totalFileCount);
}
dir.close();
PS: on the ESP32 I have to use "**/**mp3s" to specify files and directories. There seems to be a wrapper for the SD class expecting this.
Edit: With 20 Mhz SPI Speed (not only 10 Mhz) its a bit faster with ~85ms per file. But still too slow. 
-
And i have commented everything Inside the Loop. Serial and name() is not the problem. Reading Files is possible with 900kb/sec.
I have some Arduino Pro Mini here. I have done a few tests with the small 3.3V 8Mhz.
Just to check the behavior with a normal Arduino (not ESP32).
And ..... the result is very interesting. The Arduino Pro Mini behaves exactly like the ESP32. Sure, it is much slower .... but the times for listing the files are "relative" proportionally identical.
Each listing of a file slows down the next listing.
The first file can be retrieved quite quickly with openNextFile (). the second already a bit slower .... the third even slower ..... etc.
I read on the internet that some people blame the garbage collector. Anyway, it does not seem like a good idea to list many files.
Here are my measurements.
[files attached]
Found my solution: switch to SDfat library. It can list 99 files in ~112ms. (SD library 11994 ms)!
DrDooom:
Found my solution: switch to SDfat library. It can list 99 files in ~112ms. (SD library 11994 ms)!
Well, looks like SdFat is the updated version of the stock SD library (and it's weird that, in the new releases of the IDE, the first one is still not included instead of the old one; even when all the necessary wrappers are implemented to maintain compatibility with already existing sketches, except for name()).
Since you said that it takes longer the further it goes, makes me think two things:
- Card formatted as FAT32 (only for the root directory is a reason, otherwise it's the same as FAT16).
- It's like every time you call openNextFile(), rewindDirectory() is called as well; which (as you can see) is very inefficient.
DrDooom:
I read on the internet that some people blame the garbage collector.
Nonsense, even an ESP isn't powerful enough to afford a garbage collector.
Although some sort of dynamic memory management do exist, it's far away from being a gc. When triggered, a real gc in an Arduino or ESP may "freeze" the execution of the program (including interrupts to avoid race condition) from nearly one to a few seconds; which is not desirable in 99% of the cases.
DrDooom:
Found my solution: switch to SDfat library. It can list 99 files in ~112ms. (SD library 11994 ms)!
Hi DrDoom, I am wondering, how did you manage to work with the SdFat library and the ESP32... I have been trying to make them play together... but so far I couldnt come to a point where the SD card is recognized.
Would you mind sharing any necessary adaption or hints to take into account?
Thanks in advance!!
