Scanning simple numeric filenames in SD card

Hi,

I'm going to be creating a lot of files in a SD card with an arduino and I was thinking to use a simple numerial filename system. Start at 1.txt, then 2.txt, etc. The way I though about doing this is just to have a loop and check if a file exists, once it gets to a filename that does not exits, it creates a file with that name and runs the rest of the code.

This is how I was trying to achieve it:

  String fileName = String(); 
  word filenumber = 1;
  while(!filenumber==0) {
    fileName= filenumber + ".txt";
    if (SD.exists(FileName)) {
      fileName +=" exists.";
      Serial.println(fileName);
    }
    else {
      Serial.println("Creating file...");
      dataFile = SD.open(fileName, FILE_WRITE);
      filenumber = 0;
    }
  }

This doesn't compile and it says that there is not matching function for Sd.exists with arguments as (String*), instead it suggests to go for (char*). So I tried using the followings without luck:

char* charFileName = &fileName;
if (SD.exists(charFileName)) {
if (SD.exists(&FileName))
if (SD.exists((char *) &FileName)) {
if (SD.exists((char *) FileName)) {

As a last resort I could create a file just to keep track of the number of files in the SD cards, but I would ideally like to do it this way.

Thank you for your help!

The String class has a toCharArray() method. Use that to extract the actual character data to supply to the SD function.

Thanks!

That solved the compiling problem. Here is the modified code:

  I have updated the solution a couple of post below this.

I’m unable to test it today, but if it doesn’t work tomorrow I’ll update this.

Thanks again :slight_smile:

Depending on how many files you need, you might consider to include subdirectories. I used such a strategy for a fast spooling system (LPR/LPD unix).

Instead of making file 6054.txt you create the path \6\0\5\4.txt

The reason is that searching through the filesystem if one exists might be faster.

  • assuming 10.000 files - The idea is that if they are all in one directory you might have to go through all 10.000 entries before you know the file exist or not. On average you need half of it (depends heavily on the app). If you use 3 layers of directories, testing max 10 entries per level makes a total of 40 entries. (~factor 200) Even for 100 files it is the difference between 100 and 20 (~factor 5). The difference in the numbers of entries to read is 10 ^ levels versus 10 * levels (Orders of magnitude)

You could do a similar trick with ASCII filenames. In general you apply a function on the filename to "calculate" the path. In the above numeric example the function is "split in digits"

You should write some benchmark tests to see how much the acces will improve as it is pretty access pattern dependant. Please post the results.

I tested it today and it needed one or two changes. Here is the code I'm currently using:

  String fileName = String();
  unsigned int filenumber = 1;
  while(!filenumber==0) {
    fileName = "file_";
    fileName += filenumber;
    fileName += ".txt";
    char charFileName[fileName.length() + 1];
    fileName.toCharArray(charFileName, sizeof(charFileName));
    
    if (SD.exists(charFileName)) { 
      fileName += " exists.";
      Serial.println(fileName);
      filenumber++;
    }
    else {
      dataFile = SD.open(charFileName, FILE_WRITE);
      fileName += " created.";  
      Serial.println(fileName);
      filenumber = 0;
    }
  }

About using subdirectories, it's a very good point and I didn't take that into consideration. It is true that if the number of files gets quite big I'll need to find a better way to scan through, but for the time being it's very unlikely to be checking more than a 100 files and I need to get the rest of the system working. If I finish with some time to spare will come back to this, user a few different algorithms and do a few benchmarks.

Thanks for all the input.

As string objects can be RAM intensive I rewrote the code snippet . It is not tested but it would give some idea how its done with char arrays only

  char fileName[16];                // SDcard uses 8.3 names so 16 bytes is enough NOte room for the '\0' char is needed!
  unsigned int nr = 1;  
  
  while (nr != 0)
  {
    sprintf(fileName, "file_%03d.txt", nr);
    if (SD.exists(fileName) == false) break;
    Serial.print(fileName);
    Serial.println(" exists.");
    nr++;
  }
  Serial.print(filename);
  Serial.println( " created.");
  dataFile = SD.open(fileName, FILE_WRITE);

...