Finding available file name too slow with SD.exists()

I am taking pictures with TTL Serial JPEG cameras from adafruit:

To save to SD card, I have a simple code to find the next available file name:

  for (int i = 0; i < 10000; i++) 
  {
    sprintf(_f_name_buffer,"%04d.JPG",i); // Generate a serial number
    strcpy(_f_name+len, _f_name_buffer); // Add the number to the file name
    //Serial.print(F("Trying file name:"));
    //Serial.println(_f_name); // /DCIM/100ARDUI/IMGL0000.JPG
    // create if does not exist, do not open existing, write, sync after write
    if (! SD.exists(_f_name)) 
    {
      //Serial.println(F("File name available!"));
      return true;
    }
    //Serial.println(F("File already exists!"));
  }
  return false;

Essentially it generates a file name like "/DCIM/100ARDUI/IMGL0000.JPG", test with SD.exists() and returns true when it finds the next available file name.

This process is rather slow after Arduino has taken say a few hundred images overnight. Is there any faster way to do this? Maybe I should do this only once at the beginning and just create a new file name from there and tests that file name? Thanks.

create a text file with the last number used?

read it
add 1
save it
use it for jpeg name

Do a binary search of the name. If there can be up to 100 files per directory, a binary search would require at most 7 tests.

Pete

el_supremo:
Do a binary search of the name. If there can be up to 100 files per directory, a binary search would require at most 7 tests.

Pete

Sounds good. How do I do a binary search?

The search would be something like this. Assume that the function file_exists(int n,char *dirpath) checks whether the file name whose index is 'n' (e.g. IMGL0011.JPG when n=11) exists within the directory whose pathname is "dirpath" and returns non-zero if it does.
I'm also assuming that a directory will have no more than 100 files numbered from 0 to 99.

int find_file(char *dirpath)
{
  int lo,hi,mid;

  lo = 0;
  hi = 99;

  // Set up the initial conditions for the binary search that the first file (0) must exist
  // and the last one (99) must not
  
  // if index zero doesn't exist, it is the next file (or, at least, it indicates that
  // this directory is empty)
  if(!file_exists(lo,dirpath)) return(0);
  // if index 99 does exist then this directory is full, return an error/full indication
  if(file_exists(hi,dirpath))return(-1);

  do {
    mid = (hi-lo+1)/2;
    // If the mid-point file exists, change the lo pointer
    // otherwise change the hi pointer
    if(file_exists(mid,dirpath)) lo = mid;
    else hi = mid;
    // If hi and lo are just one apart, we've found the boundary between the two
    // and then next filename is hi
  } while(hi-lo > 1);
  return(hi);
}

If there are 1000 files per directory, just change "hi=99;" to "hi=999;". In that case, the number of tests will be a maximum of about 12 (including the two initial tests of 0 and 999).

Pete

Thanks. I had a misunderstanding. I thought "binary search" was reading some raw binary numbers from SD card and search within them. So it is binary search in general that you are talking about :slight_smile:

Yes. It just improves on a linear search through all possible filenames. I don't know anything about the internals of the SD directory structure :slight_smile:

Pete

Here is a sketch that finds the next file name in optimal time, one pass through the directory file.

This sketch will create ten empty files each time it is run.

#include <SD.h>

#define IMAGE_DIR "/DCIM/100ARDUI"

// SD chip select pin
const uint8_t SD_CS_PIN = SS;

// 
// return next file index or negative for error
int nextNumber() {
  dir_t dir;
  File dirFile;
  int maxIndex = -1;
  
  // open directory file
  dirFile = SD.open(IMAGE_DIR, O_READ);

  if (!dirFile){
    // open of dir file failed
    return -1;
  }
  // read all entries in directory
  while (dirFile.available()) {
    if (dirFile.read(&dir, 32) != 32) {
      dirFile.close();
      // read of dir file failed
      return -2;
    }
    // break if all entries read
    if (dir.name[0] == 0) break;
    
    // check for IMGLnnnn.JPG
    if (strncmp((char*)dir.name, "IMGL", 4) 
      || strncmp((char*)&dir.name[8], "JPG", 3)) {
      continue;
    } 
    int n = 0;
    for (int i = 4; i < 8; i++) {
      int c = dir.name[i] - '0';
      if (c < 0 || c > 9) {
        n = -1;
        break;
      }
      n = 10*n + c;
    }
    if (n > maxIndex) maxIndex = n;
  }
  dirFile.close();
  return maxIndex + 1;
}


void setup() {
  char path[32];
  File image;

  Serial.begin(9600);
 
  Serial.println("type a character to run");
  while (!Serial.available());
  if(!SD.begin(SD_CS_PIN)) {
    Serial.println("begin");
    while(1);
  }
  for (int i = 0; i < 10; i++) {
    int n = nextNumber();
    if (n < 0) {
      Serial.print("ERROR ");
      Serial.println(n);
      while(1);
    }
    sprintf(path, "%s/IMGL%04d.JPG", IMAGE_DIR, n);
    Serial.println(path);
    image = SD.open(path, FILE_WRITE);
    if (!image) {
      Serial.println("open image");
      while(1);
    }
    image.close();
  }
}

void loop() {
}

Here is typical output:

type a character to run
/DCIM/100ARDUI/IMGL0021.JPG
/DCIM/100ARDUI/IMGL0022.JPG
/DCIM/100ARDUI/IMGL0023.JPG
/DCIM/100ARDUI/IMGL0024.JPG
/DCIM/100ARDUI/IMGL0025.JPG
/DCIM/100ARDUI/IMGL0026.JPG
/DCIM/100ARDUI/IMGL0027.JPG
/DCIM/100ARDUI/IMGL0028.JPG
/DCIM/100ARDUI/IMGL0029.JPG
/DCIM/100ARDUI/IMGL0030.JPG

Thank you fat16lib! I forgot the subfolders are files themselves. I used to know enough fat12 to do stuff to floppies :slight_smile: