Go Down

Topic: Scanning simple numeric filenames in SD card (Read 6452 times) previous topic - next topic

zeroneo

Hi,

I'm going to be creating a lot of files in a SD card with an arduino and I was thinking to use a simple numerial filename system. Start at 1.txt, then 2.txt, etc. The way I though about doing this is just to have a loop and check if a file exists, once it gets to a filename that does not exits, it creates a file with that name and runs the rest of the code.

This is how I was trying to achieve it:

Code: [Select]

 String fileName = String();
 word filenumber = 1;
 while(!filenumber==0) {
   fileName= filenumber + ".txt";
   if (SD.exists(FileName)) {
     fileName +=" exists.";
     Serial.println(fileName);
   }
   else {
     Serial.println("Creating file...");
     dataFile = SD.open(fileName, FILE_WRITE);
     filenumber = 0;
   }
 }


This doesn't compile and it says that there is not matching function for Sd.exists with arguments as (String*), instead it suggests to go for (char*). So I tried using the followings without luck:
Code: [Select]

char* charFileName = &fileName;
if (SD.exists(charFileName)) {

Code: [Select]

if (SD.exists(&FileName))

Code: [Select]

if (SD.exists((char *) &FileName)) {

Code: [Select]

if (SD.exists((char *) FileName)) {


As a last resort I could create a file just to keep track of the number of files in the SD cards, but I would ideally like to do it this way.

Thank you for your help!
   

PaulS

The String class has a toCharArray() method. Use that to extract the actual character data to supply to the SD function.
The art of getting good answers lies in asking good questions.

zeroneo

#2
Apr 04, 2011, 12:47 am Last Edit: Apr 04, 2011, 10:47 pm by zeroneo Reason: 1
Thanks!

That solved the compiling problem. Here is the modified code:

Code: [Select]

 I have updated the solution a couple of post below this.


I'm unable to test it today, but if it doesn't work tomorrow I'll update this.

Thanks again :)

robtillaart


Depending on how many files you need, you might consider to include subdirectories. I used such a strategy for a fast spooling system (LPR/LPD unix).

Instead of making file 6054.txt you create the path  \6\0\5\4.txt

The reason is that searching through the filesystem if one exists might be faster.

- assuming 10.000 files -
The idea is that if they are all in one directory you might have to go through all 10.000 entries before you know the file exist or not. On average you need half of it  (depends heavily on the app). If you use 3 layers of directories, testing max 10 entries per level makes a total of 40 entries. (~factor 200) Even for 100 files it is the difference between 100 and 20 (~factor 5). The difference in the numbers of entries to read is   10 ^ levels   versus   10 * levels  (Orders of magnitude)

You could do a similar trick with ASCII filenames. In general you apply a function on the filename to "calculate" the path. In the above numeric example the function is "split in digits"

You should write some benchmark tests to see how much the acces will improve as it is pretty access pattern dependant. Please post the results.











Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

zeroneo

#4
Apr 04, 2011, 10:46 pm Last Edit: Apr 04, 2011, 10:48 pm by zeroneo Reason: 1
I tested it today and it needed one or two changes. Here is the code I'm currently using:

Code: [Select]

 String fileName = String();
 unsigned int filenumber = 1;
 while(!filenumber==0) {
   fileName = "file_";
   fileName += filenumber;
   fileName += ".txt";
   char charFileName[fileName.length() + 1];
   fileName.toCharArray(charFileName, sizeof(charFileName));
   
   if (SD.exists(charFileName)) {
     fileName += " exists.";
     Serial.println(fileName);
     filenumber++;
   }
   else {
     dataFile = SD.open(charFileName, FILE_WRITE);
     fileName += " created.";  
     Serial.println(fileName);
     filenumber = 0;
   }
 }


About using subdirectories, it's a very good point and I didn't take that into consideration. It is true that if the number of files gets quite big I'll need to find a better way to scan through, but for the time being it's very unlikely to be checking more than a 100 files and I need to get the rest of the system working. If I finish with some time to spare will come back to this, user a few different algorithms and do a few benchmarks.

Thanks for all the input.

robtillaart

As string objects can be RAM intensive I rewrote the code snippet . It is not tested but it would give some idea how its done with char arrays only

Code: [Select]


  char fileName[16];                // SDcard uses 8.3 names so 16 bytes is enough NOte room for the '\0' char is needed!
  unsigned int nr = 1; 
 
  while (nr != 0)
  {
    sprintf(fileName, "file_%03d.txt", nr);
    if (SD.exists(fileName) == false) break;
    Serial.print(fileName);
    Serial.println(" exists.");
    nr++;
  }
  Serial.print(filename);
  Serial.println( " created.");
  dataFile = SD.open(fileName, FILE_WRITE);

...
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Go Up