Bugs with SD file creation

Using Arduino Programming Questions
1

I'm using the Adafruit data logging shield to write log files to an SD card. Like the example code in the documentation, I want a new log file to be created every time the Arduino powers on. Originally I used a modified form of the example:

// create a new file
  char filename[] = "LOGGER00.CSV";
  for (uint8_t i = 0; i < 100; i++) {
    filename[6] = i/10 + '0';
    filename[7] = i%10 + '0';
    if (! SD.exists(filename)) {
      // only open a new file if it doesn't exist
      logfile = SD.open(filename, FILE_WRITE); 
      break;  // leave the loop!
    }
  }

However, my application needs to be able to support a thousand files instead of a hundred, so even with optimization, this loop was too slow. Instead, I am using a master file to keep track of which log file number I am on to make log file creation easy and fast.

I put the code in the loop function to test it. I should be able to start it and let it create every file from TEST000 to TEST999, but random file creation errors occur every time I run it. I tried including the loop method as a backup to run if the master file failed, but even with this backup, file creation often fails.

My full code is attached; it was a little too big for the post

Last time I ran the code, this was the readout:

Debug: filepath = TESTLOGS/TEST054.TXT
Opening file
Debug: logNumber = 055
Debug: logNumber[0] = 0
Debug: logNumber[1] = 5
Debug: logNumber[2] = 5
File creation successful
Debug: filepath = TESTLOGS/TEST055.TXT
Opening file
Debug: logNumber = 056
Debug: logNumber[0] = 0
Debug: logNumber[1] = 5
Debug: logNumber[2] = 6
File creation successful
Debug: filepath = TESTLOGS/TEST056.TXT
Opening file
Error: file creation failed, retrying...
Debug: filepath = TESTLOGS/TESTÿÿÿ.TXT
Error: file creation failed again, attempting recovery cycle
Attempting to recover
Main loop: 1
Secondary loop 1: 1
Secondary loop 1: 2
Secondary loop 1: 3
Secondary loop 1: 4
Secondary loop 1: 5
Secondary loop 1: 6
Tertiary loop 1a: 1
Tertiary loop 1a: 2
Tertiary loop 1a: 3
Tertiary loop 1a: 4
Tertiary loop 1a: 5
Tertiary loop 1a: 6
Tertiary loop 1a: 7
Attempting to create file: TESTLOGS/TEST057.TXT
Hello
Break 4
Error: File not created, trying again...
File creation failed, program terminated
File creation failed

It successfully created files 000-055, then failed on 056. It moved on to my recovery loop, which tried to create 057, but that also failed, so the program terminated. I have also had programs that randomly stop after a call to logFile = SD.open(filepath, FILE_WRITE). Another very common error is a read error from the master file, which did happen in the readout above. After failing to create file 056, it attempted to reread the number from the mater file. When it did so, some error occurred, because instead of getting "056" again, it ended up with "ÿÿÿ". I checked the master file on my computer, and it was not corrupted; it still contains the string "056". Also, both files 056 and 057 were created. It seems that the problem may be with assigning the file to the variable logFile.
I can't find anything in my code that would cause problems like this. It seems that they must be bugs, but if anyone has any ideas, I would greatly appreciate it.
Thanks!

SD_Library_Test.ino (9.59 KB)

2

In the code you posted, you make loads of calls to open a file e.g

    logFile = SD.open(filepath, FILE_WRITE);

...

    masterFile = SD.open("MASTER.TXT", FILE_WRITE);

...
    masterFile = SD.open("MASTER.TXT");

    logFile = SD.open(filepath, FILE_WRITE);


      masterFile = SD.open("MASTER.TXT");

But I can only see one instance where you have closed a file

      masterFile.close();

I suspect you have run out of file handles.

Did you really intend to have loads and loads of files open at the same time, or did you intend to log data to separate files and close after the data had been written
(in which case you are missing a load of close() commands)

Edit..

[See SD - Arduino Reference](http://See SD - Arduino Reference)
Note you don't need to call flush if you are about to close a file as close also flushes
SD - Arduino Reference

3

I would open the master.txt file when you startup the arduino, drop the number into a variable; then when you create a file update the variable and update the master.txt (just write to it, don't open it.) That way you have a lot less read writes to the sd card and if you reboot it will read in the current number from the master.

jasit.

4

So it turns out I had two problems. The first was that, as you pointed out, I had forgotten to call logFile.close(), so I fixed that. Once I fixed that, I ran the program again, and this time it crashed after about 160 files. All the memory card operations overflowed my SRAM. However, my real application only needs to open one new file each time it runs, so this won;t be an issue; the loop was just a test to make sure it named all the files correctly. Thanks for the help!