Numbered Text Files

Hi, all!

I have a datalogger set up that names files sequentially. If the user stops recording and then re-records, the first file will be TLOG0 and the second will be TLOG1.

This works well until the user turns off the device and then turns it back on. The arduino will lose its place!

I see two solutions:

  • Store the fileNumber in a Nubmers.txt or something and read from that for the addendum
  • Get the number at the end of the last file

I'd prefer number two, but I don't know how to do it.

Any ideas? Thanks! - Runningman

Store the last number in EEPROM.

I did something similar and I selected your option number 1. mostly because it is relatively simple to implement and fast to get the counter value. I have a file called sysinit.dat in the SD card and it contains a binary data for a struct and the counter to the filename is one entry in the struct:

// Type for system initialization variables
typedef struct {
 // Arduino boot counter (incremented in every boot)
 uint16_t bootCount;

 // Variable for unique data file name on SD card
 uint16_t fileName;
} sysinit_t;

I update the file when needed (for example when I increment file counter value).

I did consider also using EEPROM but the project was hitting Arduino Uno's boundaries for example at the end I was using 31636 bytes of flash (out of 32256 bytes) and also RAM consumption was near end and because of that I could not use other libraries / functionalities that would increase flash/RAM usage.

My solution:

void start_new_file(){  
  //Getting current written number
 File trackFile = SD.open( "TRACK.TXT", FILE_WRITE );
  int decLoop = 0;
  int result = 0;
  while( trackFile.seek(decLoop) ){
    char charNum = trackFile.read();
    int myNumber = charNum-'0';
    if( myNumber >= 0 ){
    result *= pow( 10, decLoop );
    result += myNumber;
    }
    decLoop ++;
  }
  trackFile.close();
  SD.remove( "TRACK.TXT" );
  File newTrackFile = SD.open( "TRACK.TXT", FILE_WRITE );
  int nextResult = result + 1;
  newTrackFile.print( nextResult );
  newTrackFile.close();
  String fileName = "TLOG" + String( nextResult) + ".csv";
  char __fileName[sizeof(fileName) + (decLoop - 1) + 4]; 
  fileName.toCharArray(__fileName, sizeof(__fileName));
  myFile = SD.open( __fileName, FILE_WRITE );
}

This algorithm goes through a while loop and reads off one number at a time and then offsets it by the indexed power of ten to create the number, increments it, deletes the file, recreates an identical one, and saves it for the next time. Is this inefficient?

Runningman:
Is this inefficient?

There are many ways to do things and some might be proper solution for one project whereas the same solution is not proper one for other project. You seem to use SD.remove, which you can avoid and you use String class, which I do not use due to its excessive memory footprint etc… But as said if you use String class in several other locations as well then maybe it is the correct solution for your project.

Below is a summary of how I implemented file access related functionality. First the struct that is written to file as a binary data (not ASCII as you have) and an instance of it:

// Type for system initialization variables
typedef struct {
  // Arduino boot counter (incremented in every boot)
  uint16_t bootCount;

  // Variable for unique data file name on SD card
  uint16_t fileName;
} sysinit_t;

// Variable that holds system initialization data
sysinit_t sysinit;

Then my read functionality that reads the data directly to the struct from the SD card and is executed in every boot looks like:

    // Read the stored values from SD card
    File file = SD.open(SYSINIT_DAT);
    if (file) {
      uint8_t* pSysinit = (uint8_t*)&sysinit;
      while(file.available()) {
        *pSysinit++ = file.read();
      }
      file.close();
    }

After I have updated the content of the struct (for example incremented fileName value) I update the file, but instead of removing the old file I just overwrite the old content of the file:

  // Write the data (updated or initial values) back to SD card
  File file = SD.open(SYSINIT_DAT, FILE_WRITE);
  if (file) {
    file.seek(0); // Overwrite existing content instead of appending
    file.write((uint8_t*)&sysinit, sizeof(sysinit_t));
    file.close();
  }

Then the last part is the usage of fileName, because fileName is a plain number (uint16_t) I need to use specific tricks (itoa/strcpy/strlen) to convert that number to ASCII that can be used by the SD library:

// Buffer for filename (8 + . + 3 + 0x0)
char logFileNameBuf[13] = {0};

char* logFileName() {
  if (logFileNameBuf[0] == 0x0) {
    initLogFileName();
  }

  return logFileNameBuf;
}

void initLogFileName() {
  itoa(sysinit.fileName, logFileNameBuf, 10);
  strcpy(&logFileNameBuf[strlen(logFileNameBuf)], ".txt");
}

As you can see logFileName() function is the one that returns the actual filename that I use directly in SD.open calls. It returns the filename in the format “XXX.txt” where XXX is the unique number of the filename for example 123.txt

Here is a method I use for numbered files. It will take some time to find the name if you have many existing files.

I modified the SD ReadWrite example to demonstrate the method. The line with the SD.exists() statement checks for existing files.

/* SD ReadWrite example modified for file numbers by Bill Greiman */

const uint8_t SD_CS = 10;

#include <SPI.h>
#include <SD.h>

File myFile;

void setup()
{
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }
  Serial.print("Initializing SD card...");
  if (!SD.begin(SD_CS)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");

  // Find an unused file name

  const int LIMIT = 1000;
  char fileName[13];
  for (int n = 0; n < LIMIT; n++) {
    sprintf(fileName, "TLOG%.3d.TXT", n);
    if (SD.exists(fileName)) continue;
    myFile = SD.open(fileName, FILE_WRITE);
    break;      
  }
  // if the file opened okay, write to it:
  if (myFile) {
    Serial.print("Writing to");
    Serial.print(fileName);
    Serial.print("...");
    myFile.println("testing 1, 2, 3.");
    // close the file:
    myFile.close();
    Serial.println("done.");
  } else {
    // if the file didn't open, print an error:
    Serial.println("error opening test.txt");
  }

  // re-open the file for reading:
  myFile = SD.open(fileName);
  if (myFile) {
    Serial.print(fileName);
    Serial.println(':');

    // read from the file until there's nothing else in it:
    while (myFile.available()) {
      Serial.write(myFile.read());
    }
    // close the file:
    myFile.close();
  } else {
    // if the file didn't open, print an error:
    Serial.println("error opening test.txt");
  }
}

void loop()
{
  // nothing happens after setup
}

Here is output for a few cases:

Initializing SD card…initialization done.
Writing toTLOG000.TXT…done.
TLOG000.TXT:
testing 1, 2, 3.
Initializing SD card…initialization done.
Writing toTLOG001.TXT…done.
TLOG001.TXT:
testing 1, 2, 3.
Initializing SD card…initialization done.
Writing toTLOG002.TXT…done.
TLOG002.TXT:
testing 1, 2, 3.
Initializing SD card…initialization done.
Writing toTLOG003.TXT…done.
TLOG003.TXT:
testing 1, 2, 3.

sprintf uses several KB of flash. This example uses very little flash to generate a series of numbered file names.

// Increment ASCII number 
bool addOne(char* bgn, char* end) {
  while (1) {
    if (*end != '9') {
      *end += 1;
      return true;
    }
    *end = '0';
    if (end-- == bgn) return false;
  }
}
char fileName[] = "TEST000.TXT";
void setup() {
  Serial.begin(9600);
}

void loop() {
  Serial.println(fileName); 
  if (!addOne(&fileName[4], &fileName[6])) {
    Serial.println("Fail");
    while(1);
  }
}

Here is the first and last part of the series:

TEST000.TXT
TEST001.TXT
TEST002.TXT
TEST003.TXT
TEST004.TXT

TEST997.TXT
TEST998.TXT
TEST999.TXT
Fail