Arduino Forum

Using Arduino => Storage => Topic started by: MaxG on Jul 01, 2017, 12:13 pm

Title: SD Card: dynamic file name
Post by: MaxG on Jul 01, 2017, 12:13 pm
I'd like to write a new file to the SD card. every day
The file name is derived from the real time clock, to like so YYYYMMDD.CSV
However, while the file name prints on the serial monitor, the file is not created on the SD card and responds with "error opening". When I hard code the file name it works.

What I am I missing?
Any hint appreciated.


Code: [Select]

#include <Wire.h>
#include <DS3231.h>                               // RTC library
#include <SPI.h>
#include <SD.h>

#define PIN_SD_CHIP_SELECT        4

const unsigned int BAUD_RATE_HW          =  9600; // hardware serial baud rate
const unsigned long MEASUREMENT_INTERVAL = 10000; // [ms] time interval for taking measurements

DS3231 clock;                                     // RTC
RTCDateTime dt;                                   // instantiate RTC

File dataFile;                                    // file to write to




void logData() {
  char fileName[13];
  strcat(fileName, clock.dateFormat("Ymd", dt));
  strcat(fileName, ".CSV");
  Serial.print("fileName: ");
  Serial.println(fileName);

  dataFile = SD.open(fileName, FILE_WRITE);

  if (dataFile) {
    Serial.print(F("-> Writing data to log file: "));
    Serial.println(fileName);
    dataFile.println("recordOut");
    dataFile.close();
  } else {
    Serial.println(F("-> Error opening "));
  }
}

void setup() {

  pinMode(PIN_SD_CHIP_SELECT, OUTPUT);
  pinMode(SS, OUTPUT);                            // Docu says to do this even if not used

  Serial.begin(BAUD_RATE_HW);                     // start hardware serial port with 9600 bps
  delay(500);

  Wire.begin();

  Serial.println(F("-> Clock initialised ..."));
  clock.begin();                                  // Initialise clock object


  Serial.print(F("-> Initialising SD card ... "));

  if (!SD.begin(PIN_SD_CHIP_SELECT)) {
    Serial.println(F("initialisation failed!"));
    return;
  }
  Serial.println(F("initialisation done."));
}


void loop() {

  static unsigned long lastTimeMeassured = 0;     // last time measured value
  unsigned long timeStamp = millis();             // memorise current time

  if (timeStamp - lastTimeMeassured > MEASUREMENT_INTERVAL) {
    lastTimeMeassured = timeStamp;                // present time
    dt = clock.getDateTime();                     // get RTC datum
    logData();
  }
}
Title: Re: SD Card: dynamic file name
Post by: MaxG on Jul 01, 2017, 12:22 pm
Adding:

Code: [Select]

  fileName[0] = '\0';


after

Code: [Select]

  char fileName[13];


solved the issue. (so I thought, unfortunately not, see further posts)

I thought I leave the post; maybe someone else can use it.
Title: Re: SD Card: dynamic file name
Post by: MaxG on Jul 02, 2017, 12:25 am
However...
When the day changes, it creates an error once only...

[later edit / update: but does not create this error again, despite not writing a file at all...]

Code: [Select]

-> Writing data to log file: 20170701.CSV
-> Error opening: 20170702.CSV
-> Writing data to log file: 20170702.CSV


The docu says that if the file does not exist it will be created.
How can I avoid this error?
Title: Re: SD Card: dynamic file name
Post by: chucktodd on Jul 02, 2017, 03:39 am
However...
When the day changes, it creates an error once only.

Code: [Select]

-> Writing data to log file: 20170701.CSV
-> Error opening: 20170702.CSV
-> Writing data to log file: 20170702.CSV


The docu says that if the file does not exist it will be created.
How can I avoid this error?

char filename[13];


allocates 13 bytes in RAM.  20170702.CSV/0 is more than 13 characters long, it is actually 14.  

You have to count the trailing null so your variable declaration should be at least:

char filename[14];


When you did your strcat() operations, it kept pushing the null (/0) to the right, it messed up something when it moved outside of its defined storage space.

 
Chuck.
Title: Re: SD Card: dynamic file name
Post by: MaxG on Jul 02, 2017, 04:53 am
Thank you for your reply!

Hmm, at first I thought: darn, have I been caught out again with this nasty mistake; but the 8.3 file name is 12 chars long plus \0 = 13.

Code: [Select]

123456789012
20170702.csv


I have done further testing by using the standard examples; list file, readwrite, card info... they all work fine.
So it must be something in my code.

I have also deleted a file, after that, the directory (on Windows) looks like this -- see attachment...

I have no further news at present, but will try the SDfat library... and report back.


Title: Re: SD Card: dynamic file name
Post by: chucktodd on Jul 02, 2017, 06:58 am
Thank you for your reply!

Hmm, at first I thought: darn, have I been caught out again with this nasty mistake; but the 8.3 file name is 12 chars long plus \0 = 13.

Code: [Select]

123456789012
20170702.csv


I have done further testing by using the standard examples; list file, readwrite, card info... they all work fine.
So it must be something in my code.

I have also deleted a file, after that, the directory (on Windows) looks like this -- see attachment...

I have no further news at present, but will try the SDfat library... and report back.
You're right, I cannot count 8 + 4 =12 +1 = 13.

Somehow my brain was seeing a reference to filename[13], which is actually the 14th character.

Sorry,

Now, on a closer inspection, I have concerns on your DS3231 library.  You reference DS3231.h, who's library are you using?

the reason I ask is that this function call:

strcat(fileName, clock.dateFormat("Ymd", dt));


Has me really concerned. I found a github library https://github.com/jarzebski/Arduino-DS3231 (https://github.com/jarzebski/Arduino-DS3231) that seems equivalent.

Looking at it's dateFormat() function causes shivers up my spine.

Basically their code is like this:
Code: [Select]

char * dateFormat(const char *formatstring, RTCDateTime td){
char buffer[256]

//fill buffer according to formatstring[]

return buffer;
}


First,they are allocating a large buffer 256 bytes, 1/8th of the UNO's Ram. UNO's only have 2048 total bytes of RAM.

Second, they return a pointer to a buffer that no longer exists? they hope that nothing else has disturbed the stack before you use this formatted value?

Your weird random behavior may be a result of someone's poor programming habits.

Add this function to your sketch, call it and display the results.  It will give you a instantaneous free memory count in bytes.  see if you are running out of RAM.
Code from AdaFruit: Measuring Memory Usage (https://learn.adafruit.com/memories-of-an-arduino/measuring-free-memory)
Code: [Select]

int freeRam ()
{
  extern int __heap_start, *__brkval;
  int v;
  return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
}


Chuck.

Title: Re: SD Card: dynamic file name
Post by: MaxG on Jul 02, 2017, 09:48 am
Wow... great spotting...

I was thinking of not using any library for the DS3231 (I think I remember it chewed up an extra 2k)...

Well, in order to isolate the problem, I will go back to basics, and remove any reference to the DS3231... and start over...
... will report back...

Title: Re: SD Card: dynamic file name
Post by: PaulS on Jul 03, 2017, 12:43 pm
Using the DS3231 library to get/set the time is fine. Just don't use any functions to get information as strings.

Use sprintf() and your own buffer and format specifiers and variables to get the string you want.

Code: [Select]
int year = 2017; // Get these from the clock
int month = 7;
int day = 3;

char fileName[13];
sprintf(fileName, "%4d%2d%2d.CSV", year, month, day);
Title: Re: SD Card: dynamic file name
Post by: el_supremo on Jul 03, 2017, 06:24 pm
A slight correction/mod. I always use:
Code: [Select]
sprintf(fileName, "%4d%02d%02d.CSV", year, month, day);
so that the month and day are always 2 digits long.

Pete
Title: Re: SD Card: dynamic file name
Post by: MaxG on Jul 07, 2017, 04:56 am
Thanks for the replies :)
Very helpful...
I removed the library and the problem is gone.
Since I had to deploy the data logger the next day, I had no chance to dig deeper into the problem.
Title: Re: SD Card: dynamic file name
Post by: Nick_Pyner on Jul 07, 2017, 05:51 am
I was thinking of not using any library for the DS3231 (I think I remember it chewed up an extra 2k)...
2k sounds a lot, but I have seen the use of libraries being rather inefficient. I just want time so I stick with
http://bildr.org/2011/03/ds1307-arduino/
No change was required in order to use the DS3231. I use the same as in reply #8, and start a new file at midnight.
Title: Re: SD Card: dynamic file name
Post by: Graseo on Apr 15, 2019, 02:49 pm
Wow... great spotting...

I was thinking of not using any library for the DS3231 (I think I remember it chewed up an extra 2k)...

Well, in order to isolate the problem, I will go back to basics, and remove any reference to the DS3231... and start over...
... will report back...


can you share the program that you have made? I happen to be doing a similar project now
Title: Re: SD Card: dynamic file name
Post by: Geek Emeritus on Apr 15, 2019, 08:37 pm
this is how I did that:

way up at the beginning:

Code: [Select]
File dataFile;                      // variable for SD
char filename[]  = "00000000.CSV";  // filemame array for datalogger
char logString[] = "000000000000";  // build a string for the event log


this:
Quote
char logString[] = "000000000000";  // build a string for the event log
is not needed for deriving the file name, it is just included to make the rest of the code understandable

code to set up SD card not included

the routine that derives the filename is called every time an event is logged. it gets the date from LocalTime: time_t - the offset from UTC and the offset for DST if necessary:

Code: [Select]
///////////////////////////////////////////////////////////////////////////////////////////////
// SD & DATALOGGER FUNCTIONS //////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////////////////

void getFilename( time_t LocalTime )
     {         
      sprintf(filename, "%04d%02d%02d", year(LocalTime), month(LocalTime), day(LocalTime));
     }
           
void logEvent(int sw)
{
 time_t eventTime = LocalTime();
      {
       getFilename( eventTime );
       lcd4.setCursor(3,0); lcd4.print("Filename:"); lcd4.print(  filename );
       File dataFile = SD.open(filename, FILE_WRITE);
       if (dataFile)                                                   // if the file is available, write to it:
         {
          lcd4.setCursor(2,0); lcd4.print("O");
          sprintf(logString, "%02d,%02d,%02d,%02d", sw,
          hour(eventTime), minute(eventTime), second(eventTime));
          dataFile.println(logString);
          dataFile.close();                                           // close the file after writing
          lcd4.setCursor(2,0); lcd4.print("*"); lcd4.setCursor(9,1); lcd4.print(logString);
         }
       
 /*     dataFile = SD.open(filename);
       
       if (dataFile)                                                  // if the file is available, read from it:
       {
         while (dataFile.available())
         {
          Serial.write(dataFile.read());
         }
         {
          dataFile.close();
          Serial.print( "data file "); Serial.print(filename); Serial.println( " closed after reading" );
         }
       }       
       
       else
       {
        Serial.print("error opening "); Serial.println(filename);                  //if the file isn't open, pop up an error:     
       }*/
 
   }

}


this:

Code: [Select]
lcd4.setCursor(3,0); lcd4.print("Filename:"); lcd4.print(  filename );
lcd4.setCursor(2,0); lcd4.print("*"); lcd4.setCursor(9,1); lcd4.print(logString);


is a status display and an easter egg, not necessary for deriving the filename. the file status is preceded by "SD:" on the display.

the part between /* & */ was for troubleshooting and verifying. not needed, but there is no benefit to throwing it away

sw is the loop counter for the loop that reads the sensor array.

note that
Quote
char filename[]  = "00000000.CSV";  // filemame array for datalogger
is 12 characters and it works just fine all day every day