Go Down

Topic: SD Card: dynamic file name (Read 2837 times) previous topic - next topic

MaxG

I'd like to write a new file to the SD card. every day
The file name is derived from the real time clock, to like so YYYYMMDD.CSV
However, while the file name prints on the serial monitor, the file is not created on the SD card and responds with "error opening". When I hard code the file name it works.

What I am I missing?
Any hint appreciated.


Code: [Select]

#include <Wire.h>
#include <DS3231.h>                               // RTC library
#include <SPI.h>
#include <SD.h>

#define PIN_SD_CHIP_SELECT        4

const unsigned int BAUD_RATE_HW          =  9600; // hardware serial baud rate
const unsigned long MEASUREMENT_INTERVAL = 10000; // [ms] time interval for taking measurements

DS3231 clock;                                     // RTC
RTCDateTime dt;                                   // instantiate RTC

File dataFile;                                    // file to write to




void logData() {
  char fileName[13];
  strcat(fileName, clock.dateFormat("Ymd", dt));
  strcat(fileName, ".CSV");
  Serial.print("fileName: ");
  Serial.println(fileName);

  dataFile = SD.open(fileName, FILE_WRITE);

  if (dataFile) {
    Serial.print(F("-> Writing data to log file: "));
    Serial.println(fileName);
    dataFile.println("recordOut");
    dataFile.close();
  } else {
    Serial.println(F("-> Error opening "));
  }
}

void setup() {

  pinMode(PIN_SD_CHIP_SELECT, OUTPUT);
  pinMode(SS, OUTPUT);                            // Docu says to do this even if not used

  Serial.begin(BAUD_RATE_HW);                     // start hardware serial port with 9600 bps
  delay(500);

  Wire.begin();

  Serial.println(F("-> Clock initialised ..."));
  clock.begin();                                  // Initialise clock object


  Serial.print(F("-> Initialising SD card ... "));

  if (!SD.begin(PIN_SD_CHIP_SELECT)) {
    Serial.println(F("initialisation failed!"));
    return;
  }
  Serial.println(F("initialisation done."));
}


void loop() {

  static unsigned long lastTimeMeassured = 0;     // last time measured value
  unsigned long timeStamp = millis();             // memorise current time

  if (timeStamp - lastTimeMeassured > MEASUREMENT_INTERVAL) {
    lastTimeMeassured = timeStamp;                // present time
    dt = clock.getDateTime();                     // get RTC datum
    logData();
  }
}
Mainly using UNOs. Everything needs to be defined.

MaxG

#1
Jul 01, 2017, 12:22 pm Last Edit: Jul 02, 2017, 05:00 am by MaxG
Adding:

Code: [Select]

  fileName[0] = '\0';


after

Code: [Select]

  char fileName[13];


solved the issue. (so I thought, unfortunately not, see further posts)

I thought I leave the post; maybe someone else can use it.
Mainly using UNOs. Everything needs to be defined.

MaxG

#2
Jul 02, 2017, 12:25 am Last Edit: Jul 02, 2017, 05:01 am by MaxG
However...
When the day changes, it creates an error once only...

[later edit / update: but does not create this error again, despite not writing a file at all...]

Code: [Select]

-> Writing data to log file: 20170701.CSV
-> Error opening: 20170702.CSV
-> Writing data to log file: 20170702.CSV


The docu says that if the file does not exist it will be created.
How can I avoid this error?
Mainly using UNOs. Everything needs to be defined.

chucktodd

However...
When the day changes, it creates an error once only.

Code: [Select]

-> Writing data to log file: 20170701.CSV
-> Error opening: 20170702.CSV
-> Writing data to log file: 20170702.CSV


The docu says that if the file does not exist it will be created.
How can I avoid this error?

char filename[13];


allocates 13 bytes in RAM.  20170702.CSV/0 is more than 13 characters long, it is actually 14.  

You have to count the trailing null so your variable declaration should be at least:

char filename[14];


When you did your strcat() operations, it kept pushing the null (/0) to the right, it messed up something when it moved outside of its defined storage space.

 
Chuck.
Currently built mega http server, Now converting it to ESP32.

MaxG

Thank you for your reply!

Hmm, at first I thought: darn, have I been caught out again with this nasty mistake; but the 8.3 file name is 12 chars long plus \0 = 13.

Code: [Select]

123456789012
20170702.csv


I have done further testing by using the standard examples; list file, readwrite, card info... they all work fine.
So it must be something in my code.

I have also deleted a file, after that, the directory (on Windows) looks like this -- see attachment...

I have no further news at present, but will try the SDfat library... and report back.


Mainly using UNOs. Everything needs to be defined.

chucktodd

#5
Jul 02, 2017, 06:58 am Last Edit: Jul 02, 2017, 07:05 am by chucktodd Reason: add memory free space calc
Thank you for your reply!

Hmm, at first I thought: darn, have I been caught out again with this nasty mistake; but the 8.3 file name is 12 chars long plus \0 = 13.

Code: [Select]

123456789012
20170702.csv


I have done further testing by using the standard examples; list file, readwrite, card info... they all work fine.
So it must be something in my code.

I have also deleted a file, after that, the directory (on Windows) looks like this -- see attachment...

I have no further news at present, but will try the SDfat library... and report back.
You're right, I cannot count 8 + 4 =12 +1 = 13.

Somehow my brain was seeing a reference to filename[13], which is actually the 14th character.

Sorry,

Now, on a closer inspection, I have concerns on your DS3231 library.  You reference DS3231.h, who's library are you using?

the reason I ask is that this function call:

strcat(fileName, clock.dateFormat("Ymd", dt));


Has me really concerned. I found a github library https://github.com/jarzebski/Arduino-DS3231 that seems equivalent.

Looking at it's dateFormat() function causes shivers up my spine.

Basically their code is like this:
Code: [Select]

char * dateFormat(const char *formatstring, RTCDateTime td){
char buffer[256]

//fill buffer according to formatstring[]

return buffer;
}


First,they are allocating a large buffer 256 bytes, 1/8th of the UNO's Ram. UNO's only have 2048 total bytes of RAM.

Second, they return a pointer to a buffer that no longer exists? they hope that nothing else has disturbed the stack before you use this formatted value?

Your weird random behavior may be a result of someone's poor programming habits.

Add this function to your sketch, call it and display the results.  It will give you a instantaneous free memory count in bytes.  see if you are running out of RAM.
Code from AdaFruit: Measuring Memory Usage
Code: [Select]

int freeRam ()
{
  extern int __heap_start, *__brkval;
  int v;
  return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
}


Chuck.

Currently built mega http server, Now converting it to ESP32.

MaxG

Wow... great spotting...

I was thinking of not using any library for the DS3231 (I think I remember it chewed up an extra 2k)...

Well, in order to isolate the problem, I will go back to basics, and remove any reference to the DS3231... and start over...
... will report back...

Mainly using UNOs. Everything needs to be defined.

PaulS

Using the DS3231 library to get/set the time is fine. Just don't use any functions to get information as strings.

Use sprintf() and your own buffer and format specifiers and variables to get the string you want.

Code: [Select]
int year = 2017; // Get these from the clock
int month = 7;
int day = 3;

char fileName[13];
sprintf(fileName, "%4d%2d%2d.CSV", year, month, day);
The art of getting good answers lies in asking good questions.

el_supremo

A slight correction/mod. I always use:
Code: [Select]
sprintf(fileName, "%4d%02d%02d.CSV", year, month, day);
so that the month and day are always 2 digits long.

Pete
Don't send me technical questions via Private Message.

MaxG

Thanks for the replies :)
Very helpful...
I removed the library and the problem is gone.
Since I had to deploy the data logger the next day, I had no chance to dig deeper into the problem.
Mainly using UNOs. Everything needs to be defined.

Nick_Pyner

I was thinking of not using any library for the DS3231 (I think I remember it chewed up an extra 2k)...
2k sounds a lot, but I have seen the use of libraries being rather inefficient. I just want time so I stick with
http://bildr.org/2011/03/ds1307-arduino/
No change was required in order to use the DS3231. I use the same as in reply #8, and start a new file at midnight.

Graseo

Wow... great spotting...

I was thinking of not using any library for the DS3231 (I think I remember it chewed up an extra 2k)...

Well, in order to isolate the problem, I will go back to basics, and remove any reference to the DS3231... and start over...
... will report back...


can you share the program that you have made? I happen to be doing a similar project now

Geek Emeritus

#12
Apr 15, 2019, 08:37 pm Last Edit: Apr 15, 2019, 08:42 pm by Geek Emeritus
this is how I did that:

way up at the beginning:

Code: [Select]
File dataFile;                      // variable for SD
char filename[]  = "00000000.CSV";  // filemame array for datalogger
char logString[] = "000000000000";  // build a string for the event log


this:
Quote
char logString[] = "000000000000";  // build a string for the event log
is not needed for deriving the file name, it is just included to make the rest of the code understandable

code to set up SD card not included

the routine that derives the filename is called every time an event is logged. it gets the date from LocalTime: time_t - the offset from UTC and the offset for DST if necessary:

Code: [Select]
///////////////////////////////////////////////////////////////////////////////////////////////
// SD & DATALOGGER FUNCTIONS //////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////////////////

void getFilename( time_t LocalTime )
     {         
      sprintf(filename, "%04d%02d%02d", year(LocalTime), month(LocalTime), day(LocalTime));
     }
           
void logEvent(int sw)
{
 time_t eventTime = LocalTime();
      {
       getFilename( eventTime );
       lcd4.setCursor(3,0); lcd4.print("Filename:"); lcd4.print(  filename );
       File dataFile = SD.open(filename, FILE_WRITE);
       if (dataFile)                                                   // if the file is available, write to it:
         {
          lcd4.setCursor(2,0); lcd4.print("O");
          sprintf(logString, "%02d,%02d,%02d,%02d", sw,
          hour(eventTime), minute(eventTime), second(eventTime));
          dataFile.println(logString);
          dataFile.close();                                           // close the file after writing
          lcd4.setCursor(2,0); lcd4.print("*"); lcd4.setCursor(9,1); lcd4.print(logString);
         }
       
 /*     dataFile = SD.open(filename);
       
       if (dataFile)                                                  // if the file is available, read from it:
       {
         while (dataFile.available())
         {
          Serial.write(dataFile.read());
         }
         {
          dataFile.close();
          Serial.print( "data file "); Serial.print(filename); Serial.println( " closed after reading" );
         }
       }       
       
       else
       {
        Serial.print("error opening "); Serial.println(filename);                  //if the file isn't open, pop up an error:     
       }*/
 
   }

}


this:

Code: [Select]
lcd4.setCursor(3,0); lcd4.print("Filename:"); lcd4.print(  filename );
lcd4.setCursor(2,0); lcd4.print("*"); lcd4.setCursor(9,1); lcd4.print(logString);


is a status display and an easter egg, not necessary for deriving the filename. the file status is preceded by "SD:" on the display.

the part between /* & */ was for troubleshooting and verifying. not needed, but there is no benefit to throwing it away

sw is the loop counter for the loop that reads the sensor array.

note that
Quote
char filename[]  = "00000000.CSV";  // filemame array for datalogger
is 12 characters and it works just fine all day every day
All PMs will be deleted unopened due to arrogant argumentative pot stirring Malfoys.
 If you have not read "How to use this forum - please read", in particular: "7. If you are posting code or error messages, use "code" tags":  expect rude responses

Go Up