String to character formatting for SD logging.

I want to get the date string from my DS1307 RTC and use that as a file name, checking if one exists and creating it if not. So that it will create a new file every day.

I'm trying to use this to save and write to a file where the name of the file is a string File_name retrieved from the RTC clock. But it doesn't work

 for(int i=1;;i++)
  {
    String temp = File_name;
    temp.concat(i);
    temp.concat(".txt");
    char filename[temp.length()+1];
    temp.toCharArray(filename, sizeof(filename));

    if(!SD.exists(filename))
    {
      File dataFile = SD.open(filename,FILE_WRITE);
      break;
    

    if (dataFile) 
    {
      dataFile.println(".");
      dataFile.close();
    }
    }
  }

If I use File dataFile = SD.open("Newfile.txt",FILE_WRITE);it creates the file, but doesn't write to it.

Sometimes errors become more obvious when you form the habit of formatting the source code correctly using Ctrl-T. In your code, you check to see if the file doesn't exist and, if it doesn't, you then define datafile and promptly execute the break statement, which transfers program control to whatever statement exists after the closing brace of the for loop. Therefore, nothing gets written to the file. It seems to me that you need to get rid of the break statement.

Also, try to compile the code and you'll find several other compile-time errors. Always try to get rid of those first. Why did you omit expression 2 in the for loop? If expression 2 (i.e., usually a relational test to see if another iteration of the loop is needed) is missing, it is logic True, which attempts to write an infinite number of data files. Why?

I didn't write the code I just copied it from somewhere, but I guess its just ment to loop until the filename array is full and then terminates with the break. Wouldn't it create a file name each time a character is concatenated? It seems a while loop would be better. It doesn't seem very efficient to say the least.

I didn't notice that the println was inside the for loop.

Im having trouble dealing with the variable scope. So the variables created inside the for loop are limited to that block of code? But I can't declare filename at setup because I don't know the length of it yet.

You are using the String class which can adjust the file name length dynamically, so that shouldn't matter.

filename is a Char array. I don't understand what it does in this bit of code when in the loop?

if(!SD.exists(filename))
    {
      File dataFile = SD.open(filename,FILE_WRITE);
      break;
    }

Does it only break at the end? Isn't filename being concatenated every loop and therefore dataFile is pointing to new files all the time? But you can only open one file at a time right?

I just want to pass a variable into myFile = SD.open(filename, FILE_WRITE) How can I convert a string into a char effectively? I tried to use

File myFile;
String temp="newfile.txt";
char filename[temp.length()+1];
temp.toCharArray(filename, sizeof(filename));

But that gives me: array bound is not an integer constant

syphex:
char filename[temp.length()+1];

Not possible, must be a constant value.
A value 40 I think is sufficient for a file name.

char filename[40];

Ok I got it working using this before the setup():

File myFile;
String temp=("newfile");

And this inside the setup():

temp+=".txt";
char filename[temp.length()+1];
temp.toCharArray(filename, sizeof(filename));
myFile = SD.open(filename, FILE_WRITE);

But I need to do it during the loop, and it won't let me declare filename outside of the setup loop so I can't use it in the loop. Also my SD reader has no CD pin and I need to know when its in there and when someone takes it out is there anyway to do it??

and it won't let me declare filename outside of the setup loop

It won't? What the hell is it?

There is no "setup loop". There is a setup FUNCTION.

Where IS the code you are having problems with?

By the way, do you really need the String crutch? The length of a file name on an SD card is limited (12 characters total). There is no reason to use a dynamically sizable construct to hold a fixed amount of data. Learn to use sprintf() and char arrays.

Finally, there is a world of difference between a string (a NULL terminated array of chars) and a String. The terms are NOT interchangeable.

Yeah your right, it doesn't really need to exceed 12 characters and I guess I don't need to use Stringobjects at all.

I have a question though, I want to be able to take the SD out while its still logging. But without a CD pin I can't tell it when to start writing to EEPROM, so I'll have to leave that out.

Now I want to write time in minutes for the day to the SD card only when the arduino is reset, so I can insert the SD card and restart to indicate to write to SD again. I need to organise it so that excel can detect it in a file for logging. I've been using comma delimiters so far. This is what I mean:

60, 20, 50
, 20, 50
, 20, 50
, 20, 50
//SD card removed
...

//SD card inserted and arduino reset
120, 24, 45
, 24, 45
, 24, 45
...

Then Excel can add the new data to cell 120, and leave a gap between minute 60 and minute 120. Is there any better way to do this? How much memory does a blank space use? I guess a byte to store the character reference?

I calculate a 4gb SD card has enough memory for over a thousand years of 5 byte one minute logs... is this correct?

4GB = (((4294967296 bytes/5)/60)/24)/356 = 1675.6 years??

But excel has enough trouble loading the data as it is. Maybe I should log the seconds and use matlab instead? Or whats a program dedicated to graphing lots of log data?

But without a CD pin I can't tell it when to start writing to EEPROM, so I'll have to leave that out.

You REALLY should add a switch that defines whether to log to the SD card, or not. You should NOT remove the card when the switch is in the log position. Doing so can corrupt the file, if it is open when you remove the card. When the switch is flipped to the not-log position, you should close the file, if you keep it open.

I calculate a 4gb SD card has enough memory for over a thousand years of 5 byte one minute logs... is this correct?

Not even close. You are writing 5 byte-sized values as strings. A byte-sized value, like 100, can take 3 bytes to represent as a string ('1', '0', and '0'). Add a separator between strings and 5 bytes can use 19 characters. Plus the carriage return and line feed between records, and each record could be as many as 21 bytes, not 5.

PaulS:
Finally, there is a world of difference between a string (a NULL terminated array of chars) and a String. The terms are NOT interchangeable.

Isn't a string an ordered set of bytes, and a "c-string" an ordered set of bytes saved to a memory location with a null terminator added to the end of the original string?

Isn't a string an ordered set of bytes

It's an ordered set of bytes that is NULL terminated.

and a "c-string"

If we were discussing the differences between C and PHP and Java, I might feel compelled to clarify which language's string I was talking about. Since this forum is about the Arduino, which is programmed in C++, which is a superset of C, I don't feel it necessary to qualify the particular language's string I am talking about.

I don't feel it necessary to qualify the particular language's string I am talking about.

What ever floats your boat. :slight_smile:

syphex:
I have a question though, I want to be able to take the SD out while its still logging. But without a CD pin I can't tell it when to start writing to EEPROM, so I'll have to leave that out.

See this: Gammon Forum : Electronics : Microprocessors : Temperature and humidity sensor - battery powered

I wrote a temperature logger with an SD card. To address the issue of removing the SD card, there is a red LED adjacent to it.

It flashes once as you start taking a reading and then 10 times just before opening the SD card file. Then it writes to the SD and flashes twice after the file system is synced and closed.

The idea is that if you are reaching to eject the SD, and the LED starts flashing, hold off until you see the two flashes. The 10 flashes give you plenty of time to either move your hand away, or quickly finish the eject process if they happen to start once you have started pushing on the card.

So far I haven't had any corruption.

I have implemented a switch to detect writing to the SD or EEPROM which tells me to "insert sd card" or "Remove sd card" depending on its previous value. What is the best way to write a line of 3 bytes and possibly one unsigned long to the EEPROM?

For example I want to write 28,24,42 to the EEPROM every minute and then be able to read the data for every minute at once when the SD switch is activated. From what I can gather each byte must be written to a new address so this poses a slight problem for the long number, which get written as 1430,28,24,42 when the SD switch is flicked on or off, where the 1430 is the elapsed daily minutes. Can I use the write anything example?

Can I use the write anything example?

Sure. You have our permission.

Yours is good for low power consumption but I want to log every minute, and remove/insert the SD when a switch is flipped. I think timestamping every minute would use up my 4gb SD card too quickly so I wanted to do a hybrid idea where only the minutes for the day is timestamped and only when the data switches between the EEPROM and SD via the switch. But I don't know how to format it for a spreadsheet, which would have to check if the timestamp is there and then move the data to the correct position in the spreadsheed, it seems intuitive to have the minute-stamp at the start of the data and blank otherwise such as:

minutes,temp,avg,humidity:

1049,24,18,56
,24,18,56
... etc until the SD is switched

BUT if I'm only minute-stamping the elapsed daily minutes when the SD/EEPROM is activated

I just want three main questions answered:

**1.)**Does it matter in terms of efficient data storage if the "optional" timestamp is at the start or end of the data? Or would it be better to make a dedicated entry for the elapsed minutes instead of including the temp/avg/humidity? And then how would the spreadsheet recognise it as a timestamp and not data, it would then have to check for absense of comma delimiters or something.

**2.)**How can I ensure that the data is only written every minute based on the RTC? Is code such as this sufficient/reliable enough?

if (lastminute=minute-1) 
{
  mycount+=1
  if (SD inserted/removed)
   {
    //write mycount as timestamp
   }
//write rest of data to SD/EEPROM
}
lastminute=minute;

Is there a chance that it could miss a count now and then? And especially if lastminute=minute is used every step to avoid (lastminute=minute-1) being infinitely false, then the data won't get logged for that minute and the actual time the data is logged would gradually drift behind the real elapsed minutes since I am relying on every entry being logged a minute apart until the SD switch is flipped on or of and the elapsed minute is stamped somewhere.

**3.)**I don't understand the EEPROM read/write anything library, how does it store the data? If I write a line of data, how do I know which variables are the temperature etc? Especially if I'm writing an "optional" timestamp which will have to be a long int whereas the rest of the data would be bytes?

Is code such as this sufficient/reliable enough?

No.

if (lastminute=minute-1)

Use your space key!

Consider using ==, instead of =.