SD Library and RAM usage

Hi All

Is there a method for working out how much free ram is needed to use the SD library? My code was running with about 300 free ram in the main loop but the SD writes were failing when called. Fiddled a few things and got free ram up to about 370 and now works fine. Obviously when the routine is running its eating up all the free ram and failing - so is there a way to work out from the code how much is needed to successfully do the write?

Stocky

Obviously when the routine is running its eating up all the free ram and failing - so is there a way to work out from the code how much is needed to successfully do the write?

Not correct. This will provide basic overview Arduino Memories

The SD library requires 512 buffer plus the library structures in SRAM. Constants stay in flash. Initialized variables exist both in flash and in SRAM.

By subtraction (deleting the library from the sketch) I found that SdFat requires about 650 bytes of ram and the SD wrapper uses an additional 75 or so.

I've checked the free ram in setup(), early in my main loop and after opening files and writing. The free ram was roughly the same in all cases. I think that it is allocated before you actually do anything. Of course calling the methods will use some additional stack but I can't believe that would be anything close to 300 bytes.

What is the symptom of failure?

When it goes to do the write it gets as far as creating/opening the file then fails to the “Error opening log.txt”. Its not getting as far as recording ANY data values

You can see in the code below I’m logging time/date and various data values which are floats.
Once Free ram gets down to about 330 it fails I have fond with more playing.
I now have 401 free with a bit more optimisation so its not an issue now but I’d like to understand more about how much it uses or how to work it out for future reference

Here is the routine that does the actual write (yes its a snippet!)

//========================================================================
// Data Logging
//========================================================================
void LogData(){
  digitalWrite(WriteLEDpin,HIGH);                                  // Turn on SD WRITE LED
  File dataFile = SD.open("log.txt", FILE_WRITE);              // Open the file
  if (dataFile) {                                                  //    
    if (hour()<10){                                                //
      dataFile.print("0");                                         // Leading zero if reqd 
      }                                                            //
    dataFile.print(hour());                                        // Log HOUR
    dataFile.print(":");                                           // 
    if (minute()<10){                                              // Leading zero if reqd
      dataFile.print("0");                                         // 
      }                                                            //  
    dataFile.print(minute());                                      // Log MINUTE
    dataFile.print(",");                                           // 
    dataFile.print(day());                                         // Log DAY 
    dataFile.print("-");                                           // 
    dataFile.print(month());                                       // Log MONTH 
    dataFile.print("-");                                           // 
    dataFile.print(year());                                        // Log YEAR  
    dataFile.print(",");                                           //
    dataFile.print(tRa);                                            // Log RANGE
    dataFile.print(",");                                           //
    dataFile.print(rhRa);
    dataFile.print(",");                                           //
    dataFile.println(wRa);
    dataFile.close();                                              // close file
    Serial.println(F("SD : Data Logged"));
  }                                                                //
                                                                   // 
else {                                                             // 
    Serial.println(F("SD : error opening log.txt"));                // error message if cant open file  
}
delay(990);                                                        //  
digitalWrite(WriteLEDpin,LOW);                                     // turn of SD WRITE LED 

}

Okay, I duplicated it by allocating additional ram in my sketch until it failed.

299 bytes - before open() 268 bytes - after sucessful open() 312 bytes - just before writing to the card

If I reduced the free ram by one more byte it failed to open the file. I traced where it was failing to line 430 in SdFile.cpp:

if ((oflag & (O_CREAT | O_WRITE)) != (O_CREAT | O_WRITE)) return false;

At that location the free memory was 140 bytes when it failed; 141 or more it would succeed.

I'm surprised that it was 159 bytes less than before calling open().

Yeah its very RAM hungry thing the SD library!

I'm trying to add an alarm timer (timealarms) back into the sketch and it keeps tripping me up!

I'll keep plodding along

Adding TimeAlarms and the ONE alarm I need back into my sketch is costing me 69 bytes of RAM and thats enough to break the SD routines.

I need to stop an event from recurring for 60 minutes after it triggers - any suggestions on an alternative to TimeAlarms?

I have multiple rolling averages going on in the background - thats where all my RAM is being gobbled up - but they are needed so I cant play with them

Trying a rough method using millis() - just not sure how its going to handle things when millis() rolls over I guess I will find out in about 50 days!

AlertDelayTime = (millis() + 3600000);      // add 60 minutes to current millis()
if ((millis() >= AlertDelayTime) && (AlertDelay == true)){
  Serial.println("clear");
  ClearAlert();
}

I've made some changes after reading http://www.gammon.com.au/forum/?id=12127

Supposedly that will fix the overflow issue :-)

Use SdFat instead of SD and you will save another 60-90 bytes. SD is a wrapper for an old version of SdFat. It's easy to use SdFat instead; the changes to your code to switch would be minimal.

I am also working to limit my ram usage. Your experience has shown me that I have less free ram than I thought.

Stocky: Yeah its very RAM hungry thing the SD library!

I'm trying to add an alarm timer (timealarms) back into the sketch and it keeps tripping me up!

I'll keep plodding along

I agree. Which is why if SD is solely for logging, I off-load the effort to a separate 328P only supporting the SD card: Dedicated SD logger Of course, this is a one-way trip from the host uC to the SD card but that fits my requirements.

Ray

mrburnette: Which is why if SD is solely for logging, I off-load the effort to a separate 328P only supporting the SD card

A hardware solution may be an option, but it consumes more space and power which can be an issue for some projects. A different uP instead of a second one, or some external ram might also work.

Besides struggling to reduce ram in one's sketch there are other software approaches as well. You can modify libraries sometimes. There are minimal versions of FAT for SD cards, like Petit Fat and uFat. They have limitations but will reduce ram usage significantly.

I tested my sketch with SdFat replacing SD. I found that with SdFat I had 63 bytes more free ram going into the open() call. Inside open() there were an additional 65 bytes more free ram as compared to when I used SD. So using SdFat actually increased free ram by 128 bytes in my sketch. It also shrunk the sketch by 3.5K.

Thanks - a second micro is not an option for me, so I’ll have a look at SDfat

I’ve got around it for the time being - until i try to add some more features later and run into it again!