Problem with SD Library and long strings

If I try to use the SD library in some of my bigger sketches, the SD.begin function restarts the Arduino. I have replicated the problem with a couple of long strings. Try opening the SD ReadWrite example sketch and adding the following two lines of code anywhere

String whatNow = "While the Arduino does have some on-board memory it's really not very much, only 1 kilobyte. If you're building something like an MP3 player, picture viewer or data logger you're going to need a lot more memory than 1 kilobyte. A microSD card is really just memory that you can read and write to; even better is that with Arduino you can implement a file system, like FAT16 or FAT32, which lets you easily create, read and write to files. microSD cards are available with memory sizes of 256 MB all the way up to 32 GB and higher! You can pick any size card you'd like to suit you application without modifying the sketch. If you want to start digging around with some example sketches here's a good starting point:";
String ahhh = "In order to write information to a microSD card we need to have access to a file. If a file doesn't exist yet it must be created. Before the file can be created, though, we must initialize the microSD card to use a FAT file system, initialize a volume on the file system, and then finally open the root directory in the volume. Once all of this is done we can create a file. Luckily for us, the SdFat library (written by William Greiman) does most of the work for us! First create the objects that are needed for the FAT library to work. This code must go at the start of the sketch before both the loop and the setup sections:";

If you run the sketch now, you should see the arduino go into an infinite loop. It will print "Initializing SD card..." to serial each time it restarts.

I can stop this from happening by commenting out line 370 in Sd2Card.cpp in the SD Library. Of course, this breaks the SD functionality but it stops the arduino from resetting.

  // transfer data
  n = count - 1;
  for (uint16_t i = 0; i < n; i++) {
    while (!(SPSR & (1 << SPIF)));
// dst[i] = SPDR;
    SPDR = 0XFF;
  }

Can any reproduce this? Any insight into why it's happening?

Any insight into why it's happening?

Where is dat[] defined? What size is it? Do your strings fit in the dat array, or are you overflowing a buffer?

PaulS:

Any insight into why it's happening?

Where is dat[] defined? What size is it? Do your strings fit in the dat array, or are you overflowing a buffer?

Sorry if I wasn't explicit enough. The dst array is in the function Sd2Card::readData, which is part of the SD Library included with the Arduino 0022 code base. Were you able to replicate this? I am hoping it's not just me and that there is a bug in the SD Library that can be fixed.

The dst array (not dat - old eyes) is input to the readData function. How are you calling that function?

The two strings in your example are using 1,344 characters, of the 2048 available in SRAM. That's 67.2% of all available memory.

I am not calling the function myself. It is being called by SD.begin(), I just traced the problem to the ReadData function.
The strings are designed to take up most of the available memory. I have a big sketch with a lot of code that takes up a ton of memory and I want to use the SD Library in it. When I tried to add it, I ran into this problem. Instead of posting that huge sketch, I figured out how to reproduce the problem with just those two strings.

I'm afraid you are not understanding me. If you have an arduino, an ethernet shield, a micro SD card, and the current version of the code (22) handy, give this a shot.

  1. Open the Example Sketch SD >> ReadWrite.
  2. Run the example on your board and open the serial window. Make sure you see the expected output. It should say it opened, wrote, and read the file and print the file contents.
  3. Now add those two really long strings from my original post as global variables at the top of the file (or in the setup function, or in the loop function, it doesn't matter).
  4. Run the example again. Instead of seeing the expected output, my board gets past the first serial.print call, and then resets when SD.begin is called. Since something in the setup function is causing the board to reset itself, I get an infinite loop with the board writing that first line "Initializing SD card..." to the serial window over and over.

I commented out the SD.begin function and the board stopped restarting. So then I opened the SD Library code and started commenting things out. Eventually, I tracked the problem down to the readData function. I can't figure out why that is causing the problem.

My first question is, can someone else reproduce this problem, or is it just me.
If someone can reproduce it, then something is wrong with the library and it needs to be fixed, or at least noted that the library needs a certain amount of memory free before it can be used.

I'm afraid you are not understanding me.

On the contrary, I do understand you.

Now add those two really long strings from my original post as global variables at the top of the file

Now, use up all available SRAM.

then something is wrong with the library and it needs to be fixed

The problem is that you are running out of SRAM. Changing the library so that it doesn't work won't help. Rather, it won't be useful.

or at least noted that the library needs a certain amount of memory free before it can be used.

More likely for this to happen, but the question is how much? And how/when do you measure that? SRAM usage is not static. What might be just barely enough in one sketch will not be enough in another sketch.

or at least noted that the library needs a certain amount of memory free before it can be used.

More likely for this to happen, but the question is how much? And how/when do you measure that? SRAM usage is not static. What might be just barely enough in one sketch will not be enough in another sketch.

Although SRAM usage is dynamic, I think my problem is too much static usage because my program barely gets off the ground before it crashes.
It looks like the SD library needs at least 25% of your SRAM to function based on it's cache:

// SdVolume class
/**
 * \brief Cache for an SD data block
 */
union cache_t {
           /** Used to access cached file data blocks. */
  uint8_t  data[512];
           /** Used to access cached FAT16 entries. */
  uint16_t fat16[256];
           /** Used to access cached FAT32 entries. */
  uint32_t fat32[128];
           /** Used to access cached directory entries. */
  dir_t    dir[16];
           /** Used to access a cached MasterBoot Record. */
  mbr_t    mbr;
           /** Used to access to a cached FAT boot sector. */
  fbs_t    fbs;
};

I think the library reference page should note that it takes a good chunk of your resources to run. Especially considering the consequences of going over the limit.

The sketch I am having a problem with is a web server, which has a bunch of static strings that make up the webpage I am trying to serve. These strings are probably taking up more than half of my SRAM and trying to add SD Library to the mix is causing the crash. I already have a bunch of strings in program memory, but I will try to move more of them there. I will also try to reduce the number of static arrays I am using.

Any other tips for reducing my SRAM usage?
Can I measure the amount of SRAM I am using at compile time?

Thanks for your help, PaulS. I really appreciate it.

I fought almost this same problem for a long time. Because the problem would come and go, and because it seemed to respond to the order and when variables, and particularly char arrays were declared, It only dawned slowly that it was a memory problem. Considering the time spent chasing the problem, the move to a megaArduino was a cheap fix. I

I now declare the char array buffers globally so they don't impact ram in unexpected ways.

I use one of the RAM reporting functions and stick a Serial.print in here and there while debugging to see how RAM is coming. I see lots of discussion about the relative merits of the RAM programs, but if it continues to report 3000 bytes available in the Mega implementation, who cares about precision.