Reading from SD fails after 223 successful reads

As a part of a larger project I am reading writing to and reading from an SD often. I noticed that after a while I can no longer read from the SD card.

I made a small test program and found that reading fails after 223 reads with all three test files

#include <SD.h>

int Tries = 1;
char rBuff[32];


void setup()
{
  // put your setup code here, to run once:


  Serial.begin(9600);

  delay(1000);
  Serial.println("Starting");

}


void loop()
{
  Serial.print("Read ");
  Serial.print(Tries, DEC);
  Serial.print(": ");
  if (LoadFromFile( rBuff, "A.TXT", 31))
  {
    ++Tries;
    Serial.print(rBuff);
    Serial.print(" Data Length= ");    
    Serial.println(strlen(rBuff));
  }
  else
  {
    Serial.println("Fail");
    while (1);
  }
}

char *LoadFromFile(char *dBuff, char *fName, int lMax)
{
  
  File myFile;
  int i = 0;

  dBuff[0] = 0;

  //determine if SD has started
  //SD.rewindDirectory(); //added 03 jul 2016
  if(!SD.open("/"))//tested ok 10 feb 2016
    if (!SD.begin(4))
      return 0; //was return dBuff;

  if (myFile = SD.open(fName))
  {
    cli(); //added 24 july 2016
    if (myFile.size() < lMax)
    {
      //read the file
      while (myFile.available())
      {
        dBuff[i] = (char)myFile.read();
        ++i;
      }
      dBuff[i] = 0;
    }
    //close the file
    myFile.close();
    sei(); //added 24 july 2016
  }
  else
  {
    //the file did not open
    return 0; //added 26 june 2016
  }
  return dBuff;
}

Hardware: Freetronics EtherMega

Any suggestions?

B.TXT (16 Bytes)

A.TXT (8 Bytes)

C.TXT (29 Bytes)

Move SD.begin() to setup and only run once.

I don’t understand the test on opening the root but I don’t think it is necessary and it is creating a problem.

if (!SD.open("/")) //tested ok 10 feb 2016

I don’t see why disabling and enabling interrupts is required, but that doesn’t seem related to your current problem.

#include <SPI.h>
#include <SD.h>

int Tries = 1;
char rBuff[32];

void setup()
{
  Serial.begin(9600);

  delay(1000);
  Serial.println("Starting");

  //move SD.begin to setup
  if (!SD.begin(4))
    //return 0; //was return dBuff;
  {
    Serial.println("initialization failed");
    while (1);
  }
}

void loop()
{
  Serial.print("Read ");
  Serial.print(Tries, DEC);
  Serial.print(": ");

  if (LoadFromFile( rBuff, "A.TXT", 31))
  {
    ++Tries;
    Serial.print(rBuff);
    Serial.print(" Data Length= ");
    Serial.println(strlen(rBuff));
  }
  else
  {
    Serial.println("Fail");
    while (1);
  }
}

char *LoadFromFile(char *dBuff, char *fName, int lMax)
{

  File myFile;
  int i = 0;

  dBuff[0] = 0;

  //determine if SD has started
  //SD.rewindDirectory(); //added 03 jul 2016

  //if (!SD.open("/")) //tested ok 10 feb 2016

  if (myFile = SD.open(fName))
  {
    //cli(); //added 24 july 2016
    if (myFile.size() < lMax)
    {
      //read the file
      while (myFile.available())
      {
        dBuff[i] = (char)myFile.read();
        ++i;
      }
      dBuff[i] = 0;
    }
    //close the file
    myFile.close();
    //sei(); //added 24 july 2016
  }
  else
  {
    //the file did not open
    return 0; //added 26 june 2016
  }
  return dBuff;
}

The real project runs Software Serial to talk on Rs485 so disabling the interrupts is necessary.

As in the real program is split into many functions across several libraries, you need a way of seeing if SD.begin has been called as trying to call SD.begin when it already has causes issues.

if(!SD.open("/"))//tested ok 10 feb 2016
    if (!SD.begin(4))
      return 0; //was return dBuff;

I will try just calling SD.begin in the setup() (Something I was trying to avoid.)

Thanks for the suggestion.

The suggestion of removing:

if(!SD.open("/"))//tested ok 10 feb 2016
    if (!SD.begin(4))
      return 0; //was return dBuff;

worked for this test program. At least for 25k reads.

However on the larger program it still failed. In the larger program there is a SD based FIFO buffer, for that there are 3 reads, one write, 3 deletes, and 3 writes per entry.

Since the change the larger program fails on the writes at about the 300th cycle.

I'll write a test program for the write and post an update.

Maybe you are running out of SRAM? SD.open requires 512 bytes of free SRAM to succeed. Add this to your code and call it just before your code attempts to open the file.

int freeRam() {
  extern int __heap_start,*__brkval;
  int v;
  return (int)&v - (__brkval == 0 ? (int)&__heap_start : (int) __brkval);  
}

// then call it like this
Serial.print(F("SRAM left: "));
Serial.println(freeRam());

Maybe you are running out of SRAM?

The problem is this:

if(!SD.open("/"))//tested ok 10 feb 2016

open() allocates about 30 bytes of memory from the heap.

 _file = (SdFile *)malloc(sizeof(SdFile));

Since the file is not closed, a memory leak is created.

This is another bug I warned Arduino about when they did their SD.h wrapper for the now seven year old version of SdFat.

@fat16lib: Then the 512 byte memory requirement to open each file is no longer applicable? Is it possible to open more than one file with either SD or SdFat? I'm using SdFat now.

edit: What about reading and writing to the SD card? Does it now buffer only a few bytes at a time? Or does it allocate more memory to read or write?

Is it possible to open more than one file with either SD or SdFat? I'm using SdFat now.

SdFat never had a limit and SD.h was modified 4-5 years ago to remove the limit.

I assume the 223 reads happened on a Mega since each file takes something over 30 bytes . I got about 25 open files when I ran it on an Uno.

SD.h always has a single 512 byte cache for one block. Newer versions of SdFat have two blocks for larger boards like Mega. You can restrict this to one block by editing the config file.

The cache is shared so there is an impact on performance if you alternate between several open files with small transfers.

If you do a fairly large transfer for each file access the impact will be less.

The only limit on the number of open files is memory so you can have over 200 open files on a Mega.

Thanks for the suggestions.

For clarification I am only ever opening one file at a time. Each is closed before further operation.

As I said I have removed this:

if(!SD.open("/"))//tested ok 10 feb 2016
    if (!SD.begin(4))
      return 0; //was return dBuff;

From all functions.

The project is now failing on a Write. I am trying to isolate the problem now then I will post again.

A thought. I don't recall having this problem writing to the root of the SD card. Could using a directory/folder be a part of the problem? I would not imagine so but thought I would ask.

How did the SRAM test from reply #5 turn out? How much do you have remaining before opening the file?

I have not yet added the SRAM test from reply #5 yet, I'll do that Sunday night.

I changed the larger program so that if the SD Load or Write functions fail you wait 50ms and try again. 35 000 read writes with out issue.

As an example:

  if (!LoadFromFile(Data, fPath, 32))
  {
    delay(50); //wait and try again
    if (!LoadFromFile(Data, fPath, 32))
      //read failed
      return 0;
  }
  return Data;

This code would need tweaking to run in the test file I originally posted.

I will run a full 24 hr test Monday ~ 240 000 read writes