next topic »

[on:]

I'm running simple loops using the Teensyduino iterating a variable (x) from 1 to, say, 10,000. It's a scaled down problem in an attempt to find a solution for a bug in a much larger code. The problem is that after writing a certain number of lines to the SD, it will stop writing for a certain quantity of values, then start back up again, e.g. In the most recent trial it recorded from 1 - 4293, then stopped, and began on the next line (Excel line 4294) with 14581, then continued until I cut it off around 15350.
Also, the odd time it will print two values on the same line instead of skipping to the next one.

I've been running the trials alongside Serial port recordings using the PuTTY program (very useful program) which creates a .txt or .csv file with the serial port output.

It's confusing, as in both the larger code and this simple one, the serial outputs are always exactly what is desired, and it's not a fault with the SD card as I've tried using a few different ones and the same problem emerges.

I'm thinking it might be a problem with opening and closing the datafile so often and quickly (would anyone have a better way of doing this?), or else maybe with the SD library.
Has anyone else had a similar problem?

Here's the code:

Code:

/*
  Purpose of this program is to check for faults with writing to the SD card.
*/

#include <SD.h>

const int chipSelect = 5;
long int x = 0;

void setup()
{
  Serial.begin(9600);
 

  if (!SD.begin(chipSelect))
  {
    Serial.println("Card failed");

    return;
  }
 
  Serial.println("SD card initialized.");
 
  pinMode(10, OUTPUT);           
  digitalWrite(10, HIGH);             
 
  pinMode(5, OUTPUT);             
 
  digitalWrite(SS, HIGH);
  digitalWrite(PIN_D7, HIGH);
 
  toBegin();
}

void loop()

{
  x = x + 1;
 
 SD.begin(chipSelect);

  File dataFile = SD.open("datalog.csv", FILE_WRITE);
  if (dataFile)
  {
    dataFile.println(x);
   
    dataFile.close();

    Serial.println(x);
   
    } 
  else
  {
    Serial.println("error opening datalog.csv");
  } 
 
  delay(10);
}

void toBegin() 
 {
   while (Serial.available() <= 0)
   {
     Serial.println('1', BYTE);   
     delay(300);       
   }
 }

Any help would be greatly appreciated!

[Reply #1 on:]

Code:

  File dataFile = SD.open("datalog.csv", FILE_WRITE);
  if (dataFile)
  {
    dataFile.println(x);
   
    dataFile.close();

Open the file. Write a value to the file. Close the file. Repeat for every value. As the file begins to fill up, it takes longer and longer to write to the file.

Depending on where the data is coming from, it may not be necessary to open and close the file every time. One certainly would not do that on a PC.

[Reply #2 on:]

"One certainly would not do that on a PC."

Indeed.  But on a PC you tend to have a more orderly shutdown procedure than a microcontroller.

Any suggestions for a more fail-safe procedure for closing files ?

[Reply #3 on:]

Any suggestions for a more fail-safe procedure for closing files ?

I'd still like to know where the data is coming from, and how much of it there is.

What is the risk if data is lost? How much data can you afford to lose? You can write more than one byte at a time to the file. Buffer data (as much as you can afford to lose), and open the file, write the buffered data, and close the file. If the Arduino has problems, like loss of power while the file is not open, then the only data lost will be what was in the buffer, not yet written to the file.

You need to, of course, balance the buffer size with the time needed to write that data to the file.

The idea is to minimize both the amount of time the file is open (and corruptible) and the number of times the file needs to be opened.

[Reply #4 on:]

Cheers, I'll implement a buffer to prevent the SD opening and closing so frequently, which should solve it.
Thanks for all your help.

[Reply #5 on:]

After running a series of programs using buffers, it still didn't solve the problem. I ran dozens of trials over the past few days, making small edits to the above code each time, and finally found a solution.

When I removed the SD.begin(chipSelect) line from the loop, the code runs perfectly every time. I will still most likely use a buffer in order to make to code more efficient.

Still, I don't understand why the SD.begin in the loop would cause the program to skip over 10,000 values at the same position every time, after working for more than 4000. For now I'm just satisfied the program runs properly.

Thanks again!

[Reply #6 on:]

SD.begin() initializes the SD card, mounts the FAT volume, and clears any unwritten data in internal buffers.

Calling it in loop discards a lot of data and may cause file system corruption.  SD.h was not designed to call SD.begin() multiple times.  SD.h is a wrapper for an old version of SdFat.  The wrapper has bugs that prevent multiple calls from always working.

SdFat allows multiple calls to begin/init if all files are closed first.  This allows one SD card to be removed and another SD card to be inserted.

[Reply #7 on:]

you want to call sync() to flush the data rather than closing. Close does call sync ... but it closes as well which is the bit you don't want.

Look at the SD library code to see about sync().