Pages: [1]   Go Down
Author Topic: SD card file write is skipping data  (Read 1387 times)
0 Members and 1 Guest are viewing this topic.
Ireland
Offline Offline
Newbie
*
Karma: 0
Posts: 3
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I'm running simple loops using the Teensyduino iterating a variable (x) from 1 to, say, 10,000. It's a scaled down problem in an attempt to find a solution for a bug in a much larger code. The problem is that after writing a certain number of lines to the SD, it will stop writing for a certain quantity of values, then start back up again, e.g. In the most recent trial it recorded from 1 - 4293, then stopped, and began on the next line (Excel line 4294) with 14581, then continued until I cut it off around 15350.
Also, the odd time it will print two values on the same line instead of skipping to the next one.

I've been running the trials alongside Serial port recordings using the PuTTY program (very useful program) which creates a .txt or .csv file with the serial port output.

It's confusing, as in both the larger code and this simple one, the serial outputs are always exactly what is desired, and it's not a fault with the SD card as I've tried using a few different ones and the same problem emerges.

I'm thinking it might be a problem with opening and closing the datafile so often and quickly (would anyone have a better way of doing this?), or else maybe with the SD library.
Has anyone else had a similar problem?

Here's the code:
Code:
/*
  Purpose of this program is to check for faults with writing to the SD card.
*/

#include <SD.h>

const int chipSelect = 5;
long int x = 0;

void setup()
{
  Serial.begin(9600);
 

  if (!SD.begin(chipSelect))
  {
    Serial.println("Card failed");

    return;
  }
 
  Serial.println("SD card initialized.");
 
  pinMode(10, OUTPUT);           
  digitalWrite(10, HIGH);             
 
  pinMode(5, OUTPUT);             
 
  digitalWrite(SS, HIGH);
  digitalWrite(PIN_D7, HIGH);
 
  toBegin();
}

void loop()

{
  x = x + 1;
 
 SD.begin(chipSelect);

  File dataFile = SD.open("datalog.csv", FILE_WRITE);
  if (dataFile)
  {
    dataFile.println(x);
   
    dataFile.close();

    Serial.println(x);
   
    } 
  else
  {
    Serial.println("error opening datalog.csv");
  } 
 
  delay(10);
}

void toBegin() 
 {
   while (Serial.available() <= 0)
   {
     Serial.println('1', BYTE);   
     delay(300);       
   }
 }

Any help would be greatly appreciated! smiley
Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 617
Posts: 49463
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Code:
  File dataFile = SD.open("datalog.csv", FILE_WRITE);
  if (dataFile)
  {
    dataFile.println(x);
   
    dataFile.close();
Open the file. Write a value to the file. Close the file. Repeat for every value. As the file begins to fill up, it takes longer and longer to write to the file.

Depending on where the data is coming from, it may not be necessary to open and close the file every time. One certainly would not do that on a PC.
Logged

Offline Offline
Faraday Member
**
Karma: 62
Posts: 3032
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

"One certainly would not do that on a PC."

Indeed.  But on a PC you tend to have a more orderly shutdown procedure than a microcontroller.

Any suggestions for a more fail-safe procedure for closing files ?
Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 617
Posts: 49463
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
Any suggestions for a more fail-safe procedure for closing files ?
I'd still like to know where the data is coming from, and how much of it there is.

What is the risk if data is lost? How much data can you afford to lose? You can write more than one byte at a time to the file. Buffer data (as much as you can afford to lose), and open the file, write the buffered data, and close the file. If the Arduino has problems, like loss of power while the file is not open, then the only data lost will be what was in the buffer, not yet written to the file.

You need to, of course, balance the buffer size with the time needed to write that data to the file.

The idea is to minimize both the amount of time the file is open (and corruptible) and the number of times the file needs to be opened.
Logged

Ireland
Offline Offline
Newbie
*
Karma: 0
Posts: 3
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Cheers, I'll implement a buffer to prevent the SD opening and closing so frequently, which should solve it.
Thanks for all your help.
Logged

Ireland
Offline Offline
Newbie
*
Karma: 0
Posts: 3
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

After running a series of programs using buffers, it still didn't solve the problem. I ran dozens of trials over the past few days, making small edits to the above code each time, and finally found a solution.

When I removed the SD.begin(chipSelect) line from the loop, the code runs perfectly every time. I will still most likely use a buffer in order to make to code more efficient.

Still, I don't understand why the SD.begin in the loop would cause the program to skip over 10,000 values at the same position every time, after working for more than 4000. For now I'm just satisfied the program runs properly.

Thanks again!
Logged

0
Offline Offline
Edison Member
*
Karma: 64
Posts: 1639
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

SD.begin() initializes the SD card, mounts the FAT volume, and clears any unwritten data in internal buffers.

Calling it in loop discards a lot of data and may cause file system corruption.  SD.h was not designed to call SD.begin() multiple times.  SD.h is a wrapper for an old version of SdFat.  The wrapper has bugs that prevent multiple calls from always working.

SdFat allows multiple calls to begin/init if all files are closed first.  This allows one SD card to be removed and another SD card to be inserted.
Logged

Offline Offline
Full Member
***
Karma: 0
Posts: 179
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

you want to call sync() to flush the data rather than closing. Close does call sync ... but it closes as well which is the bit you don't want.

Look at the SD library code to see about sync().


Logged

Pages: [1]   Go Up
Jump to: