Two questions: SdFat saving garbage, power-down data corruption

Hi, using the latest incarnation SdFat and I am getting a string of garbage ("O/O/O/O/O/O") at the head of the file.

Q1: Thoughts on what might be going on and how to fix it?

Q2: This is used in a vehicle and the power can be shut off without notice. How can I try to get the data written to the SD-card so that an unexpected power-down doesn't write garbage? I don't mind losing the last line of data at power-down if there is some way to make sure the rest of the file is OK. T

he garbage characters are making it a pain to have to edit the file before passing to the Spreadsheet.

Thanks.

void sdWrite() {
  if (sdCardOK) {     // This is checked in Setup and the bool sdCardOK is set there
    char fd[14] {0};  // "P" below needed as FName cannot start with a Number
    sprintf(fd,"P%d%02d%02d.CSV",localYear-2000,localMonth,localDay);
    // e.g. P20170318.CSV
    if (!SD.exists(fd))  // Create the CSV file
    {  testData = SD.open(fd,FILE_WRITE);
       testData.println(F("Date, Time"));
       testData.flush();
    } else {
      testData = SD.open(fd,FILE_WRITE);  // Just open it as it already exists
    }
    testData.print(localDate);
    testData.print(F(","));
    testData.print(localTime);
    testData.println();
    testData.close();
    smartDelay(1000);        // Show the "W" for a little bit
}
    {  testData = SD.open(fd,FILE_WRITE);
       testData.println(F("Date, Time"));
       testData.flush();

Why do you ASSume that the file was opened?

Why do you open the file only of SD.begin succeeded, but than write to it even of SD.begin() failed AND/OR the file couldn't be opened?

PaulS: Why do you ASSume that the file was opened?

Always a joy to get a civil response. See corrected code and still not approaching a solution to the problem.

If the file didn't exist, then how would it be writing garbage to it sometimes Rhetorical of course.

Q1 and Q2 still stand open. :)

void sdWrite() {
  if (sdCardOK) {     // This is checked in Setup and the bool sdCardOK is set there
    char fd[14] {0};  // "P" below needed as FName cannot start with a Number
    sprintf(fd,"P%d%02d%02d.CSV",localYear-2000,localMonth,localDay);
    // e.g. P20170318.CSV
    if (!SD.exists(fd))  // Create the CSV file
    {  testData = SD.open(fd,FILE_WRITE);
        if (SD.exists(fd) {
         testData.println(F("Date, Time"));
         testData.flush();
       }
    } else {
      testData = SD.open(fd,FILE_WRITE);  // Just open it as it already exists
    }
    if (SD.exists(fd) {
      testData.print(localDate);
      testData.print(F(","));
      testData.print(localTime);
      testData.println();
      testData.close();
      smartDelay(1000);        // Show the "W" for a little bit
    }
}

I'd suggest creating the file in setup instead:

void setup() {


    ... other setup stuff...

  if (sdCardOK) {
    char fd[14];  // "P" below needed as FName cannot start with a Number
    sprintf(fd,"P%d%02d%02d.CSV",localYear-2000,localMonth,localDay);
    // e.g. P20170318.CSV

    testData = SD.open(fd,FILE_WRITE);
    testData.println(F("Date, Time"));
  }
}

Then your writing routine is just this:

void sdWrite()
{
   if (testData.isOpen()) {
      testData.print(localDate);
      testData.print( ',' );
      testData.println(localTime);
      testData.flush();
      //  NO!     smartDelay(1000);
   }
}

Using a delay in a GPS logging program is never a good idea. You should read the "Blink without delay" or "How to do several things" examples in Useful Links for a different, non-blocking technique.

Without the complete sketch, I can't tell if you are really opening multiple files at different times. If you are, open them at the point you know a new files should be created (e.g., a button press) and write the header line. That keeps sdWrite simple.

You should also be aware that flushing the file after every write will slow the process down a lot. Instead of ~100us per write, it could be ~100ms or more. If you are only logging once every 1000ms (i.e., 1s), it's no big deal. But if you have other things to do, you may have to change your "flush policy" to something like once every 20 lines, or once every 30 seconds. This would mean you might lose the last 20 lines (or the last 30 seconds of data).

Another common solution is to close the file when a button is pushed. The LED can be used to indicate that logging is active (file is open) or inactive (file is closed). Closing the file makes sure the last records are flushed and the file system is updated, "nicely".

The presence of smartDelay implies that you started with a TinyGPS example program. It is not a smart delay, BTW. And those example programs are not structured properly. You are, no doubt, running into trouble modifying the sketch to do other things, like log to the SD card. :-/ They also have issues when the GPS device emits multiple sentences every second.

You might take a look at my NeoGPS library. The examples are structured properly, and then can usually be modified without breaking them. There is even an SD logging example. If you want to try it, NeoGPS is available from the Arduino IDE Library Manager, under the menu Sketch -> Include Library -> Manage Libraries.

Even if you don't try it, I would suggest reading the Troubleshooting pages, as they describe many of the problems you might encounter (have already encountered?). Those problems are one of the reasons I wrote NeoGPS. Speed and size were other major reasons... NeoGPS is much smaller and faster than all other libraries.

Cheers, /dev