Gaps in record while logging to SD Card

Projects Programming
1

The code below is the logging part of a much larger code of a data logger. Its working per se but there is one issue though ... when I open the recorded file in either Notepad or Excel I find empty rows in random . The logging interval is set for 500ms but in reality it happens about 10 to 20 millis more than 500 ms. That really is no issue as the data is used only for trend data plot.

But I wonder what is the reason for the gaps among records ... I have enclosed both the code and a snap shot of the Excel file. Hardware : Arduino Mega + 3.2" Touch TFT shield with SD Card interface from Waveshare.

// >>>>>>>>>>>>>>>>>>>>> LOG DATA TO DISK >>>>>>>>>>>>>>>>>>>>>>>>>

void logDataToSDC() {

  if (f_logDataToDisk) {

    // **************** WRITE HEADER DATA TO SDC ******************

    if (f_writeHeaderData) {
      char header[40];
      f_writeHeaderData = false;
      logFile = SD.open("FTPLOG.CSV", FILE_WRITE);   // Open the file to write

      if (logFile) {
        sprintf(header, "Log_Start, %02d-%02d-20%02d , %02d:%02d:%02d", DayOfMonth, Month, Year, Hour, Minute, Second);
        logFile.println(header);
        logFile.println("MilliSecond, Flow(lpm) , Press(Bar) , Temp(DegC)"); // Column header data
        logFile.close();
      }
      else {
        tft.setTextSize (2);
        tft.setTextColor(TFT_RED);
        tft.fillRect(0, 0, 320, 25, TFT_WHITE);
        print_at(10, 5, "SD CARD ERROR.NO LOGGING!"); // 25 char fill one line exact...
      }
    }

    //**************** WRITE REAL TIME DATA TO SDC ****************

    if ( millis() - loggingMs > loggingInterval )        // Time to write to SD card ??
    {
      int recInterval = millis() - loggingMs;
      loggingMs = millis();
      logFile = SD.open("FTPLOG.CSV", FILE_WRITE);       // Open the file to write

      char lpmData[10];
      dtostrf(lpmValFloat, 6, 2, lpmData);
      char pressData[10];
      dtostrf(barValFloat, 6, 2, pressData);
      char tempData[10];
      dtostrf(tempValFloat, 6, 2, tempData);
      if ( millis() - fileSizeMillis > fileSizeInterval )
      {
        fileSizeMillis = millis();
        char fileKb[10];
        sprintf(fileKb, "LOGGING. FileSize:%3dkB", logFile.size() / 1000);
        tft.setTextSize (2);
        tft.setTextColor(TFT_BLACK);
        tft.fillRect(0, 0, 320, 25, TFT_GREEN);
        print_at(10, 5, fileKb);
      }

      if (logFile)                                       // Check if the file opened okay
      {
        logFile.print(recInterval);                        // Yes now write the data
        logFile.print(',');
        logFile.print(lpmData);
        logFile.print(',');
        logFile.print(pressData);
        logFile.print(',');
        logFile.println(tempData);                      // One row of data complete.

        logFile.close();                                 // Write the data to SDC
      }
      else
      {
        tft.setTextSize (2);
        tft.setTextColor(TFT_RED);
        print_at(10, 5, "                         ");
        print_at(10, 5, "SD CARD ERROR.NO LOGGING!"); // 25 char fill one line exact...
      }
    }
  }
}

Excel_SnapShot.JPG
Excel_SnapShot.JPG394×529 46.8 KB

Excel_SnapShot.JPG
Excel_SnapShot.JPG394×529 46.8 KB

2

I'm not sure about the gaps, but I am curious why you are converting your float values to strings before print()'ing them to the SD card. The print() function can do that directly

     char lpmData[10];
      dtostrf(lpmValFloat, 6, 2, lpmData);
      //...
        logFile.print(lpmData);

vs.

        logFile.print(lpmValFloat,2);
3

Do I see you opening and closing the file each time you write to it? If so, why?

Paul

4

I have seen 'gaps' in SD card records myself.

Some brands\types of SD cards have the problem, some do not.

5

To find the answer, you need to look at the raw data file on the SD card, not the processed data from a spread sheet.

Paul

6

blh64:
I'm not sure about the gaps, but I am curious why you are converting your float values to strings before print()'ing them to the SD card. The print() function can do that directly

     char lpmData[10];

dtostrf(lpmValFloat, 6, 2, lpmData);
      //...
        logFile.print(lpmData);



vs.

logFile.print(lpmValFloat,2);

Yes of course !! Why did I do that ?

Thanks for pointing it out.

7

Paul_KD7HB:
Do I see you opening and closing the file each time you write to it? If so, why?

Paul

Just to make sure that if something goes wrong I don't loose records. OK maybe I dont need to write every record and maybe write once in 10 Seconds or so when I update the File size on display.

Will do that.

8

Paul_KD7HB:
To find the answer, you need to look at the raw data file on the SD card, not the processed data from a spread sheet.

Paul

Have also viewed the raw data on a notepad and also in the serial monitor. The gaps are there. Maybe its got something to do with the SD card type.

I was particular that I dont do anything more than this write to disk function and hence the loop() looks like this

void loop(void) {

  if (f_logDataToDisk == false) readSerial();  // Read Serial port only when Logging is OFF.

  if (f_mainScreen) {
    processMainScreen();
  }
}