Problem - SD Card file does not match serial monitor output-SD file/watchdog

I am using the Arduino uno and the Catalex SD card reader.

This sketch uses the watchdog timer for a 8 second period, an SD card to write an output file and a print to the serial monitor for troubleshooting. The sketch blinks an LED for about .8 sec, there is a 1.2 sec delay, then the chip is put on low power for 8 seconds. I'm seeing an anomaly between the SD file and the serial printout especially with respect to the loop counter 'j'. On the serial monitor I get a smooth progression in 'j' from 1,2,3,4,...etc. as I expect. But on the SD file, the 'j' counter jumps around, eg, 1,2,1,1,2,1,2,3 before settling down to the expected progression 1,2,3,4,5,6,etc. Sometimes it even starts over again later in the progression. It's almost like the watchdog is doing some sort of timeout reset before settling down but if that's the case why the difference between the SD file and the serial monitor. It's pretty frustrating. I have no idea the cause but I've tried different things to no avail.

Any help would be greatly appreciated. This is my first use of the watchdog timer and an SD card but I'm not sure if it's the problem.

Thanks for any assistance and insight you can give.

// This version builds on the previous CitizenSensing file plus SD card and 12 hour circuit
// In this sketch, the LED's flash every 10 seconds, then at each 12 hour period, the relay is activated
// for 12 seconds (2 times per day).

// The SD card routine based largely on Aaron Lee's Tutorial
// Source: Aaron Lee Youtube: "Arduio Tutorial Lesson 5.1 - SD Card Write"

#include <SD.h>; // Arduino SD card library
#include <SPI.h>; // Although not shown in Aaron's tutorial, it is shown on some others. Will not compile
                  // without it in this sketch
/*
 * This sketch write text data to an SD card
 * Our SD card communicatres over the following pins
 * MOSI = pin 11 : This pin cannot be changed (SPI protocol requires this pin - assigned by library?)
 * MISO = pin 12 : This pin cannot be changed (SPI protocol requires this pin - assigned by library?)
 * Clock = pin 13 : This pin cannot be changed (SPI protocol requires this pin - assigned by library?)
 * Chip Select = pin 10 : This pin CAN be changed - ie yours may be different
 */

  File myFile;  // Create a data file for the SD card routine


// Source: citizen-sensing.org/2013/07/arduino-watchdog/ plus added code by JimC for LED routine
// Also I put the Source: code and created a function called sleepNow().

// Check this out for good explanation on the Watchdog timer and code.
// http://www.leonardomiliani.com/en/2013/impariamo-ad-usare-il-watchdog-2/

// SEE NOTE at the end on WDT reset issue

#include <avr/sleep.h>
#include <avr/power.h>
#include <avr/wdt.h>


int j = 1; // loop counter
int prevJ = 0; // previous j value 
float timePass = 0.0; // cummulative time
float hourSet = 12.0; // Set for 1/2 day (every 12 hours) relay activation cycle
float hourSetPrev = 0.0; // Previous hourSet

// Create sleepNow function
void sleepNow() {

  wdt_reset();
  wdt_disable();
  
  set_sleep_mode(SLEEP_MODE_PWR_DOWN); // Selecting the lowest power setting
   
// Set watchdog parameters
  MCUSR &= ~(1<<WDRF); // reset status flag
  WDTCSR |= (1<<WDCE) | (1<<WDE); // required to enable configuration changes
  WDTCSR = (1<<WDP0) | (1<<WDP3); // set the pre-scalar for 10, ie 8 seconds delay 
  WDTCSR |= (1<<WDIE); // enable interrupt mode
  
// Disable the ADC
  ADCSRA &= ~(1<<ADEN); // Set bit 7 in ADC register to 1 to disable the ADC  - Source Kevin Darrah  - the '~' invert symbol flips the bits so after
                        // setting bit 7 to 1, all the bits are flipped setting all other bits to 1 and bit 7 to 0
    
// Ready for sleep
  sleep_enable(); // Enable sleep function
  sleep_mode(); // Go to sleep.  Will wake up upon interrupt activation via Watchdog timer in this sketch.

// disable sleep after waking up
  sleep_disable(); // Disable sleep for now after Watchdog timer interrupt activation

}

void setup() {
  
  pinMode(4, OUTPUT); // set pin 4 as output for LED
  pinMode(7, OUTPUT); // set pin 7 as output for relay
  Serial.begin(9600);

  Serial.println("Initializing the SD card..."); // Print message so we know where we are at

  if (!SD.begin(10))  // if statement to test for SD initialization. The "10" refers to the chip select pin
                      // and specifies which chip we are talking to. Otherwise program goes to the first chip it finds.
  {
    Serial.println("Initialization failed!!!"); // ie, if SD.begin(10) not true
    return; // stops program here - exits the setup function and doesn't proceed
  }
    Serial.println("Initialization complete.");

}

void loop() {

  myFile = SD.open("pumpLog.txt", FILE_WRITE); // OPEN (create if does not exist already) the file to open it

  if (myFile) { // check if opened
    Serial.println("SD file opened successfully");
    
    myFile.println("SD File opened successfully"); // print to SD that file opened successfully
    myFile.print("J counter = ");
    myFile.print(j); // Print j counter to SD file
    myFile.print(" - timePass / hourSetPrev / hourSet = ");
    myFile.print(timePass);
    myFile.print(" / ");
    myFile.print(hourSetPrev);
    myFile.print(" / ");
    myFile.println(hourSet);

// Trouble shooting printout
    Serial.print("J counter = ");
    Serial.print(j); // Print j counter to SD file
    Serial.print(" - timePass / hourSetPrev / hourSet = ");
    Serial.print(timePass);
    Serial.print(" / ");
    Serial.print(hourSetPrev);
    Serial.print(" / ");
    Serial.println(hourSet);
   
    myFile.close(); // Must close file after writing to it.
  } else {
    Serial.println("Error in opening file");
  }

  // LED loop and period counter
  for (int i = 0; i < 2; i++) {
  digitalWrite(4, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(200);               // wait for 0.2 sec
  digitalWrite(4, LOW);    // turn the LED off by making the voltage LOW
  delay(200);              // wait for 0.2 second
  }
  delay(1200); // Total LED flash loop and this delay equals to 2 sec
 
  sleepNow(); // Start sleep/Watchdog routine

  timePass = ((2.0 + 8.0)*j)/3600.; // 2 seconds per LED flash loop + 8 second watchdog low power setting. timePass is the cummulative fraction of 1 hour
                                    // ie, 10 seconds occurs 6 times a minute x 60 minutes per hour
// Trouble shooting printout
  Serial.print("Time passed = ");
  Serial.println(timePass);

  prevJ = j;
  j = prevJ + 1; // 'j' is the multiplier counter used to increment the cummulative timePass period. It never resets to zero.

// Trouble shooting printout
   Serial.print("The values of prevJ and j are ");
   Serial.print(prevJ);
   Serial.print(" and ");
   Serial.println(j);

    if (timePass > hourSet) { // When timePass exceeds each 12 hour period, sketch then activates the relay for 12 seconds
    digitalWrite(7, HIGH); // Turn on relay
    delay(12000); // Activate relay for 12 seconds
    digitalWrite(7,LOW);
    hourSetPrev = hourSet;
    hourSet = hourSetPrev + 12.0; // Reset for next 12 hours
    
// Trouble shooting printout
    Serial.print("hourSet / hourSetPrev = ");
    Serial.print(hourSet);
    Serial.print(" / ");
    Serial.println(hourSetPrev);
    } 
}


ISR (WDT_vect) {
   
}

// The problem is that a RESET does not turn off the watchdog timer once it has been turned on, and neither does the bootloader.  So you set 
// things up to have the watchdog cause a reset in 15ms, and the reset occurs. Now the bootloader starts up, and starts its LED-flashing, and 
// OOPS another 15ms has gone by with no one feeding the watchdog, and it resets again.  This repeats until a power cycle turns off the watchdog.  
// (This behavior is explicitly described in the ATmega datasheet...)
  j = prevJ + 1; // 'j' is the multiplier counter used to increment the cummulative timePass period. It never resets to zero.

If you used a name that made sense, you wouldn't have to leave your self notes like this.

One thing I would change would be to close the file as soon as I got done writing to it. That flushes the buffer, committing the data to the file. The way the code is written now, if the device goes to sleep between writing to the card and closing the file, data could be lost.

What is the purpose of this sketch? Sleeping for 8 seconds at time, with 12 second delays and relays doesn't make sense to me. Is the Arduino battery powered? Is the thing that the relay controls battery powered, too?

Yes. The final version will be battery powered on a PCB. As a newbie I include a lot of comments make it easier for me to remember certain things. Once the final version is complete, I will remove most of the commentary. Since the WDT max interval is only 8 seconds, my objective is to put the circuit on low power for 80-90% of the time. The LED interval is just a visual way for me to see that the program is running as intended. In the final version the LED on-off period will be minimized to maximize the % time the circuit is sleeping. Re the relay, it will activate a separate motor (6-12v) that will be on its own battery. I'm using the WDT since sleep mode needs to be interrupted automatically but 8 seconds is the max interval unfortunately.

I tried moving the myFile.close(); statement to right before where the Serial.print statements start but this did not seem to change anything. Somehow it appears to be writing to the SD card for a few loops before writing to the serial monitor. Weird. It's very frustrating but I know there has to be an explanation that I just don't see right now.

I'll play with it more..maybe a new sketch and remove the WDT timer to see what happens.

Thanks so much for at least looking at this. ANY insights might lead to a solution and I'm grateful for any help.

You're missing the point of Paul's comment about the note. Use decent names for all your variables. And NEVER use single character names for global variables.

Ok. Got it.

To gain more insight, I added a millis() statement and printed it out on the SD file and the serial monitor. I get the results below. It seems to show that writing to the SD card begins even before the millis() print statement is recognized (???) and there is an extra '21 J counter' line printed to the SD card. (Note: Before running the sketch, I make sure to delete the SD file from the mirco SD card:

SD file:

SD File opened successfully
J counter = 1 - timePass / hourSetPrev / hourSet = 0.00 / 0.00 / 12.00
SD File opened successfully
J counter = 2 - timePass / hourSetPrev / hourSet = 0.00 / 0.00 / 12.00
SD File opened successfully
timeFromStart = 21 J counter = 1 - timePass / hourSetPrev / hourSet = 0.00 / 0.00 / 12.00
SD File opened successfully
timeFromStart = 21 J counter = 1 - timePass / hourSetPrev / hourSet = 0.00 / 0.00 / 12.00
SD File opened successfully
timeFromStart = 10115 J counter = 2 - timePass / hourSetPrev / hourSet = 0.00 / 0.00 / 12.00
SD File opened successfully
timeFromStart = 20175 J counter = 3 - timePass / hourSetPrev / hourSet = 0.01 / 0.00 / 12.00

Serial.monitor output:

Initializing the SD card...
Initialization complete.
SD file opened successfully
timeFromStart = 21 J counter = 1 - timePass / ... etc
SD file opened successfully
timeFromStart = 10115 J counter = 2 - timePass / ... etc
SD file opened successfully
timeFromStart = 20175 J counter = 3 - timePass / ... etc

Thanks for any help.

Well....since the serial monitor printout reflects what I expect I decided to dismiss the SD card file's first few lines and proceed with the rest of the project.

I decided to run the circuit via battery just to make sure things ran ok and then examined the SD card file ...LO and behold...the file reflects the serial monitor printout perfectly...OK..so now there must be gremlins when hooked up to the USB power.

Go figure!

Maybe you should disable your WDT before you write to file and enable it afterwards. Currently you only enable it.

This is based on the comment at the end of your code that a reset does not disable the watchdog; I did not check the datasheet if this is the case.

The diference in outputs you are seeing from Serial to SD Card is probably because you are not giving the SD Card enough time, and since the files are only saved when you correctly close it, your are losing some data.

SD Card prints take much longer then Serial, thats why you are only seeing the errors in one of them. I do something very similar but I save it to an array and only print it to the SD Card when the array is almost full, or once per hour.