Data Logger Crash

I am using Arduino UNA and try to make a data logger with PZEL004-T. This device works well. Data are recorded on a SD card every 8 sec. A new file is created every day and the flush() used every 50 record.
The problem is that after about 3 days, the system crash and sometimes the SD card need reformating.
I have tried SD ad SdFAT library, other SD card, other SD model but no difference
Does anyone has an idea about the cause of that problem

Welcome to the forum

The problem is almost certainly in your code. Do you think it might help if you posted it here ?

@michou51, your topic has been moved to a more suitable location on the forum. Installation and Troubleshooting is not for problems with (nor for advice on) your project.

The idea that came to my mind is "millis()" Does this short posting help you?

I guess it does not help. There is no way around posting your code so please post your

complete sketch

You can post code by using this method that adds the code-tags
There is an automatic function for doing this in the Arduino-IDE
just three steps

  1. press Ctrl-T for autoformatting your code
  2. do a rightclick with the mouse and choose "copy for forum"
  3. paste clipboard into write-window of a posting

best regards Stefan

Hi, @michou51
Welcome to the forum.

Thanks.. Tom... :smiley: :+1: :coffee: :australia:

This this the code of the problem. Record crash after about 3200 lines. Auto file generation is well working. Crash occurs in the middle of the day but not specific action. Power is OK

/*
  Copyright (c) 2021 Jakub Mandula

  Example of using one PZEM module with Software Serial interface.
  ================================================================

  If only RX and TX pins are passed to the constructor, software
  serial interface will be used for communication with the module.
*/

#include <PZEM004Tv30.h>
#include <SoftwareSerial.h>
//#include <SPI.h>
#include <SD.h>
#include <RTClib.h>
#include <LowPower.h>
#include <Wire.h>

#define ledRec 7
#define ledOFF 8

String POWER = "";

int count2max = 100; // nb d'enregistrement de dara avant flush()
int count2 = 0;
int jour = 0;

//#if defined(ESP32)
//#error "Software Serial is not supported on the ESP32"
//#endif

/* Use software serial for the PZEM
   Pin 2 Rx (Connects to the Tx pin on the PZEM)
   Pin 3 Tx (Connects to the Rx pin on the PZEM)
*/
#if !defined(PZEM_RX_PIN) && !defined(PZEM_TX_PIN)
#define PZEM_RX_PIN 3
#define PZEM_TX_PIN 2
#endif


SoftwareSerial pzemSWSerial(PZEM_RX_PIN, PZEM_TX_PIN);
PZEM004Tv30 pzem(pzemSWSerial);
File myFile;
RTC_DS1307 rtc;

//_______________________________________________
void setup() {
  /* Debugging serial */
  Serial.begin(9600);

  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }
  pinMode(ledRec, OUTPUT);

  Serial.print(F("Ini SD cardN.."));

  if (!SD.begin(10)) {
    Serial.println(F("ini failed!"));
    digitalWrite(ledOFF, HIGH);
    while (1);  // la ledOFF reste allumée en continu pas la ledRec verte
  }
  Serial.println("ini done.");

  rtc.begin();

  if (! rtc.isrunning()) {

    //Serial.println("RTC is NOT run.set the time!");
    // When time needs to be set on a new device, or after a power loss, the
    // following line sets the RTC to the date & time this sketch was compiled
    rtc.adjust(DateTime(F(__DATE__), F(__TIME__)));
    // This line sets the RTC with an explicit date & time, for example to set
    // January 21, 2014 at 3am you would call:
  }
  pzem.resetEnergy();
}


//______________________________________________________
void loop() {

  Serial.print("Cust Addr:");
  Serial.println(pzem.readAddress(), HEX);
  // if (!SD.begin(10)) {
  //  Serial.println(F("ini failed!"));
  // digitalWrite(ledOFF, HIGH);
  //  while (1);
  // }
  DateTime now = rtc.now();
  delay(100);
  if (jour !=  now.day()) {

    myFile.close();  //on ferme le fichier pour en ouvrir un autre

    //char POWER[13];
    jour = now.day();
    int heure = now.hour();
    int MIN = now.minute();

    POWER = "";
    POWER = String(jour) + "-" + String(heure) + "-" + String(MIN) + ".txt";

    myFile = SD.open(POWER, FILE_WRITE);
    //count=count+1;
    myFile.println(F("ann,mois,jour,H,min,sec,Volt,Amp,Power,KWh,Freq,PF"));

    // close the file:
    //myFile.close();
    Serial.println("Titres OK");
    // Serial.println("done.");
    delay(500);
    //N=0;
  }

  //myFile = SD.open(POWER, FILE_WRITE);

  // Read the data from the sensor
  float voltage = pzem.voltage();
  float current = pzem.current();
  float power = pzem.power();
  float energy = pzem.energy();
  float frequency = pzem.frequency();
  float pf = pzem.pf();

  // Check if the data is valid
  if (isnan(voltage)) {
    Serial.println("Error readvoltage");
    digitalWrite(ledOFF, HIGH);
    // pas de voltage, vert et OFF allomé
  }
  if (voltage < 10  || voltage > 300) {
    Serial.println("Error readvoltage");
    digitalWrite(ledOFF, HIGH);
    // mauvais voltage, vert et OFF allomé
    digitalWrite(ledRec, HIGH);
  } else {
    // Print the values to the Serial console
    // Serial.print("Voltage: ");      Serial.print(voltage);      Serial.println("V");
    //Serial.print("Current: ");      Serial.print(current);      Serial.println("A");
    //Serial.print("Power: ");        Serial.print(power);        Serial.println("W");
    //Serial.print("Energy: ");       Serial.print(energy,3);     Serial.println("kWh");
    //Serial.print("Frequency: ");    Serial.print(frequency, 1); Serial.println("Hz");
    //Serial.print("PF: ");           Serial.println(pf);

    if (myFile) {

      digitalWrite(ledOFF, LOW);
      DateTime now = rtc.now();
      delay(100);
      myFile.print(now.year(), DEC);
      myFile.print(';');
      myFile.print(now.month(), DEC);
      myFile.print(';');
      myFile.print(now.day(), DEC);
      myFile.print(";");
      myFile.print(now.hour(), DEC);
      myFile.print(';');
      if (now.minute() < 10) {
        myFile.print('0');
      }
      myFile.print(now.minute(), DEC);
      myFile.print(';');
      if (now.second() < 10) {
        myFile.print('0');
      }
      myFile.print(now.second(), DEC);
      myFile.print(",");
      myFile.print(voltage); myFile.print(",");
      myFile.print(current); myFile.print(",");
      myFile.print(power);   myFile.print(",");
      myFile.print(energy, 3); myFile.print(",");
      myFile.print(frequency, 1); myFile.print(",");
      myFile.println(pf);
      //myFile.close();
      // N=N+1;
      count2 = count2 + 1;
      digitalWrite(ledRec, HIGH);
      delay(200);
      digitalWrite(ledRec, LOW);
      if (count2 > count2max) {
        myFile.flush();
        delay(500);
        count2 = 0;
      }
    }
    //delay(1000);
  }

  //delay(500);
  LowPower.powerDown(SLEEP_8S, ADC_OFF, BOD_OFF);
  //LowPower.powerDown(SLEEP_8S, ADC_OFF, BOD_OFF);

}    // loop finish

Hi,

What is your power supply?

Thanks.. Tom... :smiley: :+1: :coffee: :australia:

I have never used the low-power-mode of an arduino.
Thi means I have no experience what happens if you go to low-power-mode and what happens on wake up to connected hwardware and how the SPI-interface deals with this.

I would do a flush and closing the file each and every time before you enter low-power-mode.

best regards Stefan

The power is 12volt about 6000mah capacity from lithium pack
Presently under test the line where voltage is between 10 and 300volt... in case pzem04t returns strange character then written

I will tell you in 4 days if there is improvement
Ps with or without low consumption option makes no difference except hang after 3 to 4 days if low power and about every 30 h if constant acquisition

-------- Message d'origine --------

Initially flush was done every 100 record, now reduced to 50trecord.. no change
I will test flush every time after this trial

-------- Message d'origine --------

For testing purposes:

Can you connect a computer to the USB-cable connected to your arduino?
Terminalsoftware can record and store the serial received data to the harddisk.

This will enable to see if your code crashes or something goes wrong.

I have another suggestion:
SD needs quite a lot of memory for buffers.
write a modifcated version of your code that makes use of PString or SafeString
instead of variable-String
Variabletype String eats up new memory each time you assign a new value
This can lead to crashes.

best regards Stefan

Your application for me is an interesting testcase for using the SafeString-library

I modified your code to use SafeString and I modified each and every hardcoded string with the F-macro.

example:

 Serial.println( F("ini done.") );

the f-macro stores the characters into flash-rom and this means you get more free RAM
After the modifications the RAM-usage is
Global Variables use 1420 Bytes (69%) of RAM , 628 Bytes free for local variables

Your original version had
Global Variables use 1349 Bytes (65%) of RAM , 699 Bytes free for local variables

With SafeString the RAM-use of the SafeString stays constant.

Without SafeString it can be brought down to
Global Variables use 1305 Bytes (63%) of RAM , 743 Bytes free for local variables

This means the SafeString-libary needs 115 bytes of additional RAM but it stays at that.

@J-M-L : is there a possability to "measure" free RAM on runtime?

If it is really a too less RAM-problem how about using an Arduino Mega?

here is the SafeString-code

/*
  Copyright (c) 2021 Jakub Mandula

  Example of using one PZEM module with Software Serial interface.
  ================================================================

  If only RX and TX pins are passed to the constructor, software
  serial interface will be used for communication with the module.
*/

#include <PZEM004Tv30.h>
#include <SoftwareSerial.h>
//#include <SPI.h>
#include <SD.h>
#include <RTClib.h>
#include <LowPower.h>
#include <Wire.h>
#include <SafeString.h>

#define ledRec 7
#define ledOFF 8

createSafeString(POWER, 13); 

//String POWER = "";

int count2max = 100; // nb d'enregistrement de dara avant flush()
int count2 = 0;
int jour = 0;

//#if defined(ESP32)
//#error "Software Serial is not supported on the ESP32"
//#endif

/* Use software serial for the PZEM
   Pin 2 Rx (Connects to the Tx pin on the PZEM)
   Pin 3 Tx (Connects to the Rx pin on the PZEM)
*/
#if !defined(PZEM_RX_PIN) && !defined(PZEM_TX_PIN)
#define PZEM_RX_PIN 3
#define PZEM_TX_PIN 2
#endif


SoftwareSerial pzemSWSerial(PZEM_RX_PIN, PZEM_TX_PIN);
PZEM004Tv30 pzem(pzemSWSerial);
File myFile;
RTC_DS1307 rtc;

//_______________________________________________
void setup() {
  /* Debugging serial */
  Serial.begin(9600);

  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }
  pinMode(ledRec, OUTPUT);

  Serial.print(F("Ini SD cardN.."));

  if (!SD.begin(10)) {
    Serial.println(F("ini failed!"));
    digitalWrite(ledOFF, HIGH);
    while (1);  // la ledOFF reste allumée en continu pas la ledRec verte
  }
  Serial.println( F("ini done.") );

  rtc.begin();

  if (! rtc.isrunning()) {

    //Serial.println("RTC is NOT run.set the time!");
    // When time needs to be set on a new device, or after a power loss, the
    // following line sets the RTC to the date & time this sketch was compiled
    rtc.adjust(DateTime(F(__DATE__), F(__TIME__)));
    // This line sets the RTC with an explicit date & time, for example to set
    // January 21, 2014 at 3am you would call:
  }
  pzem.resetEnergy();
}


//______________________________________________________
void loop() {

  Serial.print( F("Cust Addr:") );
  Serial.println(pzem.readAddress(), HEX);
  // if (!SD.begin(10)) {
  //  Serial.println(F("ini failed!"));
  // digitalWrite(ledOFF, HIGH);
  //  while (1);
  // }
  DateTime now = rtc.now();
  delay(100);
  if (jour !=  now.day()) {

    myFile.close();  //on ferme le fichier pour en ouvrir un autre

    //char POWER[13];
    jour = now.day();
    int heure = now.hour();
    int MIN = now.minute();
    //                 123456789012                   
    //POWER = "";    //31-23-59.txt 
    //POWER = String(jour) + "-" + String(heure) + "-" + String(MIN) + ".txt";
    POWER = "";
    if (jour < 10) {
      POWER += F("0"); 
    }
    POWER += jour;  
    POWER += F("-");
    
    if (heure < 10) {
      POWER += F("0"); 
    }    
    POWER += heure;
    POWER += F("-"); 

    if (heure < 10) {
      POWER += F("0"); 
    }
    POWER += MIN;
    POWER += F(".txt");

    myFile = SD.open(POWER.c_str(), FILE_WRITE);
    //count=count+1;
    myFile.println( F("ann,mois,jour,H,min,sec,Volt,Amp,Power,KWh,Freq,PF") );

    // close the file:
    //myFile.close();
    Serial.println( F("Titres OK") );
    // Serial.println("done.");
    delay(500);
    //N=0;
  }

  //myFile = SD.open(POWER, FILE_WRITE);

  // Read the data from the sensor
  float voltage = pzem.voltage();
  float current = pzem.current();
  float power = pzem.power();
  float energy = pzem.energy();
  float frequency = pzem.frequency();
  float pf = pzem.pf();

  // Check if the data is valid
  if (isnan(voltage)) {
    Serial.println( F("Error readvoltage") );
    digitalWrite(ledOFF, HIGH);
    // pas de voltage, vert et OFF allomé
  }
  if (voltage < 10  || voltage > 300) {
    Serial.println( F("Error readvoltage") );
    digitalWrite(ledOFF, HIGH);
    // mauvais voltage, vert et OFF allomé
    digitalWrite(ledRec, HIGH);
  } else {
    // Print the values to the Serial console
    // Serial.print("Voltage: ");      Serial.print(voltage);      Serial.println("V");
    //Serial.print("Current: ");      Serial.print(current);      Serial.println("A");
    //Serial.print("Power: ");        Serial.print(power);        Serial.println("W");
    //Serial.print("Energy: ");       Serial.print(energy,3);     Serial.println("kWh");
    //Serial.print("Frequency: ");    Serial.print(frequency, 1); Serial.println("Hz");
    //Serial.print("PF: ");           Serial.println(pf);

    if (myFile) {

      digitalWrite(ledOFF, LOW);
      DateTime now = rtc.now();
      delay(100);
      myFile.print(now.year(), DEC);
      myFile.print( F(";") );
      myFile.print(now.month(), DEC);
      myFile.print( F(";") );
      myFile.print(now.day(), DEC);
      myFile.print( F(";") );
      myFile.print(now.hour(), DEC);
      myFile.print( F(";") );
      if (now.minute() < 10) {
        myFile.print( F("0") );
      }
      myFile.print(now.minute(), DEC);
      myFile.print( F(";") );
      if (now.second() < 10) {
        myFile.print( F("0") );
      }
      myFile.print(now.second(), DEC);
      myFile.print( F("," ) );
      myFile.print(voltage); myFile.print( F(",") );
      myFile.print(current); myFile.print( F(",") );
      myFile.print(power);   myFile.print( F(",") );
      myFile.print(energy, 3); myFile.print( F(",") );
      myFile.print(frequency, 1); myFile.print( F(",") );
      myFile.println(pf);
      //myFile.close();
      // N=N+1;
      count2 = count2 + 1;
      digitalWrite(ledRec, HIGH);
      delay(200);
      digitalWrite(ledRec, LOW);
      if (count2 > count2max) {
        myFile.flush();
        delay(500);
        count2 = 0;
      }
    }
    //delay(1000);
  }

  //delay(500);
  LowPower.powerDown(SLEEP_8S, ADC_OFF, BOD_OFF);
  //LowPower.powerDown(SLEEP_8S, ADC_OFF, BOD_OFF);

}    // loop finish

Edit: additional

If the Stringvariable "POWER" is local to a function the content of the variable is stored in a different way and the RAM is freed after each exit of the function.

So I put the realted commands into a function

WriteToSD()

RAM usage 1289 bytes free RAM for local variables 759 bytes.

/*
  Copyright (c) 2021 Jakub Mandula

  Example of using one PZEM module with Software Serial interface.
  ================================================================

  If only RX and TX pins are passed to the constructor, software
  serial interface will be used for communication with the module.
*/

#include <PZEM004Tv30.h>
#include <SoftwareSerial.h>
//#include <SPI.h>
#include <SD.h>
#include <RTClib.h>
#include <LowPower.h>
#include <Wire.h>


#define ledRec 7
#define ledOFF 8

int count2max = 100; // nb d'enregistrement de dara avant flush()
int count2 = 0;
int jour = 0;

//#if defined(ESP32)
//#error "Software Serial is not supported on the ESP32"
//#endif

/* Use software serial for the PZEM
   Pin 2 Rx (Connects to the Tx pin on the PZEM)
   Pin 3 Tx (Connects to the Rx pin on the PZEM)
*/
#if !defined(PZEM_RX_PIN) && !defined(PZEM_TX_PIN)
#define PZEM_RX_PIN 3
#define PZEM_TX_PIN 2
#endif


SoftwareSerial pzemSWSerial(PZEM_RX_PIN, PZEM_TX_PIN);
PZEM004Tv30 pzem(pzemSWSerial);
File myFile;
RTC_DS1307 rtc;

//_______________________________________________
void setup() {
  /* Debugging serial */
  Serial.begin(9600);

  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }
  pinMode(ledRec, OUTPUT);

  Serial.print(F("Ini SD cardN.."));

  if (!SD.begin(10)) {
    Serial.println(F("ini failed!"));
    digitalWrite(ledOFF, HIGH);
    while (1);  // la ledOFF reste allumée en continu pas la ledRec verte
  }
  Serial.println( F("ini done.") );

  rtc.begin();

  if (! rtc.isrunning()) {

    //Serial.println("RTC is NOT run.set the time!");
    // When time needs to be set on a new device, or after a power loss, the
    // following line sets the RTC to the date & time this sketch was compiled
    rtc.adjust(DateTime(F(__DATE__), F(__TIME__)));
    // This line sets the RTC with an explicit date & time, for example to set
    // January 21, 2014 at 3am you would call:
  }
  pzem.resetEnergy();
}


void WriteToSD() {

  DateTime now = rtc.now();

  String POWER = "";

  if (jour !=  now.day()) {

    myFile.close();  //on ferme le fichier pour en ouvrir un autre

    //char POWER[13];
    jour = now.day();
    int heure = now.hour();
    int MIN = now.minute();
    //                 123456789012
    //POWER = "";    //31-23-59.txt
    POWER = String(jour) + "-" + String(heure) + "-" + String(MIN) + ".txt";
    myFile = SD.open(POWER, FILE_WRITE);

    // Read the data from the sensor
    float voltage = pzem.voltage();
    float current = pzem.current();
    float power = pzem.power();
    float energy = pzem.energy();
    float frequency = pzem.frequency();
    float pf = pzem.pf();

    // Check if the data is valid
    if (isnan(voltage)) {
      Serial.println( F("Error readvoltage") );
      digitalWrite(ledOFF, HIGH);
      // pas de voltage, vert et OFF allomé
    }
    if (voltage < 10  || voltage > 300) {
      Serial.println( F("Error readvoltage") );
      digitalWrite(ledOFF, HIGH);
      // mauvais voltage, vert et OFF allomé
      digitalWrite(ledRec, HIGH);
    } else {
      // Print the values to the Serial console
      // Serial.print("Voltage: ");      Serial.print(voltage);      Serial.println("V");
      //Serial.print("Current: ");      Serial.print(current);      Serial.println("A");
      //Serial.print("Power: ");        Serial.print(power);        Serial.println("W");
      //Serial.print("Energy: ");       Serial.print(energy,3);     Serial.println("kWh");
      //Serial.print("Frequency: ");    Serial.print(frequency, 1); Serial.println("Hz");
      //Serial.print("PF: ");           Serial.println(pf);

      if (myFile) {

        digitalWrite(ledOFF, LOW);
        DateTime now = rtc.now();
        delay(100);
        myFile.print(now.year(), DEC);
        myFile.print( F(";") );
        myFile.print(now.month(), DEC);
        myFile.print( F(";") );
        myFile.print(now.day(), DEC);
        myFile.print( F(";") );
        myFile.print(now.hour(), DEC);
        myFile.print( F(";") );
        if (now.minute() < 10) {
          myFile.print( F("0") );
        }
        myFile.print(now.minute(), DEC);
        myFile.print( F(";") );
        if (now.second() < 10) {
          myFile.print( F("0") );
        }
        myFile.print(now.second(), DEC);
        myFile.print( F("," ) );
        myFile.print(voltage); myFile.print( F(",") );
        myFile.print(current); myFile.print( F(",") );
        myFile.print(power);   myFile.print( F(",") );
        myFile.print(energy, 3); myFile.print( F(",") );
        myFile.print(frequency, 1); myFile.print( F(",") );
        myFile.println(pf);
        //myFile.close();
        // N=N+1;
        count2 = count2 + 1;
        digitalWrite(ledRec, HIGH);
        delay(200);
        digitalWrite(ledRec, LOW);
        if (count2 > count2max) {
          myFile.flush();
          delay(500);
          count2 = 0;
        }
      }
      //delay(1000);
    }
  }
}



//______________________________________________________
void loop() {

  Serial.print( F("Cust Addr:") );
  Serial.println(pzem.readAddress(), HEX);
  // if (!SD.begin(10)) {
  //  Serial.println(F("ini failed!"));
  // digitalWrite(ledOFF, HIGH);
  //  while (1);
  // }
  delay(100);
  WriteToSD();
  
  //delay(500);
  LowPower.powerDown(SLEEP_8S, ADC_OFF, BOD_OFF);
  //LowPower.powerDown(SLEEP_8S, ADC_OFF, BOD_OFF);

}    // loop finish

best regards Stefan

Why not just use C style strings and be done with it ?

post the code-modification that does it this way

The main change appears to be this

    jour = now.day();
    int heure = now.hour();
    int MIN = now.minute();
    char POWER[30]; //plenty of room for the output
    sprintf(POWER, "%s-%s-%s.txt", jour, heure, MIN);

does this add leading zeros for numbers < 10?

No, but a quick change of format specifier (and fixing my data type mistakes) will do the job

sprintf(POWER, "%02d-%02d-%02d.txt", jour, heure, MIN);

Not sure why this is just for me but yes and no. You can look at the number of bytes between the stack and heap - that’s your headroom but this won’t account for holes you might have poked into the heap through dynamic allocation (see https://www.seeedstudio.com/blog/2021/04/26/managing-arduino-memory-flash-sram-eeprom/)

I think the headroom is the amount of contiguous memory theoretically available for dynamic allocation at that time. However, from that must be subtracted overhead for the allocation processes itself and of course margin for stack growth (downward). Even the call to malloc() itself will cause the stack to grow and reduce the headroom.

The "holes" in the area between the bottom of the heap and its current top are actually more free that can be allocated. malloc() will (should?) try to allocate that memory the next time it's called. It will (should?) allocate the smallest "hole" that's at least as large as the requested number of bytes. If no "holes" are large enough, it will allocate from the current top of the heap upwards ... causing the headroom to be reduced.

Have you got that going to the DC jack or Vin of the UNO?

Can we please have a circuit diagram?
An image of a hand draw schematic will be fine, include ALL power supplies, component names and pin labels.
Please include all hardware you have that uses the 5V pin of the UNO.

There is a linear 5V regulator on the UNO, can you feel if it is getting hot?

Thanks... Tom... :smiley: :+1: :coffee: :australia: