Saving Data during power cycle.

I developed a people counting device but I find Arduino pro mini's (Clone) EEPROM very unreliable after using about two-month memory is getting corrupted. Is there any other reliable solution to save data during power cycle?

How often have you been writing to it? It wears out after 10^5 write cycles or so.

How often does a power-cycle happen -- or is this not something that can be predicted?

I get the impression that you're using EEPROM as primary memory, and thus are writing to it every time the count changes. If so, then consider this:

Come up with an "Arduino UPS" system [essentially an Uninterruptible Power Supply]. This could be as simple as a nine volt Alkaline battery connected across a Higher Voltage Supply [HVS] [like 12V or something] -- with diode switching. The higher voltage supply would be the one that power cycles. Each supply [the HVS, and the 9V Battery] would have a diode in series [with the Cathode on the Arduino side]. Both Cathodes would connect to the Vin pin. Why "Higher Voltage"? So, when it is connected, the diodes block current from the battery, and when it's not connected, the battery supplies power to the Arduino.

You could also use a MOSFET to switch power supplies when the main power supply goes down -- then the two supplies could be at the same voltage -- i.e. you could use a USB connection to power the Arduino, and have a 5V Li-Ion battery system as the backup--assuming it's a 5V Arduino--otherwise make that 3.3V [Here's a thread on that topic: http://forum.arduino.cc/index.php?topic=225004.0].

Then, place a voltage divider on the HVS, and feed the divided voltage [set to around 4.5V nominal] to one of the External Interrupt Input pins [typically pin 3] on the Arduino. [Here's a thread on that topic: http://forum.arduino.cc/index.php?topic=551378.new#new]. When the HVS is disconnected, the battery will keep the Arduino going long enough to allow it to save the latest count [maintained in volatile memory] to EEPROM, then put the Arduino to Sleep [to minimize load on the battery].

Here's a forum thread on how to do the Sleep thing, and how to wake up on external interrupt: https://forum.arduino.cc/index.php?topic=389768.0

The external interrupt would, of course, be caused by the main power supply being reconnected. The Interrupt routine would, then, read the count back, from the EEPORM, and resume people counting activities.

The idea being, to limit EEPROM activity to the [hopefully] occasional power-down backups -- and power up restores.

MarkT: How often have you been writing to it? It wears out after 10^5 write cycles or so.

Saving it every 20 minutes.

each device is running every day about 12-14 hour. it should last over 2000 days

Why not write the data to EEPROM only at power down?

See: https://forum.arduino.cc/index.php?topic=547472.0

ReverseEMF:
How often does a power-cycle happen – or is this not something that can be predicted?

I get the impression that you’re using EEPROM as primary memory, and thus are writing to it every time the count changes. If so, then consider this:

Come up with an “Arduino UPS” system [essentially an Uninterruptible Power Supply]. This could be as simple as a nine volt Alkaline battery connected across a Higher Voltage Supply [HVS] [like 12V or something] – with diode switching. The higher voltage supply would be the one that power cycles. Each supply [the HVS, and the 9V Battery] would have a diode in series [with the Cathode on the Arduino side]. Both Cathodes would connect to the Vin pin. Why “Higher Voltage”? So, when it is connected, the diodes block current from the battery, and when it’s not connected, the battery supplies power to the Arduino.

You could also use a MOSFET to switch power supplies when the main power supply goes down – then the two supplies could be at the same voltage – i.e. you could use a USB connection to power the Arduino, and have a 5V Li-Ion battery system as the backup–assuming it’s a 5V Arduino–otherwise make that 3.3V [Here’s a thread on that topic: http://forum.arduino.cc/index.php?topic=225004.0].

Then, place a voltage divider on the HVS, and feed the divided voltage [set to around 4.5V nominal] to one of the External Interrupt Input pins [typically pin 3] on the Arduino. [Here’s a thread on that topic: http://forum.arduino.cc/index.php?topic=551378.new#new]. When the HVS is disconnected, the battery will keep the Arduino going long enough to allow it to save the latest count [maintained in volatile memory] to EEPROM, then put the Arduino to Sleep [to minimize load on the battery].

Here’s a forum thread on how to do the Sleep thing, and how to wake up on external interrupt: https://forum.arduino.cc/index.php?topic=389768.0

The external interrupt would, of course, be caused by the main power supply being reconnected. The Interrupt routine would, then, read the count back, from the EEPORM, and resume people counting activities.

The idea being, to limit EEPROM activity to the [hopefully] occasional power-down backups – and power up restores.

Thanks for your suggestion.

The device needs to be switch off every day when the market is closed. I am saving data to the web server every 5 min. But for power failure/server failure I am also saving data in EEPROM. I can’t use UPS because everything needs to be powered off when the market is closed.

James_CA: Thanks for your suggestion.

The device needs to be switch off every day when the market is closed. I am saving data to the web server every 5 min. But for power failure/server failure I am also saving data in EEPROM. I can't use UPS because everything needs to be powered off when the market is closed.

Then, you only need the UPS feature during the time the Market is Open. During that time, only save to the EEPROM if there is a power failure. In fact, this can happen so fast, you could just use a very large capacitor--just large enough to ensure the there is enough time for the save. I mean, if anybody wants to raise a stink about it, ask them how different this is from any piece of electrical equipment that takes a few milliseconds to power down. The same mechanism would trigger an EEPROM save at the close of Market. Then, on power up, the Setup() function would retrieve the count from EEPROM.

So, most of the time, there will be one write [to EEPROM] at close of market, and one read at the beginning of the day. I.e. two accesses per day, vs around 96 per day [if the Market is open for 8hrs?]. A 1:48 reduction in activity.

Update: I assumed, when you said that you save to a Web Server every 5 min, that the EEPROM save frequency was the same. But, I read the other posts and see that it's every 20 min. So, that's 24 times a day, vs 2 or a 12:1 reduction.

Also, as MarkT pointed out, the datasheet for the ATmega328/P states that the EEPROM should survive at least 100,000 Write/Erase cycles. So, at 24 per day, that's 4167 days, or a little more than 11 years. And, at 2 per day, that's 137 years -- far longer than the expected data retention period of 20 years.

But, returning to the expected 11 years before failure estimate, has this thing been running for something like that long? Or is this a case of early failure? And, has there been more than one failure?

I dug into the datasheet a bit more and found this:

12.4.2. Preventing EEPROM Corruption During periods of low VCC, the EEPROM data can be corrupted because the supply voltage is too low for the CPU and the EEPROM to operate properly. These issues are the same as for board level systems using EEPROM, and the same design solutions should be applied. An EEPROM data corruption can be caused by two situations when the voltage is too low. First, a regular write sequence to the EEPROM requires a minimum voltage to operate correctly. Secondly, the CPU itself can execute instructions incorrectly, if the supply voltage is too low. EEPROM data corruption can easily be avoided by following this design recommendation: Keep the AVR RESET active (low) during periods of insufficient power supply voltage. This can be done by enabling the internal Brown-out Detector (BOD). If the detection level of the internal BOD does not match the needed detection level, an external low VCC reset Protection circuit can be used. If a reset occurs while a write operation is in progress, the write operation will be completed provided that the power supply voltage is sufficient.

Also, all Interrupts should be disabled during an EEPROM write. This may already be taken care of by whatever routines you are using to write to the EEPROM, but it couldn't hurt to do this anyway. Disable interrupts before the write, then enable them again after the write --UNLESS there is some sort of callback involved, then manually disabling/reenabling interrupts could cause problems.

24 EEPROM failure out of 58 devices. others are running fine (11 months). I am counting pulses via the Interrupts. So I didn't disable the Interrupts as it will miss the count.

James_CA: 24 EEPROM failure out of 58 devices. others are running fine (11 months). I am counting pulses via the Interrupts. So I didn't disable the Interrupts as it will miss the count.

That's fine, as long as an interrupt doesn't occur during an EEPROM write.

James_CA: I developed a people counting device but I find Arduino pro mini's (Clone) EEPROM very unreliable

Is it possible that the Clone Pro Minis are using non-geniune ATMega328s ?

"I am saving data to the web server every 5 min."

How?

If you use an ESP8266 for that, then you don't need a ProMini. The ESP has enough (4MB) SPIFFS memory to store 5min data for many months. Leo..

You're not saving much data..So why not use your E2prom in a ring buffer?

Write to each successive address for eg 10,000 times, then move on.

Allan