Hi,
I am using an ESP32- Wrover module with the eeprom.h EEPROM emulation library to store program constants and other settings used during progam execution.
No EEPROM data is changed during program execution once the initial setup is done.
The problem is that after several days / weeks of normal program execution, the contents of the above "EEPROM locations" somehow become corrupted and data is lost.
Has anyone else encountered anything similar?
Any ideas?
It's probably better to drop this EEPROM library and move to the Preferences library
https://docs.espressif.com/projects/arduino-esp32/en/latest/tutorials/preferences.html
Thanks for the suggestion.
I will need to modify my sketch to use the new library above.
Are you aware of similar problems with the EEPROM.h library?
I stopped using the EEPROM simulation a long time ago. I don't remember having faced issues.
Hmm transition from eeprom.h to preferences.h doesnt seem too easy..
I use an eeprom "memory map" like this
const int memStatus = 1;
const int PCB_VER = 5;
const int PCB_REG = 6;
const int memDevType = 7;
const int memSwitches = 8;
const int memDevSnet = 10;
const int memDevID = 11;
const int memMac = 12;
const int memDevRem = 20;
...
.
Any ideas for an easy transition?
create a structure type representing everything you want to store permanently. You then just write the structure to memory or read it back. no need to handle adresses etc
When I was transitioning from the ESP EEPROM emulation to the Preferences class, I did the following using the original struct to minimise changes to the rest of the program. However, I changed rapidly to using the native Preferences getter and setter functions throughout.
Config.h
#ifndef _Config_h
#define _Config_h
#include <Arduino.h>
namespace nsConfig {
// change this to force a clean out and rebuild of the EEPROM data structures.
const uint16_t config_t_version = 1002 ; // increment this if you change config_t
struct config_t {
uint16_t eepromLayoutVersion; // copy of config_t_version
unsigned int runNumber ; // incremented on reboot
bool networkMode ; // false = standalone.
//screen calibration
float xScale ;
float xShift ;
float yScale ;
float yShift ;
//wlan
char ssid[32];
char psk[64];
char remoteHost[32] ;
// number translation rules
char ruleIntlFrom[6] ; // eg "00"
char ruleIntlTo[6] ; // eg "+"
char ruleLocalFrom[6] ; // eg "0"
char ruleLocalTo[6] ; // eg "+41"
char phpRetrievePath[32] ;
char phpStorePath[32] ;
char spamLevel ;
} ;
extern config_t config ;
// prototypes
void eepromLoad() ;
void eepromFetch() ;
void PrintEepromVariables() ;
void setRequestEepromReset() ;
void setup() ;
} // namespace nsConfig {
#endif
Config.cpp
/*
Config.cpp
Handle persistent configuration data
Uses ESP32 Preferences class
Adapted from an existing EEPROM model which used a struct to store the data.
In this "Preferences" model there are 2 keys, "config_t_ver" which indicates the
version of the struct config and "configKey" which is the struct itself. If the
format or size of the struct changes, then increment config_t_version which is
used to force a force a factory refresh and prevent the use of mismatched data.
An alternative would be to use the keys directly within a name space.
*/
#include "Config.h"
#include <Preferences.h>
namespace nsConfig {
Preferences preferences;
config_t config;
bool requestEepromReset = false;
char buf[120] = { 0 }; // make global ??
void ICACHE_FLASH_ATTR eepromFetch() {
Serial.println("in nsConfig::eepromFetch()");
// EEPROM => config
size_t configKeySize = preferences.getBytesLength("configKey");
// configPtr = (config_t *)&config;
if (sizeof(config) == configKeySize) {
} else {
snprintf(buf, sizeof(buf) - 1, "sizeof(config) struct: (%d); sizeof(config) key: (%d)\r\n", sizeof(config), configKeySize);
Serial.print(buf);
}
preferences.getBytes("configKey", &config, sizeof(config));
}
void ICACHE_FLASH_ATTR eepromLoad() {
Serial.println("in nsConfig::eepromLoad()");
// config => EEPROM
// EEPROM.put(0, config);
// delay(200); // taken over from ESP8266 version
// EEPROM.commit();
preferences.putBytes("configKey", &config, sizeof(config));
}
void setRequestEepromReset() {
requestEepromReset = true;
}
void forceFactoryReset() {
Serial.println("Config.cpp: Eeprom forceFactoryReset()");
// 'factory' settings
// For convenience add your own settings to avoid having to enter
// these initially in APmode through the web interface.
// SSID/psk are best removed at the end
// of any development and before publication.
config.eepromLayoutVersion = config_t_version; // redundant here
config.runNumber = 0;
strcpy(config.psk, "ggggg");
strcpy(config.ssid, "gggg");
config.networkMode = false;
// screen model specific - dependent on rotation / flipping of touch screen
config.xScale = 1.0 / 11.0;
config.xShift = 170;
config.yScale = 1.0 / 14.9;
config.yShift = 220;
strcpy(config.remoteHost, "www.gggg.com"); //
strcpy(config.ruleIntlFrom, "00");
strcpy(config.ruleIntlTo, "+");
strcpy(config.ruleLocalFrom, "0");
strcpy(config.ruleLocalTo, "+41");
strcpy(config.phpRetrievePath, "/Json.php");
strcpy(config.phpStorePath, "/storeData.php");
config.spamLevel = '1';
preferences.putUShort("config_t_ver", config_t_version);
eepromLoad();
}
void PrintEepromVariables() {
Serial.println("\nEEprom dump:");
snprintf(buf, sizeof(buf) - 1, "sizeof(config): (%d)\r\n", sizeof(config));
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "eepromLayourVersion: (%d) \r\n", config.eepromLayoutVersion);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "networkMode: (%d) \r\n", config.networkMode);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "ssid: (%s) \r\n", config.ssid);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "psk: (%s) \r\n", config.psk);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "remoteHost: (%s) \r\n", config.remoteHost);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "xScale: (%08.3f) \r\n", config.xScale);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "xShift: (%08.3f) \r\n", config.xShift);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "yScale: (%08.3f) \r\n", config.yScale);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "yShift: (%08.3f) \r\n", config.yShift);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "ruleIntlFrom: (%s) \r\n", config.ruleIntlFrom);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "ruleIntlTo: (%s) \r\n", config.ruleIntlTo);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "ruleLocalFrom: (%s) \r\n", config.ruleLocalFrom);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "ruleLocalTo: (%s) \r\n", config.ruleLocalTo);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "phpRetrievePath: (%s) \r\n", config.phpRetrievePath);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "phpStorePath: (%s) \r\n", config.phpStorePath);
Serial.print(buf);
snprintf(buf, sizeof(buf) - 1, "spamLevel: (%c) \r\n", config.spamLevel);
Serial.print(buf);
Serial.println("end of EEprom dump");
}
void setup() {
preferences.begin("config", false); // open RW mode
if (preferences.isKey("config_t_ver")) {
if (preferences.getUShort("config_t_ver") == config_t_version) {
snprintf(buf, sizeof(buf) - 1, "No change: preferences config_t_version= (%d) ; this config_t_version= (%d) \r\n", preferences.getUShort("config_t_ver"), config_t_version);
Serial.print(buf);
eepromFetch();
}
else {
snprintf(buf, sizeof(buf) - 1, "Mismatch: preferences config_t_version= (%d) ; this config_t_version= (%d) \r\n", preferences.getUShort("config_t_ver"), config_t_version);
Serial.print(buf);
forceFactoryReset();
}
}
else {
Serial.println("config_t_version key not found");
forceFactoryReset();
}
snprintf(buf, sizeof(buf) - 1,"There are: %u entries available in the namespace table.\r\n", preferences.freeEntries() );
Serial.print(buf);
PrintEepromVariables();
}
} // namespace nsConfig
Later, I used something similar to this showing an example of setting and getting individual preferences:
#include <Preferences.h> // ESP32 core
Preferences preferences;
. . .
preferences.begin("config", false); // open RW mode name space "config"
// if configuration information is not found in preferences then set default values
if (!preferences.isKey("input1")) preferences.putString("input1", "Factory input1");
if (!preferences.isKey("input2")) preferences.putString("input2", "Factory input2");
if (!preferences.isKey("input3")) preferences.putString("input3", "Factory input3");
if (!preferences.isKey("buttonPressedA")) preferences.putUShort("buttonPressedA", 0);
Serial.printf("input1 has value: (%s) \n", preferences.getString("input1").c_str());
Serial.printf("input2 has value: (%s) \n", preferences.getString("input2").c_str());
Serial.printf("input3 has value: (%s) \n", preferences.getString("input3").c_str());
Serial.printf("buttonPressedA has value: (%d) \n", preferences.getUShort("buttonPressedA"));
However, if you did not use a struct originally, then just replace each access to your variables with either a getter or setter function as appropriate.
Thanks for the example.
However, if you did not use a struct originally, then just replace each access to your variables with either a getter or setter function as appropriate.
No i didnt use a struct.
I just use
// definitions.h
const int memSSID = 500;
and then for example:
EEPROM.get(memSSID,ssid[0]);
The above takes place for around 200 different variables (of different sizes)
...replace each access to your variables with either a getter or setter function as appropriate.
You mean a different function for each variable ? (!)
Oh!
I think see what you are doing. Then your corruption problem could then be that you have made an error in your memory map somewhere say an overlap by not leaving enough space for a variable that you update at runtime.
Anyway, it does look like the struct option is the best for you if you have ~200 variables and don't want a large change throughout your code. My example shows you how to copy the "EEPROM" (now Preferences) into the struct, how to load the "EEPROM"(now Preferences) from data in the struct and how to access variables in the struct.
The alternative is a new getter and setter for each of your ~200 variables:
// set vars
preferences.putString("ssid", "PSK9oUJJj");
preferences.putUShort("devID", 126);
// use vars
Serial.printf("SSID has the value: (%s) \n", preferences.getString("ssid").c_str());
Serial.printf("devID has the value: (%d) \n", preferences.getUShort("devID"));
I thought of that too but its highly unlikely because:
(a) Storing data only happens once. The rest of the sketch just access it.
(b) I use the same "EEPROM memory map" with atmega1284p and external EEPROM with no problems for ...years. (!)
(c) I have re-checked the addressing again just in case with no errors
As for the transition, I wish there was someway to just do a find-and-replace on get/set commands in the existing sketch...
If you post the sketch we might be able to help…
Dealing manually with 200 parameters’ addresses is worth a one time investment to get rid of this and use a struct. It would be worth it as well for you on MCUs with real EEPROM too as it makes your life so much easier afterwards…
I am pretty sure that if you replace your 200 variables with a struct and if you have a variable foo then it would become config.foo from the struct, then ask chatGPT to replace all the places where you access one of the variables from the struct with config.variableName it will do it.
On an ESP32 an int is 4 bytes. On a atmega1284p it is 2 bytes. Have you consistently used int16_t etc. ?
Yeah - managing addresses should be done with sizeof and not hardcoded values if you want portability
Yeap!
I only asked because I saw this, although here it is not relevant for the storage size of the actual data item:
Anyway, corruption of flash storage, if that is indeed what it is, is a worrying phenomenon. It may not limit itself to the EEPROM area and may cause all sorts of havoc. Again, you could post your code so it could be checked for possible problem areas.
If you are migrating directly from a atmega1284p to an ESP32 there are other considerations regarding the EEPROM emulation, for example you must explicitly save the EEPROM area with EEPROM.commit();
This is indeed the correct way to handle the big block of data that you want to store and retrieve. The .putBytes() and .getBytes() methods can be used.
Here's a simple example.
#include <Preferences.h>
Preferences prefs;
void setup()
{
Serial.begin(115200);
struct settings
{
int onbefor;
int offabove;
bool automatic;
bool cent;
};
settings save1 = { 10, 18, true, false };
settings retrieve1 = { 0, 0, 0, 0 };
prefs.begin("Settings"); //namespace
prefs.putBytes("Settings", &save1, sizeof(save1));
prefs.getBytes("Settings", &retrieve1, sizeof(retrieve1));
Serial.println(retrieve1.onbefor);
Serial.println(retrieve1.offabove);
Serial.println(retrieve1.automatic);
Serial.println(retrieve1.cent);
}
void loop() {}