thoughts on esp32 EEPROM management?

i'm wondering how to manage saving configuration parameters in eeprom, recognizing when the eeprom has valid data and dealing with esp32 being used with different applications storing data in the eeprom

i believe it makes sense to read parameters from eeprom at system startup. but shouldn't be done if the eeprom doesn't have valid data - is virgin (all zeros), contains some other applications data or is corrupted. So a magic value similar to those used in some file formats seems useful and some sort of CRC/checksum to verify the the data is valid.

but considering that the esp32 could be used for other purposes using the eeprom, i was thinking that instead of using the lower address space of the eeprom, to start at a higher one (e.g. 1k).

one other consideration is format/content changes. adding a version field would allow an older save of the eeprom to be used instead of completely overwritting newer parameters with uninitialized eeprom data.

not sure how adequate a simple checksum is. wondering what other options there are.

of course many of these issues are irrelevant once the project is completed, but the first user of a product is its developers

wondering what others have done and what issues i haven't recognized?

thanks
greg

Have you considered using SPIFFS ? It would allow you to store data in named files thus avoiding reading data written by another program

would that require that any other application also use SPIFFs?

Use SPIFFs. It will be MUCH simpler, and you'll have FAR more space to play with.

Regards,
Ray L.

gcjr:
would that require that any other application also use SPIFFs?

Only if those other "applications" needed to read/write to SPIFFS

how is the EEPROM space allocated for SPIFFs protected against normal EEPROM access?

another aspect of this I'm trying to avoid is creating a struct with all the eeprom parameters. That of course would make it easy to store/retrieve a single object from either memory or a file.

the reason is to keep var names short because their pointer values are used in tables and i'd like to keep the table descriptions short

the table describing the var locations and lengths serves other purposes as well

i'm eager for an answer to reply #5 in order to better understand SPIFFs

This is from Libraries · ESP8266 Arduino Core and although it relates to the EPS8266 I would expect the ESP32 to follow suit

EEPROM
This is a bit different from standard EEPROM class. You need to call EEPROM.begin(size) before you start reading or writing, size being the number of bytes you want to use. Size can be anywhere between 4 and 4096 bytes.

EEPROM.write does not write to flash immediately, instead you must call EEPROM.commit() whenever you wish to save changes to flash. EEPROM.end() will also commit, and will release the RAM copy of EEPROM contents.

EEPROM library uses one sector of flash located just after the SPIFFS.

gcjr:
how is the EEPROM space allocated for SPIFFs protected against normal EEPROM access?

It's not. SPIFFS uses FLASH, not EEPROM.

Regards,
Ray L.

ESP32 does not have EEPROM.

ESP32, use SPIFF. SPIFFS Filesystem - ESP32 - — ESP-IDF Programming Guide latest documentation

thanks

it looks like the esp32 EEPROM is limited to 4k bytes out of the 4x16 MBytes of flash. so eeprom and spiffs are separate areas of flash

From EEPROM maximum size - ESP32 Forum

ESP32 Arduino libraries emulate EEPROM using a sector (4 kilobytes) of flash memory. The total flash memory size is (for most of the mass produced modules) 4MB, although some are produced with 8 or 16 MB. The entire space is split between bootloader, application, OTA data, NVS, SPIFFS, and EEPROM. You can find the exact layout in the following partition table: https://github.com/espressif/arduino-es … efault.csv. Other files in the same directory provide alternative partition tables. You can read more about partition tables in ESP-IDF programming guide: Partition Tables - ESP32 - — ESP-IDF Programming Guide latest documentation

thanks
wonder if EEPROM isn't simply a SPIFF file or the last few blocks of the files system.

just getting back to this.

various web pages describe python and Arduino IDE plug-ins to create/upload SPIFF file

but I need to be able send the app to someone and the .ino file recognize no file exist and create and possibly create the SPIFFS.

am i understanding this correctly?

i'm struggling trying to find the correct function and arguments to write binary data (uint8_t) to a file. FS.h include via SPIFFS.h describes

   int read() override;
    size_t read(uint8_t* buf, size_t size);
    size_t readBytes(char *buffer, size_t length)

my code. createFile() and readFile() write/read a single string. struct Desc_t describes a struct displayed, written and read using descDisp() descWrite() and descRead().

i think descwrite() may work as expected, but descRead() is not; it seems to read the same three bytes several times before the read() returns 0.

// https://techtutorialsx.com/2018/08/05/esp32-arduino-spiffs-writing-a-file/

const char version [] = "Spiff - 200501c";

#include "SPIFFS.h"

const char *filename = "/test.dat";
File  file;

#define N_CHAR 80
char s [N_CHAR];

// -----------------------------------------------------------------------------
void createFile (
    const char *s )
{
    Serial.println (__func__);
    file = SPIFFS.open (filename, FILE_WRITE);

    if (file.print (s))
        Serial.println ("File was written");
    else
        Serial.println ("File write failed");

    file.close ();
}

// ---------------------------------------------------------
// attempt to read file
void readFile (void)
{
    Serial.println (__func__);

    file = SPIFFS.open (filename, FILE_READ);

    int n = file.readBytes (s, N_CHAR-1);
    s [n] = 0;

    Serial.print   ("s: ");
    Serial.println (s);
}

// -----------------------------------------------------------------------------
typedef struct {
    int  a;
    int  b;
    int  c;
} Desc_t;

Desc_t desc [] = {
    {   1,   2,   3 },
    {  10,  29,  10 },
    { 101,  80, 999 },
    {   0,   0,   0 },
};

// ---------------------------------------------------------
void
descDisp (
    Desc_t  *d )
{
    for ( ; d->a; d++)  {
        sprintf (s, " %6d %6d %d", d->a, d->b, d->c);
        Serial.println (s);
    }
}

// ---------------------------------------------------------
void
descRead (void)
{
    Desc_t  d;

    Serial.println (__func__);
    file = SPIFFS.open (filename, FILE_WRITE);

    while (file.read (file.read ((uint8_t*) &d, (size_t)sizeof(Desc_t))))  {
        if (0 == d->a)
            break;

        sprintf (s, " %6d %6d %d", d->a, d->b, d->c);
        Serial.println (s);
    }

    file.close ();
}

// ---------------------------------------------------------
void
descWrite (
    Desc_t *d )
{
    Serial.println (__func__);
    file = SPIFFS.open (filename, FILE_WRITE);

    for ( ; d->a; d++)  {
        if (! file.write ((uint8_t*)d, sizeof(Desc_t)))  {
            Serial.println ("write failed");
            return;
        }
    }

    file.close ();
}

// -----------------------------------------------------------------------------
void setup () {
    Serial.begin (115200);
    Serial.println (version);

    // check if file exist or needs to be created
    if (!SPIFFS.begin (true)) {
        Serial.println ("SPIFFS.begin () failed");
    }
}

// -----------------------------------------------------------------------------
// read a string terminated by \n
int
readString (
    char *s,
    int   maxChar )
{
    int  n = 0;

    Serial.print ("> ");
    do {
        if (Serial.available()) {
            int c    = Serial.read ();

            if ('\n' == c)
                break;

            s [n++] = c;
            if (maxChar == n)
                break;
        }
    } while (true);

    return n;
}

// ---------------------------------------------------------
// process single character commands from the PC
#define MAX_CHAR  10

void
pcRead (void)
{
    static char s [MAX_CHAR] = {};
    static int  val = 0;

    if (Serial.available()) {
        int c = Serial.read ();

        switch (c)  {
        case '0':
        case '1':
        case '2':
        case '3':
        case '4':
        case '5':
        case '6':
        case '7':
        case '8':
        case '9':
            val = c - '0' + (10 * val);
            break;

        case 'd':
            descDisp ();
            break;

        case 'c':
            createFile ("how now brown cow");
            break;

        case 'R':
            readFile();
            break;

        case 'r':
            descRead ();
            break;

        case 'v':
            Serial.print ("\nversion: ");
            Serial.println (version);
            break;

        case 'w':
            descWr (desc);
            break;

        case '\n':          // ignore
            break;

        case '"':
            while ('\n' != Serial.read ())     // discard linefeed
                ;

            readString (s, MAX_CHAR-1);
            Serial.println (s);
            break;

        case '?':
            Serial.println ("\npcRead:\n");
            Serial.println ("    [0-9] append to #");
            Serial.println ("    c   - create file");
            Serial.println ("    r   - read file");
            Serial.println ("    v   - print version");
            Serial.println ("    \"   - read string");
            Serial.println ("    ?   - this list of commands");
            break;

        default:
            Serial.print ("unknown char ");
            Serial.println (c,HEX);
            break;
        }
    }
}

// -----------------------------------------------------------------------------
void
loop (void)
{
    pcRead ();
}

think i sorted out above -- if begin fails, just write a file