EEPROM Management: A Modest Proposal

Other Hardware Storage
1

ABSTRACT:

A convention to make EEPROM and perhaps other memory management easier at low cost.

BACKGROUND:

Arduino devices provide a small amount of on-board EEPROM memory. This can be quite useful to store infrequently changed information, such as a calibration table for a touch screen or identification information.

However, the standard EEPROM library only provides read and write access for absolute addresses. This is fine if only a single library is used in a project. Multiple libraries may attempt to use the same addresses and interfere. Managing multiple libraries requires modification of the libraries or mutual awareness of the libraries, which is suboptimal.

What is needed is a mechanism by which multiple libraries can use an EEPROM space without modification and the need to be mutually aware.

At the same time, the EEPROM is limited (1K on the Arduino Uno), so fancy storage mechanisms such as resource forks would be costly.

PROPOSAL:

That a convention be adopted for EEPROM and possible other memory management, as follows:

  1. That memory be allocated as a series of contiguous blocks, starting at address 0 as per the EEPROM library.
  2. That the first 1 byte (or more) of each block shall be a UTF-8 encoded number giving the length of the block in bytes, excluding the UTF-8 length.
  3. That the sequence of blocks be terminated by a single null byte.

RATIONALE:

A block with fewer than 128 bytes only requires a single byte to determine the length, which isn't much. No block on the standard Arduino Uno EEPROM will require more than two extra bytes. The Arduino has a lot more power than memory, so a linear search isn't bad. Sketches that consume blocks can recognize the blocks they need by length and content and can easily skip over blocks they do not need.

If there is something that does the same thing, even if differently, I'll be happy to use it. If not, I'd like to see if we can get agreement and come up with a standard. I'm willing to write a library for this, which should be easy, and let Arduino promulgate it.

If not, I'll do it anyway, because I need it for two libraries I have written, which may or may not be used together.

If there is a more appropriate forum for this, please feel free to move it. I was unable to find a better one.

2

Good idea,

First: are you familiar with - Arduino Playground - HomePage -

some years ago I posted - AGHW-Database handling-EDB library - #3 by robtillaart - Libraries - Arduino Forum - and started prototyping an EEprom database called ADB based on that post but never finished it quite satisfactory. As other projects were screaming for time too this project tumbled down my todo list :frowning:

  1. That the sequence of blocks be terminated by a single null byte.

If you put the length in the first byte yo do no need a terminator. It would even bring problems if some sketch stores a binary 0 in it

I would advise to have a look at the how 1st generation of floppies did their administration. It could have 112 files in the root folder and that was it. The directory was a hardcoded table with a linked list of blocks of bytes. As the UNO memory is rather small I would propose 16 files in the root folder. A root folder entry looks like

struct fileEntry 
{
  int offset;  // points to memory chunk
  int deleted: 1;  // only one bit
  int future: 7;  // other meta info can be stored here
} root[16];

struct memoryChunk
{
  int offset; to next chunck or 0;
  byte data[CHUNKSIZE];
}

my 2 cents