EEPROMex library - An extension of the standard Arduino EEPROM library

Hi,

For my ongoing clock project, I wanted to persistently store some data after turning off the Arduino. This should be easy since Arduino board has an on-board EEPROM. Unfortunately the Arduino standard EEPROM library only implements functionality for reading and writing a single bytes, which makes the EEPROM quite cumbersome to use.

This is why I wrote the EEPROMex library, an extension of the standard Arduino EEPROM library. It extends the functionality of the original Arduino EEPROM library with:

  • Reading, writing to basic types like bytes, longs, ints, floats & doubles
  • Reading, writing to single bits
  • Reading, writing of any data format (for example structs)
  • Reading, writing of arrays of any format
  • Updating only changed bytes rather than rewriting all (thus reducing wear)
  • (Very) basic memory management
  • Debug functionality: limit number of writes. limit writing in memory range

You can download the library here:
http://thijs.elenbaas.net/downloads/?did=3

And find detailed explanation and samples of the functionality here
http://thijs.elenbaas.net/2012/07
/extended-eeprom-library-for-arduino

A simple example of reading and writing ints and floats:

#include <EEPROMex.h>
// Always get the EEPROM in the same order
// This will ensure that each variable gets
// the same adress after every restart
int addressInt   = EEPROM.getAddress(sizeof(int));
int addressFloat = EEPROM.getAddress(sizeof(float));

double IntVarInput    = 1234;
double IntVarOutput   = 0;

double FloatVarInput  = 101.101;
double FloatVarOutput = 0;

void setup()
{
Serial.begin(9600);

// Write an int to EEPROM, and then read it back
EEPROM.writeInt(addressInt, IntVarInput);
IntVarOutput = EEPROM.readInt(addressInt);
Serial.print(IntVarOutput);
Serial.println();

// Write a float to EEPROM, and then read it back
EEPROM.writeFloat(addressFloat, FloatVarInput);
FloatVarOutput = EEPROM.readDouble(addressFloat);
Serial.print(FloatVarOutput);
Serial.println();
}
void loop() {}

Let me know what you find!

[UPDATE: The previous version had some bugfixes. Please download the update version. The new version also comes with support for newer Arduino boards and Teensy boards!]

Before I start spending more time in this library than needed, some questions:

  • Are many of you arduino aficionado's out there are using the EEPROM for storage? If you aren't using it, is that because you don't need it, or because of some lacking functionality?
  • Are there people using external SPI connected EEPROM (or even RAM)? Would it be useful to generalize the backend of this library so that it would also work for SPI connected storage?
  • Are there people using the very nice flash library by Mikal Hart (Flash | Arduiniana)? Would it be useful to, in a similar manner, implement object oriented variables that can recall /store themselves from EEPROM?

Please let me know if you find bugs or have suggestions for future improvements.

There is also Arduino Playground - EEPROMWriteAnything available already.

Indeed! In fact, there are a lot of snippets and small libraries around that implement some form of EEPROM reading/writing to extend the standard library. The one you mention is probably one of the most general (and useful?), but there are also

http://arduino.cc/playground/Code/EepromUtil
http://tushev.org/articles/electronics/47-modified-eeprom-library-for-arduino
http://blog.ncode.ca/?p=42

And so on. There is no rocket science in this library, it is meant mostly for convenience. What I figured was lacking was a library that I could use as a drop-in replacement for the EEPROM library, that implemented both the basic types like, int, float, double with a syntax very similar to the standard lib, as well as more complex types like structs and arrays.

Still, this library introduces some functionality that I could not really find elsewhere:

  • Updating only changed bytes, instead of blindly rewriting all of them. In practice I am often writing more or less the same data to EEPROM. Updating instead of writing then saves a lot EEPROM wear.

  • A maximum on the number of bytes that you can write. In theory you could burn out a memory cell in a few minutes: a write/erase cycle takes approximately 4 ms, so writing 100.000 times to a single cell takes 6 1/2 min. It happened to me that I put an EEPROM write statement within a loop rather than outside, an only after a minute or so running noticed that my Arduino didn't seem to be doing anything useful. So far, it seems that my trusted Arduino survived :).

  • Basic Memory allocation functionality. It is basically just a counter of the last allocated byte. This is also implemented in newer AVR libraries, but not in the one that ships with the Arduino.

  • Targeting single bits. I haven't needed it myself yet, but I found it as a suggestion somewhere, and figured it should be easy enough to add. The idea is that writing single bits can help you use the 1Kb that you have with maximum efficiency.

if this tests out,
it would be a great improvement / addition to the standard lib.

drjiohnsmith:
if this tests out, it would be a great improvement / addition to the standard lib.

I agree. I was fiddling with the builtin EEPROM library, making a way to store events and timestamps, and ran into the problem that timestamps wouldn't store into 1 byte. I eventually wound up packing them in a really odd way with a 12.3 second resolution or some such so I could fit them in 2 bytes--as I recall, the first byte was the week and the second byte was split into the day & time in some strange way. It didn't really work too well. A library like this would have been really helpful.

drjiohnsmith:
if this tests out,
it would be a great improvement / addition to the standard lib.

Well, that would of course be nice :slight_smile: In fact, I would prefer that the Arduino came with a lot more user-created libraries by default (or would provide an easy way of getting to them, perhaps using a package manager), so people would start building on work of others rather that reinventing the wheel every time.

But for now, I would appreciate it very much if people would testdrive the library and see what is missing and if bugs pop up. I put the library on the Playground here, and am in the process of writing an library on top of EEPROMex that implements variables that can recall /store themselves from EEPROM.

Any chance of getting this to work with the 24LC256 I2C eeprom?

@mdn15: Not without some real work. I have thought about this: the class could call different EEPROM writers/readers, which would make the underlying storage transparent to the user. However, I have not had many request for this (not many people seem to use external EEPROMs) so it is not high on my priority list.

What was higher on the list was to fix a small mistake in capitalization (a capital mistake?), which would give build errors on case-sensitive environments. This has now been fixed.

Great - I just stumbled across the limitations of the standard arduino EEPROM library, and this will be real helpful

Thanks for sharing!

tip: make a note about memory used if you include this library, this is always an issue on the 168/328p based boards
tip: most RTC ds1302 modules sold on ebay include a i2c EEPROM - supporting external EEPROMs and in particular this module with a simple example would be a great idea. Most vendors don't even seem to know it is on their module (let alone the optional ds18b20!)

Any chance of getting this to work with the 24LC256 I2C eeprom?

seen this - Arduino Playground - LibraryForI2CEEPROM - one?

wrappers for other datatypes are fairly simple, something like this

void writeFloat(unsigned int address, float value)
{
  writeBlock(address, &value , sizeof(float));
}

This is a great idea to fix the Arduino EEPROM library but why not use the excellent support already available in AVR?

The compiler supports marking a variable as stored in EEPROM with the attribute EEMEM. This allows symbolic handling of EEPROM addresses instead of the manual method forced by the Arduino EEPROM library. Also there are already all the needed functions for different data size access in AVR EEPROM, avr-libc: <avr/eeprom.h>: EEPROM handling

#include <avr/eeprom.h>

int ex EEMEM;
...
eeprom_write_word(&ex, 42);
...
int x = (int) eeprom_read_word(&ex);

The really bad new is that the Arduino build does not support the initialization of EEPROM. This is not uploaded with the sketch. Otherwise you could initialize the EEPROM variables directly.

int ex EEMEM = 42;

Using EEPROM such 24LC256 requires access through an I2C/TWI driver. There are a few around.
Cheers!

@kowalski

How does the program know/map the location of the variables when using EEMEM?
Or does it just start at address 0 etc?

But if I reorder my vars, does it reorder the mem addresses?

(I know I can write a few sketches to find out)

it just sequentially stores them
if you change the order in which you 'declare' the variables you mess it up
you can set a starting address

look at the example 'EEPROMEx' - run it and examine the output - it more or less explains itself

not specifically noted - but you can hard code the address - or use a variable of your own

this is just a convenient way to store variables without overlap or empty gaps

I haven't looked into the other examples - they seem to use another (smarter?) scheme to store variables

if you want - I'd be happy to share a code snippet of what I have so far

edit oops sorry - misread and thought you were talking about the EEPROMEx library

if you change the order in which you 'declare' the variables you mess it up

What I expected, thanks for the explanation

robtillaart:
How does the program know/map the location of the variables when using EEMEM?
Or does it just start at address 0 etc?
But if I reorder my vars, does it reorder the mem addresses?

The Arduino EEPROM library requires you to keep track of the variables in memory. Using the attribute EEMEM works just as any other variable in global space (SRAM) but allocated in the EEMEM address space. The address of a variable is the answer to your first question. Second question, yes, the compiler/linker starts from address zero(0) in the EEPROM. Third, changing the order of EEMEM tagged variables will reorder them in EEPROM but you should be using the symbolic reference (&ex).

int ex EEMEM;
...
eeprom_write_word(&ex, 42); // &ex is the address to the variable
...
int x = (int) eeprom_read_word(&ex);

The actual address of the variable in EEPROM will be calculated by the compiler/linker when building the sketch. This means that libraries may have variables in EEPROM without forcing the library user to allocate them in the address space. This is what the Arduino EEPROM library forces. This is totally unreasonable for larger software systems with many components/libraries. This type of dependency must be avoided.

The AVR EEPROM support and GCC does the work for you.

Cheers!

robtillaart:

if you change the order in which you 'declare' the variables you mess it up

What I expected, thanks for the explanation

@robtillaart

Sorry for the very late reply. Missed your questions.

The bottom-line here is that when using the EEMEM attribute the variables become symbolic. Otherwise you are handling the eeprom address space yourself. Possible but tedious.

I wouldn't go as far as calling it a mess up when the compiler/linker remaps the variables when they are changed, removed, etc. The program should use the symbols and not care about the actual address.

Having that said there are some performance optimizations possible with eeprom as most devices have a page mode which is fast update of a "row" in the memory. Padding variables in page boundaries can give higher performance.

Last, consider if the SRAM would be handled as the Arduino EEPROM library. Programmers would have to keep a map of the memory usage.

Cheers!

No problemo - you're quite busy with other thingies :slight_smile:

Hi there,

I need some help with the example code for the EEPROMex-library shown below:

#include <EEPROMex.h>

// ID of the settings block
#define CONFIG_VERSION "ls1"

// Tell it where to store your config data in EEPROM
#define memoryBase 32

bool ok  = true;
int configAdress=0;

// Example settings structure
struct StoreStruct {
    char version[4];   // This is for mere detection if they are your settings
    int a, b;          // The variables of your settings
    char c;
    long d;
    float e[6];
} storage = { 
    CONFIG_VERSION,
    220, 1884,
    'c',
    10000,
    {4.5, 5.5, 7, 8.5, 10, 12}
};

void setup() {
  EEPROM.setMemPool(memoryBase, EEPROMSizeUno); //Set memorypool base to 32, assume Arduino Uno board
  configAdress  = EEPROM.getAddress(sizeof(StoreStruct)); // Size of config object 
  ok = loadConfig();
}

void loop() {
  // [...]
  int i = storage.c - 'a';
  // [...]
  storage.c = 'a';
  if (ok)
    saveConfig();
  // [...]
}

bool loadConfig() {
  EEPROM.readBlock(configAdress, storage);
  return (storage.version == CONFIG_VERSION);
}

void saveConfig() {
   EEPROM.writeBlock(configAdress, storage);
}

I do not understand how to read one item from my store struct. If I try then I always get strange results. For example, what does the line

  int i = storage.c - 'a';

really do? It does not make much sense to me. Even if I output i over the serial monitor I only get strange results.

A more commented example would be very very helpful. Maybe one which reads and writes each element of the sorage struct. I just don´t get it.
Thanks for any help in advance.

Greetings,
Jan

kowalski:
The really bad new is that the Arduino build does not support the initialization of EEPROM. This is not uploaded with the sketch. Otherwise you could initialize the EEPROM variables directly.

I implemented this on Teensy 2.0. In hindsight, it turned out to be less than ideal.

The main problem is when a user writes "EEMEM unsigned long my32bits;" they expect it to retain its value across code uploads. But unfortunately, that's not how the compiler works. It will create an EEMEM section with all uninitialized variables set to zero. So when you build a tool like Teensy Loader (or hypothetically, some future version of Arduino) that automatically detects the EEMEM section and writes it to the eeprom, you wipe out the previous state on every upload. That's technically the correct behavior based on the ELF file the compiler generates, but it isn't a behavior that makes Arduino users happy. In fact, people generally hate it.

To really make this work in a way that users intuitively expect, what's needed are two EEMEM sections, perhaps EEMEM and EEMEM.NOINIT. The variables without initialized values would go into the other section, which would not be overwritten during upload. Initialized variables would get set to their specific value on every upload.

I may yet do this for Teensy3, because it uses a custom linker file, as is the norm when compiling for nearly all ARM chips. For AVR, straying from the built-in linker script is just asking for big trouble.