bitmaps to External EEPROM

Hi all,
I have seen plenty of tutorials on writing 1 or 2 bytes to external eeprom. But I was wondering how would I be able to write a whole bitmap (like an image for a ST7789 oled display) to the eeprom chip. anyone has any Idea/resources/code examples to share? Should I do something like for looping wire writing until all the bits are written? thanks!

Arduino has

void setup() // for things that need to run once first or just run once

void loop() // for things that need to run over and over

You don't need an extra loop in void loop(), just a view of code that is not a top to bottom path from start to end.

void loop()
{
.........

if ( data available )
{
if ( data is not end marker )
{
// write the data
}
}
}

It's the same as writing 2 times 1 byte, just repeat more times until it's all written.

By the way, most external EEPROM take blocks of 64 bytes at a time. More efficient like that. See this library as example. Works just like the built-in EEPROM class, including writing complete data structures regardless of size (as is implemented by the EEPROM.put() and EEPROM.get() functions). As long as it fits in the Arduino's RAM, this allows you to read/write the bitmap in one go.

So the EEPROM library has a buffer, maybe a double-buffer?

wvmarle:
It's the same as writing 2 times 1 byte, just repeat more times until it's all written.

By the way, most external EEPROM take blocks of 64 bytes at a time. More efficient like that. See this library as example. Works just like the built-in EEPROM class, including writing complete data structures regardless of size (as is implemented by the EEPROM.put() and EEPROM.get() functions). As long as it fits in the Arduino's RAM, this allows you to read/write the bitmap in one go.

Thanks! It looks promising, but how do I find the library's documentation? I'm always a bit lost on GitHub, I can't find a list of all the functions/objects etc. included in the library in order to learn how to use it.

There is a whole manual's worth of material on the main Arduino site.

https://www.arduino.cc/en/Reference/EEPROM

There are 3 horizontal white bars on the right side of the teal band at the top of my forum pages. I click that to get to the RESOURCES section that leads to most of that material. The forum is in the COMMUNITY section and so is the PLAYGROUND with tons more material. The forum itself is also an info source but it's harder to search.

Here is an off-Arduino-site link to the C libraries that Arduino IDE uses;
https://www.nongnu.org/avr-libc/user-manual/modules.html

I never wrote formal documentation for it. Look at the standard EEPROM library's docs for an overview of available functions and/or just read the .h file of my library. It's pretty short and has lots of comments. I basically re-implemented the EEPROM library to work with external I2C EEPROM.
Note that I never used it with other EEPROM chips as I never needed them, I expect it to work just the same as long as the structure of the EEPROM and the I2C interface are the same. This seems to be pretty standard.

GoForSmoke:
So the EEPROM library has a buffer, maybe a double-buffer?

The library I linked to is my implementation of the internal EEPROM interface but then for use with external EEPROM. This to make it very easy to use either type. I was mostly after the put() and get() functions as those allow you to write/read any data type. The 24LC256 EEPROM chip has a 64-byte write buffer, so you can write 64 bytes of data with some 70 bytes of total I2C communication (including start/stop, I2C address, command). That's not buffered on the Arduino but in the I2C chip. Reading likewise: the chip allows you to request up to 64 bytes at a time, but as the Arduino has only 32 bytes of I2C buffer have to do this 32 bytes at a time. The ESP8266 can read chunks of 64 bytes just fine.

The Arduino EEPROM library does not buffer at all, it basically writes (reads) bytes one at a time to (from) the Arduino's internal EEPROM.

GoForSmoke:
There is a whole manual's worth of material on the main Arduino site.

Arduino - EEPROM

There are 3 horizontal white bars on the right side of the teal band at the top of my forum pages. I click that to get to the RESOURCES section that leads to most of that material. The forum is in the COMMUNITY section and so is the PLAYGROUND with tons more material. The forum itself is also an info source but it's harder to search.

Here is an off-Arduino-site link to the C libraries that Arduino IDE uses;
avr-libc: Modules

Yeah, I really like Arduino ressource page, however when I'm working with adafruit libraries for instance, they pretty much only have their GitHub and a very simple explantation of how to use a given library on their website, but no thorough list of functions. I guess what I have left in that case is the .h file, but it's not very intuitive, I can't tell which lines are actually declaring the functions, their arguments and what they are used for.

wvmarle:
I never wrote formal documentation for it. Look at the standard EEPROM library's docs for an overview of available functions and/or just read the .h file of my library. It's pretty short and has lots of comments. I basically re-implemented the EEPROM library to work with external I2C EEPROM.

Thank you so much

JCSB:
guess what I have left in that case is the .h file, but it's not very intuitive, I can't tell which lines are actually declaring the functions, their arguments and what they are used for.

The public block is where you see the functions you can use to interface with the library.

The private block is where you see functions and variables that are private and for internal use only, and which can not be accessed from the outside.

They usually also have a /examples directory with, you guessed it, examples on how to use the library. Usually those are all you need to get going, and those examples are a great test case. I'm using those examples a lot when testing sensors, just to make sure it's all wired up and readings are as expected etc.

I forgot that the EEPROM is external. It's real nice to know that the chip has its own buffer.

Is the chip SPI interface? If the serial is slow enough then Arduino can feed and read 1 byte at a time but SPI is not so slow that I'd be comfortable doing that.

Those EEPROM chips all seem to use I2C. At least the 24LC series iirc. The project I needed this for had only I2C available so never looked for SPI.

The real slowness is in writing: every time you write a page (or a single byte - that was another reason to really want to write 64 bytes at a time) it takes significant time to store this page. Also an issue with the Arduino's built-in EEPROM.

External Flash chips on the other hand are generally SPI. More memory for less money and much faster writes but can handle only a couple thousand writes vs. 10 million or so for the EEPROM. Come to think of it, SPI Flash may actually be more appropriate for the OP.

wvmarle:
Those EEPROM chips all seem to use I2C. At least the 24LC series iirc. The project I needed this for had only I2C available so never looked for SPI.

The real slowness is in writing: every time you write a page (or a single byte - that was another reason to really want to write 64 bytes at a time) it takes significant time to store this page. Also an issue with the Arduino's built-in EEPROM.

External Flash chips on the other hand are generally SPI. More memory for less money and much faster writes but can handle only a couple thousand writes vs. 10 million or so for the EEPROM. Come to think of it, SPI Flash may actually be more appropriate for the OP.

You think so? I was thinking about storing the bitmaps in the chip with an individual sketch, then simply read it when needed in my real sketch to print on displays. Which flash chip should be used? something like this? 5 PCS MX25L12835FM2I-10G SOP-8 MX 25L12835F M2I-10G SOIC-8 FLASH MEMORY | eBay
it actually looks more expensive than EEPROM, I bought 60 chips for 4$

With a battery backup to prevent power loss, possibly SPI RAM could work and not run out.

With AVR flash you have to write 128 byte blocks. If you write only 1 byte to the block, all 128 get written == wear. You get time to write all 128 but if you don't, 128 still get writ.

I have been told here that AVR EEPROM writes in 4 byte blocks and IIRC can change at least 100K times.

So far the only flash I write data to at runtime is on SD.

You think so? I was thinking about storing the bitmaps in the chip with an individual sketch, then simply read it when needed in my real sketch to print on displays.

If you know the patterns ahead of time, store them in flash using PROGMEM code.

GoForSmoke:
If you know the patterns ahead of time, store them in flash using PROGMEM code.

That's what I used to do, but it took like 20% of program space for a single bitmap. thats why I came up with the EEPROM solution. Won't it be better?

If you don't have room then yes since it is that or buy a bigger controller chip. The ATmega1284P has 128K flash and 16K RAM for instance (you can get boards with that chip or roll your own) and the Arduino Mega2560 has 256K flash. However I think that external EEPROM on an Uno would be cheaper.

Another possible is to use an SD module and card. You get gigs of space and can read/write SD on your PC.

So that'd be some 6 kB per image. Not much for an image file but a lot for an Arduino.

Flash chips are cheaper per byte - but maybe cost a bit more per piece. For your storage needs I can imagine an EEPROM chip is big enough. Slowish to write, but that's a one off.

Another thing to concern: EEPROM has a wide voltage range, 2-5.5V or so, while SPI flash requires 3.3V, max 3.6V or so, making it harder to interface with a 5V Arduino. Requires level shifting for the signals as well.

GoForSmoke:
With AVR flash you have to write 128 byte blocks. If you write only 1 byte to the block, all 128 get written == wear. You get time to write all 128 but if you don't, 128 still get writ.

With an ESP8266 it's even worse. Its blocks are 4096 bytes, and its EEPROM is emulated in Flash, a single block of it (so you get 4096 bytes of it). Flash wears out after some 10k writes or even less, much faster than EEPROM. That's where my project stems from. I have to write fairly regularly to EEPROM, a few times an hour or so, and that's not comfortable in flash when the project is supposed to operate for years.

AVR datasheet -- Write/Erase Cycles: 10,000 Flash/100,000 EEPROM

Those are guaranteed cycles, I would not be greatly surprised to get 33-50% more.

I have seen tests with the EEPROM of an ATmega which claimed to reach a couple million writes, before it broke.

The 24LC256 datasheet says you have at least 1 million writes.

GoForSmoke:
I have been told here that AVR EEPROM writes in 4 byte blocks and IIRC can change at least 100K times.

The AVR EEPROM is written in 4 byte ONLY by an external programmer. When written from a running program it is always accessed as single bytes. There is no way to write a whole page (to write faster) but you don't wear out other bytes in the page.

The size of Flash page depends of total Flash amount of given chip. It is always erased and written as a whole page. AFAIK only erasing (turning all bytes in a page to 0xff) causes Flash wear, not writing. But writing cannot turn 0 into 1.

Smajdalf:
The AVR EEPROM is written in 4 byte ONLY by an external programmer. When written from a running program it is always accessed as single bytes. There is no way to write a whole page (to write faster) but you don’t wear out other bytes in the page.

If I write byte 0 and then byte 7, what happens to bytes 1,2,&3?

The size of Flash page depends of total Flash amount of given chip. It is always erased and written as a whole page. AFAIK only erasing (turning all bytes in a page to 0xff) causes Flash wear, not writing. But writing cannot turn 0 into 1.

Like old Etch-a-Sketch. The old data is not preserved and new data is scraped locations. If you want to change less than the whole page you need a pre-erase-and-write copy, a 128 byte buffer.

Battery back the Arduino and spi SRAM will do for storage that won’t wear out.