off chip data storage

I have looked around and havnt found any solutions that fit my needs, though i assume this is quite a common problem.

I need to find a way of storing nested matrixs when power is turned off. The nested matrices will be storing map information that my robot has gathered, so i will need to be able to read/write/alter the information during runtime.

All of the methods i have read up on either

do not allow matrix data
only allow data to be written during initial program upload
loose data when power turns off

if anyone has any solutions or idea they would be much appreciated

Have you looked into using an SD card shield.

I'd go with the SD card. If you wanted to really yak shave though you could serialize the data in EEPROM.

i have gotten halfway through McGivering a solution using the chips onboard EEPROM,

SD card still sounds like a better option, looks like they have come down in price quite a bit since i last bought one. how hard is it to format data onto one ? i've never used one in my projects before.

thanks for the suggestions

after reading up on SD cards, maybe something using I2C instead of SPI would be nice, since an I2C bus is already included on my custom PCB.

would there be a way to store matrix data in a larger offchip EEPROM that used I2C ?

You may want to have a look at:

SD card storage and really easy to use. Connects to a serial port (software serial port will work too).

See here for Arduino libraries:

http://www.roguerobotics.com/wikidocs/code/library/arduino/roguesd

You could save your matrices on shutdown, and load them back up on restart.

Heck, I'd even help you write the code to save a load the data. Piece of cake.

b

How big are the matrices?

If they are 512 bytes, or less, use the onboard EEPROM.

You can load and store at any time in the program, just do it when it's needed.
The code is pretty simple.

It's a holiday, so though this looks a bit homeworkey ...
Here's a pseudo-example of loading and storing a 16x16 'maze' (it compiles, but I HAVE NOT TESTED IT, so be careful).
Reading back should be comparably straightforward.

/* EEPROM Write Matrix
 * Stores values from a matrix into EEPROM for later retrieval.
 */

#include <EEPROM.h>

int done = false;  // flag to avoid repeatedly writing
#define WIDTH (16)
#define HEIGHT (16)
unsigned char maze[WIDTH][HEIGHT];

void setup() {}

void loop()
{
  if (done) return;  // finished, so just get out of here
  
  write_matrix();
  
  done = true;
}

void write_matrix()
{
  for (int addr=0; addr < sizeof(maze); ++addr) {
    EEPROM.write(addr, maze[addr/WIDTH][addr%HEIGHT]);
  }
}

A slightly less obvious, but actually simpler, and more general purpose approach is:

void write_matrix01()
{
  for (int addr=0; addr < sizeof(maze); ++addr) {
    EEPROM.write(addr, *((unsigned char*)maze+addr));
  }
}

This has the small benefit that it cares less what the data looks like. It could be an array of struct's and it would still work. If this is homework, you might want to avoid this approach unless you can explain it to the instructor, as it is a bit tricky.

loop() contains a flag to avoid repeatedly writing the matrix to EEPROM for this demo. The on-board EEPROM can be used over 100,000 times, but I still try to avoid writing unnecessarily. It takes about 3-4 milli seconds to write a byte, so the whole 256 bytes will take about a second. This is the significant downside.

If you need to save a few matrices, then make sure they don't overlap in EEPROM! You can do this using a bit of arithmetic and sizeof() which will give the memory size in bytes (char) of each matrix.

If all the data fits, and speed isn't a problem, then this is oodles easier and infinitely lower cost (you already have the Arduino :-)) than using an external memory.

To test it, put some easy to recognise data into the matrix before writing it to EEPROM, write it, then create a second sketch/program to read it back and print the values to the serial monitor.

HTH
GB-)

not homework :stuck_out_tongue: just something to keep my brain busy.

currently the matrix is 31x31 boolean. (side note, inside arduino do int and boolean variables use the same amount of ram ?)

i woud like to have it bigger as this would afford me greater precision, however memory requirements quickly get too large for the atmega 328

speed is not a concern, however i am almost positive my 31x31 matrix will overflow the on board EEPROM.

i have considered just feeding a few key coordinates into the EEPROM such as locations of doors, charging station, and grid position at power off, as these could be almost as useful as storing the whole matrix, however i really would like to get the data logging down, if for nothing else than knowing how to do it for my next project.

i will give your suggestion a test,
would the same method work one of those cheap external I2C EEPROMs?

regards

(side note, inside arduino do int and boolean variables use the same amount of ram ?)

The easy way to find the size of any variable is:

    Serial.println(sizeof(var));

upload, and the program will tell you how big the storage is.

currently the matrix is 31x31 boolean.

If you use a different storage technique, that matrix is only 31x4 = 124 bytes, which will easily fit into the onboard 512 byte EEPROM.
A boolean value can also be packed into a single bit (it only represents LOW or HIGH, true or false), 8 values in a byte, and 32 in 4 bytes.

So, if you pack booleans into bits, then 512 bytes of EEPROM are enough to store 4096 values, or a 64 x 64 matrix. It does make the code a bit more complex, but careful use of macros could make it bearable.

would the same method work one of those cheap external I2C EEPROMs?

I'm not sure which part of the method you mean. The way to read and write external EEPROM over I2C is pretty different.

HTH
GB-)

I ordered a cheap external EEPROM, i think for now i will settle for just storing critical coordinates, instead of the whole matrix,

GB your solution of crunching the data is intriguing, but i think its a little above my level at the time, i will defiantly come back to that idea when i have a bit more programming skill.

Thanks to all of you for the help

I ordered a cheap external EEPROM, i think for now i will settle for just storing critical coordinates, instead of the whole matrix

A particular method to keep in mind (if most of the area you are mapping is considered "empty") is the concept of "sparse arrays":

There are tons of techniques out there to "pack" what seems to be an amazing amount of data into a very small space - two of the best areas to learn about these techniques are from the game programming and demo coder worlds (especially those demos optimized for very small memory footprints; you would be floored by what can be done in 256 bytes).

Also, a review of certain "large" computer games from the 1970s and 1980s might be beneficial; many of these games (especially the larger scope turn-based and real-time strategy games) used fairly interesting techniques to pack their "worlds" (analogous to your maps) in small areas (aka, floppies) - granted, floppies have more space than you have available (most generally), but the techniques could be helpful.

:slight_smile:

GB your solution of crunching the data is intriguing, but i think its a little above my level at the time, i will defiantly come back to that idea when i have a bit more programming skill.

Bit packing is straightforward, and probably less complex than getting the off-chip storage to work.

for example, you could have something like

boolean maze[31][31];
unsigned long mazebits[31]; // same number of values as maze[31][32]

void setup() {}

void loop ()
{
int row = 7;
int column = 23;
boolean val;

maze[row][column] = true;
bitWrite(mazebits[row], column, true);

maze[row][column] = false;
bitWrite(mazebits[row], column, false);

val = maze[row][column];
val = bitRead(mazebits[row], column);

maze[row][column] = val;
bitWrite(mazebits[row], column, val);
}

Use bitWrite(), and bitRead() to handle a boolean value in the "row" of values.
mazebits[] is only 31*4 (size of long) = 124 bytes because each value of true or false is packed into 1 bit.

HTH
GB