Writing data to flash memory; a practical solution?

Moderators, please feel free to merge with my previous thread if appropriate

I'd like to control a large number of devices by reading from an array that contains information regarding which devices to turn on and when to turn them on/off. For various reasons, these devices have 4 parameters associated with them. These parameters include time, device address and a few other special parameters.

Basically, the rough idea was to implement a code that looks something like:

for(int i = 0; i <= 720; ++i){
   if (millis() >= *Array[0][i] && millis() < *Array[0][i] + 10) {
      digitalWrite(Array[1][i], HIGH);
      digitalWrite(Array[2][i], HIGH);
   }
}

In reality, it is slightly more complicated, but the crux of the matter is that the first row of the array has to store 4 bytes per entry. I only need to store 1 byte for the rest of the entries and I posted a thread earlier regarding this problem.

It occurred to me, however, that I could do all of my manipulations on this array on a separate, larger processor with ample RAM and then pass this to the Arduino. I thought maybe I could store this array in the flash memory (of which I have ~ 30 KB, IIRC) and just read it from there.

So, is it possible or practical to do what I suggest? Is there a simple way to implement this? I'm confused about the PROGMEM library and how to use it.

You can use two sketches. One loads the data in eeprom and the other is the main one that uses the data you programmed.

I posted too soon. I couldn't find the documentation I wanted so I posted here only to find the desired documentation 10 minutes later :roll_eyes:

I'm still interested to hear people's thoughts regarding the practicality of what I suggest, but for anyone else searching on this same topic, have a look around the following:

http://www.nongnu.org/avr-libc/user-manual/pgmspace.html

Thanks, Keith. I'm not sure that will help in my case because the EEPROM available is less than the SRAM. My full array (without doing anything "fancy" to reduce the size) is on the order of 11 KB. I can whittle this down to roughly 5 KB but that is still more than the 4 KB of EEPROM available on the Mega.

   if (millis() >= *Array[0][i] && millis() < *Array[0][i] + 10) {

How is Array defined? Using the * that way seems inappropriate. It makes me think you are doing something wrong.

Hi Paul,

That is extremely likely. At this point, though, I'm just asking general questions to sketch out some pseudo-code before I actually start coding. Originally, I had the following definition:

long *Array [i][j];

And everything seemed to work fine for small arrays - e.g. with i = 2 and j = 2 or etc. Of course, in the actual application, I want i = 4 and j = 720, which is too big for the SRAM, of course.

tms8c8:
I'm confused about the PROGMEM library and how to use it.

site search on the word progmem, top link:

Don't just look, work with the last example to make it do something else till you are clear.

Example

The following code fragments illustrate how to read and write unsigned chars (bytes) and ints (2 bytes) to PROGMEM.

#include <avr/pgmspace.h>


// save some unsigned ints
PROGMEM  prog_uint16_t charSet[]  = { 65000, 32796, 16843, 10, 11234};

// save some chars
prog_uchar signMessage[] PROGMEM  = {"I AM PREDATOR,  UNSEEN COMBATANT. CREATED BY THE UNITED STATES DEPART"};

unsigned int displayInt;
int k;    // counter variable
char myChar;  

// read back a 2-byte int
 displayInt = pgm_read_word_near(charSet + k)

// read back a char 
myChar =  pgm_read_byte_near(signMessage + k);

Arrays of strings

It is often convenient when working with large amounts of text, such as a project with an LCD display, to setup an array of strings. Because strings themselves are arrays, this is in actually an example of a two-dimensional array.

These tend to be large structures so putting them into program memory is often desirable. The code below illustrates the idea.

/*
 PROGMEM string demo
 How to store a table of strings in program memory (flash), 
 and retrieve them.

 Information summarized from:
 http://www.nongnu.org/avr-libc/user-manual/pgmspace.html

 Setting up a table (array) of strings in program memory is slightly complicated, but
 here is a good template to follow. 

 Setting up the strings is a two-step process. First define the strings.

*/

#include <avr/pgmspace.h>
prog_char string_0[] PROGMEM = "String 0";   // "String 0" etc are strings to store - change to suit.
prog_char string_1[] PROGMEM = "String 1";
prog_char string_2[] PROGMEM = "String 2";
prog_char string_3[] PROGMEM = "String 3";
prog_char string_4[] PROGMEM = "String 4";
prog_char string_5[] PROGMEM = "String 5";


// Then set up a table to refer to your strings.

PROGMEM const char *string_table[] = 	   // change "string_table" name to suit
{   
  string_0,
  string_1,
  string_2,
  string_3,
  string_4,
  string_5 };

char buffer[30];    // make sure this is large enough for the largest string it must hold

void setup()			  
{
  Serial.begin(9600);
}


void loop()			  
{
  /* Using the string table in program memory requires the use of special functions to retrieve the data.
     The strcpy_P function copies a string from program space to a string in RAM ("buffer"). 
     Make sure your receiving string in RAM  is large enough to hold whatever
     you are retrieving from program space. */


  for (int i = 0; i < 6; i++)
  {
    strcpy_P(buffer, (char*)pgm_read_word(&(string_table[i]))); // Necessary casts and dereferencing, just copy. 
    Serial.println( buffer );
    delay( 500 );
  }
}

Originally, I had the following definition:

Which is a 2D array of pointers to longs, NOT a 2D array of longs.

Why are you creating a 2D array of pointers?

And everything seemed to work fine for small arrays - e.g. with i = 2 and j = 2 or etc. Of course, in the actual application, I want i = 4 and j = 720, which is too big for the SRAM, of course.

Depends on the board of course.
A '1284P with 16K SRAM would not be a problem. Here's one that's Duemilanove/Uno- like, 32 IO, shield compatible. Offboard FTDI, or will take an onboard MIKROE483 FTDI module from Mouser ($11)
$5 for a bare board, build it up as you like
Cross Roads Electronics PL listed here

PaulS:
Why are you creating a 2D array of pointers?

Out of ignorance? Like I said I'm in the beginning stages of planning this code and initially I was needing to swap columns. It was suggested that I work with pointers instead so I could just change pointers and not have to actually move the data entries.

I found a number of program examples showing how to store character strings and vectors of integers in AVR program memory but no good example of how to put a const struct into program memory. After playing around I came up with a design pattern for this. It uses a number C++ tricks to hide the details. Typically the struct data type need to be defined with a program memory reference version. There is also a need for member data access functions and a copy function to move the whole data structure from program memory to data memory.

Please see Sketchbook/ProgramMemory at master · mikaelpatel/Sketchbook · GitHub for more details.

I hope this can be of some aid.

Cheers, Mikael

Now try doing that at run time.