Putting constant data into program memory (PROGMEM)

I've done a tutorial on putting arrays of strings into PROGMEM, using a somewhat less tedious approach than is described on the PROGMEM reference page:

http://www.gammon.com.au/forum/?id=12615

Comments are welcome.

Did you try version 1.5.7 ? I think the 1.0.5 (not tested 1.0.6) uses: prog_uchar message[] PROGMEM = { "Hello" }; The new 1.5.7 uses: const message[] PROGMEM = { "Hello" }; The 'const' should be there for version 1.5.7 as far as I know.

I think any tutorial on PROGMEM should include a reference to the f() macro.

Can you imagine the number of "PROGMEM" AND Serial.print("(char*) pgm_read_word(&(menuPointer[index]))") type statements I have in my code? smiley-evil

A simple Serial.print(f("system time is ")) achieves the job more effectively and is far more readable.

KenF: I think any tutorial on PROGMEM should include a reference to the f() macro.

It's the F macro, not the f macro, and on that page I have:

The first thing we can do is use the F() macro, which cunningly expands out to print directly from PROGMEM, thus saving RAM. So we change the line above to read:

Peter_n: Did you try version 1.5.7 ?

No, I am sticking to full releases, not Beta releases right now.

Thanks,
i think I have a real use for it.
I am doing some FFT , so far only on audio signals and naturally on small samples ( 64 max) so I could use "extra RAM’ to store these samples ( 16 bits int) and than process them as doubles. ( It looks as PROGMEM does not store / works on doubles)
Actually I will need to compare FFT data between different samples and that is where I really need more real RAM and PROGMEM RAM. .
I’ll let yo know how it goes in few days, I hope.
Cheers Vaclav

Every variable, structure, union, table can be in PROGMEM.

A 'double' is converted to a 'float' on a Arduino Uno.

No mention of http://arduiniana.org/libraries/flash/ ?

I had never heard of it, I must admit. I'll add a link.

However the method I present solves the problem of string arrays fairly neatly*. Also putting structures into PROGMEM.

  • Assuming the strings are similar sizes to each other.

Peter_n: Every variable, structure, union, table can be in PROGMEM.

A 'double' is converted to a 'float' on a Arduino Uno.

And your point i s?

Not according to this.

http://arduino.cc/en/Reference/PROGMEM

As mentioned above, it is important to use the datatypes outlined in pgmspace.h. Some cryptic bugs are generated by using ordinary datatypes for program memory calls. Below is a list of variable types to use. Floating point numbers in program memory do not appear to be supported. prog_char - a signed char (1 byte) -127 to 128 prog_uchar - an unsigned char (1 byte) 0 to 255 prog_int16_t - a signed int (2 bytes) -32,767 to 32,768 prog_uint16_t - an unsigned int (2 bytes) 0 to 65,535 prog_int32_t - a signed long (4 bytes) -2,147,483,648 to * 2,147,483,647. prog_uint32_t - an unsigned long (4 bytes) 0 to 4,294,967,295 Example

For a structure with values I would use the memcpy_P.

For Arduino IDE 1.0.6 a new typedef is created for float :wink:

// Arduino IDE 1.0.6 with float in PROGMEM
// Tested with Arduino Uno
typedef float PROGMEM prog_float;
prog_float table[] PROGMEM = { 1.0, 34.234, 324.234, 23.1, 52.0, 3.6, 5.6 } ;

void setup() {
  Serial.begin(9600);

  for (unsigned int i = 0; i < (sizeof(table) / sizeof(float)); i++) {
    Serial.println( pgm_read_float(&table[i]));
  }
}

void loop() {
}

A float in PROGMEM in Arduino 1.5.7 is even easier:

// Arduino IDE 1.5.7 with float in PROGMEM
// Tested with Arduino Uno
const float table[] PROGMEM = { 1.0, 34.234, 324.234, 23.1, 52.0, 3.6, 5.6 } ;

void setup() {
  Serial.begin(9600);

  for (int i = 0; i < (sizeof(table) / sizeof(float)); i++) {
    Serial.println( pgm_read_float(&table[i]));
  }
}

void loop() {
}

Vaclav: Floating point numbers in program memory do not appear to be supported.

Any data type is supported (including structures) if you use the technique described in my linked post..

After all PROGMEM is just memory and can hold anything.

The technique is to put what you want to store into a struct and memcpy_P that struct to an instance of what you want to use.

You can simplify slightly by making a library that copies from PROGMEM to any type:

PROGMEM_readAnything.h:

#include <Arduino.h>  // for type definitions

template <typename T> void PROGMEM_readAnything (T * sce, T& dest)
  {
  memcpy_P (&dest, sce, sizeof (T));
  }

Now the sample code above can be rewritten (in IDE 1.0.6) as:

#include <PROGMEM_readAnything.h>

float PROGMEM table[]  = { 1.0, 34.234, 324.234, 23.1, 52.0, 3.6, 5.6 } ;

void setup() 
{
  Serial.begin(115200);

  for (size_t i = 0; i < (sizeof(table) / sizeof(float)); i++) 
  {
    float thisOne;
    PROGMEM_readAnything (&table[i], thisOne);
    Serial.println(thisOne);
  }
}

void loop() { }

This same technique could be used to read structures from PROGMEM.

Or for things that can be copied (like floats) you could simplify the way you use it:

PROGMEM_readAnything.h:

#include <Arduino.h>  // for type definitions

template <typename T> void PROGMEM_readAnything (T * sce, T& dest)
  {
  memcpy_P (&dest, sce, sizeof (T));
  }

template <typename T> T PROGMEM_getAnything (T * sce)
  {
  static T temp;
  memcpy_P (&temp, sce, sizeof (T));
  return temp;
  }

Example code:

#include <PROGMEM_readAnything.h>

float PROGMEM table[]  = { 1.0, 34.234, 324.234, 23.1, 52.0, 3.6, 5.6 } ;

void setup() 
{
  Serial.begin(115200);

  for (size_t i = 0; i < (sizeof(table) / sizeof(float)); i++) 
  {
    Serial.println(PROGMEM_getAnything (&table[i]));
  }
}

void loop() { }

Now a single line retrieves the value from PROGMEM.

That is very cool. Thanks.

Vaclav: it is important to use the datatypes outlined in pgmspace.h

The typedefs in pgmspace.h never really worked and their use has been deprecated for a while now. See http://www.nongnu.org/avr-libc/user-manual/group__avr__pgmspace.html#gaa475b6b81fd8b34de45695da1da523b6

It's only the Arduino "documentation" that perpetuates the rumour that the typdefs are useful or that there is any issue with floats.

And this leads to my comment:

In my opinion, Nick's writeup should:

(1) refer to the underlying AVR LIBC documentation. Given that it's the basis on which all the layered Arduino goodness is built, it should be given in reference (2) Many AVR LIBC functions for string manipulation are available in _P flavours that work with PSTR(). I have no idea if those would work with F() -- I never use F() myself -- which kind works with what should be addressed somehow (3) there should be a minimal treatment of strcpy_P(), strcmp_P(), strstr_P(), sprintf_P() with refs to the underlying AVR LIBC docs.

well done Nick!

IIRC the advantage of the "Putting the strings themselves into PROGMEM" method is that you should be able to use strings of unequal length.

question that came up: Would there be a graceful way to put a linked list in PROGMEM... or a binary tree

Would there be a graceful way to put a linked list in PROGMEM... or a binary tree

Linked lists and binary trees are generally used because the data is not constant. What is the purpose of a constant linked list or binary tree?

PaulS:
What is the purpose of a constant linked list or binary tree?

I think that would be the natural way to express a menu system. It would be nice if you could easily declare a PROGMEM based menu structure with sub-menus and such and then have a menu library just take the root of the linked structure.

I do not see any reason this could not be done using PROGMEM as it currently exists. Pointers to other constants are handled gracefully. The main issue is that there is no attribute(progmem) notation that works right on string constant initializers, so you gave to do the two-step:

const char s1[] PROGMEM = "the first";
const char s2[] PROGMEM = "another one";
const char *slist[] PROGMEM = { s1, s2, ... };

I would like to second robtillaart’s suggestion that Nick update his example of this to actually show different-lengths strings being used.

I personally use constant length declarations like this, even when there is a good bit of variation in string lengths and I don’t mind wasting the last element either as I like to be able to use the sentinel.

const char menu[][10] PROGMEM ={ "opt one", "opt two", "" };

Compared to ram, there is mountains of flash – I just keep the max short and go for it.

gardner: In my opinion, Nick's writeup should:

(1) refer to the underlying AVR LIBC documentation. Given that it's the basis on which all the layered Arduino goodness is built, it should be given in reference

I've added more links, and also a copy of the template function for reference.

I'm trying to keep it fairly simple, so not all of your suggestions have been implemented. :)