Reading PROGMEM string array char by char

Projects Programming
1

I'm trying to make a program which makes use of the least possible RAM when reading strings from flash memory. Particularly, I'm trying to read Strings char by char in order to avoid creating a copy of the whole string in the RAM as strcpy_P does.
So say I have the following arrays:

const prog_char line1[] PROGMEM ="line number one";
const prog_char line2[] PROGMEM ="line number two";
const prog_char *myArray[] PROGMEM = { line1, line2 };

And I want to create a function that, given a line number (0 or 1 in this case), it outputs the content of that line.
My first attempt was to just to access it using two indexes:

do {
      c = (char) pgm_read_byte(&arrP[i][j++]);
      Serial.print(c);
    } while (c!='\0');

But this doesn't work unless:
a) the value of i is know at compile time
b) arrP is stored in RAM instead of Flash memory (removing the PROGMEM modifier)

I'm trying to figure out how this all works, so any hints are welcome.
Here are my guesses so far: I guess is that I have to read the content of arrP first in order to obtain a valid pointer to the corresponding line, and then read that line char by char. When arrP is in the RAM then there's no problem, but when it is not, I would have to use some kind of pgm_read function to retrieve it. but how? There's no pgm_read_pointer (let alone pgm_read_prog_char_pointer)... so what can I do? Did I even get it right?

2

Have you read this page: PROGMEM - Arduino Reference

But this doesn't work unless:
a) the value of i is know at compile time
b) arrP is stored in RAM instead of Flash memory (removing the PROGMEM modifier)

The value of i shouldn't need to be known at compile time, but the variable i definitely needs to be declared (which you don't show), and assigned to some value (which you also don't show).

Also, pgm_read_byte ONLY works on progmem, so b) doesn't make any sense at all. You also do not show where/how arrP is declared/initialized either. From your previous code snippet, it appears you should be using myArray, so where does arrP come from?

3

fyi, the following code works just fine:

#include <avr/pgmspace.h>

prog_char line0[] PROGMEM = "First Test";
prog_char line1[] PROGMEM = "Second Test";

PROGMEM const char * myArray[] =
{
    line0,
    line1
};

void setup()
{
    
    Serial.begin(115200);
    
    byte i = 0, j = 0;
    char c = 0;
    do {
      c = (char) pgm_read_byte(&myArray[i][j++]);
      Serial.print(c);
    } while (c!='\0');
    Serial.println();
    
    i = 1, j = 0;
    do {
      c = (char) pgm_read_byte(&myArray[i][j++]);
      Serial.print(c);
    } while (c!='\0');
    Serial.println();
}

void loop(){}
4

I include a full example to make things clrearer.

fyi, the following code works just fine:

But in that example, the value of i can be established at compilation time. For some reason, when it's not (like when i is a function's parameter) it doesn't work (see code below)

Also, pgm_read_byte ONLY works on progmem, so b) doesn't make any sense at all.

So why does the following code work when using arr, but not when using arrP ???

#include <avr/pgmspace.h>
/**
 * Test storing & retrieving 1D & 2D arrays from progmem
 */

const prog_char line1[] PROGMEM ="line number one";
const prog_char line2[] PROGMEM ="line number two";

const prog_char *arr[] = { line1, line2 };
const prog_char *arrP[] PROGMEM = { line1, line2 };


void setup(){
    Serial.begin(9600);
    test(1);
}


void test(int index) {
    int i=0; char c;

    do {
      c = (char) pgm_read_byte(&arr[index][i++]); //this works (arr is in RAM)
      //c = (char) pgm_read_byte(&arrP[index][i++]); //this won't work (arrP is in Flash)
      Serial.print(c);

    } while (c!='\0');
}

void loop(){
}
5

I see the problem now. Curiously enough, my sample code shouldn't actually work at all. My only guess is that, with it all being in the same function, the compiler applied some optimizations that eliminated the multiple dereferencing of the flash memory locations.

Anyways, the problem is essentially that you've got yourself a pointer to a pointer, but when working with progmem, you have to dereference each pointer yourself, since in GCC pointers don't actually work with progmem, hence the need to utilize all the pgm_read macros.

Here's the proper code:

#include <avr/pgmspace.h>

prog_char line0[] PROGMEM = "First Test";
prog_char line1[] PROGMEM = "Second Test";

const char * myArray[] PROGMEM =
{
    line0,
    line1
};

void setup()
{
    
    Serial.begin(115200);
    
    test(0);
    test(1);
}

void loop(){}

void test(int index)
{
    unsigned int flash_address = pgm_read_word(&myArray[index]);
    char c = 0;
    do {
      c = (char) pgm_read_byte(flash_address++);
      Serial.print(c);
    } while (c!='\0');
    Serial.println();    
}

As you can see, it's a two step process. The first is to call pgm_read_word to read out the memory address of the string index passed in. Again, this isn't actually a pointer, it's just the 16bit address of the flash location that the specified indexed string is saved to, so we store that in an unsigned int.

We then use that address to read out our string with pgm_read_byte, until we reach the end of the string.

Again, multilevel dereferencing (pointer to a pointer) won't work, because the compiler can't read the values out of flash without the use of the pgm macros, so you have to handle each level of dereferencing yourself.

6

Thank you jraskell, very nicely explained, I think I've finally got it.

7

jraskell:
Here's the proper code:

Three years later this gets appreciated again, thank you! Creating a test() certainly helps.

8

Three years later, things have changed a lot. You shouldn't use his old examples, see here for an up to date example:

9

Many things may have changed in three years, but the example at PROGMEM - Arduino Reference is basically identical to the example at the same page from December 27, 2011 (Arduino - PROGMEM).

Looks like there was just some small formatting changes.

The main point here, the reference example doesn't answer the OP's (and my) question of how to create a function that outputs multiple strings from an array one char at a time.

Anyway thanks for the code and explanation jraskell.

10

That has a reasonable number of examples.

11

Hello world

On Nick's page

we can see

PROGMEM_readAnything function which reads pgm area byte by byte/char by char

why Nick used this techniques and not the copy whole array at once ?

for example for any 1D array

template <typename T, size_t N> void ArrayCopy (T& dest, const T (&src) [N]) noexcept
{
  // array
  memcpy_P (dest, src, sizeof(src));
}

in the same mode we can overload and for string, etc