PROGMEM const not working

Hi,

I want to put a table into flash and used “PROGMEM const” as in the attached code.

The code uses less RAM (means the table is indeed in flash) however it prints junk.

When the keyword “PROGMEM const” are removed, the code uses more RAM, as expected, and runs fine printing the table.

Whats wrong with the code?

Thanks in advance,
WonderfulIOT

#define ONE "one"
#define TWO "two"
#define THREE "three"
#define FOUR "four"
#define FIVE "five"
#define SIX "six"
#define SEVEN "seven"
#define EIGTH "eight"
#define NINE "nine"

typedef struct 
{
  char *ptr_command;
  unsigned int size;
}StructATCommand;

PROGMEM const StructATCommand at_command_table[]=
{
  { ONE, sizeof(ONE)},    //0
  {TWO, sizeof(TWO)},  //1
  {THREE, sizeof(THREE)}, //2
  {FOUR,sizeof(FOUR)}, //3
  {FIVE,sizeof(FIVE)}, //4
  {SIX,sizeof(SIX)}, //5
  {SEVEN,sizeof(SEVEN)}, //6
  {EIGTH,sizeof(EIGTH)}, //7
  {NINE,sizeof(NINE)}, //8
};

void print_char_string(char* ptr)
{
  while(*ptr != 0)
  {
    Serial.print(*ptr);
    ptr++;
  }
  Serial.println("");
}

void setup() {
  Serial.begin(19200);
  // put your setup code here, to run once:
  
  for(int i=0;i<sizeof(at_command_table)/sizeof( StructATCommand);++i)
  {
    print_char_string(at_command_table[i].ptr_command);
  }
}

void loop() {
  // put your main code here, to run repeatedly:

}

Upon further reading I realized that there is a special function to read the string stored in flash. The code has been modified accordingly, however it still does not work. There is no output now
Here is the full code (only 1 line has changed from before)

#define ONE "one"
#define TWO "two"
#define THREE "three"
#define FOUR "four"
#define FIVE "five"
#define SIX "six"
#define SEVEN "seven"
#define EIGTH "eight"
#define NINE "nine"

typedef struct 
{
  char *ptr_command;
  unsigned int size;
}StructATCommand;

PROGMEM const StructATCommand at_command_table[]=
{
  { ONE, sizeof(ONE)},    //0
  {TWO, sizeof(TWO)},  //1
  {THREE, sizeof(THREE)}, //2
  {FOUR,sizeof(FOUR)}, //3
  {FIVE,sizeof(FIVE)}, //4
  {SIX,sizeof(SIX)}, //5
  {SEVEN,sizeof(SEVEN)}, //6
  {EIGTH,sizeof(EIGTH)}, //7
  {NINE,sizeof(NINE)}, //8
};

void print_char_string(char* ptr)
{
  while(*ptr != 0)
  {
    Serial.print(*ptr);
    ptr++;
  }
  Serial.println("");
}

void setup() {
  Serial.begin(19200);
  char buff[32];
  // put your setup code here, to run once:
  
  for(int i=0;i<sizeof(at_command_table)/sizeof( StructATCommand);++i)
  {
    strcpy_P(buff, (char*)pgm_read_word(at_command_table[i].ptr_command));
    print_char_string(buff);
  }
}

void loop() {
  // put your main code here, to run repeatedly:

}

wonderfuliot:
Whats wrong with the code?

It doesn’t store the C strings in FLASH, and it doesn’t retrieve ptr_command from FLASH using the special pgm_read_XXX function. See comments:

// Make sure these C strings are stored in PROGMEM, too.
static const char ONE  [] PROGMEM = "one";
static const char TWO  [] PROGMEM = "two";
static const char THREE[] PROGMEM = "three";
static const char FOUR [] PROGMEM = "four";
static const char FIVE [] PROGMEM = "five";
static const char SIX  [] PROGMEM = "six";
static const char SEVEN[] PROGMEM = "seven";
static const char EIGTH[] PROGMEM = "eight";
static const char NINE [] PROGMEM = "nine";

typedef struct 
{
  const char *ptr_command;
//  unsigned int size; // not needed
} StructATCommand;

PROGMEM const StructATCommand at_command_table[]=
  {
    { ONE   },
    { TWO   },
    { THREE },
    { FOUR  },
    { FIVE  },
    { SIX   },
    { SEVEN },
    { EIGTH },
    { NINE  },
  };

static const size_t AT_COMMAND_COUNT =
                       sizeof(at_command_table)/sizeof(at_command_table[0]);
                       
void setup() {
  Serial.begin(19200);
  
  for(int i=0; i < AT_COMMAND_COUNT; ++i)
  {
    // Because the array is in PROGMEM, you have to use pgm_read_XXX to
    //   get any values from an array elemment.  You pass the *address*
    //   of the element piece that you want, and pgm_read_XXX fetches it
    //   for you.
    void *cmd = pgm_read_ptr( &at_command_table[i].ptr_command );
    // At this point, `cmd` contains the PROGMEM address of a
    //   C string constant.
    
    // You /could/ use pgm_read_byte to step through it, reading each byte.
    //   It's easier here to just cast the `cmd` pointer to a special type 
    //   that's used for FLASH strings.  This type is what the F macro returns.
    Serial.println( (const __FlashStringHelper *) cmd );
  }
}

void loop() {}

Thanks -dev, it really helps.
However at_command_table*.ptr_command[/b] already contains the address of the string, why should I pass the address of this pointer (makes it a pointer to pointer :confused: ) to the below function*
void cmd = pgm_read_ptr( &at_command_table.ptr_command );[/b]*
Thanks and Regards,
WonderfulIOT

Because you store two different things in flash, the array with pointers to all objects and the objects. So reading the array in flash it will give you another pointer in flash to where the object is :slight_smile:

Is below code perfectly legal?

const char text[] PROGMEM =  "This is a long____________________________________________________text";
char temp_buffer[200];

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  strcpy_P(temp_buffer, text);
  Serial.println(temp_buffer);
}

void loop() {
  // put your main code here, to run repeatedly:
}

Legal, yes. Useful, no. Because just creating a 200 byte array is wasteful.

Hello.
I have the same problem: getting scrambled data from a PROGMEM array ,both fixed numbers and strings.

I defined 4 const arrays in the code, one of them is a set of strings, different length.
I retrieved the values like using array without PROGMEM.

How can I retrieve the values in a simplest way?
For the following commands I expect to see on the display "text_265".

Please help me to understand how o pass the PROGMEM address.
I have the char array with 25 of strings (9-15 characters) and 3 arrays each with 25 values.
My whole code for Nano is 26600bytes and ~1615bytes SRAM now. I run on short memory.
I hope the SRAM memory optimisation (~120 bytes) does worth the supplementary code to be inserted.

//define arrays

const char * const listaParametriMeniu[] PROGMEM =
{
 "txt_00_",
 "text 1111",
 "text_2", 
 "string_3_",
 "string__4_-",
 "text_!_5",
};

const uint8_t default_parameters[6] PROGMEM ={81,1,65,10,70,6};

//print data of the 3rd pointer from both arrays

lcd.print(listaParametriMeniu[2]) ; lcd.print(default_parameters[2]);
const char str_0[] PROGMEM = "txt_00_";
const char str_1[] PROGMEM = "text 1111";
const char str_2[] PROGMEM = "text_2";
const char str_3[] PROGMEM = "string_3_";
const char str_4[] PROGMEM = "string__4_-";
const char str_5[] PROGMEM = "text_!_5";

const char * const listaParametriMeniu[] PROGMEM = {
  str_0, str_1, str_2, str_3, str_4, str_5,
};

const uint8_t default_parameters[6] PROGMEM = {81, 1, 65, 10, 70, 6};

void setup() {
  Serial.begin(250000);
  Serial.print((__FlashStringHelper*)pgm_read_word(&listaParametriMeniu[2]));
  Serial.println(pgm_read_byte(&default_parameters[2]));
}
void loop() {}
text_265

Strangely enough the wonders of optimization make even the wrong code work.
That's quite irritating (at least to me).

const char * const listaParametriMeniu[] PROGMEM =
{
  "txt_00_",
  "text 1111",
  "text_2",
  "string_3_",
  "string__4_-",
  "text_!_5",
};

const uint8_t default_parameters[6] PROGMEM = {81, 1, 65, 10, 70, 6};

void setup() {
  Serial.begin(250000);
  Serial.print(listaParametriMeniu[2]);
  Serial.println(default_parameters[2]);
}
void loop() {}
text_265
lcd.print(listaParametriMeniu[2]) ; lcd.print(default_parameters[2]);

You need to be careful when testing code this way, since you are specifying a particular element of the array (2 in this case), the compiler will often assume it is more efficient to directly replace the array reference with the actual data from the array, and never actually create the array. Then the test code will appear to properly retrieve the data from PROGMEM, when in fact it will not, because the data is not actually being stored in PROGMEM.

Better to use a variable that the compiler knows will be changing, so that the actual array has to be used:

const char str_0[] PROGMEM = "txt_00_";
const char str_1[] PROGMEM = "text 1111";
const char str_2[] PROGMEM = "text_2";
const char str_3[] PROGMEM = "string_3_";
const char str_4[] PROGMEM = "string__4_-";
const char str_5[] PROGMEM = "text_!_5";

const char * const listaParametriMeniu[] PROGMEM = {
  str_0, str_1, str_2, str_3, str_4, str_5
};

const uint8_t default_parameters[6] PROGMEM = {81, 1, 65, 10, 70, 6};


void setup() {
  char textBuffer[12];
  Serial.begin(9600);
  for (byte i = 0; i < 6; i++) {
    Serial.print((__FlashStringHelper *) pgm_read_word(&listaParametriMeniu[i]));
    Serial.println(pgm_read_byte(&default_parameters[i]));

    //alternative way to print out the text, by first copying it into a buffer in RAM
    strlcpy_P(textBuffer, pgm_read_word(&listaParametriMeniu[i]), sizeof(textBuffer));
    Serial.println(textBuffer);
    
    Serial.println();
  }
}

void loop() {

}

You may want to read over some reference material for PROGMEM:

Putting constant data into program memory

here is an example I wrote for another discussion thread with a structure gathering PROGMEM stored data in a PROGMEM based structure

#include <avr/pgmspace.h>

enum situation_t : uint8_t {single, couple, RIP};

const char s0[] PROGMEM = "single";
const char s1[] PROGMEM = "couple";
const char s2[] PROGMEM = "rest in peace";

const char* const descriptionSituation[] PROGMEM = {s0, s1, s2};

struct person_t {
  const char * firstName;
  const uint8_t d;
  const uint8_t m;
  const uint16_t y;
  situation_t situation;
};


// for char array, you define them first somewhere in flash memory
const char auguste[] PROGMEM = "Auguste";
const char camille[] PROGMEM = "Camille";
const char victor[] PROGMEM = "Victor";
const char juliette[] PROGMEM = "Juliette";
const char someone[] PROGMEM = "Mr Unknown";

const person_t population[] PROGMEM =
{
  {auguste, 12, 11, 1840, RIP}, // the structure holds a pointer in memory for the first name
  {camille, 8, 12, 1864, RIP},
  {victor, 26, 2, 1802, RIP},
  {juliette, 10, 4, 1806, RIP},
  {someone, 1, 1, 2000, single}
};


void setup() {
  Serial.begin(115200);
  for (const auto& member : population)
  {
    Serial.println((const __FlashStringHelper *) pgm_read_word(&(member.firstName)));
    Serial.print  ((uint8_t) pgm_read_byte_near(&(member.d)) ); Serial.print("/");
    Serial.print  ((uint8_t) pgm_read_byte_near(&(member.m)) ); Serial.print("/");
    Serial.println((uint16_t) pgm_read_word_near(&(member.y)) );
    Serial.println((const __FlashStringHelper *) pgm_read_word(&(descriptionSituation[(uint8_t) pgm_read_byte_near(&(member.situation))])));
    Serial.println(F("------------------------------"));
  }
}

void loop() {}

may be that will help?

Thanks everyone for the effort to respond.
I applied the solution presented by @Whandall for 3 data (numbers) arrays and one array with strings.
My code example was exactly like you wrote but I extracted only the parts which were in discussion considering "voids" by default.

The PROGMEM optimization works well. It does what I want.
I gained back 456 bytes from SRAM and just few bytes added to flash. That' a lot. It is almost 25% of the total SRAM.

Thanks @david_2018 for the recommendation to read more about PROGMEM.

I am not a hard coder. I am just a hobbyist and search for specific solution at the moment I want to implement something I do not know how.

If you have any Serial.print() or lcd.print() commands with literal text, you can save some SRAM by using the F() macro, which places the literal in PROGMEM. As an example:

//literal stored in SRAM
Serial.println("Some random text");

//literal stored in PROGMEM
Serial.println(F("Some random text"));

david_2018:
If you have any Serial.print() or lcd.print() commands with literal text, you can save some SRAM by using the F() macro, which places the literal in PROGMEM

Not a big deal, but just for the sake of semantics — PROGMEM is not really a place where you store things (or really just by extension - short for PROGram MEMory). The memory that will be used to hold your data is called Flash and in practice PROGMEM is just a keyword you use as a variable modifier in the programming language to instruct the compiler to store that variable in flash memory.

david_2018:
If you have any Serial.print() or lcd.print() commands with literal text, you can save some SRAM by using the F() macro, which places the literal in PROGMEM.

I already know this trick since 2-3 years ago. I also reccommend this to anyone.

offtopic

My project is an automation for heating system (wood boiler, heat buffer, domestic hot water heater, radiators in the rooms) I started in 2015 and added a lot of features growing in dimension. As a consequence I had to optimize the code to occupy less space.
I used the ArduinoIDE 1.5.6 with which I started until this january (28kbytes compiled program)
I switched to the version 1.8.7 and the compiled file dropped considerably to 23kb.

david_2018:
If you have any Serial.print() or lcd.print() commands with literal text, you can save some SRAM by using the F() macro, which places the literal in PROGMEM.

I already know this trick since 2-3 years ago. I also reccommend this to anyone.

offtopic

My project is an automation for heating system (wood boiler, heat buffer, domestic hot water heater, radiators in the rooms) I started in 2015 and added a lot of features growing in dimension. As a consequence I had to optimize the code to occupy less space.
I used the ArduinoIDE 1.5.6 with which I started until this january (28kbytes compiled program)
I switched to the version 1.8.7 and the compiled file dropped considerably to 23kb.