Problem with retrieving the same value multiple times from program memory

I just started to work with PROGMEM and got to store strings, retrieve them and display everything on a LCD quite nice. However, soon as I try to display the same string twice the program stops at the point where i try to display the same value the second time.
It would be nice is you could have a look and maybe point me at a work around or to what I am missing. Since the whole sketch is quite long, I just post the important parts where the problem occurs.

#include <ShiftLCD.h>
#include <avr/pgmspace.h>


prog_char menu_0[] PROGMEM = "Sensitivity";
prog_char menu_1[] PROGMEM = "Room photo cut";
prog_char menu_2[] PROGMEM = "Room photo limit";
prog_char menu_3[] PROGMEM = "OS photo cut";
prog_char menu_4[] PROGMEM = "OS photo limit";
prog_char menu_5[] PROGMEM = "Room 1";
   prog_char submenu_5_1[] PROGMEM = "R1 PIR delay";
   prog_char submenu_5_2[] PROGMEM = "R1 HT1 Active";
   prog_char submenu_5_3[] PROGMEM = "R1 Timer 1";
   prog_char submenu_5_4[] PROGMEM = "R1 HT2 Active";
   prog_char submenu_5_5[] PROGMEM = "R1 Timer 2";
   prog_char submenu_5_6[] PROGMEM = "R1 HT3 Active";
   prog_char submenu_5_7[] PROGMEM = "R1 Timer 3";
prog_char menu_6[] PROGMEM = "Room 2";
prog_char menu_7[] PROGMEM = "Room 3";
prog_char menu_8[] PROGMEM = "Room 4";
prog_char menu_9[] PROGMEM = "Room 5";
prog_char menu_10[] PROGMEM = "Room 6";
prog_char menu_11[] PROGMEM = "Room 7";
prog_char menu_12[] PROGMEM = "Room 8";
prog_char menu_13[] PROGMEM = "Room 9";
prog_char menu_14[] PROGMEM = "Room 10";
prog_char menu_15[] PROGMEM = "AC 1";
prog_char menu_16[] PROGMEM = "AC 2";
prog_char menu_17[] PROGMEM = "AC 3";
prog_char menu_18[] PROGMEM = "AC 4";

prog_char msg_0[] PROGMEM = "Not Used";
prog_char msg_1[] PROGMEM = "Saving....";
prog_char msg_2[] PROGMEM = "Setup mode";
prog_char msg_3[] PROGMEM = ">";

PROGMEM const char *menu_table[] = {
  menu_0,
  menu_1,
  menu_2,
  menu_3,
  menu_4,
  menu_5,
  menu_6,
  menu_7,
  menu_8,
  menu_9,
  menu_10,
  menu_11,
  menu_12,
  menu_13,
  menu_14,
  menu_15,
  menu_16,
  menu_17,
  menu_18
};

PROGMEM const char *submenu_5_table[] = {
  submenu_5_1,
  submenu_5_2,
  submenu_5_3,
  submenu_5_4,
  submenu_5_5,
  submenu_5_6,
  submenu_5_7
};

PROGMEM const char *msg_table[] = {
  msg_0,
  msg_1,
  msg_2,
  msg_3
};

That's the initializing and storing part of it. Next where I encounter the problem:

if(menuOption == 3){
        lcd.clear();
        lcd.print(strcpy_P(buffer, (char*)pgm_read_word(&(menu_table[2]))));
        lcd.setCursor(0, 1);
        lcd.print("NOT USED");
      }
      if(menuOption == 4){
        lcd.clear();
        lcd.print(strcpy_P(buffer, (char*)pgm_read_word(&(menu_table[3]))));

      }
      if(menuOption == 5){
        lcd.clear();
        lcd.print(strcpy_P(buffer, (char*)pgm_read_word(&(menu_table[4]))));
        lcd.setCursor(0, 1);
        lcd.print("NOT USED");

As you can see under menuOption 3 and menuOption 5 I have the lcd.print statement "lcd.print("NOT USED")".
If I replace 1 of this statements with "lcd.print(strcpy_P(buffer, (char*)pgm_read_word(&(msg_table[0]))));" it will work fine but soon as I try to read the message "NOT USED" the second time from the program memory everything freezes.
Any help would be kindly appreciated.

Thanks for having a look at.

Have you tried printing to the serial monitor, instead of the LCD, to see if that works?

I tried it, using Serial rather than lcd, and it just printed the expected results over and over. Test code appears below.

Maybe something funny is going on in lcd, but I don't see it. Or, maybe something's wrong with the code you're not showing. My personal favorite crystal-ball guess is that buffer is declared somewhere as a character pointer, or it's declared as a character array that's too small to hold the message, and strcpy_P() overruns the buffer and writes into other memory locations. You didn't show that part of the code, though, so it's just a guess.

I'll suggest that you write a minimal sketch that demonstrates the problem you're trying to solve, using Serial rather than lcd. I think you'll get more participation if you post something that compiles, and doesn't make us assemble hardware to try it for ourselves. I think you'll also get more help if you make the effort to isolate the problem to a short, compilable sketch, so that we don't have to pore over miles of code, or write our own code to get your snippets to run.

See the sticky post, "How to use this forum - please read," at the top of this forum listing. In particular, see item #11, where it says, among other things,

Post a complete sketch (program code)! If you don't you waste time while people ask you to do that. However, with coding problems, if possible post a "minimal" sketch that demonstrates the problem - not hundreds of lines of code. If the problem goes away in the minimal sketch, it wasn't where you thought it was.

Test code follows. I deleted a whole bunch of declarations that aren't relevant to the posted code.

prog_char menu_0[] PROGMEM = "Sensitivity";
prog_char menu_1[] PROGMEM = "Room photo cut";
prog_char menu_2[] PROGMEM = "Room photo limit";
prog_char menu_3[] PROGMEM = "OS photo cut";
prog_char menu_4[] PROGMEM = "OS photo limit";

prog_char msg_0[] PROGMEM = "Not Used";
prog_char msg_1[] PROGMEM = "Saving....";

PROGMEM const char *menu_table[] = {
  menu_0, menu_1, menu_2, menu_3, menu_4
};

PROGMEM const char *msg_table[] = {
  msg_0, msg_1
};

char buffer[40];  // Plenty big

void setup() {
  Serial.begin(115200);
}

void loop() {
  
  for (int menuOption=0;menuOption<=6;menuOption++) {

    Serial.print(menuOption);
    Serial.print(": ");

    if(menuOption == 3){
      Serial.print(strcpy_P(buffer, (char*)pgm_read_word(&(menu_table[2]))));
      Serial.print("  ");
      Serial.println(strcpy_P(buffer, (char*)pgm_read_word(&(msg_table[0]))));
    }

    if(menuOption == 4){
      Serial.println(strcpy_P(buffer, (char*)pgm_read_word(&(menu_table[3]))));
    }

    if(menuOption == 5){
      Serial.print(strcpy_P(buffer, (char*)pgm_read_word(&(menu_table[4]))));
      Serial.print("  ");
      Serial.println(strcpy_P(buffer, (char*)pgm_read_word(&(msg_table[0]))));
    }

    if ((menuOption < 3) || (menuOption > 5)) {
      Serial.println("No message.");
    }

    delay(100);
  }
  Serial.println();
}

Sample output:

2: No message.
3: Room photo limit  Not Used
4: OS photo cut
5: OS photo limit  Not Used
6: No message.

0: No message.
1: No message.
2: No message.
3: Room photo limit  Not Used
4: OS photo cut
5: OS photo limit  Not Used
6: No message.

I think I read somewhere recently that Serial.print() has some special code to read from Progmem which is not included in LCD.print(). I didn't take much notice as I was not planning to use Progmem.

Could you copy the string to a char array and print that to the LCD. A single char array could be used for all the strings so it wouldn't take up too much RAM.

...R

Robin2:
... copy the string to a char array and print that to the LCD ...

I'm pretty sure that's what happens here, quoting from the OP's sketch:lcd.print(strcpy_P(buffer, (char*)pgm_read_word(&(menu_table[2]))));and in the code the OP complains about:lcd.print(strcpy_P(buffer, (char*)pgm_read_word(&(msg_table[0]))));

strcpy_P() seems to copy a C string from an address in program memory to an address in RAM. The first argument is a pointer to a RAM buffer, and the second is a pointer to a character array in program memory. Best I can tell, strcpy_P() returns the address of the destination, just like strcpy(). So, I think the code moves data from program memory to RAM, and then lcd.print()'s from RAM. That's what seems to happen when I Serial.print() it.

As for whether Serial.print() distinguishes between character pointers to RAM and to program memory, it appears to me that it doesn't. Here's test code:

prog_char menu_0[] PROGMEM = "Sensitivity";

char buffer[40];  // Plenty big

void setup() {
  Serial.begin(115200);
}

void loop() {
  Serial.println("From program memory:");
  Serial.println(menu_0);

  Serial.println();

  Serial.println("From RAM:");
  Serial.println(strcpy_P(buffer, menu_0));

  Serial.println();
  Serial.println();
  delay(100);
}

Using strcopy_P(), the output is, "Sensitivity," as expected. Using a pointer to program memory, the output is blank. Because menu_0 likely points to unused RAM, I think it could print just about anything.

As for whether Serial.print() distinguishes between character pointers to RAM and to program memory, it appears to me that it doesn't.

Of course it doesn't (it cannot) - that's what the F macro is there for.

Thank you guys for the help and sorry for the long code but the whole thing is about 1700 lines of code and I thought I just pig the important part to show what is not working. I guess it was my buffer declaration which I made char buffer[20]. Since the longest string is not more than 15 characters long I thought it's enough. Thanks tdm3, I set the buffer size to 40 and it works. Thanks again.

Checking all the messages with this statement:
Serial.print(strlen(strcpy_P(buffer, (char*)pgm_read_word(&(msg_table[i])))));shows a longest length of 16, for "Room photo limit", menu_2, menu_table[2]. The minimum size of the array that would contain it is 17, allowing for the string-ending 0x00 at the end. So,

divedj:
... the longest string is not more than 15 characters long ...

is off by a couple, but I'd expect that a buffer sized at 20 would work. I sized it at 17, and it worked; sized it at 16, and I got quirky output, but it ran; sized at 10, again odd output, but it ran. But, I'm running a simple test program - if I overwrite some RAM, it's not so likely that it'll do something to mess up the program flow.

With the longest output string needing only 17 characters, a buffer size of 20 ought to work. If increasing the buffer size worked, I suspect that something else is afoot, and that it's still lurking in your code. Maybe you're using the buffer someplace else, where it's too small.

I'll ask that you do this: Start with your code, and delete stuff and add stuff until it looks more or less like the code I posted. What I want most is that the string definitions you're using right now survive into the reduced program unchanged. Then check and see if it works.

Thanks tmd3, I took your advice and it's running with a buffer size of 20 with no problems. Another problem seems to be that progmem or the lcd doesn't like to output ">" as a single character as defined in "prog_char msg_3[] PROGMEM = ">";

prog_char msg_0[] PROGMEM = "Not Used";
prog_char msg_1[] PROGMEM = "Saving....";
prog_char msg_2[] PROGMEM = "Setup mode";
prog_char msg_3[] PROGMEM = ">";

I took it out from the table added another one with special char's :

const byte char_table[] PROGMEM = {
  B01111110,                          //Arrow right
  B01111111,                           //Arrow left
  B00111110                         //right
};

and retrieve it with:

lcd.write(pgm_read_byte(&char_table[0]));

which works fine.

I am using another buffer somewhere else but it's declared as "char read_buffer[64];" and as far as I know they should not interfere with each other.
Please let me know when I am wrong.

Thanks again for your help.