PROGMEM and strange Serial.print output [solved]

Projects Programming
1

Hello,

I am trying to store an array of strings to have the Arduino dynamically come up with a name. I tried doing it with SRAM, but was having trouble with my program not running with no error generated. It turns out that my string arrays containing the names were too big for the memory. So, I switched to PROGMEM and everything loads fine, but now doesn't retrieve the names from the array correctly.

Here is my code:

#include <avr/pgmspace.h>

char* name_male[]   PROGMEM = {"Tony", "Warren", "Wendell", "Will", "William"};
char* name_last[]   PROGMEM = {"Vinsant", "Ward", "Wheeler", "Wolfe", "Young"}; 

#define array_male_length   ((sizeof(name_male)/sizeof(char *)))
#define array_last_length   ((sizeof(name_last)/sizeof(char *))) 
   
char* first_name;
char* last_name;

int randomNumber;
int index;

void setup() 
{
  Serial.begin(9600);
}

void loop() 
{
  while(index < array_last_length)
  {
    generateName();
  }  
}


void generateName()
{
  randomNumber = random(5);
  first_name = name_male[randomNumber];
  last_name = name_last[randomNumber];
   
  Serial.print("\nIndex:");
  Serial.print(index);
  
  Serial.print("\t Random:");
  Serial.println(randomNumber);
  
  Serial.print("Indexed name is "); 
  Serial.print(name_male[index]);
  Serial.print(" ");
  Serial.println(name_last[index]);

  Serial.print("Generated name is ");
  Serial.print(first_name);
  Serial.print(" ");
  Serial.println(last_name);

  Serial.print("Direct name is ");
  Serial.print(name_male[0]);
  Serial.print(" ");
  Serial.println(name_last[0]);
  
  index++;
      
  delay(1000);

}

and here is the output:

Index:0	 Random:2
Indexed name is  m:2
Indexed name is  
Generated name is  
Direct name is Tony Vinsant

Index:1	 Random:4
Indexed name is ×ÿ÷ÿÿÿ  is ×ÿ÷ÿÿÿ  is ×ÿ÷ÿÿÿ  is ×ÿ÷ÿÿÿ  
Generated name is  
Direct name is Tony Vinsant

Index:2	 Random:3
Indexed name is  Random:3
Indexed name is  Random:3
Indexed namee
Generated name is  
Direct name is Tony Vinsant

Index:3	 Random:3
Indexed name is  
Generated name is  
Direct name is Tony Vinsant

Index:4	 Random:0
Indexed name is  
Generated name is  
Direct name is Tony Vinsant

It works if I address the array element directly, but not when I dynamically point to the array. Any ideas? I feel like I have tried everything. I am getting some crazy Serial output, so my guess is it is another memory issue.

Thanks!

Ryan

2

This isn't doing what you would expect:

char* name_male[]   PROGMEM = {"Tony", "Warren", "Wendell", "Will", "William"};

There are two things being defined here. name_male[] is an array of pointers, and that array will be in progmem.
Then there are the strings that are pointed to: "Tony", "Warren", etc. These are NOT in progmem.

To make your script run, you'll have to do something like:

  first_name = pgm_read_word(name_male + randomNumber);

This will dereference the array properly and get a correct string pointer.

To get the strings to ALSO go into progmem, I recommend reading

or
http://www.nongnu.org/avr-libc/user-manual/pgmspace.html

3

To get a handle on using C strings with PROGMEM I developed this tutorial for myself. Hopefully you can get some references, techniques, and ideas from it.

/*
 * ProgmemCstrings - Techniques to read character strings from PROGMEM
 * Reference: http://www.nongnu.org/avr-libc/user-manual/pgmspace.html
 * (version "Automatically generated by Doxygen 1.7.6.1 on Tue Jan 3 2012.")
 *
 * By Chris "Sembazuru" Elliott, SembazuruCDE (at) GMail.com
 * 2013-05-27
 */

#include <avr/pgmspace.h>

// Lets set up some sample string tables in program space.
// Syntax as suggested by Hashma in http://forum.arduino.cc/index.php?topic=165518.msg1239841#msg1239841
const char PROGMEM string_1[] = "First shalt thou take out the Holy Pin, then shalt thou count to three, no more, no less.";
const char PROGMEM string_2[] = "Three shall be the number thou shalt count, and the number of the counting shall be three.";
const char PROGMEM string_3[] = "Four shalt thou not count, neither count thou two, excepting that thou then proceed to three.";
const char PROGMEM string_4[] = "Five is right out.";
const char PROGMEM string_5[] = "Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who being naughty in My sight, shall snuff it.";
// Pointer table assignment copied directly from the reference source at nongnu.org w/o thinking.
// This syntax (PGM_P) still seems to compile and work but is it considered good programming practice?
PGM_P string_table[] PROGMEM =
{
  string_1,
  string_2,
  string_3,
  string_4,
  string_5
};

void setup()
{
  Serial.begin(115200);
  while (!Serial) { // Wait for serial port to connect. Needed for Leonardo only.
  }
  delay(1000); // Simply to allow time for the ERW versions of the IDE time to automagically open the Serial Monitor. 1 second chosen arbitrarily.
  Serial.println();  // Newlines printed to separate consecutive runs on Serial Monitor
  Serial.println();
  Serial.println(F(" ***** Instructions for using the Holy Hand Grenade of Antioch, stored in PROGMEM"));
  Serial.println();
}

void loop()
{
  long startMicros = 0;  // Timer to calculate how long it takes to read and printout strings.
  long stopMicros = 0;  // Should be able to subtract out the actual serial traffic because both output will be the same.

  startMicros = micros();
  recommendedStringReferencing();
  stopMicros = micros();
  Serial.print(F(" ** The amount of microseconds to fetch and display the strings from PROGMEM using the suggested strcpy_P method is: "));
  Serial.println(stopMicros - startMicros);
  Serial.println();

  startMicros = micros();
  alternateStringReferencing();
  stopMicros = micros();
  Serial.print(F(" ** The amount of microseconds to fetch and display the strings from PROGMEM using my byte-by-byte method is: "));
  Serial.println(stopMicros - startMicros);
  Serial.println();
  while (1)
  {
    // Busy spin to basically stop execution.
    // Yeah, I could have put everything into setup() to run once and leave loop() empty, but I didn't.
  }
}

/*
 * This is the recommended method to fetch a string from PROGMEM. It does require having a buffer that is >= the longest string.
 * If one changes one of their strings, then they need to manually verify that the buffer is still long enough.
 * Tests on my UNO show at serial baud 2400 this is (repeatable) 1970044us.
 * Tests on my UNO show at serial baud 4800 this is (repeatable) 986208us.
 * Tests on my UNO show at serial baud 9600 this is (repeatable) 491924us.
 * Tests on my UNO show at serial baud 14400 this is (repeatable) 328736us.
 * Tests on my UNO show at serial baud 19200 this is (repeatable) 245964us.
 * Tests on my UNO show at serial baud 28800 this is (repeatable) 163188us.
 * Tests on my UNO show at serial baud 38400 this is (repeatable) 122980us.
 * Tests on my UNO show at serial baud 57600 this is (repeatable) 80412us.
 * Tests on my UNO show at serial baud 115200 this is (repeatable) 40208us.
 */
void recommendedStringReferencing()
{
  char buffer[200];  // Buffer to hold the characters for the suggested method of reading strings. This _should_ be longer than the longest string in progmem.

  for (unsigned char i = 0; i < 5; i++) // copied directly from the reference source w/o thinking.
  {
    strcpy_P(buffer, (PGM_P)pgm_read_word(&(string_table[i])));
    Serial.println(buffer);
  }
  return;
}

/*
 * This is an alternate method that I'm playing with. If all one wants to do with the string is push it out of the Arduino using a method that can
 * take individual bytes, why not only use a single byte buffer and read the string byte by byte, stopping when one reaches a string termination
 * character?
 *
 * This has the benefit of making code maintenance easier because one doesn't need to verify that any changed strings won't overflow a buffer character array.
 * Tests on my UNO show at serial baud 2400 this is (repeatable) 1970044us, making it within 4us (minimum micros() resolution) as strcpy_p().
 * Tests on my UNO show at serial baud 4800 this is (repeatable) 986208us, making it within 4us as strcpy_p().
 * Tests on my UNO show at serial baud 9600 this is (repeatable) 491920us, making it between 4us and 8us faster than strcpy_p().
 * Tests on my UNO show at serial baud 14400 this is (repeatable) 328736us, making it within 4us as strcpy_p().
 * Tests on my UNO show at serial baud 19200 this is (repeatable) 245960us, making it between 4us and 8us faster than strcpy_p().
 * Tests on my UNO show at serial baud 28800 this is (repeatable) 163188us, making it between 4us and 8us faster than strcpy_p().
 * Tests on my UNO show at serial baud 38400 this is (repeatable) 122980us, making it within 4us as strcpy_p().
 * Tests on my UNO show at serial baud 57600 this is (repeatable) 80412us, making it within 4us as strcpy_p().
 * Tests on my UNO show at serial baud 115200 this is (repeatable) 40208us, making it within 4us as strcpy_p().
 *
 * I wonder if it is the same method strcpy_P() uses since the timing is so similar.
 * I provide this sketch as a learning reference.
 */
void alternateStringReferencing()
{
  char buffer;  // Buffer to hold a single character to allow checking for NULL.
  unsigned int offset;  // Incrementing offset to step through PROGMEM to step through the string.

  for (byte i = 0; i < 5; i++)  // increment through my 5 strings.
  {
    offset = 0;  // (Re)set off set to 0 at the start of each string loop.
    do  // Want to read and act on one character before considering dropping out.
    {
      buffer = pgm_read_byte((PGM_P)pgm_read_word(&(string_table[i])) + offset);  // Read a single character.
      if (buffer)  // Check to see if it is any thing other than a null. A null would signify the end of the string.
      {
        Serial.print(buffer);  // Found something other than a null, so print it without a line feed.
      }
      else
      {
        Serial.println();  // Found a null, so print a line feed.
      }
      offset++;  // increment PROGMEM pointer for the current string offset for the next byte.
    }
    while (buffer);  // Loop until a null found.
  }
}

Yeah... I've been waiting for an excuse to post this. :wink:

4

Thank you for your examples. I tried modifying my existing code by adding gardner's mod to my program:

  first_name = pgm_read_word(name_male[randomNumber]);
  last_name  = pgm_read_word(name_last[randomNumber]);

But now I am getting an "invalid conversion from 'uint16_t' to 'char*' error when I compile. I tried to read through the PROGMEM examples, but they all require you to list out the elements in the array at single pointers. While I could do this, it would REALLY a pain to list each individual entry. I have another routine I run on a separate machine that generates my array. If possible, I would prefer to avoid having to manually edit this array.

So, is there a way to keep my array in this format:

char* name_male[]   PROGMEM = {"Tony", "Warren", "Wendell", "Will", "William"};

But still address the elements while avoiding having to make the equivalent of this:

PGM_P string_table[] PROGMEM =
{
  string_1,
  string_2,
  string_3,
  string_4,
  string_5
};

Basically, I can't control how many elements will be in my array, what their values will be, and they come in a CSV format. Having to manually parse that into a string_table format would be a nightmare. I would still like to be able to retrieve the value with something like last_name = name_last[randomNumber];
and it would be great if the string_table could be dynamically generated by evaluating something like the length of the array in PROGMEM.

Am I completely missing the point? I have ventured from tinkerer into computer science territory, and this is above my level of understanding.

Many thanks.

5

81Pantah:
Thank you for your examples. I tried modifying my existing code by adding gardner's mod to my program:

  first_name = pgm_read_word(name_male[randomNumber]);

last_name  = pgm_read_word(name_last[randomNumber]);




But now I am getting an "invalid conversion from 'uint16_t' to 'char*' error when I compile.

Sorry, you will need to cast the result of pgm_read_word() to char *, thus:

  first_name = (char *) pgm_read_word(name_male + randomNumber);
  last_name  = (char *) pgm_read_word(name_last + randomNumber);

Please note that I have changed the inside of the pgm_read_word() from a[ i ] to a+i. This is an important change.

6

81Pantah:
Thank you for your examples. I tried modifying my existing code by adding gardner's mod to my program:

  first_name = pgm_read_word(name_male[randomNumber]);

last_name  = pgm_read_word(name_last[randomNumber]);




But now I am getting an "invalid conversion from 'uint16_t' to 'char*' error when I compile. I tried to read through the PROGMEM examples, but they all require you to list out the elements in the array at single pointers. While I could do this, it would REALLY a pain to list each individual entry. I have another routine I run on a separate machine that generates my array. If possible, I would prefer to avoid having to manually edit this array.

Well, unfortunately, the only way to actually get the names into PROGMEM (instead of your first attempt which just put pointers to SRAM addresses in PROGMEM which is kinda pointless...) this is how it has to be done. Maybe have the routine that generates your array also generate that bit of code for you? Or... (see lower...)

So, is there a way to keep my array in this format:

char* name_male[]   PROGMEM = {"Tony", "Warren", "Wendell", "Will", "William"};

But still address the elements while avoiding having to make the equivalent of this:

PGM_P string_table[] PROGMEM =

{
  string_1,
  string_2,
  string_3,
  string_4,
  string_5
};




Basically, I can't control how many elements will be in my array, what their values will be, and they come in a CSV format. Having to manually parse that into a `string_table` format would be a nightmare. I would still like to be able to retrieve the value with something like `last_name = name_last[randomNumber];`
and it would be great if the `string_table` could be dynamically generated by evaluating something like the length of the array in PROGMEM.

Am I completely missing the point? I have ventured from tinkerer into computer science territory, and this is above my level of understanding.

Many thanks.

Well, unless you have other constraints that limit this, since you are generating lists in (or was that from) CSV format on a computer, instead of having to recompile your Arduino program every time you change the list why not save the list as CSV (either a subset of your master list, or the entire master CSV) to a SD (or microSD) card. Re-jigger your code to read the names from an SD card.

My first attempt at such (with my current level of ignorance) would be to open the CSV file (would either need to be the only file on the SD card, or a very specific name hard-coded into your Arduino sketch) and count the number of lines. I'm not sure if there is an automatic function or method for that, but it is a simple matter of readln()'ing into nothing while incrementing a counter until the end of the file. Might be a bit smarter to also build an array to store the fileread pointer (not sure exactly what it is called right now w/o looking at the SD library documentation) for the beginning of each line into an array so you can jump to (for example) line#93 instead of scanning from the beginning to that line.

Then when reading out randomly chosen names, get the fileread pointer to the beginning of the randomly chosen line (either by readln()'ing into nothing from the beginning until the line before the one you need, or using the filepointer with a command that moves the fileread pointer to an absolute bit position in the file). Then read each character watching for quotation marks and commas to figure out where in the CSV line format you are, and read the value you want.

I hope that made sense. I'm in kind of a rush and don't have much time to edit...

7

Sembazuru:
the only way to actually get the names into PROGMEM ... this is how it has to be done.

It's not that bad. You could also do:

#define MAX_NAME 15
char name_male[][MAX_NAME] PROGMEM = {"Tony", "Warren", "Wendell", "Will", "William"};

This has the downside of padding each name string to 15, or whatever, characters. But flash is abundant and maybe this is a win.

8

Thanks everyone.

I ended up using Gardner's solution of first_name = (char *) pgm_read_word(name_male + randomNumber); which seamlessly integrated with my current code and works flawlessly.

Many thanks.