PROGMEM and arrays

Projects Programming
1

Hi, I'm in process of lowering memory footprint of little program of mine, main culprit being that I have need of quite a few arrays.
I thought I could put arrays into progmem, and then reading them with pgm_read_byte_near() as needed.

Example of array:

int shiftTimeMap[14][12] PROGMEM {
{999,       0,   10,   20,   30,   40,   50,   60,   70,   80,   90,   100 },   //shift pressure %
//-----------------------------------------------------------------------
{ -20,    1100, 1100, 1100, 1100, 1100, 1100, 1100, 1100,  900,  800,  800 },
{ -10,    1100, 1100, 1100, 1100, 1100, 1100, 1100,  900,  900,  800,  800 },
{   0,    1100, 1100, 1100, 1100, 1100, 1100,  900,  900,  900,  800,  800 },
{  10,    1100, 1100, 1100, 1100, 1100, 1100,  900,  900,  900,  800,  800 },
{  20,    1100, 1100, 1100, 1000, 1000, 1000,  900,  900,  900,  800,  800 },
{  30,    1100, 1100, 1100, 1000, 1000,  900,  900,  800,  700,  700,  600 },
{  40,    1000, 1000, 1000, 1000,  900,  800,  700,  700,  700,  500,  500 },
{  50,    1000, 1000, 1000,  900,  900,  800,  700,  700,  500,  500,  500 },
{  60,    1000, 1000, 1000,  900,  800,  700,  600,  500,  450,  450,  450 },
{  70,    1000, 1000,  900,  900,  800,  700,  600,  500,  450,  320,  320 },
{  80,    1000,  900,  900,  800,  800,  700,  600,  500,  320,  300,  300 },
{  90,    1000,  900,  800,  800,  800,  700,  600,  500,  320,  300,  300 },
{ 100,    1000,  900,  800,  800,  800,  700,  600,  500,  320,  300,  300 }};
//oil temp

and I have general purpose readMap() function to fetch the data from array:

int readMap(int theMap[14][12], int x, int y) {
  int xidx = 0; // by default near first element
  int xelements = LEN(theMap[0]);

  int distance = abs(pgm_read_byte_near(&theMap[0][xidx]) - x); 
  for (int i = 1; i < xelements; i++)
  {
    int d = abs(pgm_read_byte_near(&theMap[0][i]) - x);
    if (d < distance)
    {
      xidx = i;
      distance = d;
    }
  }
  int yidx = 0; // by default near first element
  int yelements = LEN(*theMap);

  distance = abs(pgm_read_byte_near(&theMap[yidx][0]) - y); 
  for (int i = 1; i < yelements; i++)
  {
    int d = abs(pgm_read_byte_near(&theMap[i][0]) - y);
    if (d < distance)
    {
      yidx = i;
      distance = d;
    }
  }

  int mapValue = pgm_read_byte_near(&theMap[yidx][xidx]);
  return mapValue;
}

I still fail to see more free dynamic memory for some reason.
Have I understood something wrong with PROGMEM or does this make any sense whatsoever?

Thanks for your guidance:)

2

For PROGMEM to work, the array has to be declared const like:

const int shiftTimeMap[14][12] PROGMEM { 
  ... 
};

// array should be referred to as const int...
//
int readMap( const int theMap[14][12], int x, int y)
{
  ...
}

The compiler should have issued a warning about PROGMEM
You can turn warnings on in the Arduino IDE: File, Preferences, Compiler Warnings

Yours,
TonyWilk

3

Thanks, tried defining array and function parameter as const but no change in available dynamic memory.
Compiler has warnings on, however, can I trust compilers idea of free memory in this case at all, or do I see reflected free memory when actually running the code?

4

I also noticed that if I change location of PROGMEM definition like this:

const int PROGMEM shiftTimeMap[14][12] {
{999,       0,   10,   20,   30,   40,   50,   60,   70,   80,   90,   100 },   //shift pressure %
//-----------------------------------------------------------------------
{ -20,    1100, 1100, 1100, 1100, 1100, 1100, 1100, 1100,  900,  800,  800 },
..

compiler will indeed give me a warning, but without reason:

sketch/transmap.h: In function 'void polltrans()':
sketch/transmap.h:22:38: warning: '__progmem__' attribute ignored [-Wattributes]
 const int PROGMEM shiftTimeMap[14][12] {
5

ah, solved it, arrays defined with PROGMEM need to be globally defined or have "static".
I just said

static const int shiftTimeMap[14][12] PROGMEM {

and everything seems well, thank you for your help!:slight_smile:

6

Also to be noted that in documentation:
https://playground.arduino.cc/Main/PROGMEM

in example there are pgm read functions wrong way around, imo.

Also it would be nice to have mentioned that writable information needs to live globally.

Also it would be equally lovely to have compiler to give out warning on all the possible syntaxes with PROGMEM definition.

7

mui:
ah, solved it, arrays defined with PROGMEM need to be globally defined or have "static".

Great!

Re: the documentation at Arduino Playground - HomePage

The pgm_read_byte_near() function takes an address as the parameter, so there are several different ways you can express the pointer and get the same result. With a simple array you could get an element by any one of:

  pgm_read_byte_near( &(myArray[4]) );      // address of element [4]
  pgm_read_byte_near( &(myArray[0]) + 4 );  // address of 1st element + 4
  pgm_read_byte_near( myArray + 4 );        // address of array + 4

The array having to be global... yeh, unfortunately it's one of those things that is only obvious once you know ::slight_smile:

Yours,
TonyWilk

8

I tried to say that in first example:

// read back a 2-byte int
 displayInt = pgm_read_word_near(charSet + k)

// read back a char 
myChar =  pgm_read_byte_near(signMessage + k);

I believe pgm_read_word_near should be used for signMessage and pgm_read_byte_near for charSet?

9

mui:
I believe pgm_read_word_near should be used for signMessage and pgm_read_byte_near for charSet?

Ah, I see what you mean, but no...

the example does use a pretty stupid name "charSet" for an array which is not char, it is uint6_t

Snippet from the PROGMEM page:

// save some unsigned ints
const uint16_t charSet[]  PROGMEM  = { 65000, 32796, 16843, 10, 11234};

// save some chars
const char signMessage[] PROGMEM  = {"I AM PREDATOR..."};

// read back a 2-byte int
displayInt = pgm_read_word_near(charSet + k);

// read back a char 
myChar =  pgm_read_byte_near(signMessage + k);

signMessage is an array of char, each element is one byte.

charSet is an array of uint16_t, each element is 16bits, which is two bytes also known as a word

The function pgm_read_word_near() is passed a pointer and returns a word, aka uint16_t, from that location.

Yours,
TonyWilk

10

ah, "obviously" :smiley: my bad.