PROGMEM float arrays

Problem: trying to put polynomial coefficients for thermocouple interpolation into PROGMEM. Below is a short example that shows the problem. I would like to use two-dimensional arrays, but have been experimenting with both to see what works.

#include <avr/pgmspace.h>
#include <WProgram.h>
PROGMEM float coeff1[] = {
 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 ,
 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9 ,
 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9 
};
PROGMEM float coeff2[3][10] = {
 { 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 },
 { 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9 },
 { 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9 }
};
void setup(){
  Serial.begin(57600);
  for( uint8_t i = 0; i < 30; i++ ) {
    Serial.print( pgm_read_float_near( coeff1[i] )); // NG
    Serial.print("  ");
    Serial.print( pgm_read_float_near( coeff1 + i )); // OK
    Serial.print("  ");
    uint8_t j = i / 10;    uint8_t k = i % 10;
    Serial.print( pgm_read_float_near( coeff2[j][k] )); // NG
    Serial.print("  ");
    Serial.print( pgm_read_float_near( coeff2 + i )); // first 3 are OK
    Serial.println();
  }
}
void loop(){}

The results:

0.00  0.00  0.00  0.00
0.00  0.10  0.00  1.00
0.00  0.20  0.00  2.00
0.00  0.30  0.00  0.00
0.00  0.40  0.00  -2.03
0.00  0.50  0.00  -0.00
0.00  0.60  0.00  0.00
0.00  0.70  0.00  -0.00
0.00  0.80  0.00  0.00
0.00  0.90  0.00  0.00
0.00  1.00  0.00  -0.00
0.00  1.10  0.00  -0.00
0.00  1.20  0.00  -0.00
0.00  1.30  0.00  -0.00
0.00  1.40  0.00  -0.00
0.00  1.50  0.00  0.00
0.00  1.60  0.00  -0.00
0.00  1.70  0.00  -0.00
0.00  1.80  0.00  0.00
0.00  1.90  0.00  -0.00
0.00  2.00  0.00  -0.00
0.00  2.10  0.00  0.00
0.00  2.20  0.00  -0.00
0.00  2.30  0.00  -0.00
0.00  2.40  0.00  -0.00
0.00  2.50  0.00  -0.00
0.00  2.60  0.00  -0.00
0.00  2.70  0.00  -0.00
0.00  2.80  0.00  -0.00
0.00  2.90  0.00  -0.00

Only the second column, the one that uses a 1-dimensional array and is addressed by (coeff1 + i ) works correctly. So there is something about referencing float arrays in PROGMEM that I’m not getting right.

How can I rewrite this so that I can address a 2-dimensional array, ideally using (coeff2 + i) syntax?

Jim

coeff + (i * 10) + k

AWOL: coeff + (i * 10) + k

Thanks for the suggestion. Since I have i set up to run 0 to 29, and j goes 0 to 2, I assume you intended that to be coeff + (j * 10) + k . Tried it, no luck. Exact same result as coeff + i .

Either PROGMEM stores multi-dimensional arrays differently, or the initialization of multi-dimensional arrays does not actually result in the data being stored. I'm not sure how to determine which is the case.

Jim

1. Elements of 1-D are stored sequentially in memory, and elements of 2-D arrays are stored sequentially in memory in row-major order, whether the arrays are in program memory or not.

2. Initialization of 1-D arrays and initialization of 2-D arrays work the same in program memory as they do in conventional memory.

3. Accessing data in program memory with one of the avr-libc pgm_read_whatever() functions requires that you give it the address of the element

    float *pf = (float *)coeff2;                           // Address if coeff2[0][0]
    
    for( uint8_t i = 0; i < 30; i++ ) {
        Serial.print( pgm_read_float( &coeff1[i]) );       // Give it the address
        Serial.print("  ");
        Serial.print( pgm_read_float( coeff1 + i ) );      // Index from the first address
        Serial.print("  ");
        uint8_t j = i / 10;
        uint8_t k = i % 10;
        Serial.print( pgm_read_float( &coeff2[j][k] ) );    // Give it the address
        Serial.print("  ");
        Serial.print( pgm_read_float( &coeff2[0][0] + i ) );// Index from the address of the first array element
        Serial.print("  ");
        Serial.print( pgm_read_float( coeff2[j] + k ) );    // Index from the first address on row j
        Serial.print("  ");
        Serial.print( pgm_read_float( pf + i ));            // Index from the value of pf
        Serial.println();
    }

Output:


0.00  0.00  0.00  0.00  0.00  0.00
0.10  0.10  0.10  0.10  0.10  0.10
0.20  0.20  0.20  0.20  0.20  0.20
0.30  0.30  0.30  0.30  0.30  0.30
0.40  0.40  0.40  0.40  0.40  0.40
0.50  0.50  0.50  0.50  0.50  0.50
0.60  0.60  0.60  0.60  0.60  0.60
0.70  0.70  0.70  0.70  0.70  0.70
0.80  0.80  0.80  0.80  0.80  0.80
0.90  0.90  0.90  0.90  0.90  0.90
1.00  1.00  1.00  1.00  1.00  1.00
1.10  1.10  1.10  1.10  1.10  1.10
1.20  1.20  1.20  1.20  1.20  1.20
1.30  1.30  1.30  1.30  1.30  1.30
1.40  1.40  1.40  1.40  1.40  1.40
1.50  1.50  1.50  1.50  1.50  1.50
1.60  1.60  1.60  1.60  1.60  1.60
1.70  1.70  1.70  1.70  1.70  1.70
1.80  1.80  1.80  1.80  1.80  1.80
1.90  1.90  1.90  1.90  1.90  1.90
2.00  2.00  2.00  2.00  2.00  2.00
2.10  2.10  2.10  2.10  2.10  2.10
2.20  2.20  2.20  2.20  2.20  2.20
2.30  2.30  2.30  2.30  2.30  2.30
2.40  2.40  2.40  2.40  2.40  2.40
2.50  2.50  2.50  2.50  2.50  2.50
2.60  2.60  2.60  2.60  2.60  2.60
2.70  2.70  2.70  2.70  2.70  2.70
2.80  2.80  2.80  2.80  2.80  2.80
2.90  2.90  2.90  2.90  2.90  2.90

Regards,

Dave

Footnote:
The name of an array is a const pointer to the data type of the array. The value of that pointer is the address of the first element of the array.

coeff1 is an array of floats in program memory, so the following two are exactly the same to a C++ compiler:

pgm_read_float_near( coeff1 + i )

Exadtly same as

pgm_read_float_near( &coeff1[0] + i )

Also:
In general if p is a pointer data type and n is an integer data type, the following two are treated exactly the same by the compiler:

*p + n

Exactly the same as

p[n]

So, for the convenience of humans any pointer expression can be “converted” to an array expression and vice versa. Please note that I am not saying that pointers are the same as arrays (they are not). It’s just notation.

Now, coeff2[3][10] is an array of three arrays. Each of these is an array of ten floats.

That is

coeff2[0] is an array of ten floats
coeff2[1] is an array of ten floats
coeff2[2] is an array of ten floats

The identifier coeff2 is a pointer to an array of ten floats. Its value is the address of coeff2[0]. The item at that address is the floating point value of coeff2[0][0]

Then coef2+1 is a pointer to an array of ten floats. Its value is the address of coeff2[1], which is ten floats after the address of coeff2[0]. That’s why I introduced a pointer to float equal to the address of coeff2[0][0] so that the loop variable i can be used to index directly into the 2-D array memory space.

The Flash library by Mikal Hart makes it possible to use conventional array and 2-D array notation to access data in program memory. (Does some other interesting stuff as well…)

Extremely helpful. Thank you.

Why aren't these equivalent?

Serial.print( pgm_read_float( &coeff2[0][0] + i ) );// Index from the address of the first array element
Serial.print( pgm_read_float( coeff2 + i ) );// Index from the address of the first array element

Both should evaluate to an address that is offset by i elements from coeff2[0][0]. Wouldn't they be equivalent for a non-PROGMEM array?

Jim

JimG:

Why aren’t these equivalent?

Serial.print( pgm_read_float( &coeff2[0][0] + i ) );// Index from the address of the first array element

Serial.print( pgm_read_float( coeff2 + i ) );// Index from the address of the first array element



Both should evaluate to an address that is offset by i elements from coeff2[0][0]. Wouldn't they be equivalent for a non-PROGMEM array?
...

No, they would not. See Footnote

Here’s the thing: In C and C++ (and, therefore, in Arduino) there is actually no such thing as a 2-D array. Well, from a notational point of view there is, but it’s actually treated by the compiler as an array of arrays. This is a Big Deal for cases like this.

I’ll try again:

So, coeff2[3][10] is an array of three arrays of floats. Each of the three is an array of ten floats.

Used as an identifier in an expression, coeff2 is a pointer to an array of ten floats. The value of coeff2 is the address of coeff2[0], which is the same numerically as the address of coeff2[0][0].

The value of coeff2 + 1 is greater than the value of coeff2 by the size of an array of ten floats (40 bytes). That’s the whole idea behind pointer arithmetic and that’s why, in general, that array indexing and pointer arithmetic can be equivalent.

In “normal” programs, the programmer doesn’t care about the actual values of pointers and just uses array notation or pointer arithmetic to access whatever the pointers are pointing to, but when trying to nail home a point like this, maybe it would be helpful to print out some of them:

#include <avr/pgmspace.h>
#include <WProgram.h>

PROGMEM float coeff2[3][10] = {
 { 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 },
 { 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9 },
 { 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9 }
};
void setup()
{
    Serial.begin(9600); // Or whatever...


    // Make sure the size of a pointer is the same as the size of an unsigned int
    // so that we can see the decimal values of the pointers:
    Serial.print("sizeof(float *) = ");Serial.print(sizeof(float *));
    Serial.print(" and sizeof(unsigned) = ");Serial.println(sizeof(unsigned));
    if (sizeof(float *) != sizeof(unsigned)) {
        Serial.println("Hmmm---I really thought they were the same size in avr-gcc.");
        while (1)
           ;
    }
    Serial.println();
    Serial.print("&coeff2[0][0] = ");Serial.println((unsigned)&coeff2[0][0]);
    for (int i = 0; i < 3; i++) {
        Serial.print("&coeff2[");Serial.print(i);
        Serial.print("] = ");Serial.println((unsigned)&coeff2[i]);
    }
    Serial.println();
    
    Serial.print("coeff2 = ");Serial.println((unsigned)coeff2);
    for (int i = 0; i < 3; i++) {
        Serial.print("coeff2 + ");Serial.print(i);
        Serial.print(" = ");Serial.println((unsigned)(coeff2+i));
    }
    Serial.println();
    
    for (int i = 0; i < 3; i++) {
        Serial.print("&coeff2[");Serial.print(i);
        Serial.print("][0] = "); Serial.println((unsigned)(&coeff2[i][0]));
    }
    Serial.println();
    for (int i = 0; i < 3; i++) {
        float *fp1 = &coeff2[0][0] + i;     // coeff2[0][0] is a float
        float *fp2 = (float *)(coeff2 + i); // coeff2 is a pointer to an array of ten floats
        Serial.print("i = ");Serial.print(i);
        Serial.print(": fp1 = ");Serial.print((unsigned)(fp1));
        Serial.print(", fpp2 = ");Serial.println((unsigned)fp2);
        Serial.print("*fp1 = ");Serial.print(pgm_read_float(fp1));
        Serial.print(", *fp2 = ");Serial.println(pgm_read_float(fp2));
        Serial.println();
    }
}
void loop(){}

Output:


sizeof(float *) = 2 and sizeof(unsigned) = 2

&coeff2[0][0] = 104
&coeff2[0] = 104
&coeff2[1] = 144
&coeff2[2] = 184

coeff2 = 104
coeff2 + 0 = 104
coeff2 + 1 = 144
coeff2 + 2 = 184

&coeff2[0][0] = 104
&coeff2[1][0] = 144
&coeff2[2][0] = 184

i = 0: fp1 = 104, fpp2 = 104
*fp1 = 0.00, *fp2 = 0.00

i = 1: fp1 = 108, fpp2 = 144
*fp1 = 0.10, *fp2 = 1.00

i = 2: fp1 = 112, fpp2 = 184
*fp1 = 0.20, *fp2 = 2.00

Regards,

Dave

Footnote:

Arrays not in program memory:

float coeff2[3][10] = {
 { 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 },
 { 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9 },
 { 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9 }
};
void setup()
{
    Serial.begin(9600); // Or whatever...


    // Make sure the size of a pointer is the same as the size of an unsigned int
    // so that we can see the decimal values of the pointers:
    Serial.print("sizeof(float *) = ");Serial.print(sizeof(float *));
    Serial.print(" and sizeof(unsigned) = ");Serial.println(sizeof(unsigned));
    if (sizeof(float *) != sizeof(unsigned)) {
        Serial.println("Hmmm---I really thought they were the same size in avr-gcc.");
        while (1)
           ;
    }
    Serial.println();
    Serial.print("&coeff2[0][0] = ");Serial.println((unsigned)&coeff2[0][0]);
    for (int i = 0; i < 3; i++) {
        Serial.print("&coeff2[");Serial.print(i);
        Serial.print("] = ");Serial.println((unsigned)&coeff2[i]);
    }
    Serial.println();
    
    Serial.print("coeff2 = ");Serial.println((unsigned)coeff2);
    for (int i = 0; i < 3; i++) {
        Serial.print("coeff2 + ");Serial.print(i);
        Serial.print(" = ");Serial.println((unsigned)(coeff2+i));
    }
    Serial.println();
    
    for (int i = 0; i < 3; i++) {
        Serial.print("&coeff2[");Serial.print(i);
        Serial.print("][0] = "); Serial.println((unsigned)(&coeff2[i][0]));
    }
    Serial.println();
    
    for (int i = 0; i < 3; i++) {
        float *fp1 = &coeff2[0][0] + i;     // coeff2[0][0] is a float
        float *fp2 = (float *)(coeff2 + i); // coeff2 is a pointer to an array of ten floats
        Serial.print("i = ");Serial.print(i);
        Serial.print(": fp1 = ");Serial.print((unsigned)fp1);
        Serial.print(", fpp2 = ");Serial.println((unsigned)fp2);
        Serial.print("*fp1 = ");Serial.print(*fp1);
        Serial.print(", *fp2 = ");Serial.println(*fp2);
        Serial.println();
    }
}
void loop(){}

Output:


sizeof(float *) = 2 and sizeof(unsigned) = 2

&coeff2[0][0] = 407
&coeff2[0] = 407
&coeff2[1] = 447
&coeff2[2] = 487

coeff2 = 407
coeff2 + 0 = 407
coeff2 + 1 = 447
coeff2 + 2 = 487

&coeff2[0][0] = 407
&coeff2[1][0] = 447
&coeff2[2][0] = 487

i = 0: fp1 = 407, fpp2 = 407
*fp1 = 0.00, *fp2 = 0.00

i = 1: fp1 = 411, fpp2 = 447
*fp1 = 0.10, *fp2 = 1.00

i = 2: fp1 = 415, fpp2 = 487
*fp1 = 0.20, *fp2 = 2.00

OK. I think I finally got it. :P

Thanks.

Jim

Dave -

I rewrote my test sketch like this:

#include <avr/pgmspace.h>
#include <WProgram.h>
class pCoeff {
  public:
  pCoeff(){}
  static PROGMEM float coeff1[30];
  static PROGMEM float coeff2[3][10];
};
float pCoeff::coeff1[30] = {
 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 ,
 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9 ,
 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9 
};
float pCoeff::coeff2[3][10] = {
 { 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 },
 { 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9 },
 { 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9 }
};
float poly( float x, float* c, uint8_t rowdim, uint8_t coldim ) {
  uint8_t idx = (rowdim - 1 ) * coldim;
  float fx = 0.0;
  for( ; ; ) {
    fx = x * fx + pgm_read_float_near( c + idx );
    if( idx == 0 )
      break;
    idx -= coldim;
  }
  return fx;
}
pCoeff pc;
void setup(){
  Serial.begin(57600);
  Serial.println( poly( 1.0, &pc.coeff2[0][5], 3, 10 ) );
  Serial.println( poly( 1.0, &pc.coeff1[5], 3, 10 ) );
}
void loop(){}

The poly function evaluates (in this example) a quadratic equation using Horner’s rule and iterating up the selected column to get the coefficients. I wanted to show that the coefficient array could be declared as either 1-D or 2-D and still work. I also wanted to make sure all of the PROGMEM stuff was going to work before rewriting my main code.

The good news is that the same code, when applied to the real program, reduced the RAM requirements per thermocouple type from over 200 bytes each, to only 20 bytes each. In an application that only has 700 bytes of free RAM, this makes a huge difference.

Thanks again for your patient assistance.

Jim