Strange things (Duemilanov and PROGMEM)

Forum 2005-2010 (read only) Software Syntax & Programs
1

First off, I apologize for the length of the code I'm posting, but the issue is difficult, (As in I'm not able to), to reproduce if anything is cut out. My situation is this: If you look towards the end of the code sample there is a function 'nodeOn_P()' which is commented out. If you run the code with this function commented out, you get the expected output, (an output of the dataArray[] in binary), but if you uncomment it you get a completely different output even though that function is not called anywhere else in the program! Does anyone know what is going on with this?

Again, sorry about the code length, I'm just trying to put some arrays in flash so that I can free up more ram.

Anyway, the code:

#include <avr/pgmspace.h>
 
const int mappingArray[86][2] = {
  {7, 2},      // Node 0
  {2, 8},      // Node 1
  {5, 2},      // Node 2
  {2, 10},      // Node 3
  {5, 3},      // Node 4
  {1, 3},      // Node 5
  {4, 4},      // Node 6
  {2, 3},      // Node 7
  {3, 7},      // Node 8
  {4, 8},      // Node 9
  {0, 9},      // Node 10
  {6, 2},      // Node 11
  {5, 10},      // Node 12
  {3, 8},      // Node 13
  {4, 1},      // Node 14
  {3, 3},      // Node 15
  {5, 1},      // Node 16
  {1, 7},      // Node 17
  {1, 4},      // Node 18
  {2, 4},      // Node 19
  {3, 6},      // Node 20
  {5, 7},      // Node 21
  {2, 6},      // Node 22
  {5, 9},      // Node 23
  {4, 5},      // Node 24
  {5, 5},      // Node 25
  {7, 10},      // Node 26
  {1, 10},      // Node 27
  {7, 8},      // Node 28
  {7, 3},      // Node 29
  {3, 1},      // Node 30
  {7, 9},      // Node 31
  {0, 0},      // Node 32
  {6, 1},      // Node 33
  {6, 4},      // Node 34
  {0, 3},      // Node 35
  {6, 7},      // Node 36
  {2, 7},      // Node 37
  {5, 6},      // Node 38
  {2, 1},      // Node 39
  {4, 3},      // Node 40
  {6, 9},      // Node 41
  {3, 9},      // Node 42
  {5, 8},      // Node 43
  {1, 8},      // Node 44
  {4, 9},      // Node 45
  {4, 2},      // Node 46
  {6, 10},      // Node 47
  {1, 0},      // Node 48
  {3, 0},      // Node 49
  {7, 5},      // Node 50
  {4, 7},      // Node 51
  {1, 6},      // Node 52
  {7, 6},      // Node 53
  {4, 6},      // Node 54
  {2, 2},      // Node 55
  {6, 3},      // Node 56
  {3, 4},      // Node 57
  {7, 0},      // Node 58
  {4, 0},      // Node 59
  {3, 10},      // Node 60
  {0, 8},      // Node 61
  {0, 10},      // Node 62
  {0, 1},      // Node 63
  {4, 10},      // Node 64
  {0, 7},      // Node 65
  {7, 4},      // Node 66
  {3, 5},      // Node 67
  {1, 9},      // Node 68
  {7, 7},      // Node 69
  {6, 6},      // Node 70
  {2, 5},      // Node 71
  {1, 5},      // Node 72
  {3, 2},      // Node 73
  {6, 8},      // Node 74
  {2, 0},      // Node 75
  {1, 1},      // Node 76
  {6, 0},      // Node 77
  {5, 0},      // Node 78
  {2, 9},      // Node 79
  {0, 4},      // Node 80
  {5, 4},      // Node 81
  {6, 5},      // Node 82
  {0, 6},      // Node 83
  {7, 1},      // Node 84
  {0, 5}      // Node 85
  };

const prog_int8_t mappingArray_P[86][2] PROGMEM = {
  {7, 2},      // Node 0
  {2, 8},      // Node 1
  {5, 2},      // Node 2
  {2, 10},      // Node 3
  {5, 3},      // Node 4
  {1, 3},      // Node 5
  {4, 4},      // Node 6
  {2, 3},      // Node 7
  {3, 7},      // Node 8
  {4, 8},      // Node 9
  {0, 9},      // Node 10
  {6, 2},      // Node 11
  {5, 10},      // Node 12
  {3, 8},      // Node 13
  {4, 1},      // Node 14
  {3, 3},      // Node 15
  {5, 1},      // Node 16
  {1, 7},      // Node 17
  {1, 4},      // Node 18
  {2, 4},      // Node 19
  {3, 6},      // Node 20
  {5, 7},      // Node 21
  {2, 6},      // Node 22
  {5, 9},      // Node 23
  {4, 5},      // Node 24
  {5, 5},      // Node 25
  {7, 10},      // Node 26
  {1, 10},      // Node 27
  {7, 8},      // Node 28
  {7, 3},      // Node 29
  {3, 1},      // Node 30
  {7, 9},      // Node 31
  {0, 0},      // Node 32
  {6, 1},      // Node 33
  {6, 4},      // Node 34
  {0, 3},      // Node 35
  {6, 7},      // Node 36
  {2, 7},      // Node 37
  {5, 6},      // Node 38
  {2, 1},      // Node 39
  {4, 3},      // Node 40
  {6, 9},      // Node 41
  {3, 9},      // Node 42
  {5, 8},      // Node 43
  {1, 8},      // Node 44
  {4, 9},      // Node 45
  {4, 2},      // Node 46
  {6, 10},      // Node 47
  {1, 0},      // Node 48
  {3, 0},      // Node 49
  {7, 5},      // Node 50
  {4, 7},      // Node 51
  {1, 6},      // Node 52
  {7, 6},      // Node 53
  {4, 6},      // Node 54
  {2, 2},      // Node 55
  {6, 3},      // Node 56
  {3, 4},      // Node 57
  {7, 0},      // Node 58
  {4, 0},      // Node 59
  {3, 10},      // Node 60
  {0, 8},      // Node 61
  {0, 10},      // Node 62
  {0, 1},      // Node 63
  {4, 10},      // Node 64
  {0, 7},      // Node 65
  {7, 4},      // Node 66
  {3, 5},      // Node 67
  {1, 9},      // Node 68
  {7, 7},      // Node 69
  {6, 6},      // Node 70
  {2, 5},      // Node 71
  {1, 5},      // Node 72
  {3, 2},      // Node 73
  {6, 8},      // Node 74
  {2, 0},      // Node 75
  {1, 1},      // Node 76
  {6, 0},      // Node 77
  {5, 0},      // Node 78
  {2, 9},      // Node 79
  {0, 4},      // Node 80
  {5, 4},      // Node 81
  {6, 5},      // Node 82
  {0, 6},      // Node 83
  {7, 1},      // Node 84
  {0, 5}      // Node 85
  };

// All the node numbers, so the last number of each array is negative to mark the end of the array
const prog_int8_t aRing0[] PROGMEM = {0, -1};
const prog_int8_t aRing1[] PROGMEM = {1 , 2 , 3 , 4 , 5, -1};
const prog_int8_t aRing2[] PROGMEM = {6 , 7 , 8 , 9 , 10, 11, 12, 13, 14, 15, -1};
const prog_int8_t aRing3[] PROGMEM = {16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, -1};
const prog_int8_t aRing4[] PROGMEM = {31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, -1};
const prog_int8_t aRing5[] PROGMEM = {46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, -1};
const prog_int8_t aRing6[] PROGMEM = {61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, -1};
const prog_int8_t aRing7[] PROGMEM = {76, 77, 78, 79, 80, 81, 82, 83, 84, 85, -1};
const prog_int8_t aRing8[] PROGMEM = {-1}; // Dummy for when more nodes are added
const prog_int8_t aRing9[] PROGMEM = {-1}; // Dummy for when more nodes are added
  
const prog_int8_t* aRingPtr[] PROGMEM = {aRing0, aRing1, aRing2, aRing3, aRing4, aRing5, aRing6, aRing7, aRing8, aRing9}; 

volatile byte dataArray[11];  // Essentially the main frame buffer  

void setup()                    
{
  Serial.begin(9600);
}

void loop()                    
{
  for(int i = 0; i <= 9; i++){
    groupOn_P((prog_int8_t*)pgm_read_byte(&aRingPtr[i]));
    delay(1);
    printArray();
    clearArray();
  }
}

void printArray(){
  for(int i = 0; i < 11; i++){
    Serial.print(dataArray[i], BIN);
    Serial.print(", ");
  }
  Serial.println();
}  

void clearArray(){
  for(int i = 0; i < 11; i++){
    dataArray[i] = 0;
  }
}

void groupOn_P(prog_int8_t group[]){
  int index = 0;
  prog_int8_t currentElement = pgm_read_byte(&group[index]);
  
  while(currentElement >= 0){
    nodeOn(currentElement);    
    index++;
    currentElement = pgm_read_byte(&group[index]);
  }
}

/* Why is it that commenting this out produces different output IF I NEVER CALL IT IN THE MAIN LOOP!?
void nodeOn_P(prog_int8_t nodeNum){
  prog_int8_t registerVal = pgm_read_byte(&mappingArray_P[nodeNum][1]);
  prog_int8_t value = pgm_read_byte(&mappingArray_P[nodeNum][0]);

  bitWrite(dataArray[registerVal], value, HIGH);
}
*/

void nodeOn(int nodeNum){
  bitWrite(dataArray[mappingArray[nodeNum][1]], mappingArray[nodeNum][0], HIGH);
}
2 
  prog_int8_t currentElement = pgm_read_byte(&group[index]);

You really want this to be just

  int8_t currentElement = pgm_read_byte(&group[index]);

because although you are reading from PROGMEM, you are reading into a normal RAM variable.
The pgmspace address space is separate from RAM address space, so what is PROBABLY happening is that currentElement (and other local variables you have delcared in a similar manner) ends up allocated as if it were in pgmspace, but the address ends up treated as RAM, and the variable ends up overlapping some area of RAM and the assignment causes some RAM location to be trashed. When your final function is uncommented, you change the pgmspace addresses that are allocated, and you change which RAM locations are corrupted.

3

Ah. That makes a lot of sense. Thanks for the quick reply. I've programmed in other languages for a while but C is taking me a little time to get my head around memory management. Anyway, again thanks for taking the time to look through it.

-tsg

4

Well, I went through all the prog_int8_t declarations and corrected them per your suggestion and I'm still getting the exact same behavior. I believe your suggestion is on the right track, in that there is something trashing ram, but I'm not sure where I'm going wrong yet. My suspicion is that it has something to do with how I'm accessing a multi-dimensional array in nodeOn_P, I just don't understand how it affects my output if I never call it- especially now that I should be reading into ram where appropriate. Here's the updated code for anyone who wants to take a look.

sorry about the long arrays.

#include <avr/pgmspace.h>
 
const int mappingArray[86][2] = {
  {7, 2},      // Node 0
  {2, 8},      // Node 1
  {5, 2},      // Node 2
  {2, 10},      // Node 3
  {5, 3},      // Node 4
  {1, 3},      // Node 5
  {4, 4},      // Node 6
  {2, 3},      // Node 7
  {3, 7},      // Node 8
  {4, 8},      // Node 9
  {0, 9},      // Node 10
  {6, 2},      // Node 11
  {5, 10},      // Node 12
  {3, 8},      // Node 13
  {4, 1},      // Node 14
  {3, 3},      // Node 15
  {5, 1},      // Node 16
  {1, 7},      // Node 17
  {1, 4},      // Node 18
  {2, 4},      // Node 19
  {3, 6},      // Node 20
  {5, 7},      // Node 21
  {2, 6},      // Node 22
  {5, 9},      // Node 23
  {4, 5},      // Node 24
  {5, 5},      // Node 25
  {7, 10},      // Node 26
  {1, 10},      // Node 27
  {7, 8},      // Node 28
  {7, 3},      // Node 29
  {3, 1},      // Node 30
  {7, 9},      // Node 31
  {0, 0},      // Node 32
  {6, 1},      // Node 33
  {6, 4},      // Node 34
  {0, 3},      // Node 35
  {6, 7},      // Node 36
  {2, 7},      // Node 37
  {5, 6},      // Node 38
  {2, 1},      // Node 39
  {4, 3},      // Node 40
  {6, 9},      // Node 41
  {3, 9},      // Node 42
  {5, 8},      // Node 43
  {1, 8},      // Node 44
  {4, 9},      // Node 45
  {4, 2},      // Node 46
  {6, 10},      // Node 47
  {1, 0},      // Node 48
  {3, 0},      // Node 49
  {7, 5},      // Node 50
  {4, 7},      // Node 51
  {1, 6},      // Node 52
  {7, 6},      // Node 53
  {4, 6},      // Node 54
  {2, 2},      // Node 55
  {6, 3},      // Node 56
  {3, 4},      // Node 57
  {7, 0},      // Node 58
  {4, 0},      // Node 59
  {3, 10},      // Node 60
  {0, 8},      // Node 61
  {0, 10},      // Node 62
  {0, 1},      // Node 63
  {4, 10},      // Node 64
  {0, 7},      // Node 65
  {7, 4},      // Node 66
  {3, 5},      // Node 67
  {1, 9},      // Node 68
  {7, 7},      // Node 69
  {6, 6},      // Node 70
  {2, 5},      // Node 71
  {1, 5},      // Node 72
  {3, 2},      // Node 73
  {6, 8},      // Node 74
  {2, 0},      // Node 75
  {1, 1},      // Node 76
  {6, 0},      // Node 77
  {5, 0},      // Node 78
  {2, 9},      // Node 79
  {0, 4},      // Node 80
  {5, 4},      // Node 81
  {6, 5},      // Node 82
  {0, 6},      // Node 83
  {7, 1},      // Node 84
  {0, 5}      // Node 85
  };

const prog_int8_t mappingArray_P[86][2] PROGMEM = {
  {7, 2},      // Node 0
  {2, 8},      // Node 1
  {5, 2},      // Node 2
  {2, 10},      // Node 3
  {5, 3},      // Node 4
  {1, 3},      // Node 5
  {4, 4},      // Node 6
  {2, 3},      // Node 7
  {3, 7},      // Node 8
  {4, 8},      // Node 9
  {0, 9},      // Node 10
  {6, 2},      // Node 11
  {5, 10},      // Node 12
  {3, 8},      // Node 13
  {4, 1},      // Node 14
  {3, 3},      // Node 15
  {5, 1},      // Node 16
  {1, 7},      // Node 17
  {1, 4},      // Node 18
  {2, 4},      // Node 19
  {3, 6},      // Node 20
  {5, 7},      // Node 21
  {2, 6},      // Node 22
  {5, 9},      // Node 23
  {4, 5},      // Node 24
  {5, 5},      // Node 25
  {7, 10},      // Node 26
  {1, 10},      // Node 27
  {7, 8},      // Node 28
  {7, 3},      // Node 29
  {3, 1},      // Node 30
  {7, 9},      // Node 31
  {0, 0},      // Node 32
  {6, 1},      // Node 33
  {6, 4},      // Node 34
  {0, 3},      // Node 35
  {6, 7},      // Node 36
  {2, 7},      // Node 37
  {5, 6},      // Node 38
  {2, 1},      // Node 39
  {4, 3},      // Node 40
  {6, 9},      // Node 41
  {3, 9},      // Node 42
  {5, 8},      // Node 43
  {1, 8},      // Node 44
  {4, 9},      // Node 45
  {4, 2},      // Node 46
  {6, 10},      // Node 47
  {1, 0},      // Node 48
  {3, 0},      // Node 49
  {7, 5},      // Node 50
  {4, 7},      // Node 51
  {1, 6},      // Node 52
  {7, 6},      // Node 53
  {4, 6},      // Node 54
  {2, 2},      // Node 55
  {6, 3},      // Node 56
  {3, 4},      // Node 57
  {7, 0},      // Node 58
  {4, 0},      // Node 59
  {3, 10},      // Node 60
  {0, 8},      // Node 61
  {0, 10},      // Node 62
  {0, 1},      // Node 63
  {4, 10},      // Node 64
  {0, 7},      // Node 65
  {7, 4},      // Node 66
  {3, 5},      // Node 67
  {1, 9},      // Node 68
  {7, 7},      // Node 69
  {6, 6},      // Node 70
  {2, 5},      // Node 71
  {1, 5},      // Node 72
  {3, 2},      // Node 73
  {6, 8},      // Node 74
  {2, 0},      // Node 75
  {1, 1},      // Node 76
  {6, 0},      // Node 77
  {5, 0},      // Node 78
  {2, 9},      // Node 79
  {0, 4},      // Node 80
  {5, 4},      // Node 81
  {6, 5},      // Node 82
  {0, 6},      // Node 83
  {7, 1},      // Node 84
  {0, 5}      // Node 85
  };

// All the node numbers, so the last number of each array is negative to mark the end of the array
const prog_int8_t aRing0[] PROGMEM = {0, -1};
const prog_int8_t aRing1[] PROGMEM = {1 , 2 , 3 , 4 , 5, -1};
const prog_int8_t aRing2[] PROGMEM = {6 , 7 , 8 , 9 , 10, 11, 12, 13, 14, 15, -1};
const prog_int8_t aRing3[] PROGMEM = {16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, -1};
const prog_int8_t aRing4[] PROGMEM = {31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, -1};
const prog_int8_t aRing5[] PROGMEM = {46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, -1};
const prog_int8_t aRing6[] PROGMEM = {61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, -1};
const prog_int8_t aRing7[] PROGMEM = {76, 77, 78, 79, 80, 81, 82, 83, 84, 85, -1};
const prog_int8_t aRing8[] PROGMEM = {-1}; // Dummy for when more nodes are added
const prog_int8_t aRing9[] PROGMEM = {-1}; // Dummy for when more nodes are added
  
const prog_int8_t* aRingPtr[] PROGMEM = {aRing0, aRing1, aRing2, aRing3, aRing4, aRing5, aRing6, aRing7, aRing8, aRing9}; 

volatile byte dataArray[11];  // Essentially the main frame buffer  

void setup()                    
{
  Serial.begin(9600);
}

void loop()                    
{
  for(int i = 0; i <= 9; i++){
    groupOn_P((int8_t*)pgm_read_byte(&aRingPtr[i]));
    delay(1);
    printArray();
    clearArray();
  }
}

void printArray(){
  for(int i = 0; i < 11; i++){
    Serial.print(dataArray[i], BIN);
    Serial.print(", ");
  }
  Serial.println();
}  

void clearArray(){
  for(int i = 0; i < 11; i++){
    dataArray[i] = 0;
  }
}

void groupOn_P(int8_t group[]){
  int index = 0;
  int8_t currentElement = pgm_read_byte(&group[index]);
  
  while(currentElement >= 0){
    nodeOn(currentElement);    
    index++;
    currentElement = pgm_read_byte(&group[index]);
  }
}

/* Why is it that commenting this out produces different output IF I NEVER CALL IT IN THE MAIN LOOP!?
void nodeOn_P(int8_t nodeNum){
  int8_t registerVal = pgm_read_byte(&mappingArray_P[nodeNum][1]);
  int8_t value = pgm_read_byte(&mappingArray_P[nodeNum][0]);

  bitWrite(dataArray[registerVal], value, HIGH);
}
*/
void nodeOn(int nodeNum){
  bitWrite(dataArray[mappingArray[nodeNum][1]], mappingArray[nodeNum][0], HIGH);
}
5

Does either run of code give you the numbers you expect?

groupOn_P((int8_t*)pgm_read_byte(&aRingPtr[i]));

this looks wrong; int8_t* is longer than a single byte, so you'll probably need pgm_read_word() at least, and maybe pgm_read_dword(), but I'm not that's sufficient to show all the strangeness you're seeing.

I'd stick additoinal debugging in there to make sure you first pgm_reads return the expected values, before doing another indirection on progam space to get additional values.

6

Actually, if I do run it with that function commented out I get the expected output. I thought that int8_t used 8 bits which was what the 8 stood for, (something like 0-127 with the last bit indicating the sign). I thought int16_t and int where both the same in terms of them both using 2 bytes to store a given number. Is there a reference that indicates how much memory each datatype uses? I looked on google for a while and couldn't find anything definite. I could use a byte as my datatype for that data as I'm never going to go above the number 97. I'll put some additional debugging code in to see if I narrow it down somewhat.

EDIT: I did find a few sites that indicated that int_8 should be 8 bits, ( -128 through +127 ), so I think I'm doing that right with doing the pgm_read_byte() function. Of course, with the way this is working right now, I'm beginning to think I may be in a mental asylum contemplating my navel...

EDIT2: I narrowed it down to the following lines:

  int8_t registerVal = pgm_read_byte(&mappingArray_P[nodeNum][1]);
  int8_t value = pgm_read_byte(&mappingArray_P[nodeNum][0]);

If I take either of those two lines and put it in my setup function after replacing nodeNum with a number like:

  int8_t registerVal = pgm_read_byte(&mappingArray_P[2][1]);

...then it starts outputting wierdness. So it's got to be something with how I'm accessing a 2d array within flash memory. I'm on the right track now, so I should be able to solve it soon. Thanks for helping me think it through.

7

int8_t* is longer than a single byte

I thought that int8_t used 8 bits which was what the 8 stood for

Yes, but int8_t* is a pointer to a single byte, and the pointer will be 16 bits...

8

Using pgm_read_word() fixed it. I think I understand why that call to groupOn() was incorrect. See if I'm understanding correctly. groupOn() expects an array of type int8_t, and since an array is nothing more than a pointer to the first int8_t of many, what it is really accepting is a 2 byte address. Since my call to that function got the address by using only one byte, I was only getting half the address. So what was probably happening was that it worked sometimes because using just the first half of the address still got me the info I needed, but when I put other items into program space, it shifted my array into a different address space that no longer could be addressed by a single byte. Is that about right?

9

It's a very good description and I believe you've hit the nail on the head!

10

I just wanted to give a hearty thanks to westfw. Without your help it would have been exceptionally painful to find that issue. Everything has been working as expected, (For now!), and I just wanted say you're awesome. Hope you're having a great weekend.

-tsg