Question on memory usage when assigning values to an array

Hey all,

So I'm working on a project to try and drive a grid of RGB LEDs (using an Arduino Uno R3) so they can display various patterns when I press a button. I'm finding myself reeeeeeally crunched on dynamic memory, so I've been trying a bunch of stuff to get it down (compression algorithm, variable type reduction, yanking non-essential code, etc).

Just this morning though I noticed something weird. Near the end of my code I have a section where I move a set of values for assigning the RGB values of the LEDs from one array that updates frequently, to another array designed to hold those values until they need to change. Well that operation appears to be taking up around half of my dynamic memory.

Something is going on here that I clearly don't understand, so I was wondering if anyone else knows what's happening? Also, any thoughts on how to reduce memory usage would be very welcome. I've got around 12 patterns that I was hoping to load and right now I'm maxed out at around 6.

Screenshots of what I'm seeing and a copy of the code are attached.

Thanks very much

LEDStripsVer6p4.ino (4.9 KB)

Hi. Please do not post images of the ide, just cut & paste the important text into your post between code tags. Also post your code between code tags instead of attaching the .ino file, if you want help quickly from a range of forum members. Most forum members use phones and tablets most of the time and will not be able to read a .ino file until they fire up their PC/laptop, by which time they may have forgotten about your question.


I can guess why there is such a large difference in memory usage. The Optimiser, which is a part of the C compiler, is very clever. I suspect it knows, when you comment out those lines, that the arrays are set but never used, so it removes them completely. I think if you switch on the detailed compiler warnings, it will tell you that this is what it has done.

Hey there, apologies for the formatting of my question, I'll be sure to post correctly going forward. I appreciate you taking the time to let me know.

I'll check the the detailed compiler when I have a chance to sit down with the script again and see if what you described is what's happening. Thanks for the explanation!

PaulRB:
I can guess why there is such a large difference in memory usage. The Optimiser, which is a part of the C compiler, is very clever. I suspect it knows, when you comment out those lines, that the arrays are set but never used, so it removes them completely. I think if you switch on the detailed compiler warnings, it will tell you that this is what it has done.

This appears to be what is happening, the leds[] array alone takes 972 bytes of dynamic memory (324 pixels x 3 bytes per pixel), leaving only 107 bytes being used for other variables when you had the lines commented out.
If the pattern arrays are never going to change, you can store them in program memory, see the following references on PROGMEM:

Arduino Reference page for PROGMEM

Putting constant data into program memory (PROGMEM)

so I really love the program memory idea. If I can take all these arrays out of dynamic memory then that solves the issue.

I've been trying to get something working this morning, but I'm having a real bear of a time with it. Storing all the arrays with PROGMEM was pretty straight forward and showed a substantial memory savings, but trying to access them afterward threw me for a loop.

I started with the pgm_read_word_near command, but couldn't figure out how to get it to work with the array structure I'd set up. Then I went out and grabbed the PGMWrap library to see if I could get that to work, but after setting everything up it:

  1. Doesn't appear to have had any impact on memory usage over not using it
  2. The script no longer works when trying to set the led patterns

Any thoughts on what might be happening or how to structure this more effectively?

Thank you

#include <PGMWrap.h>
#include <FastLED.h>
#define LED_PIN 7
#define NUM_LEDS 324
CRGB leds[NUM_LEDS];
const short buttonPin = 2; 
const short buttonPin2 = 4; 
short buttonState = 0;
short buttonlatch = 0; 
short patternValue = 0;
short buttonState2 = 0;
short buttonlatch2 = 0; 
short patternIndex = 0;
short ledIndex,ledColor;
short currentColor = 0;
short colors[6][3] = {{0,0,0},{1,0,0},{0,1,0},{0,0,1},{1,1,1},{1,1,0}};
//short pattern0[]  = {10003,10211,10253,10451,10483,10501,10523,10551,10573,10581,10603,10621,10653,10771,10803,10831,10873,10881,10893,10901,10983,11021,11053,11091,11163,11181,11213,11221,11293,11321,11353,11371,11403,11411,11433,11441,11463,11481,11503,11531,11553,11591,11663,11691,11723,11741,11763,11781,11803,11811,11833,11841,11903,11931,12043,12071,12113,12121,12153,12191,12253,12281,12303,12321,12343,12351,12403,12441,12473,12591,12623,12671,12693,12721,12743,12761,12793,12991,13033};
//short pattern1[]  = {10005,10233,10315,10403,10505,10573,10695,10773,10845,10863,10875,10973,11055,11133,11195,11293,11415,11453,11575,11673,11795,11833,11955,12053,12115,12193,12275,12373,12385,12403,12475,12553,12675,12743,12845,12933,13015};
//short pattern2[]  = {10001,10604,10661,10725,10731,10764,10785,10794,10861,10934,11005,11024,11051,11065,11071,11095,11114,11145,11164,11195,11204,11241,11284,11315,11334,11355,11384,11405,11421,11464,11475,11494,11505,11544,11555,11574,11601,11644,11665,11714,11735,11764,11781,11824,11855,11874,11905,11924,11935,11954,11961,12005,12024,12045,12064,12095,12104,12141,12194,12275,12284,12315,12331,12355,12361,12384,12481,12584,12641,12695,12701};
//short pattern3[]  = {10002,10604,10662,10764,10862,10934,10942,10954,11032,11044,11052,11104,11122,11134,11142,11204,11212,11224,11242,11284,11302,11334,11372,11404,11422,11464,11482,11504,11562,11584,11602,11644,11662,11684,11742,11764,11782,11824,11842,11874,11912,11944,11962,12004,12022,12034,12042,12104,12112,12124,12142,12194,12202,12214,12292,12304,12312,12384,12482,12584,12642};
//short pattern4[]  = {10001,10105,10121,10245,10271,10455,10460,10475,10481,10605,10620,10635,10641,10785,10800,10815,10841,10975,11000,11025,11031,11125,11140,11165,11181,11335,11350,11365,11381,11525,11530,11545,11571,11675,11700,11715,11721,11851,11865,11880,11895,11911,12065,12080,12105,12121,12215,12230,12245,12271,12405,12430,12445,12461,12605,12610,12625,12641,12765,12770,12785,12791,12975,13001,13125,13141};
//short pattern5[]  = {10003,10201,10223,10251,10273,10291,10313,10371,10403,10421,10443,10461,10483,10511,10533,10561,10583,10601,10613,10641,10653,10681,10693,10701,10713,10751,10773,10781,10793,10821,10833,10851,10873,10931,11053,11101,11243,11271,11433,11451,11613,11631,11793,11811,11973,12001,12143,12191,12313,12371,12393,12411,12423,12451,12463,12471,12493,12531,12543,12551,12563,12591,12603,12631,12643,12661,12683,12711,12733,12761,12783,12801,12823,12841,12873,12931,12953,12971,12993,13021,13043};
//short pattern6[]  = {10000,10050,10090};
//short pattern7[]  = {10001,10050,10091};
//short pattern8[]  = {10002,10050,10092};
//short pattern9[]  = {10003,10050,10093};
//short pattern10[] = {10004,10050,10094};
//short pattern11[] = {10005,10050,10095};

int16_p pattern0[]  = {10003,10211,10253,10451,10483,10501,10523,10551,10573,10581,10603,10621,10653,10771,10803,10831,10873,10881,10893,10901,10983,11021,11053,11091,11163,11181,11213,11221,11293,11321,11353,11371,11403,11411,11433,11441,11463,11481,11503,11531,11553,11591,11663,11691,11723,11741,11763,11781,11803,11811,11833,11841,11903,11931,12043,12071,12113,12121,12153,12191,12253,12281,12303,12321,12343,12351,12403,12441,12473,12591,12623,12671,12693,12721,12743,12761,12793,12991,13033};
int16_p pattern1[]  = {10005,10233,10315,10403,10505,10573,10695,10773,10845,10863,10875,10973,11055,11133,11195,11293,11415,11453,11575,11673,11795,11833,11955,12053,12115,12193,12275,12373,12385,12403,12475,12553,12675,12743,12845,12933,13015};
int16_p pattern2[]  = {10001,10604,10661,10725,10731,10764,10785,10794,10861,10934,11005,11024,11051,11065,11071,11095,11114,11145,11164,11195,11204,11241,11284,11315,11334,11355,11384,11405,11421,11464,11475,11494,11505,11544,11555,11574,11601,11644,11665,11714,11735,11764,11781,11824,11855,11874,11905,11924,11935,11954,11961,12005,12024,12045,12064,12095,12104,12141,12194,12275,12284,12315,12331,12355,12361,12384,12481,12584,12641,12695,12701};
int16_p pattern3[]  = {10002,10604,10662,10764,10862,10934,10942,10954,11032,11044,11052,11104,11122,11134,11142,11204,11212,11224,11242,11284,11302,11334,11372,11404,11422,11464,11482,11504,11562,11584,11602,11644,11662,11684,11742,11764,11782,11824,11842,11874,11912,11944,11962,12004,12022,12034,12042,12104,12112,12124,12142,12194,12202,12214,12292,12304,12312,12384,12482,12584,12642};
int16_p pattern4[]  = {10001,10105,10121,10245,10271,10455,10460,10475,10481,10605,10620,10635,10641,10785,10800,10815,10841,10975,11000,11025,11031,11125,11140,11165,11181,11335,11350,11365,11381,11525,11530,11545,11571,11675,11700,11715,11721,11851,11865,11880,11895,11911,12065,12080,12105,12121,12215,12230,12245,12271,12405,12430,12445,12461,12605,12610,12625,12641,12765,12770,12785,12791,12975,13001,13125,13141};
int16_p pattern5[]  = {10003,10201,10223,10251,10273,10291,10313,10371,10403,10421,10443,10461,10483,10511,10533,10561,10583,10601,10613,10641,10653,10681,10693,10701,10713,10751,10773,10781,10793,10821,10833,10851,10873,10931,11053,11101,11243,11271,11433,11451,11613,11631,11793,11811,11973,12001,12143,12191,12313,12371,12393,12411,12423,12451,12463,12471,12493,12531,12543,12551,12563,12591,12603,12631,12643,12661,12683,12711,12733,12761,12783,12801,12823,12841,12873,12931,12953,12971,12993,13021,13043};
int16_p pattern6[]  = {10000,10050,10090};
int16_p pattern7[]  = {10001,10050,10091};
int16_p pattern8[]  = {10002,10050,10092};
int16_p pattern9[]  = {10003,10050,10093};
int16_p pattern10[] = {10004,10050,10094};
int16_p pattern11[] = {10005,10050,10095};


int16_p* patterns[12] = {pattern0,pattern1,pattern2,pattern3,pattern4,pattern5,pattern6,pattern7,pattern8,pattern9,pattern10,pattern11};

void setup() {
  // put your setup code here, to run once:
  //Serial.begin(9600);

FastLED.addLeds<WS2812, LED_PIN, GRB>(leds, NUM_LEDS);
pinMode(buttonPin, INPUT);
pinMode(buttonPin2, INPUT);
}
void loop() {
  // put your main code here, to run repeatedly:
      //Serial.print(currentColor);
      //Serial.print(patternValue);
      //delay(100);
      //Serial.print("hello");
buttonState = digitalRead(buttonPin);
buttonState2 = digitalRead(buttonPin2);
  if (buttonState == HIGH && buttonlatch == 0) {
    // select pattern and latch button
    if (patternValue < 11 ){
      patternValue++;
      buttonlatch = 1;
    }
    else{
      patternValue = 0;
      buttonlatch = 1;
      }
  } 
  else if (buttonState2 == HIGH && buttonlatch2 == 0) {
    // select pattern and latch button
    if (patternValue > 0 ){
      patternValue--;;
      buttonlatch2 = 1;
    }
    else{
      patternValue = 11;
      buttonlatch2 = 1;
      }
  }
  else if (buttonState == LOW && buttonlatch == 1){
    // unlatch button:
    buttonlatch = 0;
    }
  else if (buttonState2 == LOW && buttonlatch2 == 1){
    // unlatch button:
    buttonlatch2 = 0;
    }
  else {
  //do nothing
  }
  
  for (short i = 0; i < 324;i++)
  {
    ledColor = patterns[patternValue][patternIndex]%10;
    ledIndex = (patterns[patternValue][patternIndex] - 10000 - ledColor) / 10;
    //Serial.print(ledColor);
    //Serial.print(" ");
    //Serial.print(ledIndex);
    //Serial.print(" "); 
    //Serial.print(i);
    //Serial.print(" ");
    //delay(1000);        
    if(i == ledIndex)
    {
      currentColor = ledColor;
      patternIndex++;

    }
    leds[i] = CRGB(colors[currentColor][0],colors[currentColor][1],colors[currentColor][2]);
  }
 patternIndex = 0;
  
FastLED.show();
delay(100);
}

You have a rather unusual structure for your pattern arrays, because you are indexing into an array of pointers to the pattern arrays, while at the same time indexing into the individual pattern array. I'm a bit surprised that actually works, although I didn't test your original code to see if it functioned correctly. With PROGMEM, accessing the value in the pattern array is a two step process, first retrieve the pointer to the appropriate pattern array, then retrieve the element within that particular array. I'm not familiar with the PGMWrap library, so not sure if it can be done with that, the following code uses pgm_read_word to retrieve the data:

#include <FastLED.h>
#define LED_PIN 7
#define NUM_LEDS 324
CRGB leds[NUM_LEDS];
const short buttonPin = 2;
const short buttonPin2 = 4;
short buttonState = 0;
short buttonlatch = 0;
short patternValue = 0;
short buttonState2 = 0;
short buttonlatch2 = 0;
short patternIndex = 0;
short ledIndex, ledColor;
short currentColor[3] = {0, 0, 0};
short colors[6][3] = {{0, 0, 0}, {1, 0, 0}, {0, 1, 0}, {0, 0, 1}, {1, 1, 1}, {1, 1, 0}};
const short pattern0[] PROGMEM = {10003, 10211, 10253, 10451, 10483, 10501, 10523, 10551, 10573, 10581, 10603, 10621, 10653, 10771, 10803, 10831, 10873, 10881, 10893, 10901, 10983, 11021, 11053, 11091, 11163, 11181, 11213, 11221, 11293, 11321, 11353, 11371, 11403, 11411, 11433, 11441, 11463, 11481, 11503, 11531, 11553, 11591, 11663, 11691, 11723, 11741, 11763, 11781, 11803, 11811, 11833, 11841, 11903, 11931, 12043, 12071, 12113, 12121, 12153, 12191, 12253, 12281, 12303, 12321, 12343, 12351, 12403, 12441, 12473, 12591, 12623, 12671, 12693, 12721, 12743, 12761, 12793, 12991, 13033};
const short pattern1[] PROGMEM = {10005, 10233, 10315, 10403, 10505, 10573, 10695, 10773, 10845, 10863, 10875, 10973, 11055, 11133, 11195, 11293, 11415, 11453, 11575, 11673, 11795, 11833, 11955, 12053, 12115, 12193, 12275, 12373, 12385, 12403, 12475, 12553, 12675, 12743, 12845, 12933, 13015};
const short pattern2[] PROGMEM = {10001, 10604, 10661, 10725, 10731, 10764, 10785, 10794, 10861, 10934, 11005, 11024, 11051, 11065, 11071, 11095, 11114, 11145, 11164, 11195, 11204, 11241, 11284, 11315, 11334, 11355, 11384, 11405, 11421, 11464, 11475, 11494, 11505, 11544, 11555, 11574, 11601, 11644, 11665, 11714, 11735, 11764, 11781, 11824, 11855, 11874, 11905, 11924, 11935, 11954, 11961, 12005, 12024, 12045, 12064, 12095, 12104, 12141, 12194, 12275, 12284, 12315, 12331, 12355, 12361, 12384, 12481, 12584, 12641, 12695, 12701};
const short pattern3[] PROGMEM = {10002, 10604, 10662, 10764, 10862, 10934, 10942, 10954, 11032, 11044, 11052, 11104, 11122, 11134, 11142, 11204, 11212, 11224, 11242, 11284, 11302, 11334, 11372, 11404, 11422, 11464, 11482, 11504, 11562, 11584, 11602, 11644, 11662, 11684, 11742, 11764, 11782, 11824, 11842, 11874, 11912, 11944, 11962, 12004, 12022, 12034, 12042, 12104, 12112, 12124, 12142, 12194, 12202, 12214, 12292, 12304, 12312, 12384, 12482, 12584, 12642};
const short pattern4[] PROGMEM = {10001, 10105, 10121, 10245, 10271, 10455, 10460, 10475, 10481, 10605, 10620, 10635, 10641, 10785, 10800, 10815, 10841, 10975, 11000, 11025, 11031, 11125, 11140, 11165, 11181, 11335, 11350, 11365, 11381, 11525, 11530, 11545, 11571, 11675, 11700, 11715, 11721, 11851, 11865, 11880, 11895, 11911, 12065, 12080, 12105, 12121, 12215, 12230, 12245, 12271, 12405, 12430, 12445, 12461, 12605, 12610, 12625, 12641, 12765, 12770, 12785, 12791, 12975, 13001, 13125, 13141};
const short pattern5[] PROGMEM = {10003, 10201, 10223, 10251, 10273, 10291, 10313, 10371, 10403, 10421, 10443, 10461, 10483, 10511, 10533, 10561, 10583, 10601, 10613, 10641, 10653, 10681, 10693, 10701, 10713, 10751, 10773, 10781, 10793, 10821, 10833, 10851, 10873, 10931, 11053, 11101, 11243, 11271, 11433, 11451, 11613, 11631, 11793, 11811, 11973, 12001, 12143, 12191, 12313, 12371, 12393, 12411, 12423, 12451, 12463, 12471, 12493, 12531, 12543, 12551, 12563, 12591, 12603, 12631, 12643, 12661, 12683, 12711, 12733, 12761, 12783, 12801, 12823, 12841, 12873, 12931, 12953, 12971, 12993, 13021, 13043};
const short pattern6[] PROGMEM = {10000, 10050, 10090};
const short pattern7[] PROGMEM = {10001, 10050, 10091};
const short pattern8[] PROGMEM = {10002, 10050, 10092};
const short pattern9[] PROGMEM = {10003, 10050, 10093};
const short pattern10[] PROGMEM = {10004, 10050, 10094};
const short pattern11[] PROGMEM = {10005, 10050, 10095};
//size_t endpoint;

const short* const patterns[12] PROGMEM = {pattern0, pattern1, pattern2, pattern3, pattern4, pattern5, pattern6, pattern7, pattern8, pattern9, pattern10, pattern11};

//char buf[64]; //text buffer for sprintf command

void setup() {
  // put your setup code here, to run once:
  //Serial.begin(9600);
  FastLED.addLeds<WS2812, LED_PIN, GRB>(leds, NUM_LEDS);
  pinMode(buttonPin, INPUT);
  pinMode(buttonPin2, INPUT);
}
void loop() {
  // put your main code here, to run repeatedly:

  buttonState = digitalRead(buttonPin);
  buttonState2 = digitalRead(buttonPin2);
  if (buttonState == HIGH && buttonlatch == 0) {
    // select pattern and latch button
    if (patternValue < 11 ) {
      patternValue++;
      buttonlatch = 1;
    }
    else {
      patternValue = 0;
      buttonlatch = 1;
    }
  }
  else if (buttonState2 == HIGH && buttonlatch2 == 0) {
    // select pattern and latch button
    if (patternValue > 0 ) {
      patternValue--;;
      buttonlatch2 = 1;
    }
    else {
      patternValue = 11;
      buttonlatch2 = 1;
    }
  }
  else if (buttonState == LOW && buttonlatch == 1) {
    // unlatch button:
    buttonlatch = 0;
  }
  else if (buttonState2 == LOW && buttonlatch2 == 1) {
    // unlatch button:
    buttonlatch2 = 0;
  }
  else {
    //do nothing
  }

  for (short i = 0; i < 324; i++)
  {
    //pgm_read_word(&patterns[patternValue]) retrieves the pointer to the pattern array
    //patternIndex * sizeof(short) adds the offset for the array element pointed to by patternValue
    ledColor = pgm_read_word((pgm_read_word(&patterns[patternValue]) + patternIndex * sizeof(short))) % 10;
    ledIndex = (pgm_read_word((pgm_read_word(&patterns[patternValue]) + patternIndex * sizeof(short))) - 10000 - ledColor) / 10;
    //sprintf(buf, "patternIndex=%3d pattern=%5d i=%3d ledColor=%d ledIndex=%3d", patternIndex, pgm_read_word(pgm_read_word(&patterns[patternValue]) + patternIndex * sizeof(short)), i, ledColor, ledIndex);
    //Serial.println(buf);

    if (i == ledIndex)
    {
      //currentColor = colors[patternValue][ledColor];
      currentColor[0] = colors[ledColor][0];
      currentColor[1] = colors[ledColor][1];
      currentColor[2] = colors[ledColor][2];
      patternIndex++;
    }
    leds[i] = CRGB(currentColor[0], currentColor[1], currentColor[2]);
  }
  patternIndex = 0;

  FastLED.show();
  delay(100);
}

Just got a chance to sit down and try this. It works perfectly! I'm bouncing off the walls a little bit haha. This was the last piece I needed to pull the whole project together. Thank you so much for the help!