[SOLVED] VERY weird memory usage problem

OK, I have a sketch that uses libraries etc. And I Have a library that stores a rather large array into flash, but the size of this array has no bearing on the issue i’m presenting, cause I have cut it down to 1 element and I get the exact same results …

So … here’s whats odd this problem … when I include the one line that simply references an index number of the array, my compiled memory consumption goes to over 100% but if I comment out that line, then it goes back to 70-something percent … It’s best if I show by example what Im talking about.

const unsigned int ciscoCmds[][] PROGMEM ={{3250,3350,800,2550,800,2550,800,850,800,2550,800,2500,850,900,750,2550,800,2550,750,900,800,2550,750,2550,800,900,750,900,800,2550,750,900,800,900,750,2500,850,900,750,900,750,2600,750,900,750,900,750},
{3300,3350,800,2500,800,2550,800,900,750,2600,750,2600,750,2550,800,2550,750,2600,750,900,750,2600,750,2600,750,900,750,900,750,2600,750,900,750,850,850,850,800,850,800,900,750,2550,800,850,800,900,800},
{3250,3350,800,2550,800,2600,750,900,750,2500,850,2600,750,850,800,2600,750,2550,750,950,750,800,850,900,750,900,750,900,750,2600,750,850,800,950,750,2550,750,850,850,900,750,2550,800,2550,750,2550,800}};

const unsigned long tvcmds[] = {0xe0e0e01f,0xe0e0d02f,0xe0e0f00f,0xe0e058a7,0xe0e0d22d,0xe0e0f807,0xe0e09e61,0xe0e0f906,0xe0e01ae5,0xe0e0b44b,0xe0e036c9,0xe0e028d7,0xe0e0a857,0xe0e06897,0xe0e0cf30,0xe0e0a25d,0xe0e052ad,0xe0e012ed,0xe0e0629d,0xe0e0807f,0xe0e0a659,0xe0e046b9,0xe0e006f9,0xe0e08679,0xe0e016e9};


   void send_command(int index, int device) {
        if (device == CISCO) tv.sendRaw_p(ciscoCmds[index],sizeof(ciscoCmds[index])/sizeof(ciscoCmds[index][0]),hz); //THIS IS THE LINE I COMMENT OUT
        if (device == TV) tv.sendSAMSUNG(tvcmds[index],32);
	}
	
	
	void  IRsend::sendRaw_p (const unsigned int buf[],  unsigned int len,  unsigned int hz)
{
    enableIROut(hz);
    for (unsigned int i = 0;  i < len;  i++) {
        if (i & 1)  space(pgm_read_word_near(&(buf[i])));
        else        mark (pgm_read_word_near(&(buf[i])));
    }
    space(0);  // Always end with the LED off
}

And HERE is the output from the compiler if I comment that one line out:

Sketch uses 23984 bytes (78%) of program storage space. Maximum is 30720 bytes.
Global variables use 1206 bytes (58%) of dynamic memory, leaving 842 bytes for local variables. Maximum is 2048 bytes.

AND when I uncomment that line and try to compile, this is what I get:

Sketch uses 33136 bytes (107%) of program storage space. Maximum is 30720 bytes.
Global variables use 1206 bytes (58%) of dynamic memory, leaving 842 bytes for local variables. Maximum is 2048 bytes.
Sketch too big; see http://www.arduino.cc/en/Guide/Troubleshooting#size for tips on reducing it.
Error compiling for board Arduino Nano.

What in the world is going on here? Anyone?

Thank you,

Mike

const unsigned int ciscoCmds PROGMEM = = {{3250,

What's the purpose of the double '='?

Also:

void IRsend::sendRaw_p (const unsigned int buf PROGMEM, unsigned int len, unsigned int hz)

Pretty sure that 'PROGMEM' does nothing.

Also:

if (device == CISCO) tv.sendRaw_p(ciscoCmds[index], sizeof(ciscoCmds[index]) / sizeof(ciscoCmds[index][0]), hz);

I'm pretty sure that's wrong. 'sizeof' is a compiler directive. But, the value of index is not known at compile-time.

OK, so I narrowed the problem down even further... take a look:

This compiles with only 78% memory usage:

private:
    void send_command(int index, int device) {
        if (device == CISCO) {
           // int elements = sizeof(ciscoCmds[index])/sizeof(ciscoCmds[index][0]);
            int elements = 0;
            tv.sendRaw_p(ciscoCmds[index],elements,hz);
        }
        if (device == TV) tv.sendSAMSUNG(tvcmds[index],32);

AND THIS compiles with 107% memory usage

private:
    void send_command(int index, int device) {
        if (device == CISCO) {
            int elements = sizeof(ciscoCmds[index])/sizeof(ciscoCmds[index][0]);
            tv.sendRaw_p(ciscoCmds[index],elements,hz);
        }
        if (device == TV) tv.sendSAMSUNG(tvcmds[index],32);
}

So the problem is in the way that I extract the number of elements in the array that I am trying to pass into the IR library ....

sizeof(ciscoCmds[index]) / sizeof(ciscoCmds[index][0])

any multidimensional array experts in here who could chime in?

gfvalvo:
Also:Pretty sure that 'PROGMEM' does nothing.

You mean ... other than storing my array into flash memory so it does not consume any memory that the program itself uses ....

EasyGoing1:
any multidimensional array experts in here who could chime in?

Reply #3. Why bother asking for help if you’re going to ignore the responses?

EasyGoing1:
You mean … other than storing my array into flash memory so it does not consume any memory that the program itself uses …

Not in the function definition it isn’t

Also:

if (device == TV) tv.sendSAMSUNG(tvcmds[index], 32);

Do you really want to send 32 bytes (or elements?) from position ‘index’? What if index points to the last element of your array?

EDIT:
Maybe it’s 32 bits? Never mind.

gfvalvo:
Reply #3. Why bother asking for help if you're going to ignore the responses?

I have not ignored any relevant responses.

gfvalvo:
Not in the function definition it isn't

But ... I'm NOT declaring the array in any function definition ...

gfvalvo:
Do you really want to send 32 bytes (or elements?) from position 'index'? What if index points to the last element of your array?

EDIT:
Maybe it's 32 bits? Never mind.

That second line is only referencing a simple one dimensional array of type unsigned long ... and each element only contains short hex numbers like this 0xe0e0e01f ... the only reason I left that in the code for this post is because I copied and pasted straight from the library and left nothing out ... because that's what you're supposed to do ... and I've already shown which part of the code is causing my problem ... unless you think there is something of interest that is relevant to my problem in the line you commented on (tv.sendSAMSUNG(tvcmds[index],32);)

gfvalvo:
Also:I'm pretty sure that's wrong. 'sizeof' is a compiler directive. But, the value of index is not known at compile-time.

The reason I didn't respond to this was because while you were giving this response, I was further clarifying my problem and showing that it is definitely related to sizeof ...

But upon further experimentation ... where I replaced all that sizeof stuff with a single integer, it didn't fix the problem.

EasyGoing1:
But ... I'm NOT declaring the array in any function definition ...

Right Here:
void IRsend::sendRaw_p (const unsigned int buf PROGMEM, unsigned int len, unsigned int hz)
^^^^^^^^^
The "PROGMEM" does not belong there.

You also have not answered my question in Reply #1.

Often behaviour like this can be caused by compiler optimisation. GCC is very clever and is often able to determine when variables are not used. When you comment out that line and set elements = 0, the compiler may look in sendRaw_p() and see that when elements==len==0, the loop will never be executed. Since ciscoCmds doesn't appear to be used anywhere for anything with side effects, the compiler just dumps it.

Since you state that the same amount of memory gets used irrespective of the size of ciscoCmds

EasyGoing1:
the size of this array has no bearing on the issue i'm presenting, cause I have cut it down to 1 element and I get the exact same results ...

maybe you are seeing some other issue.

It might be quicker to post all your code for someone else to compile for themselves for a Nano board.

PS: Agree with gfvalvo re PROGMEM in function definition - useless.

gfvalvo:
Right Here:
void IRsend::sendRaw_p (const unsigned int buf PROGMEM, unsigned int len, unsigned int hz)
^^^^^^^^^
The "PROGMEM" does not belong there.

You also have not answered my question in Reply #1.

OH THAT ONE!!! Yes, I agree with you ... I don't think it actually does anything.

arduarn:
Often behaviour like this can be caused by compiler optimisation. GCC is very clever and is often able to determine when variables are not used. When you comment out that line and set elements = 0, the compiler may look in sendRaw_p() and see that when elements==len==0, the loop will never be executed. Since ciscoCmds doesn't appear to be used anywhere for anything with side effects, the compiler just dumps it.

Since you state that the same amount of memory gets used irrespective of the size of ciscoCmds maybe you are seeing some other issue.

It might be quicker to post all your code for someone else to compile for themselves for a Nano board.

PS: Agree with gfvalvo re PROGMEM in function definition - useless.

YES, you are correct ... I've been playing with the code some more and the problem is in the actual declaration of the 2D array. I had forgotten that I cut the number of elements in half last night and I didn't reflect that in the declaration, and in the second dimension, I had it set to 100 when all I needed was 49, so when I declared that array like this, it brought my memory consumption down to 84% which works for me:

const unsigned int ciscoCmds[29][49] PROGMEM =

And I realize that in my original post, I left the dim sizes blank and that was because I had been messing with the code prior to posting here and forgot to fully reverse my changes ... so I apologize to anyone who would have caught that out of the gate.

gfvalvo:
What's the purpose of the double '='?

I assume you mean - why am I declaring the array in 2 dimensions? Because I'm storing infra red remote codes in raw format and I want them all in a single array to simplify the code... referencing a bunch of raw IR codes with a single index number for 50 of them ... vs having a separate line for each IR code ... makes the code REALLY CLEAN and SIMPLE.

Heres the final array for this devices IR codes:

const unsigned int ciscoCmds[29][49]  PROGMEM =
        {{3250,3350,800,2550,800,2550,800,850,800,2550,800,2500,850,900,750,2550,800,2550,750,900,800,2550,750,2550,800,900,750,900,800,2550,750,900,800,900,750,2500,850,900,750,900,750,2600,750,900,750,900,750},
         {3300,3350,800,2500,800,2550,800,900,750,2600,750,2600,750,2550,800,2550,750,2600,750,900,750,2600,750,2600,750,900,750,900,750,2600,750,900,750,850,850,850,800,850,800,900,750,2550,800,850,800,900,800},
         {3250,3350,800,2550,800,2600,750,900,750,2500,850,2600,750,850,800,2600,750,2550,750,950,750,800,850,900,750,900,750,900,750,2600,750,850,800,950,750,2550,750,850,850,900,750,2550,800,2550,750,2550,800},
         {3250,3400,750,2600,750,2550,750,950,750,2600,700,2650,700,950,700,950,750,950,700,2550,800,900,750,950,700,900,750,950,700,2650,700,950,700,950,750,2600,700,2600,750,2550,800,950,700,2650,700,2600,750},
         {3250,3350,800,2600,750,2500,800,900,800,2500,800,2550,800,900,750,900,800,2550,750,2550,800,900,750,900,800,850,800,850,800,2550,800,900,750,900,750,2600,750,2600,750,850,800,850,800,2550,800,2550,800},
         {3250,3400,750,2550,800,2550,800,850,800,2600,750,2500,800,900,800,2550,750,900,800,850,800,900,750,900,750,900,750,950,750,2550,750,900,800,900,750,2550,800,850,800,2500,850,2550,750,2600,750,2600,750},
         {3250,3400,750,2550,800,2550,800,850,800,2550,800,2550,750,2600,750,2600,750,900,750,2600,750,900,750,900,800,900,750,900,750,2600,750,900,750,900,750,900,750,900,800,2550,750,850,850,2500,800,2600,750},
         {3300,3400,750,2600,750,2550,800,850,800,2550,800,2550,800,2550,750,2600,750,900,750,900,800,900,750,900,750,850,800,900,750,2600,750,900,750,900,800,900,750,900,750,2550,800,2600,750,2550,750,2600,750},
         {3250,3400,750,2600,750,2550,800,900,750,2600,750,2550,800,900,750,900,750,2550,800,850,800,900,750,900,750,900,800,850,800,2550,800,900,750,900,750,2600,750,2550,800,850,800,2550,800,2550,750,2550,800},
         {3300,3400,750,2600,750,2600,750,900,750,2600,750,2550,750,2600,750,2600,750,2550,800,2550,750,900,800,2550,750,900,800,900,750,2550,800,850,800,900,750,900,750,900,750,850,850,850,800,2600,750,800,850},
         {3250,3350,800,2550,800,2550,750,950,750,2550,750,2600,750,900,750,900,800,850,800,850,800,2600,750,2550,800,900,750,900,750,2550,800,850,800,900,750,2600,750,2600,750,2550,800,2500,800,900,800,900,750},
         {1150,200,1800,3450,650,2700,700,2650,650,1000,650,2700,650,2700,650,2650,700,850,800,950,700,1050,650,2650,650,2650,700,1000,650,1000,700,2600,750,850,800,850,800,900,800,2550,750,2600,750,2600,750,900,750,950,700},
         {3300,3400,750,900,750,900,750,2600,750,2600,750,2550,800,2550,750,900,800,2550,750,2600,750,2550,800,2600,750,2600,750,2550,750,950,750,850,800,850,800,850,800,2550,800,850,800,850,800,900,800,900,750},
         {3250,3450,700,2650,700,2600,750,950,700,2600,750,2650,700,1000,650,900,750,2650,700,2650,700,900,750,2650,700,950,700,1000,650,2600,750,1000,650,950,700,2600,750,2600,750,900,750,900,750,2650,700,850,800},
         {3250,3350,800,2600,750,2550,800,900,750,2550,800,2550,750,950,750,900,750,2600,750,950,700,2600,750,2500,800,950,750,950,700,2550,800,900,750,950,700,2600,750,2600,750,900,750,2650,700,900,750,900,750},
         {3250,3350,800,2600,700,2650,700,950,700,2600,750,2550,800,2600,750,2550,800,900,750,900,750,2550,800,2600,750,900,750,900,750,2500,850,900,750,900,750,900,750,900,800,2550,750,2600,750,900,750,950,750},
         {3200,3400,750,2650,700,2700,650,800,850,2650,700,2600,750,900,750,900,750,900,750,2650,700,2600,750,900,750,1050,600,950,700,2750,600,1000,650,1000,700,2600,700,2650,700,2650,700,950,700,950,700,2600,750},
         {3250,3350,800,2550,800,2600,700,850,800,2600,750,2600,750,2600,750,900,750,950,700,2600,750,900,750,2600,750,900,750,900,750,2600,750,950,700,950,750,900,750,2600,750,2550,750,950,750,2550,750,950,750},
         {3300,3400,750,2600,750,2500,850,900,750,2600,750,2500,800,2600,750,2600,750,900,750,900,750,2600,750,900,750,950,750,900,750,2550,800,800,850,850,800,900,750,900,750,2600,750,2600,750,900,750,2550,800},
         {3300,3350,800,2600,700,2600,750,950,700,2550,800,2600,750,2600,750,900,750,850,800,950,700,2600,750,950,700,900,800,900,750,2600,750,900,750,950,700,850,800,2600,750,2600,750,2550,800,900,750,2600,750},
         {3250,3400,750,2550,800,2550,750,950,750,2500,800,2550,800,950,700,950,750,900,750,2550,800,850,800,2500,850,900,750,900,750,2600,750,900,750,900,750,2600,750,2600,750,2550,800,900,750,2600,750,800,850},
         {3200,3400,750,2600,750,2600,750,900,750,2600,750,2600,750,900,750,900,750,900,750,850,850,2600,700,950,750,900,750,850,800,2600,750,950,700,950,700,2600,750,2500,850,2600,750,2550,750,850,850,2550,750},
         {3300,3400,750,2550,800,2550,800,900,750,2600,750,2550,750,2600,750,2600,750,2500,850,900,750,2600,750,900,750,900,750,950,700,2600,750,900,750,850,850,900,750,900,750,900,750,2600,750,900,750,2550,800},
         {3300,3400,750,2600,750,2550,800,900,750,2600,750,2600,750,2600,700,850,850,2550,750,950,750,2600,700,850,850,900,750,900,750,2600,750,850,800,850,800,950,700,2600,750,850,800,2600,750,950,700,2650,700},
         {3200,3400,750,2600,750,2600,750,900,750,2600,750,2550,750,850,850,2600,700,850,850,850,800,2550,800,900,750,850,800,900,750,2600,750,950,700,950,750,2550,750,950,700,2650,700,2550,800,900,750,2600,750},
         {3250,3350,800,2600,750,2550,750,950,700,2600,750,2600,750,900,750,950,700,2600,750,950,700,2650,700,850,800,850,850,850,800,2500,850,900,750,950,700,2600,750,2600,750,850,800,2600,750,900,750,2600,750},
         {3300,3350,800,2550,750,2600,750,900,750,2600,750,2600,750,900,750,2600,750,2600,750,900,750,2600,750,850,800,850,800,950,700,2600,750,900,750,950,700,2600,750,900,750,900,800,2600,700,900,800,2550,750},
         {3200,3350,800,2600,750,2600,750,900,750,2600,750,2550,750,900,800,900,750,850,800,2600,750,2550,800,850,800,900,750,900,750,2550,800,900,750,900,750,2600,750,2600,750,2600,750,900,750,900,750,2550,800},
         {3300,3350,800,2500,800,2600,750,900,750,2600,750,2600,750,2550,800,900,750,900,750,2550,800,2550,800,900,750,850,800,900,750,2600,750,900,750,900,750,950,750,2550,750,2550,800,950,700,950,750,2550,750}};

… and I think that the reason my memory usage went down when I excluded references to the array, is because the compiler will ignore the array and would not compile it if there are no references to it.

const unsigned int ciscoCmds PROGMEM = = {{

EasyGoing1:
I assume you mean - why am I declaring the array in 2 dimensions?

I believe he meant why were there two = in your originally posted code?

const unsigned int ciscoCmds[][] PROGMEM = = {{

evanmars:
const unsigned int ciscoCmds PROGMEM = = {{
I believe he meant why were there two = in your originally posted code?

const unsigned int ciscoCmds[][] PROGMEM = = {{

OH ... TYPO - straight up!

.... actually thats not true ... it came from me messing with the code and copying and pasting different things and I didn't look the code over carefully enough before posting it ... but it was totally a mistake ... im not sure that would even compile ... would it?