Force runtime array allocation and initialization with list syntax

Originally here c - Force runtime array allocation and initialization with list syntax - Stack Overflow

I am working on an atmega328 that only has 2K of RAM.

I use a script to create 10bytes arrays from 10 bitmaps that represent 10 digits of my LCD timer. Those byte arrays are stored in my source code but since I only write one digit at time there is no reason to keep all of them in RAM. This way I can use bigger digits and consume less RAM!

This is what I am trying to do:

    void load_digit_ba(uint8_t digit, uint8_t digit_ba[]) {

      // from 0 to 9 I copy the current digit (byte array) into digit_ba
      if (digit == 0) {
        uint8_t digit_ba_tmp[10] = { 24, 40, 0, 0, 0, 224, 240, 248, 252, 254 };

        memcpy(digit_ba, digit_ba_tmp, 10);
      }
    ...

    }

But it seems that the compiler is statically reserving memory for the array.

Sketch uses 7,310 bytes (23%) of program storage space. Maximum is 30,720 bytes.
Global variables use 1,695 bytes (82%) of dynamic memory, leaving 353 bytes for local variables. Maximum is 2,048 bytes

If I add a cell to the array [11] and just pass 10 value as initializers it will instead allocate the memory at runtime (it seems):

    void load_digit_ba(uint8_t digit, uint8_t digit_ba[]) {

      if (digit == 0) {
        uint8_t digit_ba_tmp[11] = { 24, 40, 0, 0, 0, 224, 240, 248, 252, 254 };

        memcpy(digit_ba, digit_ba_tmp, 10);
      }
    ...

    }

Sketch uses 11,396 bytes (37%) of program storage space. Maximum is 30,720 bytes.
Global variables use 597 bytes (29%) of dynamic memory, leaving 1,451 bytes for local variables. Maximum is 2,048 bytes.

At least this is what the Arduino IDE (AVRDude) says.
Same behavior if I do uint8_t digit_ba_tmp[] letting the compiler figure the length.

Why adding a cell does that and is it clean? Doesn't seem to me.
The assumption here is that since the length of the digit array is fixed for every digit, just changes the content, and I send one digit at time over serial to the display, it makes sense to just load the current digit into ram and then send it. I don't see any heap/stack memory fragmentation issue here, am I right? :frowning:

But it seems that the compiler is statically reserving memory for the array.

Which of the two arrays in that bit of code is "the array"?

If I add a cell to the array [11] and just pass 10 value as initializers it will instead allocate the memory at runtime (it seems):

No. The behavior is the same whether array has 10 elements or 11.

With the trick

What trick? And, what about it?

Why adding a cell does that and is it clean?

Does what?

Perhaps you should be asking over at http://snippets-r-us.com, or posting ALL of your code here.

No. The behavior is the same whether array has 10 elements or 11.

That's what I thought, avrdude is just not smart enough?

Which of the two arrays in that bit of code is "the array"?

The one with literals, digit_ba is local to the calling function, I am probably going to set it static or global since is going to stay there all the time.

What trick? And, what about it?

Does what?

The +1 cell trick, but it seems that I am just tricking avrdude and not changing anything right?

Perhaps you should be asking over at http://snippets-r-us.com, or posting ALL of your code here.

Sure I am probably going to push the whole project on github :slight_smile:

I guess my real question is:

We have 32K of Flash storage for our program to be stored (let's pretend I am not using a bootloader) and just 2K of RAM right?
That means that we are not loading the whole program in RAM because it would not fit.

I guess it will continuously read from the flash and put instruction into RAM as the program advance, or is there another memory space that I am not aware of? Since my assumption is that there isn't another memory space, what I am trying to do is to keep those arrays out of RAM and load them from code one at time when I need them.

P.S. I tried already PROGMEM and it works, but I just want to to learn something today :smiley:

It's a Harvard architecture processor ; no part of the the program ever goes in RAM

Thanks @AWOL.

I read this AVR microcontrollers - Wikipedia very interesting.

Program execution[edit]
Atmel's AVRs have a two stage, single level pipeline design. This means the next machine instruction is fetched as the current one is executing. Most instructions take just one or two clock cycles, making AVRs relatively fast among eight-bit microcontrollers.
The AVR processors were designed with the efficient execution of compiled C code in mind and have several built-in pointers for the task.

That means that the guy here just fetches one instruction at time and we are talking about assembly instruction (right?), well then, this makes me think that I should be able to copy an array from flash (source code) to RAM (statically reserved digit_ba) and not reserve 10122 bytes but just 1122 (digit_ba, the container for the current digit).

I don't see why you have to copy anything from program memory to RAM - why not just leave it in program memory?

Because for example I need to transfer that over serial, and I don't want to rewrite the function since the function takes an array as parameter.

Just for reference: avr-libc: Data in Program Space

You can send a PROGMEM string over serial:

char foo [] PROGMEM = "Hello, world!";

void setup ()
  {
  Serial.begin (115200);
  Serial.println ((__FlashStringHelper *) foo);
  }  // end of setup

void loop () { }

Originally here c - Force runtime array allocation and initialization with list syntax - Stack Overflow

this is just a post of the same question on another forum.

  1. avrdude just uploads the program. It has nothing to do with where the data is. Reading/writing flash is a complex process and is slow. See datasheet.

  2. You can't extend an array at run time but you could use the heap.

  3. You can use progmem - see the playground - to place data in program space but you need to read it back to SRAM before you can use it.

Mark

@holmes4

this is just a post of the same question on another forum.

I don't understand what you are trying to point out here, I wrote this post both here and on stackoverflow, is there a problem with that? I said that and you're even quoting me, I don't get it.

  1. avrdude just uploads the program. It has nothing to do with where the data is. Reading/writing flash is a complex process and is slow. See datasheet.

AVRDude does static memory analysis after compilation, that's what I am talking about.

You can't extend an array at run time but you could use the heap.

I don't think I have ever mentioned that I wanted to do that.

You can use progmem - see the playground - to place data in program space but you need to read it back to SRAM before you can use it.

I said that myself in one of my posts earlier.

  1. Read the freaking topic before ranting 8)

As others have suggested, the cleaner way to avoid holding all the data in RAM would be to store your font data as PROGMEM variables and load each character into RAM as you needed it. But the behaviour you found with your solution does seem very strange:

kilianciuffolo:
But it seems that the compiler is statically reserving memory for the array.

Can you post a complete sketch that demonstrates this in the simplest possible way? (Perhaps two sketches - before and after the change?)

The two sets of compiler output are showing radically different metrics and it seems strange that just increasing the size of an array by one caused such a big difference. This makes me wonder whether there were any other changes in the code not shown. Also, which IDE version are you using?

kilianciuffolo:
I don't understand what you are trying to point out here, I wrote this post both here and on stackoverflow, is there a problem with that? I said that and you're even quoting me, I don't get it.

Well this is just cross-posting on a grander scale, right?

AVRDude does static memory analysis after compilation, that's what I am talking about.

To the extent that it can.


What was the problem with my earlier reply?


But it seems that the compiler is statically reserving memory for the array.

I don't understand your point. You are defining an array and the compiler is reserving memory for it. What would you expect it to do?

Are you referring to this snippet?

void load_digit_ba(uint8_t digit, uint8_t digit_ba[]) {

      // from 0 to 9 I copy the current digit (byte array) into digit_ba
      if (digit == 0) {
        uint8_t digit_ba_tmp[10] = { 24, 40, 0, 0, 0, 224, 240, 248, 252, 254 };

        memcpy(digit_ba, digit_ba_tmp, 10);
      }
    ...

    }

And this is the problem with working with snippets. I can't reproduce that. Taking my best shot at it:

void load_digit_ba(uint8_t digit, uint8_t digit_ba[]) 
{
      // from 0 to 9 I copy the current digit (byte array) into digit_ba
      if (digit == 0) {
        uint8_t digit_ba_tmp[10] = { 24, 40, 0, 0, 0, 224, 240, 248, 252, 254 };

        memcpy(digit_ba, digit_ba_tmp, 10);
      }
    }
void setup() 
{
  load_digit_ba (5,NULL);
}
void loop() { }

Compiler output (IDE 1.5.4):

Sketch uses 572 bytes (1%) of program storage space. Maximum is 32,256 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2,039 bytes for local variables. Maximum is 2,048 bytes.

Now changing the array like this:

        uint8_t digit_ba_tmp[11] = { 24, 40, 0, 0, 0, 224, 240, 248, 252, 254 };

I now get:

Sketch uses 572 bytes (1%) of program storage space. Maximum is 32,256 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2,039 bytes for local variables. Maximum is 2,048 bytes.

So, no change in program size at all.

You need to submit code that proves your point, not a whole lot of disconnected assertions.