RAM usage question: PROGMEM vs const vs #define

Hello,

I'm trying to save RAM for a sketch, and on top of trying to code more efficiently (use less variables, or variables that use a smallest footprint, be careful about variable scope), I looked into other methods to put into Flash (or program) memory variables that are constant.

So far it seems there is 4 methods to achieve this:

  • using simply a number in the code, as in: digitalRead(8);

  • using #define, as in:

#define SENSORPIN 8
...
digitalRead(SENSORPIN);
  • using const, as in:
const int sensorPin = 8;
...
digitalRead(sensorPin);
  • using PROGMEM feature, as in:
#include<avr/pgmspace.h>

...
prog_uchar sensorPin PROGMEM = 8;
digitalRead(pgm_read_byte(&sensorPin));

from what I understand, simply writing explicitely the value is good, exept if you want to use that value in several places, and maybe need to change it later, where #define or const can be better (const being prefered because it allows the compiler to know the type of the variable)

Then I wonder what would really be the use of the last method: PROGMEM

it seems to need more code, is more complicated, so what is the point?

It is advertised for storing arrays of data, such as arrays of strings for instance, but is there a bigger memory use if I use:

Serial.println("text to be displayed");

instead of:

#include<avr/pgmspace.h>

...

prog_char string[] PROGMEM = "text to be displayed";
char buffer[30];
strcpy_P(buffer, (char*)pgm_read_word(&(string))); 
Serial.println(buffer);

I would tend to think eventually that the second method is worst, because of the use of the buffer variable.

Am I right, or wrong? or does it allow also to modify variables?

Or maybe is it only desirable for array usage?
is const a problem in that case?

const int numbers[10] = {0,1,2,3,4,5,6,7,8,9};
//then use number[i] in some way

lgabiot:
I would tend to think eventually that the second method is worst, because of the use of the buffer variable.

The advantage of the second method is that the same buffer (RAM) can be used to multiple progmem (FLASH) strings. If you use string constants they all get loaded into RAM at the same time and RAM is MUCH more limited than FLASH.

Thanks for your answer,

so you mean that when "Serial.println("second text");" in the following code is executed:

Serial.println("first text");

Serial.println("second text");

the string "first text is still in memory and will never be erased?

#define is just a macro definition. The complier goes through the code and replaces the keyword with whatever value you tell it before compiling.

#define pin 8
digitalWrite(pin, HIGH);

is exactly the same as:
digitalWrite(8, HIGH);

The trick is that 8 in this case, is a constant. Constants can still use RAM. The complier may put the constant into RAM and then reference that constant throughout the code.

They keyword const still consumes RAM. The word "const" only tells the complier that the code can't change the value. So if you try to assign a const a value, the complier will throw an error.

Then I wonder what would really be the use of the last method: PROGMEM
it seems to need more code, is more complicated, so what is the point?

The point is that off the 4 (which is really 3) methods you listed, this is the ONLY method which will not leave stuff in RAM. The pointers will pull the constants out of PROGMEM but they won't ever get copied to RAM (unless you do it in your code.)

thanks a lot for your answer.

The point is that off the 4 (which is really 3) methods you listed, this is the ONLY method which will not leave stuff in RAM. The pointers will pull the constants out of PROGMEM but they won't ever get copied to RAM (unless you do it in your code.)

Ok, got that point.

Constants can still use RAM. The complier may put the constant into RAM and then reference that constant throughout the code.

So there is no rule to know when the memory used by a constant is released? is it the same scope than a regular variable?

So if for instance I use in the void loop() {} function a Serial.println("text"); "text" will be destroyed when I exit loop()?
Or in the following case :

void loop() {
    ...

    while ( n > 0 ) {
        Serial.println("text");
    }
    ....
}

"text" will be destroyed when exiting the while loop?

I bet the first 3 methods generate the same compiled result.

Typically, if you have a large constant array such as some values you use or some strings, you store them in PROGMEM and only recall them when you need them to SRAM buffers. Storing one variable in PROGRAM is not going to save too much.

FYI here is my blog post about optimizing arduino memory:

To force the compiler's hand, static should be included in the declaration...

static const int sensorPin = 8;

@ Coding Badly
I believed static had the opposite effect: creating a variable that wouldn't be destroyed when going outside of scope?

Could be useful, but not in the special need of saving RAM, or am I wrong?

@ liudr
Thanks I'll read your blog.

And to go back to my previous post, I still wonder about the scope of written constants (such as "text" in Serial.println("text");). Does it work in the same way as normal variables (i.e. living between curly braces)?

lgabiot:
I believed static had the opposite effect: creating a variable that wouldn't be destroyed when going outside of scope?

static has two effects. The one you described. And, it makes the name module scope instead of global scope. If sensorPin is global scope then the compiler may have to allocate storage space in case code from another module takes the address of sensorPin. By declaring sensorPin to be module scope and not referring to it by address, the compiler is free to do what it wants with sensorPin and the value. In the case of AVR-GCC, the compiler correctly treats it as a simple constant allowing very aggressive optimization.

Could be useful, but not in the special need of saving RAM, or am I wrong?

So long as your code does not refer to the constant by address and the name is module scope then you are wrong.

lgabiot:
I still wonder about the scope of written constants (such as "text" in Serial.println("text");). Does it work in the same way as normal variables (i.e. living between curly braces)?

No. String constants are copied from FLASH to RAM before your program starts and stay in memory forever.

Only the LPM instruction can read the FLASH memory space so any function not designed explicitly to use FLASH/progmem can only use RAM addresses.

I would tend to think eventually that the second method is worst, because of the use of the buffer variable.

To print strings from PROGMEM you can copy them out a byte at a time, so the buffer variable is one byte long. And this variable (which in practice the compiler will probably optimize into a register) is re-used for all your PROGMEM data.

lgabiot:
Or in the following case :

void loop() {

...

while ( n > 0 ) {
        Serial.println("text");
    }
    ....
}




"text" will be destroyed when exiting the while loop?

The "text" variable will still exist, ready for next time you enter loop. However if you happen to use "text" somewhere else in your program the compiler will share that memory location because string literals don't change.

I should point out that variables are not really destroyed, that would involve some sort of pyrotechnics. However the space used for them might be re-used under certain circumstances. Clearly however the space for literals cannot be re-used because how would it know what literal to put back there?

For example "auto" variables (ie. local variables inside a function) are allocated on the stack using memory previously used by other function calls (hence this is why their initial contents are not defined, if you don't initialize them). Then when that function exits, that stack is available for a different functions auto variables.

thanks a lot for all your answers!

I have to admit the subject is fascinating, but a little beyond my actual knowledge, so I'm slow to ingest your informations.

@ Coding Badly: thanks for your explanation of the module scope of static.

johnwasser wrote:

No. String constants are copied from FLASH to RAM before your program starts and stay in memory forever.

Thanks a lot for the information. Then using PROGMEM makes sense. especially because:

Nick Gammon wrote:

To print strings from PROGMEM you can copy them out a byte at a time, so the buffer variable is one byte long. And this variable (which in practice the compiler will probably optimize into a register) is re-used for all your PROGMEM data.

So indeed it will take far less memory...

I have to admit that I fail to see the reason why the strings literals have to stay in memory the whole time, and not been treated as either: instructions (who seems to stay in flash as far as I understand), or be considered as variables with the usual scope (curly braces to be short).

But then I think I lack the general knowledge of low level memory handling...

Anyway, if I understand well:

char A[10] = "some text";

will take 20 byte in RAM: 10 byte for the "some text" literal, and then 10 byte for the A array.

I'm going OT here, so discard this if you wish:
It make me also wonder how computers are using their memory, since there is no flash/ram stuff. I guess the ram is splitted in two: one part for instructions, one part for variables etc...?

Any reliable book on the subject (computer / microcontroller architecture) that would be a good entry for a beginner like me to understand more this?

Memory and instructions are separate.

lgabiot:
Anyway, if I understand well:

char A[10] = "some text";

will take 20 byte in RAM: 10 byte for the "some text" literal, and then 10 byte for the A array.

I think in that particular case (if statically defined) it would take 10 bytes, because the initialization would copy the text (once) from PROGMEM. However if inside a function, yes "some text" would have to also exist.

I have to admit that I fail to see the reason why the strings literals have to stay in memory the whole time, and not been treated as either: instructions (who seems to stay in flash as far as I understand), or be considered as variables with the usual scope (curly braces to be short).

It would be something to do with the fact that the instructions to access program memory are different to those to access RAM. Perhaps a smarter compiler might automatically generate such instructions, but not today. Plus, and this is probably important, since the two memories are separate a given address would have different meanings. So address 0x0100 in program memory would be different to address 0x0100 in RAM. For the compiler to keep track of two "sorts" of addresses might be a stretch.

Thanks again for your answer.

Is there any penalties using flash memory to store constant instead of RAM?
(I could optimise all the constant of my sketch to use PROGMEM for instance, or just the biggest ones?).
I guess it also an issue of time to write code, to keep it as simple as possible to be maintainable, and not over-optimise.

Could it slow down the execution, or is the flash ram less resistant to multiple read? (I seems the number of write is smaller).

(and don't answer if you are tired of my questionning, it is really out of curiosity, I've more than enough information to continue to code now).

lgabiot:
Is there any penalties using flash memory to store constant instead of RAM?

Yes, CPU time. Loading data from SRAM is a two cycle machine instruction. Loading data from Flash is three cycles. But, Loading from Flash requires using the Z register which is at least another two cycles to initialize. And, having to dedicate the Z register can interfere with nearby code generation (less than optimal). A good rule of thumb: Flash data is about three times slower to access than SRAM data.

Unless you are developing very time critical code, the CPU time penalty is irrelevant.

(I could optimise all the constant of my sketch to use PROGMEM for instance, or just the biggest ones?).

Biggest and least frequently accessed are the low hanging fruit.

I guess it also an issue of time to write code, to keep it as simple as possible to be maintainable, and not over-optimise.

Welcome to the Flash Library...
http://arduiniana.org/libraries/flash/

I've encountered verification errors a couple of times in the past using the flash library. I don't know what the cause for the errors were, but when I switched to using PROGMEM... No more verification errors. Has anyone else had this problem?

Just my 2 cents,
DJ

@ Coding Badly

thanks for your informations and explanations.

I've finaly went the PROGMEM way, all goes well, but I'll test in the future the flash library.

@DigitalJohnson...

The two cannot possibly be directly related. For the purposes of uploading and verifying, it's all just bytes.

Which board were you using at the time?

@Coding Badly
I used a mega1280. I'm certainly not qualified to know why I was getting verification errors using the flash library, nor do I still have the code that produced the error. I changed the code to use progmem instead and all was well. What I do remember is that my sketch was around 35K in size and I was storing about 200-250 bytes of 8-bit constants in a couple of arrays using the flash library, some as chars and some as bytes. It would compile fine but when I uploaded I got a verification error. I posted my problem to the forum at the time with no luck as to why the error. When I stored the same data in the same sketch but using progmem, no more errors. I've uploaded a few even bigger sketches since without errors so I don't believe it's a bad memory location. If you'd like more specifics, I'll try to answer as best I can recall. I'd still like to know what the problem was.

Thanks,
DJ