Go Down

Topic: RAM usage question: PROGMEM vs const vs #define (Read 15319 times) previous topic - next topic

lgabiot

Nov 18, 2011, 07:10 pm Last Edit: Nov 18, 2011, 07:17 pm by lgabiot Reason: 1
Hello,

I'm trying to save RAM for a sketch, and on top of trying to code more efficiently (use less variables, or variables that use a smallest footprint, be careful about variable scope), I looked into other methods to put into Flash (or program) memory variables that are constant.

So far it seems there is 4 methods to achieve this:

- using simply a number in the code, as in:
Code: [Select]
digitalRead(8);

- using #define, as in:
Code: [Select]
#define SENSORPIN 8
...
digitalRead(SENSORPIN);


- using const, as in:
Code: [Select]
const int sensorPin = 8;
...
digitalRead(sensorPin);


- using PROGMEM feature, as in:
Code: [Select]
#include<avr/pgmspace.h>

...
prog_uchar sensorPin PROGMEM = 8;
digitalRead(pgm_read_byte(&sensorPin));



from what I understand, simply writing explicitely the value is good, exept if you want to use that value in several places, and maybe need to change it later, where #define or const can be better (const being prefered because it allows the compiler to know the type of the variable)

Then I wonder what would really be the use of the last method: PROGMEM

it seems to need more code, is more complicated, so what is the point?

It is advertised for storing arrays of data, such as arrays of strings for instance, but is there a bigger memory use if I use:
Code: [Select]
Serial.println("text to be displayed");

instead of:

Code: [Select]
#include<avr/pgmspace.h>

...

prog_char string[] PROGMEM = "text to be displayed";
char buffer[30];
strcpy_P(buffer, (char*)pgm_read_word(&(string)));
Serial.println(buffer);


I would tend to think eventually that the second method is worst, because of the use of the buffer variable.

Am I right, or wrong? or does it allow also to modify variables?

Or maybe is it only desirable for array usage?
is const a problem in that case?
Code: [Select]
const int numbers[10] = {0,1,2,3,4,5,6,7,8,9};
//then use number[i] in some way




johnwasser


I would tend to think eventually that the second method is worst, because of the use of the buffer variable.


The advantage of the second method is that the same buffer (RAM) can be used to multiple progmem (FLASH) strings.  If you use string constants they all get loaded into RAM at the same time and RAM is MUCH more limited than FLASH.
Send Bitcoin tips to: 1G2qoGwMRXx8az71DVP1E81jShxtbSh5Hp

lgabiot

Thanks for your answer,

so you mean that when "Serial.println("second text");" in the following code is executed:

Code: [Select]
Serial.println("first text");

Serial.println("second text");


the string "first text is still in memory and will never be erased?



cmiyc

#define is just a macro definition.  The complier goes through the code and replaces the keyword with whatever value you tell it before compiling.

#define pin 8
digitalWrite(pin, HIGH);

is exactly the same as:
digitalWrite(8, HIGH);

The trick is that 8 in this case, is a constant.  Constants can still use RAM.  The complier may put the constant into RAM and then reference that constant throughout the code.  

They keyword const still consumes RAM.  The word "const" only tells the complier that the code can't change the value.  So if you try to assign a const a value, the complier will throw an error.

Quote
Then I wonder what would really be the use of the last method: PROGMEM
it seems to need more code, is more complicated, so what is the point?

The point is that off the 4 (which is really 3) methods you listed, this is the ONLY method which will not leave stuff in RAM.  The pointers will pull the constants out of PROGMEM but they won't ever get copied to RAM (unless you do it in your code.)
Capacitor Expert By Day, Enginerd by night.  ||  Personal Blog: www.baldengineer.com  || Electronics Tutorials for Beginners:  www.addohms.com

lgabiot

thanks a lot for your answer.


Quote
The point is that off the 4 (which is really 3) methods you listed, this is the ONLY method which will not leave stuff in RAM.  The pointers will pull the constants out of PROGMEM but they won't ever get copied to RAM (unless you do it in your code.)


Ok, got that point.

Quote
Constants can still use RAM.  The complier may put the constant into RAM and then reference that constant throughout the code. 


So there is no rule to know when the memory used by a constant is released? is it the same scope than a regular variable?

So if for instance I use in the void loop() {} function a Serial.println("text"); "text" will be destroyed when I exit loop()?
Or in the following case :
Code: [Select]
void loop() {
    ...

    while ( n > 0 ) {
        Serial.println("text");
    }
    ....
}


"text" will be destroyed when exiting the while loop?




liuzengqiang

I bet the first 3 methods generate the same compiled result.

Typically, if you have a large constant array such as some values you use or some strings, you store them in PROGMEM and only recall them when you need them to SRAM buffers. Storing one variable in PROGRAM is not going to save too much.

FYI here is my blog post about optimizing arduino memory:

http://liudr.wordpress.com/2011/02/04/how-to-optimize-your-arduino-memory-usage/
Serial LCD keypad panel,phi_prompt user interface library,SDI-12 USB Adapter

Coding Badly


To force the compiler's hand, static should be included in the declaration...

Quote
static const int sensorPin = 8;

lgabiot

@ Coding Badly
I believed static had the opposite effect: creating a variable that wouldn't be destroyed when going outside of scope?

Could be useful, but not in the special need of saving RAM, or am I wrong?

@ liudr
Thanks I'll read your blog.


And to go back to my previous post, I still wonder about the scope of written constants (such as "text" in Serial.println("text");). Does it work in the same way as normal variables (i.e. living between curly braces)?


Coding Badly

I believed static had the opposite effect: creating a variable that wouldn't be destroyed when going outside of scope?


static has two effects.  The one you described.  And, it makes the name module scope instead of global scope.  If sensorPin is global scope then the compiler may have to allocate storage space in case code from another module takes the address of sensorPin.  By declaring sensorPin to be module scope and not referring to it by address, the compiler is free to do what it wants with sensorPin and the value.  In the case of AVR-GCC, the compiler correctly treats it as a simple constant allowing very aggressive optimization.

Quote
Could be useful, but not in the special need of saving RAM, or am I wrong?


So long as your code does not refer to the constant by address and the name is module scope then you are wrong.

johnwasser


I still wonder about the scope of written constants (such as "text" in Serial.println("text");). Does it work in the same way as normal variables (i.e. living between curly braces)?


No.  String constants are copied from FLASH to RAM before your program starts and stay in memory forever. 

Only the LPM instruction can read the FLASH memory space so any function not designed explicitly to use FLASH/progmem can only use RAM addresses.
Send Bitcoin tips to: 1G2qoGwMRXx8az71DVP1E81jShxtbSh5Hp

nickgammon

Quote
I would tend to think eventually that the second method is worst, because of the use of the buffer variable.


To print strings from PROGMEM you can copy them out a byte at a time, so the buffer variable is one byte long. And this variable (which in practice the compiler will probably optimize into a register) is re-used for all your PROGMEM data.



Or in the following case :
Code: [Select]
void loop() {
    ...

    while ( n > 0 ) {
        Serial.println("text");
    }
    ....
}


"text" will be destroyed when exiting the while loop?


The "text" variable will still exist, ready for next time you enter loop. However if you happen to use "text" somewhere else in your program the compiler will share that memory location because string literals don't change.

I should point out that variables are not really destroyed, that would involve some sort of pyrotechnics. However the space used for them might be re-used under certain circumstances. Clearly however the space for literals cannot be re-used because how would it know what literal to put back there?

For example "auto" variables (ie. local variables inside a function) are allocated on the stack using memory previously used by other function calls (hence this is why their initial contents are not defined, if you don't initialize them). Then when that function exits, that stack is available for a different functions auto variables.
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

lgabiot

thanks a lot for all your answers!

I have to admit the subject is fascinating, but a little beyond my actual knowledge, so I'm slow to ingest your informations.

@ Coding Badly: thanks for your explanation of the module scope of static.

johnwasser wrote:
Quote
No.  String constants are copied from FLASH to RAM before your program starts and stay in memory forever.


Thanks a lot for the information. Then using PROGMEM makes sense. especially because:

Nick Gammon wrote:
Quote
To print strings from PROGMEM you can copy them out a byte at a time, so the buffer variable is one byte long. And this variable (which in practice the compiler will probably optimize into a register) is re-used for all your PROGMEM data.


So indeed it will take far less memory...

I have to admit that I fail to see the reason why the strings literals have to stay in memory the whole time, and not been treated as either: instructions (who seems to stay in flash as far as I understand), or be considered as variables with the usual scope (curly braces to be short).

But then I think I lack the general knowledge of low level memory handling...

Anyway, if I understand well:
Code: [Select]
char A[10] = "some text";

will take 20 byte in RAM: 10 byte for the "some text" literal, and then 10 byte for the A array.

I'm going OT here, so discard this if you wish:
It make me also wonder how computers are using their memory, since there is no flash/ram stuff. I guess the ram is splitted in two: one part for instructions, one part for variables etc...?

Any reliable book on the subject (computer / microcontroller architecture) that would be a good entry for a beginner like me to understand more this?




nickgammon

http://en.wikipedia.org/wiki/Harvard_architecture

Memory and instructions are separate.


Anyway, if I understand well:
Code: [Select]
char A[10] = "some text";

will take 20 byte in RAM: 10 byte for the "some text" literal, and then 10 byte for the A array.


I think in that particular case (if statically defined) it would take 10 bytes, because the initialization would copy the text (once) from PROGMEM. However if inside a function, yes "some text" would have to also exist.

Quote
I have to admit that I fail to see the reason why the strings literals have to stay in memory the whole time, and not been treated as either: instructions (who seems to stay in flash as far as I understand), or be considered as variables with the usual scope (curly braces to be short).


It would be something to do with the fact that the instructions to access program memory are different to those to access RAM. Perhaps a smarter compiler might automatically generate such instructions, but not today. Plus, and this is probably important, since the two memories are separate a given address would have different meanings. So address 0x0100 in program memory would be different to address 0x0100 in RAM. For the compiler to keep track of two "sorts" of addresses might be a stretch.
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

lgabiot

Thanks again for your answer.

Is there any penalties using flash memory to store constant instead of RAM?
(I could optimise all the constant of my sketch to use PROGMEM for instance, or just the biggest ones?).
I guess it also an issue of time to write code, to keep it as simple as possible to be maintainable, and not over-optimise.

Could it slow down the execution, or is the flash ram less resistant to multiple read? (I seems the number of write is smaller).

(and don't answer if you are tired of my questionning, it is really out of curiosity, I've more than enough information to continue to code now).

Coding Badly

Is there any penalties using flash memory to store constant instead of RAM?


Yes, CPU time.  Loading data from SRAM is a two cycle machine instruction.  Loading data from Flash is three cycles.  But, Loading from Flash requires using the Z register which is at least another two cycles to initialize.  And, having to dedicate the Z register can interfere with nearby code generation (less than optimal).  A good rule of thumb: Flash data is about three times slower to access than SRAM data.

Unless you are developing very time critical code, the CPU time penalty is irrelevant.

Quote
(I could optimise all the constant of my sketch to use PROGMEM for instance, or just the biggest ones?).


Biggest and least frequently accessed are the low hanging fruit.

Quote
I guess it also an issue of time to write code, to keep it as simple as possible to be maintainable, and not over-optimise.


Welcome to the Flash Library...
http://arduiniana.org/libraries/flash/

Go Up