Code Efficiency & Variable Space Questions

Are there resources to help figure these issues out?

I’m looking to conserve as much variable memory space to use for data collection. In this case data collection speed precludes moving it off board to the serial monitor. And I wish not to fool with EEPROM, etc.

Variables within functions are created and destroyed. Does this consume program space or variable space. If program space then I could possibly lean code in that direction.

A compiler comment mentions the space used and what remains. It also warns about not going to low and how much is unused. Is this referring to space needed for what is created then destroyed? And is there anyway to know how low one can go before causing errors? Any tools available to predict how low one can go? Or is there a way to measure it during execution?

Also code efficiency knowledge would help regarding execution speed, variable usage, and coding style. By style I refer to code like “if then else” versus equations like “a = b? c : a;” is this quicker? Which is faster and what about resources used?

Currently the program being worked on uses up about 42% code space. And 55% variable space. This is with the data collect array set to near 0. This only leave 400 bytes before I reach into the reserved space. Not many bytes.

The Board is ATMega328 Uno. 2K Variable Ram.

How are “const” handled?

Debug messages are done with bool DEBUG = false; When compiling what happens with a function like

“if (DEBUG && other conditions) { code }”

is it left out of the code when debug is defined false? it would make sense to leave it out.

many questions and not being a in the guts of the compiler I have no idea how to go about figuring these things out.

:o

Does this consume program space or variable space.

Variables are created on the stack, which is part of SRAM, or, variable space in your vernacular.

A compiler comment mentions the space used and what remains. It also warns about not going to low and how much is unused. Is this referring to space needed for what is created then destroyed?

It is only concerned about global variables that exist all the time. The size of the stack depends on how many functions are called (nested depth) and what local variables the functions use. None of that is known at compile time.

And is there anyway to know how low one can go before causing errors?

You can use 100% of SRAM. Knowing how much the stack uses at a given time is possible. That's what the freeMemory() function (google to find it, if you need to) measures.

Which is faster and what about resources used?

The compiler is very good at optimizing code for speed, for size, or for memory usage. Typically, some compromise is needed, because faster code that uses 8 times the memory is usually not an acceptable trade-off.

How are "const" handled?

Correctly. What are you concerned about?

is it left out of the code when debug is defined false?

If the code is between #ifdef/#endif statements, then it may or may not be compiled. If if uses a varialble, as in your example, that might change, it must be compiled, but will be bypassed at run time (assuming the values stays false).

PaulS:
If the code is between #ifdef/#endif statements, then it may or may not be compiled. If if uses a varialble, as in your example, that might change, it must be compiled, but will be bypassed at run time (assuming the values stays false).

No, in his example, if DEBUG evaluates to false at compile time (it is presumably either a #define or const), the ENTIRE if block will be discarded by the compiler, as the if conditional can NEVER be true.
Regards,
Ray L.

"const" variables usually do not take up any space in your program, but if you do certain things (such as taking their address) then they may be assigned some space.

Stack space may be shared, or it may be additive. It depends upon how your functions are called and how they get nested.

Hints:

  • Do not use "int" when "byte" will do.
  • Use the "F()" macro where you can.
  • Make use of #ifdef/#endif where you can. This is C/C++, not Java. (Some Java compilers are smart about blocks of code that are surrounded by an "if" statement that will always be false. I do not think that gcc is this clever but I may be wrong. RayLivingston believes that gcc is this clever.)
  • Try your code different ways and make measurements so that you use the appropriate way.

The uno has 2K of RAM. Given that your code leaves you 400 bytes to play with, I suspect that however frugal/clever you are, you're unlikely to increase that to let's say 800. So how much do you actually need? If you need 20% more, perhaps there will be a way to do it. Much more and you're probably better off buying something with more memory. A Teensy perhaps.

Others here can verify, but my understanding is that globals and static's are allocated on what is called "the heap", whereas the temporary variables are created on the stack. Both the stack and the heap take their memory space from the available SRAM. The Top of the Stack (TOS) grows downward as parameters and return addresses are pushed onto the stack. When the function is exited, the TOS grows back upward as those "spent" variables are discard on function exit. The Top of the Heap (TOH) grows from the base of SRAM upwards until all globals and static's are allocated. One of the dangers of recursive function calls is a collision between the TOS and TOH when the program executes.

wildbill:
Much more and you're probably better off buying something with more memory. A Teensy perhaps.

The Teensy is a great microcontoller, but isn't cheap ($20 for 3.2 and $30 for the 3.6). I'm having some luck substituting members of the STM32F family for the Uno, Nano, and Mega's. The Blue pill can be purchased for as little as $2, has 64K or 128K of flash memory, 20K of SRAM (the real bottleneck for most programs), and scoots along at 72MHz. Also, it can be programmed in the Arduino IDE. Finally, even though some chips are sold as 64K versions, in fact some I have ordered have 128K of flash. The only thing I have found missing is EEPROM. (They can emulate EEPROM by reserving a segment of flash.)

econjack:
Others here can verify, but my understanding is that globals and static’s are allocated on what is called “the heap”, where as the temporary variables are created on the stack. Both the stack and the heap take their memory space from the available SRAM. The Top of the Stack (TOS) grows downward as parameters and return addresses are pushed onto the stack. When the function is exited, the TOS grows back upward as those “spent” variables are discard on function exit. The Top of the Heap (TOH) grows from the base of SRAM upwards until all globals and static’s are allocated. One of the dangers of recursive function calls is a collision between the TOS and TOH when the program executes.

Static data resides in the .data or .bss sections, depending on whether it’s initialized or not.


Source

The heap is used for dynamic memory (using malloc or new). The stack is where local non-static variables live.

Pieter

@PieteP--nice graphic. My discussion included the .data and .bss as part of the heap space, and those two areas were fixed at compile time. Dynamic memory allocations "grow" the heap from the left edge of the (purple) heap space towards TOS, while the TOS grows downward, as shown in the graphic, towards the heap. If the two collide (the purple and brown areas overlap), that's a stack crash and things rarely go well after that.

Thanks,

I'll reread this a couple of times and then try various approaches to conserving the stack.

I do not play with recursive function calls. I'm a hardware guy and things begin to get to complicated.

@wildBill... As to space required for data collection. Like with all monitors there is always a limit. So features such as tossing out replicated data based on Delta, start/stop capture windows, help. Right now data is an integer. Perhaps converting the data to a byte value may become useful.

@PaulS... To further explain the question.... How are constants handled? (I knew what I meant :slight_smile:

const int foo = 99;

"if (mydata = foo)" --- I would guess foo is replace with 99. simple, makes sense

but what about this?

const byte wgm2 = ((wgm2120MaskReg & myTCCR2A) | ((wgm22MaskReg & myTCCR2B) >> 1));

when first coding my thought was the compiler would replace all wgm2 references with that bit wise code, cut and paste. It does not. So I turned them into functions like.

byte wgm2() {return (wgm2120MaskReg & myTCCR2A) | (wgm22MaskReg & myTCCR2B) >> 1;}

keep in mind I am a hardware logic designer by trade. I assumed a constant will end up replacing all wgm2 instances in the code just like foo did above.

When you use the const keyword, the compiler is ONLY obligated to see that no new value is assigned to that memory address. But, it typically uses a register to hold the value, and substitutes the register address where the variable address would appear. It can't do that with an unlimited number of variables, because there are a limited number of registers available.

So, the const keyword CAN save memory, but it may not.

Are there resources to help figure these issues out?

The main one is call "nm" (avr-nm for Uno/etc (alhtough it may not matter.) Buried deep in the binary directory of your Arduino is this command-line utility, and if you run it on the .elf file that is buried deep in the build directory of your sketch, you'll get output like:

#sort by size, show sizes, demangle C++ symbols
[b][color=red]avr-nm --size-sort -S -C *.elf       [/color][/b]
[color=blue]00800158[/color] [color=maroon]00000001[/color] [color=limegreen]b[/color] timer0_fract
[color=blue]00000112[/color] [color=brown]00000002[/color] [color=limegreen]t[/color] Print::flush()
[color=blue]00800100[/color] [color=brown]00000002[/color] [color=limegreen]d[/color] thisByte
00000384 00000004 t __cxa_pure_virtual
00800159 00000004 b timer0_millis
00800154 00000004 b timer0_overflow_count
00000114 00000006 t Print::availableForWrite()
00000746 0000000a T abort
0000073a 0000000c T __tablejump2__
0000008c 00000010 T __do_clear_bss
00800102 00000012 d vtable for HardwareSerial
000002ca 00000014 t serialEventRun()
00000194 00000014 t Serial0_available()
00000076 00000016 T __do_copy_data
0000009c 00000016 T __do_global_ctors
0000017c 00000018 t HardwareSerial::available()
00000160 0000001c t HardwareSerial::peek()
0000011a 0000001e t HardwareSerial::availableForWrite()
000002de 00000024 t Print::write(char const*) [clone .constprop.15]
00000138 00000028 t HardwareSerial::read()
000001a8 00000044 t HardwareSerial::_tx_udr_empty_irq()
00000286 00000044 t HardwareSerial::flush()
000006f6 00000044 T __udivmodsi4
0000041c 0000004c T __vector_19
000000be 00000054 t Print::write(unsigned char const*, unsigned int)
0000069c 0000005a t _GLOBAL__sub_I___vector_18
00000468 00000064 T __vector_18
00000302 00000082 t Print::printNumber(unsigned long, unsigned char) [clone .constprop.12]
00000388 00000094 T __vector_16
000001ec 0000009a t HardwareSerial::write(unsigned char)
0080015d 0000009d b Serial
000004cc 000001d0 T main

The blue column is the symbol's value (this will be 800000+the RAM address for RAM on an AVR, for "reasons.") The Brown column is the size of that symbol (in hex), and the green column is the type:
"b" for bss (uninitialized data), "d" for .data (initialized data), and t for .text (code, PROGMEM)

In this example (the ASCIITable example sketch) you get to see the RAM consumed by the Serial data structure (including the buffer), but it's otherwise pretty boring.

This does not include stack variables or malloc'ed/new'ed memory, of course.

Some Java compilers are smart about blocks of code that are surrounded by an "if" statement that will always be false. I do not think that gcc is this clever but I may be wrong.

Most C compilers (including the one used by Arduino) ARE "that clever"; I've been writing code that relies on that fact for probably 20 years...

westfw:
The main one is call "nm"...

I believe some / most / all of the same information is available from a MAP file. An update to boards.txt (or one of those configuration files) will get the build process to output a MAP file. If anyone wants details let me know. I can remember doing that once before.

I find the .map files hopelessly verbose, since they seem to include data from every debug symbol produced by the compiler as well as the "important" stuff.

They do contain a cross-reference table that is sometimes useful, but ugh.
(perhaps it's just the way that Atmel Studio produces the .map, but...)

westfw:
I find the .map files hopelessly verbose...

Agree! Sifting through them is a chore.

Thanks.... some of you may have thought I had better coding practices. they are slightly better today.

:o

After going through the program files using the F() around print text, cleaning up, etc... there is 480 bytes for data collection, leaving 515 free.

6 channels enabled, 80 bytes each; Enough for my current purposes. Thanks!

Changed data capture to include an if(channelEnabled) {} wherever data is stored, printed, calculated, etc... The code is in condensed per function code blocks so to be easily configured and is held in a configuration file. Not spread around. This change made a significant difference.

free memory function will be useful in future. It returns delta between heap and stack I assume.

Presently all channels are stored with time stamp on any signal change. If one channel state changes they all get stored. If need be it may be reasonable to only store state changes and pushing them into an array of time,value,type. Type would be a byte, time int, value int (maybe byte will suffice). Perhaps a future enhancement or maybe I'll build a stand alone analyzer using a Arduino or like with deep pockets and fast clock speed to make life easier. Also logic signals can be stored as bits further conserving memory. At present storage is the issue. Plenty of time between loops.

----------- what I'm doing with this particular code, see below if interested-----------------------------

I'm using this uno to control a two motor robot. This data analyzer collects code values during operation, pin outputs, and analog input signals. Ten channels, 5 are analog. At present 8 are sampling data, storing six values. Two pairs are measuring voltage across motor and current measuring resistor. It does the math before storing result.

This is to see what is actually going on rather than guess why things are not behaving as expected. I only need 60-100 storage locations per channel for this particular robot. In the future this code might be moved to a stand alone Arduino with greater speed and greater memory and maybe faster ADCs.