How much ram is being used?

I am trying to put together a few 18b20 temp sensors, an 8x2 LCD with the lcd4bit library and a DS1305 RTC. my sketch size is about 10k of 16k.. I get to a certain point and something is corrupting the LCD stream.. I can't see how I am using up 1k of RAM.. is some of that lost to overhead?

How do I find how much RAM is in use? What can I do to find out what is overlapping?

I suspect that much AVR debuggers, like the Dragon, can give you the amount of ram your program is using, but this needs to be confirmed.

There are many reasons why data can get corrupted, like a buggy line writing too much data at some place... or many others :slight_smile:

The wierd part is if I comment out different lines of code, the problem goes away.

I didn't start having this problem until I got somewhere over 9k of sketch

So probably your analysis is right... I'd considerer buying a Dragon or similar debugger to see what's happening, or just go study the libs I use to see how much ram they potentially consume...

You can write your own function to figure out how much free RAM you have on your mega168 at any point in your execution. Please take a look at this thread for more information:

http://forum.pololu.com/viewtopic.php?f=10&t=989&view=unread#p4218

  • Ben

You can write your own function to figure out how much free RAM you have on your mega168 at any point in your execution. Please take a look at this thread for more information:

http://forum.pololu.com/viewtopic.php?f=10&t=989&view=unread#p4218

  • Ben

Calculating memory availability is tricky. The test on that link does not seem to give reliable results for the memory available to malloc, so if using code or libraries that call malloc then the memory test in the Arduino playground is better.

If there are no calls to malloc then the Pololu code is better because that test consumes an insignificant amount of memory compared to the call to malloc just for the playground test.

I expect both tests suffer from inaccuracy if the memory pool gets fragmented

FYI, the Pololu code needed to be changed from
extern void __bss_end;
to
extern int __bss_end;

to compile in my arduino environment

I was probably unclear on the indended use of the function to which I linked. The real point of this memory test function is to determine when your stack is about to overwrite your data, not to figure out how much space you have left for malloc (especially since malloc inherently provides this functionality itself by returning zero if the allocation fails). Even if you've used malloc, if this memory test function returns a small number, you are in danger of having your stack overwrite your data (which can lead to data overwriting the stack), which in turn can lead to a corrupted program with unpredictable behavior and very frustrating bugs. If you have a lot of nested functions or use a lot of RAM, you can call this test function often to make sure you are never having memory issues.

  • Ben

The Arduino is supposed to have 1k of RAM, right?
Usually, how much should be 'available' for Arduino sketch users?
I put it at the beginning of the program and after the first 4 voids that are called before my sketch seems to crash and it returns 271 each time.

Well... possibly one of my problems was a missing () when calling a void.

Then, I uncommented a bunch of serial.print("text blah"); statements and FreeMem reported -105 and the sketch crashed..

Does using "text" strings use up lots of ram? Should I move them to progmem?

The mega168 has 1k of RAM, and the Arduino environment takes up between 100 and 200 bytes of that for various globals and for the serial buffers (I don't off-hand know exactly how much). Note that on the mega168, RAM is also used to hold the stack, so your global variables are at the beginning of RAM, the heap (the dynamically allocated variables created with malloc()) grows outwards from the end of the global variable space, and the stack grows from the end of RAM towards the heap/global space.

When you're writing a program, you need to make sure you leave plenty of RAM for the stack. All non-static local variables are put on the stack, and nested functions will grow the stack. If the stack ever overruns the data in the heap or the global space, you will run into problems, so one crucial thing to know about your program is the maximum amount of stack space it will use.

Local text strings of with N characters take up N+1 bytes of stack space. Global text strings with N characters take up N+1 bytes of global space. If you have many hundreds of characters in local text strings all in the same function or in the same branch of nested functions, you could very well cause your stack to overrun your global data. If these text strings don't need to dynamically change, you should store them in and access them from program memory (i.e. in flash space). They're in flash anyway, so having them occupy both flash and RAM is neither efficient nor helpful.

If you are getting a negative result from the function in the other thread I linked, you are almost certainly having memory issues. Even if that function says you have a little bit of free space, you could still have memory issues in other parts of the program that grow the stack slightly farther.

  • Ben

So I moved those text strings to progmem and got them successfully in and only 'lost' a few bytes of RAM in the process.. but they no longer cause me to crash.
Are these functions considered nested?
if (xx==yy)
{
}
else if (xy==ya)
{
}
or only those that are like this?
if (xx==yy)
{
if (xy==ya)
{
}
}

None of this is a nested function. This term refers only to the fact of encapsulating a function into another. Here's an example I stole from Wikipedia :

float E(float x)
{
    float F(float y)
    {
        return x + y;
    }
    return F(3);
}

This is basically used to limit the scope (and the encumbering of the namespace) of a very specific function, not supposed to be reused in other contexts. You don't often use such stuff.

It could be considered, imho, as a non-OO way of creating "private" functions.

By nested functions I mean neither what you nor tehboii are saying. I mean simply something like:

void A(...)
{
some stuff
B(...);
}

void B(...)
{
some stuff
C(...);
}

void C(...)
{
some stuff
}

void loop()
{
some stuff
A(...);
}

loop() calls A(), which calls B(), which calls C(). Every function called adds more data to the stack, and this data isn't removed from the stack until the function returns. By the time you've reached C() you're now four levels deep. If you try to figure out how much stack you're using by just looking at what C() requires, you could be severely underestimating your stack usage. If you want to assess your program's stack usage, you need to look for locations that seem like they might require the most stack space and check your free memory there. This will probably be inside one of your most deeply nested functions, or inside a function with a lot of local variables.

The same problem results from recursion. Every level of recursion depth adds data to the stack, so recursing too deeply will cause problems.

  • Ben

dnear1, do you mean that the LCD crashes, but the Arduino is OK? I had this problem recently, and it was due to sending too many strings to the LCD. A little delay(50) in between cured it.

D

Actually, at one point, the whole thing crashed.
Then, as I disabled things, the LCD worked but it was displaying strings in the wrong place (and I use lcd.cursorTo(1,0) before each display update).. another time, it was displaying complete gibberish because the strings were constantly being overwritten by something..

So far, I managed to free about 350 bytes at the point the sketch starts and things appear to be behaving finally, plus I was able to re-enabled all the features after I moved 90% of my strings to PROGMEM

By nested functions I mean neither what you nor tehboii are saying. I mean simply something like:

void A(...)
{
some stuff
B(...);
}

void B(...)
{
some stuff
C(...);
}

void C(...)
{
some stuff
}

void loop()
{
some stuff
A(...);
}

loop() calls A(), which calls B(), which calls C(). Every function called adds more data to the stack, and this data isn't removed from the stack until the function returns. By the time you've reached C() you're now four levels deep. If you try to figure out how much stack you're using by just looking at what C() requires, you could be severely underestimating your stack usage. If you want to assess your program's stack usage, you need to look for locations that seem like they might require the most stack space and check your free memory there. This will probably be inside one of your most deeply nested functions, or inside a function with a lot of local variables.

The same problem results from recursion. Every level of recursion depth adds data to the stack, so recursing too deeply will cause problems.

  • Ben

Following someone else's advice, I tried to keep most functions to less than 20 lines, so you can see the whole function on-screen without scrolling. So I tried to group certain tasks into a separate function and call them multiple times from within another.
I also got rid of some 'overhead' and freed up process time by enabling/using a 1hz 'interrupt' from my rtc. this way most of the functions, including display refresh only happens once a second - which is more than fast enough. otherwise the serial 'interface' was somewhat unresponsive.

Maybe the onewire functions are giving me some of the grief.. so I will probably try to reduce recursion around those functions..

Mems, Bens, and others,

This thread has been quite helpful. You guys are champs. I've been developing code on top of a 3rd party's software base for the Arduino and running into problems with someone (?) writing to memory incorrectly. Since the 3rd party code is large and rev 1.0 (and I make goofs all the time) it was not clear who was stomping on memory incorrectly (them or me). So, after unit testing my own code elsewhere to verify it was clean and also verifying that the 3rd party's code does, indeed, work properly it seemed there was another problem coming into play. I suspected memory size constraints but lacked simple tools to measure memory consumption on the Arduino. That's when I discovered this thread and a number of others.

I've put together a sketch that appears to give good results and a complete memory profile so the remaining memory (or stack/heap collision) can be clearly indicated. It is in the next post....

Marc

Mems, Bens, and others,

Referring to the my previous post immediately above, this sketch is a straightforward memory profiler that computes the size of the .data, .bss, stack, heap, and free ram sections. I've tested it on the Arduino Duemilanove with the ATmega328 chip in the Arduino-0016 IDE and it seems to be working properly (so far).

It is based on 'Pauls' clearly written function from the Pololu forum, get_free_memory(). (this thread already hilighted his work here: http://forum.pololu.com/viewtopic.php?f=10&t=989&view=unread#p4218). It also uses the check_mem() function from the Arduino playground. I wanted to compare their results (which turned out favorably).

This code has no malloc()'s and seems to predictably compute the memory sizes of each section illustrated in the Memory Areas and Using malloc(), avr-libc: Memory Areas and Using malloc().

Given all the disclaimers and caveats you've mentioned before, this sketch shows how adding or subtracting some number of characters to the strings in the loop() function predictably adds or subtracts that exact number of bytes from the .data section size and reduces free memory.

Thanks again for posting great explanations and pointing out the Pololu link. Also, thanks to Paul at Pololu for his great get_free_memory() function illustrating how to get a handle on this memory issue.

You should be able to cut-and-paste and pop this code into a new Arduino sketch, download and check out the memory profile which should look something like this:
get_free_memory() reports [1109] (bytes) which must be > 0 for no heap/stack collision
ram size=[2047] bytes decimal
.data size=[734] bytes decimal
.bss size=[183] bytes decimal
heap size=[6] bytes decimal
stack size=[16] bytes decimal
free size=[1108] bytes decimal

Why isn't a memory profile like this included in the Arduino IDE???

Marc

/*
 * Cut-and-pasted from www.arduino.cc playground section for determining heap and stack pointer locations.
 * http://www.arduino.cc/playground/Code/AvailableMemory
 *
 * Also taken from the Pololu thread from Paul at: http://forum.pololu.com/viewtopic.php?f=10&t=989&view=unread#p4218
 *
 * Reference figure of AVR memory areas .data, .bss, heap (all growing upwards), then stack growing downward:
 * http://www.nongnu.org/avr-libc/user-manual/malloc.html
 *
 */

extern unsigned int __data_start;
extern unsigned int __data_end;
extern unsigned int __bss_start;
extern unsigned int __bss_end;
extern unsigned int __heap_start;
//extern void *__malloc_heap_start; --> apparently already declared as char*
//extern void *__malloc_margin; --> apparently already declared as a size_t
extern void *__brkval;
// RAMEND and SP seem to be available without declaration here

int16_t ramSize=0;   // total amount of ram available for partitioning
int16_t dataSize=0;  // partition size for .data section
int16_t bssSize=0;   // partition size for .bss section
int16_t heapSize=0;  // partition size for current snapshot of the heap section
int16_t stackSize=0; // partition size for current snapshot of the stack section
int16_t freeMem1=0;  // available ram calculation #1
int16_t freeMem2=0;  // available ram calculation #2


/* This function places the current value of the heap and stack pointers in the
 * variables. You can call it from any place in your code and save the data for
 * outputting or displaying later. This allows you to check at different parts of
 * your program flow.
 * The stack pointer starts at the top of RAM and grows downwards. The heap pointer
 * starts just above the static variables etc. and grows upwards. SP should always
 * be larger than HP or you'll be in big trouble! The smaller the gap, the more
 * careful you need to be. Julian Gall 6-Feb-2009.
 */
uint8_t *heapptr, *stackptr;
uint16_t diff=0;
void check_mem() {
  stackptr = (uint8_t *)malloc(4);          // use stackptr temporarily
  heapptr = stackptr;                     // save value of heap pointer
  free(stackptr);      // free up the memory again (sets stackptr to 0)
  stackptr =  (uint8_t *)(SP);           // save value of stack pointer
}


/* Stack and heap memory collision detector from: http://forum.pololu.com/viewtopic.php?f=10&t=989&view=unread#p4218
 * (found this link and good discussion from: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1213583720%3Bstart=all )
 * The idea is that you need to subtract your current stack pointer (conveniently given by the address of a local variable)
 * from a pointer to the top of the static variable memory (__bss_end). If malloc() is being used, the top of the heap
 * (__brkval) needs to be used instead. In a simple test, this function seemed to do the job, showing memory gradually
 * being used up until, with around 29 bytes free, the program started behaving erratically.
 */
//extern int __bss_end;
//extern void *__brkval;

int get_free_memory()
{
  int free_memory;

  if((int)__brkval == 0)
     free_memory = ((int)&free_memory) - ((int)&__bss_end);
  else
    free_memory = ((int)&free_memory) - ((int)__brkval);

  return free_memory;
}






void setup()                    // run once, when the sketch starts
{
  Serial.begin(57600);
}

void loop()                     // run over and over again
{
  Serial.print("\n\n--------------------------------------------");
  Serial.print("\n\nLOOP BEGIN: get_free_memory() reports [");
  Serial.print( get_free_memory() );
  Serial.print("] (bytes) which must be > 0 for no heap/stack collision");
  

  Serial.print("\n\nSP should always be larger than HP or you'll be in big trouble!");
  
  check_mem();

  Serial.print("\nheapptr=[0x"); Serial.print( (int) heapptr, HEX); Serial.print("] (growing upward, "); Serial.print( (int) heapptr, DEC); Serial.print(" decimal)");
  
  Serial.print("\nstackptr=[0x"); Serial.print( (int) stackptr, HEX); Serial.print("] (growing downward, "); Serial.print( (int) stackptr, DEC); Serial.print(" decimal)");
  
  Serial.print("\ndifference should be positive: diff=stackptr-heapptr, diff=[0x");
  diff=stackptr-heapptr;
  Serial.print( (int) diff, HEX); Serial.print("] (which is ["); Serial.print( (int) diff, DEC); Serial.print("] (bytes decimal)");
  
  
  Serial.print("\n\nLOOP END: get_free_memory() reports [");
  Serial.print( get_free_memory() );
  Serial.print("] (bytes) which must be > 0 for no heap/stack collision");
  
  
  // ---------------- Print memory profile -----------------
  Serial.print("\n\n__data_start=[0x"); Serial.print( (int) &__data_start, HEX ); Serial.print("] which is ["); Serial.print( (int) &__data_start, DEC); Serial.print("] bytes decimal");

  Serial.print("\n__data_end=[0x"); Serial.print((int) &__data_end, HEX ); Serial.print("] which is ["); Serial.print( (int) &__data_end, DEC); Serial.print("] bytes decimal");
  
  Serial.print("\n__bss_start=[0x"); Serial.print((int) & __bss_start, HEX ); Serial.print("] which is ["); Serial.print( (int) &__bss_start, DEC); Serial.print("] bytes decimal");

  Serial.print("\n__bss_end=[0x"); Serial.print( (int) &__bss_end, HEX ); Serial.print("] which is ["); Serial.print( (int) &__bss_end, DEC); Serial.print("] bytes decimal");

  Serial.print("\n__heap_start=[0x"); Serial.print( (int) &__heap_start, HEX ); Serial.print("] which is ["); Serial.print( (int) &__heap_start, DEC); Serial.print("] bytes decimal");

  Serial.print("\n__malloc_heap_start=[0x"); Serial.print( (int) __malloc_heap_start, HEX ); Serial.print("] which is ["); Serial.print( (int) __malloc_heap_start, DEC); Serial.print("] bytes decimal");

  Serial.print("\n__malloc_margin=[0x"); Serial.print( (int) &__malloc_margin, HEX ); Serial.print("] which is ["); Serial.print( (int) &__malloc_margin, DEC); Serial.print("] bytes decimal");

  Serial.print("\n__brkval=[0x"); Serial.print( (int) __brkval, HEX ); Serial.print("] which is ["); Serial.print( (int) __brkval, DEC); Serial.print("] bytes decimal");

  Serial.print("\nSP=[0x"); Serial.print( (int) SP, HEX ); Serial.print("] which is ["); Serial.print( (int) SP, DEC); Serial.print("] bytes decimal");

  Serial.print("\nRAMEND=[0x"); Serial.print( (int) RAMEND, HEX ); Serial.print("] which is ["); Serial.print( (int) RAMEND, DEC); Serial.print("] bytes decimal");

  // summaries:
  ramSize   = (int) RAMEND       - (int) &__data_start;
  dataSize  = (int) &__data_end  - (int) &__data_start;
  bssSize   = (int) &__bss_end   - (int) &__bss_start;
  heapSize  = (int) __brkval     - (int) &__heap_start;
  stackSize = (int) RAMEND       - (int) SP;
  freeMem1  = (int) SP           - (int) __brkval;
  freeMem2  = ramSize - stackSize - heapSize - bssSize - dataSize;
  Serial.print("\n--- section size summaries ---");
  Serial.print("\nram   size=["); Serial.print( ramSize, DEC ); Serial.print("] bytes decimal");
  Serial.print("\n.data size=["); Serial.print( dataSize, DEC ); Serial.print("] bytes decimal");
  Serial.print("\n.bss  size=["); Serial.print( bssSize, DEC ); Serial.print("] bytes decimal");
  Serial.print("\nheap  size=["); Serial.print( heapSize, DEC ); Serial.print("] bytes decimal");
  Serial.print("\nstack size=["); Serial.print( stackSize, DEC ); Serial.print("] bytes decimal");
  Serial.print("\nfree size1=["); Serial.print( freeMem1, DEC ); Serial.print("] bytes decimal");
  Serial.print("\nfree size2=["); Serial.print( freeMem2, DEC ); Serial.print("] bytes decimal");
  
  delay(3000);

I agree that reporting memory usage at compile time (from the elf file) would be very welcome, but for runtime use, having the get_memory_free function in a library would make the basic info accessable through just including the header instead of having to copy and past the code - Here is the function as a library:

Save as MemoryFree.h in a library directory called MemoryFree

// memoryFree header

#ifndef      MEMORY_FREE_H
#define MEMORY_FREE_H

#ifdef __cplusplus
extern "C" {
#endif

int freeMemory();

#ifdef  __cplusplus
}
#endif

#endif

Save as MemoryFree.c in the same library directory

 /*
 * MemoryFree.c
 * returns the number of free RAM bytes
*/

#include "WProgram.h"  
#include "MemoryFree.h"

extern unsigned int __data_start;
extern unsigned int __data_end;
extern unsigned int __bss_start;
extern unsigned int __bss_end;
extern unsigned int __heap_start;
extern void *__brkval;


int freeMemory()
{
  int free_memory;

  if((int)__brkval == 0)
     free_memory = ((int)&free_memory) - ((int)&__bss_end);
  else
    free_memory = ((int)&free_memory) - ((int)__brkval);

  return free_memory;
}

Usage in a sketch:

#include <MemoryFree.h>

void setup()                    // run once, when the sketch starts
{
  Serial.begin(9600);
}

void loop()                     // run over and over again
{
  Serial.print("freeMemory() reports ");
  Serial.println( freeMemory() );
}