Dynamic allocation of memory

Forum 2005-2010 (read only) Software Development
1

I have been looking a bit into "__malloc_heap_end" lately, and I think that there might me a way to malloc some memory (responsibly).

If anyone is curious and wants to help me look into this, start by reading avr-libc: Memory Areas and Using malloc().

Especially this bit:
"If __malloc_heap_end is 0, the allocator attempts to detect the bottom of stack in order to prevent a stack-heap collision when extending the actual size of the heap to gain more space for dynamic memory"

If anyone has done any research on this matter, please feel free to post some knowledge!

2

Hi InvalidApple

I've been looking into heap management quite a bit. I'll jump right in if that's ok. Can I start by asking what you mean by

there might me a way to malloc some memory (responsibly)

The reason I ask is that I can't see a problem with malloc. So far as I've looked, it seems to do a good job both of not corrupting the stack (if it's used correctly) and of managing the freelist efficiently. (I havn't yet looked at realloc).

Where I DO see a problem is that the stack doesn't appear to show the same courtesy and respect the heap.

With regards to __malloc_heap_end, that seems fine too. It appears to be used both as a flag (zero) where heap is in SRAM and as a "high water mark" if the heap has been moved to external RAM (talking ATmega generally here rather than Arduino specifically). For the Arduino it's correctly set to 0 informing the heap handler that heap is in SRAM and therefore below the stack and therefore it needs to take SP into account when an allocation would extend the heap. In the case of an ATmega with external RAM, if you decided to move the heap there, then __malloc_heap_end would be set to the upper limit you wanted to use for heap and that would be watched instead of SP.

I have some vague un-thought-out idea at the moment of perhaps enhancing the stack push routine so that it checks for heap collision. The problem I'm thinking though is: so what if it does avoid corrupting the heap? What's it then going to do? In a larger system you'd get a "stack overflow" error. But what would you do with it on the Arduino? - anyway - like I said - just a vague thought at the moment.

3

so what if it does avoid corrupting the heap? What's it then going to do?

That is the right question - and really there is no good answer other than not using the heap at all. Also, why would you use it in the first place - what is it you can do with dynamic memory that you can not do with static memory?

In a cooperative multitasking environment, dynamic memory makes sense, because it allows RAM to be shared between independent tasks. However, it is also probably the single biggest source of f-up's in the history of programming. No wonder why Microsoft decided to abandon application control of memory allocation altogether for .NET. On microcontrollers there are no other tasks that can benefit from shared RAM.

To play it safe, you really need to know the worst case scenario (the most RAM you will ever need at the same time for the lifetime of the application). Once you have that answer, you may as well allocate it statically.

4

Forgive me a little fun in my reply - it's all meant in the best possible taste. :slight_smile:

and really there is no good answer other than not using the heap at all

well I'm not sure that even makes "the problem" go away. As I said earlier the heap does NOT crash into the stack if used properly, however the stack DOES crash into the heap. BUT I think the stack doesn't stop there. I need to double check this, but I'm pretty sure the stack will happily trash the .bss segment for the same reason. I wouldn't be surprised if it carried right on down to .data as well. Of course by then the game is well over.

what is it you can do with dynamic memory that you can not do with static memory?

Well speaking as an old C programmer... in a generalized case, building linked lists, binary trees - well - erm... dynamic data structures would have been a challenge without dynamic memory. Compiler writing for one example would have been a nightmare without it. BUT - I take your point - on a small system like the arduino there is less of a case perhaps.

it is also probably the single biggest source of f-up's in the history of programming

Agreed, or at least one of them anyway - but perhaps a tad unfair to blame the tools - if "programmers" who haven't a clue what they're doing are allowed loose it's hardly the fault of the "heap".

No wonder why Microsoft decided to abandon application control of memory allocation altogether for .NET

Woah there - now you've gone way to far. You're surely not citing microsoft as any sort of font of wisdom? :o

To play it safe, you really need to know the worst case scenario (the most RAM you will ever need at the same time for the lifetime of the application).

Well perhaps but I find bottle necks are usually easier to find by tracking them rather than trying to predict them but maybe that's just me.

But back on the case. Personally I don't have a problem with the heap on the Arduino. It's not compulsory to use it but it's there if a need does arise for it. Maybe a competition - best and worst use of the heap - or maybe not.

All good fun. :slight_smile:

5

Before I comment, please understand that the prior post (nor this) in any way is meant as a grumpy - I have all the answers post. There is an element of punch however to give a different perspective on using malloc on microcontrollers. :slight_smile:

As I said earlier the heap does NOT crash into the stack if used properly, however the stack DOES crash into the heap. BUT I think the stack doesn't stop there. I need to double check this, but I'm pretty sure the stack will happily trash the .bss segment for the same reason. I wouldn't be surprised if it carried right on down to .data as well. Of course by then the game is well over.

When two parties crash, it is more of an academic issue who's fault it is, the damage is inflicted nevertheless (the insurance company may see it differently however).

Also, keep in mind that when the stack trash the heap, any live heap pointer into the (now) stack region, will trash the stack - and as you put it - game over.

in a generalized case, building linked lists, binary trees - well - erm... dynamic data structures would have been a challenge without dynamic memory.

Static memory does not limit you in any way with regards to data structures/models. You can pre-allocate your data structures (arrays, struct's, ...) and build your references runtime for any conceivable list or tree you like. Doing it in static memory is just as easy (or difficult) as with dynamic memory.

Woah there - now you've gone way to far.

Microsoft bashing is an art I'm not entertaining. Look at what they do well and learn from that. The .NET memory management model solves a lot of problems difficult to control with other schemes. This is no solution for microcontrollers however.

Well perhaps but I find bottle necks are usually easier to find by tracking them rather than trying to predict them but maybe that's just me.

I think we should do it anyway that best serves the purpose. The point is to find issues before we ship a product.

If you want to study malloc disaster in practice, I suggest you Google Arduino and String library. That's a winning combination for horror stories on "shouldn't this work, unexpected behavior, seem to hang ...). None of the library calls to malloc are checked for errors (and why should they - there is nothing meaningful they can do about it).

6

Hmmmmmmmmmmmm! I think we're going to have to agree to differ on an awful lot of this.

Back to the Arduino specifically. Would it be of benefit to the arduino community:

If memory corruption occurs we leave it as it is and have our application go ferrel with no particular details as to why.

OR if it was practicable

If memory corruption is about to occur, our application closes in a controlled manner and perhaps even makes some info available as to why.

7

and perhaps even makes some info available as to why.

And that is indeed a real dilemma, as an embedded controller with no OS there can not be any assumption on what I/O resources would be available to show or report any kind of run time error. Even the pin 13 led might being used as an input pin.

Lefty

8

there can not be any assumption on what I/O resources would be available

Less of the negative waves, Moriarty. How about, we just haven't thought of one yet?

The stumbling block at the moment (of interest to me at least) is whether it's possible/practicable and if so how, to "trap" each stack push and make a check on the SP.