malloc() => free() = all RAM reclaimed but malloc() again increments, crashes

OK, can anyone tell me what the hell is going on here? It's really a pretty funky, roundabout code structure, but it's the most dynamic way I could find to create a dynamic menu construct from a bit mask and description array.

char prefs_mask[][15] PROGMEM = { "1Cmd Auto .5ms", "1Cmd Auto 1s", "1Cmd Auto 5s", "1Cmd Sticky" };
char FLAGS_mask[][15] PROGMEM = { "EDVF: All dead", "EDV1: 0%", "Overload", "Valid Dischrge", "N/A", "Valid Charge", "CurrTaper Term", "Temp Alarm", "Overcurrent", "Low Temp Fault", "Overvoltage", "N/A", "Charge Control", "0/NiMh 1/LiOn", "PSTAT", "% Rel/Abs Sel" };
char Status_mask[][15] PROGMEM = { "Error Bit 1", "Error Bit 2", "Error Bit 3", "Error Bit 4", "Fully Dischrgd", "Fully Charged", "Discharging", "Initialized", "RemainTimeAlrm", "RemainCapAlarm", "N/A", "TermDischgAlrm", "OverTempAlarm", "N/A", "TermChargeAlrm", "OverChargeAlrm" };
uint16_t prefs = 0xE;

unsigned int DisplayBitList(unsigned int bitField, prog_char* labelsList, byte labelCount) {
  phi_prompt_struct dynaList = topMenu;
  unsigned int x;
  dynaList.high.i=labelCount-1; // Last item of the list is size of the list - 1.
  dynaList.width = lcd_columns-1; // room for scroll bar and nothing more
  dynaList.option |= phi_prompt_list_in_ram;
  dynaList.option &= ~phi_prompt_arrow_dot;
  char* dynaListBuf = (char*)malloc((dynaList.high.i + 1) * 16);
  char** dynaListPtrs = (char**)malloc(dynaList.high.i + 1);
  for (x=0; x<=dynaList.high.i; x++) {
    if (bitField & (1 << x)) dynaListBuf[16 * x] = 126;
    else dynaListBuf[16 * x] = 165;
    strcpy_P(dynaListBuf+(16 * x)+1,(prog_char*)(labelsList + (x*15)));
    *(dynaListPtrs + x) = &dynaListBuf[16 * x];
  }
  dynaList.ptr.list = dynaListPtrs;
  dynaList.low.i = 0;
  while (wait_on_escape(25)) ; // wait for buttons to be up, may have residual press from menu
// debug print & wait, shows free mem and addresses assigned to data by malloc()
  lcd.clear();
  i2c_msg_lcd(PSTR("Free: "));
  lcd.print(freeMemory(), DEC); // is always the same value once i put free() in the right place and the right order
  lcd.setCursor(0,1);
  i2c_msg_lcd(PSTR("d.p.l: "));
  lcd.print((uint16_t)dynaList.ptr.list, HEX); // unpredictably jumps around through addresses, sometimes crapping out on the 2nd re-use (assigning something like 0x1F39 address, subsequently recycling past the end of RAM and rebooting Arduino
  lcd.setCursor(0,2);
  sprintf(i2cBuffer, "dPtrs: %.3X%.3X%.3X", (uint16_t)dynaListPtrs[0], (uint16_t)dynaListPtrs[1], (uint16_t)dynaListPtrs[2]);
  lcd.print(i2cBuffer);
  lcd.setCursor(0,3);
  i2c_msg_lcd(PSTR(" << Back | >> Go "));
  while ((x = wait_on_escape(1000)) == 0) ;
  if (x == 3) return bitField; // if we see it's going to crash on access, we can use back button to abort
  lcd.clear();
  while (wait_on_escape(25)) ; // wait for buttons to be up
  while (1) {
    switch (select_list(&dynaList)) {
      case -3: 
      case -1: 
        free(dynaListPtrs); // free() in FILO order, seems to work better than FIFO
        free(dynaListBuf);
        return bitField;
        break;
      case 1: // select button: toggle selected bit in bitfield
        bitField ^= (1 << dynaList.low.i); // toggle!
        if (bitField & (1 << dynaList.low.i)) *(dynaListBuf + (16 * dynaList.low.i)) = 126;
        else *(dynaListBuf + (16 * dynaList.low.i)) = 165;
        break;
    }
  }
}
void OptionsMenu() {
  prefs = DisplayBitList(prefs,(char*)&prefs_mask, sizeof(prefs_mask) / sizeof(*prefs_mask)); // generate, present, retrieve, and set the global value in one line - yay!
}
......
// part of a much larger function to parse and display various types of data based on the configured command "data type" - this is the bitfield type that I wrote the whole routine for.
      case BATT_BITFIELD:
        switch (singleCmdList.low.i) {
          case Cmd_Flags: DisplayBitList(wordBuffer,(char*)&FLAGS_mask, sizeof(FLAGS_mask) / sizeof(*FLAGS_mask)); break;
          case Cmd_BatteryStatus: DisplayBitList(wordBuffer,(char*)&Status_mask, sizeof(Status_mask) / sizeof(*Status_mask)); break;
          default: lcdQuickPrintWord((unsigned int)wordBuffer,BIN); break;
        }
        break;
......

There's also a big curiosity in using malloc(): the first pointer (in the dynaListPtrs var) is almost always corrupt on the second call (after one malloc(), malloc(), free(), free() cycle), but only when it (seemingly randomly) decides to assign a different address to the allocation than it did the first time. Only the first iteration of pointer assignment, though. That is, the first menu entry on the LCD is garbage data, but all subsequent lines are valid (unless malloc() placed the start address too close to the end, which it often does after 3-4 runs). If it uses the same address block, though, it assigns and writes everything properly, and it always works right on a "fresh boot" with new malloc() calls.

So, just, um... WTF am I doing wrong here? The malloc() header is supposed to be at *(ptr-1), which of course I don't touch here. But it seems like it's really at *(ptr) itself, so when I write the first entry it clobbers the header info. Is that what seems to be going on here...? :drooling_face:

FalconFour I think the char array pointers are word aka two bytes long. So you need twice as much space on your pointer array. Try sizeof(char*)*(high.i+1)

I'm at the mall on wife' blackberry. Will check myself when I get home.:)

Haha... dedication or pure boredom, posting from a (cough*) Blackberry! :)

Yeah, I spent a good 3...4 hours on this routine last night, figuring out how to manipulate these pointers and arrays properly, without using cheap hacks like hard-coded offsets (+2 for the word of a char*). You should see some of the dumb stuff I did, like using "dynaList.option &= !phi_prompt_list_in_ram" and after fixing nearly all the pointer inconsistencies and writing a byte-by-byte dump serial debug routine, I finally realized the reason it was spewing pure gibberish was that it was writing "0" to the option parameter with "!" instead of "~". Facepalm. Then I found all sorts of other stuff, like entirely forgetting to "fill" the pointer list with pointers in the for() loop, and then forgetting to offset the pointer addresses, it was dumping out things like ">Item... tem... m... m. >Moar..." by only offsetting by 2 bytes instead of 0x10 bytes per iteration. Yuck.

But I've now verified that all parameters are being properly written, which is wh--... hmm checks code... yep, X is initialized to 0 as well. ONLY thing I can think of is that maybe the compiler is goofing up and "optimizing" a static address into "dynaListPtrs[0]" but working properly for 1...15. With the levels of complexity added through "pointers to pointers to pointers", it's hard to keep track of... :/

(although I did finally make the connection between "dynaListPtrs[x]" and "*(dynaListPtrs + x)"... that's helped improve readability quite a bit for me!)

edit: I should also quote some numbers... I--... OH MY GOD. (epiphany moment)... NOW I see where the problem is! That's what you meant about the char* pointers taking 2 bytes (I knew that): I'm only accounting for 1 byte per pointer in dynaListPtrs' malloc()! Awh hell... that means I'm clobbering the free pool and stepping into random data... YUCK! Okay, edit time. :D

edit: That did it! You sir, are a legend: char** dynaListPtrs = (char**)malloc((dynaList.high.i + 1) * sizeof(dynaListBuf)); It works perfectly now, always recycling the same address whenever possible, no more memory leaks or overruns! Beautiful, beautiful, beautiful. Now I've got a bitfield-to-strings function, chances are I'm going to come up with a way to mix datatypes (bitfield, string, int) in a dynamic "configuration menu" list next, but I think I need to draw the line somewhere ;)

Glad I could help. Credits also go to wife's blackberry Macy and Caribou Coffee free wifi :) Just remember using sizeof is a good habit when calculating space. I could have made the same mistake but you making it will keep me immune for at least 15 minutes, like how long apple app store remembers your password so you can keep buying withing knowing conciously you made purchase.

So off topic, my ascii panel project is going well. Now I can use a phi-2 shield + arduino as a vt100 terminal to dump data to it with tab and return for formatting. I can also invoke phi_prompt via serial. Imagine your project needs lcd and keys you hook up to this panel via serial and you can even display yn_dialog to ask user for answer, no function call need, just Serial.print("\eBEngage photonic cannon?~"); and wait on serial for the panel to get back to you with 'Y' or 'N'. \e is escape, which is followed by control function B, indicating yn_dialog. Then a sentence of prompt ending with ~.

liudr: I could have made the same mistake but you making it will keep me immune for at least 15 minutes, like how long apple app store remembers your password so you can keep buying withing knowing conciously you made purchase.

LOL'd hard! XD

liudr: So off topic, my ascii panel project is going well. Now I can use a phi-2 shield + arduino as a vt100 terminal to dump data to it with tab and return for formatting. I can also invoke phi_prompt via serial. Imagine your project needs lcd and keys you hook up to this panel via serial and you can even display yn_dialog to ask user for answer, no function call need, just Serial.print("\eBEngage photonic cannon?~"); and wait on serial for the panel to get back to you with 'Y' or 'N'. \e is escape, which is followed by control function B, indicating yn_dialog. Then a sentence of prompt ending with ~.

Oh, sweet! That sounds really awesome. I remember the ol' days of escape codes, but wasn't savvy enough to understand them back then. Seems like a cool way to control simple functions. Oh great, another toy to play around with! :D

like hard-coded offsets (+2 for the word of a char*).

"+ sizeof(char*)" is similarly hard-coded, but is by no means a hack, and will make your code more portable.

Plus, I think sizeof() is a macro that gets replaced by a constant number during compilation so there is no overhead like function calls require. On the other hand, you can't use it to dynamically determine stuff length either. BTW, you can't use sizeof() in preprocessor directives like #if #define etc, according to some online resources. I've never tried personally.

http://c-faq.com/cpp/ifexpr.html

BTW, you can't use sizeof() in preprocessor directives like #if #define etc

That interesting - I've been using #define SIZEOF_ARRAY(x)  (sizeof(x)/sizeof((x)[0])) for a long time without mishap. "sizeof" is also used in "offsetof", which is also a macro, IIRC.

AWOL:

BTW, you can't use sizeof() in preprocessor directives like #if #define etc

That interesting - I've been using #define SIZEOF_ARRAY(x)  (sizeof(x)/sizeof((x)[0])) for a long time without mishap. "sizeof" is also used in "offsetof", which is also a macro, IIRC.

There is no problem with your usage in #define.

However, sizeof will not be evaluated by the pre-processor in #if. For example:

if 1 < 2

// ok

endif

if sizeof(char) < sizeof(int)

// not ok ... compilation error

endif

The sizeof operator is evaluated by the compiler after the pre-processor stage.

Good to know. Like I said, I read of internet but never tested it myself :sweat_smile: