Strange memory problem......

Hi all,

Using a MEGA R3, I was testing a program that loads a LOT of font bitmaps into PROGMEM to be used on a KS-108 type display (a VFD actually).

I have the VFD and a 16x2 LCD connected so that I can view font samples on the VFD while at the same time viewing the font NAME on the LCD (so I know which ones need tweaking).

Anyway, the problem is this: If I load all the fonts into PROGMEM, the LCD display won’t initialize.

If the compiled sketch is over about 80K, that’s where the problem is.

Most everything is in PROGMEM… I’ve got 6904 bytes of ram free, so it’s not a ram problem.

I tried something which may point to the problem:

I did the LCD.begin (16, 2); and a LCD.print (“Hello”); followed by a 5 second delay. The “Hello” printed on the LCD, then after the delay expired the VFD initialized and began to display the font samples, but the LCD just sat there dead, saying “Hello” instead of displaying the font names.

I doubt this will help, but here’s the sketch:

#include <Noritake_VFD_GUU100.h>
#include <fonts/allFonts.h>
#include <LiquidCrystal.h>

// LCD pins connections
#define _V0  33 // unused in VFD
#define _RS  32
#define _RW  31
#define _EN  30
#define _D7  29
#define _D6  28
#define _D5  27
#define _D4  26
#define _D3  25
#define _D2  24
#define _D1  23
#define _D0  22

static Noritake_VFD_GUU100 VFD;

static LiquidCrystal LCD (_RS, _RW, _EN, _D0, _D1, _D2, _D3, _D4, _D5, _D6, _D7);

uint8_t c;

char *buffer; // pointer for malloc'd buffer

int main (void)
{
	init ();
	Serial.begin (115200);

	VFD.init ();
	LCD.begin (16, 2);

	while (1) {
		for (c = 0; c < 30; c++) {
			_delay_ms (5000);
			switch (c) {
				case 0: { VFD.setFont (ati_8x16); demo ("ati_8x16"); break; }
				case 1: { VFD.setFont (coco3); demo ("coco3"); break; }
				case 2: { VFD.setFont (deco_8x16); demo ("deco_8x16"); break; }
				case 3: { VFD.setFont (dutch_6x8); demo ("dutch_6x8"); break; }
				case 4: { VFD.setFont (fixednums15x31); demo ("fixednums15x31"); break; }
				case 5: { VFD.setFont (fixednums7x15); demo ("fixednums7x15"); break; }
				case 6: { VFD.setFont (fixednums8x16); demo ("fixednums8x16"); break; }
				case 7: { VFD.setFont (font_20x32_digits); demo ("font_20x32_digits"); break; }
				case 8: { VFD.setFont (font_5x7_data); demo ("font_5x7_data"); break; }
				case 9: { VFD.setFont (font_5x7); demo ("font_5x7"); break; }
				case 10: { VFD.setFont (font_5x8_descender); demo ("font_5x8_descender"); break; }
				case 11: { VFD.setFont (font_5x8); demo ("font_5x8"); break; }
				case 12: { VFD.setFont (font_6x8); demo ("font_6x8"); break; }
				case 13: { VFD.setFont (glcdfont); demo ("glcdfont"); break; }
				case 14: { VFD.setFont (gothic_8x16); demo ("gothic_8x16"); break; }
				case 15: { VFD.setFont (hd44780u); demo ("hd44780u"); break; }
		//		case 16: { VFD.setFont (ibm_8x14); demo ("ibm_8x14"); break; }
				case 17: { VFD.setFont (ibm_8x16); demo ("ibm_8x16"); break; }
				case 18: { VFD.setFont (ibm_8x8); demo ("ibm_8x8"); break; }
				case 19: { VFD.setFont (italics_8x16); demo ("italics_8x16"); break; }
				case 20: { VFD.setFont (moderne_8x16); demo ("moderne_8x16"); break; }
				case 21: { VFD.setFont (nums_4x7); demo ("nums_4x7"); break; }
				case 22: { VFD.setFont (ocr_8x16); demo ("ocr_8x16"); break; }
				case 23: { VFD.setFont (roman_8x16); demo ("roman_8x16"); break; }
				case 24: { VFD.setFont (scrawl_8x16); demo ("scrawl_8x16"); break; }
				case 25: { VFD.setFont (sev_seg_8x16); demo ("sev_seg_8x16"); break; }
				case 26: { VFD.setFont (slant_8x16); demo ("slant_8x16"); break; }
				case 27: { VFD.setFont (swiss_8x16); demo ("swiss_8x16"); break; }
		//		case 28: { VFD.setFont (thin14); demo ("thin14"); break; }
				case 29: { VFD.setFont (thin16); demo ("thin16"); break; }
				case 30: { VFD.setFont (thin8); demo ("thin8"); break; }
				case 31: { VFD.setFont (tseng_8x16); demo ("tseng_8x16"); break; }
			}
		}
	}
}

void demo (const char *str)
{
	// display stuff on the VFD
	VFD.clearScreen ();
	VFD.setLine (0, 0);
	VFD.print ("ABCDEFGHIJKLMNOPQRSTUVWXYZ");
	VFD.setLine (0, 1);
	VFD.print ("abcdefghijklmnopqrstuvwxyz");
	VFD.setLine (0, 2);
	VFD.print ("~!@#$%^&*()_+`-={}|[]\\;':\",./<>?");
	VFD.setLine (0, 3);
	VFD.print ("01234567890");

	// show the font name
	LCD.clearScreen ();
	LCD.setLine (0, 0);
	LCD.print (str);

	// show memory free
	buffer = (char *) malloc (24 * sizeof (char));
	sprintf (buffer, "RAM: %4lu B", freeMemory ());
	LCD.setLine (0, 1);
	LCD.print (buffer);
	free (buffer);
}

/// end of test.ino

I recall past comments on the forum that the bootloader may have some problem with handling large amounts of PROGMEM data in large programs. 80K is pretty large. I don't know the details, but maybe you can track it down. Also, westfw, the bootloader author, would know. Maybe search on "PROGMEM, bootloader, westfw".

oric_dan: I recall past comments on the forum that the bootloader may have some problem with handling large amounts of PROGMEM data in large programs. 80K is pretty large. I don't know the details, but maybe you can track it down. Also, westfw, the bootloader author, would know. Maybe search on "PROGMEM, bootloader, westfw".

I'm not sure I understand what you mean. The compiled sketch loads just fine. The bootloader is doing it's job.

(edit to add): I think I see what you mean. Maybe it appears to load but in reality not all is loading. I can check that by burning the code directly in with my ISP - Thanks for the idea!

The problem is that something "chokes" when around 80+K is used. It's the same, working binary machine code, just loaded a bit higher in memory as it gets bigger.

In use, I will never have all the fonts loaded (usually no more than 2 or three and most usually just one), but I'm trying to figure out what the problem is. If I have a microcontroller with 256K of flash, I want to be able to use all of it (less the space the bootloader uses of course - to address the Captain Obvious posters). If the chip chokes somewhere when more than 80K is used, obviously there is a problem or bug somewhere, and obviously if the chip has 256K of memory it's supposed to be able to USE all of it.

(which gives me an idea... this probably isn't it, but it MAY be bad flash cells on that particular board - easily checked by using a different board - of which I have several).

I doubt that's it though... I may have to go through the tedious task of finding exactly where it chokes and then try to debug it. I can also load the code directly with my ISP MKII and see if it works. I'll fill 'er up with 200K of junk and see if it still works.

Thanks for the idea!

-- Roger

The use of malloc and free is not a good idea with the arduino. It leads to fragmentation of sRam. I’d suggest replacing it with a simple char array.

I haven’t looked into this at all, but how do the Arduino mega’s handle pointers when the address space is more than 64K?

Do you use the latest Arduino IDE beta? It solved a memory problem for me and my Megas 2560.

KenF: The use of malloc and free is not a good idea with the arduino. It leads to fragmentation of sRam. I'd suggest replacing it with a simple char array.

Actually, I did that to see if I had a memory leak. I suppose that checking free ram WITHOUT doing malloc/free would have told me anyway. I was doing this at around 4:00 AM... brain was in powersave mode. :)

guix: Do you use the latest Arduino IDE beta? It solved a memory problem for me and my Megas 2560.

No I use 1.0.6 with custom modifications (nothing to do with memory allocation though).

My custom version has:

  • Option in Preferences to compile and link floating point library or not
  • More serial speeds in the selector (up to 921600 baud)
  • A "build, then monitor" button that kicks in the serial monitor after an upload
  • Fixed the serial monitor so it handles CR, LF, backspace, tab, bell, etc...
  • Options in Preferences to set paths to compiler, avrdude, etc (to use my own compiled AVR toolchain)
  • Fixed the horrible screwed up multiple IDE fonts problem
  • Option to record status & error messages to disk.
  • Option to set certain AVRDUDE command line options
  • Hooks to call external programs on events (such as load sketch, compile sketch, upload sketch). These are used to (optionally) start or stop external programs (for example stop an external terminal program to free the serial port before upload, then re-enable it after)

There are a few tests I can do to rule out (or implicate) the IDE:

  • upload a compiled sketch via my ISP
  • Compile the code directly with AVR-GCC (not even using the IDE) and then upload the .hex file with the ISP

Especially the second one will for sure rule out (or not) the IDE.

econjack: I haven't looked into this at all, but how do the Arduino mega's handle pointers when the address space is more than 64K?

I don't know... but there's something about a "-relax" option on the compiler command line to allow large programs to work. Using this option or not made no difference. I'm going to try the things I mentioned above (as well as testing a bare vanilla IDE right from a fresh download and also try the 1.5.x beta IDE).

I don't need to use that much PROGMEM anyway... it was just a test and now that I ran into the problem I'm curious and I won't be happy until I know why (and fix it).

OK, i did the search for you myself. Here is one reference to such problems. Don’t know all details or later fixes, but large programs have been an issue.
http://code.google.com/p/arduino/issues/detail?id=1067