Go Down

Topic: Mega2560 fails when sketch size is larger than approx. 52418 bytes (Read 4445 times) previous topic - next topic

UdoZ

Hi, I am using an Arduino Mega2560, programmed via Arduino 0022 running under Windows 7 x64.
During development of my project I have uploaded bit by bit of code, in total several hundred times successfully. The project and so the sketch size was growing fine. However, since my code has grown beyond 52418 bytes, which is well below the limit of the device, it fails to load correctly.

I have verified that the problem is not with my latest code additions. (I removed the new bits -> uploads and runs fine. I then added a "void dummy() routine" which consists only of Serial.print("some text"); statements. This void dummy routine is never called. It only adds code to the sketch. By carefully adding more and more lines of similar unused Serial.print statements, I could show that the sketch loads and runs fine up to 52418 bytes. The moment I go beyond that limit, the sketch seems still to upload fine, at least that's reported by Arduino 022. The sketch even starts to run (part of a connected display is written), but then stops. Obviously the code loaded is corrupt, indicating an issue with the Arduino 0022 environment.

As it stands my project cannot  be realised as I really need more usable code space. That's why I selected the Mega2560 in the first place. Does anybody have advice what I should do or try next. Thanks in advance.
Best regards, Udo

johnwasser

In my experience, when a sketch fails to run to completion after adding debug statements (Serial.print("Some text");) it's because my poor little processor has run out of RAM to put all those string constants into.  :(

You can load string constants from FLASH memory as needed to save a lot of RAM space.
Send Bitcoin tips to: 1G2qoGwMRXx8az71DVP1E81jShxtbSh5Hp

Grag38


UdoZ

Hi, thanks for that but I have a comment/question: I thought also that RAM could be a problem, hence I added for testing purposes in the last sketch before failure before setup() an Integer Array of dimension [1000] and the sketch did not fail to upload. Alternatively when I added the extra code, which did not have too many statements, it failed. Nevertheless, I will follow this up more. However, this brings me to the next question. Why would a Serial.print("some text") statement use lots of RAM? I had expected that the "some text" is added to Program storage as this is rather a constant than a char array. I talk about text which never changes and if I had to programm that in Assembler I surely would not have placed anything into RAM. A simply indirect reference to a "text location" would do.Maybe I spent too many hours (well in the past) programming in Assembler when RAM was rather tiny and expensive.

johnwasser

I think the FLASH is in a different address space than the RAM.  You can execute the program out of FLASH but the data pointers are all assumed to point to RAM.  If you pass a FLASH address/pointer to a function like Serial.print() which takes a RAM address/pointer you will get the wrong data.  For that reason, all of your initialized global variables and string constants are copied from FLASH to RAM before your program begins.  Then the RAM addresses are used.

You can explicitly put constants into the FLASH address space but you have to copy them to RAM as needed if you are going to pass them to a function that takes a RAM address pointer.

I don't know if the compile/upload will fail if you have more global and static variables than your RAM can hold.  Certainly your program will fail if your data overwrites the stack space.  Unfortunately there seems to be no detection of a collision between heap and stack.
Send Bitcoin tips to: 1G2qoGwMRXx8az71DVP1E81jShxtbSh5Hp

nickgammon

You have 256 Kb of EEPROM but only 8 Kb of RAM. There is quite a potential here for shooting yourself in the foot.

Every time you add something like:

Code: [Select]
Serial.print("some text");

... you increase the EEPROM usage and the RAM usage, as John said. This is because the "raw" C code expects to find strings in RAM. You can use the pgmspace library to save that copying from EEPROM (which you have lots of) to RAM (which you don't).

See:

http://www.arduino.cc/en/Reference/PROGMEM

Quote
I added for testing purposes in the last sketch before failure before setup() an Integer Array of dimension [1000] and the sketch did not fail to upload.


Did it not upload, or did it upload, but not run?

I have known the compiler to optimize away arrays you don't use. It looks at it and says "hey, he never used that, I'll get rid of it".
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

nickgammon

There is a function you can use to see how much RAM you have spare.

Add this and add a display in setup to show what figure this returns:

Code: [Select]
int memoryFree()
{
extern unsigned long __bss_end;
extern void *__brkval;

 int freeValue;
 freeValue = 0;

 if ((unsigned long)__brkval == 0)
   freeValue = ((unsigned long)&freeValue) - ((unsigned long)&__bss_end);
 else
   freeValue = ((unsigned long)&freeValue) - ((unsigned long)__brkval);

 return freeValue;
}//end memoryFree()
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

UdoZ

Hi, I used Nick's function to check free RAM and found that a routine, even if never called (for testing purposes only, before you ask) occupies significant RAM. This doesn't make sense to me. Why writing subroutines with local variables when simply adding these routines already occupies RAM. I would expect that a subroutine/function frees its RAM when no longer active. Looking how little RAM I use for global variables etc and how inefficient the compiler seem to use RAM, I am amazed. Maybe I simply misunderstand (and need to add code to release this RAM). As it seems to be, it simply makes little sense to me. Where do I go wrong?

Furthermore I have started to experiment with  PROGMEM (which to me should certainly be part of the compilers standard when compiling those "Serial.print("text which never changes") statements).
Anyhow I declared an array   PROGMEM prog_int16_t iArray[]={0,1,2.......many more elements for testing ....}    and expected that I would need to use " = pgm_read_word_near(iRES + n)" to access for example the array's nth element. However, I get the same result when using an ordinary " = iArray[n]" . Am I allowed to do this or is there a downside to this?

Thanks for all your assistance in optimising my project.

nickgammon

Quote
I used Nick's function to check free RAM and found that a routine, even if never called (for testing purposes only, before you ask) occupies significant RAM.


Could you elaborate on that a bit? Perhaps an example? It shouldn't use any RAM, unless, and this a big unless, it uses variables declared static. Oh, yes, and if it uses strings (or probably any initialized data) then the compiler is probably generating code to copy that to RAM (as described earlier) as part of program startup, whether or not you call the function.

Quote
... which to me should certainly be part of the compilers standard when compiling those "Serial.print("text which never changes") statements) ...


g++ is a pretty generic compiler. It probably expects stuff to be in RAM (not an unreasonable expectation) and the AVR guys have added extra startup code to make sure your strings are in fact in RAM.

Quote
... and expected that I would need to use " = pgm_read_word_near(iRES + n)" to access for example the array's nth element. 


Ah, arrays. I think I would need to see a larger example, but I know that progmem strings can be a pain. If you make the array progmem, you get the string addresses (pointers) in progmem but not the strings themselves. The technique for getting them both into progmem is described on the page linked to above. Not sure about your ints, but I think more than a tiny snippet is needed so I could construct a proper test.
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

UdoZ

Hi Nick, I will look into  the first two points a little more, then come back. Regards the Array, here is a simplified example. It Serial.prints 7, 8, 9 despite using a direct reference to the Array. Maybe I made something wrong and these Integers do still sit in RAM? What do you think?

#include <avr/pgmspace.h>

PROGMEM prog_int16_t iRES[]={7,8,9};

void setup()
{Serial.begin(115200);}

void loop()
{
  for (int i=0; i<3;i++) Serial.println(iRES);
  delay(10000);
}

nickgammon

Your sketch gives me:

Code: [Select]

sketch_may15b.cpp: In function 'void loop()':
sketch_may15b:9: error: call of overloaded 'println(prog_int16_t [3])' is ambiguous
/Applications/Arduino.app/Contents/Resources/Java/hardware/arduino/cores/arduino/Print.h:54: note: candidates are: void Print::println(const String&) <near match>
/Applications/Arduino.app/Contents/Resources/Java/hardware/arduino/cores/arduino/Print.h:56: note:                 void Print::println(char, int) <near match>
/Applications/Arduino.app/Contents/Resources/Java/hardware/arduino/cores/arduino/Print.h:57: note:                 void Print::println(unsigned char, int) <near match>
/Applications/Arduino.app/Contents/Resources/Java/hardware/arduino/cores/arduino/Print.h:58: note:                 void Print::println(int, int) <near match>
/Applications/Arduino.app/Contents/Resources/Java/hardware/arduino/cores/arduino/Print.h:59: note:                 void Print::println(unsigned int, int) <near match>
/Applications/Arduino.app/Contents/Resources/Java/hardware/arduino/cores/arduino/Print.h:60: note:                 void Print::println(long int, int) <near match>
/Applications/Arduino.app/Contents/Resources/Java/hardware/arduino/cores/arduino/Print.h:61: note:                 void Print::println(long unsigned int, int) <near match>


Quote
It Serial.prints 7, 8, 9 despite using a direct reference to the Array.


Well, not for me it doesn't.

Maybe we are using different versions of the IDE. I believe I have the latest (0022).
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

johnwasser

You forgot to mark it as 'code' so the index on iRES was stripped out beacuse it looks like a tag.
Code: [Select]

#include <avr/pgmspace.h>

PROGMEM prog_int16_t iRES[]={7,8,9};

void setup()
{Serial.begin(115200);}

void loop()
{
  for (int i=0; i<3;i++) Serial.println(iRES[i]);
  delay(10000);
}


Put the index back in and it compiles without error.  I guess the compiler knows how to convert a prog_int16_t to an int.

I wonder if Serial.print will take a prog_char_t * and do the conversion.
Send Bitcoin tips to: 1G2qoGwMRXx8az71DVP1E81jShxtbSh5Hp

UdoZ

Hi, my apologies for not inserting code correctly. I was unaware of this. Back to the subject, things seem even more confusing. Check this modified code out and there is a difference whether we write individual Serial.print statements or put them in a loop. The moment we allow the index "i" to go beyond 2 (in the loop), all results of the for loop give nonsense.
Code: [Select]
#include <avr/pgmspace.h>

PROGMEM prog_int16_t iRES[9]={1,2,3,4,5,6,7,8,9};


void setup()
{Serial.begin(115200);}

void loop()
{
 
  Serial.println(iRES[0]);
  Serial.println(iRES[1]);
  Serial.println(iRES[2]);
  Serial.println(iRES[3]);
  Serial.println(iRES[4]);
  Serial.println(iRES[5]);
  Serial.println(iRES[6]);
  Serial.println(iRES[7]);
  Serial.println(iRES[8]);
 
 
  for (int i=0; i<9;i++) {Serial.println(iRES[i], DEC);}

  delay(10000);
}


The result for the individual statements is: 1 2 3 4 5 6 7 8 9 (as you expect), where as the loop gives 8738, 25186,98, 2186 (6 times).
Unfortunately in the simple example it had worked. Now I will investigate further why all my RAM goes despite using lots of unnested subroutines/function calls.

nickgammon


The result for the individual statements is: 1 2 3 4 5 6 7 8 9 (as you expect), where as the loop gives 8738, 25186,98, 2186 (6 times).


Well I wouldn't expect it to work because you have moved to a different sort of memory. A quick disassembly shows why. The compiler was smart enough to optimize away the literal array accesses. It knew which ones you wanted and it knew they couldn't change (being in progmem) so it just literally printed what it knew the numbers were. But with the indexing into an array it couldn't be sure, so it generated memory accesses. The fact that it printed 1, 2, nonsense was just luck. For me it printed:

Code: [Select]
1
2
3
4
5
6
7
8
9
25186
25186
98
25186
25186
25186
25186
25186
25186



You need to use the function that accesses the progmem (in both cases really, you can't depend on optimizations).  You need something like this:

Code: [Select]

  for (int i=0; i<9;i++)
  {
  Serial.println(pgm_read_word (iRES + i), DEC);
  }


Now that works!
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

UdoZ

Thanks for everybodys help. Using PROGMEM resolved my problem.

Go Up