Help in programming the Atmega1284 with maniacbug-mighty-1284p.

oric_dan:
For arraysize > 4000, the program does not appear to start. The Serial.print()
statements in setup() do not print either. I also added Serial.begin() that lefty left
out. So, out of one frying pan and into the next.

Actually I didn't add the serial statements to have them output anything useful as that is why there is no Serial.begin(), it's a way someone showed me to keep the compiler from optimizing away the otherwise unused arrays. It the arrays are not referenced in the sketch they won't be created in the final compilation and thus the sketch size wont' grow in size.

Lefty

oric_dan:
avrdude has probably been patched so many times, it's probably a wonder it works
at all. There is probably an int somewheres where there should be a long. arraysize
of 4000 gives a 67808 byte size file that is just over the 16-bit boundary.

Do you suppose the .hex file address indexing is handled correctly during compilation?

I don't know, but I really do believe it's a bootloader/AVRDUDE legacy bug that has not been fully explored or explained let alone solved for the larger mega size chips when using AVRDUDE. Testing with a 644P chip showed no problem loading up to very near it's flash limit size, so I think the problem is when using sketches larger then the 644P size is.

Lefty

I remove programem and made the array smaller, although it should fit the chip it uploads but, does not work.

Binary sketch size: 3,994 bytes (of a 130,048 byte maximum)

Estimated used SRAM memory: 48,365 bytes
//#include <avr/pgmspace.h>   //To store arrays into flash rather then SRAM
// Simple sketch to create large sketch sizes for testing purposes
/*
  Blink
  Turns on an LED on for one second, then off for one second, repeatedly.
 
  This example code is in the public domain.
 */
 
// Pin 13 has an LED connected on most Arduino boards.
// give it a name:
int led = 13;

/* 
 Make arraysize = to 1500 for 328P chip, 4000 for 1280P chip?,
 3600 for 644P chip, xxxx for 1284P,  etc.
*/
const int arraysize= 3000;  // value to mostly fill avalible flash capacity

long myInts0[arraysize]  = {};  //Store initilized array into flash memory
long myInts1[arraysize]  = {};
long myInts2[arraysize]  = {};
long myInts3[arraysize]  = {};

// the setup routine runs once when you press reset:
void setup() {                
  // initialize the digital pin as an output.
  pinMode(led, OUTPUT); 
  Serial.begin(115200);
   digitalWrite(led, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(1000);               // wait for a second
  digitalWrite(led, LOW);    // turn the LED off by making the voltage LOW
  delay(1000);
  int i = random(0,arraysize);      // Work around any optimization for constant values
  Serial.print(myInts0[i]);         //  Access some random element so the array can't be optimized away.
  Serial.print(myInts1[i]);         //  Access some random element so the array can't be optimized away.
  Serial.print(myInts2[i]);         //  Access some random element so the array can't be optimized away.
  Serial.print(myInts3[i]);         //  Access some random element so the array can't be optimized away.
}

// the loop routine runs over and over again forever:
void loop() {
  
  digitalWrite(led, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(1000);               // wait for a second
  digitalWrite(led, LOW);    // turn the LED off by making the voltage LOW
  delay(1000);               // wait for a second
}

SRAM memory issue?????

So, I assume the mega2560 chips also have problems above 64KB/whatever?

I remove programem and made the array smaller, although it should fit the chip it uploads but, does not work.

Code:
Binary sketch size: 3,994 bytes (of a 130,048 byte maximum)

Estimated used SRAM memory: 48,365 bytes

Well that isn't testing with rather large flash sizes and as the 1284P does not have SRAM that big I don't think it's a legit test to hang one's hat on. We need another sketch that creates large sketch flash sizes independent of my sketch to verify it's not a sketch problem but rather just a sketch size problem.

Lefty

oric_dan:
So, I assume the mega2560 chips also have problems above 64KB/whatever?

I would bet so, but don't have a mega2560 board to test, just a arduino mega1280 and a seeeduino mega1280 board that both choke on larger sketch sizes.

Lefty

retrolefty:
Yes, that is a problem, and it's very likely that the sketch is not being loaded correctly after a certain size is reached even though it's less then maximum flash memory size. There is no AVRDUDE error reported after the upload, but obviously the blink program is not running after the upload. It's believed to be a bug in the bootloaders for large (but still legally sized) sketches. For instance when using the Uno's bootloader one can load up a sketch right up to near the flash memory maximum size limit, but with a mega1280 I can't even get close to near max size. Coding Badly is of the opinion that is a bug in older bootloaders and their interaction with AVRDUDE on large code size uploads. So if the 1284P bootloader is just a recompilation based on one of the older mega bootloader programs it may have 'inherited' the same problem. This should be investigated further but I'm afraid I'm not knowledgeable enough in software/bootloaders to be a direct help other then testing.

I guess a first step is for someone to try a different method to developing large sketch sizes just to make sure it's not my sketch causing the problem symptom. Anyone up for the challenge?

Lefty

There's a problem related to a bug inside the GNU avr-gcc compiler. If I remember well, the compiler cannot create code that can directly access data in FLASH is it is stored over the 64 kB limit. Latest versions of avr-gcc aren't affected by this bug, the one that is included with the Ardino IDE is.
You can upgrade the Avr toolchain to fix it. I did it for my Linux box.

cyclegadget:
SRAM memory issue?????

Yeah, 1284 has only 16KB of SRAM. Setting arraysize up to 960 = 44960 = 15360 bytes
works, but arraysize = 1000 crashes, since there isn't room left for the background functions
and serial buffers.

Ok, I do have a mega2560 board [although only used a couple of times], and I tested
lefty's sketch. It works the same as on the 1284 chip. For arraysize = 4000, it uploads
and runs, but at arraysize = 7000, it uploads and verifies ok, but the sketch doesn't run.

And I am using Enhanced IDE ERW v.1.03 here, so the toolchain fix doesn't seem to have
gotten over into the latest version here either.

And yet another problem. With ERW 1.03, starting Serial Monitor does NOT send a reset
to the mega2560 board. I have to reset manually. Sheesh.

Ok, I went back to test Serial Monitor on both IDE 1.0 and 1.03, and they both send a
reset to the atmega2560 board when you open Serial Monitor. So, triple sheesh - it's
aways something around this place!

starting Serial Monitor does NOT send a reset

The ERW author felt is was best to remove the reset feature/problem. It depends how you look at it. If you read some of his posts in the last few pages of his thread, he explains how to modify the serial monitor behavior.

leo72:

retrolefty:
Yes, that is a problem, and it's very likely that the sketch is not being loaded correctly after a certain size is reached even though it's less then maximum flash memory size. There is no AVRDUDE error reported after the upload, but obviously the blink program is not running after the upload. It's believed to be a bug in the bootloaders for large (but still legally sized) sketches. For instance when using the Uno's bootloader one can load up a sketch right up to near the flash memory maximum size limit, but with a mega1280 I can't even get close to near max size. Coding Badly is of the opinion that is a bug in older bootloaders and their interaction with AVRDUDE on large code size uploads. So if the 1284P bootloader is just a recompilation based on one of the older mega bootloader programs it may have 'inherited' the same problem. This should be investigated further but I'm afraid I'm not knowledgeable enough in software/bootloaders to be a direct help other then testing.

I guess a first step is for someone to try a different method to developing large sketch sizes just to make sure it's not my sketch causing the problem symptom. Anyone up for the challenge?

Lefty

There's a problem related to a bug inside the GNU avr-gcc compiler. If I remember well, the compiler cannot create code that can directly access data in FLASH is it is stored over the 64 kB limit. Latest versions of avr-gcc aren't affected by this bug, the one that is included with the Ardino IDE is.
You can upgrade the Avr toolchain to fix it. I did it for my Linux box.

I guess that could be it, but I think I've read it speculated in the past that it is something to do with the AVRDUDE/bootloader protocol of stk500 or stk500v1 that is responsible for this behaviour.

Sheesh, you would think as long as the arduino mega1280/2560 boards have been available that the exact problem and or solution would have been well known and published by now? I sure it's not common to write sketches greater then 64KB but you think a few have had the need?

Lefty

Sheesh, you would think as long as the arduino mega1280/2560 boards have been available that the exact problem and or solution would have been well known and published by now? I sure it's not common to write sketches greater then 64KB but you think a few have had the need?

Sheesh, that makes 4 sheeshs now. I'm not too surprised that Arduinophiles in general aren't
writing too many programs over 64KB, but you'd think some might have. For my part, the
control s.w. for my robot tank exceeded 32KB code and 2KB RAM way last spring, so I've
wanted a larger space chip in a small size board [ala 1284 & UNO formfactor] to keep growing,
but even 64KB will be gone quickly. Hmmm.

It's also amazing the 1284 chips have been the object of 2000-3000 posts now on multiple
threads, and the limitations aren't flagged everywheres. Guess I'll have to learn how to
bypass the bootloader.

Is it possible to develop code using the Arduino IDE, with its libraries etc, and upload those
.hex files via a regular ICSP programmer, or will they not run without having the bootloader
to bootstrap off of?

Is it possible to develop code using the Arduino IDE, with its libraries etc, and upload those
.hex files via a regular ICSP programmer, or will they not run without having the bootloader
to bootstrap off of?

Sure, the current IDE supports in the files/menu option of upload sketch using programmer that will use your selected programmer (such as arduinoISP) to burn the sketch directly into flash without needing a bootloader, in fact if the chip had a bootloader installed prior to uploading using programmer, it will end up with no bootloader, just the sketch when completed.

HOWEVER the big question is will this change the large sketch not large enough problem? That depends on if the root cause is the bootloader/AVRDUDE or if the compiler version is the root cause. I'll see if I can play around and see if I get different results using upload using programmer on my mega1280 board.

Lefty

So I tried uploading large size mega1280 sketches without using the serial bootloader, but rather the upload using programmer option using another arduino running the arduinoISP sketch as the hardware programmer. The results were no change in symptom, I still could not load a mega1280 board to near it's maximum flash size limit.

So that would tend to appear to lend credence that it's a tool chain version cause (Although there may still be an issue with AVRDUDE and the protocol it uses with the arduinoISP programmer sketch I guess, we would need someone with a different hardware programmer, such as a AVRISP mkII to test out the problem), and explain the reason the problem has not be addressed even if known by the arduino developer's group. I have read them discussing about upgrading the AVR tool chain for quite a long time but there appears to be a lot of resistance as the upgrade carries the possibility of breaking a lot of older sketches and library code and unless there was clear benefits identified, some question upgrading the tool chain?

Oh well, it's not as if I thought I could fix the problem by myself, I just wanted to more clearly understand if there was a real problem or not ( it's real ), what it's root cause is (probably the tool chain? ), and what if any fix one might apply to the situations (I'm stuck). So I've kind of reached my limit of usefulness and will just stand back unless asked something about the issue.

Lefty

lefty, do you know if avrdude was built by Atmel as part of an official tool set, or something
jinned up by freaks? If by Atmel, one might think they would have proofed it on their larger chips,
and it would be 100%. Microchip's tools like MPLAB are continually updated to reflect their newer
chips.

There's a problem related to a bug inside the GNU avr-gcc compiler. If I remember well, the compiler cannot create code that can directly access data in FLASH is it is stored over the 64 kB limit. Latest versions of avr-gcc aren't affected by this bug, the one that is included with the Ardino IDE is.
You can upgrade the Avr toolchain to fix it. I did it for my Linux box.

leo, have you tried uploading files > 64KB to your boards, and had the programs run successfully?

AVRDUDE is it's own independent open source project with no 'official' relationship to Atmel or Arduino that I am aware of.

Lefty

retrolefty:
AVRDUDE is it's own independent open source project with no 'official' relationship to Atmel or Arduino that I am aware of.

AVR Downloader/UploaDEr - Summary [Savannah]

Lefty

I notice that the avrdude version installed with IDE v1.03 is dated 2 Sept 2011, so it's fairly recently
updated. Do you know any of those people who you could contact about the > 64KB issue?

OTOH, the avrdude version supplied with enhanced ERW IDE v.1.03 is dated 5 Nov 2012, and that
IDE still didn't upload working code, re post #75. It sounds like there may be 3 problems needing
fixing: IDE toolchain, avrdude, optiboot/others.

I don't believe the issue(s) have anything to do with AVRDUDE directly. It's what protocol type any specific bootloader code is using to work with AVRDUDE, which can handle several different protocols of which some have no problem writing to large flash memory sizes such as the SK500v2 protocol. That would effect the ability to load large sketches I believe. The other related issue but different is the ablitiy to properly create large sketches using the PROGMEM attribute, which seems to be a AVR gcc compiler bug/limitation.

I truly believe we have not uncovered any new problems, or problems that don't have a work around. For instance I'm sure someone can easily create super large sketches that don't rely on using the PROGMEM command. I also believe the a 'correct' bootloader for any specific chip type that uses the SK500v2 protocol with AVRDUDE would be able to load such large sketches. It's just identifying all the correct pieces and parts and implementing them correctly in the custom hardware folder. I suspect the current 1284p bootloader is correct and would test ok with a 'real' sketch of large proportions if they don't utilize PROGMEM to build that sketch.

At least that is my current take on what we are seeing.

Lefty

I started a new thread on this issue, see if we can get any enlightenment?

http://arduino.cc/forum/index.php/topic,143876.0.html