Building custom bootloader?

More questions: I think I discovered that the linker command line options for the location of the .version section in the bootloader needs to be changed to store the version in the last two bytes of a 1K image rather than the last two bytes of a 512 byte image, but now I wonder if anyone else needs changes to take a 1K versus 512 bootloader into account? Does something in the avrdude conf file need to be changed to reflect the reduced size of available flash?

The .version section should stay in the same place (last two bytes of flash memory in the chip), but the .text segment address will change for a longer bootloader. For a 1K bootloader on atmega328, you'd want to start at the 31k mark:

atmega328: LDSECTIONS  = -Wl,--section-start=.text=[color=brown]0x7c00[/color] -Wl,--section-start=.version=0x7ffe

Instead of:

atmega328: LDSECTIONS  = -Wl,--section-start=.text=[color=brown]0x7e00[/color] -Wl,--section-start=.version=0x7ffe

BTW, I didn't have any problem adding EEPROM support (under BIG_BOOT for atmega1284, just by including <avr/eeprom.h> and using eeprom_write_byte() and eeprom_read_byte() (which are macros, so they shouldn't result in any special library requirements.)

Claghorn, do you have Arduino IDE v1.0? This comes with Optiboot v4.4, which should compile as-is using the IDE tool chain. I could add the eeprom code and recompile with no problem. You'll need to adjust the fuses for the 1k bootsector of course - either in the optiboot makefile or in boards.txt, depending on how you're uploading the new bootloader. You'll also need to set the maximum_size line in boards.txt.

Aside from the segment addresses already mentioned, I can't think of anything else.

westfw:
The .version section should stay in the same place (last two bytes of flash memory in the chip), but the .text segment address will change for a longer bootloader. For a 1K bootloader on atmega328, you'd want to start at the 31k mark

Yea, that occurred to me later :-).

BTW, I didn't have any problem adding EEPROM support (under BIG_BOOT for atmega1284, just by including <avr/eeprom.h> and using eeprom_write_byte() and eeprom_read_byte() (which are macros, so they shouldn't result in any special library requirements.)

I know I've seen copies of eeprom.h where they are macros, but in the avr-libc I have (downloaded from the fedora repos), they are real external routines (actually there are a gazillion versions of them with different names disguised by macro definitions for just the name of the routine).

In fact the whole avr toolchain in the fedora repos seems to be somewhat different since if I don't modify anything in the optiboot source, it doesn't build the same (or even a similar) .hex file. For one thing, the unmodified version built out of the box is already bigger than 512 bytes.

You mean it's easier to compile something under Windows than Linux? My world is being turned upside down!

I think it's important to have the "right" version of the AVR libs, and possibly GCC too. This may actually be harder to ensure on a Linux box. It seems perverse to recommend running Gnu software on Windows, but it may be the quickest way to achieve your goal.

I just took a look at the latest 1.8.0 avr-libc from the official avr-libc site, and it looks the same as the fedora version (no macros for eeprom routines), so I guess the fedora repos are too new :-).

I was also reading about fuses and bootloaders in the atmel datasheet, and got really confused till I got it through my head that all the tables were giving addresses and offsets in 2 byte word units rather than byte units.

Anyway, I think I may have gathered enough info to take a real stab at making this work. Thanks to everyone for all the pointers. Now I need to find a big capacitor to disable reset on the R3 I'm going to try to use as an ISP...

Check the bootloader version in your Uno (see instructions here New optiboot; beta testers welcome... - #9 by westfw - Microcontrollers - Arduino Forum). Thanks to Westfw's work on optiboot, recent versions >=4.3 don't require reset to be disabled when running ArduinoISP.

the whole avr toolchain in the fedora repos seems to be somewhat different

Yes, this is part of the reason that I added support to optiboot to compile using the exact toolchain that ships with the Mac/Windows Arduino IDE, rather than (or "in addition to") the one that happens to be on your system. Especially since changes in compiler version can drastically change the code size, making source that compiled fine with 4.3.2 be many bytes too large with 4.6.

Confirmed that eeprom.h has changed drastically between the Arduino version and the "current" latest version. Rats; that's annoying...

tim7:
Check the bootloader version in your Uno (see instructions here New optiboot; beta testers welcome... - #9 by westfw - Microcontrollers - Arduino Forum). Thanks to Westfw's work on optiboot, recent versions >=4.3 don't require reset to be disabled when running ArduinoISP.

Another good thing to know. Thanks! (Someone should update the ArduinoISP wiki to mention this :-).

recent versions >=4.3 don't require reset to be disabled when running ArduinoISP.

FWIW, things still seem to work better for me with the cap between RESET and GND...

Seemed to work OK without the cap, but I did encounter the bug with 19200 baud causing problems. Had to figure out how to get everyone to agree on 9600 baud, then I was able to burn the new bootloader.

And now I've tried to use the watchdog reset gimmick to upload a new sketch and am utterly confused by the results :-). Avrdude appears to start talking to the bootloader OK (I don't get the kind of sync errors I get without the reset), but it gets to the point where it says "Writing" and would normally print # chars for progress, but it makes no progress and eventually says "not responding".

But wait! It gets weirder. At this point, it has apparently erased the old sketch, so now it really is sitting in the bootloader doing the 1-2-3 flashy thing on pin 13. If I try to upload the new sketch again, it works perfectly.

So I have a procedure that works for uploading microcode without physical access to the board, but I have no idea why it works :-).

Anyone know if there are things I need to reset manually that a watchdog reset doesn't reset? Should I be using different lock bits or something to make a watchdog reset "harder" in some way? I'm really mystified.

If anyone is actually curious enough to want to look at my bootloader hacks and the code that uses them, you
can always find the latest version of the source pointed at by:

http://home.comcast.net/~tomhorsley/hardware/arduino/software.html

Hmm, I'm not sure I follow. Is this the sequence of events?

  • Sketch sets eeprom flag and executes watchdog reset
  • Bootloader starts running : detects and wipes eeprom flag, does three LED flashes, and starts communicating with AVRdude
  • Communication is lost for reasons unknown, and AVRdude hangs
  • Bootloader watchdog expires and bootloader resets itelf
  • On this pass the bootloader does not flash the LEDs but immediately tries to start the sketch
  • The sketch has been wiped, so nothing happens

How do you trigger the second sketch upload attempt?

Actually after the first failure, the bootloader seems to be in a loop starting itself back up over and over again. The 3 flashes keep happening at that point, so the 2nd upload always encounters the bootloader no matter when I start it. I did see some code in there that says something about temporarily reprogramming the reset vector to point to the bootloader, so perhaps it gets that far and no farther and it is just constantly hitting watchdog reset and calling bootloader over and over? Still have no idea why the first attempt fails though.

Hum. Try putting AVRdude into very verbose mode (avrdude -v -v -v -v ...) and look for the command which fails. It will also tell you if your fuses are set correctly (low=0xFF, high=0xDC, ext=0x05). You could also check that the memory is being unlocked properly before writing (lock=0x3f).

I tried ultra verbose mode, and it is still mysterious (to me anyway). The transcript of the session that failed followed by the one that worked is here:

http://home.comcast.net/~tomhorsley/hardware/arduino/files/transcript.txt

As near as I can tell the first run tells it to write the initial chunk and never gets back a response.

In the second run, everything looks identical to the first right through the first write which does get a response and then keeps going to a successful completion.

Very weird, but I guess I don't absolutely have to understand it. So far running avrdude twice in a row has always worked to do the upload, so I can upload over bluetooth now with no manual intervention.

I adjusted the boot size fuse bits for a 1K bootloader, is there something in lock bits or other fuse bits that needs tweaking?

Can you show your fuse/lock bytes? Just use avrdude in -v mode to read from it.

I suspect I don't understand something. Do I need it hooked up to the ISP to do this? If I just burn a sketch it tells me "safemode read" of all the fuse and lock bytes are zero.

Here's part of the board definition I used when I burned the new bootloader:

aardremote.bootloader.low_fuses=0xff
aardremote.bootloader.high_fuses=0xdc
aardremote.bootloader.extended_fuses=0x05
aardremote.bootloader.unlock_bits=0x3F
aardremote.bootloader.lock_bits=0x0F
aardremote.build.mcu=atmega328p

What's your programmer again? With USBtinyISP I see this:

$ avrdude -c usbtiny -p m328p -v

avrdude: Version 5.8cvs, compiled on Jan 15 2010 at 17:27:01
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2009 Joerg Wunsch

...

Reading | ################################################## | 100% 0.01s

avrdude: Device signature = 0x1e950f
avrdude: safemode: lfuse reads as FF
avrdude: safemode: hfuse reads as D6
avrdude: safemode: efuse reads as 5

avrdude: safemode: lfuse reads as FF
avrdude: safemode: hfuse reads as D6
avrdude: safemode: efuse reads as 5
avrdude: safemode: Fuses OK

avrdude done.  Thank you.

I think the ArduinoISP one might not read the fuse bytes, so you don't know for sure what they are. And perhaps you can't change them.