[SOLVED] Building ATMEGA32U4 bootloader and pin 13 behavior

[UPDATE]
In this thread, I was asking questions about how to build an ATMEGA32U4 bootloader in order to regain control of the TXLED, RXLED, and L_LED (pin 13) pins for my project. I succeeded with the help of hiduino and some online information. I started at a question on L_LED.

[ORIGINAL]
So with my barebone ATMEGA32U4/Arduino micro setup, whenever I upload code, pin 13’s LED flickers. I don’t know why it would do that. Any hint? Is it used as an output pin during program upload?

I was planning to use the pin to sense a rotary encoder channel. If the channel is closed then I fear the pin will get damaged during program upload if it is turned into output. Should I just add a 1K resistor from it to the encoder channel? Unfortunately I’ll have use use all the chip’s pins for my project so I’m not letting pin 13 just sit idling. 4 pins for sd card, 6 pins for lcd, 3 for encoder with button, 2 for I2C sensors and internal RTC, 6 in groups of 2 for 3 “input channels”, then 2 pin serial for xbee, 1 battery sense and 1 buzzer.

Any comment is appreciated!

I observed the same behavior with a leonardo clone board. The LED on pin 13 flashes while sketch is uploaded. Does someone know what is driving this pin?

This is from the doc:

• ICP3/CLKO/OC.4A – Port C, Bit 7 ICP3: If Timer 3 is correctly configured, this pin can serve as Input Capture feature. CLKO: When the corresponding fuse is enabled, this pin outputs the internal microcontroller working frequency. If the clock prescaler is used, this will affect this output frequency. OC.4A: Timer 4 Output Compare A. This pin can be used to generate a high-speed PWM signal from Timer 4 module. The pin has to be configured as an output (DDC7 set “one”) to serve this function.

I don't think any alt. functions were used so this behavior could be in the bootloader?

Yes, the Caterina bootloader used in the Micro and Leonardo uses the L LED on pin 13 as a "heartbeat breathing" to indicate it is running. It continues to run the heartbeat even during the code upload. During this code upload it interrupts the heartbeat breathing so it appears to flicker.

Thank you hiduino! You are a bootloader expert! I wish the forum gives out badges for different expertises.

Following your explanation I found these in caterina.h

		#define L_LED_OFF()		PORTC &= ~(1<<7)
		#define L_LED_ON()		PORTC |= (1<<7)
		#if DEVICE_PID == 0x0037	// polarity of the RX and TX LEDs is reversed on the Micro
			#define TX_LED_OFF()	PORTD &= ~(1<<5)
			#define TX_LED_ON()		PORTD |= (1<<5)
			#define RX_LED_OFF()	PORTB &= ~(1<<0)
			#define RX_LED_ON()		PORTB |= (1<<0)			
		#else 
			#define TX_LED_OFF()	PORTD |= (1<<5)
			#define TX_LED_ON()		PORTD &= ~(1<<5)
			#define RX_LED_OFF()	PORTB |= (1<<0)
			#define RX_LED_ON()		PORTB &= ~(1<<0)
		#endif

So essentially can I use the following to nullify the effect of the L_LED_ON()?

#define L_LED_ON()

or

#define L_LED_ON() {}

I guess I can nullify the RX and TX LED pins too and use them for I/O, right?

I wonder how to then compile the code. Any advice?

I tried make all using just Arduino IDE's included make in avr but it didn't work:

..\..\..\tools\avr\utils\bin\make all
makefile:153: ../../../../../../LUFA/LUFA-111009/LUFA/makefile: No such file or directory
The system cannot find the path specified.
The system cannot find the path specified.
make: *** No rule to make target `../../../../../../LUFA/LUFA-111009/LUFA/makefile'.  Stop.

To build a Caterina bootloader you will need to download the LUFA library and unzip it to a path accessible from the Caterina folder in your IDE install. The Caterina was built on an older release of LUFA 111009. It may work with newer LUFA releases but I think it may need modifications for that. It's simpler to just stick with the 111009 release.

Try this download link, http://www.github.com/abcminiuser/lufa/archive/LUFA-111009.zip

I have my install paths like this: \My Documents\Arduino\arduino-1.0.5-r2\hardware\arduino\bootloaders\caterina \My Documents\LUFA\LUFA-111009\

You will need to edit the Makefile to define the USB VID and PID values. See the examples in the file. Just uncomment the ones you will use for testing.

(Note: You may need to add the IDE WinAVR \avr\bin and \avr\utils\bin to the Windows PATH)

C:\> set PATH=%PATH%;C:\Documents and Settings\User\My Documents\Arduino\arduino-1.0.5-r2\hardware\tools\avr\bin;C:\Documents and Settings\User\My Documents\Arduino\arduino-1.0.5-r2\hardware\tools\avr\utils\bin

Then from command line window run the make command:

C:\Documents and Settings\User\My Documents\Arduino\arduino-1.0.5-r2\hardware\arduino\bootloaders\caterina>make

-------- begin --------
avr-gcc (WinAVR 20100110) 4.3.3
Copyright (C) 2008 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


Compiling C: Descriptors.c
avr-gcc -c -mmcu=atmega32u4 -I. -gdwarf-2 -DF_CPU=16000000UL -DF_USB=16000000UL -DBOARD=BOARD_USER -DARCH=ARCH_AVR8 -DBO
OT_START_ADDR=0x7000UL -DDEVICE_VID=0x2341UL -DDEVICE_PID=0x0037UL -D USB_DEVICE_ONLY -D DEVICE_STATE_AS_GPIOR=0 -D ORDE
RED_EP_CONFIG -D FIXED_CONTROL_ENDPOINT_SIZE=8 -D FIXED_NUM_CONFIGURATIONS=1 -D USE_RAM_DESCRIPTORS -D USE_STATIC_OPTION
S="(USB_DEVICE_OPT_FULLSPEED | USB_OPT_REG_ENABLED | USB_OPT_AUTO_PLL)" -D NO_INTERNAL_SERIAL -D NO_DEVICE_SELF_POWER -D
 NO_DEVICE_REMOTE_WAKEUP -D NO_SOF_EVENTS -D NO_LOCK_BYTE_WRITE_SUPPORT -Os -funsigned-char -funsigned-bitfields -ffunct
ion-sections -fno-inline-small-functions -fpack-struct -fshort-enums -fno-strict-aliasing -Wall -Wstrict-prototypes -Wa,
-adhlns=./Descriptors.lst -I../../../../../../LUFA/LUFA-111009/ -std=c99 -MMD -MP -MF .dep/Descriptors.o.d Descriptors.c
 -o Descriptors.o

Linking: Caterina.elf
avr-gcc -mmcu=atmega32u4 -I. -gdwarf-2 -DF_CPU=16000000UL -DF_USB=16000000UL -DBOARD=BOARD_USER -DARCH=ARCH_AVR8 -DBOOT_START_ADDR=0x7000UL -DDEVICE_VID=0x2341UL -DDEVICE_PID=0x0037UL -D USB_DEVICE_ONLY -D DEVICE_STATE_AS_GPIOR=0 -D ORDERED_EP_CONFIG -D FIXED_CONTROL_ENDPOINT_SIZE=8 -D FIXED_NUM_CONFIGURATIONS=1 -D USE_RAM_DESCRIPTORS -D USE_STATIC_OPTIONS="(USB_DEVICE_OPT_FULLSPEED | USB_OPT_REG_ENABLED | USB_OPT_AUTO_PLL)" -D NO_INTERNAL_SERIAL -D NO_DEVICE_SELF_POWER -D NO_DEVICE_REMOTE_WAKEUP -D NO_SOF_EVENTS -D NO_LOCK_BYTE_WRITE_SUPPORT -Os -funsigned-char -funsigned-bitfields -ffunction-sections -fno-inline-small-functions -fpack-struct -fshort-enums -fno-strict-aliasing -Wall -Wstrict-prototypes -Wa,-adhlns=Caterina.o -I../../../../../../LUFA/LUFA-111009/ -std=c99 -MMD -MP -MF .dep/Caterina.elf.d Caterina.o Descriptors.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/AVR8/Device_AVR8.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/AVR8/Endpoint_AVR8.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/AVR8/Host_AVR8.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/AVR8/Pipe_AVR8.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/
AVR8/USBController_AVR8.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/AVR8/USBInterrupt_AVR8.o ../../../../
../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/AVR8/EndpointStream_AVR8.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/AVR8/PipeStream_AVR8.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/ConfigDescriptor.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/DeviceStandardReq.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/Events.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/HostStandardReq.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Core/USBTask.o ../../../../../../LUFA/LUFA-111009/LUFA/Drivers/USB/Class/Common/HIDParser.o --output Caterina.elf -Wl,-Map=Caterina.map,--cref -Wl,--section-start=.text=0x7000 -Wl,--relax -Wl,--gc-sections     -lm

Creating load file for Flash: Caterina.hex
avr-objcopy -O ihex -R .eeprom -R .fuse -R .lock Caterina.elf Caterina.hex

Creating load file for EEPROM: Caterina.eep
avr-objcopy -j .eeprom --set-section-flags=.eeprom="alloc,load" \
        --change-section-lma .eeprom=0 --no-change-warnings -O ihex Caterina.elf Caterina.eep || exit 0

Creating Extended Listing: Caterina.lss
avr-objdump -h -S -z Caterina.elf > Caterina.lss

Creating Symbol Table: Caterina.sym
avr-nm -n Caterina.elf > Caterina.sym

Size after:
AVR Memory Usage
----------------
Device: atmega32u4

Program:    4058 bytes (12.4% Full)
(.text + .data + .bootloader)

Data:        190 bytes (7.4% Full)
(.data + .bss + .noinit)



-------- end --------

Then in the Caterina folder you should have a Caterina.hex bootloader file .

hiduino,

Thank you! More karma!

I followed your steps and some online tutorial:

http://galvant.ca/how-to-compiling-arduino-caterina-with-new-vidpid/

I finally got the bootloader built. It is only 4K of code. So can you safely say that once I upload even a blink sketch, the 4,832 bytes of USB handler will appear on the MCU?

I bet the large 32KB "bootloader" is just to have both bootloader and a short blink with the 4,832 bytes so the OS will register the device or something else?

I'll try this bootloader tonight when I get home.

Now I suspect if I don't have the 4,832 bytes, after bootloader finishes, my device handlers are gone and I might not be able to upload? If this doesn't work, say the comm port is not showing up, how do I add the 4,832 bytes to the bootloader?

The 4832 bytes are part of the sketch. They are not related or any part of the bootloader. They are for SerialUSB communication from your sketch and to allow for the auto-bootloader function. They are located in the core files. So they will always be there.

Theoretically you could load a different core, like Uno, to the board and loose any USB communication from your sketch and the auto-bootloader function. Then you will gain back around 4K of flash space, or even 8K of space if you don't need the bootloader. In this case you would need to upload using ICSP.

Thanks for continuing to help me here! I really appreciate it.

I am trying to add that SerialUSB together with my bootloader so my device will have the Serial to talk to a PC. I looked at the bootloader I build and Caterina-Micro.hex.

There are 32 bytes per line in Caterina-Micro.hex, it has the 4832 byte in the front, empty space in the middle and bootloader in the last 4KB. I can understand all those in theory with intel hex format.

My build has 16 byes per line, no problem. It only has the 4KB bootloader, and a line to indicate the execution starts at 0x7000

:040000030000700089

The Caterina-Micro.hex doesn't have this line though. If I were to combine the hex files to add the 4832 bytes to my bootloader, should I keep this line?

The bootloader detaches USB after it ends so I worry that without SerialUSB code the Arduino IDE may not have access to the chip.

That line can be ignored. It's used for other types of processors.

You can use that Caterina.hex bootloader as-is without the beginning 4832 bytes. It's a little tricky doing this. The first time you use it the bootloader will be running continuously until you upload a sketch. The IDE will get confused finding the COM port, but just manually press the reset button when it's ready to upload. You just need to be aware the COM port number changes with the bootloader.

Alternatively, you could build the basic Blink sketch and edit the Blink.cpp.hex file. Remove the last line end-of-file marker.

:00000001FF

Then append your Caterina.hex file to the end of it. Then used the combined hex file as your bootloader.

This will allow it to initially have a running USB link for the Auto-bootloader to run when you upload from the IDE. This is why the production Caterina-Leonardo.hex files have the combined sketch and booloader all in one file.

hiduino,

Thanks. I have tried on my home computer with win 8 without joy. I can't compile the bootloader. I could when I was using work computer running win 7.

Some error code:

      0 [main] sh 6744 sync_with_child: child 8804(0x160) died before initializa
tion with status code 0xC0000142
    224 [main] sh 6744 sync_with_child: *** child state waiting for longjmp
/usr/bin/sh: fork: Resource temporarily unavailable
      0 [main] sh 3188 sync_with_child: child 5344(0x160) died before initializa
tion with status code 0xC0000142
    318 [main] sh 3188 sync_with_child: *** child state waiting for longjmp
/usr/bin/sh: fork: Resource temporarily unavailable

After some research I think it is an issue with some versions of windows, such as 8.1 on my home machine. I tried a few things but none worked.

Just in case, this worked for some people:

http://www.avrfreaks.net/forum/windows-81-compilation-error?name=PNphpBB2&file=viewtopic&t=137881

I will have to do this development on my work computer then. Win 8.1 has broken quite a lot of things. I'm glad I have a win 7 machine at work.

Update:

Using the solution I provided above (link to a file to replace mysys-1.0.dll) and compiling in command line prompt cmd.exe under c:\system\SysWOW64\ worked. When I was compiling earlier, I was using cmd.exe in c:\system\System32. The two are different. I was using this one in win 7 but it worked without the .dll replacement. It must be the difference between win 8.1 and win 7/8. I spent a large part of the evening fighting this problem and didn't have time to compare the complied binary with the one I compiled in win7.

Combining Caterina.hex and blink.hex also worked. Now I have removed control of the TXLED, RXLED, and L_LED pins from the bootloader, increasing the usable pin amount by 3 (have to define TXLED pin to be D30).

Many thanks to hiduino!

Hello. Could you please provide any additional info, how to solve this?
What did you do in the bootloader etc?

I am facing the same problem and want to resolve it.

Thanks.