I admit that I don't really know what I'm doing here and I've never messed aroud with makefiles so I thought editing the source was the way to go. Basically I just make a new project in atmel studio and threw all of the optiboot files in. Then I added
#define F_CPU 7900000L
#define BAUD_RATE 2400
to the top of optiboot.c and and removed
#if (F_CPU/BAUD_RATE) > 280 // > 57600 for 16MHz
to use the hardware uart.. I also changes a few of these
#elif (defined(__AVR_ATmega328P__) || defined(__AVR_ATmega328__))
so that it would compile for a 328. After the changes I just let atmel studio compile it for me (it ends up at 766 bytes, but that's good enough for me). Here is what AS6 says it's doing to compile:
avr-gcc -funsigned-char -funsigned-bitfields -DDEBUG -O1 -ffunction-sections -fdata-sections -fpack-struct -fshort-enums -g2 -Wall -mmcu=atmega328 -c -std=gnu99 -MD -MP -MF "$(@:%.o=%.d)" -MT"$(@:%.o=%.d)" -MT"$(@:%.o=%.o)"
avr-gcc -Wl,-Map="$(OutputFileName).map" -Wl,--start-group -Wl,-lm -Wl,--end-group -Wl,--gc-sections -mmcu=atmega328
and to assemble:
avr-gcc -Wa,-gdwarf2 -x assembler-with-cpp -c -mmcu=atmega328 -MD -MP -MF "$(@:%.o=%.d)" -MT"$(@:%.o=%.d)" -MT"$(@:%.o=%.o)" -Wa,-g
I used my AVR dragon to burn the bootloader and to burn the BOOTSZ1, BOOTSZ0, and BOOTRST fuses. I have a feeling that the problem lies in the fuse settings, or the fact that my file is over 512 bytes, while optiboot expects itself to be less that 512 bytes (does it?). If you could provide me with a "correct" hex file for an atmega328 running at 8 MHz (7.9 to be exact) at a baud rate from 2400 to 115200 baud that would awesome.