I have optiboot for atmega1284p at 20MHz - I needed the extra 4MHz to make an application fast enough. I will upload it this evening.
This is my boards.txt entry. Use it as a guide for fuses, etc. The bootloader uses 115200 Baud, using UART1, the TX1 and RX1 pins. The status led is on PD7. If you want UART0 or a different LED pin and I will recompile it for you.
cube1284p_20.name=Cube 1284P (PCB, 20MHz)
Sorry if I'm way off base here but I've been bouncing around the datasheet for the 1284, and it appears the chip @ 20Mhz will be unstable at lower voltages, you have your brownout set to 2.7v in efuse, I would think a value of 0xFC (4.3V) would make more sense for the the efuse? Am I missing something?
My only guess here is maybe out here in arduino land we regularly give our caps a workout and seeing below 4.3v isn't going to be that uncommon? If this is the case what does the chip do when its underpowered for the current speed? just miss a few steps and pick right back up?
Also shouldn't the Low fuse be set for a full swing oscillator? Something like 0xD7 for a 20mhz crystal?