Atmega328 custom bootloader

Did you disable the auto-reset in the Arduino that's acting as the programmer?

According to http://arduino.cc/playground/Main/DisablingAutoResetOnSerialConnection
you can put a 120? resistor between 5V and Reset.

I personally didn't have a resistor like that and didn't want to make one because I'm color blind, and that means I can't read colour coding and need to test each one…
So I just put a 10µF capacitor there, it worked really nice, as it won't let the reset pin go down from 5V.
It's dangerous though, as a lot of current could enter through that pin and burn something… but meh, being a daredevil is fun.