What is your wiring, exactly?
10K resistor from +5 to RESET. Pushbutton to ground so I can reset the processor.
TX and RX connected to a Pololu AVR Programmer.
0.1uF capacitor across VCC and GND at the processor.
16 MHz crystal + capacitors from Evil Mad Science connected to XTAL pins.
AVCC connected to +5 volts.
Analog side GND connected to ground.
SCK, MOSI, and MISO connected to the Programmer.
I've tried with and without and LED + series resistor on pin 3.
Maybe put a 0.1uF capacitor there for decoupling.
Across VCC and GND?