To answer the question from AgeingHippy : YES. If you guys talk about the ATmega328 pin 1. And I agree with you Grumpy_Mike. In Every ( most ) of CPU's ( that including the computer in front of you ), a reset circuit is needed. Even a simple counter chip ( 7490 ) need a reset circuit. <-- Correct me if am wrong. In the book title "The 8088 Project Book", page 64, by Robert Grossblatt, a reset circuit is needed. It basicaly : 10 K pull-up config , push-on button in serie with 150 , 4.7 uF elec and 0.1 uF ( in parallel with the push-on button and the resistor) and 1N914 diode in parallel with the 10 K resistor. Work like this : At power-up, the cap is short for the time of RC ( 10 K * ( 4.7 uF + 0.1 uF ) <--put a LOW signal ( possibility of clearing the internal Flip-Flop - counters ) , and slowly going into HIGH for the CPU to run. When you press reset, I will do the same procedure when it start-up.
Adding a reversed biased diode across the reset pull-up resistor 'cures' the symptom.
Thank you retrolefty to remind me of the diode. I will place it in my breadboard Arduino/ ATmega358 setup.