Uploaded sketch not working after power off

I get it running by pressing the reset button after power cycle.
But why do I need to press the reset button? I don't need to do it while running the Button Example to blink LED.

Could you point me to some reference URL that a beginner should read?
I have read the foundation section from Arduino but I did not find some thing like the above case.

Thanks PaulS and Nick for taking your time to help me.