ok, that all makes sense. I think the biggest thing I don't understand is the leap from machine language to a blinking LED.
Without resorting to Senso's solution (a Computer Architecture class in university) we have to simplify things a bit. The microcontroller does all of its work essentially by moving ones and zeroes around. One of the things it can do quite easily is apply 5 volts to a specific output pin when there is a 1 at a specific bit location and remove that 5 volts when there is a zero at that same location. The 5v turns the LED on and the lack of 5v turns it off.
To my understanding, a chip like the atmega 328 is pretty much a zillion transistors with maybe some other stuff too (like what, an oscillator/resonator/whatever?), so I guess the hex loaded onto the chip is stored in the transistors themselves?
That is essentially correct.
However, this cannot be the case because the transistors would reset whenever power is lost. I guess the main thing is just how do the internals of a chip store/execute a program?
Modern microcontrollers possess internal non-volatile memory of one sort or another which would be in the category of 'other stuff' up above. The program is stored here and it is not lost when power is removed.
Don