Hello,
I'm wondering if, after programming the AVR in an Arduino, can I remove the chip (ATmega328) and supply power to it outside of the Arduino board using just a 12v battery and a 5v voltage regulator?
yes you can but its best to at least add 1 or 2 small caps to cancel out any noise.but if its just for something like leds the caps are not realy needed.
Thanks!
You might want a 16MHz crystal or resonator too - the build-in oscillator can be used but you'll need to configure fuse-bits to use it. Search the site for more details there is lots about this.
And you should have a decoupling capacitor, these are not optional components as some believe.
If you take a look at here (http://www.arduino.cc/en/Main/Standalone), it describes pretty much what you need to do.