Pretty much yes. You can pop out the chip, drop it on a board with some caps and a resonator/crystal and you're off and running.
There's a lot of good resources out there on stand alone arduinos:
http://www.imagearts.ryerson.ca/sdaniels/physcomp/tutorials/Arduino_standalone/ard_hack.html
http://www.arduino.cc/playground/Learning/AtmegaStandalone
I've been tempted to pick up a couple of these boards:
http://www.spikenzielabs.com/SpikenzieLabs/Prototino.html
They look pretty nice for minimal arduino setups.