RRRRRRRRRRBBA or running ATMEGA chip by itself.

For finished projects [I'd program the chip beforehand in an arduino board w/removable chip], how feasible is it to use the ATmega by itself like this guide suggests? Obviously the version with external 16Mhz crystal would be used.

Should it also have a voltage regulator or any other circuitry to make it more robust?

Some people base their hardware on their Arduino boards and that's a good solid approach.

My goals see my UNO as a development board rather than a piece of my target application.

Yo:
ATTiny standalone:

http://hlt.media.mit.edu/?p=1229
http://macetech.com/blog/node/81

The business card app may not be perfect but to me it is so neat and clean I bow down anyway!

More examples:
http://www.obdev.at/products/vusb/prjall.html

And there's tons more out there, some going back decades. I remember a friend playing with a 6502 based MCU with Forth in the microcode, I guess it was in the mid-80's. Yeah, the native language was Tiny-Forth I wouldn't mind finding a couple of those chips.