Decker87:
What have you found to be the easiest way to drop a programmed microcontroller into a project?
Do a search for "standalone Arduino" - that will tell you what you need to know.
As an alternative, there are smaller Arduino boards available, such as the RBBB kit:
http://shop.moderndevice.com/products/rbbb-kit
...the Arduino Mini:
http://arduino.cc/en/Main/ArduinoBoardMini
Decker87:
Are there any caveats of removing the Atmega328 and, for example, putting it in a small breadboard or protoboard and using it?
Provided you supply the power properly, add decoupling capacitors, and use the same crystal or resonator (16 MHz), there shouldn't be an issue. Just be careful about your connections and such.
Decker87:
Is it enough to simply buy a USB connector and wire it to the microcontroller? (including power, etc) Or does the Uno board contain more functionality that enable it?
No - all "standard" Arduinos with USB connectors use an interface IC to connect the serial I/O pins on the Arduino to the USB connector, which makes the device appear as a "virtual serial port" to the PC. On older Arduinos prior to the Uno, this IC was a chip manufactured by FTDI (http://www.ftdichip.com/); later Arduios use a custom ATMega8 with custom programming to perform the same function (yes, there are two ATMegas on an Uno; both are reprogrammable as well, though the ATMega8 takes a bit of work from what I understand - it doesn't have a bootloader, for instance).
If you don't feel like adding all of the extra circuitry (and really, you shouldn't, unless you have a particular reason to, like if you needed the final project to be easily plugged into a PC for some reason), there are a couple of other options. One would be to utilize a USB to Serial cable - there exist special cables called "FTDI cables" that incorporate the FTDI chipset to communicate to the Arduino (there are other cables/chipsets as well); you just bring out some header pins from the RX/TX pins and a few others. One issue with using these other cables with a pre-bootloaded ATMega328 is that you may have to do a manual reset and such to get your program on them (if you are doing "off-board" programming); there are plenty of postings and such regarding how to do this, so I won't rehash it here.
Another possibility is to simply bypass using a bootloader period. On the Arduino are a set of pins labeled something like ICSP/ISP; these pins are used on a stock Arduino to program the bootloader onto a "blank" ATMega328, via a special device called an ICSP/ISP (in-circuit serial programmer). If you are really interested in this, you can do a lot of cool things (you can also "brick" your ATMega328 - you'll need a programmer that supports "high-voltage" programming to bring it back). The Uno itself can act as an ICSP/ISP programmer via a special script which is included with the distro; all you have to do on your "standalone" Arduino is bring out the pins for ICSP/ISP, plug the two together, and load the script on your regular Arduino, then select it as the programmer from the menu in the IDE (there's an option for it).
Here is something, though, that you should avoid doing: Don't insert/remove the ATMega328 repeatedly from the socket on the Arduino. Doing this will eventually cause the spring pins on the socket to lose their springiness, causing intermittent contact, and other problems (it'll show up as a dead Arduino, or spurious I/O bugs or any number of other possibilities). In addition, it is very easy to bend and break the pins of the IC by such insertions and removals (which is why you should be very careful when you -must- do this, such as when you replace the IC on the board because of failure - which is really the -only- reason you should remove the chip today).
Instead, shields exist which you can plug into your Arduino which incorporate a special ZIF (zero-insertion force) socket that allow you to easily plug in and remove the ATMega being programmed. These shields typically use the ICSP/ISP pins as the programming interface, allowing you to easily add a bootloader to a bare chip, or push a hex binary of your compiled sketch to the chip instead. Note that it is possible that you might be able to find a shield that can plug into the Arduino, in which you remove the original ATMega328, plug in the shield, and it has its own crystal and such. I know that homebrew boards of this nature have been built, but I don't know if any commercially available boards are out there. I think most today use the ICSP/ISP pins, with the original Arduino running the ICSP/ISP sketch, etc.