Power a bare-bones project with USB 5V?

I'm more of a software than a hardware guy so forgive me if this is a naive question.
I'm working on a really minimalist project, imagine a small project box containing an ATMega328, XTAL, a couple of buttons & resistors and a two-line LCD.
Can I simply add a USB socket to the box and power it all via a USB cable connected to a PC or USB/5V wall-wart?
USB Pin1 is +5V, Pin4 is Gnd -- simple as that?

I realize USB may not be exactly 5.0V but it should still be in the working range of the components.
I've seen hints in my googling that I might need to add a cap between the +5 and Gnd.
Is that required, and if so can someone explain what it's doing (what its absence might cause)? Some kind of smoothing/filtering?
Thanks in advance!

Yes, you can just power it from a PC-connected USB cable or 5V wall-wart (as long as the wall-wawrt provides a REGULATED output -- if it doesn't say regulated, it's not).

You should indeed add a capacitor between +5V and GND. That is called a "decoupling capacitor" and it does indeed provide smoothing/filtering of the power line, necessary to keep the +5V more or less stable during period of switching activity (and there's a LOT of switching activity in a microcontroller).

More info here:

http://www.thebox.myzen.co.uk/Tutorial/De-coupling.html

--
The Flexible MIDI Shield: MIDI IN/OUT, stacking headers, your choice of I/O pins

Thanks. So, to be clear, something like this?

            +---------+
            |         |
[+5V]---+---|Vcc   Gnd|---+---[Gnd]
        |   |         |   |
  .1uF ===  | ATMega  |   |
        |   |         |  === .1uF
[Gnd]---+---|Gnd      |   |
            |     AVcc|---+---[+5V]
            |         |
            +---------+

1µF on VCC might be a bit better than 0.1.

Don't forget to provide de-coupling for your LCD too. Generally there is a decoder chip on it. In an ideal world. it would have its own decoupling capacitor. However, caps are cheap and it doesn't hurt to add another one for another active device.

Is it possible to over de-couple a circuit?

Is it possible to over de-couple a circuit?

No.

The only thing might possibly be that a switch mode power supply will refuse to start if it sees too much capacitance, but this will only happen on very large systems much more complicated than a PC.

If you can live with 8MHz or less then you don't need the external crystal. ATmegas have internal clock source.

For even more minimalizing (izzat a real word?) there's a youtube how-to shrinkify your Arduino projects.