What is the simplest circuits that uses caps?

I know they store a charge, but WHY would you want this?

Well, the simplest circuit is probably the bypass caps used in digital circuits, whose purpose is to store "some" charge very close to the pieces of circuit that switch charge around (since long wires are not "ideal" and would interfere some with rapid movements of charge.)

But most uses of capacitors do not use them so much for storing charge, but in ways that depend on the nice mathematical behavior exhibited by the voltage and current parameters around the capacitor. While a resistor is simply

V = I*R, a capacitor does

V = (1/C)(Integral(I(t), dt)). That's calculus, and may not make much sense depending on how much math you've had, but:

1) having a simple component that does "hard" math can be useful.

2) The important part is that the voltage depends on TIME as well as current and capacitance. Thus a capacitor can be used in assorted applications that involve timing things.

A particularly simple "blinking light" using a capacitor is this neon light circuit:

The resistor limits the charging current, so the voltage on the capacitor gradually builds up to a point where the neon bulb turns on (neon bulbs have certain "magic" properties that allow this to work; notably having infinite resistance until you hit a particular voltage. You can't build this with an LED.)

You might also look here:

http://www.adafruit.com/forums/viewtopic.php?f=8&t=17207