Stupid question: What's the diode across the reset pull-up do? I know I've heard of that before, but I have no idea what problem it's meant to solve.
As for capacitors, I use Fairchild's recommendation. Big(ish) electrolyitc caps on input and output side -- sized to provide adequate capacity per the application, but often 470uF on input, and 100uF on output. Parallel ceramic caps on the input and output sides of the regulator -- 100nF each (though Fairchild recommends a 330nF on the input FWIW.) Additional 100nF ceramics on Vcc pins of all ICs, as well as the AVcc of AVR chips.
None of these change the regulator's output voltage. In fact, they're there so the voltage doesn't change with load.