This is largely an academic question, I realize capacitors are cheap.
I was building a power supply last night for a circuit which was going to be powered by a 9V battery. I started wondering whether or not the standard capacitors on either side of the 7805 are really needed if the source is a battery. Do you have to worry about filtering the power coming from a battery the same way you do when the power is coming from a wall wart?
The reason for the two capacitors is that they stop the 7805 oscillating!
A voltage regulator contains a high-gain amplifier (rather like an op-amp) as well as a voltage reference and a power output transistor. If you leave out the capacitors, it may well work, but there's a risk of very high frequency oscillations. These oscillations will get transferred out on the power wires to your circuits, which will then misbehave in strange ways.
Note that even a battery will not produce a stable volatge if the current drawn from it varies (look up "internal resistance"). So you'll usually find a capacitor across the power supply of a circuit, even if the supply is not coming from rectified AC.
Thank you,
My assumption was that the capacitors were there to handle the ripple from rectified AC power. I had no idea how the 7805 worked so I learned something today.