Precisely…Ergo, lack of regulation and decoupling, am I right?
Battery voltage stability/regulation is not too bad, but it depends on what kind of battery you are using and your application. Ni-Cads and Nimh have pretty good flat discharge voltage until very near the end of their charge. Li are more like lead-acid in that the voltage decreases linear with charge capacity remaining. All would power a AVR processor chip fine as long as the voltage is not over or under the max and min specifications for the chip.
If you are using analog inputs the A/D conversion accuracy will be effected some with falling battery voltage unless you utilize the internal 1.1vdc band gap as the A/D reference or other external regulated voltage reference for the AVCC pin.
Decoupling should be OK with just the normal .1mfd bypass caps that are normally wired right at the AVR chips power pins, extra decoupling when using batteries should not be required, but that might change if you are also powering some external components (servos, motors, solenoids, etc) along with the processor chip.