If the processor is a "pico power" version, it can be powered directly from 2 AA batteries. This is by far the most efficient way to provide power.
However, other devices may not tolerate the low voltage or erratic current delivery of dying batteries. What other hardware will you be using?