Hi,
You could run rechargeable AA's which output only 1.2V or thereabouts, therefore the total would be 4.8V well within the safe voltage for the ATTiny, alternatively if you have flexibility with the configuration, 2 parrallel pairs in series for a total of 3V you'll still be able to run the ATTiny, and that should last for a very long time too (depending what else you have it running).
I often match up ATTiny85 with power from a single CR2032 3V coin cell and they run for ages.
Is there a reason you're wanting to run with 4x AAs specifically?
Cheers ! Geoff