Running Arduino-compatible from batter(y|ies) without regulator

or that 5V is important for some reason.

You are trying to imply that it isn't?

I don't think the exact voltage matters for us, so long as the microcontroller is running stable.

There is a big difference between exact voltage and consistent voltage. Batteries will give you neither.