There is no "3.3 V version" of the microprocessor; it operates up to 5.5 V. It's just that you have to use an 8 MHz crystal for 3.3 V to work rather than a 16 MHz one. Of course the 8 MHz version will work perfectly well at 5 V.
It is absurdly improbable that the capacitors on the board would be rated for less than 6 V. The only concern is whether the regulator would be harmed by back-feeding more than 3.3 V to its output. What may happen is that it will draw more quiescent current and in fact if you propose to operate it from a lithium battery you certainly should remove or disable the regulator.
For battery standby operation you also want to remove the "pilot" LED or its resistor. 