You can probably power the ATMega328. It will go down to 1.8V at lower speeds. I've actually done this before using a Renesas chip, but I don't remember exactly the voltage rating of the supercap we used, or even which chip it was. This was for a class project where we were graded on power reduction based on how long we could run a particular script off the super-cap. So it should be theoretically possible on the ATmega. However, as for the other parts, like the USB, etc, no, those need 5V.