Couple of things,
Most supercaps have very low current draw allowances... on the order of 1-5mA. You can certainly keep an RTC alive on this power supply or preserve the contents of the Arduino RAM while changing batteries, etc. but it's not meant to power a MCU unless you buy a supercap that allows much higher current draws (they tend to be 1F and up) and combine that with a boost switchmode regulator to pull all the available power out of the supercap. Multi-Farad supercaps (and you'd want a dual-layer model, if memory serves) are not cheap and a boost converter is just what the doctor ordered to realize their full value.
Even so, I would expect you would have to put the MCU to sleep most of the time to conserve power. A LiPo battery pack may be a better and cheaper solution.