It is important not to confuse the ability of the MCU to operate over a wide supply voltage range with the importance of supply bypassing - which means both "bulk" bypassing (47 µF is what is used in the UNO) and high frequency bypassing (0.1 µF being the commonly used value).
Reliable operation is almost always dependent (perhaps not when merely flashing LEDs) on the supply voltage not suddenly and transiently changing.
"Big" supply capacitors - if this is what you mean - are simply part of proper engineering design.