The primary function of decoupling capacitors is to prevent the supply voltage fluctuating as the circuit switches current - logic circuits switch very fast, an Arduino output pin can switch at millions of amps per second (even if only briefly as the current changes by 10's of mA). The power supply can't react this quickly (on timescale of 10ns so the voltage at chip's supply pins will droop sharply unless there are decoupling capacitors to provide this sudden change in current. At longer timescales the power supply will react and recharge the decoupling capacitors.
So the decoupling capacitors can't prevent voltage fluctuations at the supply but can reduce them substantially (from volts to fractions of a volt). With logic circuits you only need enough decoupling to prevent false switching - 200mV of noise would be fine but 1.5V would not.
With analog circuits you want much cleaner supply rails (but seldom have to deal with such high speed switching - except where digital and analog meet as here).
Another way of looking at is that the decoupling capacitors store charge for when there is sudden demand for more. There can be a hierarchy of decoupling capacitors, the small fast capacitors are the first line of defence, then larger slower capacitors to keep them happy, eventually the power supply circuit reacts. In fact VLSI logic chips have a significant proportion of their silicon area devoted to on-chip decoupling capacitors.