Bypass/Decoupling caps are needed due to the parasitic inductance and resistance inherent in wiring and PCB traces, inductance limits how fast current can change. When working with high frequency digital ICs, current must be able to change quickly. Adding the bypass/decoupling capacitors as close as possible to the IC's V+ and GND pins effectively reduces the input impedance of the power supply + wiring.
You could also think of the bypass caps as tiny batteries that supply the IC with power until the power supply current can arrive to both fill the capacitor again and power the IC. The capacitor also passes any high frequency noise FROM the IC to ground, preventing interference.
Multiple value capacitors are used to cover multiple frequency requirements, or to have as low as possible ESR and ESL in the capacitance. MLCC (Mono/Ceramic) capacitors have extremely low ESR/ESL, but are also only mid capacity, they are the most common type of 0.1uF capacitors used for bypass. These are put in parallel with an electrolytic, which can filter the lower frequencies, but has higher ESR and ESL. When using an electrolytic, when the capacitors ESL/ESR is greater than that of the supplying circuit, their value cancels itself out in the best case, and they make things stop working in the worse case (see "capacitor plague").
All of the above is covered in the link below, but I thought I'd give a Cliff's notes in easier to understand language.
http://www.interfacebus.com/Design_Capacitors.html -This page covers everything you need to know, from layout and how bypass/decoupling capacitors work