It's just that so many ICs have their GND and Vcc pins on the far ends... (4 and 8 for OpAmps, for example).
Luckily leaded parts are usually good enough.
Wth adjacent pins for power it'd be convenient to solder an SMD cap right on top of the power pins, next to the package itself, and just leave it in place in between experiments.