I don't think there's any frequency dependence in the terms bypass or decoupling - you can say high-frequency decoupling, low-frequency bypass - well I'm brought up on Art of Electronics and I don't think they make that distinction.
Perhaps this is a 'dialect' thing.
Anyway my two-pennies-worth on decoupling is inductance - to be effective high-speed decoupling caps need to be on a low-inductance path from the power pins of the device in question. If an electrolytic is wound so that the leads both go to the same end of the foil strip then there is a low(ish) inductance path to the first part of the foil (and it doesn't take much foil area to have 0.1uF capacitance).
If however the leads go to opposite ends of the foil before its wound up there is a much more tortuous path - more inductance, and high speed signals are not going to see that capacitance so well (or so fast). So details matter.
Of course at audio frequencies the two arrangements behave the same.