Technically you're correct, but in practice you can just use a ceramic radial mounted as close as possible to the part, and with the shortest leads possible. AFAIK, that's about as low-inductance as it gets short of surface mount parts, and I'm going to assume that's out of the question here. I use the little TDK blue ceramics from DigiKey for all my through-hole decoupling.
The PDF linked above has a really good explanation of this, and as pointed out before, it may be helpful to parallel a large(r) electrolytic to catch higher current loads.
BTW, thanks for the understanding on the LMGTFY thing. I don't mean to lecture, just trying to add perspective and keep up the supportive community here.