Reassuringly I found this investigation...
The electrolytic capacitor can be modeled at an ideal capacitor with two back-to-back zener diodes in parallel with it, with each zener representing one of the cutoff voltages. The leakage slowly increases for voltages below these cutoff voltages, and then quickly for levels above the cutoff voltages. The performance is very consistent within these bounds as well, with differences most likely accounted for by heating effects due to the increase in current flow at higher voltages. So it seems that an electrolytic can be used backwards with negligible change in circuit behaviour, as long as it is within these bounds. These bounds change on both the positive and negative sides with capacitor voltage rating. A 1uF/50V capacitor was tested, and it operated up to -16V in reverse bias, which suggests a linear relationship between forward and reverse breakdown voltages. So if you plan to use a capacitor backwards, give it a quick check first to find out where its reverse breakdown begins. A DC current of 0.1uA is a good threshold to use.
at Electrolytic - Open Music Labs Wiki
If I interpret correctly, this experimentation suggests that a reverse bias of 5V on a Capacitor rated for 25V may be well-behaved as I hoped. It also offers a way to test the voltage threshold for the runaway overheating behaviour for a given cap, which may be worth experimenting with. I may well drop down to a 1uF cap and will consider bringing up the rated voltage to 50V to minimise some potential issues, though.