USB battery bank without low current shutoff

This thread of Nick Gammon has me thinking about capacitors, specifically supercaps although the one in his example is a normal one I think.

A 1F supercap like this is less than 20mm diameter.

Could that play a part in your solution?