LukeZ:
Is there some reasonable combination of capacitors that could perhaps be installed in parallel somehow?
The problem is that for the size available and the capacitance needed, there aren't many options. For the near future, Tantalum capacitors (MnO2 or Polymer) are the most volumetrically efficient options, which is why they stick around.
This is the closest Polymer Part (6.3V is very safe to use on a 5V rail). It's 1500uF instead of 2200uF, but half the ESR and almost half the width:
The downside is that it's a relatively expensive part.