I wonder what exactly those pumps do to the power supply,
They are inductive and make the input impedance more inductive. This alters the phase angle between voltage and current and raises the overall impedance. Think of an inductor as a negitave capacitor, it electrically removes some / all of the capacitance you put across the input. This applies to DC because the interference spikes are an AC component of the applied voltage.
Would it be a good idea to solder a ceramic cap across their connections?
Yes. The closer to the terminals the better, even inside the motor like this:-
