Switching two batteries with DPDT relay

ViolentValence:
Will I need a capacitor to keep the Arduino going during the switch?

Yes. The value will depend on how fast the switchover is.

Should there be diodes for any kind of hiccup when that happens?

No, but you DO need a diode across your relay coil, to kill the back EMF when you disconnect it. Otherwise the back EMF could fry your Arduino.