I had understood that the switched pole was isolated electrically from the relay/arduino/PSU.
It is.
But the long lengths of wires act as an antenna and pick up electromagnetic interference from close by mains wiring.
You can try wiring it up using screened cable. Note only ground one end of the screen on a cable not both to prevent ground loops.