Turns out it was the relay board, but as an added precaution I connected the motors to their own supply.