Even when I turn the switch on in one of the 5 12v lines, it doesn't change anything.
The 5V output from the arduino. In your diagram the wire coming from the arduino doesn't just connect to the 3.3K resistor.
I checked this morning and the voltage is going from the 5V feed directly into the input pins even when there is no feed from the 12V side.
but it isn't and I can't figure out why.