You could use a 3 pole switch, circuit wise it should work on a DC level but in the real world is it capable of the speeds and isolation the USB requires. You would not open the ground. Not opening the data lines will cause problems. This is definitely not recommended and it is possible to generate a transient that may damage your computer and or Arduino.