Serial Port Switching w/ 74HC4066

I have two devices that require independent serial communications w/ the Arduino so I am using a bilateral switch, the 74HC4066 ( to switch both the RX and TX lines between them and the Arduino. I'm enabling and disabling the RX and TX lines independently via two control signals coming from the Arduino. Essentially, when I want to talk to device A, I disable device B and enable device A, etc. using these control signals.

My problem: the switch is not triggering properly when I have the RX and TX lines from the Arduino connected to the 4066. In fact, I have a HW switch connected between the Arduino and the 4066 for use in troubleshooting. If I short the lines so that RX and TX are connected to the 4066 then the already-enabled switch turns OFF. If I open the lines, the enabled switch turns ON. For some reason the 4066 isn't happy w/ the RX and TX signals from the Arduino connected to the device.

I'm totally confused at this point. Does anyone have an idea what might be going on here? This arrangement works fine w/ other signals (e.g. audio), but for some reason not with RX and TX lines.

I'd really like to stick with this arrangement if I can just get it working properly.

Thanks for any advice!