Have a look at Oleg's board schematic for the USB host controller. He has a neat / elegant way of using the buffer chip OE pins to his advantage by controlling the MISO translation from 3.3V to 5V using the SS signal. Unless the USB host controller chip (Max3241) is actively selected with its SS-pin pulled low, its MISO output is isolated from the SPI bus. Very elegant and effective solution.
Based on suggestions from the forum, I'm trying something similar with the TXB by pulling the SS signals high using 10K resistors. In the past, I have had issues with SD cards on the SPI bus corrupting bootloader or sketch uploads via SPI. Thus, I'd have to remove the SD card before uploading. I hope that the pullup resistor approach will avoid that problem.