Extra TTL interface(s)

It seems like I'm set up to possibly loose data with a muxing setup, even if I write my handler with an interrupt driven system to receive the data. Not to mention what happens if I actually need to send data...

What's the best way to prevent this data loss? Would a buffer stuck in front of each muxed interface be appropriate or is there a better way I'm overlooking??