A buffer won't affect the signal because of delays. You could use 2 74xx125s and two 74xx126s. Just off the top of my head maybe something like this
When the chip is reset for programming the IO pin goes tri-state and the resistor enables U1 and U2. After programming the first thing you do is set the IO pin high to enable U3 and U4 and therefore the XBEE.
Technically you don't need buffers on Tx but it's cleaner and stops the XBEE from trying to decipher all the programming traffic.
It needs a spare IO pin though and I don't know how you would insert U1 and U2 into the signal path unless you have a DIP mega328.
Rob