aargh. Have I done damage? (xbee powered from arduino 3.3V pin)

At the worst, the xbee will draw too much, the 3.3V regulator will get warm, the power will sag, you'll lose comm's.
More likely - you have 4.2 to 5V on Tx out to the Xbee, possibly causing damage to Xbee inputs that are expecting inputs no greater than VCC + 0.5V = 3.8V.
May take longer term to become apparent, on either the Arduino or the Xbee.