Which XBees are you using or planning to use? Series 1 - point to point or point to multi-point - or Series 2/2.5 - mesh network?
If you are using, or planning to use, Series 1, and there will be more than 2 or them, then one of them needs to be set up in broadcast mode, and you will not be able to change that.
The only XBee, then, that makes any sense to pair with the Arduino with the bluetooth device is the one in broadcast mode.
After initial setup, this 'master' node should also behave like an endnode (so the point-to-point / multipoint thing isn't really
neccesary i reckon).
Then, I reckon you are wrong.