Well not really.For a start you will not be able to achieve full duplex, that is sending and receiving at the same time. Only one device will be able to transmit with the rest receiving, the changeover protocol will be tricky to cope with. The sample rate will be slow giving poor quality. The lack of D/A on the Arduino will restrict the sample size to 8 bits or require an external D/A. The lack of buffer memory means the you are limited to the "clever" stuff you can do to alleviate these things.The closest project I know of is this:- http://feilipu.me/2015/04/29/xbee-walkie-talkie/
but now I already have a base from where to start.