xbee interfering with xbee

I have 12 pairs of arduinos that are each connected via xbee pros.
Each arduino only talks to its partner pair (unidirectional) and does not care what the other xbees are doing.

In order to isolate things I tried programming each xbee pro so that there were pairs. Each pair had it DL and MY the same.
So for pair number one, both xbees were programmed with
DL = 1
My =1,

Pair number 2: DL =2 , My = 2
etc.

I have not been able to get thing working so smoothly and I noticed some odd behavior. With all the xbee pairs, one is sending and the other is receiving. The receiving arduino is never transmitting anything. However, I noticed that receiving Arduinos are interfering with each other!
For example, if I have all 12 arduinos transmitting and I turn on one of the receiving arduinos it works as expected. However if I turn on another receiving arduino it suddenly affects the other one and both start missing messages and exhibit more intermittent behavior.

Is there a better way I should be setting up my pairs??