PCA9564 parallel bus to i2c-bus controller

I think the may thing i am trying to fix is a mix of communication and multi-master issues.

So this is what works:
-1 Arduino ATMega2560 talking with 8x MCP23017 (it is the master of those devices)

-1 ATMega2560 master talking with 12 ATMega2560 slaves (over I2C).

The problem is, as i perceive it, is that we need to some how get the data (and pass commands) to the 64 sets of ATMega2560 micro controller, with 8x MCP23017 per set...