dhenry:
It would take about 8.2ms just to put the data to the column shift registers plus the time i need to get that data from the other arduino
(an other 8.2ms at best) which kinda limits my refresh rate.I am not sure how your have structured your hardware but it takes about 0.1ms to shift a byte into a HC595, on a 1MIPS AVR, and much faster if you use hardware spi.
To drive 64 leds, you need to shift at most two bytes and that should be no more than 0.2ms, and likely much lower than that if you are running your chip at 16MIPS.
As fungus has pointed out, your hardware doesn't make a whole lot of sense to me and you may want to rethink how you approach the whole thing.
its 64x64=4096 leds