So why not wire up a matrix of headers, with pairs of pins that you wire your LEDs off to wherever?
It can end up looking like an 8x8 matrix less the LEDs and you can use an MAX7219 to drive them all.
See the attached - every place you see an LED, put a 2 pin connector.
Now combine the 2 pin connectors into strips of headers, and you have a nice compact 8x16 solution for remoting 64 LEDs.
