Rather than bring every wire back to the 7219's - why not wire the busses locally...
All the a-segment together, the b-segment etc all the way through to all the dp’s.
Then you'd be looking at 2x 7219 = 2x 16 wires from the display controller to the LED panel. 8x segments and 8x commons.
As noted above in #2 (and allowing for 16x 8-segment displays incl dp).
(EDIT - Tom George beat me to it!)