If you use an Arduino Mega instead of two UNOs you will have plenty of pins for both displays.
One problem with your design is that you can't draw enough power from an Arduino pin to light a whole row of LEDs (up to 10). You will need anode drivers. You may want to switch to a design where the current limit resistors are on the anodes and the cathodes are scanned. That way simple NPN transistors can be used for the cathode drivers.