flip flop it or 4051 it?

The '574 is also certainly less wiring than adding a transistor.
Here's the start of what I am envisioning.
'574 shift registers to source current into a column of anodes, and a transistor sinking current from that column (or a row, if you turn the whole thing sideways).
Another set of shift registers controls the transistors. Those can be 74HC595, are not as critical if mated with the right transistor.
Pick an NPN transistor that the shift register can drive into saturation, or use a logic level N-Channel MOSFET.
The OEs don't need to be wired to ground, they can be controlled by an Arduino output as well.