LED matrix, How to use fewer pins?

I do not.
I use SPI.transfer() for everything, nice & fast. I have not used the equivalent of shiftout(), which is what LED library does, in over 2 years now.
A lot of times I use direct port manipulation even to control the SS line.

Couple of transfers in void setup to initialize it (them),
then blink without delay kind of routine to make updates.