Well, nearly a year later I've come back to this project. I ended up finding a darlington array that can source the current my RGB matrix will require (UDN2981AT).
To try help ensure I get the circuit right, I've mocked it up. I'm going to produce it using perf-board.
Understanding this is probably going to be a bit of a headache. I've tried to label everything in the mocked version - if anything's unclear, pls let me know.
One thing I'm unsure of, is whether the 4051 multiplexer is going to be okay to control the darlington array.
Also .. the darlington array is going to have to potentially source a max of (20mA*3)*8 (480mA) on a duty cycle of 12.5%, on one output. The unit is rated for a maximum of 500mA.
I'm not great at reading datasheets -> it seems possible, but if anyone could confirm, would be grateful.http://datasheet.octopart.com/UDN2981A-T-Allegro-datasheet-10472.pdf
Advice would be appreciated.