Understanding the components of a LED matrix

20mA on 8 pins is also too much.

It isn't too much unless you are building for NASA.

In that case, you can add a swich (npn/pnp, depending on your design) on the 595's output.