Read this it will show you what you are trying to do.
http://www.thebox.myzen.co.uk/Workshop/LED_Matrix.html
Depending on how you drive it you will need to supply power for a whole row or column when you multiplex it.
so the common anodes are connected to a DC voltage, in this case I used +5 volts
No that needs to be switched as well as the cathodes. There needs to be resistors in the cathodes or you need to drive the cathodes with a constant current drive.