Only put one column high, the other low. Then put only one row high and the others low.
Where are the resistors! You need about 270R in line with each column.You never said this was a charleyplexing circuit, you drive those very differently from a normal matrix.
but I tried putting 10k resistors on each column