Hmm, or maybe put 0.3V schottky diodes on the cathode drivers so they could only sink current and not be able to source.
If the anodes for that color are already low, then driving the cathodes high at the same time just seems like overkill, would you agree?