LED matrix and WS2803

I've been experimenting with WS2803 chip lately, and so far I didn't have any problems with it. However, I tried to implement LED matrix, and then it started to behave the way I don't want. So, what I tried is this:

I've got 18 LEDs connected to WS2803. Anodes are all connected to +5V. Now, I tried to do this (I realize this isn't real LED matrix, but works the same way):

I've split 18 LEDs into groups, there are 6. Only one group of LEDs can be active at the time (so, only 3 LEDs). Switch time is 1ms.

Now, this doesn't really work. LEDs are flickering like crazy. Note that they're flickering regardless of my code running only virtual matrix, or matrix+everything else (I got other stuff going on in my code, but for this scenario I disabled everything else). So I figured that maybe shiftOut function is too slow. Then I tried SPI - same thing. At some point one method (I'm not sure if it was SPI or shiftOut) caused not only flickering, but also completely erratic behaviour of LEDs: they would all flicker, but then for few seconds few of them didn't even turn on, then they all turned on, then flicker again etc. etc.. So, does anyone have experience with driving LED matrices using this chip? Is it too slow to handle that many changes? I know my way around matrices, as I built few of them using different hardware, but bever using WS2803.

Hi, post your schematic and sketch please! (Use code tags).