Yes, having 5 isolated groups per row would make it a little trickier mechanically vs having the whole layer connected.
RGB, yes add 10 more control lines.
Could use '1284 for that, 32 IO total. Could achieve same update rate, send out 2 bytes of anode info vs just 1.