I've now tried 1K pullups instead of the 5.6K pullups. The behavior is pretty much the same, except that the planes which are "ghosting" (slightly lit up besides the planes that really should be on) are now brighter, but still not fully bright.
I've managed to figure out one more important detail: The planes which are "ghosting" are always planes 2 and 3 (planes 1 and 4 are fine). When I turn all planes off, the "ghosting" planes (2 and 3) turn on at full brightess.
There's definitely something wrong around the anode planes 2 and 3...