If I take out all pullup and pulldown resistors, then planes 1 and 4 are always on at full brightness, planes 2 and 3 behave as instructed. This is kind of the opposite of the case when I have the pullups in place and set all planes to off. In that case planes 2 and 3 are on at full brightness.
It's got to be something related to the pullups/pulldowns, but I can't figure out what is wrong...