Correct, I think. I hadn't thought the math out yet, because I am more focused on a proof of concept. But I figured speed would eventually be an issue, so later I could add a third dimension to the matrix. Essentially, giving the "switches" a second or third bit so that it knows what layer to select. Thus directing the sensor to another ADC.
Also concerning the speed, now that I think about it... I could just make multiple matrices rather than layers. Like 4 - 28 x 24 matrices (depending on the number of inputs on the ADC). Or something to that effect.
But right now I'm trying to figure out how to do the "switching" for the matrix.