am I totally wrong?
Not totally wrong but it is more complex than it needs to be.
Simply have a 4051 and wire pots to it.
You know what pot you are reading by the value you put out on the three data select pins.
Like the schematic for this project:-
http://www.thebox.myzen.co.uk/Hardware/MIDI_Footsteps.html
Just replace the foot sensors with your pots.