Using an array makes no difference to your communications- the way you’ve explained it.
As you discussed so far, the only lines that need to change for local or remote operation are -
shifter.setPin(index, mySwitch[index].toggle);
shifter.write();
Using serial A B etc is a new twist... where did that come from?
Input handling would go here...
mySwitch[index].currState = digitalRead( switchPin[index] );
I think before anyone can help you get further along, you need to sketch out a conceptual block diagram of what your whole ‘expected’ system is going to look like.