How many people will operate this panel? How many hands do they have each? This tells you how many transitions the MCU might have to deal with per second.
Only one person with hopefully no more than 2 hands.
Welcome to the forum.
What is the application that needs 100+ encoders?
What precision do you need?
Would 100+ potentiometers do the same job, I gather you will have people turning them for some reason.
What do you need the Arduino to do with encoder results.
What is its output?
I want to make a programmer for synthesizers that can store several patches. The problem with potentiometers is that if you switch patches they're pointing the wrong way and once you turn them the current stored value may jump by quite a large amount, so I'd like to use encoders to only ever increment or decrement existing values but never set a definite value. Precision isn't that important.
The arduino will output midi cc data.
128 encoders need 16 MCP23017 chips (so you need two I2C buses for that), plus two for the interrupts (bringing it down to four - too many for an Arduino? No problem for the ESP8266) and another 8-port MCP23008 to reduce that to 1 interrupt.
In this case the ESP8266 may come in handy. Only 11 total I/O but you can use any two pins as I2C bus, allowing for total 5x I2C buses (controlling up to 40 MCP23017 chips) and one spare line to handle the incoming interrupts. It also has more speed than Arduino which will be helpful.
No matter what it's not going to be a simple project in any sense of the word.
I'm happy to learn and I'm in no rush to finish this project. And actually I have a NodeMCU with the ESP8266 lying around. I'll keep this in mind, thanks. Haven't worked with interrupts in combination with i2c yet, but I guess it will just be something as "hey, drop everything you do and read the incoming data right now"?
Maybe you may use only one encoder and a switch to choose what function it should have?
If you NEED to have so many encoders for some reason I think you need a lot MCUs to read only a few of them and send results to main MCU. For example I would use bunch of ATMega48 (because they are cheap on eBay in DIP package) and let one to read about 5 of encoders and send data to main Arduino/standalone ATMega328.
I hope this won't be necessary as I only expect 1 or 2 inputs at a time. Using a switch is the very thing I try to avoid, because it implies menu diving that you have on most synthesizers and it can be very annoying.