100+ encoders

SPI should be fast indeed.

But 32 SPI slaves requires a total of 35 pins to control! (MOSI, MISO, CLK and a dedicated SS for each).