Explain which library example and which part is too slow.
If it is your own code, paste it to your message, attach a ZIP or post a link to repository e.g. GitHub.
I have never used a colour OLED. It will take 16 SPI bytes to draw 8 pixels on a colour OLED. It only takes 1 byte to draw 8 pixels on a Monochrome OLED.
I would expect a similar performance as a 128x128 TFT e.g. ST7735 or ILI9163.
The TFT libraries are very quick.
Your SSD1331 is only 96x64. So will be a lot quicker than the bigger SSD1351 128x128 OLED.
With a monochrome 128x64 it is common to manipulate a buffer in SRAM. Then blit the whole 128x64 to the OLED in one go.
Since a colour OLED requires 16x the traffic, you only draw the area that has changed. The performance of your app comes down to your "intelligence" when choosing the minimal area(s) to update.
David.