What library are you using for the display? Check in the library and see what the SPI clock divider is.
SPI can work up to half the MCU clock speed, so you can sometimes get a speed increase by changing the divider in the library from something like 8 or 4 to 2.