There are theoretical limits. A SPI display can only run at 8MHz on a Uno or Mega2560. If you use a Zero or Due, not only can you go faster but they can use DMA. The < Adafruit_ILI9341.h> library expects SPI.
The < Adafruit_TFTLCD.h> library expects a 8-bit or 16-bit parallel interface. The 16-bit using two full ports on a MEGA2560 can run at maximum speed. As a general rule, Adafruit write pretty efficient code but you can always make slight improvements.
The < UTFT.h> library is always going to be slower.
As Oliver said, it depends on your display, your Arduino and what type of jobs you want to do.
Put some numbers forwards. Then you can find out whether something is theoretically possible or not.
David.