We have a ESP8266 and and Arduino Mega connected through SPI, where the ESP is master, and the Mega as slave. The ESP uses this communication channel to retrieve values of sensors connected to the Mega.
I am wondering what frequency to set on the ESP for the SPI clock.
Is there a way to calculate the minimal / maximum or optimal setting?
That depends a bit on how fast you can deliver the information the ESP requests. If the information is constantly available and the Mega doesn't have computing intensive work to gather that information you may go up to about 4MHz with the SPI frequency. I don't think there's a technical minimal value but I would not go below about 100kHz just to have a reasonable speed.