Solar panel and supercapacitors

I have done a test with the 2 supercaps of 5F, taking sensor samples every 15 mins and sending them via WIFI once per hour. With the supercaps charged to 5V and without the solar panel connected the system was running for 29 hours.


When the voltage of the supercaps goes below 2.90V then they suddenly discharge to 1.90V.
I was checking the voltage directly in the supercaps with the multimeter. In the input of the ESP-12F it's always stable at 3.3V. Until the caps go below 3.3V, then the LDO just passes the current and the ESP-12F gets almost the same voltage as the supercaps.

The schematics so far:

I still have the problem that if it goes below 2.90V it is not able to recover itself with the solar panel, even in strong sun. I need to manually disconnect the jumper between the supercaps and the LDO voltage regulator.

So, could be that there is no sun for more than 30 hours (quite unlikely here), then when its sunny again it wouldn't recover itself.
I'm thinking to put there a transistor or a p-channel mosfet driven by a zener that would cut the connection below 3V, or something like that. With that disconnected it re-charges without problems, even from null. I'm not sure if that would work or could do something else to solve the problem.