Solar panels are really only good for directly charging batteries where the solar panel voltage is higher than the battery voltage thats being charged.
Solar panels dont work well with switching converters especially in low light, as switching converters need something that approaches a voltage source when they start, and solar panels dont behave like this.
What usually happens is that as the solar panel voltage rises as the light intensity rises, eventually the switching converter will try and start and will try and draw current from the panel , which cant deliver enough so the panel voltage collapses, and the switching converter then goes into an unstable state where it may sit there drawing current but not switching, ie it wont start.
The simplest solution is usually the best, ie use the solar panels to directly charge a battery via a blocking diode, and then run everything else off that battery.