LiPo batteries are very fussy about how they're charged (and dangerous if misused) so for any solution that can charge the LiPos you will need to use a battery controller. There is an Arduino clone that has an integral LiPo controller. If you don't use that, you'll need to research how LiPo controllers work and design/build/test your own. Note that some LiPo failure modes can be very dangerous and it's usually recommended that LiPos should not be left unattended while charging.
Other types of rechargeable battery such as NiMH and NiCd are far more tolerant and could be managed with a current limited supply and a simple peak voltage detector.
If you only want the solar panels to be used in preference to the LiPo when available (and revert to LiPo power when there is no solar power available) with no recharge capability, you could do that by arranging for the solar supply to provide a higher voltage than the LiPo, with diodes on both of them to prevent either source discharging through the other. You will suffer some voltage loss from the diodes, and if you're going for maximum efficiency that could be important. Also, it sounds as if your solar cells are producing a far higher voltage than you need and if efficiency matters to you then you would need to choose an efficient way of dropping this down to your required output voltage. For example, you might be able to use a switched mode power supply/buck converter to produce a low voltage regulated output from a high voltage unregulated input. If you use the right type of converter, this could step the solar feed up to 5V even when the solar panels are producing less than 5V, as long as they're producing enough current. (You'd need to look at the voltage/current characteristics of your solar panels to decide whether this was worth doing.)
Also ensure that you isolate the LiPos when the voltage drops below the critical thresholds as they can be damaged by over-discharge.