Hi,
What is your magical PV charger, link to specs/data please.
The only instance that I know of that the PV voltage will be much higher than battery volts while the charger is in charge mode is if the charger is a MPPT type.
Please on your circuit LABEL your pinouts, not just numbers, they have names like, GND, Vcc, Batt+, P1, P2 etc.
How big is the PV, link to specs/data.
How big is the Battery, link to specs/data.