Lithium batteries are very dangerous if overcharged, among other risk factors. to mitigate this risk, most commercially available batteries are sold with protection circuits built into the connection.
This battery has no protection.
This one is protected.
Solar cells produce a varying voltage at a constant current depending on the available light intensity.
As long your charging module has overvoltage and undervoltage protection you should be good.
In any case don't peer anything directly from your lipo and alsways test the over and undervoltage protection of every charging module before I lengthening it to your project so you know it's working properly.