What is it that you are trying to charge.
Generally, trying to power something off solar power directly is always problematic
due to the fact that Solar panels are essentially current sources and work best if they are directly charging a battery.
You can then regulate the batteries voltage , and use the regulated voltage to power whatever you like.
The battery ensures that in the absence of sufficient sunlight, the powered device still works.
Without a battery, all that will happen when there is insufficient sunlight, is that the supplied voltage will be less than 5V
and how that affects what you are powering depends on what it is.