To charge a battery you need a voltage that is higher than the battery voltage but not so high as to damage the battery.
Since you say 3.7v I assume you refer to LiPo cells which are certain to explode if they are overcharged. That means that you MUST use a suitable charger for safety. Even then I would not charge them unattended unless they are in a container that will withstand and contain a fire or explosion.
When you get a suitable charger it will probably tell you what input voltage and current it requires.
If you were charging lead-acid batteries things would be very much simpler.