Converting voltage when there is less than about a volt and a half difference between the input voltage and the output voltage can be challenging, although there are designs such as buck-boost converters that can achieve this.
For you, consider some options.
You can convert the voltage up, then down to 12 volts.
You can convert the voltage down, then up to 12 volts.
You can convert the voltage down to 3.3 volts.
You can convert the voltage down to 5 volts. You probably have a local store that sells a device that plugs into a cigarette lighter for automotive power (and has the protections you should have) and provides a USB connection for 5 volts.
Of course, you may have to also re-think the rechargeable battery.