A/D converters basically work by comparing your voltage with another voltage. Therefore, you will have to investigate how your A/D converter measures the battery voltage. Because if the battery voltage is the voltage the microcontroller uses as a reference in case it is its supply voltage, you will always measure the battery to be full.
You need to configure the A/D converter in a way that it gets a stable reference voltage. There are many that have an internal reference voltage (e.g. a bandgap reference voltage) or you can get an external one.
Have a look into the datasheet of your micro and then see how the library you are using is setting up the A/D converter.
The ESP8266 has a built-in voltage reference of about 1V, afaik the ESP32 works the same. The development boards add a voltage divider (typically 220k+100k) to give it a 3.3V range.
So measuring the battery voltage is no problem this way, just be aware of the on-board voltage divider when adding your own.
That just leaves the minor issue of there not being much of a relationship between battery voltage and battery charge level, making this voltage measurement kinda useless for determining battery level.