As jim-p said, the voltage reference is completely stable and independent of VCC level. It works well.
For the previous version of the ESP32 the accuracy was good between 0.5V and 2.5V And for the new ESP32-S3 it has improved and is not a problem anymore. Anyway better stay away from the rails.
So, in the worst case you will have good accuracy from 0.5V to 2.5V. Adjust the voltage divider to that range and test if it is enough for you. For testing the battery level should be ok. I have done and it works very well.
Put the capacitor close to the MCU pin, and take several samples in your code and average them.
And the ADC comes already calibrated, if I'm not wrong. What will be not calibrated are your resistors, so you have to adjust in software anyway.