Can I use a 3.7V battery for a 3.3V board?

I thought that most of the "blue Pill" boards had a 3.3V regulator in between the pin labeled 5V and the 3.3V supply for the chips. It should be safe to plug a 4.2V battery into the 5V pin. (usually on the back...)
image