As Leo says, traditionally, the switch would be between the charger/battery circuit and the regulator. That would eliminate all battery drain but still allow charging.
However, it appears the regulator has a Chip Enable pin. If there's a way to cut the trace that ties that pin high, and connect a switch that switches it between high and low, that would be very close to the same as an on/off switch.
Also, I notice on the bottom of the board there's a pad labeled "EN". I suspect that has to do with the processor, not the regulator, but you might investigate what that EN pad is for. It's possible the designers have provided a way to shut things down.
Edit: Yes, from the schematic, it looks like that's the CE pin of the ESP32, and it's pulled high through a resistor. But you could ground that pad to bring the CE pin low. But I don't know how much current the ESP would still draw. So it might not be a full shutdown.