There is no hall sensor.
The ESP32 has several touch sensor pads on portB, as per the documentation link I posted. The OP is using the capacitive touch sensor pads as a hall effect device. Hopefully, you can see how a capacitive touch sensor pad does not work like a hall effect switch. The OP is confused in thinking that a magnet operates a capacitive touch sensor switch.