But, when nothing is connected to the input, it is still printing LOW (as it is activated somehow).
When connecting it to 5v, it stops printing LOW.
Connecting it to GND or just let it 'float' it will print LOW.
Any idea why a floating input is activated even when i am using the PULLUP internal resistor?
I would suggest ordering another ESP32 as you probably have damaged it. You stated you placed 5V on a GPIO, that usually damages the part. "When connecting it to 5v, it stops printing LOW." The ESP32's operating voltage range is 2.2 to 3.6V . Under normal operation the ESP32 Thing will power the chip at 3.3V.