Thank you guys, apparently I actually damaged the transistor. I tested blinking an LED as you guys suggested and it just switched from bright to slightly dimmer. I tested again with a new transistor (no soldering yet) and it blinked as it should!
Wawa:
Switching the source is a weird and maybe dangerous setup for the mosfet.
If you want to keep the switch there, then at least connect R1 to gate and source, not to gate and ground.
I actually need to keep that switch there now, oops. So you're saying I should wire R1 like this?