Problem sort of solved. It is the Serial Monitor and/or monitoring the serial port in the terminal. I don't know why exactly this is causing the problem, but I have a few hunches.
It might still be in the code, but I will figure out whats up later. At least I have concluded it is not in the module hardware.