Oh my... Sometimes it is just better to stay in bed i guess 
I got it all working now... I somehow did not see that pin 0 and pin 1 are connected to the usb-serial converter on the board.
I used those pins during testing.
I am soooo stupid xD
Thank you guys for your help 