For anyone stumbling upon this issue: I have been able to play around with it some more. And I fixed the issue.
The Problem was the code I was using was waiting for a serial Connection using: while(!Serial).
Which in hindsight makes total sense that it shouldn’t work. No Cable -> No Serial Connection.
But what’s peculiar is, that this even stopped the Power led’s to turn on, so I was assuming it was hardware related and didn’t even look at the code.