Arduino Due: external power supply and USB connection

Yes, the grounds are missing.