Due-based board does not execute user code

Hello!
I have recently designed a campervan module around the Due. I based the design on Due schematics, stripped the mega16u2 and the native USB port and added all the functionality I needed.

Two assembled board have arrived. I have checked the power supply and programmed a simple blink program using SWD. The upload was succesful, but the LED wasn't blinking.
After double checking pin naming etc, the board is still not running the program.

I have checked the oscillator waveform, it is correct - and it means that the part of code that setups the 12MHz oscillator is executing. However, any pin I specify is kept in pulled up input mode.

I have a genuine Due, and have programmed it in the same way and it works correctly.
Aside having pullups and pulldowns on most of the pins, my schematic has the following changes:

DUE -> My board
TST grounded -> TST floating
32kHz oscillator in place -> 32kHz oscillator omitted
DHSDM connected to USB -> DHSDM floating
DFSDM connected to USB (39R) -> DHSDM floating
DHSDP connected to USB -> DHSDM floating
DFSDP connected to USB (39R) -> DHSDM floating
VBG connected to RC filter -> VBG floating
Vbus connected to USB -> Vbus floating
JTAGSEL grounded -> JTAGSEL floating
UOTGID connected to USB -> UOTGID floating
UOTGVBOF into USB circuitry -> UOTGVBOF floating

According to the sam3x8e datasheet, none of the changes should affect the chip.
I suspect that the code that executes before setup() (somewhere deep into the Arduino library) requires the USB elements, or the code initializes USB handling and it freezes the procesor.

Is it possible? If yes, am I able to modify this code and disable USB handling alltogether?

My schematic: ElectroBlock.pdf - Google Drive
DUE schematic: https://www.arduino.cc/en/uploads/Main/arduino-Due-schematic.pdf