Custom ATSAMD21 board fails USB enumeration at 1.8V

I'm trying to get a custom Arduino board to work under 1.8V
Bootloader version 1.8.11

I've set the NVM waitstates to 3(and confirmed in the flash viewer in the IDE that its set to 3)
And disabled the BOD33.

When running the bootloader on 3.3V it works as normal
But when i which to 1.8V it gives me an USB error:

Windows has stopped this device because it has reported problems. (Code 43)

A request for the USB device descriptor failed.

And looking in the bootloader source code, i can see it get stuck at this line in file "board_driver_usb.c:

line 182: while ( (pUsb->DEVICE.DeviceEndpoint[ep_num].EPINTFLAG.bit.TRCPT1) == 0 );

Have you guys any suggestion to things i can try?

Morten, Denmark