According to 32u4 doc:
The USB device address is set up according to the USB protocol:
• the USB device, after power-up, responds at address 0
• the host sends a SETUP command (SET_ADDRESS(addr)),
• the firmware handles this request, and records that address in UADD, but keep ADDEN cleared,
• the USB device firmware sends an IN command of 0 bytes (IN 0 Zero Length Packet),
• then, the firmware can enable the USB device address by setting ADDEN. The only accepted
address by the controller is the one stored in UADD.
ADDEN and UADD shall not be written at the same time.
The way I understand the above is, you set UADD to the received address so its highest bit ADDEN is cleared. Then you send an in packet and the set UADD to received address with highest bit set. This from Arduino IDE 1.6.5 and 1.8.4 didn't do the setting UADD with ADDEN cleared. Was UADD somehow already set by USB controller or is this missing from the code?
else if (SET_ADDRESS == r)
UDADDR = setup.wValueL | (1<<ADDEN);