I'm still playing around a bit with this. I do agree with the way it's shaping up, that CDC should be kept in to preserve the auto-reset, but maybe an override could exist for advanced users. However, HID could / should be automatically included only if it is utilized in a sketch.
In the meantime, here's what I have figured out through experimentation:
To remove all USB libraries, comment out the USB.attach(); line in cores/arduino/main.cpp
To keep USB but omit the HID component, leave main.cpp alone and edit cores/arduino/USBDesc.h and comment out the line #define HID_ENABLED
And finally, if you want HID but not CDC, do like the above line, but comment out the #define CDC_ENABLED line instead.
By comparison of sketch sizes, here is what I get for my current project:
All USB libraries: 30,356 bytes
CDC but no HID: 29,838 bytes
HID but no CDC: 29,742 bytes
No USB at all: 27,632 bytes
I don't know how thorough or effective it is to remove the #HID_ENABLED or #CDC_ENABLED defines - that was obvious and easy but either it's not very effective, or there's just a lot of overhead for USB regardless of what you do with it.
Cheers!