I think it's a good idea to include HID only when used but agree with mellis: CDC should always be included by default to preserve auto-reset for new users. That's been a core feature of Arduino for years and I think it's a very nice UX choice. With no CDC there's no serial port and no way to do the auto-reset from the IDE using the existing scheme without custom drivers.
That's not to say that there's not some value for advanced users in being able to strip CDC, too (as Steph has already done). I'd be interested in a clean way for doing this that doesn't involve modifying the core.