Atmega32u4 from 5V to 3.3V

If I have a custom board that has the Atmega32u4 using the Leonardo bootloader and I want to convert from 5V to 3.3V I believe I just modify the USB circuitry so it is no longer on the same power supply as the micro. Someone I was talking to mentioned that the crystal will have to be dropped from 16Mhz to 8Mhz as well. Can the same exact bootloader for the 5V, 16Mhz board work fine on the same board at 8Mhz 3.3V or will it have to me modified in some way? I know nothing about the bootloader or how it works.

Yes, it works.