I have this issue with several Leonardo and micro boards.
You should check the 5V power supply when powering via Vin (oscilloscope, look for 4kHz sawtooth signal).
On all of my boards, there is a sawtooth of about 800mV ptp @ 4kHz on the 5V supply !!
The frequency changes slightly when changing input voltage from 7V to 12V.
C12 is the indeed the capacitor making the acoustic noise, it is "singing" with the frequency of this sawtooth.
According to the schematics of the Leonardo and micro, the voltage regulator used on these boards should be the NCP1117, but on the boards I have, there is no number printed on this component, so I suspect an alternative was used, with these annoying results...
For a project I have for generating audio signals, this noise was unacceptable, so I cut out the regulator and powered everything with an external LM7805.