High Pitch noise from Leonardo board

No, low drop-out regulators often require ESR in a certain range - too high or too low and
oscillation. Low-drop out regulators have highly non-linear circuitry so that its hard to
satisfy Nyquist's criterion for stability across the area of operation. Go read some
datasheets... For instance, the TPS769xx family (ultra low dropout) has this warning in the datasheet:

The minimum recommended capacitance is 4.7 μF. The ESR (equivalent series resistance) of the capacitor should be between 0.2 Ω and 10 Ω. to ensure stability. Capacitor values larger than 4.7 μF are acceptable, and allow the use of smaller ESR values. Capacitances less than 4.7 μF are not recommended because they require careful selection of ESR to ensure stability.