5V to 3V3 capacitor question

it is unlikely to see a regulator oscillate, but I've see it happen.

Theoretically, the regulator should shutdown when overloaded and remain in the shutdown mode until the excessive load is removed but if there is a delay between the time the load has been shutdown and the time it is "seen" on the output, the regulator can shutdown, see the load is gone and restore power just before the load reengages and begin a repetitive cycle , essentially ossillating. This can occur if the load has some initialization period before enabling the output.