modern USB-C PD ports always start at 5V. There is a spial chip inside the interface that exchanges data with the connected device to adjust to eventually higher voltage and hher currents. You can switch on this USB port.
Another thing that might help is to add a 1000µF capacitor as near as possible to the 3.3V and a GND pin. If there occures a sudden and fast current-spike the capacitor will provide current. The wires shall be as short as possible to make the voltagedrop caused by the current as small as possible