You should not use capacitors on a raw pin output because at switch on a capacitor looks like a dead short and you will exceed the maximum current ratings in the pin. That is why you put a series resistor in the gate of an FET and those have a much smaller capacitance than we are talking about here.
Also reducing the input impedance to the audio circuit helps.