What I need is audio to go to one output port, unless an audio jack is plugged to the other output port. So I need one input and two output ports, and no extra button.
My guess is an integrated circuit can detect when output port 2 is plugged into and disconnect port 1 with a relay or similar, or the physical switch button can be at the very end of the audio jack port and be physically pressed by the audio jack itself when plugged.
You can turn the module around and count the inputs as output, the output as input, and remove the button. I would then reroute the button connections and add components at freed up space for the switching.