I am planning to hook up the aux connector to a switch so I can switch between two output sources. I don't have a switch that can turn on/off all the 5 pins coming from one aux jack at once but I do have a spare arduino
Relays are the easiest solution. There are analog switch ICs which are more "high tech" but you have to be careful with the design to avoid getting switching noises into the audio. With relays you should also have resistors to ground (maybe 10K to 100K) to keep unconnected inputs/outputs from "floating".
Or possibly you can just use some regular switches or a rotary switch...
There is no need for anything digital unless you want to control the switching digitally.
Note that it's OK to connect two or more inputs together. i.e. You can connect two (or more) amplifier inputs to your computer's line or headphone output at the same time.
However, you shouldn't "short" two outputs together... Don't connect your computer and TV to the same amplifier at the same time. Using two audio sources at once requires a mixer.
Like sterretje says - The regular Arduino doesn't have an analog output (no digital-to-analog converter). And the analog-to-digital converter is only 10-bits, and NO it's NOT fast-enough for good-quality audio.