INTP:
3.5mm jacks have an extra pin that detects a plug being inserted. A lot of 3.5mm female ends made for PCB will be clear and you can get a good look at how they work. Just google some up and look for 5 pin varieties.
If the OP is using a jack-socket connected to his Arduino the "the left audio and ground wire from a phone" will not be coming from a phone.
(By the way I am not saying your interpretation of what is really required is wrong! )
...R