The arduino should only reads the voltage of the sound incoming from the cable and not add some distortion to it because i'm sending nothing to the analog input of arduino, only receiving the information through it.
Not quite, arduino also presents a load to the signal source. You can minimize effect, simply inserting a resistor in series with arduino input, 1 - 10 k. More info on your hardware set-up would be necessary to provide more accurate advise.
think about a simple buffer isolating the arduino from the audio source. any dual opamp is probably the easiest solution.