May sound silly, but this is done quite a lot in HiFi circles where they appreciate a GOOD quality pot.
Put a little box in front of your stereo, with an audio input and an audio output. Connect the input to whatever source you're playing from. The output goes to the amplifier. Between the input and output you place a digital volume control IC; there are several kinds. Also include an Arduino of your choice and an IR receiver. Bob's your uncle.
Don't want a separate box? Place the circuitry described above in the audio path just before the existing pot. Simply disconnect the wires/tracks that go to the pot and redirect them through your circuit.
Don't forget the decoupling capacitors to block any DC offset from going into and out of your circuit.
It's not very complicated and actually a nice little project, go ahead!