Digital Pot not working with audio signal

I've been building my speaker for a while and I wanted to control the volume levels with Arduino. The only way I could think of was a digital Pot and since I know they are used in audio circuits I tried it too. I bought a x9c104p and tried it on a simple circuit with the audio. But it didn't work. I tested the pot with an LED and it worked just fine. So I wanna ask if there is some interaction with audio signal I should know about or I might need a different code or is the problem that I'm amplifying the signal afterwards? Thank you for any useful piece of advice and sorry for being so dumb.

You've neglected to post your simple circuit with the mistake in it, so we cannot find the mistake...

So you have a working digital pot. And some code that works with it. And a working speaker circuit. And some code that works with it. But when combining them, it won't work. And you suspect either interference in the signals of the two parts or coding error?