connecting arduino to a speaker

Or more simply wire a series 1mfd capacitor from the pin to the pot. That will remove the DC bias voltage, leaving only AC audio frequencies to pass. As already stated, you can't directly generate sine wave audio, just square wave that have lots of odd harmonic frequencies, so it will always sound buzzy and harsh.

I'm sure that'll "round off" the square wave, but I don't think that will center it; in other words, you'll get audio output from approx 0-5 volts, not -2.5 to +2.5...?

:-?