Analog output varies despite having something conected to it

The noise can be mitigated, somewhat, with a 0.1uF ceramic cap from the wiper (pin A4) to ground. It will shunt higher frequency noise to ground. Value not critical.