It certainly seems twitchy doesn't it? I wonder why? Perhaps a capacitor might smooth out the changes. But what changes I wonder? Maybe if you can measure the voltage you can work out whether the pot is actually giving a fluctuating reading, or if this is some artefact of the way that analogRead works.