I assume you have a normal servo where you control the position.
You want your pot to control the servo's speed. The simplest way is to process the position of the pot at regular intervals. Each time, you measure the position of the pot, calculate the deflection away from the neutral position, calculate the servo speed associated with that deflection and add a corresponding amount to the servo position.
The relationship between pot position and servo speed doesn't have to be linear - you can arrange for it to have a deadband around the 'at rest' position and you can control how quickly the speed increases as you move away from there.
Actually, I lied. The simplest approach is to replace your standard servo with a continuous rotation servo and then you can get the effect you wanted without changing the sketch at all.