seems like one of those moments you should just go back to the drawing board and redo your project (in thought). With your current insights how would you like it to behave and how would you do that.
the only thing you know is that at a certain moment you turn the potmeter and the servo will adjust to a new position. How long will it stay at that new position? until the next new value of the potmeter. So the servo moves, then it rests, it moves, it rests, move, rest, move rest etc
It is not so that the servo goes in one minute from pos A to pos B (that is what your formula looks like), reality (I think) is that will take 5 seconds to go to pos B and the servo will stay 55 seconds at pos B. Your playback tries to average/ smooth the movements in time...
In short you model (represented in the data model) should describe the behaviour you want, nothing more nothing less.
Rethink th behaviour you want, and decide what data is needed..