How are you reading the data from the serial port?
Assuming that the servo can actually get to the commanded position, servo.read() can tell you where the servo is. Adding to or subtracting from that value to get a new position is easy.
How can I fix the problem?
The first step is admitting that you have a problem. What problem do you have?