the main problem is the motor displacement is not reproducible when it rotates back and forth sometimes.
That is called backlash, and the problem is easily solved.
Just make sure you approach the final position from the same direction of rotation, every time. That may require you to overshoot the new position by some fixed amount, and then back up.