Inaccurate IR Sensor Reading

You did of course connect servo ground both to supply(-) and Arduino ground.
Leo..