Controlling servo without needing to use delay

(the servo arm is connected to a spring so the power of the (now) unresisted spring should enable a return).

Not a good idea. The servo normally goes through a worm gear and it is not a good idea to try and drive those backwards.

Any ideas?

Use the blink without delay method to write your program.