Servo goes to 180 as soon as power is applied

Servo horns, the little plastic pieces that linkages attach to, generally have several holes. Move the linkage closer to the center to get slower movement/more torque. Move the linkage farther from center to faster movement/lower torque.

The key will be finding a position where the torque is still adequate to move the piece.