I'd consider adding a servo.write(angle, duration) to make the servo move through the 'angle' and take 'duration' to do it. I like that approach because it puts the problem into a place which is waking up to adjust the servo pulse duration anyway. If I went that far, I'd probably have the servo.attach(), or some new setup-method, take a servo speed parameter to make it work well.
gbulmer, if you are interested in pursuing this I would be happy to consider a speed option for the servo library. Start a new thread and put a link to here if you have some suggestions on how this could be implimented