Tricky!
Have you tried powering the servo after the main board?
I would suspect that it would have little effect since the setting time of the servo would be many times longer that the start-up time of the board, but may be worth a go.
It may also help to set the servo position on power-down and also initialise to the same position on power-up (maybe before applying servo power)