That doesn't make sense. Perhaps your delay is too short. Try 50 ms.
I wrote a simple version, partly based on the Adafruit code : Arduino Playground - AdafruitMotorShield
You could give it a try.
The Adafruit Motor shield will not work with 5V.
Some motors and servos require a lot of current to start. So check your power supply during operation.
Some DC motors even shortcut the power, if the speed is too low. DC motors come in all sizes and qualities and can have strange unexpected behaviour.
The servo doesn't use much of the Adafruit motor shield. It only uses the +5V and an output from the Arduino.