One thing that can happen is it burns out. With inadequate supply the circuit
may malfunction and pass full supply through the servo's motor, until it overheats
and gets damaged.
I doubt that the motor will be damage due to low voltage. I've never heard of RC hobby servos being damaged by trying to power them from somewhat discharged batteries.