Power the servos separately, using a servo power supply that supplies 4.8-6V and at least 1 Ampere per servo (for small servos), 3A/servo for large ones.
A 4xAA pack will power 1 or 2 small servos, if the cells are fresh.
Don't forget to connect all the grounds.