Running a single servo at no load, you can connect the servo power to the Vin pin on the Arduino, the USB 5 v has more than enough current capacity for that. For battery power, as Zoom said, 4.8 v minimum. Your AAAs in series could easily be as low as 4.2 v.