You given us no real idea of what you have connected and where. A simple circuit diagram would help a lot. I can't see that 8 x AA batteries (12V) would do anything except perhaps cause some damage.
The servo could be powered by 4 x AA batteries connected DIRECTLY to + and - with the - also connected to the Arduino GND . Directly means not through a breadboard and definitely not from the Arduino 5V pin. That would be a good start.