Power Arduino and servo separately ?

I have an arduino, two motors rated at 1.5v with a motor driver and a Hitech HS-322HD servo. When powering it all with a 3.7v 1000mah lipo, it only runs for about 5 seconds and shuts down, which I can understand since the servo is listed as 4.8-6.0 Volts.

I also have a 4xAA pack. When I power it all using it, it runs just fine.

When I power just the arduino and two motors with the lipo it works fine.

However, when I power the arduino and the two motors with the lipo and then run the power and ground wires from the servo to the AA battery pack this happens:
The board and motors power up and work fine. The servo powers up and just jumps to one side and doesn't move ( in every other scenario the servo rotates around like the code tells it to).

I would like to run the board/motors and servo from different power sources, but it seems to freak the servo out for some reason.

Is the 4-AA pack's ground connected to the aruino ground? It should be.

groundfungus:
Is the 4-AA pack's ground connected to the aruino ground? It should be.

You internet people are all wizards. That worked, thanks!