The power source I am using is a 4 AA battery pack with 1.5V each that connects to the Arduino power jack. (6V)
That's why it can't work.
If you read the specs of an Arduino, you will see that the power jack input should be in the range of 7V .. 12V.
Arduino's power jack's input goes to a voltage regulator which needs at least 7V to operate regularly.
So you should go with 5..6 battery cells (= 7.5 ... 9V).
The following link will give you more information:
Attention: don't use the +5V pin as input when you are operating with more than 5.25V !! You might immediately fry your device.