And the servo's were powered by the Arduino, they are small hobby servo's that run on 4.8v or 6v. Eventually once the project is complete I will be running off external power.
Were?... not any more?- do you have external power now, do you mean?
It's not the voltage that's an issue, it's the current. Even if you have no mechanical load on them, those servos could be drawing 100mA or so each, up to say 500mA if you load them.... it's risking damaging your Arduino, for sure.
As a rule of thumb, any servo "funnies" are best treated by making sure power related problems are off the table by having an external supply, even if it's just a 4xAA pack for testing. Sometimes the problem goes away when you provide external power; if it doesn't, well at least you've eliminated one possible cause.