Powering Arduino makes a difference?

USB is limited to 500mA. Perhaps you are drawing more than that.