You need a battery pack capable of delivering the current required by the motors. Ordinary AA cells normally won't cut it, except (possibly) when brand new.
The stall current for the motors used on that platform is 2.8 amperes, TT Geared Motor with Back Shaft (180rpm 6v L Shape) - DFRobot and each motor will briefly draw that when it starts up. I'm surprised that it works on USB, as the current draw is far higher than allowed. You risk damaging the computer providing the power.