If you can find an old RC electric car anywhere (yard sale or thrift store) you can probably connect the Arduino in place of the RC receiver and re-use the motor drivers, motors, wheels, etc. I've found that RC cars often sell for $4 at thrift stores like Savers.
If you want commercial parts you can get the chassis, motors, and wheels for $30 from http://www.robotshop.com/dagu-mr-basic-mobile-robotic-platform-2.html You will need a motor driver shield to drive the motors. For $36 you can get a plastic chassis with four separate motors: http://sumaoutlet.com/1pcs-4wd-robot-smart-car-chassis-kits-arduino-car-with-speed-enc-p-3709.html Many other choices if you look around the RobotShop site.