As long as you replace the battery with some kind of jumper. Most battery holders expect a continuous circuit. An alternative might be to use rechargeable batteries. They tend to run around 1.2V per battery, so four of those would be very close to 5V.
Depending on what is meant by mattloto for the term "robotics", it might be best to simply use 7.2 or 9.6 VDC rechargeable packs that are used in the radio-control hobby. They are plentiful, fairly inexpensive, and lightweight. They will supply more than enough voltage for the regulator, as well as plenty of current to run the Arduino as well as extra peripherals (servos, drive motors, etc).
In order to use them in this capacity, you would want to create some form of "power distribution" system, where the battery can plug in using a standard mating plug for the pack, and has an output plug that will plug into the Arduino's barrel jack, as well as extra leads or plugs to power any other devices. Ideally this would all be implemented on a separate PCB, and there would be regulators implemented on-board to supply the voltages to the extra devices as needed, as well as perhaps a "raw" voltage source (clearly marked and positioned, of course).
