The output of the 5V regulator is a minimum of about 1V lower than the battery voltage. The protection diode for the power jack takes away another 0.6V. The ATmega328P is designed to run on as little as 3.8V (at 16 MHz). The Arduino should continue to work correctly with the battery as low as 4.8V if you use the Vin pin and 5.4V if you use the power jack.