Is USB input connected to the voltage regulator?

So I bought this battery pack with a USB output. http://www.allspectrum.com/store/product_info.php?cPath=52&products_id=1280&osCsid=8b2be41dd16020e7961129e01b31ab05&sdesc=BATTERY+HOLDER+FOR+4+x+AA-CELL+%28WITH+USB+CONNECTOR%29+%2B+SWITCH+Model+%23+BH341USB

However there is no voltage regulator in there and its outputting 6V. Will this damage the board if I power it through the USB plug?

The ATMEGA is rated to accept 6 volts maximum as a power supply, I believe. But, with batteries, 1.5volts is just a nominal voltage, and it may reach up to 1.7 each when they're all new, which ... will not be good.

Are you loading that with 4 alkaline batteries (1.5V each) or 4 rechargeables (1.2V each)? I think you'd be okay with rechargeables - measure the output voltage and see what it is.

The ATMEGA is rated to accept 6 volts maximum as a power supply,

No it is not!!!

Section 26 of the data sheet shows:-

Absolute Maximum Ratings
Maximum Operating Voltage … 6.0V

Stresses beyond those listed under “Absolute Maximum Ratings” may cause permanent dam- age to the device. This is a stress rating only and functional operation of the device at these or other conditions beyond those indicated in the operational sections of this specification is not implied. Exposure to absolute maximum rating conditions for extended periods may affect device reliability.

No way must this be construed as being rated to accept 6V.