Yes you can.
The standard brownout setting will let the chip run all the way down to 3V without too many issues.
I've tested a couple in the lab and found that the timing holds steady as long as you can provide the current at that voltage.
We didn't have to change the chip, nor any fuse bits to make it work - they just do.