fungus:
In theory a 328P should run down to 1.8V but you need to drop the clock frequency down. Try setting it to 1MHz internal clock and see what happens (use "fuse bits" for this).
You'll also have to set the brown-out-detection voltage in the fuse bits too, it will brown-out at 2.7V by default with normal Arduino bootloader IIRC. Since current consumption is less at lower clock frequencies just reducing the clock speed may allow running from button cells (if nothing else takes many mA).