chaines:
The button batteries measure 4.3V when not attached.They measure 3.2V when the program is running. I'm guessing this is the problem? I'm trying to find a small power source, the whole circuit is only a few milimeters larger than the Atmega chip, so I dont really want to use 3 AAA batteries.
The current being drawn has dropped the voltage down by 1.1V so I'm guessing it's near the limit.
In theory a 328P should run down to 1.8V but you need to drop the clock frequency down. Try setting it to 1MHz internal clock and see what happens (use "fuse bits" for this).
You could also reduce the current getting through to the LED... put two sets of batteries in parallel, etc.