Check the ATmega328p specs for the maximum clock speed for a given voltage... At 1.8V you can clock upto 4MHz, at 2.7V upto 10MHz and 4.5V upto 20MHz.
You also need to check the brown-out detection is setup correctly - you may get away with using one of the 3.3V board settings (Mini, Pro Mini) to get the Arduino IDE to setup the brownout settings right, but I'm not sure - failing that you can create your own board settings (delve in the hardware/boards.txt file - but back it up first just in case)