SOLVED: Problem when plugging heavy load devices on the same outlet as Arduino

I don't think just a capacitor would do anything. The Arduino Duemilanove already has plenty of capacitors to aid the regulation feedback loop and there is one in the wall wart too. I tried that and it didn't solve the problem.

You can turn on WDTON with a ISP programmer like the AVR ISP mkII. Atmel Studio makes it change the fuses. Alternatively, you can just use avrdude in a command-line. You can also use another Arduino as an ISP but I don't like connecting all the wires.