Atmega328 can't handle the noise and getting repro

I am using the ATmega328 to run an induction heater. When I am at resonance there can be a lot of high frequency noise on the circuit. It seems the chip gets shut down, and the only way to get it to go again is to reupload my program.

I am using the pwm pin 15 (digital 9). I have LEDs going straight to ground from pins 2 and 4 (digital 0 and 2). I have an input at pin 28 and 23 (analog 5 and 0).

Should I have all input that are not being used grounded? Do I need a resistor from the LEDs to ground? Any other suggestions?

You probably should have resistors on the LEDs, but I doubt that is the problem (may be part of it, though). What are you driving with the PWM? The noise issue may be due to inductive kickback, or something similar; you might try isolating what you are controlling away from the Arduino by using an opto-coupler device (a cheap 4N25/4N26 or similar opto-coupler should be ok if your PWM isn't running at high-frequency). Hook the Arduino's PWM output to the opto-coupler, then the opto-coupler's output to whatever you are controlling (you may need to use a transistor or a buffer IC; I can't remember what the current rating of the 4N2x opto-coupler is).