I am using a 2200uf capacitor on the input of the 7805 then on it output I am using a 10uf cap..
When I solder the 7805 on the PCB there is problem, but when I am using the same 7805 outside the PCB and supplying only 5v from the 7805 to the circuit with a long wire, then things are just fine...
There is a clue in there somewhere I think. Your description of the circuit sounds OK, although I would add a 100nF ceramic capacitor in parallel with the 2200uF on the input, and another in parallel with the 10uF on the output. Attached is a schematic of a typical 7805-based supply. The electrolytic on the input can be arbitrarily large. This one shows two 100nF capacitors on the output, these are physically located near different ICs on the board. It's very important to have a 100nF capacitor from Vcc to ground located as close as possible to the ATmega328, and another from AREF to ground.
In a noisy environment like this one, I'd put one 100nF near the 7805 along with the 10uF, another close to the ATmega328, and a third on the AREF pin, just for starters.
If it's just a 328 and a couple LEDs, it can hardly be more than 50mA, yes?
You are correct it just that much of load I have. So you are sure that it will only dray 50ma...??
I was asking more than making a statement. Hard to be sure about anything without a proper schematic.