The specification for Vin is a minimum of 7V. You can feed 5V into the 5V pin but for certain Arduinos you have to be careful not to feed 5V through the USB at the same time; it might cost you your PC.
It will be safe when using a Nano, but not when using e.g. an Uno or a Mega.
To answer your question:"Is my atmega328p of Arduino is fine?" The only answer I can give is try it, not knowing what version etc you are using. sterretje is correct. Try the blink sketch and see if you get anywhere but be sure there is nothing connected to it other then the USB cable.