Serial port? As in an RS-232 serial port?
If "yes" then the answer is "no". The RS-232 voltage levels will damage the processor.
Correct answer is Yes, with the correct chip between the 328 and the serial port. Many have done it and many are still doing it.
Search for Arduino Breadboard Serial. Here is a link to a YouTube video that would explain it to you -
I have an Arduino-on-a-breadboard that works just fine this way.
One word of warning - When I use it with an on board serial port it will work at high speeds, when connected to my home computer with a USB to 9-pin Serial cable it won't work faster than 9600.