It is easier to have a adapter from USB directly to TTL-level serial lines.
The old RS-232 uses higher voltages.
Adafruit has both of them, with some explanation:
If you did build that stand-alone, the 7404 is probably used to be able to handle that higher voltage.
I hope you read the comments for this schematic: http://arduino.cc/en/Main/StandaloneAssembly
The RXD is an input, so the 7404 port for RXD is drawn the other way around. Also the /RESET is wrong and there are more problems. So the schematic is a total mess.
Perhaps it's better to start all over again.
Do you have a bootloader programmed ?
Can you upload your schematic (and a photo) ?
Did you already use a normal Arduino ? Building one yourself is easier if you already have used one.