Grumpy_Mike:
Feeding a voltage into the wrong end of the regulator is not recommended, however I have never come across a case of this burning out the regulator.Was the 12V supply left connected up but not powerd up. That could cause a short on the input side and produce what you saw. If this is the case then a diode would prevent this from happening.
Nope 12V wasn't connected at all 
I must've wired somewhere wrong somewhere... Can't tell now...
dhenry:
What did I do wrong? Do I need put a diode between LM7805 and +5V line?
Typically a reverse-connected diode between 7805's input and output.
Thanks! You mean like this?
Would it drop my output voltage noticeably?
Also I was wondering, if I'm using my own power supply, I don't really have to connect FTDI's VCC to the circuit at all right? Just the ground and RX/TX?