Hi EtayThere are so many things that can cause that but the bottom line is your ATmega isn't responding. Since you know that it is a good IC since you were able to load the bootloader that leaves the FTDI breakout as you suggest, or almost any aspect of the setup on that board.Can you take a better photo of the actual board as it is when you're trying this, including the connections back to the FTDI, power etc? Let's see what we can see...Geoff
Why are you even bothering with a bootloader for a standalone chip? It's easy enough to use your Arduino (I assume you have one) as an ICSP, include a 6-pin ICSP header in your standalone design and program the target chip using ICSP. You don't need the FTDI converter, and you can use atmeg328p-pu chips without the bootloader, which are cheaper than chips with the bootloader already installed. See http://miscsolutions.wordpress.com/2011/08/09/prototyping-small-embedded-projects-with-arduino/.
A quick test of a loopback on your FTDI will narrow it down. Just jumper the Tx to Rx on your FTDI board and then open Serial Monitor and see if it echos back the characters you type.If it works then your FTDI is fine.Another possibility is that there could be a mismatch of serial baud rate between the bootloader and what speed the IDE is selecting. This depends on which board type you are selecting from the IDE.
When testing the FTDI board no sketch is involved, you are not talking to any Arduino. You are creating a loopback so whatever goes out the Tx pin should be picked back up on the Rx pin. Just use the Arduino IDE and select the Serial port and then open the Serial Monitor. Type some characters and press the Send button. If the same characters show back up in the window below the entry line then your FTDI board is working fine.The baud rate of the chip will be whatever the bootloader is configured to run at. For Optiboot it should be 115200 baud. This is what the Uno board selection is using. The IDE will adjust the baud rate of the FTDI based on which Board you select. So if you selected the wrong board type, say Duemilanove then the IDE will try to use 57600 baud on the FTDI board, and will not match up to the chip bootloader running at 115200 baud. So board selection is important and needs to match up to the bootloader type.
If I used Nicks gammon bootloader do you know the baud rate ?Etay