dmjlambert:
Nothing wrong with compiling the hex file using the IDE and then uploading to the chip using DFU.
Now it gets a bit confusing. Sorry for the naive questions, I so far did not venture deeper into the secrets of bootloaders than to simply use the ISP to burn a new one to some Nano clones. Then I plug in the USB to the board, select Nano in the Board menu of the IDE and download the code to the board.
I want to achieve the same with a new design having the 32U4 as the first step, then afterwards maybe also with the AT90usb1286. How can this be done? For example take the 32U4? I have the ISP signals available, so I can burn the leonardo bootloader to it. It will overwrite the DFU and the IDE sees the board like a leonardo. Is that a wrong assumption?
What does it mean compiling the code with the IDE and upload with DFU? In my understanding the DFU is the pre-burned bootloader for USB capable AVR's, which can be used to upload code via the Arduino IDE by selecting the right board....is that not the case?
You're going to have to decide what bootloader you will be using, the Leonardo bootloader or the DFU bootloader that comes pre-installed on the chip.
If you pick the Leonardo bootloader and you are using a Leonardo bootloader, then you have built a Leonardo and you can simply use the Arduino IDE to compile and upload onto it.
If you decide to use the DFU bootloader, you compile the code with the Arduino IDE and export the compiled binary. That gives you a hex file. Then download and use whatever DFU tool you want to use and from the command line or from the DFU tool's GUI, upload the hex file to the chip. Or as tf68 mentions you can use avrdude 6.3 to upload from the command line. If you choose the DFU bootloader, I am not sure exactly where you get a copy of it, other than it is on the chip when it is purchased from ATMEL. Board makers such as Arduino would have overwritten it with their own bootloader. I'm sure there is an image of it somewhere on the Internet or on the ATMEL site perhaps.
One advantage of using the DFU bootloader is the VID and PID are owned by ATMEL and since they are pre-loading the chip with the bootloader, they are obviously not expecting you to buy your own VID and PID like Adafruit and Arduino expect you to do for custom boards. If you are making a board for other than personal use, you will need to make sure you are not using somebody else's VID and PID without their permission. See this Adafruit FAQ "How does the new '8u2 affect Arduino-derivatives", this applies to Leonardo or Teensy derivatives:
You are getting into a big world of complexity for this point of avoiding an FTDI chip. If you are interested in space FT230X is something to consider. You also mention in this thread you are expecting to use the USB for other things such as interfacing with a flash or SD, which is unlikely for you to do since it is not a USB host.
I recommend to go ahead and prototype using a Leonardo and Teensy with their native bootloaders and see how far you get with the other requirements of your project before you worry too much about bootloaders.
dmjlambert:
You are getting into a big world of complexity for this point of avoiding an FTDI chip. If you are interested in space FT230X is something to consider. You also mention in this thread you are expecting to use the USB for other things such as interfacing with a flash or SD, which is unlikely for you to do since it is not a USB host.
Do you know if the FT230X is compatible with the FT232RL? Can I simply replace one with the other with minor modications? Is there any other Arduino compatible board using that chip? I mean it's preatty tempting to use, smaller size and half the price, looks like win-win 
I don't believe there are any Arduino compatible boards using it yet. I think it is interesting, too. I've read about it but not purchased one yet. You can google and find various posts were people roll their own Arduino boards or prototype with it. It all starts with getting one of those breakout boards from the link I gave, and experiment and prototype with it, and then decide.
dmjlambert:
I don't believe there are any Arduino compatible boards using it yet. I think it is interesting, too. I've read about it but not purchased one yet. You can google and find various posts were people roll their own Arduino boards or prototype with it. It all starts with getting one of those breakout boards from the link I gave, and experiment and prototype with it, and then decide.
It seems there are, well, not with the FT230X but with it's bigger brother the FT231X, which has the DTR pin available.
I found an old forum thread with the recommendation to go for the FT231X:
http://forum.arduino.cc/index.php?topic=131940.0
As well as an actual Arduino compatible board:
Sparkfun also has a breakout board using that chip:
Price-wise from DigiKey:
FT232RL: 4.00 EUR
vs.
FT231X: 2.14 EUR
Atmega32U4 6.00 EUR
vs.
Atmega328P 3.40 EUR
If I do the math, designing in a USB-capable AVR versus having a main-stream AVR accompanied by a smaller, cheaper FTDI module, I'm inclined to think the latter is the better bargain. Slightly larger space, slightly less SRAM, but lots of experience, more code space. I like a challenge and a new field to explore, but maybe the time is not ripe for that yet.
It seems Atmega launched a new variant of the 328P, the 328PB which has additional features, and costs less, see a comparison from DigiKey site:
Atmega328P in 32TQFP 3.38EUR
Atmega328P in 32VQFN 3.54EUR
Atmega328PB in 32TQFP 2.58EUR
Atmega328PB in 32VQFN 2.66EUR
Atmega pubslihed a comparison document:
Differences between 328P and 328PB
They say it is not a drop in replacement, for sure, here are the more critical diffefences I can see on first glance:
- not fully pin compatible, a GND and a Vdd pin of the original 328P had to make place for I/Os.
- it seems that the device signatures are different (Chapter 3.6), which will cause trouble with the Arduino IDE...any way to circumvent it?
- current consumption is higher
It is definitely a chip to consider, together with an FT231X, even together they cost less than a single 32U4. As much as I would love to have a board with a 32U4, less and less speaks for it.
BTW, I found potentially a way to service 2 USB devices thru a single USB port: in a new design I've recently got first prototypes, I have an FTDI and a MP3 chip, both with USB signals. I can route both U+/U- pin pairs in parallel to the USB port. Then I use a simple transfer gate/power switch to either shut down FTDI or MP3, the other has access to USB. This way I can either use the IDE to communicate with the board or use the on-board SD-card as USB device to download files. I still have a lot of experimenting to do, but first results are promising.