Help understanding ISP programming, bootloading, cores, USB to serial

I am trying to learn to embed AVR’s into my own projects without a commercial Arduino board and have them easily reprogrammed by another, less experienced user with updates I provide. Here is a little background and several questions. I’ve been doing a lot of reading but getting more confused. I think I need more of an overview before diving into the details of avrdude. Thanks in advance for your help.

Background:
I started learning to program embedded controllers with an Arduino Uno a few weeks ago. That went very smoothly. I then wanted to use AVR’s standalone or in a minimal Arduino configuration in my own hacking projects; e.g. R/C camera controller, coupling an off-the-shelf timer to a wireless light controller, etc.

I ordered a couple of ATMEGA328p’s with the bootloader and a couple of ATtiny85’s bare from Spark Fun.

I was able to use the MIT HLT ATtiny85 tutorial to program the 85 on a breadboard without any problems using Arduino as an ISP. I can’t figure out how to do the same with the 328. But, of course I am able to program the 328 (since it has the bootloader) by placing it in the Arduino socket.

I just received an AVR Dragon and will begin learning to use it.

So here are my questions. If there is not a “?” I’m looking for confirmation that my thinking is correct.

I want to have a standard programming interface (usb or 2x3 ISP headers) on my projects so that an end user can plug to the USB port on their PC and upload new firmware that I post or email to them. The key points seem to be:

Bootloader to elliminate continued need for hardware programmer
i. I can buy a bootloaded chip or bootload it with my hardware programmer and then routine firmware updates can be done without the programmer using serial programming.
ii. I was able to program the ATtiny85 because the Arduino as ISP sketch made the Arduino function as a hardware programmer.

USB to serial converter to establish hardware link to the boot loaded chip.
i. This is built into the Arduino board
ii. It can be a separate piece plugged between a boot loaded chip and the PC. It can cost as much as a hardware programmer but it should be “safer” and simpler for the end user.
iii. There are many available and they seem to support different subsets of the AVR family. Which ones are inexpensive (for end user) and will work with 85 and 328 chips?

Software to load hex file to chip.
i. avrdude
ii. I assume one could write a “batch” file (Windows) with avrdude command lines to automate this for the end user. They would need to install winavr, download the hex file and batch file, plug in their USB to serial adapter, and execute the batch to program the chip.
iii. Could it be done from the Arduino IDE by plugging directly from PC to serial converter to project ISP (no Arduino board just a boot loaded chip)? I assume I would need to develop the Arduino suport files so that my project (chip, clock speed, etc.) would appear as a menu choice on the boards menu.

A chip specific “core”?
i. I don’t know what this is unless it is a port of Arduino commands to make them work on different AVR’s.
ii. Is it used by the compiler?
iii. Are they specific for different compilers or different IDE’s or different programming (as in chip uploading) methods or just different chips?
iv. Was this part of the HLT files I downloaded to program the ATtiny using Arduino as ISP?
v. Is this what the Arduino Tiny project is?
vi. When I compile a sketch in Arduino IDE does it use this core to create the appropriate hex file for the target chip?
vii. Would I then need to edit board.txt and other files for each chip I want to create hex files for? Is this what HLT did for the 85? Can it be done for the 328 to generate hex files for ISP loading into stand alone 328’s?

Just a quick partial answer from another newbie whoes done it but you can purchase £5 USB programmer from HK that can connect via [edit]6[/edit] pins to a standalone device that can then be re-programmed via a simple batch file to upload a new .hex copy of your update sketch to your users device.

regards
Simon

Link to example
http://www.ebay.co.uk/itm/USBASP-USBISP-AVR-Programmer-AVRDude-USB-Port-/180666590548?pt=UK_AudioElectronicsVideo_Video_TelevisionSetTopBoxes&hash=item2a10916554
Si

Neuport:
I was able to use the MIT HLT ATtiny85 tutorial to program the 85 on a breadboard without any problems using Arduino as an ISP. I can’t figure out how to do the same with the 328. But, of course I am able to program the 328 (since it has the bootloader) by placing it in the Arduino socket.

Search "Circuit (targeting an AVR on a breadboard)"...

Step 1.5 is missing from the instructions at the top.

I want to have standard 2x3 headers on my projects so that an end user can plug in (USB) and upload new firmware that I post or email to them.

If you use a Teensy, the end user will only need a USB cable.

i. can buy a bootloaded chip or bootload it with my hardware programmer and then routine firmware updates can be done without the programmer using serial programming.

No. The problem with your statement is terminology. "Serial programming", in AVR vernacular, is what you did with the ATtiny85. This is a more accurate statement...

i. can buy a bootloaded chip or bootload it with my hardware programmer and then routine firmware updates can be done without the programmer using TTL serial communications.

ii. I was able to program the ATtiny85 because the Arduino as ISP sketch made the Arduino function as a hardware programmer.

Yes.

USB to serial converter to establish hardware link to the boot loaded chip. ... i. This is built into the Arduino board

Yes. Either as an FTDI chip or as an Atmega8U2 processor.

ii. It can be a separate piece plugged between a boot loaded chip and the PC.

Yes. Typically called a USB-to-serial converter. Sometimes "TTL" is included in the name.

It can cost as much as a hardware programmer

Only if you are not shopping in the right places.

but it should be “safer” and simpler for end user.

No way to respond. You have not defined "end-user". My typical end-user builds air conditioner controllers using AVR processors so they would prefer to use their own programmer.

iii. There are many available and they seem to support different subsets of the AVR family. Which ones are inexpensive (for end user) and will work with 85

None. The ATtiny85 processor does not have a serial port and really does not have enough memory or support for a bootloader.

and 328 chips?

retrolefty likes this one...

Software to load hex file to chip. ... i. avrdude

You want confirmation that AVRDUDE can be used to upload hex files? If that is the case, the answer is "yes".

ii. I assume one could right a “batch” file (Windows) with avrdude command lines to automate this for the end user.

Probably not. There is no way to know when the batch file is written what serial port is assigned to the USB-to-serial converter.

They would need to install winavr

No. AVRDUDE does not need the full WinAVR install to function.

download the hex file and batch file, plug in their USB to serial adapter, and execute the batch to program the chip.

Yes. With the caveat that the user will need to provide the serial port.

iii. Could it be done from the Arduino IDE by plugging directly from PC to serial converter to project ISP?

By "ISP" do you mean the six pin connector?

I assume I would need to develop the Arduino suport files so that my project (chip, clock speed, etc.) would appear as a menu choice on the boards menu.

If you use the 328 processor and an Arduino bootloader (like Optiboot), one of the existing choices will work just fine.

A chip specific “core”? ... i. I don’t know what this is unless it is a port of Arduino commands to make them work on different AVR’s.

Yes.

ii. Is it used by the compiler?

Yes.

iii. Are they specific for different compilers or different IDE’s or different programming (as in chip uploading) methods or just different chips?

For the most part, just different processors.

iv. Was this part of the HLT files I downloaded to program the ATtiny using Arduino as ISP?

Yes.

v. Is this what the Arduino Tiny project is?

Yes. Arduino Tiny is meant to be more than a core but time has prevented it from evolving very far past that.

vi. When I compile a sketch in Arduino IDE does it use this core to create the appropriate hex file for the target chip?

Yes.

vii. Would I then need to edit board.txt and other files for each chip I want to create hex files for?

Maybe. For the 328 processor, you can very likely use an existing entry.

Is this what HLT did for the 85?

Sort of. The Arduino IDE supports multiple boards.txt (and programmers.txt) files. I believe Alessandro Saporetti's core (the one available from HLT) includes a board.txt file. In other words, there was no need to edit the boards.txt file that is included with the Arduino IDE.

Can it be done for the 328 to generate hex files for ISP loading into stand alone 328’s?

Yes.

Thanks guys those replies were helpful. Coding, thanks for taking the time to go through the details. I had studied this quite a bit. Putting it down in my post helped gel my thoughts and your corrections and confirmation strengthed it more.

iii. There are many available and they seem to support different subsets of the AVR family. Which ones are inexpensive (for end user) and will work with 85
None. The ATtiny85 processor does not have a serial port and really does not have enough memory or support for a bootloader.

So, for the ATtiny85 the other person would also need a hardware programmer? Something like what simplesi posted or this Pocket AVR Programmer from Spark Funhttp://www.sparkfun.com/products/9825?

I assume I would need to develop the Arduino suport files so that my project (chip, clock speed, etc.) would appear as a menu choice on the boards menu.

If you use the 328 processor and an Arduino bootloader (like Optiboot), one of the existing choices will work just fine.

It's a beautiful thing! I chose the "Arduino BT w/ ATmega328" as the target board and Arduino as ISP programmed my breadboarded, bootloaded, 328P no problem. Thanks!

Neuport:
So, for the ATtiny85 the other person would also need a hardware programmer?

Yes.

Something like what simplesi posted

Maybe. There is a "3.3V" in the graphic. Hard to say if that means it supports targets running at 3.3 volts or requires the target to be running at 3.3 volts.

or this Pocket AVR Programmer from Spark Fun http://www.sparkfun.com/products/9825?

Yes.

BTW, I tried out the Dragon this evening and had a tough time getting it to talk to an ATtiny85 in circuit. Eventually I discovered that it would not work because I had PB2 (SCK) connected to drive an optoisolater. When I connected the optoisolator to a different pin all worked fine so somehow that optoisolator is causing errors in the communications for the Dragon but was not a problem for the Arduino as ISP. Maybe there is an override in the Dragon that could get past that problem?

I have not been able to get the Dragon working with the Arduino IDE. I'll try again tomorrow. I tried editing the boards.txt and programmers.txt as in the suggested links but get java errors when I try to upload a sketch using my edited entries and the Dragon.

Added to boards.txt

attiny85dragon.name=ATtiny85 1Mhz (w/ Dragon via ISP)
attiny85dragon.upload.using=dragon
attiny85dragon.upload.maximum_size=8192
attiny85dragon.build.mcu=attiny85
attiny85dragon.build.f_cpu=1000000L
attiny85dragon.build.core=attiny45_85

added to programmers.txt

dragon.name=Atmel AVR Dragon in ISP mode
dragon.communication=usb
dragon.protocol=dragon_isp

Errors

Binary sketch size: 1690 bytes (of a 8192 byte maximum)
java.lang.NullPointerException
at processing.app.debug.AvrdudeUploader.getProgrammerCommands(AvrdudeUploader.java:106)
at processing.app.debug.AvrdudeUploader.uploadUsingPreferences(AvrdudeUploader.java:68)
at processing.app.Sketch.upload(Sketch.java:1603)
at processing.app.Sketch.exportApplet(Sketch.java:1568)
at processing.app.Sketch.exportApplet(Sketch.java:1524)
at processing.app.Editor$DefaultExportHandler.run(Editor.java:2293)
at java.lang.Thread.run(Thread.java:619)

Maybe there is an override in the Dragon that could get past that problem?

How much current is needed to drive the optoisolator?

I tried editing the boards.txt and programmers.txt as in the suggested links but get java errors when I try to upload a sketch using my edited entries and the Dragon.

I believe the "boards.txt" section has to include the bootloader entries. Try this (be sure to review the fuse values; they may not be correct for you)...

attiny85dragon.bootloader.low_fuses=0xE2
attiny85dragon.bootloader.high_fuses=0xD7
attiny85dragon.bootloader.extended_fuses=0xFF
attiny85dragon.bootloader.path=empty
attiny85dragon.bootloader.file=empty85at8.hex
attiny85dragon.bootloader.unlock_bits=0xFF
attiny85dragon.bootloader.lock_bits=0xFF

I am using PS2501A-1-A-ND from Digi-Key. From the data sheet Vf=1.2-1.4 at If=10mA. With a supply of 5V I calculated 360-380 ohms for the current limiting resistor. I went with a 330 ohm resistor. I just measured 11.12ma (w/ constant voltage) through the input side. In the final project I am using the optoisolators to interface with a remote control; for now I have an LED on the output side for debugging.

I just connected the Dragon to my project and verified that it still can read and program my chip in circuit but only if I the optoisoloator is not connected to the SCK pin. Using Arduino as ISP I also can program my project just fine and it doesn't matter if I have anything else connected to SCK. If I have my optoisolator driven on the SCK pin my debug LED flashes away during programming but doesn't cause any problem. With both programmers I can see the square wave during programming.

Out of curiosity I connected the debug LED directly to SCK; and the Dragon worked fine! I measure 8.9mA through the LED with a constant voltage. I started increasing the resistor on the optoisolator input and at 510 Ohms the Dragon worked, the measured current through the input of the optoisolator was 7.5mA with a constant voltage input.

Apparently if a current drawing component is connected to SCK (and maybe some other ISP pins, I haven't tried them all with various loads)somewhere around 10mA Dragon stops working. Something to definately keep in mind when choosing pins for a project.

I wonder if this explains a lot of the reliability during programming complaints about AVR Dragon on the forums.

Thanks again for the suggestions.

I'll try the boards.txt modification later.

I wonder if this explains a lot of the reliability during programming complaints about AVR Dragon on the forums.

Good observation and certainly a reasonable theory.