Go Down

Topic: Writing an Arduino using ISP without damaging bootloader? (Read 2032 times) previous topic - next topic

jgalak

I've been having a lot of issues programming my Arduino UNO of late, with failure to connect issues from avrdude.  I suspect a USB problem on the computer side.  In the meanwhile, I was wondering if I can use an ISP to program the board.

I have an Atmel-ICE programmer, but I've read that using ISP programming wipes the bootloader.  Is there any way to avoid this? 

DrAzzy

Not without combining the bootloader and sketch hex files, then writing that manually using avrdude. Only being able to do a whole chip erase is part of their security model - that's also the only way to clear the lock bits (which can be set to keep people from reading the flash).

But - you can always put the bootloader back with "burn bootloader" and that ISP programmer.
ATTinyCore for x4/x5/x61/x7/x8/x41/1634/828/x313 megaTinyCore for the megaavr ATtinies - Board Manager:
http://drazzy.com/package_drazzy.com_index.json
ATtiny breakouts, mosfets, awesome prototyping board in my store http://tindie.com/stores/DrAzzy

jgalak

Thanks.  I was afraid of that.  Oh well, back to beating my head against the USB brick wall (which is the correct solution anyway....)

CrossRoads

"Not without combining the bootloader and sketch hex files,"

which the Sketch:Export Compiled Binary option will create for you.

Using an ISP like Nick Gammon's sketch can download that for you.
http://www.gammon.com.au/uploader
and which runs on my Standalone Programmer board as well
http://www.crossroadsfencing.com/BobuinoRev17/


Designing & building electrical circuits for over 25 years.  Screw Shield for Mega/Due/Uno,  Bobuino with ATMega1284P, & other '328P & '1284P creations & offerings at  my website.

pert

Thanks.  I was afraid of that.  Oh well, back to beating my head against the USB brick wall (which is the correct solution anyway....)
Why are you so afraid of just doing a Tools > Burn Bootloader when you want to go back to using the bootloader again? It's just as easy as doing an Upload Using Programmer; you just need to select the correct board from the Tools > Board menu first.

jgalak

Mostly because I've not done it before, and was looking for a quick fix to the USB problem.  I'm sure I can burn the bootloader, but that becomes a new thing to do.  I don't need another thing to do right now :)

bperrybap

I'm sure I can burn the bootloader,
I wouldn't be so sure. avrdude has an issue with libusb and due to the way avrduded is coded, atmel ISP programmers like AVR dragon, and AVR ISP MK II, can't be used with the IDE to burn a bootloader.
I'm not sure if this affects the atmel ICE as well, but it is likely.

I've been trying to get this fixed for MANY years.
I even provided the patch to avrdude to fix it (it is less than 5 lines of code) but the main author refuses to pull it in claiming it is a libusb issue, when in fact it is avrdude that creates the issue.

I've also tried for years to get the arduino.cc devs to change the way they use avrdude when burning a bootloader to work around it. They could do it in one avrdude command vs two.

What causes it is doing two back to back avrdude commands. The second avrdude command will fail as it won't be able to locate the USB ISP device because avrdude reset the USB on the fist command and when the second avrdude command runs, the USB ISP device has not enumerated yet so the device doesn't exist at that point in time as far as the OS is concerned.

The fix in avrdude and the work around in the IDE are both really small and simple and don't affect anything else
but so far after many years, it remains an open issue in both savanna for avrdude and on github for the arduino IDE.

---bill


CrossRoads

"atmel ISP programmers like AVR dragon, and AVR ISP MK II, can't be used with the IDE to burn a bootloader."

Baloney. I use my AVR ISP MK II all the time, here it is connected to a Mini2560 board burning a bootloader.

and to a 1284P


I disabled the Jungo driver that Atmel Studio wants to use, and use the Libusb driver instead. Have been for 7 years now.

Designing & building electrical circuits for over 25 years.  Screw Shield for Mega/Due/Uno,  Bobuino with ATMega1284P, & other '328P & '1284P creations & offerings at  my website.

CrossRoads

Designing & building electrical circuits for over 25 years.  Screw Shield for Mega/Due/Uno,  Bobuino with ATMega1284P, & other '328P & '1284P creations & offerings at  my website.

bperrybap

"atmel ISP programmers like AVR dragon, and AVR ISP MK II, can't be used with the IDE to burn a bootloader."

Baloney. I use my AVR ISP MK II all the time, here it is connected to a Mini2560 board burning a bootloader.
While the Atmel USB ISP devices definitely can program a bootloader using avrdude, as far as I've seen
it still cannot be done using the Arduino IDE and this has been an issue for many years.
Actually, it has never worked.
I now mostly use a USBasp device with my custom f/w which is much faster than the Dragon,
I occasionally use my dragon as well, but not with the arduino IDE using the avrdude it comes with.
To use the Arduino IDE requires using a custom version of avrdude with a fix.
The low level USB code in avrdude for the jtagMKII devices (which includes the dragon and MK II) resets the USB just before it exits for AVR dragon and MK II devices.
The result is that on back to back commands the 2nd command will fail.
Here is the full output from IDE version 1.8.5 attempting to burn a bootloader into an UNO type device that shows the issue and error on the 2nd avrdude command.
(The first avrdude command worked just fine)



home/bill/Documents/devel/arduino/arduino-1.8.5/hardware/tools/avr/bin/avrdude -C/home/bill/Documents/devel/arduino/arduino-1.8.5/hardware/tools/avr/etc/avrdude.conf -v -patmega328p -cdragon_isp {program.extra_params} -e -Ulock:w:0x3F:m -Uefuse:w:0xFD:m -Uhfuse:w:0xDE:m -Ulfuse:w:0xFF:m


avrdude: Version 6.3, compiled on Jan 17 2017 at 06:01:25
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "/home/bill/Documents/devel/arduino/arduino-1.8.5/hardware/tools/avr/etc/avrdude.conf"
         User configuration file is "/home/bill/.avrduderc"
         User configuration file does not exist or is not a regular file, skipping

         Using Port                    : usb
         Using Programmer              : dragon_isp
avrdude: usbdev_open(): Found AVRDRAGON, serno: 00A20000756E
avrdude: jtagmkII_getsync(): sign-on command: status -1
avrdude: jtagmkII_getsync(): sign-on command: status -1
JTAG ICE mkII sign-on message:
Communications protocol version: 1
M_MCU:
  boot-loader FW version:        255
  firmware version:              1.01
  hardware version:              1
S_MCU:
  boot-loader FW version:        255
  firmware version:              1.09
  hardware version:              7
Serial number:                   00:a2:00:00:75:6e
Device ID:                       AVRDRAGON
         AVR Part                      : ATmega328P
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PC2
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         ByteDelay                     : 0
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                  Block Poll               Page                       Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom        65    20     4    0 no       1024    4      0  3600  3600 0xff 0xff
           flash         65     6   128    0 yes     32768  128    256  4500  4500 0xff 0xff
           lfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           hfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           efuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           lock           0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00
           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00

         Programmer Type : DRAGON_ISP
         Description     : Atmel AVR Dragon in ISP mode
         Vtarget         : 4.0 V
         SCK period      : 8.00 us

avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.15s

avrdude: Device signature = 0x1e950f (probably m328p)
avrdude: erasing chip
avrdude: reading input file "0x3F"
avrdude: writing lock (1 bytes):

Writing | ################################################## | 100% 0.06s

avrdude: 1 bytes of lock written
avrdude: verifying lock memory against 0x3F:
avrdude: load data lock data from input file 0x3F:
avrdude: input file 0x3F contains 1 bytes
avrdude: reading on-chip lock data:

Reading | ################################################## | 100% 0.05s

avrdude: verifying ...
avrdude: WARNING: invalid value for unused bits in fuse "lock", should be set to 1 according to datasheet
This behaviour is deprecated and will result in an error in future version
You probably want to use 0xff instead of 0x3f (double check with your datasheet first).
avrdude: 1 bytes of lock verified
avrdude: reading input file "0xFD"
avrdude: writing efuse (1 bytes):

Writing | ################################################## | 100% 0.06s

avrdude: 1 bytes of efuse written
avrdude: verifying efuse memory against 0xFD:
avrdude: load data efuse data from input file 0xFD:
avrdude: input file 0xFD contains 1 bytes
avrdude: reading on-chip efuse data:

Reading | ################################################## | 100% 0.05s

avrdude: verifying ...
avrdude: 1 bytes of efuse verified
avrdude: reading input file "0xDE"
avrdude: writing hfuse (1 bytes):

Writing | ################################################## | 100% 0.06s

avrdude: 1 bytes of hfuse written
avrdude: verifying hfuse memory against 0xDE:
avrdude: load data hfuse data from input file 0xDE:
avrdude: input file 0xDE contains 1 bytes
avrdude: reading on-chip hfuse data:

Reading | ################################################## | 100% 0.05s

avrdude: verifying ...
avrdude: 1 bytes of hfuse verified
avrdude: reading input file "0xFF"
avrdude: writing lfuse (1 bytes):

Writing | ################################################## | 100% 0.06s

avrdude: 1 bytes of lfuse written
avrdude: verifying lfuse memory against 0xFF:
avrdude: load data lfuse data from input file 0xFF:
avrdude: input file 0xFF contains 1 bytes
avrdude: reading on-chip lfuse data:

Reading | ################################################## | 100% 0.05s

avrdude: verifying ...
avrdude: 1 bytes of lfuse verified

avrdude done.  Thank you.


/home/bill/Documents/devel/arduino/arduino-1.8.5/hardware/tools/avr/bin/avrdude -C/home/bill/Documents/devel/arduino/arduino-1.8.5/hardware/tools/avr/etc/avrdude.conf -v -patmega328p -cdragon_isp {program.extra_params} -Uflash:w:/home/bill/Documents/devel/arduino/arduino-1.8.5/hardware/arduino/avr/bootloaders/optiboot/optiboot_atmega328.hex:i -Ulock:w:0x0F:m


avrdude: Version 6.3, compiled on Jan 17 2017 at 06:01:25
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "/home/bill/Documents/devel/arduino/arduino-1.8.5/hardware/tools/avr/etc/avrdude.conf"
         User configuration file is "/home/bill/.avrduderc"
         User configuration file does not exist or is not a regular file, skipping

         Using Port                    : usb
         Using Programmer              : dragon_isp
avrdude: usbdev_open(): did not find any USB device "usb" (0x03eb:0x2107)

avrdude done.  Thank you.

Error while burning bootloader.





So you can see the 2nd avrdude command failed because avrdude couldn't locate the device.
This is because the previous avrdude command reset the USB and the USB ISP device had not yet enumerated.

--- bill

CrossRoads

If it wasn't so late, I'd pull out some bits and show the results of a successful bootload. Everyboard I make, I bootload with the Atmel AVR ISP MKii and then load Blink with an FTDI Basic clone to demo basic functionality, all using the IDE.

I haven't seen the error you demonstrated. Is it because I am running the 2 programming steps manually from the IDE and there is more time between steps than what you are doing running from avrdude directly?
Designing & building electrical circuits for over 25 years.  Screw Shield for Mega/Due/Uno,  Bobuino with ATMega1284P, & other '328P & '1284P creations & offerings at  my website.

bperrybap

I haven't seen the error you demonstrated. Is it because I am running the 2 programming steps manually from the IDE and there is more time between steps than what you are doing running from avrdude directly?
Not sure what you mean by "I am running the 2 programming steps manually from the IDE"
When you click on [Tools]->Burn Bootloader
The IDE does it all under the hood and will use two avrdude commands to burn a bootloader.
One command to set the fuses
And a 2nd command to actually burn the bootloader.

The problem is that avrdude resets the USB which resets the Dragon and I'm assuming the MK II as well.
The device then has to startup again and re-enumerate with the OS.
If the 2nd command comes in before all that has happened, then the current avrdude code will not see the device.
(The patch I created simply looks for a few seconds rather than just once before giving up)

It appears that at least on the Dragon that Atmel was really dumb and resets the actual processor when a USB bus reset happens. This is really really dumb. There is no need to reset the device's processor just because the USB bus reset. The power up code on the Dragon takes a few seconds.

Now if the USB host is really slow, then the device might have enough time to come up and be ready by the time the 2nd avrdude command runs.
But it is many seconds so this is normally not really a possibility.

However, some versions of the IDE had a 1 second sleep between commands.
Not sure what the current IDE does, but that can add margin that might make it work on slow hosts/PCs.

Another possibility is that Atmel could dramatically speed up its code or change its design so that the enumeration could happen quicker, then the issue might not show it self.

The issue is very real and needs to be corrected in avrdude as it is avrdude that is creating the problem.

The IDE could work around this issue by simply doing everything in a single avrdude command vs multiple.

After a couple of years of pestering, I was able to get westfw to alter the makefile for optiboot to burn the bootloader all in a single avrdude command rather than multiple commands.

--- bill


pert

I can reproduce the problem bperrybap mentioned using AVRDUDE 6.0.1 with the libusb-win32 driver. However, if I use AVRDUDE 6.0.1 with the libusbK driver or AVRDUDE 6.3.0 with either driver it works correctly. The results are the same for me across 3 different computers, Windows 7 (32 bit and 64 bit) and Windows 10.

So it's not a simple always works/never works sort of a problem. Luckily not many people use the affected programmers and of the people who do not all of are affected.

pert


bperrybap

I don't do windows.... or mac so I can't speak for what is happening there.
The avrdude that comes with the linux package doesn't work and still has the issue.
Here is the original issue from back in 2011.
https://github.com/arduino/Arduino/issues/650

The issue is caused by code in usbdev_close() in usb_libusb.c
Here is the latest code from avrdude 6.3

Code: [Select]
static void usbdev_close(union filedescriptor *fd)
310 {
311  usb_dev_handle *udev = (usb_dev_handle *)fd->usb.handle;
312
313  if (udev == NULL)
314    return;
315
316  (void)usb_release_interface(udev, usb_interface);
317
318 #if defined(__linux__)
319  /*
320   * Without this reset, the AVRISP mkII seems to stall the second
321   * time we try to connect to it.  This is not necessary on
322   * FreeBSD.
323   */
324  usb_reset(udev);
325 #endif
326
327  usb_close(udev);
328 }


usb_reset() resets the USB and causes the device to disappear.
The fix is that if the avrdude code is going to call usb_reset() to reset the USB bus, it also needs poll for a few seconds when looking for a device to give it time to recover.

Joerg didn't like this solution as he uses avrdude in an undocumented way to locate and print out all the devices on his network and this added polling time would effectively break what he likes to do since he scans for many devices that may not be there or turned on.


A simple solution would be to modify the IDE to not use 2 avrdude commands.
Not only would this keep the issue from happening but it would make burning bootloaders a little bit faster.
--- bill

Go Up