Go Down

Topic: Avoiding mismatch errors from verification (Read 3590 times) previous topic - next topic

pschaeffer

I have a modest suggestion for the 'Arduino as ISP' sketch. Basically, it should verify that a chip erase has been one before trying to store anything in the flash. The problem is that NOT doing a chip erase doesn't produce any errors, until the verification phase. In other words, the upload of a sketch will appear to work, with some sort of mismatch error being reported during verification.

The chip erase operation is easy to detect. If it hasn't been done, the flash can't be updated (although update attempts don't appear to return any errors). This program could be reported several ways. Perhaps the first few byte of the verification data could actually be set to a error message ('Chip erase required before updating flash - try Upload Using Programmer')

This problem can be resolved with the existing code just by using the 'Upload Using Programmer' file menu entry. Still making this change might help a few folks avoid mysterious 'mismatch' verification errors.

Of course, I found out about this the hard way... By NOT using the  'Upload Using Programmer' file menu entry when I needed to.

If it matters, I can make the code changes and submit them (assuming this is a desirable change).

Here is a different idea. Change the Arduino IDE so that any attempt to use the Upload button produces an error message if a programmer has been specified. The message could be something like 'Upload Using Programmer required with programmer xxxxxxxx'. Indeed, the Upload button could even invoke 'Upload Using Programmer' if a programmer was detected in the boards.txt file.

The overall idea is to make the upload process a bit more foolproof.

Coding Badly


Two things...
1. IDE version?
2. You have jumped to a solution without describing the actual problem.

westfw

The problem is that "verification" is done by avrdude, not by the ArduinoISP sketch.  Avrdude uses low-level commands to write one page at a time, and then more low-level commands to read back the memory (one page at a time), and compares the read-back version to the sent version.
(Also, the protocol used between avrdude and arduinoISP is pretty stupid, and doesn't offer any good ways to send back a detailed error indication like "I don't think I can program that page because it hasn't been erased.")

Quote
Change the Arduino IDE so that any attempt to use the Upload button produces an error message if a programmer has been specified.

Huh?  The IDE won't use the configured programmer unless you use the  "upload using programmer" option.  (or it's shortcut - SHIFT-UPLOAD.)  (Is that what this is about?  SHIFT-UPLOAD used to be "verbose upload" and now (post 1.0) it is "upload using programmer."   This makes a bunch of old advice obsolete...)

Quote
If [chip erase] hasn't been done, the flash can't be updated

That's not strictly true.  You can change "one bits" into "zero bits" in flash, even if there is already data there.
And the bootloader NEVER does a chip-erase; it can do a one-page erase before each page is programmed, assuming that avrdude doesn't decide to "optimize" by skipping pages that are already all ones.

pschaeffer

Coding Badly,

IDE version is 1.0.5.

The actual problem (as I see it) is that it is possible (easy) to get a quite mysterious error message (verification mismatch) using the IDE, when the IDE (or even the ArduinoISP sketch) could detect the problem in advance and produce a much more intelligible error message or perhaps avoid the error by taking a different programming path.

What actually happens is this. Boards.txt has the upload protocol for some board set to stk500v1 (as in mighty_opt.upload.protocol=stk500v1). Of course, it doesn't have to just be stk500v1. Any ISP type programmer would probably produce the same error. Note that I specified stk500v1 because I was using the ArduinoISP sketch (running on an Uno R3) to load other sketches (blink for example) onto my target system (ATMega1284p).

You then click the standard upload button. The IDE generates the Avrdude command below.

C:\Users\Peter\arduino-1.0.5\hardware/tools/avr/bin/avrdude -CC:\Users\Peter\arduino-1.0.5\hardware/tools/avr/etc/avrdude.conf -v -v -v -v -patmega1284p -cstk500v1 -P\\.\COM4 -b19200 -D -Uflash:w:C:\Users\Peter\AppData\Local\Temp\build8679411521618747254.tmp\Blink.cpp.hex:i

Note that -c is set to stk500v1 (from boards.txt) and -p is atmega1284p (my target chip). Of great importance is the -D flag. The -D flag blocks the chip erase making sure that the sketch will never actually be stored on the ATMega1284p chip. Of course, you eventually get an error message.

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0159
         0xee != 0xec
avrdude: verification error; content mismatch

Now if you (intelligently) use the 'Upload Using Programmer' menu item the Avrdude command is.

C:\Users\Peter\arduino-1.0.5\hardware/tools/avr/bin/avrdude -CC:\Users\Peter\arduino-1.0.5\hardware/tools/avr/etc/avrdude.conf -v -v -v -v -patmega1284p -cstk500v1 -P\\.\COM4 -b19200 -Uflash:w:C:\Users\Peter\AppData\Local\Temp\build8679411521618747254.tmp\Blink.cpp.hex:i

Since the -D flag is missing a chip erase gets done. The following is from the Arvdude output

avrdude: NOTE: FLASH memory has been specified, an erase cycle will be performed
         To disable this feature, specify the -D option.
avrdude: erasing chip
avrdude: Send: V [56] . [ac] . [80] . [00] . [00]   [20]

Of course, you don't see all of those messages about erasing the flash if you use the standard upload button.

What could be done about this? Several things come to mind. The ArduinoISP sketch could simply refuse to try to program any flash if it had not received a prior chip erase command (V [56] . [ac] . [80] . [00] . [00]   [20]). The standard Atmel chips don't allow flash to be programmed until a chip erase command has been received.

Changing the ArduinoISP sketch would work, but the sketch has no easy way to report errors back to the user. Most folks probably don't run with verbose turned on and wouldn't see any special response the ArduinoISP sketch might generate.

However, the upload button could detect that Avrdude is about to be invoked with a combination of options that won't work. Setting -c to stk500v1 with the -D flag simply won't work. Of course, many other possible values for -c won't work the -D flag. The one value for -c that will work with -D is -carduino. Why? Because -carduino uses the bootloader and the bootloader is allowed to update the flash even with the -D flag. In other words, chip erase is not required with -carduino because the bootloader updates the flash in a very different way.

If the upload button detected  that -c specified a programmer (other than -carduino) it could simply produce an error message telling the user to switch to 'Upload Using Progammer'. Of course, it could also just remove the -D flag and proceed. This will work. However, the downside is material. Chip erase wipes out the bootloader (if you have one installed) and that might be a very unexpected side effect.

pschaeffer

westfw,

"The problem is that "verification" is done by avrdude, not by the ArduinoISP sketch.  Avrdude uses low-level commands to write one page at a time, and then more low-level commands to read back the memory (one page at a time), and compares the read-back version to the sent version."

Agreed. However, the ArduinoISP sketch can reasonably expect that page write commands will fail (with no error indications) if a chip erase command has not been previously received.

"(Also, the protocol used between avrdude and arduinoISP is pretty stupid, and doesn't offer any good ways to send back a detailed error indication like "I don't think I can program that page because it hasn't been erased.")"

Agreed. My (lame) idea is to send the error message back in the verification data. This is quite doable. However, only someone with verbose turned on would ever see the error message (and then only if they knew to look for it).

"Huh?  The IDE won't use the configured programmer unless you use the  "upload using programmer" option."

This is not correct. I went into boards.txt and set mighty_opt.upload.protocol=zebras. I then clicked on the standard Upload button and got

C:\Users\Peter\arduino-1.0.5\hardware/tools/avr/bin/avrdude -CC:\Users\Peter\arduino-1.0.5\hardware/tools/avr/etc/avrdude.conf -v -v -v -v -patmega1284p -czebras -P\\.\COM4 -b19200 -D -Uflash:w:C:\Users\Peter\AppData\Local\Temp\build7604730678725251018.tmp\Blink.cpp.hex:i

Of course, Avrdude didn't like it,

"That's not strictly true.  You can change "one bits" into "zero bits" in flash, even if there is already data there."

Agreed. However, as a practical matter... You have to do a chip erase to get a useful result from flash programming. Stated more broadly its always true that you can only change "one bits" into "zero bits". Chip erase simply forces all of the flash bits to "one bits" so that setting some to zero gives that correct code in flash memory.

"And the bootloader NEVER does a chip-erase; it can do a one-page erase before each page is programmed, assuming that avrdude doesn't decide to "optimize" by skipping pages that are already all ones."

Agreed. However, I am using the ArduinoISP sketch to upload sketches.

The underlying idea is that the Upload button shouldn't issue an Avrdude command with a -c set to anything other than arduino and the -D flag set, because the combination isn't likely to work. Instead it should point the user to the 'Upload Using Programmer' menu entry or just drop the -D flag (probably too dangerous because of the side effects).








Coding Badly

IDE version is 1.0.5.


Got it.

Quote
The actual problem (as I see it) is that it is possible (easy) to get a quite mysterious error message (verification mismatch) using the IDE, when the IDE (or even the ArduinoISP sketch) could detect the problem in advance and produce a much more intelligible error message or perhaps avoid the error by taking a different programming path.

What actually happens is this. Boards.txt has the upload protocol for some board set to stk500v1 (as in mighty_opt.upload.protocol=stk500v1). Of course, it doesn't have to just be stk500v1. Any ISP type programmer would probably produce the same error. Note that I specified stk500v1 because I was using the ArduinoISP sketch (running on an Uno R3) to load other sketches (blink for example) onto my target system (ATMega1284p).


The problem has nothing to do with avrdude nor Arduino ISP.  The problem is you are using a[font=Courier New] boards.txt [/font]entry designed to be used with a bootloader.  I can no longer recall why bootloader based entries include the dash-D but they do.

If you post the complete[font=Courier New] boards.txt [/font]entry I can probably help.  Or, this may help...
http://forum.arduino.cc/index.php?topic=56114.0

westfw

The problem you have is not what you think it is, and is so rare that I've never seen it before.

ArduinoISP, and the Bootloader use the same communications protocol (STK500v1.)  When you hit the UPLOAD button and the IDE generates
C:\Users\Peter\arduino-1.0.5\hardware/tools/avr/bin/avrdude -CC:\Users\Peter\arduino-1.0.5\hardware/tools/avr/etc/avrdude.conf -v -v -v -v -patmega1284p -cstk500v1 -P\\.\COM4 -b19200 -D -Uflash:w:C:\Users\Peter\AppData\Local\Temp\build8679411521618747254.tmp\Blink.cpp.hex:i
It THINKS that it is talking to the arduino bootloader.  The bootloader does not support chip-erase (but does per-page erases), and it would be inappropriate to use the chip-erase option when talking to the bootloader.   But you have ArduinoISP in there, and it is seeing the commands instead of the bootloader, and it SHOULD have a chip-erase because ISP programming doesn't support a per-page erase.)   Normally the bootloader actually runs at a different bitrate (115200 for optiboot) than ArduinoISP (19200 or 9600), which prevents this sort of confusion.  It seems that your boards.txt either has the wrong bitrate specified for the bootloader, or perhaps it should contain the magic commands that prevent the IDE from ever using the bootloader. (Note that the bitrate in boards.txt is JUST for using the bootloader.  Using "upload using programmer" causes the bitrate to be determined from other data (perhaps hard coded in the IDE.)

Go Up