The mystery of serial,Diecimilla and factory reset

Hi all,

I thought I had a problem with my Duemillanove w ATMega328; and so I thought I'd perform a "factory reset" - that is, burn the bootloader on flash. For this, I'm trying to use an Arduino Diecimilla as a programmer, and the Duemillanove as a target.

Needless to say, I have a problem flashing the bootloader - and I have already seen:

I pretty much agree that the problem is the auto-reset done performed by serial (more details below) - however; that is not the mystery for me.

The mystery is that, just a couple of months ago, I performed the exactly same procedure (Diecimilla programmer, Duemillanove target, Arduino 0018 IDE, Ubuntu 10.04) - and it worked flawlessly, with no problems whatsoever! :o

So, if anyone has an idea how come everything worked few months ago - maybe it will be possible to see what is different now? And, if it is possible to perform the bootloader-burn easily again (meaning, without performing any hardware hacks)? :-?

While I admit the problem is (most likely) the serial reset, below are a few more details. Otherwise, the only significant change that I can remember, is that there was a kernel update update recently - and currently I have Linux 2.6.32-25-generic; could that, also, be the problem?

Originally, I was so surprised the first 'bootloader flashing' went so smoothly (I had written very bad code, that kept resetting the Duemillanove each 500ms, thereby making it impossible to reprogram regularly from the IDE), I took a picture:

You'll notice I'm using wire with alligator clips, going directly to the ICSP header (at that time, I hadn't seen the tutorials above - but I believe the ICSP header pins are anyway connected to P10, P11, P12, P13 as in the tutorials); also, I'm using extern power supply on the target (Duemillanove). I cannot remember whether I did anything with the reset button (as described in How to upload code without auto reset) - I just remember that:

  1. First I chose Diecimilla/ATMega168 as Board (the programmer), and choose ArduinoISP sketch, and upload that to the Diecimilla
  2. Second I chose Duemillanove/ATMega328 as Board (the target), and Tools / Burn Bootloader / w/ Arduino as ISP

and the target Duemillanove had the bootloader reprogrammed succesfully..

Right now, I have pretty much the same setup; the Duemillanove board seems not to be exactly the same one, though. Also, unlike the original case, where the target Duemillanove was essentially bricked - here the bootloader is, basically, still fine - as I can still program the Duemillanove directly through USB. I have tried with both external power, and power supplied from the programmer Diecimilla, no change - meaning that the second step in the process above will fail with:

         Using Port            : /dev/ttyUSB0
         Using Programmer      : stk500v1
         Overriding Baud Rate  : 19200
avrdude: Send: 0 [30]   [20] 
avrdude: Send: 0 [30]   [20] 
avrdude: Send: 0 [30]   [20] 
avrdude: ser_recv(): programmer is not responding
avrdude: stk500_recv(): programmer is not responding
avrdude: Send: Q [51]   [20] 
avrdude: ser_recv(): programmer is not responding
avrdude: stk500_recv(): programmer is not responding

(Note that I edited ~/.arduino/preferences.txt, and added build.verbose=true and upload.verbose=true to get this output from the IDE, and get the right command lines used)

I first used the alligator-clips-wire as in the above image; then after a few failures, I thought I'd measure pins P10-P13 on a scope. I could then see that P11 and P12 (MISO/MOSI) are "heavily" floating whenever I attach the alligator clips (it's as if those wires with clips, acted as antennas for 50Hz mains). So, I ditched these wires, and simply cut smaller pieces of regular wire, and connected through the sockets as in the tutorial. Now there is significantly less of this floating noise, but that didn't result with success of the burning process.

Then I looked around, and found about this serial reset issue.. Then I read a bit through the ArduinoISP sketch source; and I realised there is a "heartbeat" (a PWM signal on P9) for the programmer - so I observed that on scope too. The behavior is:

  • After finish of programming via USB - or after press on RESET button - P9 stays low for about a second, and then 'heartbeat' starts.
  • If heartbeat is started - any click on any of the Tools menu options (including Tools menu itself) will freeze the 'heartbeat' (note, this doesn't look like reset, since with a real reset, the 'heartbeat' should start again after a second - which it does not do in this case).
  • Also, most serial commands from terminal will freeze the 'heartbeat' (except 'stty -a -F /dev/ttyUSB0' which will reset)
  • If a 'heartbeat' has frozen, you need to press the RESET button again - in order to get it running anew

With this in mind - I tried experimenting a bit with trying to 'burn bootloader' and closely press the RESET button on the programmer Diecimilla; and there are, basically, these outcomes:

  1. The RESET button was not released on time, and the 'heartbeat' is frozen; then the failure is:
avrdude: ser_recv(): programmer is not responding
avrdude: stk500_recv(): programmer is not responding
  1. The RESET button has been released on time; however, the 'heartbeat' has not kicked yet in - then the failure is:
avrdude: Device signature = 0x1e9406
avrdude: Expected signature for ATMEGA328P is 1E 95 0F
         Double check chip, or use -F to override this check.

Finally, if I decide to use this '-F' option -- by running manually the command line that Arduino IDE generates:

/path/to/avrdude -C/path/to/avrdude.conf -v -v -v -v -patmega328p -cstk500v1 -P/dev/ttyUSB0 -b19200 -e -Ulock:w:0x3F:m -Uefuse:w:0x05:m -Uhfuse:w:0xDA:m -Ulfuse:w:0xFF:m

-- and if I also release the RESET button (on the programmer Diecimilla) at the right time, then avrdude will try to program, and finally fail during verification:

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0000
         0x3f != 0x00
avrdude: verification error; content mismatch

As I understand the ISP protocol, the first action that should be made, is that RESET (P10 on programmer) should be pulled low; I can only observe that on scope, when RESET (button) is released on time.

Now, I'd say that this behavior is nicely summed up in this comment:

This could be because the auto reset circuit is being triggered and making your seeeduino think the sketch is for it. You could try disabling it.

In respect to disabling the auto-reset, I also found 'Arduino Forum - disable auto-reset by serial connection':

The solution lies in proper setup of the serial connection, and requires no extra hardware or hardware modification. You must have to disable HUPCL on linux and DTR on Windows

Unfortunately, checking my serial port settings gives '-hupcl', which I guess means hupcl is disabled (that setting is also enforced by running the Arduino IDE programming via USB, too):

$ stty -a -F /dev/ttyUSB0 
speed 19200 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^A; eol = <undef>;
eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd cs8 -hupcl -cstopb cread clocal -crtscts
ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr -icrnl -ixon -ixoff
-iuclc -ixany -imaxbel -iutf8
-opost -olcuc -ocrnl -onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
-isig -icanon -iexten -echo -echoe -echok -echonl -noflsh -xcase -tostop -echoprt
-echoctl -echoke

That pretty much exhausts my approaches for now; apparently the serial reset is the problem - but I cannot circumvent it by pressing the RESET button on time... But still - how could I have brought back a bricked Diecimilla earlier, and not have any such problems then; with more or less the same software/hardware combo? Did the Linux kernel maybe make a significant change in how serial calls are executed??

Well, as I'd like to know what's going on - I'd love to hear some comments or suggestions for further troubleshooting (as long as they don't involve desoldering type hardware hacks :slight_smile: ) ,
Thanks,
Cheers!

.. And, Arduino forums to the rescue :slight_smile:

As a hardware hack, there is the 110 ohm addition - unfortunately, the resistor has to have its value chosen pretty closely, and I currently do not have access to any resistors with such values.

Then I saw this:

Arduino Forum - disable auto-reset by serial connection #14:

The easiest way is to stick a larger value capacitor between gnd and the reset pin. I use a 10uF - works a treat. It has the effect of 'swallowing' the small ground pulse transferred by the 0.1uF auto reset capacitor and preventing the board from resetting. It avoids the current drain of holding it high with a resistor. too.

Arduino Forum - please add a jumper to disable auto reset #56

Just my 2 cents:
Place a capacitor of 1 or 2 microfarad between reset and ground.
You don't have to try different resistors.
The reset from the ftdi chip is disabled and the reset button works normally without the "huge" current of 40 mA.
Remove the cap and everything is normal.

And I could only found a 100 nF cap - not even electrolytic; just a plain plastic capacitor... Stuck one leg in the RESET pin of the Diecimilla programmer; and the other via those alligator clips to ground - and kept on observing the heartbeat on oscilloscope. Finally, when clicking the Tools menu in the IDE - the heartbeat was NOT stopping anymore; and finally - "burn bootloader" succeeded :slight_smile:

So, while this solved my problem - it still haven't solved the mystery of: how come I did not go through all this, the first time I tried to "reset/restore factory settings" (that is, to 'burn the bootloader')? :-? (... though, it isn't that much of a pressing issue for me now :wink: )

Cheers!

The capacitor trick with the 10uF was mine, it looks like it doesn't take much to stop it resetting if a 100nF works :slight_smile: Like you a 10uF was what I had to hand, I have some 100nF as well but I didn't think that would be big enough.

The capacitor trick with the 10uF was mine, it looks like it doesn't take much to stop it resetting if a 100nF works Smiley Like you a 10uF was what I had to hand, I have some 100nF as well but I didn't think that would be big enough.

Cheers, pluggy - thanks for sharing! [smiley=thumbup.gif] I'm so glad it could have been done with less - I couldn't, for the life of me, find any other cap than that one in the picture; so glad using this one did the trick :slight_smile:

Sorry for the bump, but it seems the mystery is coming to a close; as one of my suspicions - the Linux kernel being changed in a recent update (btw, my current uname -r says 2.6.32-25-generic) - seems to be confirmed by a recent post (Aug 2010), "ftdi_sio serial adapter data toggle problem.":

The problem is that the ftdi-sio driver's data toggle and the ftdi-sio
adapter's data toggle are now at risk of starting out (opening of ttyUSB0) out
of sync. I'm sure this is the problem because it's seemingly random, and, when
it occurs, it corrects after a few data transfer attempts. As you'll see below,
this is a new problem which the ftdi-sio driver didn't used to have. He's
using a Debian system, so my references to kernel releases are what that
distribution calls them.
His ftdi-sio adapter worked fine up until, and including, kernel 2.6.32-3-686.
Then, when he upgraded to kernel 2.6.32-5-686-18, the problem started to occur.
It still occurs on Debian Unstable's kernel 2.6.32-5-686-19, and it even still
occurs on Debian Experimental's kernel 2.6.35-rc6-686, so I suspect this is a
problem which hasn't been addressed yet.

....

Ah, the infamous dtr/rts issue.

....

Ok, we need to get this settled once for good. I copied some other
people who reported similar trouble. Let me try to summarize again what
I believe is the problem.
The USB protocol of this chip has two functions to modify the hardware
handshake lines. One is FTDI-SIO-SET-FLOW-CTRL-REQUEST and the other
is FTDI-SIO-SET-MODEM-CTRL-REQUEST. To fully enable or disable DTR/RTS
lines, appearantly both commands need to be sent to the chip.
... [snip - good discussion, read at source] ...
Unfortunately, I see no good way to fix this and still keep all existing
users happy as they're simply dealing with with wrong settings and the
driver can't guess which mode the user actually needs. So people should
really fix their clients.

Ok, so at least I know something changed now, and it is not really trivial to solve it on a driver level... hence all other software using the driver may/will exhibit problems (in respect to previous use cases). Not really a solution, but I least I know now what's going on :wink:

Cheers!

PS: and just to emphasize the key quote, relevant to this:

Then Alan's patch 4175f3e31 ("tty-port: If we are opened non blocking we
still need to raise the carrier") came in in 2.6.32-rc8 which changed
the behaviour for all serial drivers by directly raising the DTR/RTS
lines upon each device open.

PSS: Now, I even bump into problems programming the Duemillanove as usual from the Arduino IDE! I never used to press "RESET" button to have it programmed; now, if I don't press it, I could get "programmer not responding" - so I had to resort to the good old: hold RESET button -> click Upload -> release RESET button quickly, to have stuff uploaded.. :o Good the button is still there, though :slight_smile: - And yes, just to be more precise - the button seems to be needed for uploading only when you have the capacitor on...

So thats why my trusty software fudge with DTR doesn't work when I changed my host's OS to Debian Squeeze....

Implementing the hardware fudge has really improved it though - it always used to reset the first time it read the arduino after I reconnected it.