After using Arduino as ISP, only able to program with FT232R once

EmptySet · February 12, 2019, 9:37pm

I have an ATmega2560 which I am using MegaCore with. GitHub - MCUdude/MegaCore: Arduino hardware package for ATmega64, ATmega128, ATmega165, ATmega169, ATmega325, ATmega329, ATmega640, ATmega645, ATmega649, ATmega1280, ATmega1281, ATmega2560, ATmega2561, ATmega3250, ATmega3290, ATmega6450, ATmega6490, AT90CAN32, AT90CAN64 and AT90CAN128

To interface with this, I have an Arduino as ISP hooked up to its SPI pins. This works well, but I don't want to always be laden with the extra breadboard coming off my circuit just for reprogramming. I have an FT232R chip hooked up to my AVR's Tx/Rx as well to hopefully alleviate this. The desired end goal here is to have to use the Arduino as ISP 0% of the time, and the FT232R 100% of the time.

As hinted by the subject line, I can't do this yet. The FT232R can successfully upload to the AVR immediately after the bootloader has been programmed to it, but it cannot upload in any other case. The upload procedure likely destroys the bootloader it needs.

So, I need to know how to do one of these to remove reliance on the Arduino as ISP:

Re-burn bootloader with the FT232R interface (which doesn't work for me now)
Preserve bootloader when reprogramming with my FT232R.
Upload with the FT232R interface without needing a bootloader present

Circuit details: I have a 0.1uF capacitor connected between my FT232R and !Reset of my AVR, as I saw recommended somewhere. The rest of the circuit (Rx/Tx connections) should be correct as, again, I am able to reprogram when a bootloader is present.

Here's the current "state machine" I observe when trying various things in the IDE.

Tools Menu Settings
Board:"ATmega2560"
Clock:"8 MHz internal"
BOD:"2.7v"
Compiler LTO:"Disabled (default)"
Pinout:"AVR pinout"

Entry State: Any
USB Cable:Plugged into ISP Arduino
Programmer:"Arduino as ISP"
Command:"Burn Bootloader"
Result: Success

Entry State: Any
USB Cable:Plugged Into FT232R
Programmer:"AVRISP mkII"
Command:"Burn Bootloader"
Result: Error, message below

C:\Program Files (x86)\Arduino\hardware\tools\avr/bin/avrdude -CC:\Users\REDACTED\AppData\Local\Arduino15\packages\MegaCore\hardware\avr\2.0.1/avrdude.conf -v -patmega2560 -cstk500v2 -Pusb -e -Ulock:w:0x3f:m -Uefuse:w:0xfd:m -Uhfuse:w:0xd6:m -Ulfuse:w:0xe2:m 

avrdude: Version 6.3-20171130
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "C:\Users\REDACTED\AppData\Local\Arduino15\packages\MegaCore\hardware\avr\2.0.1/avrdude.conf"

         Using Port                    : usb
         Using Programmer              : stk500v2
Error while burning bootloader.
avrdude: usbdev_open(): did not find any USB device "usb" (0x03eb:0x2104)

avrdude done.  Thank you.

Entry State: Any
USB Cable:Plugged Into FT232R
Programmer:"AVR ISP"
Command:"Burn Bootloader"
Result: Error, message below

C:\Program Files (x86)\Arduino\hardware\tools\avr/bin/avrdude -CC:\Users\REDACTED\AppData\Local\Arduino15\packages\MegaCore\hardware\avr\2.0.1/avrdude.conf -v -patmega2560 -cstk500v1 -PCOM11 -e -Ulock:w:0x3f:m -Uefuse:w:0xfd:m -Uhfuse:w:0xd6:m -Ulfuse:w:0xe2:m 

avrdude: Version 6.3-20171130
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "C:\Users\REDACTED\AppData\Local\Arduino15\packages\MegaCore\hardware\avr\2.0.1/avrdude.conf"

         Using Port                    : COM11
         Using Programmer              : stk500v1
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 1 of 10: not in sync: resp=0x03
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 2 of 10: not in sync: resp=0x03
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 3 of 10: not in sync: resp=0x03
avrdude: ser_recv(): read error: The I/O operation has been aborted because of either a thread exit or an application request.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 4 of 10: not in sync: resp=0x03
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 5 of 10: not in sync: resp=0x03
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 6 of 10: not in sync: resp=0x03
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 7 of 10: not in sync: resp=0x03
Error while burning bootloader.
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 8 of 10: not in sync: resp=0x03
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 9 of 10: not in sync: resp=0x03
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 10 of 10: not in sync: resp=0x03
avrdude: ser_drain(): read error: Access is denied.



avrdude done.  Thank you.

Entry State: Bootloader successfully burned via Arduino as ISP
USB Cable:Plugged Into FT232R
Programmer:"AVRISP mkII"
Command:"Upload" (normal upload)
Result: Success

Entry State: Program successfully uploaded via FT232R
USB Cable:Plugged Into FT232R
Programmer:"AVRISP mkII" or "AVR ISP"
Command:"Upload" (normal upload)
Result: Error, message below

avrdude: Version 6.3-20171130
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "C:\Users\REDACTED\AppData\Local\Arduino15\packages\MegaCore\hardware\avr\2.0.1/avrdude.conf"

         Using Port                    : COM11
         Using Programmer              : arduino
         Overriding Baud Rate          : 38400
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 1 of 10: not in sync: resp=0xcd
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 2 of 10: not in sync: resp=0xcd
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 3 of 10: not in sync: resp=0xcd
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 4 of 10: not in sync: resp=0xcd
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 5 of 10: not in sync: resp=0xcd
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 6 of 10: not in sync: resp=0xcd
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 7 of 10: not in sync: resp=0xcd
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 8 of 10: not in sync: resp=0xcd
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 9 of 10: not in sync: resp=0xcd
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 10 of 10: not in sync: resp=0xcd

avrdude done.  Thank you.

Problem uploading to board.  See http://www.arduino.cc/en/Guide/Troubleshooting#upload for suggestions.

pert · February 13, 2019, 12:19am

EmptySet:
So, I need to know how to do one of these to remove reliance on the Arduino as ISP:

Re-burn bootloader with the FT232R interface (which doesn't work for me now)

You can't do that (OK technically there is some complex and slow way to do it, but put it out of your mind).

EmptySet:
2. Preserve bootloader when reprogramming with my FT232R.

I don't think this is a matter of the bootloader being lost. When you do the Burn Bootloader process, the rest of the flash memory is erased. Since there is no sketch to run, the bootloader remains permanently activated until you upload the first sketch. After uploading that first sketch, the ATmega2560 must be reset at just the right time during the upload to activate the bootloader. By connecting the DTR pin of the FT232 to the reset pin of the ATmega2560 via a 0.1 uF capacitor, the upload process is able to automatically reset the ATmega2560 at just the right time during the upload. When that auto-reset circuit is not working, it causes exactly the symptoms you are experiencing.

I have a recommendation for a quick test you can try to verify that the bootloader is indeed still intact even when you can't upload. This requires that you have some way to manually reset the ATmega2560. If your circuit doesn't have a reset button, you can simply momentarily connect the reset pin of the ATmega2560 to ground using a wire to do the reset. Here are the instructions:

Burn the bootloader
Disconnect the programmer and connect the FT232.
Sketch > Upload. This will use up your first upload after Burn Bootloader that always works.
File > Preferences > Show verbose output during: upload (check)
Click "OK"
Sketch > Upload
Watch the contents of the black console window at the bottom of the Arduino IDE window. As soon as you see something like "Global variables use 9 bytes (0%) of dynamic memory, leaving 8183 bytes for local variables. Maximum is 8192 bytes.", reset the ATmega2560.

If the above instructions cause the second upload after Burn Bootloader to work, then you know that the bootloader is still intact and that the problem is with your auto-reset circuit.

EmptySet · February 13, 2019, 4:33pm

Thanks for the reply.

I tried the procedure you mentioned, this was my output.

avrdude: Version 6.3-20171130
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "C:\Users\REDACTED\AppData\Local\Arduino15\packages\MegaCore\hardware\avr\2.0.1/avrdude.conf"

         Using Port                    : COM17
         Using Programmer              : arduino
         Overriding Baud Rate          : 38400
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 1 of 10: not in sync: resp=0x85
avrdude: ser_recv(): read error: The I/O operation has been aborted because of either a thread exit or an application request.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 2 of 10: not in sync: resp=0x85
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 3 of 10: not in sync: resp=0x85
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 4 of 10: not in sync: resp=0x85
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 5 of 10: not in sync: resp=0x85
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 6 of 10: not in sync: resp=0x85
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 7 of 10: not in sync: resp=0x85
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 8 of 10: not in sync: resp=0x85
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
the selected serial port avrdude: stk500_recv(): programmer is not responding
 does not exist or your board is not connected
avrdude: stk500_getsync() attempt 9 of 10: not in sync: resp=0x85
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 10 of 10: not in sync: resp=0x85
avrdude: ser_drain(): read error: Access is denied.



avrdude done.  Thank you.

As soon as the reset pin of my AVR is pulled low, the avrdude operation aborts and all its remaining 'attempts' fail quickly. Based on the error messages printed, it looks to me like the serial port is closed when the reset pin is connected. I'm not sure why, exactly - things might work out if the reset was allowed to happen without this disturbance.

And just for clarification, if I don't connect anything to the reset pin of my AVR, it will fail like I described in my first post (just a slow count to 10 timeouts).

You mentioning that I had to connect a 0.1uF capacitor from !Reset to DTR (pin 2) made me realize that I had accidentally connected it to DTS (pin 9) previously. I don't know how much was impacted since it's not a pin used by the controller in the Arduino or my pinout, but to be safe I did my testing above with the capacitor entirely removed. Afterwards, however, I tried connecting the capacitor to DTR and testing again.

Unfortunately, with the capacitor from DTR to !Reset, the FT232R port seems to never be recognized by my computer ("Port" is grayed out). I've "tricked it" by connecting the cap only after the com port is set and then clicking upload, but then it behaves similarly to the above. Specifically, in this case the serial port seems to be closed immediately and the upload attempt fails.

Putting a scope on the !Reset of my AVR, I noticed something sporadically pulling it low through DTR for very brief intervals. The frequency was maybe once every 200ms or so. This seemed to go on even after my upload procedure failed.

I also verified that my usage of the FT232R in my circuit was identical to the Arduino's. I used this image to compare. https://media.digikey.com/Photos/RDL/Arduino%20Nano%20-%20Schematic.png

I think I can rule out any major wiring issues since the FT232R can reprogram the AVR properly when the bootloader has been newly burnt. I'm wondering that perhaps I've damaged the FT232R at some point though, and I think my next step is replacing that part. At the least, I can't think of another explanation for the serial port getting screwed up with a cap on DTR, or maybe more specifically, AVR !Reset being set to ground momentarily.

Do you have any other thoughts pertaining to the issues I'm seeing here?

pert · February 14, 2019, 12:47am

EmptySet:
I had accidentally connected it to DTS (pin 9) previously.

The datasheet shows that on the SSOP package, physical pin 9 is DSR. Is that what you meant? You can actually use either DTR or CTS for the auto-reset circuit, but I don't know if you can use DSR.

EmptySet:
Unfortunately, with the capacitor from DTR to !Reset, the FT232R port seems to never be recognized by my computer ("Port" is grayed out).

That definitely is not right. Other than the DTR pin, is the ATmega2560 pin only connected to VCC via a 10K ohm pull up resistor, as shown in the MegaCore schematic?

EmptySet:
Putting a scope on the !Reset of my AVR, I noticed something sporadically pulling it low through DTR for very brief intervals. The frequency was maybe once every 200ms or so. This seemed to go on even after my upload procedure failed.

You should get one low pulse whenever a serial connection is opened to the FT232. That's how the auto-reset works. Is this happening when the FT232 is not creating a port on your computer? It seems to me like the FT232 is repeatedly powering up and creating a pulse, then shutting down due to some bad operating condition. You can try restarting your computer to see whether this is just caused by some zombie AVRDUDE process that is stuck running in a loop in the background.

EmptySet · February 14, 2019, 8:20pm

pert:
The datasheet shows that on the SSOP package, physical pin 9 is DSR. Is that what you meant?

Yes.

pert:
Other than the DTR pin, is the ATmega2560 pin only connected to VCC via a 10K ohm pull up resistor, as shown in the MegaCore schematic?

Actually, in my previous testing, I didn't have !Reset connected to anything other than to DTR through the 0.1u cap. I checked its level in this state and it was 5V. Checking the AVR datasheet, looks like it's an internal pull-up to VCC. I decided to try connecting it to VCC through a 10k anyway, and as expected it didn't seem to change any results.
I swapped out my FT232 for another one and I see similar behavior here - but more explanation of that below.

pert:
Is this happening when the FT232 is not creating a port on your computer? ... You can try restarting your computer to see whether this is just caused by some zombie AVRDUDE process that is stuck running in a loop in the background.

I tried to characterize when this happens a little more.

Some other things worth noting before I get started: I checked my schematic looking out from the FT232R and it looks identical to the pinout I linked previously. Well, the only difference I noted is that I don't have 3v3 connected since I'm not using it - shouldn't matter, I imagine. And again, I'm not too worried that there's a basic wiring issue since I checked again and confirmed with my most recent circuitry that it is capable to upload when there is only a fresh bootloader present (after burning with AVR ISP). Also, I did try restarting several times, no changes.

Here are some findings with several different circuit options:

No cap connected:
With the new FT232, no difference in behavior from my previous posts. I will note that I tried this with and without pulling !Reset to ground and tried various timings around the invocation of avrdude to trigger it at the right moment. Nothing seemed to do the trick. This makes me wonder - is it perhaps possible that uploading through the FT232R once can erase the bootloader? I thought this wasn't the case but perhaps if I have some settings in the Arduino IDE incorrect this can happen?

Cap on DTR:
When I have DTR connected via 0.1uF cap to !Reset with the newer FT232 and I power my circuit and connect the USB, scoping out !Reset doesn't show any activity (just a steady 5V). At this time, my device is actually seen as a serial port, and everything seems fine.

Only when avrdude is invoked in my upload procedure do I see this intermittent blip pulling !Reset to ground start. Then even after avrdude fails, the blips continue (seems to trigger pretty consistently every 500ms). My "Port" also becomes grayed out at this point. The blips and undetectable port will continue indefinitely until that DTR-!Reset connection is broken. After this, even if the connection is restored, the port is detected again and the blips will stop.

It's the same as I've seen before, but to be clear, here's the error message when I attempt to upload (using AVRISP mkII) with the cap on DTR:

avrdude: Version 6.3-20171130
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "C:\Users\REDACTED\AppData\Local\Arduino15\packages\MegaCore\hardware\avr\2.0.1/avrdude.conf"

         Using Port                    : COM18
         Using Programmer              : arduino
         Overriding Baud Rate          : 38400
avrdude: ser_drain(): read error: The I/O operation has been aborted because of either a thread exit or an application request.


avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_drain(): read error: Access is denied.


avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_drain(): read error: Access is denied.


avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 1 of 10: not in sync: resp=0xa3
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_send(): write error: sorry no info avail

avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 2 of 10: not in sync: resp=0xa3
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 3 of 10: not in sync: resp=0xa3
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 4 of 10: not in sync: resp=0xa3
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 5 of 10: not in sync: resp=0xa3
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 6 of 10: not in sync: resp=0xa3
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 7 of 10: not in sync: resp=0xa3
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 8 of 10: not in sync: resp=0xa3
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 9 of 10: not in sync: resp=0xa3
avrdude: ser_send(): write error: sorry no info avail
avrdude: ser_recv(): read error: Access is denied.


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 10 of 10: not in sync: resp=0xa3
avrdude: ser_drain(): read error: Access is denied.



avrdude done.  Thank you.

Cap on CTS:
The behavior is a bit different when I have CTS hooked up, as you suggested. In this case, it's pretty easy - The !Reset pin stays at a steady 5V at all times, and the port is always available. But of course this also means it's still failing... error message below:

avrdude: Version 6.3-20171130
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "C:\Users\REDACTED\AppData\Local\Arduino15\packages\MegaCore\hardware\avr\2.0.1/avrdude.conf"

         Using Port                    : COM18
         Using Programmer              : arduino
         Overriding Baud Rate          : 38400
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 1 of 10: not in sync: resp=0xb1
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 2 of 10: not in sync: resp=0xb1
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 3 of 10: not in sync: resp=0xb1
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 4 of 10: not in sync: resp=0xb1
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 5 of 10: not in sync: resp=0xb1
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 6 of 10: not in sync: resp=0xb1
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 7 of 10: not in sync: resp=0xb1
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 8 of 10: not in sync: resp=0xb1
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 9 of 10: not in sync: resp=0xb1
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 10 of 10: not in sync: resp=0xb1

avrdude done.  Thank you.

Problem uploading to board.  See http://www.arduino.cc/en/Guide/Troubleshooting#upload for suggestions.

So in the end, I'm still fairly stuck here. I appreciate the help though. If you have any suggestions for further troubleshooting let me know.

pert · February 15, 2019, 1:11am

EmptySet:
I didn't have !Reset connected to anything other than to DTR through the 0.1u cap. I checked its level in this state and it was 5V. Checking the AVR datasheet, looks like it's an internal pull-up to VCC. I decided to try connecting it to VCC through a 10k anyway, and as expected it didn't seem to change any results.

Correct. You can get by without an external pull-up on the reset pin. The internal pull-up is not very strong so it's possible that strong EMI could cause spurious resets. That would result in a lot of confusion so it's generally considered best practices to add the external pull-up.

EmptySet:
is it perhaps possible that uploading through the FT232R once can erase the bootloader? I thought this wasn't the case but perhaps if I have some settings in the Arduino IDE incorrect this can happen?

I think there is some way it can happen, but I'm not sure exactly what it is. I think the mechanism is that some sketch code would end up being written to the boot section. Even if this didn't overwrite the bootloader, it could still make the bootloader not work. After your board is in the "can't upload" state, what happens when you reset it? If the bootloader is running, you should get a couple of fast blinks on pin 13 after reset.

I connected an Arduino as ISP to an Arduino Mega and burned the bootloader with the same MegaCore configuration you're using and was able to upload multiple times after that. Which version of MegaCore are you using? If you installed it via Boards Manager, you can find the version number like this:

Tools > Board > Boards Manager
Wait for downloads to finish.
Scroll down until you see "MegaCore by MCUdude". There, it will say something like "version 2.0.1 installed".

EmptySet · February 15, 2019, 9:27pm

pert:
Which version of MegaCore are you using?

2.0.1.

pert:
If the bootloader is running, you should get a couple of fast blinks on pin 13 after reset.

Interesting. I'm not quite following "pin 13" - is that considering that I am using the ATmega2560 with the AVR pinout? In that case, pin 13 is PH5. If you're assuming I use the Arduino Mega pinout, it looks like it's PB7 instead.

I decided to scope out both of these pins. I tried PH5 first - didn't see anything on it, it stayed at 0V. When I tried PB7, and when I had only bootloader on the chip, I did see two 5V pulses in a brief period every second or so - likely bootloader starting over and over as the program code loops.

So that's interesting - it seems like the bootloader doesn't change based on which Megacore pinout scheme you use. I guess it doesn't matter in the end, though.

Anyway, after programming the chip with the FT232R once, the bootloader 'blips' no longer showed up, not even once when I cycled power on my board. This does seem like pretty strong evidence that the bootloader actually is getting erased by my first upload with the FT232R.

The question now is - why?

Here's the specific bootloader for reference. You can also see my settings in the command.

C:\Program Files (x86)\Arduino\hardware\tools\avr/bin/avrdude -CC:\Users\REDACTED\AppData\Local\Arduino15\packages\MegaCore\hardware\avr\2.0.1/avrdude.conf -v -patmega2560 -carduino -PCOM4 -b19200 -Uflash:w:C:\Users\REDACTED\AppData\Local\Arduino15\packages\MegaCore\hardware\avr\2.0.1/bootloaders/optiboot_flash/atmega2560/optiboot_flash_atmega2560_UART0_38400_8000000L.hex:i -Ulock:w:0x0f:m

pert · February 15, 2019, 11:45pm

EmptySet:
2.0.1.

OK, I just confirmed that I can burn bootloader with an Arduino as ISP using the same MegaCore version and settings on my Arduino Mega 2560 board and afterwards upload multiple times with no problem.

EmptySet:
Interesting. I'm not quite following "pin 13" - is that considering that I am using the ATmega2560 with the AVR pinout? In that case, pin 13 is PH5. If you're assuming I use the Arduino Mega pinout, it looks like it's PB7 instead.

Sorry about that. The bootloader blinks PB7 no matter what pinout, so I was wrong to say pin 13 since you're using the AVR pinout. I'm just used to the Arduino pinout.

EmptySet:
Anyway, after programming the chip with the FT232R once, the bootloader 'blips' no longer showed up, not even once when I cycled power on my board. This does seem like pretty strong evidence that the bootloader actually is getting erased by my first upload with the FT232R.

I agree.

What are you uploading to the board? Is it a large sketch, or just a bare minimum or blink sort of thing to test?

EmptySet · February 18, 2019, 2:39pm

pert:
What are you uploading to the board? Is it a large sketch, or just a bare minimum or blink sort of thing to test?

I was trying with a small sketch that didn't do much of anything, now I'm trying with an unmodified "Blink".

So, the plot thickens here after scoping out PB7 a bit more.

After flashing with the ISP, the device is in constant bootloader. PB7 shows the "bootloader pattern" about once every two seconds.
After uploading once with the FT232, the Blink application is present and PB7 cycles at 0.5Hz (since it's the builtin LED pin, used by both BL and blink). Note that this is my one 'good' upload, and I perform this upload without anything on DTR/CTS. Also, when power cycling in this state, I do not see the bootloader pattern at all. Blink is the first thing to change PB7.

All this led me to believe bootloader was not present. But when trying a few things to get the second upload working, I did see bootloader activity again.

When attempting the second upload with the FT232 with identical wiring to the first upload, the application 'ignores' the chip and never stops running Blink. Arduino fails eventually after several attempts yielding "programmer is not responding". This makes enough sense.
When attempting the second upload with the FT232 and a 0.1uF cap between DTR and the AVR's !Reset, it still fails of course, my COM port becomes grayed out until I power cycle, and I see the !Reset line getting pulled low every 400-500ms or so. This is all old information, but I noticed something new when scoping PB7. In time with the constant resets, I see the bootloader pulses coming from my AVR again (so effectively happening every 400-500ms).

This leaves me with several mysteries...

Why don't I see bootloader pulses when powering up my AVR?
What's different about this constant reset pattern to cause bootloader to be entered by my AVR? Edit: Seems like the distinction here may be in a 'cold start' vs. power-on reset. I can see the bootloader pattern when manually tapping ground to !Reset as well.
And as always, where the heck are these constant resets coming from?

I'm assuming the resets are happening from the FT232 side. I'll check the datasheet/internet and try to figure out what's happening, but it's easier said than done!

kprims · February 18, 2019, 3:28pm

I'm assuming the resets are happening from the FT232 side. I'll check the datasheet/internet and try to figure out what's happening, but it's easier said than done!

Check for a short between pins1&2 on your FTDI232R chip.

EmptySet · February 18, 2019, 9:20pm

Found the issue!

The reset pin of the FT232 was connected to the reset pin of the AVR. I didn't notice this at first since the 'RST' label I had in my schematic was supposed to only go to a breakout pin.

This error meant the FT232 was resetting itself and the AVR through DTR. This is probably also why the reset blips I saw were very short, since the FT232 stopped driving the pin when it reset. Once it reset, whatever avrdude was doing to set DTR in the first place kept happening over and over, hence the frequent resets. I guess that kills the serial port too, so good to know.

With this fixed, everything works as expected. The last mystery is that I don't see any bootloader pulses when starting up 'cold'; I still don't fully understand why not.

Anyway, thanks for your guidance.

pert · February 18, 2019, 11:53pm

Hi. I accidentally gave you some incorrect information earlier:

pert:
You can actually use either DTR or CTS for the auto-reset circuit

That should have said:
"You can actually use either DTR or RTS for the auto-reset circuit".
I don't know about CTS. I apologize for any confusion this might have caused.

I'm very glad to hear you found the solution. Enjoy!
Per

DrAzzy · February 19, 2019, 12:28am

CTS (as well as the 3 other modem control lines other than DTR and RTS) are inputs, active low. DTR and RTS are outputs (also active low)

With your finer serial consoles, the state of those pins will be shown visually - I've connected them to various parts of my project to monitor the state of pins to aid in debugging.

system · May 6, 2021, 6:08pm