Ariadne Bootloader on Leonardo Ethernet board

Hello

I am working with a Leonardo Ethernet board.

Want i want to do is to change the bootloader in order to be able to flash program over Ethernet, from my Desktop computer, instead of plugin USB cable.

I have downloaded, compile and install this bootloader with arduino ISP:

Here are some problems:

When this bootloader is installed, i cannot upload a program from Arduino IDE, via classic USB port. The documentation says it is possible to do it when the board is reseted, during a few seconds when the led13 is blinking quickly. I can see this led blinking, but the serial port is not available in Arduino IDE Ports submenu...

When the led13 is blinking at startup, i am able to transfert a new program via tftp, but i cannot do this when the program is running. I suppose the bootloader is not alive when the program is running, but is there a way to call by myself a part of the bootloader from the program ? What i want to do is to send an http request to my program with new program hex file, and i want the program to flash itself.

With this bootloader, i can run programs written in pure C. I mean programs written in low-level C and compiled with avr-gcc by hand. But i cannot run a program written with arduino IDE. The program is flashed, but it does not run
Here the operation i have done:
In arduino IDE, i am choosing "Arduino Leonardo ETH" in Tools > Board submenu. Then, i click on Verify/Compile menu.
I am looking for informations in black output window at the bottom of the screen (i have checked verbose output in preferences).
I can see: /var/folders/XX/YYYYYYYYYYYYYYYYYYYYYYYY/T/arduino_build_294885/myprogram.hex

avr-objcopy -I ihex /var/folders/XX/YYYYYYYYYYYYYYYYYYYYYYYY/T/arduino_build_294885/myprogram.hex -O binary hex.bin

Then, i put hex.bin file via tftp. There are no error messages.
When i reset the board, i can see led13 blinking quickly during a few second, but my program is not running then...

Thanks for your help

That’s frustrating to me that they copied the Ariadne bootloader repository rather than forking it. It’s free open source code and that is permitted of course but there’s absolutely no good reason and it’s really harmful because I never knew about the work they’ve done on Ariadne since it wasn’t linked to the original repository. The work to add ATmega32U4 support is very important but they made no effort to share it with the original author. Also, they did a half-assed job of it because they never bothered to add an entry in boards.txt for the Leonardo ETH or update the documentation, which would have only taken a couple extra minutes but made the work much more accessible to the user.

testpresta2:
When this bootloader is installed, i cannot upload a program from Arduino IDE, via classic USB port. The documentation says it is possible to do it when the board is reseted, during a few seconds when the led13 is blinking quickly. I can see this led blinking, but the serial port is not available in Arduino IDE Ports submenu…

The Leonardo ETH is quite different from other boards supported by Ariadne, yet they made very minimal changes to the bootloader source when they added Leonardo ETH support. It’s quite likely that the loss of USB upload capability is normal when using the bootloader on that board but the author just didn’t make the minimal effort to document this in any way.

testpresta2:
When the led13 is blinking at startup, i am able to transfert a new program via tftp, but i cannot do this when the program is running. I suppose the bootloader is not alive when the program is running, but is there a way to call by myself a part of the bootloader from the program ? What i want to do is to send an http request to my program with new program hex file, and i want the program to flash itself.

The bootloader only runs for several seconds after reset. You can manually press reset before starting the upload but that defeats the purpose of uploading over Ethernet, right? So you need a way to remotely reset the Arduino to activate the bootloader before starting the TFTP upload. You can do this by using the watchdog, something like:

#include <avr/wdt.h>
if (reset command was received) {
  wdt_enable(WDTO_15MS);  //enable the watchdog timer with a 15 millisecond timeout duration
  while (true);  //perpetual loop until the watchdog times out and resets the microcontroller
}

So you will send the reset command over the network to your Arduino, then immediately start the TFTP upload. How you choose to implement the reset system is up to you. There is an example of one implementation included with Ariadne here:

testpresta2:
With this bootloader, i can run programs written in pure C. I mean programs written in low-level C and compiled with avr-gcc by hand. But i cannot run a program written with arduino IDE. The program is flashed, but it does not run

Hopefully someone else will comment on this because I’m not very familiar with the way Arduino works with ATmega32U4 boards. I know the Arduino IDE adds extra code to the compilation for an ATmega32U4 based board such as Leonardo ETH to handle the different way uploads work (open serial port at 1200 baud to trigger reset). This is why you’ll notice the size of sketches compiled for one of these boards is always much larger than compiled for e.g. an Uno. So it may be that extra code added in from the core is interfering with the program running on the Leonardo ETH which has the Ariadne bootloader installed.

Just for your reference, this is the most active fork of Ariadne:

It’s owned by the author of Ariadne. Development is now being done on that fork since codebender, owner of the original repository, is no longer involved with the project. LoathingKernel is pushing changes made there back to codebendercc/Ariadne-Bootloader so that repository will also probably stay up to date

Thanks for this very nice answer.

Do you know if it is very hard to add support of ATmega32U4 on "official" Ariadne bootloader.
I am new to arduino and i think it will be for me a very good exercice in order to understand low level arduino programming, but i think i will need some help :slight_smile:

Thanks again

You can see the changes that were made here:

It's surprisingly minimal. The original Ariadne code had some first steps towards adding ATmega32U4 support in place so it's clear it was a goal at that time. The official version of Ariadne has progressed a bit since VirtusLab's fork (W5200, W5500, and ATmega1284P support are the most major changes) but even so I think it would be easy enough to pull the VirtusLab Leonardo Ethernet commit in. In fact the W5500 support added since is a great help as this is required for Leonardo ETH support so that's one less thing from the VirtusLabs repo to merge. The question is whether that commit actually works. Your experience tells me it does indeed work on some level. My take would be that, even with the limitations you've discovered, it's still an improvement and doesn't impose any overhead so why not merge it in? However, I'm only a minor contributor to the project so the person whose opinion really matters is LoathingKernel. It would be worth at least opening an issue in his repository to see if he's receptive.

Ideally we would be able to get USB uploading working but that would likely end up requiring that the Ariadne code be merged into the Caterina bootloader usually used with the ATmega32U4 boards, rather than some code being added into the current Ariadne bootloader code to support the ATmega32U4. I really don't think losing USB upload capability is a deal breaker. Once I have Ariadne bootloader installed on a board I never use USB uploads anymore, in fact I had proposed that there be an option to disable USB uploads for all boards to get the bootloader to fit in the next smallest boot section, freeing up program memory. I suspect that since you now have a modified version of optiboot, a UART bootloader, installed on your Leonardo Ethernet you could actually do a standard upload over serial if you connected a USB-serial adapter such as FT232RL to the UART pins. It's only the upload using the native USB capability of the chip that's not possible with the optiboot-based bootloader.

The issue of code compiled in the Arduino IDE is a much bigger issue for me. I did some further research on this after my last reply but this USB stuff is a bit beyond me. I still think it's something about how the Arduino core expects to interact with a now non-existent Caterina bootloader that's preventing the code from running but I could definitely be wrong. The program you compiled outside of the Arduino IDE doesn't have that code and thus runs fine. The goal would be to selectively disable that code without losing all the special USB functionality of the ATmega32U4. The secondary goal would be to do so while still referencing the Arduino AVR Boards core, rather than being forced to maintain our own modified version of the core in the Ariadne repository, which would be a struggle to keep up to date as the Ariadne project is not under very active development. LoathingKernel is super smart so he may be able to tell us what the issue is and what needs to be done.

I have a fork of Ariadne that I created at the request of LoathingKernel to do some development while he was on an extended break from being involved with the project. I have some different opinions on how things should work (Boards Manager installation support for example) and so far I haven't convinced him that my changes should be merged into the main repository. Meanwhile there has been some excellent work done on the main repository (W5500/W5200 support most especially) and even some minor contributions by me that I haven't merged into my fork. I'm torn between going off in my own direction and letting my fork permanently diverge with a new name or giving up on some of my pet features that LoathingKernel doesn't agree with in favor of the main repository. So it is possible that if limited ATmega32U4 support ends up being rejected by LoathingKernel I will end up merging this feature into my fork.

Playing with bootloader code is lots of fun. It's a big step for the average Arduino user who has never ventured outside of the IDE but clearly you're well past that point so I say jump into that source code and see what you can do! I am a big fan of Ariadne and I'm definitely willing to help out with this as my skills allow but to be honest I don't understand half the code in there and get especially lost with all the ATmega32U4 USB stuff. When it comes to the Arduino IDE hardware definitions I'm much more experienced if it gets to the point where that's needed. Unlike many developers I actually like to write documentation and am reasonably good at it, mostly because I'm still enough of a beginner that I can write documentation to their level of understanding.

Thanks again for your help.

I am wondering something: Do you think it is possible to write a program with Arduino IDE (I mean a program with setup() and loop()) which will be able to flash itselfs ?

My idea is to keep the original bootloader and write a program by myself in user space (ouside bootloader) that will download a .hex and write it on the flash memory (erasing the program itself)

Thanks

I think you would need to modify the Caterina bootloader to be able to write to flash from the application. This ability is usually only possible from the bootloader but if you add some code to the bootloader it can be done. This was done for the optiboot bootloader by forum user majek and there is actually some discussion in that thread regarding what you're talking about trying to do:

I see a project called arduino other the air (OTA), but it seems to only work with ESP8266 Wifi chip.
What is the easiest way ? Do you think i should modify Caterina bootloader or modify OTA project in order to make it work on Ethernet ?
Thanks

I'd need to look at it to be sure, but I don't think the OTA project will provide anything of use for flashing ATmega32U4 over Ethernet.

Let me make sure I understand your requirements correctly, correct me if these statements are wrong in any way.

  • You want to be able to generate a .hex file from an Arduino sketch and upload it over Ethenet to a Leonardo Ethernet but when you upload this file using the VirtusLab bootloader the code doesn't run.
  • Being able to upload via the USB jack on your Arduino Ethernet is not important.

Yes it is that

And i have a second question: I tried to reboot with watchdog like you said.
It reboots the board, but the ariadne bootloader is skipped. Do you know how i can force a reset to bootloader without having to unplug power supply ? Thanks

testpresta2:
Yes it is that

Then I think the easiest approach is to hunt down the code in the Arduino core libraries that’s being added to the sketch that prevents it from running when uploaded via the Ariadne bootloader. The USB code in the core libraries is pretty confusing but you don’t need to completely understand it, you just need to disable the right part of it. If you do something that causes you to no longer be able to upload over USB with the standard bootloader installed then you’re probably on the right track. The trick is to avoid disabling the other USB function that aren’t related to USB uploads, the HID functions (Keyboard and Mouse) and Serial communication over USB. The HID functions are found here:

but they may depend on some of the core code also.

Here’s some hints to get you on the right track:

The Arduino IDE automatically adds the line:

#include <Arduino.h>

to every sketch. That file includes the core library files:

I believer USBAPI.h is the one of interest:

In turn that includes USBDesc

and USBCore:

Also, you have Main.cpp, which is the primary source file that runs everything else:
https://github.com/arduino/Arduino/blob/1.8.2/hardware/arduino/avr/cores/arduino/main.cpp
It has the lines:

#if defined(USBCON)
	USBDevice.attach();
#endif

So I’d start systematically commenting out code, upload, test, repeat. Make sure to write down your results as you go so you can remember it as you get to later tests.

I would start by commenting the USBAPI.h include in Arduino.h.

testpresta2:
And i have a second question: I tried to reboot with watchdog like you said.
It reboots the board, but the ariadne bootloader is skipped. Do you know how i can force a reset to bootloader without having to unplug power supply ? Thanks

The Ariadne bootloader should be activated by a watchdog reset. That is the recommended way to upload to the board:

  • Uploader sends command to reset
  • Arduino does watchdog reset
  • Uploader sends file using TFTP

The package even includes a library for this:

If you look through the source you'll see that indeed it uses the watchdog.

We are used to seeing serial bootloaders not be activated by watchdog resets. I believe the reason is that a serial upload can do a hardware reset so it's better to skip the delay after a watchdog reset. So it might be that the code to activate the bootloader needs to be updated to add ATmega32U4 support. The way the bootloader code detects a watchdog reset is via a register which is called MCUSR on some AVRs and MCUCSR on others. I know it's MCUSR on ATmega328P and MCUCSR on ATmega32U4 so they may not have written the code so that it can handle both register names.