Micro USB loader firmware not working! no source file?

Using Arduino Installation & Troubleshooting
#1

Hi

I just procured an Arduino Micro as the simplest microprocessor to be used as a gadget connected to a Linux host.

The problem of the day is that avrdude is not able to send anything to the /dev/ttyACM0 port in a way that the Micro firmware respond as an application firmware loader function. Verified with a kermit null modem session, all what the Micro does is to send "TESTATA" repeatedly once the USB bus turns active.

Unfortunately, I could not find the firmware source code for what is in the unit on my desk. (Not very glamorous from an organization claiming to "love open-source" ...)

See below what lsusb -v reports for this USB device.

See also below what dmesg reports when the reset button is pressed, then released.

The kernel device driver source file /usr/src/linux-3.16.1/drivers/usb/class/cdc-acm.c is handling this USB device and creates the /dev/ttyACM0 device for the avrdude application. I resolved the read-write permission and other issues with avrdude.

Without the source code for exactly what is in this Micro board (as reflected in the USB descriptors reported by lsusb), I can not investigate any further.

Notably, it puzzles me that the USB descriptors report an HID function capability.

The list of software tarballs that I looked at is included below.

As it stands now, this Arduino Micro project is a failure, except for the (incomplete) learning experience. Apparently something is wrong with the factory-shipped firmware.

I would appreciate any help either with access to the exact source code for the firmware, or additional hints about linux kernel (3.16.1, 3.18.4 had the same symptoms) handling of this USB device.

Regards,

  • Thierry Moreau

============================
Bus 003 Device 026: ID 2a03:8037
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
idVendor 0x2a03
idProduct 0x8037
bcdDevice 1.00
iManufacturer 1 Arduino Srl
iProduct 2 Arduino Micro
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 100
bNumInterfaces 3
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 500mA
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 0
bInterfaceCount 2
bFunctionClass 2 Communications
bFunctionSubClass 2 Abstract (modem)
bFunctionProtocol 1 AT-commands (v.25ter)
iFunction 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 2 Communications
bInterfaceSubClass 2 Abstract (modem)
bInterfaceProtocol 0 None
iInterface 0
CDC Header:
bcdCDC 1.10
CDC Call Management:
bmCapabilities 0x01
call management
bDataInterface 1
CDC ACM:
bmCapabilities 0x06
sends break
line coding and serial state
CDC Union:
bMasterInterface 0
bSlaveInterface 1
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0010 1x 16 bytes
bInterval 64
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 10 CDC Data
bInterfaceSubClass 0 Unused
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 3 Human Interface Device
bInterfaceSubClass 0 No Subclass
bInterfaceProtocol 0 None
iInterface 0
HID Device Descriptor:
bLength 9
bDescriptorType 33
bcdHID 1.01
bCountryCode 0 Not supported
bNumDescriptors 1
bDescriptorType 34 Report
wDescriptorLength 101
Report Descriptors:
** UNAVAILABLE **
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x84 EP 4 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 1
Device Status: 0x0000
(Bus Powered)

[76797.934658] usb 3-2: USB disconnect, device number 26
[76798.806746] usb 3-2: new full-speed USB device number 27 using xhci_hcd
[76798.973727] usb 3-2: New USB device found, idVendor=2a03, idProduct=8037
[76798.973729] usb 3-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[76798.973731] usb 3-2: Product: Arduino Micro
[76798.973733] usb 3-2: Manufacturer: Arduino Srl
[76798.974121] cdc_acm 3-2:1.0: This device cannot do calls on its own. It is not a modem.
[76798.974791] cdc_acm 3-2:1.0: ttyACM0: USB ACM device
[76798.976226] input: Arduino Srl Arduino Micro as /devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2:1.2/0003:2A03:8037.0018/input/input36
[76798.976278] hid-generic 0003:2A03:8037.0018: input: USB HID v1.01 Mouse [Arduino Srl Arduino Micro] on usb-0000:00:14.0-2/input2

83fee56c18d5b633f13d1bf6dbfda095d9e2eb82 /home/tmoreau/distrib/arduino-1.6.11-linux64.tar.xz
90e5d0e9e19cb6118cf402bc6e9e7a8e1f5200db /home/tmoreau/distrib/Arduino-master.zip
c6a84df8f6392b671e546efab42dff64e60cfcab /home/tmoreau/distrib/avrdude-6.3.tar.gz
bed0bc1538735ab7049ab7af0b7d889b1a6f8956 /home/tmoreau/distrib/avr-libc-1.6.7.tar.bz2
5fecb9f9b0fcc37831772ced9cb40d8b8bca225e /home/tmoreau/distrib/LUFA-111009.zip

#2

The bootloader source is at {IDE installation folder}/hardware/arduino/avr/bootloaders/caterina or https://github.com/arduino/Arduino/tree/master/hardware/arduino/avr/bootloaders/caterina.

#3

This is where I looked first.

But the USB descriptors do not match: VID, PID, manufacturer name ...

(unless I am totally misreading something)

Sorry for challenging your volunteer help!

  • Thierry
#4

Is it one of the VID/PID values found in the Micro entry?

When I reset my micro it shows up as Arduino Micro bootloader in Windows Device Manager. Then once the bootloader exits it shows up as Arduino Micro. If I select Leonardo from the Tools > Board menu and Burn Bootloader then it shows up as Arduino Leonardo bootloader/Arduino Leonardo. So I think that the bootloader has a different VID/PID than the board when the bootloader is not running, when the VID/PID matches the one in boards.txt. There are multiple different values for VID/PID and product name that can be set:

micro.build.vid=0x2341
micro.build.pid=0x8037
micro.build.usb_product="Arduino Micro"

is used for:

micro.vid.0=0x2341
micro.pid.0=0x0037
micro.vid.1=0x2341
micro.pid.1=0x8037
micro.vid.2=0x2A03
micro.pid.2=0x0037
micro.vid.3=0x2A03
micro.pid.3=0x8037

Then there are the Genuino specific entries:

micro.vid.4=0x2341
micro.pid.4=0x0237
# If the board is a 2341:0237 use 2341:8237 for build and set
# other parameters as well
micro.vid.4.build.vid=0x2341
micro.vid.4.build.pid=0x8237
micro.vid.4.build.usb_product="Genuino Micro"
micro.vid.4.bootloader.file=caterina/Caterina-Genuino-Micro.hex

micro.vid.5=0x2341
micro.pid.5=0x8237
# If the board is a 2341:8237 use 2341:8237 for build and set
# other paramters as well
micro.vid.5.build.vid=0x2341
micro.vid.5.build.pid=0x8237
micro.vid.5.build.usb_product="Genuino Micro"
micro.vid.5.bootloader.file=caterina/Caterina-Genuino-Micro.hex

In the first case the bootloader file caterina/Caterina-Micro.hex is used, which is different from the bootloader used for Genuino.

#5

You ask:

Is it one of the VID/PID values found in the Micro entry?

If you look in my original post, it is 2a03:8037 (vid:pid), i.e. micro.vid.3 and micro.pid.3

Otherwise, when you refer to "Burn Bootloader", this is a function not available to me because a) I wish to use the USB to burn flash (application), so I have no ICSP tools, and b) I am having this difficulty at the very USB support by the Linux kernel interacting with this particular USB device unit.

About this possibility for the bootloader (with a first set of USB descriptors identified by a specific vid:pid) to time-out and then reset the USB with a new set of USB descriptors identified by a different vid:pid, I could not see this occurring from the kernel dmesg traces (or otherwise).

So my original question remains: where can I find the bootloader source code that corresponds to the USB descriptors shown in my original post.

You may observe that there is no entry micro.vid.3.bootloader.file entry in boards.txt.

Additionally, a limited reverse engineering effort targeting caterina/Caterina-Micro.hex provided clues that it is NOT a firmware that might have reported the above USB descriptors.

Stepping back from the specifics, I wonder if I might trust the Linux support for ICSP tools given the bad experience with the entry-level Micro (the ICSP tools would also be interfaced through USB CDC class and/or USB serial emulation I guess).

Regards,

  • Thierry
#6

thierry-moreau:
Additionally, a limited reverse engineering effort targeting caterina/Caterina-Micro.hex provided clues that it is NOT a firmware that might have reported the above USB descriptors.

I see now the manufacturer reported is Arduino Srl so that means it's probably the arduino.org firmware, which is located here:
https://github.com/arduino-org/Arduino/tree/master/hardware/arduino/avr/bootloaders/Caterina-ORG

#7

Getting closer ... thanks.

But not an exact fit: the product id is 8037 (on my desk) versus 0037 (on the archive build_all_target.h and Descriptors.c)

Also, the USB version is 2.0 (on my desk) versus 1.1 (in the archive Descriptors.c)

From a quick look, I do not see a string like TESTATA that the unit on my desk is outputting to the host.

Maybe I can learn something from this archive, but uncertainty remains as to exactly what is in this Micro unit.

Regards,

  • Thierry
#8

I just procured an Arduino Micro

The problem of the day is that avrdude is not able to send anything to the /dev/ttyACM0 port in a way that the Micro firmware respond as an application firmware loader function.

What ISP tools are you using? Have you tried uploading a sketch using the Arudino IDE? I'm not sure that the Micro works the way you seem to be expecting it to...

I believe that the way the Arduino Micro works is that it uses the HID port for actual upload. The serial port is just for user communication AND it has a magic reset command (set bitrate to 1200?) that invokes the bootloader (which then loads code over the HID.) So it's not expected that the normal sketch firmware respond to serial programmers/uploaders. It sounds like you have some sort of test sketch in there that sends "something."

You probably have the correct bootloader source code. Arduino/etc got a bit sensitive about people recompiling with changes while continuing to use the official VID/PID combos, and stopped distributing the source with their VID preset. I think. I'm not sure if there are any other differences.

You can check the Arduino.org site (they match the VID you report) and/or the Adafruit site (they designed the Micro, AND they're really evangelical about open source.)

There's also a weird reset dance with these native USB devices. When they reset and start the bootloader, they'll appear as one device, but when/if they run a user sketch with USB drivers, that will re-enumerate, possibly with different codes and/or ending up at a different place in the USB chain (different /dev/... serial port, for example.) There have been reports of this happening with just the wrong sort of timing on some hosts, leaving the USB drivers in "strange" states.

#9

(Looks like I was only partially correct. It DOESN'T Use the HID device, but it IS dependent on the reset sequence to run the bootloader code (a normal running sketch will NOT understand bootloader commands.)

Here's a log of a Leonardo (same CPU) being uploaded, with some comments... (IDE version 1.6.9, on a Mac.)

Sketch uses 4,592 bytes (16%) of program storage space. Maximum is 28,672 bytes. [color=brown][color=limegreen](end of compilation)[/color]
[/color]Global variables use 149 bytes (5%) of dynamic memory, leaving 2,411 bytes for local variables. Maximum is 2,560 bytes.

(Causing auto-reset. Note that this is done by the IDE.)

Forcing reset using 1200bps open/close on port /dev/cu.usbmodemfd3131

(Now the IDE waits for the interface to reappear, to address the timing issues I mentioned)

PORTS {/dev/cu.Bluetooth-Incoming-Port, /dev/cu.Bluetooth-Modem, /dev/cu.usbmodemfd3131, /dev/cu.usbmodemfd31131, /dev/cu.usbserial-FTD61T6Q, /dev/tty.Bluetooth-Incoming-Port, /dev/tty.Bluetooth-Modem, /dev/tty.usbmodemfd3131, /dev/tty.usbmodemfd31131, /dev/tty.usbserial-FTD61T6Q, } / {/dev/cu.Bluetooth-Incoming-Port, /dev/cu.Bluetooth-Modem, /dev/cu.usbmodemfd31131, /dev/cu.usbserial-FTD61T6Q, /dev/tty.Bluetooth-Incoming-Port, /dev/tty.Bluetooth-Modem, /dev/tty.usbmodemfd31131, /dev/tty.usbserial-FTD61T6Q, } => {}
PORTS {/dev/cu.Bluetooth-Incoming-Port, /dev/cu.Bluetooth-Modem, /dev/cu.usbmodemfd31131, /dev/cu.usbserial-FTD61T6Q, /dev/tty.Bluetooth-Incoming-Port, /dev/tty.Bluetooth-Modem, /dev/tty.usbmodemfd31131, /dev/tty.usbserial-FTD61T6Q, } / {/dev/cu.Bluetooth-Incoming-Port, /dev/cu.Bluetooth-Modem, /dev/cu.usbmodemfd31131, /dev/cu.usbserial-FTD61T6Q, /dev/tty.Bluetooth-Incoming-Port, /dev/tty.Bluetooth-Modem, /dev/tty.usbmodemfd31131, /dev/tty.usbserial-FTD61T6Q, } => {}
PORTS {/dev/cu.Bluetooth-Incoming-Port, /dev/cu.Bluetooth-Modem, /dev/cu.usbmodemfd31131, /dev/cu.usbserial-FTD61T6Q, /dev/tty.Bluetooth-Incoming-Port, /dev/tty.Bluetooth-Modem, /dev/tty.usbmodemfd31131, /dev/tty.usbserial-FTD61T6Q, } / {/dev/cu.Bluetooth-Incoming-Port, /dev/cu.Bluetooth-Modem, /dev/cu.usbmodemfd3131, /dev/cu.usbmodemfd31131, /dev/cu.usbserial-FTD61T6Q, /dev/tty.Bluetooth-Incoming-Port, /dev/tty.Bluetooth-Modem, /dev/tty.usbmodemfd3131, /dev/tty.usbmodemfd31131, /dev/tty.usbserial-FTD61T6Q, } => {/dev/cu.usbmodemfd3131, /dev/tty.usbmodemfd3131, }

(ah hah! There it is! Now start avrdude using the appropriate protocol. Quickly, before the bootloader times out!)

Found upload port: /dev/cu.usbmodemfd3131
/Applications/arduino/Arduino-1.6.9.app/Contents/Java/hardware/tools/avr/bin/avrdude -C/Applications/arduino/Arduino-1.6.9.app/Contents/Java/hardware/tools/avr/etc/avrdude.conf -v -patmega32u4 -cavr109 -P/dev/cu.usbmodemfd3131 -b57600 -D -Uflash:w:/var/folders/jz/5yb8f2hr8xjcpf0059bsfz4r0000gn/T/build6caae9a3141ae5f60042a4271a07b2b7.tmp/Blink.ino.hex:i

(Avrdude runs successfully)

avrdude: Version 6.0.1, compiled on Apr 14 2015 at 16:30:25
#10

Hi!

"What ISP tools are you using? Have you tried uploading a sketch using the Arudino IDE?"

Actually, I want to "look under the hood" of any Integrated Development Environment, such that any "magic" in the open source technology is mastered as a traceable engineering issue. This is motivated by a self-defense strategy against any opportunity to inject a bug/Trojan horse in the device or host. (This exact motivation should be irrelevant.)

This is why I focused on the USB Linux support for the USB descriptors reported by the device. These are the only reliable engineering data, because I still miss the exact source code for the bootloader firmware.

Initially, I was confident that the basic ISP tool exploiting the USB ISP capability would do the job. Now I am almost convinced of the reverse. That is, I see the JTAG/ICSP firmware loading mechanism as the only reliable one: a future procurement of additional Micro units may well be loaded with a different firmware (different "magic").

Your post is well appreciated as informative. Paradoxically, it hints that I am not advanced towards a satisfactory solution. Let me focus on the "open source" dedication.

You write:

"You probably have the correct bootloader source code. Arduino/etc got a bit sensitive about people recompiling with changes while continuing to use the official VID/PID combos, and stopped distributing the source with their VID preset."

Not blaming the messenger, this is antagonistic to the open source principle: if controlled registration rules conflict with the right to modify the open source work, the latter should prevail. (Wikipedia reports 0xF055 as the ad-hoc VID as a mitigation attempt in this specific instance.)

You also point to

"the Adafruit site (they designed the Micro, AND they're really evangelical about open source.)"

I was led to https://forums.adafruit.com and found no download opportunity, neither for software/firmware nor for hardware design data. Evangelistic speech is yet unable to bring Heaven to Earth, I guess!

As a last piece of information, I filed a support issue with Arduino.org support team (so far acknowledged, but unanswered) as the actual hardware vendor. I should be able to report a relevant outcome in the present forum thread.

Regards,

  • Thierry
#11

want to "look under the hood" of any Integrated Development Environment,

Sure, but don't you think you should see if it actually works in the intended environment first?
Right now it looks like "I bought a product with a product-specific bootloader and I think it's broken because I can't get it to load code using a different product."

I focused on the USB Linux support for the USB descriptors reported by the device. These are the only reliable engineering data, because I still miss the exact source code for the bootloader firmware.

But the data "reported by the device" is dynamic. I've already said that the device re-enumerates in between the bootloader and the user sketch; if you're looking at threat models, you should consider that the sketch will wait until 3am and then re-enumerate as a CD drive with a malware autorun payload. The Arduino bootloader is not a secure operating system that protects the USB port, and it doesn't have ANY ability to stop the sketch from doing something entirely different from anything in the bootloader source.

I see the JTAG/ICSP firmware loading mechanism as the only reliable one: a future procurement of additional Micro units may well be loaded with a different firmware (different "magic").

That's certainly correct. People have gotten arduino boards without a bootloader installed, or with an unexpected bootloader (atmegaboot vs optiboot.) (and yet some want to remove the "version number" that I carefully added to Optiboot!) People have (somehow) damaged their bootloaders so that they need to be reloaded. The only fully reliable loading method is what is built into the chip (not that some chips have USB bootloader built into their ROM.)

this is antagonistic to the open source principle

Yeah, well, that's what happens when OSSW meets semi-open SW (like LUFA) and "tightly paranoid" industry trade groups like the USB Implementers Forum. The whole F055 thing seems to be a complete fiasco (there was a site supporting registration of PIDs/etc, but they "disappeared" and took their database with them. Other attempts at "sharing" a VID have been vehmently attacked by the USB-IG.)
Have you looked into the build procedure for the Caterina Bootloader yet? You have to pull in LUFA separately (because of its licensing requirements), and you should be careful to make sure all your compiler and library versions match. It looks like the VID/PID is specified in the makefile rather than in the source code. At some point, you can try a build with the same VID/PID and see if you get the same binary (dumped via ICSP from the chip?) But it's not "instant" and it's not trivial.

I was led to https://forums.adafruit.com and found no download opportunity

Yes, that would be their user forums. Try the product pages and/or their github repository (actually, it looks like the Arduino Micro mostly points to the the Arduino.cc pages for product documentation and tutorials...)
But: GitHub - adafruit/Caterina-Bootloader has more hex files and more info about which versions of various other pieces should be than most places...

#12

thierry-moreau:
I see the JTAG/ICSP firmware loading mechanism as the only reliable one: a future procurement of additional Micro units may well be loaded with a different firmware (different "magic").

Just build your own copy Caterina and then burn that bootloader to every Micro as soon as you get it, then you don't need to worry about what it ships with. I always burn a fresh bootloader to all my clones when I receive them, not because of security but because I want to be sure I don't have some antiquated bootloader version running on them. Most of the Pro Micros I get show up as "Arduino Leonardo" when I they arrive, which isn't a big deal to me but would confuse anyone with less experience with Arduino.

#13

I did a bit of looking, and it seems that there are about three versions of the Caterina bootloader source floating around.

  • There's the version in hardware/.../avr/bootloaders/caterina - I think this is the original version, and it hasn't changed since when the leonardo was first introduced.
  • It looks like there were some changes in about 2012 to support the Arduino Robot and lilypad. This source is in .../caterina-Arduino_Robot/ Arduino.org is apparently using this version as well. (I don't see any reason it shouldn't run on a Micro as well.)
  • There's the adafruit version. This actually looks closer to the original, with the addition of the "doubletap reset" feature.

(This is just comparing "Caterina.c" The VID/PID are entered using the Makefile, so they NEVER appear in the source code directly.)

#14