Why have I successfully achieved what I desired (control of VID/PID), even though I haven't done anything that makes sense (to me) in boards.txt ?

I have many arduinos, but recently I have been working on a project that requires me to use two (genuine) Arduino Due's connected to the same computer at the same time. When actually doing the job they are designed to do, each provides USBmidi inputs and outputs to the mac via their native USB ports. I generally program them via their programming ports (and usually when only one Due is connected) and I have no problems programming them. In fact the Dues do their job very well.

However, recently I realised that the midi-applications on the mac communicating with the Dues via midi were getting confused by both Due's calling themselves "Arduino Due" and having the same USB PID and VID.

So I learned enough about how to create "custom hardware" descriptions in my scripts folders so that (via careful editing of boards.txt files) I could persuade the Due's to call themselves different things, and report different PIDs and VIDs to what they would have had by default.

That, too, works, so in principle I should count myself happy that one of the devices now shows up in the mac's systems USB device list as "OrganStops" instead of "Arduino Due" and it has a new PID and VID. The client programs talking to them are even happier than before. I am really happy!

However ....... and here is my reason for posting a question here ...... I found it nigh on impossible while I was doing all that to really work out what the PID and VID settings in boards.txt were really doing. The reason this was hard was because the mac was clearly caching a number of settings, and so nine times out of ten (even after reboots) the mac would sometimes report out of date information that was clearly not what the Due was broadcasting or had picked up from boards.txt . My success at triggering a USB cache flush felt not better than random shots in the dark with probability 20% of success after each change to boards.txt, even though I ended up in a good place eventually. This made it super hard to really work out which of the changes I was making to boards.txt were actually helping me, which were inconsequential or even counter productive.

Now, I am not asking for help with the mac or help to get it to update its USB devices.

But I would like to ask for help/advice about boards.txt.

Firstly, here is the "default" boards.txt for a Due which I got on my mac from $HOME/Library/Arduino15/packages/arduino/hardware/sam/1.6.12/boards.txt :

arduino_due_x_dbg.name=Arduino Due (Programming Port)
arduino_due_x_dbg.vid.0=0x2341
arduino_due_x_dbg.pid.0=0x003d
arduino_due_x_dbg.vid.1=0x2A03
arduino_due_x_dbg.pid.1=0x003d
arduino_due_x_dbg.upload.tool=bossac
arduino_due_x_dbg.upload.protocol=sam-ba
arduino_due_x_dbg.upload.maximum_size=524288
arduino_due_x_dbg.upload.use_1200bps_touch=true
arduino_due_x_dbg.upload.wait_for_upload_port=false
arduino_due_x_dbg.upload.native_usb=false
arduino_due_x_dbg.build.mcu=cortex-m3
arduino_due_x_dbg.build.f_cpu=84000000L
arduino_due_x_dbg.build.usb_manufacturer="Arduino LLC"
arduino_due_x_dbg.build.usb_product="Arduino Due"
arduino_due_x_dbg.build.board=SAM_DUE
arduino_due_x_dbg.build.core=arduino
arduino_due_x_dbg.build.extra_flags=-D__SAM3X8E__ -mthumb {build.usb_flags}
arduino_due_x_dbg.build.ldscript=linker_scripts/gcc/flash.ld
arduino_due_x_dbg.build.variant=arduino_due_x
arduino_due_x_dbg.build.variant_system_lib=libsam_sam3x8e_gcc_rel.a
arduino_due_x_dbg.build.vid=0x2341
arduino_due_x_dbg.build.pid=0x003e

arduino_due_x.name=Arduino Due (Native USB Port)
arduino_due_x.vid.0=0x2341
arduino_due_x.pid.0=0x003e
arduino_due_x.vid.1=0x2A03
arduino_due_x.pid.1=0x003e
arduino_due_x.upload.tool=bossac
arduino_due_x.upload.protocol=sam-ba
arduino_due_x.upload.maximum_size=524288
arduino_due_x.upload.use_1200bps_touch=true
arduino_due_x.upload.wait_for_upload_port=true
arduino_due_x.upload.native_usb=true
arduino_due_x.build.mcu=cortex-m3
arduino_due_x.build.f_cpu=84000000L
arduino_due_x.build.usb_manufacturer="Arduino LLC"
arduino_due_x.build.usb_product="Arduino Due"
arduino_due_x.build.board=SAM_DUE
arduino_due_x.build.core=arduino
arduino_due_x.build.extra_flags=-D__SAM3X8E__ -mthumb {build.usb_flags}
arduino_due_x.build.ldscript=linker_scripts/gcc/flash.ld
arduino_due_x.build.variant=arduino_due_x
arduino_due_x.build.variant_system_lib=libsam_sam3x8e_gcc_rel.a
arduino_due_x.build.vid=0x2341
arduino_due_x.build.pid=0x003e

You can see that it has two sections, the first for the programming port, the second for the native port.

If I delete everything except the USB/PID/VID lines to emphasise what's different in those we have the following: (this deletion is just for the post ... I didn't actually delete lines in the file!)

arduino_due_x_dbg.name=Arduino Due (Programming Port)
arduino_due_x_dbg.vid.0=0x2341
arduino_due_x_dbg.pid.0=0x003d
arduino_due_x_dbg.vid.1=0x2A03
arduino_due_x_dbg.pid.1=0x003d
arduino_due_x_dbg.build.usb_product="Arduino Due"
arduino_due_x_dbg.build.vid=0x2341
arduino_due_x_dbg.build.pid=0x003e

arduino_due_x.name=Arduino Due (Native USB Port)
arduino_due_x.vid.0=0x2341
arduino_due_x.pid.0=0x003e
arduino_due_x.vid.1=0x2A03
arduino_due_x.pid.1=0x003e
arduino_due_x.build.usb_product="Arduino Due"
arduino_due_x.build.vid=0x2341
arduino_due_x.build.pid=0x003e

and if I compress the above lines further (just for the post, not in the real file) by deleting duplicate information so that we can more easily compare the two sections we have the following.

x_dbg.name=Arduino Due (Programming Port)
x_dbg.build.usb_product="Arduino Due"   # later changed by me to "OrganStops" 
x_dbg.      vidpid.0=0x2341 0x003d       
x_dbg.      vidpid.1=0x2A03 0x003d
x_dbg.build.vidpid  =0x2341 0x003e      # later changed by me to 0x2340 0x003c

x.name=Arduino Due (Native USB Port)
x.build.usb_product="Arduino Due"       # later changed by me to "OrganStops" 
x.      vidpid.0=0x2341 0x003e
x.      vidpid.1=0x2A03 0x003e
x.build.vidpid. =0x2341 0x003e

In the above I have also notated the changes that I ended up with in my "final" version of boards.txt that left me with a usefully working board (albeit with very little confidence as to why it worked).

The default boards.txt file seems already a bit "mysterious" to be in sense that the Native port is seen to have 0x003e as the PID everywhere, but (for a reason I could not find explained anywhere) the Programming port uses PID 0x003d in two places but 0x003e in one. Is this important? I couldn't tell.

More worryingly, in the final boards.txt configuration that worked for me (though who knows, the mac may still be caching some earlier config without telling me?) the VID and PID I altered were in the lines relating to the PROGRAMMING port (see above), yet I have the device plugged in by only the NATIVE PORT ONLY, and yet the mac is reporting to me that it sees the device as VID 0x2340 PID 0x003c

This leaves me thoroughly confused. Why on earth do changes I make to the PROGRAMMING PORTS section of boards.txt show up on a board I have only had plugged in on the Native port??? (It;s only been plugged in on that cable for the entirety of the time that the mac has been on today). OK, it could be more mac caching issues, but I am beginning to think that actually, in this instance, the mac really is reporting what it sees. And this leaves me very confused as to what the difference between the upper and lower half of boards.txt really is. I had started by assuming that the top half is specifying VID/PID numbers for the programming port and the bottom have VID/PID numbers for the native port. But perhaps that idea was completely wrong (indeed it seems experimentally to have beeb wrong given what is working for me!).

Finally, I was hopeful when I began fiddling with boards.txt that I could make the Due present one PID/VID when plugged in only via its native port, and a different PID/VID when only plugged in via its Native port. However, I am now beginning to wonder whether that's possible at all, because I realise that although my OrganStop controller works, my mac appears to only ever have reported a single VID-PID pair in the "system report" window that lets me look at the USB devices, even though I sometimes have the object connected by one port and sometimes by another.

Can any of you give me tips/pointers as to what I am failing to understand about boards.txt ?

There are some docs at Platform specification - Arduino CLI that are trying to help, but don't really make much sense, and in any case appear to be a bit out of date (e.g. build.pid and build.vid get zero mentions, even though they are the only things I had success after changing).

Thanks,

Christopher

Actually, I should make a small correction/update to the above. Above I said that mac os had never given me two different instances of the board visible in its USB device list. Though this was (so far as I can tell) true yesterday, it is no longer true now, today.

Until 10 minutes ago, today I had only had the Native port connected (with no board.txt configuration changes or reflashes since configs I used yesterday and reported in my post above) and the mac's OS USB device list was showing me:

===================================
OrganStops:

  Product ID:	0x003c
  Vendor ID:	0x2340
  Version:	1.00
  Serial Number:	MIDI
  Speed:	Up to 480Mb/sec
  Manufacturer:	Arduino LLC
  Location ID:	0x14400000 / 2
  Current Available (mA):	500
  Current Required (mA):	500
  Extra Operating Current (mA):	0
===================================

however, 10 mins ago I decided to see what would happen if I plugged in the Programming port, and now the OS reports:

===================================
OrganStops:

  Product ID:	0x003c
  Vendor ID:	0x2340
  Version:	1.00
  Serial Number:	MIDI
  Speed:	Up to 480Mb/sec
  Manufacturer:	Arduino LLC
  Location ID:	0x14400000 / 2
  Current Available (mA):	500
  Current Required (mA):	500
  Extra Operating Current (mA):	0
===================================
Arduino Due Prog. Port:

  Product ID:	0x003d
  Vendor ID:	0x2341
  Version:	0.01
  Serial Number:	34239313335351B09182
  Speed:	Up to 12Mb/sec
  Manufacturer:	Arduino (www.arduino.cc)
  Location ID:	0x14630000 / 7
  Current Available (mA):	500
  Current Required (mA):	100
  Extra Operating Current (mA):	0
===================================

I cannot be 100% sure whether I can now see two things above only because the osx cache timed out and refreshed, or if I just missed it yesterday. But either way, it's good/reassuring to see two entries appear.

Nonetheless, it's still seems to me very strange, in fact even stranger than before, that the PID 0x003c that appears on the thing with Serial Number MIDI and which is on the Native part was set by me in the part of boards.txt that contains all the mentions of the Programming port (not the native port).

I still don't know if this is all really just cache issues in the mac rather than expected behaviour.

I think I might be starting to make progress.

I am beginning to get the feeling that the two sections of boards.txt are not (as I had previously assumed) designed to configure the two different ports, but are instead each designed to configure the entire board, but differently depending on which port was used to program the board.

My fairly strong working hypothesis at the moment is that:

  • the arduino_due_x_dbg.BLAH section of boards.txt configures the entire board when it is programmed over the Programming Port
  • the arduino_due_x.BLAH section of boards.txt configures the entire board when it is programmed over the Native Port
  • in both sections the build.usb_product, build.vid and build.pid parts ONLY affect the way that the Native Port appears to USB Host in normal use.

In addition, without much evidence yet to support it, (but I will try to test later), I am guessing that, perhaps,

  • in both sections the vid.0 and pid.0 always control what the Programming Port returns
  • in both sections the vid.1 and pid.1 are deliberately different so that some kind of board-discovery process interrogating the programming port can determine whether the board was last programmed over the native or last programmed over the programming port

I have yet to determine whether the name field in both sections sets the name of the programming port, but I know that the programming port is called "Arduino Due Prog. Port" on my boards which I program via the Programming Port. This is not the same as the name field, but it could be the os removing some characters and simplifying it??? Alternatively the name of the programming port can perhaps not be changed via boards.txt and is set somewhere else.

[EDIT: I just noticed that:

$HOME/Library/Arduino15/packages/arduino/hardware/sam/1.6.12/firmwares/atmega16u2/arduino-usbserial/Descriptors.c

contains a line

.UnicodeString = L"Arduino Due Prog. Port" ]

The given firmware tells the PC how the port is named.

For the native port, the topic Change board name, multiple boards of the same type describes how one can change the name of the board. It's quite lengthy due to discussion up and down. Note that this was for Leonardo/Pro Micro but the same principles should apply to the Due.

Thank you @sterretje -- there do seem to be some very good/useful pointers in the thread you have linked. [ I'd tried to find useful other threads on here before, but that one escaped me. ]