Working on ArduinoCores...

Development Libraries
1

So... If you're working on an existing Arduino core (ArduinoCore-SAMD, say), is there some trick to attaching the core that you're working on to an existing IDE binary, for testing? (Short of just copying the changed files into the IDE's packages/... directories)

What's the status of automatic testing (just to see if everything that used to compile still compiles, or if binaries match for source changes that aren't supposed to affect the binary, and etc)?

And finally, are the various steps of ARDUINO_SAMD_VARIANT_COMPLIANCE defined anywhere?

2

I use file system link.

3

I probably am out of my depth because I don't understand what you mean by binary in this context, but I'll make a feeble attempt to help anyway:

is there some trick to attaching the core that you're working on to an existing IDE binary

How about installing it into the portable folder in the IDE installation?
https://www.arduino.cc/en/Guide/PortableIDE

What's the status of automatic testing (just to see if everything that used to compile still compiles

We have been using my Travis CI script in all the MCUdude repositories to compile all the example sketches on every commit/pull request for the last 1.5 years, and I think it's been helpful.

Here's the script:

Here's the Travis CI configuration used in MightyCore:

In this case, the configuration is fairly complex because not all of the examples compile for all the MightyCore boards. In a core where that was not the case, it is much more simple. In this case, only the library examples are compiled but you can compile additional test sketches if you like.

There are actually some newer features of the script to check the structure and metadata of libraries and example sketches which haven't been used in the MCUdude repos yet. I can give details on those (or anything else) if you're interested.

The main alternative for my script is this:

Although much more widely used than mine, I think it's inferior. Of course, I'm biased.

Another interesting, but quite different, project is this:

I haven't gotten around to using it yet.

4

We have been using my Travis CI script in all the MCUdude repositories to compile all the example sketches on every commit/pull request for the last 1.5 years, and I think it's been helpful.

man, I am just not understanding "Travis CI." Every tutorial or example I look at seems to assume some understanding of ... something ... that I don't have, and I cant figure out what it is until after I understand it.
In this case, I con't think I'm interested in "continuous integration", since I don't want to be adding .travis.yml files to repositories that don't have them. And it's a core rather than the IDE, so there isn't any "Build configuration" file. (and moreover, a core that has to be installed with the board manager...)
The particular patch I'm working on, I'd like to do more that just a "see if the build fails" - I want to compare binaries with the pre-patch version. Does Travis-CI do this sort of thing?

5

westfw:
man, I am just not understanding "Travis CI." Every tutorial or example I look at seems to assume some understanding of ... something ... that I don't have, and I cant figure out what it is until after I understand it.

I think the trouble with the Travis CI documentation is that it assumes you're using one of the common languages/build systems/whatever the correct term is, that they have set it up to "just work" with. In that case, you can basically just specify the language in .travis.yml and it will give you some level of useful functionality right from the start. As you need additional sophistication, you can dive in deeper.

But Arduino is not supported in this way. You need to do a little work to set .travis.yml up to compile an Arduino project and the documentation really sucks (at least the last time I looked) at easing you into that sort of configuration. I had a lot of frustration. Combine that with the fact that the Arduino IDE CLI has the annoying habit of trying the open the GUI when something goes wrong. On the headless system of a Travis CI build, this means the build just hangs for 10 minutes before timing out and you get absolutely no feedback on what went wrong.

My script is intended to make it easier for the average Arduino user to get started using Travis CI for continuous integration of their projects. I think we all benefit from more people doing this. I frequently see code released with bugs that would have been caught by compiling. I also see repository maintainers wasting time reviewing PRs that continuous integration would have automatically revealed were flawed.

I actually much prefer Circle CI to Travis CI. The only advantage to Travis CI is it doesn't impose per-build duration limits on the free plans. That limit of Circle CI made it not suitable for the very long builds of some of the more complex projects I'm involved in (and they're volunteer projects with no budget so signing up for a monthly bill won't work). If it wasn't for that, I would have switched over. I do plan to make my script support both Travis CI and Circle CI. Travis CI is the most popular by far so I think that was the best place to start.

westfw:
moreover, a core that has to be installed with the board manager...

That does make things a bit more tricky, but you can install cores that were intended to be installed via Boards Manager by simply copying the right files to the right places. That should be easy enough to set up in the .travis.yml file.

My script does not currently have a function to assist in this process. So far, I have only used it with hardware packages that allow manual installation. The script does that for you with a single function call.

westfw:
I want to compare binaries with the pre-patch version. Does Travis-CI do this sort of thing?

Essentially, anything you can do from the Linux command line, you can do with Travis CI. Essentially, each build is just a configurable Linux (or macOS) virtual machine running on someone else's server, with a report on the exit status of the commands you ran on that machine. The .travis.yml file just describes which VM configuration you want, and how you want the report to deal with the exit statuses of the commands. The YAML format can be a bit frustrating for more advanced scripting operations but you can always just write your own script in your choice of language, and then run that script from the .travis.yml file.