westfw:
man, I am just not understanding "Travis CI." Every tutorial or example I look at seems to assume some understanding of ... something ... that I don't have, and I cant figure out what it is until after I understand it.
I think the trouble with the Travis CI documentation is that it assumes you're using one of the common languages/build systems/whatever the correct term is, that they have set it up to "just work" with. In that case, you can basically just specify the language in .travis.yml and it will give you some level of useful functionality right from the start. As you need additional sophistication, you can dive in deeper.
But Arduino is not supported in this way. You need to do a little work to set .travis.yml up to compile an Arduino project and the documentation really sucks (at least the last time I looked) at easing you into that sort of configuration. I had a lot of frustration. Combine that with the fact that the Arduino IDE CLI has the annoying habit of trying the open the GUI when something goes wrong. On the headless system of a Travis CI build, this means the build just hangs for 10 minutes before timing out and you get absolutely no feedback on what went wrong.
My script is intended to make it easier for the average Arduino user to get started using Travis CI for continuous integration of their projects. I think we all benefit from more people doing this. I frequently see code released with bugs that would have been caught by compiling. I also see repository maintainers wasting time reviewing PRs that continuous integration would have automatically revealed were flawed.
I actually much prefer Circle CI to Travis CI. The only advantage to Travis CI is it doesn't impose per-build duration limits on the free plans. That limit of Circle CI made it not suitable for the very long builds of some of the more complex projects I'm involved in (and they're volunteer projects with no budget so signing up for a monthly bill won't work). If it wasn't for that, I would have switched over. I do plan to make my script support both Travis CI and Circle CI. Travis CI is the most popular by far so I think that was the best place to start.
westfw:
moreover, a core that has to be installed with the board manager...
That does make things a bit more tricky, but you can install cores that were intended to be installed via Boards Manager by simply copying the right files to the right places. That should be easy enough to set up in the .travis.yml file.
My script does not currently have a function to assist in this process. So far, I have only used it with hardware packages that allow manual installation. The script does that for you with a single function call.
westfw:
I want to compare binaries with the pre-patch version. Does Travis-CI do this sort of thing?
Essentially, anything you can do from the Linux command line, you can do with Travis CI. Essentially, each build is just a configurable Linux (or macOS) virtual machine running on someone else's server, with a report on the exit status of the commands you ran on that machine. The .travis.yml file just describes which VM configuration you want, and how you want the report to deal with the exit statuses of the commands. The YAML format can be a bit frustrating for more advanced scripting operations but you can always just write your own script in your choice of language, and then run that script from the .travis.yml file.