Arduino command-line interface.

It's surprising to me how much buzz this has gotten.

It's getting hyped as "Arduino now has an officially supported command-line interface" but here's the thing: A command line interface has existed for years:

This is a part of Arduino's campaign to split out individual components that used to be all crammed into the Arduino IDE. The other parts of this project are arduino-builder and arduino-preprocessor.

The primary nice thing about arduino-cli over using the existing Arduino IDE CLI is that you can skip installing the GUI components if you're not going to use them. That could be really great for online continuous integration services (e.g Travis CI, Circle CI) where you previously had to download hundreds of megabytes of Arduino IDE every single time you ran a CI build.

It seems arduino-cli might have some added features over the Arduino IDE's CLI. It's unfortunate that the two aren't intercompatible.

However, this is definitely the time for any interested parties to get involved if they want to have influence in the development of arduino-cli. Arduino is actively seeking input right now. Once the tool is in production use it's going to be much more difficult to get any significant changes made.

I'm very much interested in more efficient continuous integration testing of Arduino projects. However, an important aspect of testing for me is backwards compatibility. That can only be done via the standard Arduino IDE CLI and it seems very messy to use the IDE's CLI for testing compatibility with old IDE versions, then arduino-cli for testing new ones. So I don't see myself benefiting much from arduino-cli for some years to come.

I am excited to see what other people will use it for though. The hope is that this will allow better integration with alternative IDEs. I also wouldn't mind seeing it slow the popularity of PlatformIO because use of that platform has lead to a lot of code being published that's advertised as "Arduino" but actually isn't even compatible with the Arduino IDE.

I have been using the existing command line interface for quite some time. I rarely use the IDE.

I hope the developers will take the trouble to explain the differences if there are any.

...R

I know this is an old thread but I just spend a couple of days installing and getting the arduino-cli to work (compile /upload) with my uno and stm32 boards. Using it with the uno was fine but getting it to work with my stm32 board was difficult. There is very little documentation available, really just the readme file.

The CLI is written in GOLANG which I have never used before, so going through the code did not help me.

I learned more from reading through the issues related to problems I was having.

I did use Robins python script to build and upload from Geany and that worked great and more straight forward than the new CLI. I went back to windows and could not get the script running with geany on windows. I have switched back to linux for good now so I may just re-load that again.

The associated issue report:

I agree that the documentation of arduino-cli has a lot of room for improvement and that there are rough edges in the interface as well. You can see from the issue that the two primary developers of the project take your issue report seriously and are going to work to improve this.

The arduino-cli project only just went from alpha to beta phase and is under very active development. I think the heavy publicity arduino-cli received on sites like Hackaday, etc. happened a little too early and without providing nearly enough warning that it was in the alpha development phase at that time. This resulted in a lot of people using it with the unreasonable expectation that it was in a production-ready development stage. The fact is that you are all beta testers, whether you like it or not. We really need as many people to get involved with testing this project, providing useful feedback (like you did), and contributing development work in order to assure that arduino-cli ends up being the best tool it can be. Now is our chance to have a say in the basic design of the interface. Once it goes out of beta, it will be much more difficult to convince the developers to make breaking changes, even if they are clearly an improvement. In addition to its use from the command line, arduino-cli is going to be the heart of the Arduino IDE, Arduino Web Editor, and several 3rd party IDE's. So this is a really important thing to get right.

pert:
We really need as many people to get involved with testing this project, providing useful feedback (like you did),

I asked in Reply #2 for the developers to explain the differences between the older command-line system that I use and the new system - with the usual deafening silence by way of response.

Based on the comments in Reply #3 I can't find the enthusiasm to try to learn yet-another-more-complicated-way-of-doing-the-same-thing.

On the other hand if someone were to take the trouble to explain the differences I might come to the conclusion that the newer way offers enough advantages to make it worth investigating.

@Pert. these comments are not intended as any criticism of you. However I am aware that you are better placed than I am to bring them to the attention of the right people.

...R

I've actually used arduino-cli a lot since this thread started and have played a minor role in its development and documentation, so I think I am now qualified to answer your question Robin2.

The first thing I should say is that arduino-cli has come a long way since the time of my original comment. The only obvious functionality I now notice missing compared to the IDE is Burn Bootloader, but the Arduino IDE's CLI doesn't have that either. In addition, arduino-cli does have quite a bit of functionality that the Arduino IDE CLI doesn't have, and even some that the Arduino IDE's GUI doesn't have.

The main bonus feature of arduino-cli for the average user is the ability to set custom configurations for each sketch. This includes the ability to associate a specific board and port with each sketch, which will be used by default when compiling/uploading that sketch unless another board and/or port is specified in the command. This is a feature that has been requested many times by users of the Arduino IDE over the years.

arduino-cli also has output functionality that the Arduino IDE CLI doesn't have. For example, you can search for installed libraries or libraries available from Library Manager. Same with boards and Boards Manager. You can choose to have the output formatted as JSON to make it easy for a machine to parse. That is likely more of interest to people who are integrating arduino-cli into an application than for the average user.

As I said before, the really big advantage of arduino-cli vs. using the Arduino IDE's CLI is that arduino-cli is tiny compared to the Arduino IDE. That will be most helpful for people using a CI service where they are likely installing the IDE or arduino-cli for every CI build, but it could also be helpful to people who are concerned about disk space usage or bandwidth.

The Arduino IDE CLI has the incredibly annoying behavior of opening the GUI when something goes wrong with a CLI command. That is most horrible on a headless system where there is no way for the GUI to run, so it just hangs, without giving any indication of what went wrong. I have spent countless frustrating hours trying to troubleshoot problems in Travis CI builds due to this. arduino-cli doesn't have that problem.

arduino-cli is faster than the Arduino IDE CLI. For example, compiling BareMinimum for Uno took a little over 5 seconds with Arduino IDE CLI. On arduino-cli the same compile took under 4 seconds the first time. The Arduino IDE CLI clears the compilation cache every time, so every compile takes that same 5 s. arduino-cli is able to use the previous compilation cache so subsequent compilations only took a little over 1 second.

The great thing is that the Arduino IDE will eventually use arduino-cli so most of the useful features added to arduino-cli are also going to benefit users of the Arduino IDE.

Very many thanks for all that info. It's nice to have real contact with the inside. I just wish it was not left to you to do everything.

The main bonus feature of arduino-cli for the average user is the ability to set custom configurations for each sketch. This includes the ability to associate a specific board and port with each sketch,

I have that capability built into my Python program.

arduino-cli also has output functionality that the Arduino IDE CLI doesn't have. For example, you can search for installed libraries or libraries available from Library Manager. Same with boards and Boards Manager.

I try to avoid using them. I prefer the bare metal :slight_smile:

arduino-cli is faster than the Arduino IDE CLI. For example, compiling BareMinimum for Uno took a little over 5 seconds with Arduino IDE CLI. On arduino-cli the same compile took under 4 seconds the first time. The Arduino IDE CLI clears the compilation cache every time, so every compile takes that same 5 s. arduino-cli is able to use the previous compilation cache so subsequent compilations only took a little over 1 second.

That might be nice but I have survived so far without it - why didn't they just build that into the IDE CLI ?

...R

PS ... why is this in Bar Sport ?

Thanks for that over view, pert.

You are definitely not bumping into things in the dark like I am.

After reading a few articles on-line about it. I thought the arduino-cli tool was released.

I guess I found out the hard way. My two biggest obstacles with the tool is how the arduino-cli.yaml
file is used. Like what goes in there and what can't. Along with what goes on the command line itself.

As you could glean from the issue I started it was a lot of mucking around and a good measure of luck to get it working.

I'm sure once they get a wiki together it will be easier to get up to speed with the tool especially for 3rd party boards.

Robin2:
I have that capability built into my Python program.

That's great, but also think it's nice to have an official system that will be used universally by everyone once arduino-cli reaches full integration. It might even have some beneficial applications for people publishing sketches. For example, Arduino Web Editor has a similar sketch metadata file that, in addition to associating a board with the sketch, can define the specific version of a library dependency to use when compiling that sketch (because in Arduino Web Editor all versions of every library in the Arduino Library Manager index are pre-installed).

Robin2:
I try to avoid using them. I prefer the bare metal :slight_smile:

Do you mean you avoid using libraries and hardware packages altogether, or just that you avoid using Library Manager and Boards Manager and do manual installations of libraries and hardware packages instead?

Robin2:
That might be nice but I have survived so far without it

Yeah, a few seconds isn't a huge deal to me. Even so, I always find myself surprised by that fast compile of arduino-cli, thinking "that was too quick, something must be wrong!".

Robin2:
why didn't they just build that into the IDE CLI ?

I'm not sure. The Arduino IDE deletes all its temporary files, including the compile cache when it exits and I think that is a good behavior. So you could say that it's annoying that arduino-cli doesn't clean up after itself. However, in the Arduino IDE the clean up really doesn't cost you much of the benefits of the caching, since you are usually compiling many times for each IDE session. In the case of the CLI, you only get one compilation per session. Probably the best solution would be to add a command to arduino-cli to clear all the temporary files. Then people can easily do the clean up when they're ready for it.

Robin2:
PS ... why is this in Bar Sport ?

Good question. I think we've had some valuable discussion here that merits another forum section. I'll move it to "General Discussion".

noweare:
You are definitely not bumping into things in the dark like I am.

Ha! I definitely am doing plenty of bumping around in the dark when it comes to arduino-cli. I have very little knowledge of the Go programming language, so most of what I know about arduino-cli was from trial and error. I do have a lot of experience with things like FQBN, because these are also used in the Arduino IDE, but I know less about Java than Go, so I got most of my knowledge of the working of the Arduino IDE through trial and error as well.

noweare:
After reading a few articles on-line about it. I thought the arduino-cli tool was released.

There have been several releases, but all except the newest were "preview" releases of the tool in its alpha development phase, mostly intended for use by beta (alpha?) testers. The first beta release was only done two weeks ago.

noweare:
how the arduino-cli.yaml
file is used

I really think this is something that needs to be documented better. It is mentioned in the readme in relation to installing a 3rd party hardware package, but there are also other configuration options you can set in that file that aren't documented anywhere other than in the source code. You can also store arduino-cli.yaml files in multiple locations and those locations, as well as their precedence, is not documented anywhere.

noweare:
Along with what goes on the command line itself.

The situation is better for the command line documentation because arduino-cli has built-in documentation for all the commands accessible via:

arduino-cli help [command]

or

arduino-cli [command] --help

There are probably some places where that documentation could be improved, but it's pretty good already. I haven't decided whether it would make sense to also provide all that information in the external documentation content. The advantage is that this would allow us to provide short, to the point descriptions in the built-in documentation and more detailed information in the external documentation. The disadvantage is this would lead to a lot of duplicate content to maintain.

noweare:
As you could glean from the issue I started it was a lot of mucking around and a good measure of luck to get it working.

Well, always feel free to ask questions here on the forum when you get stuck if you like. It's true that a lot of the forum regulars are not experienced with arduino-cli, but a good deal of the general knowledge we have of the Arduino IDE also applies to arduino-cli (such as the FQBN thing).

noweare:
I'm sure once they get a wiki together it will be easier to get up to speed with the tool especially for 3rd party boards.

I'm hoping I'll get the opportunity to contribute to that effort. I have done a lot of work on documenting arduino-cli, but it's all been specific to an advanced gRPC interface that is only really useful to people integrating it into their own applications (such as an IDE), so not something useful to the average user.

pert:
Do you mean [...] or just that you avoid using Library Manager and Boards Manager and do manual installations of libraries and hardware packages instead?

Yes.

Partly because I'm a Luddite - so don't take me too seriously.

More seriously because I am wary of systems that have a central repository of libraries in case an upgrade needed for one purpose causes another to break.

And because I got my fingers burned years ago with what were allegedly convenience tools for working with Ruby that were a nightmare for a newbie to understand when I didn't want to do exactly what the tool was written for. It turned out to be a lot less trouble to learn how to do the thing from the command line but a lot of time was wasted thinking the tool must be the best and quickest way.

It's the old choice between giving a man a fish or teaching him how to catch fish. I much prefer the latter and IMHO it would take a lot less effort on the part of the developers just to write decent instructions rather than writing programs to avoid the need for instructions (which never actually works as Reply #8 indicates). But we all know that programmers prefer writing code to writing documentation.

...R

No shortage of evidence in this Thread to support the much greater value of writing the fishing instructions rather than handing out fish. :slight_smile: :slight_smile:

...R

Robin2:
I am wary of systems that have a central repository of libraries in case an upgrade needed for one purpose causes another to break.

I don't see how that's any less of a problem if you do manual installations?

Robin2:
it would take a lot less effort on the part of the developers just to write decent instructions rather than writing programs to avoid the need for instructions

I don't think the developers of arduino-cli have any intention to avoid the need for instructions. This is a complex command line tool. It would be crazy to think it could ever be designed in a way that wouldn't require instructions.

It is true that the Library Manager and Boards Manager features of arduino-cli allow you to accomplish things by following more simple instructions. Let's look at some examples:

Instructions for manually installing the ArduinoJson library:

Instructions for installing the ArduinoJson library using arduino-cli:

  • arduino-cli lib update-index
  • arduino-cli lib install ArduinoJson

Let's say I want to update all my installed libraries to their newest release version. The instructions for doing this in arduino-cli:

arduino-cli lib update-index
arduino-cli lib upgrade

or to update all the installed cores:

arduino-cli core update-index
arduino-cli core upgrade

Now imagine the crazy instructions you'd have to write to tell someone how to do that manually!

Here are the instructions for installing the ESP8266 core manually:

Note that those instructions don't get into the process of installing git and python, which can be a bit challenging on Windows.

Here are the instructions for installing the ESP8266 core using arduino-cli:

If you don't already have an arduino-cli.yaml configuration file, run this command:

arduino-cli config init

and note the location of the file that will be shown by the command's output.

Open arduino-cli.yaml in a text editor.

Add these lines to the file and save:

board_manager:
  additional_urls:
    - http://arduino.esp8266.com/stable/package_esp8266com_index.json
arduino-cli core update-index
arduino-cli core install esp8266:esp8266

This one is still a little bit complex because of needing to edit the configuration file, but much less so than the manual alternative. I suspect they'll eventually add a way to update the configuration file using an arduino-cli command, as you can already do using the Arduino IDE CLI.

Robin2:
No shortage of evidence in this Thread to support the much greater value of writing the fishing instructions rather than handing out fish.

Arduino does provide some fairly good instructions for manual library installation:
https://www.arduino.cc/en/guide/libraries#toc5
My only complaint is those instructions assume the library is in the root of the folder that comes out of the .zip file, which is not always the case. I suspect that's the problem in the thread you linked.

pert:
I don't see how that's any less of a problem if you do manual installations?

I guess I wasn't as clear as I could have been.

It was in my mind that a person doing a manual installation would be more aware of what libraries, and versions of libraries s/he has already installed.

However my main issue is not to put things in a library manager at all - just make them part of each separate project.

I can never imagine wanting to do something like this

Let's say I want to update all my installed libraries to their newest release version.

To my mind it is a recipe for disaster.

Even if each of the library developers was diligent in providing a list of changes and advice about anything that might not be backwards compatible it would be a monumental task for a user to read all those change notices and make a sensible decision.

...R

My interest in CLI is for use in automated continuous integration testing. For this purpose, it makes sense to always test against the newest release versions of the library dependencies.

pert:
My interest in CLI is for use in automated continuous integration testing. For this purpose, it makes sense to always test against the newest release versions of the library dependencies.

That makes sense. But I suspect it is a specialised niche compared to the needs of the 99% of Arduino users.

...R

99.99% of Arduino users are going to be using an IDE solely and will never touch the CLI. Of those who do use the CLI directly, I'm certain a high percentage will be using it for automated testing.

The sort of person who prefers to manually use a command line interface to a GUI, or use the command line interface in their own application is quite rare. The intersection between that sort of person and the sort of person who uses Arduino is minuscule. But the CLI interface is the only option for someone doing automated testing so 100% of them are going to be using one. It's true that automated testing is not a very widespread practice among Arduino users, but it's getting more and more common every day. Helping this is the availability of high quality free CI services like Travis CI and the new GitHub Actions, which make it much easier to set up.

I have no quibble with what you say.

The Arduino system is Open Source so anyone can extend it or otherwise develop it in any direction that takes their fancy - and more power to them.

However what would concern me is if the core Arduino developers are spending time on a niche product at the expense of work that would benefit the 99.99%.

And, of course, I am concerned that the command-line-system that I use every day will be abandoned in the future in favour of a more complex system that offers no advantages for me.

...R

Robin2:
However what would concern me is if the core Arduino developers are spending time on a niche product at the expense of work that would benefit the 99.99%.

You need to understand that arduino-cli is going to be used to provide the core functionality of every IDE (both official and 3rd party). So practically every Arduino user (excepting Platformio users, since Platformio likely won't switch to arduino-cli) is going to be using arduino-cli. They just won't be using the command line interface and won't even know it's there.

It is true that we already had a working IDE and that this move of extracting all the non-GUI functionality from the IDE's Java code and moving it to dedicated external tools is likely driven by the Arduino Web Editor project, but it's really the right thing to do from a long term perspective. Arduino inherited this approach from the Wiring IDE (which even had parts of the hardware core system hardcoded into the Java code), then just continued to build on it over the years. I'm no expert in the IDE's Java code, but I've heard many times that it is very difficult to work with, so it is challenging for contributors to get involved with its development and there are even parts that the Arduino developers don't like to touch unless it's absolutely necessary.

Robin2:
I am concerned that the command-line-system that I use every day will be abandoned in the future in favour of a more complex system that offers no advantages for me.

I can't make any promises, but I would suspect that the Arduino IDE's CLI interface will be left the same as it has always been. That should be quite easy to do and the alternative would be to cause a ton of breakage. Every installation of the Arduino IDE will contain arduino-cli, so if the user wants to use the arduino-cli CLI, they can just use that tool directly. If they want to use the classic Arduino IDE, then they can do that through the IDE's executable, just like always.