obra
October 19, 2020, 4:53am
1
I'm clearly doing something wrong, but I'm having a heck of a time figuring out how arduino-cli wants me to escape strings containing " marks in values passed to --build-properties.
A slightly cut down example:
$ export LOCAL_CFLAGS='"-I/tmp/x" "-I/tmp/y"'
$ arduino-cli compile --fqbn my:valid:fqbn --build-properties "compiler.cpp.extra_flags=$LOCAL_CFLAGS" Sketch.ino
Gets me:
Error: invalid argument "compiler.cpp.extra_flags= "-I/tmp/x" "-I/tmp/y"" for "--build-properties" flag: parse error on line 1, column 26: bare " in non-quoted-field
I've been through just about every permutation I can think of, but can't figure out the correct escaping to be able to pass a string containing a double-quote in as a value for --build-properties.
A search of github issues for arduino-cli doesn't seem to turn up anyone else who's run into this, which leads me to suggest that I'm just missing something really obvious.
Anyone have an idea of what I'm doing wrong?
Thanks!
pert
October 19, 2020, 5:46am
2
It has been reported as an off-topic comment on this issue:
opened 03:04PM - 20 Jul 20 UTC
type: enhancement
topic: documentation
criticality: high
topic: build-process
### Describe the request
Establish two sets of standard "extra flags" build p… roperties:
- one set for use by platform developer
- one set for use by the user
These should be clearly documented as being dedicated to the exclusive use by either the platform developer or the user.
The properties proposed here are only arbitrary properties, without any special use by the build system, so no changes would required in the build system. It is only a matter of setting an example for 3rd party platform developers by adding these additional properties to the `platform.txt` files of the official platforms with comments clearly explaining their intent.
Ideally, the properties would be documented in the [Arduino platform specification](https://arduino.github.io/arduino-cli/latest/platform-specification/) as well (https://github.com/arduino/arduino-cli/issues/985).
### Describe the current behavior
Arduino has established a convention of providing "extra flags" properties in [`platform.txt`](https://arduino.github.io/arduino-cli/latest/platform-specification/#platformtxt). These properties are intended to allow the compilation process to be customized by the addition of arbitrary flags in the compilation commands.
For example:
https://github.com/arduino/ArduinoCore-avr/blob/1.8.6/platform.txt#L39-L49
```text
# This can be overridden in boards.txt
build.extra_flags=
# These can be overridden in platform.local.txt
compiler.c.extra_flags=
compiler.c.elf.extra_flags=
compiler.S.extra_flags=
compiler.cpp.extra_flags=
compiler.ar.extra_flags=
compiler.objcopy.eep.extra_flags=
compiler.elf2hex.extra_flags=
```
These properties are referenced in the appropriate compilation recipes in `platform.txt` ([example](https://github.com/arduino/ArduinoCore-avr/blob/1.8.6/platform.txt#L58)).
The empty definitions in `platform.txt` ensure that if the extra flags are not defined elsewhere, the compilation recipes referencing them will still work. Those default empty definitions can be overrriden from any of the other platform configuration files:
- [`boards.txt`](https://arduino.github.io/arduino-cli/latest/platform-specification/#boardstxt)
- [`boards.local.txt`](https://arduino.github.io/arduino-cli/latest/platform-specification/#boardslocaltxt)
- [`platform.local.txt`](https://arduino.github.io/arduino-cli/latest/platform-specification/#platformlocaltxt)
- [`global platform.txt`](https://arduino.github.io/arduino-cli/latest/platform-specification/#global-platformtxt)
or from the command line:
- [`arduino-cli compile --build-property`](https://arduino.github.io/arduino-cli/latest/commands/arduino-cli_compile/#options)
- [`arduino --pref`](https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc#options)
---
From the comments in the official `platform.txt` files, it seems that `build.extra_flags` is intended to be for the use of the boards platform developer, while the other "extra flags" properties are intended to be for the use of the user (because a platform developer has no need for `platform.local.txt`).
A common use case of these properties for the user would be to add `-D` flags (e.g., https://github.com/arduino/arduino-cli/issues/210#issuecomment-493892258, https://github.com/arduino/Arduino/issues/6344), but there are other uses.
However, platform developers sometimes use these properties in their `boards.txt` file ([example](https://github.com/MCUdude/MightyCore/blob/v2.0.5/avr/boards.txt#L103-L111)). This presents the problem that the user risks inadvertently overriding the flags defined by the boards platform if they use the "extra flags" properties to modify the compilation command ([example](https://github.com/arduino/arduino-cli/issues/2041)).
### Arduino CLI version
N/A
### Operating system
All
### Operating system version
Any
### Additional context
#### Re: Possible Harmful Effect on APIs
On a certain level, this does provide the user the global macro setting capability that has been often requested, and often rejected:
- https://github.com/arduino/arduino-builder/issues/15
- https://github.com/arduino/arduino-builder/pull/29
- https://github.com/arduino/arduino-builder/pull/282
- https://github.com/arduino/arduino-builder/issues/283
- https://github.com/arduino/arduino-builder/issues/354
- https://github.com/arduino/arduino-ide/issues/1837
- https://github.com/arduino/Arduino/issues/421
- https://github.com/arduino/Arduino/issues/2547
- https://github.com/arduino/Arduino/issues/3635
- https://github.com/arduino/Arduino/pull/3717
- https://github.com/arduino/Arduino/pull/3757
- https://github.com/arduino/Arduino/issues/4528
- https://github.com/arduino/Arduino/issues/5084
- https://github.com/arduino/Arduino/issues/5390
- https://github.com/arduino/Arduino/issues/7189
- https://github.com/arduino/Arduino/issues/11108
- https://github.com/arduino/arduino-cli/issues/159
- https://github.com/arduino/arduino-cli/issues/210
- https://github.com/arduino/arduino-cli/pull/1117
- https://github.com/arduino/arduino-cli/pull/1517
- https://github.com/arduino/tooling-rfcs/issues/9
- https://github.com/arduino/tooling-rfcs/issues/22
However, I think this proposal strikes the right balance of:
- Allowing advanced users to do advanced things
- Not exposing standard users to additional complexity that makes the Arduino development software less beginner friendly
- Being very easy for Arduino to implement
The concern has been that allowing users to set global macros will result in library developers unnecessarily using these as part of the API, making that API less beginner friendly (https://github.com/arduino/arduino-builder/issues/15#issuecomment-145558252). However, if the user is required to use a command line interface or create a specially formatted configuration file in a difficult to access location, the average user won't be interested, and so it will only make sense for a library developer to use this feature for advanced, rarely used features of the library (e.g., enable debug output for troubleshooting of the library). We have already had this "extra flags" capability [since 2014](https://github.com/arduino/ArduinoCore-avr/commit/2b9f022eaad0d107b1516c22ac02bdac4977223b), yet I haven't seen it result in any significant incorporation of user set global macros into the user-level API of Arduino libraries.
### Issue checklist
- [X] I searched for previous requests in [the issue tracker](https://github.com/arduino/arduino-cli/issues?q=)
- [X] I verified the feature was still missing when using the latest [nightly build](https://arduino.github.io/arduino-cli/dev/installation/#nightly-builds)
- [X] My request contains all necessary details
I requested they create a dedicated issue report about it, but they never did it. I did create an item in the Arduino Tooling Team's "to do" list so it wouldn't be forgotten by us. However, it is nice to have a public report in the issue tracker as a dedicated place to collect all information related to the issue, so you are welcome to create one.
I have also spent some time messing around with various permutations and also have not been able to find any workaround.
One thing I found confusing with the report from @bxparks was this statement:
Note that the Arduino IDE's --pref flag handles the parsing correctly, so the CLI's --build-properties is not a replacement for the IDE's --pref:
This could be construed as indicating that arduino --pref supported quotes in. However, if you read closely, it's actually saying that arduino --pref supports commas, which arduino-cli --build-properties does not. But arduino --prefalso doesn't support quotes.
obra
October 19, 2020, 5:59am
3
I'm happy to create a fresh issue against arduino-cli.
Most of the Google hits I see for the error message are actually about go's csv parser, and the solutions seem to involve enabling a 'LazyQuote' feature in the parser, but having never dug inside arduino-cli, I have no idea if it's the same codepath.
If @bxparks were talking about arduino-builder, their comment might make sense. Because arduino-builder's -pref is "dumb" and does those quotes just fine
obra
October 19, 2020, 6:07am
4
