Compiler option defines overriding ESP32 and not compiling

concurrents · May 11, 2022, 10:31pm

I have the same code base for ESP-32 and M5-Stick-C-Plus. But there are a couple #ifdef around specific sections (such as M5.Lcd.print).

I have a global Defines.h that specifies which configuration is being compiled.
I now want to specify the option at the compile level (but in a makefile).

I want to define ** ESP_M5_SMART_CLICKER_CONFIGURATION** which is specific to the M5.

compileM5:
        /opt/homebrew/bin/arduino-cli compile "build.extra_flags=\"-DESP_M5_SMART_CLICKER_CONFIGURATION\"" --fqbn esp32:esp32:m5stick-c | tee output

But after trying the various build.extra_flag syntax, when compiling for the M5 the ESP32 doesn't get defined, and a compile error shows up in library file: M5StickCPlus.h

#error “This library only supports boards with ESP32 processor.”

Example Defines.h file included in all my code. Currently I comment out the option I want.

//turn on or off which configuration is desired..
//#define ESP_M5_SMART_CLICKER_CONFIGURATION
//#define ESP_32_FEEDER

If instead this is modified in the file included, the compile works fine.

So somehow the build.extra_flags is wiping out other defines. One comment in my research showed that these flags should only be used by the platform builders (or things like I'm seeing show up).

With GitHub, I wanted a single install without having to modify that Defines.h file for each different build.

Any help would be appreciated.
thanks.
scott

in0 · May 11, 2022, 11:26pm

Hi @concurrents. You can see the reason here:

https://github.com/espressif/arduino-esp32/blob/2.0.3/platform.txt#L136

build.extra_flags=-DESP32 -DCORE_DEBUG_LEVEL={build.code_debug} {build.loop_core} {build.event_core} {build.defines} {build.extra_flags.{build.mcu}}

When you define a custom build.extra_flags property via an arduino-cli compile --build-property flag, you overwrite the one defined in the platform.

Flags, no, but as for the build.extra_flags property that is also the impression I get about its purpose, based on this comment:

https://github.com/espressif/arduino-esp32/blob/2.0.3/platform.txt#L122

# This can be overriden in boards.txt

boards.txt is the domain of the platform author, so if the purpose of the flag is to make board-specific build command customizations defined in the individual boards definitions, then that property is not intended to be used by the user and you are risking just such a usage collision if you decide to modify it as a user.

There is another set of "extra_flags" properties, and the comment on those indicates to me they are intended for the user:

https://github.com/espressif/arduino-esp32/blob/2.0.3/platform.txt#L112-L120

# These can be overridden in platform.local.txt
compiler.c.extra_flags=
compiler.c.elf.extra_flags=
compiler.S.extra_flags=
compiler.cpp.extra_flags=
compiler.ar.extra_flags=
compiler.objcopy.eep.extra_flags=
compiler.elf2hex.extra_flags=
compiler.libraries.ldflags=

The platform.local.txt file is intended to allow the user to modify a platform configuration without touching the primary files. The platform author would have no need to use that file since platform.txt is their domain. So properties intended for use in platform.local.txt must be reserved for the user. They are both more flexible and more inconvenient than the build.extra_flags property in that they are specific to the compilation commands for a given language (e.g., compiler.cpp.extra_flags is only used when compiling C++ and Arduino Language files), but you can always define multiple if you need to do something like set a define for C++ and C files.

Something you should be aware of is that some platform authors have been naughty and also used these properties internally (even though there is no need for that since they are able to define an unlimited number of such arbitrary properties for their own use. In those platforms, you will run into this same problem of a use collision when your custom definition overwrites the one from the platform. You are of course welcome to copy the definition from the platform and append your additional flags to that.

Arduino holds much of the blame for such encroachment on the user's properties because they have never documented these flags beyond the obscure and vague comments in the platform.txt file. There is discussion on this situation here:

github.com/arduino/arduino-cli

Provide dedicated sets of "extra flags" properties for platform developers and users

opened 03:04PM - 20 Jul 20 UTC

per1234

type: enhancement topic: documentation criticality: high topic: build-process

### Describe the request Establish two sets of standard "extra flags" build p…roperties: - one set for use by platform developer - one set for use by the user These should be clearly documented as being dedicated to the exclusive use by either the platform developer or the user. The properties proposed here are only arbitrary properties, without any special use by the build system, so no changes would required in the build system. It is only a matter of setting an example for 3rd party platform developers by adding these additional properties to the `platform.txt` files of the official platforms with comments clearly explaining their intent. Ideally, the properties would be documented in the [Arduino platform specification](https://arduino.github.io/arduino-cli/latest/platform-specification/) as well (https://github.com/arduino/arduino-cli/issues/985). ### Describe the current behavior Arduino has established a convention of providing "extra flags" properties in [`platform.txt`](https://arduino.github.io/arduino-cli/latest/platform-specification/#platformtxt). These properties are intended to allow the compilation process to be customized by the addition of arbitrary flags in the compilation commands. For example: https://github.com/arduino/ArduinoCore-avr/blob/1.8.6/platform.txt#L39-L49 ```text # This can be overridden in boards.txt build.extra_flags= # These can be overridden in platform.local.txt compiler.c.extra_flags= compiler.c.elf.extra_flags= compiler.S.extra_flags= compiler.cpp.extra_flags= compiler.ar.extra_flags= compiler.objcopy.eep.extra_flags= compiler.elf2hex.extra_flags= ``` These properties are referenced in the appropriate compilation recipes in `platform.txt` ([example](https://github.com/arduino/ArduinoCore-avr/blob/1.8.6/platform.txt#L58)). The empty definitions in `platform.txt` ensure that if the extra flags are not defined elsewhere, the compilation recipes referencing them will still work. Those default empty definitions can be overrriden from any of the other platform configuration files: - [`boards.txt`](https://arduino.github.io/arduino-cli/latest/platform-specification/#boardstxt) - [`boards.local.txt`](https://arduino.github.io/arduino-cli/latest/platform-specification/#boardslocaltxt) - [`platform.local.txt`](https://arduino.github.io/arduino-cli/latest/platform-specification/#platformlocaltxt) - [`global platform.txt`](https://arduino.github.io/arduino-cli/latest/platform-specification/#global-platformtxt) or from the command line: - [`arduino-cli compile --build-property`](https://arduino.github.io/arduino-cli/latest/commands/arduino-cli_compile/#options) - [`arduino --pref`](https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc#options) --- From the comments in the official `platform.txt` files, it seems that `build.extra_flags` is intended to be for the use of the boards platform developer, while the other "extra flags" properties are intended to be for the use of the user (because a platform developer has no need for `platform.local.txt`). A common use case of these properties for the user would be to add `-D` flags (e.g., https://github.com/arduino/arduino-cli/issues/210#issuecomment-493892258, https://github.com/arduino/Arduino/issues/6344), but there are other uses. However, platform developers sometimes use these properties in their `boards.txt` file ([example](https://github.com/MCUdude/MightyCore/blob/v2.0.5/avr/boards.txt#L103-L111)). This presents the problem that the user risks inadvertently overriding the flags defined by the boards platform if they use the "extra flags" properties to modify the compilation command ([example](https://github.com/arduino/arduino-cli/issues/2041)). ### Arduino CLI version N/A ### Operating system All ### Operating system version Any ### Additional context #### Re: Possible Harmful Effect on APIs On a certain level, this does provide the user the global macro setting capability that has been often requested, and often rejected: - https://github.com/arduino/arduino-builder/issues/15 - https://github.com/arduino/arduino-builder/pull/29 - https://github.com/arduino/arduino-builder/pull/282 - https://github.com/arduino/arduino-builder/issues/283 - https://github.com/arduino/arduino-builder/issues/354 - https://github.com/arduino/arduino-ide/issues/1837 - https://github.com/arduino/Arduino/issues/421 - https://github.com/arduino/Arduino/issues/2547 - https://github.com/arduino/Arduino/issues/3635 - https://github.com/arduino/Arduino/pull/3717 - https://github.com/arduino/Arduino/pull/3757 - https://github.com/arduino/Arduino/issues/4528 - https://github.com/arduino/Arduino/issues/5084 - https://github.com/arduino/Arduino/issues/5390 - https://github.com/arduino/Arduino/issues/7189 - https://github.com/arduino/Arduino/issues/11108 - https://github.com/arduino/arduino-cli/issues/159 - https://github.com/arduino/arduino-cli/issues/210 - https://github.com/arduino/arduino-cli/pull/1117 - https://github.com/arduino/arduino-cli/pull/1517 - https://github.com/arduino/tooling-rfcs/issues/9 - https://github.com/arduino/tooling-rfcs/issues/22 However, I think this proposal strikes the right balance of: - Allowing advanced users to do advanced things - Not exposing standard users to additional complexity that makes the Arduino development software less beginner friendly - Being very easy for Arduino to implement The concern has been that allowing users to set global macros will result in library developers unnecessarily using these as part of the API, making that API less beginner friendly (https://github.com/arduino/arduino-builder/issues/15#issuecomment-145558252). However, if the user is required to use a command line interface or create a specially formatted configuration file in a difficult to access location, the average user won't be interested, and so it will only make sense for a library developer to use this feature for advanced, rarely used features of the library (e.g., enable debug output for troubleshooting of the library). We have already had this "extra flags" capability [since 2014](https://github.com/arduino/ArduinoCore-avr/commit/2b9f022eaad0d107b1516c22ac02bdac4977223b), yet I haven't seen it result in any significant incorporation of user set global macros into the user-level API of Arduino libraries. ### Issue checklist - [X] I searched for previous requests in [the issue tracker](https://github.com/arduino/arduino-cli/issues?q=) - [X] I verified the feature was still missing when using the latest [nightly build](https://arduino.github.io/arduino-cli/dev/installation/#nightly-builds) - [X] My request contains all necessary details

concurrents · May 12, 2022, 12:05am

Thanks for the long reply. After a couple test options I found out that compiler.cpp.extra_flags was the winning ticket.

Now my makefile can compile both:

compile:
        /opt/homebrew/bin/arduino-cli compile --build-property compiler.cpp.extra_flags=-DESP_32_FEEDER --fqbn esp32:esp32:esp32 | tee output

compileM5:
        /opt/homebrew/bin/arduino-cli compile --build-property compiler.cpp.extra_flags=-DESP_M5_SMART_CLICKER_CONFIGURATION --fqbn esp32:esp32:m5stick-c | tee output

Next, do you know how to have GitHub run the arduino-cli compiler?

concurrents · May 12, 2022, 12:25am

One more question: How do I provide these -D commands to the Arduino SDK? As I switch back and forth often, changing platforms.txt isn't a good solution. Or even the preferences.txt file requires re-starting Ardunio.

thanks.

in0 · May 12, 2022, 2:48am

It is not clear to me what you mean by "GitHub". Please provide more details about what you are hoping to accomplish.

By "Arduino SDK", do you mean the Arduino IDE? If so, which version?

concurrents · May 12, 2022, 2:51am

Arduino SDK (the main graphic editor, download to boards, etc), version 1.8.19

As for GitHub, I can ask that in another thread. This is the GitHub that has continuous integration and can compile after files are checked in.

in0 · May 12, 2022, 3:04am

We call that the "Arduino IDE". It will avoid a lot of confusion if you use that term here instead of "Arduino SDK".

You can set build properties via the Arduino IDE 1.x command line using the --pref flag (like how you used --build-property with Arduino CLI):

github.com

arduino/Arduino/blob/master/build/shared/manpage.adoc

// Generate a manpage with: a2x -f manpage manpage.adoc
// or HTML with: a2x -f xhtml manpage.adoc
//
// This file uses {empty}:: in some places, to allow putting multiple
// paragraphs inside a single label list item. This is a bit ugly and
// non-semantic, but it seems this is the best way to do this. Asciidoc
// also supports putting a plus sign on a line by itself to join two
// paragraphs into a single list item. However, the indentation on the
// second paragraph makes that formatted with a fixed-size font.
// Removing the indentation completely makes the asciidoc source very
// unreadable. Also, for the --board option, there is a a paragraph,
// followed by a list, followed by another paragraph. The + approach can
// only put the latter paragraph into the inner list, not the outer
// one...

ARDUINO(1)
==========
:doctype: manpage

NAME

This file has been truncated. show original

OK, I'll answer over there. FYI, the thing you are talking about is called "GitHub Actions"

concurrents · May 12, 2022, 3:07am

The Arduino IDE provides a nice buttons to compile, upload, etc. When using the IDE I don't see a command line option, only the buttons.

So my question is how to provide those -D switches from the IDE.

thanks.

in0 · May 12, 2022, 3:10am

You can't provide arbitrary compilation command flags via the Arduino IDE GUI. You can only do it from the command line or the boards platform configuration files.

This was done intentionally by Arduino to encourage library authors to provide users with friendly interfaces rather than preprocessor-based messes.

concurrents · May 12, 2022, 3:20am

"friendly" ?? That's subjective.

Anyway, in my original question, I have the same code base for two distinct executables. Arduino setup the environment constraints (eg. src folder) which is fine.

But I want to be able to easily, and friendly without messes, compile between the two options using the Arduino IDE (not command line).

Are you saying that's not possible?

thanks,

in0 · May 12, 2022, 3:27am

Make a board definition with custom board options that define the -D flags you need. Then you can select which you want from the Tools menu in the Arduino IDE:

https://arduino.github.io/arduino-cli/latest/platform-specification/#custom-board-options

concurrents · May 12, 2022, 3:30am

I'll look into that. In that case my other developers would need those boards too.

I'll probably just stick with commenting out my define for the 2 cases.

thanks.

in0 · May 12, 2022, 3:35am

You can make it easy to install and update your custom boards platform by providing Boards Manager installation support:

https://arduino.github.io/arduino-cli/latest/package_index_json-specification/

You can reference all the resources from the stock ESP32 boards platform (which they can also install via Boards Manager), so your custom platform can consist of a boards.txt file alone.

Or if you are not interested in going the Boards Manager route, you can distribute a boards.local.txt file to them with instructions for them to drop it into the ESP32 platform installation folder.

concurrents · May 12, 2022, 3:36am

thanks, will look into it.

Juraj · May 12, 2022, 4:10am

Per, it doesn't work for esp32 and esp8266 core. problem is the tools folder. {it should be renamed to system). and the other problem is that tools are in tools (system) and not installed in the right tools folder.

github.com/esp8266/Arduino

esp8266 Arduino package can't be used as referenced package

opened 10:33AM - 16 Feb 19 UTC

JAndrassy

waiting for feedback

Arduino IDE 3rd party Hardware specification section ["Referencing another core,… variant or tool"](https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5-3rd-party-Hardware-specification#referencing-another-core-variant-or-tool) specifies that a boards package can be referred from custom hardware definitions or other boards packages. esp8266 package can't be used now as referred, because platform.txt file entries use variable {runtime.platform.path}, which points to the folder of selected board's boards.txt file. In case of a referring definition it is not the right folder to access sdk and tools. solved in PR https://github.com/esp8266/Arduino/pull/5770

github.com/arduino/arduino-cli

Uploader prefs miss {build.core.path}, {build.system.path} and {build.variant.path}

opened 04:00PM - 15 May 19 UTC

JAndrassy

topic: code type: imperfection

The keys `{build.core.path}`, `{build.system.path}` and `{build.variant.path}` a…re not evaluated for platform.txt tools recipes (`tools.xy.upload.pattern`). Then the tools recipes must use `runtime.platform.path`. But then there are problems to use the platform as referenced core, because `runtime.platform.path` is the referencing board platform path. For example in the SAMD package the `upload.pattern` contains `{runtime.platform.path}/variants/{build.variant}/{build.openocdscript}` instead of `{build.variant.path}/{build.openocdscript}`. A board in referencing board platform can use in board.txt an existing variant of the referenced core platform in `xy.build.variant` for compilation, but the upload tool will not find the script for the variant. I try to make the esp8266 package to be used as as referenced core. Now it uses `{runtime.platform.path}/tools` all over platform.txt and `{runtime.platform.path}` will evaluate to referencing board platform, not to referenced core platform. They could use `{build.core.path}/../../tools` or rename the folder to `system` and use `{build.system.path}` variable. It works for building, but it doesn't evaluate in `uploader.pattern`. I debugged the Arduino IDE 1 and found out that compiler recipes use prefs evaluated by arduino-builder, while Uploaders use the preferences map of the IDE. So it is not a simple fix. I think, the best way for fixing this would be to pass the prefs evaluated by `Compiler` over `SketchController` from build() to upload(). Same in class `processing.app.Base`.

in0 · May 15, 2022, 1:56am

Thanks @Juraj. I remember the ESP8266 issue, but didn't extrapolate that to ESP32 platform as well.

The boards.local.txt solution is unaffected by the problem.

As for the creation of a referencing platform, it seems that it will work fine if you copy the entire tools folder from the stock ESP32 platform to your own platform. Other than that, the only file needed is a boards.txt.

in0 · May 15, 2022, 1:57am

Have you found success with this endeavor @concurrents?

concurrents · May 15, 2022, 2:49am

Thanks for the query. Yes the GitHub continuous integration is what I'm looking for. I haven't worked it yet, but actually don't see how unless the arduino compiler can be placed at the GitHub server. That would be powerful.

Anyway, I did have good success with the makefile compiling various options as stated earlier. That at least lets me send of a couple compiles to make sure the code works with various define configurations.
thanks.

in0 · May 15, 2022, 3:27am

It can be done easily, and is already done by hundreds of Arduino project repositories.

GitHub actions provides you with a selection of virtual machines. You can run anything you like on those machines. "Actions" are pre-packaged reusable components you can build your workflow from, and so these are the most convenient way to use GitHub Actions, but you are also welcome to just run shell commands directly as you need or prefer. So essentially anything you can do in a headless manner on your local machine you can do on GitHub Actions as well.

I'll share some options in order of my own preference:

`arduino/compile-sketches` action

This action provides a simple yet flexible interface to:

Install sketch dependencies
Compile sketches
Collect compilation results data

This is the approach used in all of the official Arduino firmware repositories.

High level introduction:

Example of a workflow compiling the examples of the "Arduino_ConnectionHandler" library for a selection of boards including ESP32:

https://github.com/arduino-libraries/Arduino_ConnectionHandler/blob/master/.github/workflows/compile-examples.yml

`arduino/setup-arduino-cli` action

This action installs Arduino CLI in the GitHub Actions runner machine. It is then up to you to use Arduino CLI however you like in subsequent steps of the job.

High level introduction:

Workflow demonstrating its use:

https://github.com/arduino/arduino-cli-example/blob/master/.github/workflows/test.yaml

Install and use Arduino CLI directly from the runner shell

As I mentioned, you can execute commands in the runner shell just the same as you would on your machine. This approach will tend to result in a more complex workflow that takes more effort to maintain, especially if you are using it in multiple projects. But you have full control over the behavior.

Documentation here:

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun

And of course the installation and command line usage instructions for Arduino CLI are also relevant:

https://arduino.github.io/arduino-cli/latest/

Honorable mentions

I don't have personal experience with these, but they are popular alternatives created by talented members of the Arduino community so worth a mention:

`ArminJo/arduino-test-compile` action

Adafruit's build script

This is not an action, but a script Adafruit installs and runs via GitHub Actions workflows in their ~200 Arduino project repositories. It seems somewhat specific to their own requirements, but quite a few 3rd party projects are using it.

The origins of this project go all the way back to the days when Travis CI was the go to automation service. Adafruit was a pioneer in the use of Travis CI for automated testing of Arduino projects and their work served as a valuable reference even though I found the scripts did not meet my own specific needs.

concurrents · May 15, 2022, 3:28am

Well there you go.. thanks that's awesome.

I'll work on that soon.

thanks,

Topic		Replies	Views
Reference for .ino file structure? Beyond setup() and loop() Arduino Command Line Tools	26	445	September 11, 2025
Per-sketch way to set extra debug build properties (sketch project file?) Arduino Command Line Tools	0	36	January 19, 2026
[Arduino IDE] Feinheiten und Tricks Deutsch	97	24839	January 21, 2026
How to add compile options to Arduino IDE 1.8.13 General Discussion	6	15554	May 24, 2021
Using --build-property to set optimize and define symbols Arduino Command Line Tools	6	1226	October 30, 2024

arduino/compile-sketches action

arduino/setup-arduino-cli action

Install and use Arduino CLI directly from the runner shell

Honorable mentions

ArminJo/arduino-test-compile action

Adafruit's build script

Related topics

`arduino/compile-sketches` action

`arduino/setup-arduino-cli` action

`ArminJo/arduino-test-compile` action