I'm 1 day into Arduino, 35 years into writing C, and I have a very basic question about the structure of the .ino file. I've read the arduino-cli documentation about the sketch (folder) and the permissive subfolders and how they are treated, but I haven't found a reference for what is permissible in the .ino file itself.
I surmise it is sourced as a translation unit and the void setup (void) and void loop (void) functions are then called in main() (wherever that hides), but other than writing all code so setup() initializes everything, #define XYZ ... providing literals, and then loop() being the program loop - are there any other "features" or "limitations" the .ino structure provides? Any other magic incantations that can be used?
Postscript:
Kudos for the arduino-cli burn-bootloader command. I acquired my two Nanos from my son who graduated with an ESET degree from A&M, and the bootloader on one of the nano boards had met an unfortunate demise while driving one motor in a robotics project. The burn-bootloader command fixed it right up after downloading, building and loading the ArduinoISP on the working nano. Pretty slick!
It is defined in the main.cpp file in the core of the board you are compiling for. You can see the file from the core of the "Arduino AVR Boards" platform that adds support to Arduino IDE for the classic Nano board here:
All .ino and .pde files in the sketch folder (shown in the Arduino IDE as tabs with no extension) are concatenated together, starting with the file that matches the folder name followed by the others in alphabetical order.
This is something you might have already learned from reading the Arduino Sketch Specification:
So this means you are free to break your code up into multiple .ino files. This can facilitate navigating a large sketch codebase, as you can jump to a given subset of the code by selecting the tab of that .ino file in Arduino IDE.
You can do the same by adding .cpp, .h, .c, etc. files to the sketch, but that is different in that the source files will be a separate translation unit, whereas the multiple .ino files will all be a single file (and therefore a single translation unit) when presented to the compiler. Of course there are benefits to separating the codebase into multiple translation units. However, this makes things more complicated for someone who is not already familiar with those concepts. Likewise, the function prototype generation is only done for code in .ino files, so if you find that feature to be convenient then that is another advantage for using multiple .ino files to break up a large codebase (and conversely if you don't find prototype generation to be useful, then that is a benefit of using .cpp files instead).
The .cpp filename extension is then added to the resulting file.
Here you can see why the code in the .ino file is in a dedicated translation unit.
If not already present, #include <Arduino.h> is added to the sketch. This header file (found in the core folder for the currently selected board) includes all the definitions needed for the standard Arduino core.
(should say "declarations" instead of "definitions")
So this is why you can reference the Arduino core functions and macros in your sketch code even though there isn't a visible sign of how the declarations are made available. Likewise, if you prefer to make that explicit, you are free to add the #include directive. Furthermore, if for some reason you don't want that, you can do something like this:
#if false
#include <Arduino.h>
#endif
The sketch preprocessor will recognize that you have explicitly provided #include directive for Arduino.h, and so will refrain from automatically adding the #include directive, but since you have disabled the directive via the #if false conditional, the declarations will not be provided to the .ino file.
Prototypes are generated for all function definitions in .ino/.pde files that don't already have prototypes.
So you can reference functions defined later in the .ino file, even without explicitly adding a function prototype (AKA "forward declaration") for the function. Likewise, if you prefer to explicitly provide the prototypes in your code (or if you encounter a situation where automatic prototype generation is either not supported, or is not working correctly), you are free to do that.
#line directives are added to make warning or error messages reflect the original sketch layout.
This is self-explanatory and not terribly interesting, but perhaps worth noting if you are interested in the details of how sketch preprocessing works.
Thank you! That hit the nail on the head. I had read the part about the multiple .ino files and the src and data subdirectories, I was really just curious if there were any gotchas, or additional features. I'll go through the #pre-processing link and see what I can uncover.
Thank you again for you detailed and salient answer.
There are. The preprocessor (in the past known as the Arduino builder) is not perfect and might place prototypes in the wrong position in the .ino file. Some examples
The C++ compiler used for AVRs does not support all of C++: it does not have most of the stuff normally in libc++, nor from the C++ STL.
(If you're a C programmer, that probably won't bother you to much...)
Certain C functions also have limited functionality (for example, printf()/etc doesn't support the full range of format specifiers, normally does not include floating point support, and "stdio" is not normally initialized or connected to anything. Then it has some extra features to deal with the embedded environment (interrupts, for instance.) The features of avr-libc are described here: https://www.nongnu.org/avr-libc/user-manual/
"double" on an AVR is 32bits (the same as float), so it has limited precision and range compared to what you might expect.
If you end up using an ARM or ESP processor, you'll have a more complete and standard-ish C++ environment.
Also some standard C/C++ functions are replaced by macros which can result in unexpected behaviour.
For example, abs() is replaced by #define abs(x) ((x)>0?(x):-(x)) which is vulnerable to side effects say in the possible but unlikely case of abs(++i)
You might like to note that you don't actually need to use the setup()/loop() sketch format if you don't want to
If you put a main() function in your code then the compiler will not use the board specific one described previously. The disadvantage of doing that is that you have to include code to initialise the Arduino hardware and #include the Arduino specific functions if you want to use them
That is one other curiosity I had. I've been picking though the sources, and will give it a try. The only other embedded RISC chip I build for is the MilkV-Duo, the rest are Arm, either TI or Pi. It was quite refreshing to have the Nano be able to upload over USB directly rather than having to chain two boards together to use as a SWD probe. (pico, and I see there is a Nano RP2040 connect -- that will be nice)
Macro provided UB, gotta love it. That is definitely something to lookout for, but the text-replacement for efficiency with small boards makes sense. I've gotten to where I try and avoid pre/post-increment as anything other than a loop increment. Otherwise, an additional line of code with a += or -= helps readability (especially for older eyes), and is a small price to pay for certainty regardless of whether the value goes into a macro or function call.
That is what I was looking for as well. As with any new platform, finding and collecting all the documentation is always a fun scavenger hunt. It's a volume thing no matter what board you pick up. The Arduino docs are good, but it is still a bit of a hunt. You learn anything new the same way you eat a whale ... one byte at a time...
Now that would have been a surprise if I wasn't looking for it. I haven't run into that yet, but I'm not much beyond the proverbial microcontroller "Hello World!" yet. But along with the pair of Nanos, I also got several bags full of IC chips (of course with their legs crumpled against the base from being tossed into a backpack). I'd turned my Nano "Hello World" blink into a fun little LM324 and 741 tester to both ease into the AVR library and to test and toss any ICs that didn't make it. (pins 6-9 to the non-inverting inputs, a voltage divider providing 2.5V to the inverting inputs, and then sequential blink with a 2-200 ms delay between toggling each opamp on/off while the LEDs chase each other :)
Thank you all for your help and guidance. These are exactly the type of gotchas I would stumble on, that if aware of before are a mere speed-bump, but if ignorant of, I may chase for quite some time before arriving at the cause. (we've all been there) So for that -- a double Thank You to all.
Fortunately, that I did know. It's not uncommon on smaller boards and, at least the C standard leaves it to the implementation (I'd have to check the C++ standard, but can't see it being any different).
One area where I'm still scratching my head is the arduino-cli compile command. Is the --build-property build.extra_flags=-DCHGDIR the proper way to pass preprocessor defines on the command-line? In the example I want to pass a change-direction define to control conditional inclusion of code during compile. Looking at --build-property output, that seems to work adding to the recipe.c.o.pattern=... property??
It is probably already clear to you, but for the benefit of others with similar question who will read this topic in the future I think it is worth noting that some of the things the participants in this discussion have mentioned are not in any way specific to the Arduino framework.
It is important for people to understand that, beyond the Arduino sketch preprocessing described in the documentation I referenced previously, the "Arduino language" is just C++ (as implemented by the industry standard GCC toolchain of the platform, e.g., avr-gcc) and the vast amount of information about C++ and the variant of GCC in use is completely applicable.
Not strictly. If you look at the platform configuration:
The comment indicates the build.extra_flags property is intended for internal use by the platform, not as a dedicated interface for free use by the user to inject compiler arguments.
And sure enough if we look in boards.txt we see that build.extra_flags is indeed used by some board definitions:
and so on...
When you set a value of a property via the --build-property flag, you are overwriting the value of that property set by the platform configuration. The ability to do that can be very useful to an advanced user, but if you don't take into consideration how the property is used by the platform, then overwriting it might cause problems.
The platform does provide a set of properties that are dedicated for the exclusive use by the user to inject arguments into the compilation commands:
So these properties will be the most safe to use with the --build-property flag.
Of course you are welcome to set any platform property you like via the --build-property flag, but you should understand that this is a very powerful and advanced capability that is only intended to be used by those who have a good understanding of the internal details of how the specific platform in use is configured.
You should be able to use the set of properties I mentioned above with any of the official boards platforms maintained by the Arduino company (e.g., "Arduino AVR boards"/arduino:avr), but when it comes to 3rd party platforms there are no guarantees as providing these properties is only an informal convention established by Arduino, but not enforced by the platform framework. Unfortunately, for absolutely no reason, some 3rd party platform developers made use of these properties in the platform configuration (even though they could just have easily used an arbitrary property for that purpose and left the extra_flags property free for use by the user), meaning that if you set the value via the --build-property flag you will overwrite the value the platform configuration set for it, and which is intended to be part of the compilation command. It is also possible that some 3rd party platforms don't reference these properties in the compilation command "patterns", in which case setting the value of the property via the --build-property flag would have absolutely effect on the compilation command.
If you are interested in the subject, there is some related discussion here:
Please only comment on the GitHub issue thread if you have new technical information that will assist with the resolution. General discussion and support requests are always welcome here on Arduino Forum.
For this specific use case, the --build-property flag is the correct feature to use. It is only the specific platform property you chose to use with the flag that is questionable.
However, it is worth mentioning that some boards definitions provide "custom board options" which allow the user to adjust the configuration of the definition. If the configuration you need is provided by a custom board option, then you should use that as this is a public interface of the board definition the platform developer has explicitly provided to the user (meaning it is less prone to breakage or unexpected behavior than setting platform properties directly, which the average 3rd party platform developer likely doesn't ever consider the user might do). You can set the custom board options via the --board-options flag, or if you prefer via the --fqbn flag value (reference).
Ah, thank you. I had indeed looked at the language flags:
but my intent was to "Append" instead of "Override", so I looked for an ....extra_flags= that wasn't already populated for the nano -- without understanding the implications for other boards.
Which I suspect is what the github topic is for -- and I will stop by and comment. A simple way to "Add" or "Append" without a complete need to "Copy existing + Append" to add a preprocessor flag would be quite helpful.
Even for small test files, like the LED example above, I may have several defines that conditionally include code to isolate specific areas of code I'm digesting, or to simply allow for changing the behavior of the code (like changing the direction [sequence] in which the LM324 opamps are called from the code to reverse the directions of LEDs.
I had thought that arduino-cli may just "Pass what's Leftover" as part of the command-line allowing it to be treated like a typical call to gcc, etc.. But I see the challenge with the build environment combining C/C++/Assembly compilation in a single command.
Question - is this were interacting though the gRPC daemon provides a finer-grained way of interacting with the build process? (I haven't fully digested how to interact with it other than looking over the various properties it exposes). If that allows the ability to grab the current, e.g. compiler.c.extra_flags= values and then be able to append the wanted define - that would make that process easier. (I'll read further there)
Thanks again for the education on where my use of build.extra_flags= could run into problems (and for all of your other sage advice)
That is exactly what the flags I pointed out are used for:
The platform's definitions of the properties are empty, so the arguments you provide via the --build-property flag and these will simply be added to the compilation command, without removing anything from the command.
No. It only provides an option for a tighter integration of Arduino CLI into other applications (e.g., Arduino IDE) than would be possible using the command line interface.
You can get the value using the command line by running the arduino-cli compile command with the --show-properties flag.
As I already explained, you don't need to worry about "appending" as long as you use the properties that are provided for this purpose.
Got it! That is where I had a bit of confusion. I didn't know whether the, e.g. compiler.c.extra_flags= could be used elsewhere internally like the build.extra_flags= property. If the former is for dedicated use of the user, then we are good to go. Thank you again.
They could be. There is nothing in the platform framework that prevents the platform developer from doing something annoying like that. As I already explained:
For example, you can see it done here for absolutely no good reason in the popular "MiniCore" boards platform:
However, as I already explained:
So if you are using the arduino:avr platform (i.e., arduino-cli compile --fqbn arduino:avr:nano) then you don't need to worry about that because the Arduino developers were smart enough to actually leave the properties free for exclusive use by the user as intended.
Here is an interesting caveat that I didn't expect. Setting the define with compiler.c.extra_flags=-DCHGDIR isn't sufficient to include the define as part of the build. Using the compile string1:
$ acli compile --fqbn arduino:avr:nano sketches/lm324-led-chase --warnings all
--build-property compiler.c.extra_flags=-DCHGDIR
This doesn't alter the resulting recipe.preproc.macros property used in the build. Presumably due to the final build being done with avr-g++. Setting also the property compile.cpp.extra_flags does make the define visible to the final part of the build/link, e.g.
Arduino ultimately using avg-g++ for the build, this does make sense. Another week or so feeling like I'm drinking from a fire-hose digesting all the new Arduino process details, and I may have a solid beginner's understanding for using the Nano.
Thanks again for helping fill in the gaps.
footnotes:
(1) acli is aliased to arduino-cli (saves hunting and pecking for keys)
Someone should point out that modifying the compile/etc commands is NOT “the arduino way”, and is not commonly done to achieve the sort of results that you are describing. Better to use a config.h that your sketch includes. (Note that the extra flags we’ve been talking about will be applied to the entire arduino core and any libraries as well as your sketch code, which could result in surprises.) (and I’m not sure the builder will get dependencies right if all you change is a compile flag.)
Yeah, that makes it harder to compile different option-versions of a sketch without actually editing a file, but that’s what the ide is for…
Thank you for explaining the application to the entire arduino core. I like to be able to build manually, from the command line when learning a build system. I long ago learned that if you understand how your code is actually built, you can then tell and IDE how you want it done instead of just hoping the IDE config gets it right.
I want to make sure I understand, if using, e.g. compile.c.extra_flags isn't what one is supposed to do, then what are those build properties for? Yes, I could include a config.h or, for that matter, just manually edit the primary sketch file each time I want to isolate particular code, but that seems to ignore the intended purpose of being able to specify build properties, options, etc. on the command line.
The build system works very well, but not being able to do something as simple as pass a #define does limit its flexibility in a significant way and the way code must be structured. Don't get me wrong, I'm not throwing rocks at the system, it's all part of learning "the arduino way" (and how to tweak it to do what I need it to do :)
Hopefully the github issue to add an easy way to do this will get some movement and provide a solution that doesn't have potential unwanted side effects. It's been open for a while, so maybe it's nearing the point of implementation.
Please only comment on the GitHub issue thread if you have new technical information that will assist with the resolution. General discussion and support requests are always welcome here on Arduino Forum.