Guidance/ Issues with using defines in .ino for use in libraries

Hi all!

A little background of myself: I’m a junior test engineer and I do C# at work (it’s my first job and I’ve been doing it since half a year), but I have no real C/C++ experience, only things I have done so far is some basic arduino stuff. So please go easy on me, I’m coming here for guidance.

The last month(s) I have been busy writing a library that contains a variance of LED classes that I use within my projects. Mainly as a C++ exercise, but also because I find myself reprogramming blinking leds, fading leds, sparkling leds over and over again.

My targeted devices at the moment are the Arduino and the ESP8266.

Since I’m planning to share it with the community soon, I’m trying to make it in a way that it is implemented in a good way.

for those who are already interested, it’s on github:

UtiLEDy

This to provide a little background in what I want to do.

It is already functional, however, I struggle to implement something, which I don’t know how to call it, so I’ll try to explain it by example.

Let’s say I have a sketch “ledsketch.ino”.
In this sketch I want to use a Fader, which is a LED that fades in/out.
For this I include a header file: “fader.h”, in which the class ‘Fader’ is located. There is also a “fader.cpp” file.
The Fader is derived from a class ‘GammaLED’ which is a gamma- corrected LED.
So GammaLED is the base class for the Fader.
The “gammaLED.h” file is included in the header file of “fader.h” and also has a source file “gammaLED.cpp”.

Inside of “gammaLED.h”, I do something like:

// gammaLED.h //

...

#ifdef BRIGHTNESS_RESOLUTION_12BIT
 #define BRIGHTNESS_TYPE uint16_t 
 #define BRIGHTNESS_TYPE_MAX 4095
#elif defined BRIGHTNESS_RESOLUTION_10BIT
 #define BRIGHTNESS_TYPE uint16_t 
 #define BRIGHTNESS_TYPE_MAX 1023
#elif defined BRIGHTNESS_RESOLUTION_8BIT
 #define BRIGHTNESS_TYPE uint8_t 
 #define BRIGHTNESS_TYPE_MAX 255
#else // Default: 12bit
 #define BRIGHTNESS_TYPE uint16_t 
 #define BRIGHTNESS_TYPE_MAX 4095
#endif


#include "GammaTables.h"

...

After that, I include a header file “GammaTables.h” within “gammaLED.h”, which picks a gamma correction table accordingly.

// GammaTables.h //

...

#ifdef BRIGHTNESS_RESOLUTION_12BIT
    const BRIGHTNESS_TYPE gammaCorrectionLookupTable[] = { ... } // Contains 4095 unsigned ints. 
#elif defined BRIGHTNESS_RESOLUTION_10BIT
    const BRIGHTNESS_TYPE gammaCorrectionLookupTable[] = { ... } // Contains 1023 unsigned ints. 
#elif defined BRIGHTNESS_RESOLUTION_8BIT
    const BRIGHTNESS_TYPE gammaCorrectionLookupTable[] = { ... } // Contains 255 unsigned shorts. 
#else // Default: 12bit
    const BRIGHTNESS_TYPE gammaCorrectionLookupTable[] = { ... } // Contains 4095 unsigned shorts.  
#endif

...

I have done it this way, so the unnecessary gamma tables would not be included during compilation, and so would not end up in the code that is uploaded to the Arduino/ESP.
For the Arduino, the 12 bit lookuptable would almost (or does?) completely occupy the memory, which is not good.

Now, for the issue/question…

If I put the define for the resolution, let’s say for 10 bit, in the “GammaLED.h” in front of both the above pieces of code, then it works as it should. But I would like to have the define in the sketch itself:

// ledsketch.ino //

#define BRIGHTNESS_RESOLUTION_10BIT
#include<fader.h>

Fader led1 = Fader(...);

Fader led2 = Fader(...);

Fader led3 = Fader(...);


void setup() { ... }

void loop() { ... }

...

But then something weird happens:

If I calculate the count of the array “gammaCorrectionLookupTable” from within the skech, this returns 1023 items, which is correct.
But the Fader class itself, uses the default (12 bit) variant of the “gammaCorrectionLookupTable”, since its brightness values go up to 4095.

I think the classes are compiled without the “#define BRIGHTNESS_RESOLUTION_10BIT”, but the header files are compiled with the define.

Anyone who can help me overcome this?

Also, the issue aside… is this a good way to handle this kind of compiling pre-selection?
If not, what could be a better solution and then please also provide me with a sub code example?

Any help would be appreciated!

Thanks <3

You have to understand how .ino files are compiles. If you have more than one .ino file in the project, they are all combined, along with some function definitions into (basically) a .cpp file that is then compiled. Other .cpp files are compiled separately and then all the compiled objects are linked together.

If you want to do what you suggest, you will have to put all your code in the .h file so it is included into the current .ino file

I believe the software I2C library does this....

blh64:
You have to understand how .ino files are compiles. If you have more than one .ino file in the project, they are all combined, along with some function definitions into (basically) a .cpp file that is then compiled. Other .cpp files are compiled separately and then all the compiled objects are linked together.

If you want to do what you suggest, you will have to put all your code in the .h file so it is included into the current .ino file

I believe the software I2C library does this…

Thanks for the reply!

The SoftI2C library indeed does something similar, it uses the #defines from the sketch for use in the library. However it does only consist out of a header .h file. Not a .cpp source file. The strange thing that happens with me is that the header files are compiled correctly, but the source files are not. It seems that the .h file uses the defines from the sketch, but the source files do not.

At the start of the source file, I do include the header also:

// GammaLED.cpp

#include <GammaLED.h>

...

Could this be an issue?

The strange thing that happens with me is that the header files are compiled correctly

Header files are NOT compiled. Only source files are.

PaulS:
Header files are NOT compiled. Only source files are.

Hmm, did not know that.. it certainly gives a new perspective to the matter.

Looking at the previous post of blh64:

...you will have to put all your code in the .h file so it is included into the current .ino file.

I see that I didn't read it well enough and that I should not use .cpp files if I want to achieve what is described in my main post.

It feels a bit like avoiding the problem, rather than solving it.

I'm thinking then what would be the best solution here.

  • Doing what I want, defines in the sketch and facing that I have to put all the code in a header.
  • Keeping the defines in the main GammaLED.h file and having to change the library every time I make a new project with a different resolution.

I'd go for the first, but I would like to have a more experienced standpoint for this matter..

The only other option I've seen is to modify the hardware definition of the boards you compile the library for so that it can include files from the sketch folder (http://forum.arduino.cc/index.php?topic=329709.msg2279367#msg2279367). However, since your goal is to share this library that's not a very good solution since everyone who wanted to use your code would need to go through this complex process.

Keep in mind that you don't necessarily need to move all the code to the .h, only the code that is affected by the configuration defines.

PaulS:
Header files are NOT compiled. Only source files are.

Then how does a function in a .h file work at all?

My understanding is that when it is #included, it is written into the source file which is then pre-processed and compiled.

Then how does a function in a .h file work at all?

My understanding is that when it is #included, it is written into the source file which is then pre-processed and compiled.

Exactly. It is the source file that contains the #include statement that is compiled, not the header file by itself.

As a quick fix, I have made a "Config.h" file that is included in each .cpp file before any other header files are included. Seems to work without issues, so I have to think if I keep doing it this way OR place all my code in the header files.

pert:
The only other option I've seen is to modify the hardware definition of the boards you compile the library for so that it can include files from the sketch folder (http://forum.arduino.cc/index.php?topic=329709.msg2279367#msg2279367). However, since your goal is to share this library that's not a very good solution since everyone who wanted to use your code would need to go through this complex process.

Keep in mind that you don't necessarily need to move all the code to the .h, only the code that is affected by the configuration defines.

The process described in the link seems to be very complex for something simple. So I think I will not go down that path. Thanks for the suggestion though!

Regarding to the .h : I could be selective and only place the code in the header file, but that would only exclude a few functions and so I'd then prefer to put everything inside header files.

I'm still thinking what would be the best solution for the described problem in the main post, but now I have an extra option:

  • Place the defines in the header file of the base GammaLED class.;
  • Use only header files.
  • Place the defines in a dedicated 'config.h' and include it in every .cpp and .h file.

Thoughts on this and/or alternatives are very much appreciated!

Header files are NOT compiled. Only source files are.

Then how does a function in a .h file work at all?

The header file is include in EACH source file where it is #include'ed, individually.

That means if you put in your sketch:

#define MYLIB_MYOPTION 200
#include "mylib.h"

whatever is in mylib.h that uses MYLIB_MYOPTION will have the "200" definition, but mylib.h doesn't get compiled separately, so when your mylib.cpp library is compiled, it does NOT know about the 200.

Look at this scenario:

// SomeThing.h
void SomeThing();

// SomeThing.cpp
#include "SomeThing.h"

void SomeThing()
{
#ifdef SOMETHING
  Serial.println("something");
#else
  Serial.println("another thing");
#endif
}

// Sketch.ino
#define SOMETHING
#include "SomeThing.h"

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  SomeThing();
  delay(1000);
}

No matter what you do in the sketch, you will never get "something" written to Serial. The reason being that "SomeThing.cpp" does not include "Sketch.ino" and hence it cannot "see" the "SOMETHING" define :slight_smile:

You need to put the define in a place where "SomeThing.cpp" can see it (eg. in "SomeThing.h").

I have a friend who got sick of the non-standard Arduino IDE so he got Atmel Studio that follows the rules. He won’t go back.

As far as your constant data tables on Arduino, you might be able to fit them in flash by using PROGMEM.

But when you write Arduino board, there is a wide range of Arduino & compatible boards. A board with ATmega 1284P on it has 16K RAM, 4K EEPROM, 128K flash, 2 serial ports and 40 pins. That chip is just over $5 ea, a mini-1284 board runs < $30.

You have options you may not know.

GoForSmoke:
non-standard Arduino IDE so he got Atmel Studio that follows the rules.

In the context of this thread, which "rules" are you talking about?

Act like a standard C/C++ IDE/compiler rules about included files was my issue.
For him, Nishant is a forum member and he may have a list of reasons that I don’t know all of.

Coding examples here, you just avoid what once you did not. Now that I can run it, I’m not sure if Studio is still free.

GoForSmoke:
Act like a standard C/C++ IDE/compiler rules about included files was my issue.

Huh? This makes no sense to me. The Arduino IDE uses gcc and therefore any/all the “standard compiler rules about included files” are no different whether the Arduino IDE is used vs using custom hand written makefiles.

The only goofyness (and granted it is a big IMO) in the Arduino build process is all the automatic prototyping and sketch .ino to .cpp massaging that goes on along with the unfortunate way the IDE handles Arduino “libraries” which are not libraries at all.
And then the IDE does not install its binary executables properly on the host, including the compiler, so the built in compiler default include paths for the gcc system include files is not correct which then requires using extra -I options to locate system header files.

but none of that has anything to do rules about included files.
The rules for #include foo.h and #include <foo.h> and the -I options work as documented regardless of what tools sit on top of the compiler.


What I have seen is that many people, particularly here on the Arduino forum, simply do not understand how compilers and linkers work to generate images, and often are very ignorant as to some of the big differences in using C/C++ in a full implementation like on a PC host running a full blown OS vs in an embedded environment.

In this case, the OP is unaware how source code modules are compiled and linked and the concepts related to separate compilation units. It has nothing to do with the language being used nor the tool set.
He has created separate compilation units but is attempting to do things that requires them to be one.
This is not a tool or language issue but rather an educational issue.
What is desired is simply not possible given the way that the code being used has been structured.
The fact that the Arduino IDE is being used to build the image is irrelevant.
The same would be true if using make with hand built makefiles or even custom scripts.

Now, is there a way to provide what the OP is wanting. Yes there is. But it requires changing the way the code is structured to ensure that all the code that needs to see a macro defined in the .ino is compiled along with the .ino file vs as a separate compilation unit.

— bill

bperrybap:
Huh? This makes no sense to me. The Arduino IDE uses gcc and therefore any/all the "standard compiler rules about included files" are no different whether the Arduino IDE is used vs using custom hand written makefiles.

Sure but the whole bit about .ino files is definitely non-standard and doesn't play by the same rules. The Arduino program is turning the .ino into proper source files to be compiled. And it is that process, not the compilation of the source files but the conversion of .ino to .cpp, that caused things not to work the same in a .ino as you would expect.

I moved to Eclipse with my own makefiles over this.

I was writing a library when what worked for years before on other platforms C++ did not. So I found a way around it and moved on.

Delta_G:
And it is that process, not the compilation of the source files but the conversion of .ino to .cpp, that caused things not to work the same in a .ino as you would expect.

The following things are all that happens during the conversion of .ino to .cpp:

  • Add #include <Arduino.h>.
  • Concatenate .ino files.
  • Generate prototypes for all functions in .ino files that don’t already have them.
  • Add #line directives so that compiler messages will point to the expected places in the sketch.

I don’t see how any of that would cause things not to work in the context of this thread.

Although not specific to the .ino to .cpp conversion, the other “Arduino magic” is arduino-builder does a dependency resolution process where it determines which paths must be specified to the compiler via -I flags according to the #include directives in your sketch and libraries. I also don’t see how would be relevant.

I suppose maybe you two are talking about the Arduino IDE’s lack of support for easily changing compiler flags, which could be used to either configure the library via -D or allowing the library to #include a configuration .h file from the sketch folder via -I?