Can library include a configuration file located in the sketch folder?

User library including a configuration file located in the sketch folder was discussed in https://github.com/arduino/Arduino/issues/1734 and developed in https://github.com/GDV0/Arduino/commit/e868241db1a7ba533214f21e66d31884d0ae311e Does Arduino have this feature, or is there some other way to for library to include a configuration file located in the sketch folder?

I want to do this because I have a user library that has an array samples[SAMPLE_COUNT]. Currently SAMPLE_COUNT is defined in the library's config file. But it would be nice to place the config file in the sketch folder so that users can tweak SAMPLE_COUNT to their needs.

Also read_pins_t is defined in the library's config file, as one of:

typedef uint8_t read_pins_t;
typedef uint16_t read_pins_t;
typedef uint32_t read_pins_t;

The config file needs to be included at pre-compile time for typedef to work. read_pins_t is used in several library files. Would be nice if user could define typedef read_pins_t in the sketch folder.

Thank you.

Move the full library files in your sketch folder and then you can play around with your local copy (not exactly what you ask for but achieves same goal - of course make sure that's the one being used!)

J-M-L: Move the full library files in your sketch folder

Do you mean "Move the full library files to your sketch folder"?

Yes in, to, into, inside, below,...

Myapp
    Myapp.ino
    lib.cpp
    lib.h
    libconfig.h

If you use double quotes for your #include rather than angle brackets it will look locally

Thanks J-M-L. That will work, although every library updated becomes error-prone busy work. Especially if the user has several sketches using the library.

You could use the absolute path to the configuration file in the library include but that’s not a very good idea because it’s not portable.

By modifying the boards.txt file for the selected board(s) it can be done as explained here:

This is a better solution but the problem is it will need to be redone on every update of the IDE and/or hardware platform version. It’s also not very friendly to sharing the library since you will either need to make a custom hardware package to go with the library or the library user will need to make the modifications to their boards.txt.

The solution I’ve found to allow preprocessor macro configuration of the library from the sketch is to move all the library’s code that relies on the macro to the library header file and then define the macro in the sketch before the include of the library. However, I’m not sure if that will work for your specific application.

The way I’ve written my libraries where I want the user to be able to configure a buffer size from the sketch is to pass a parameter to the library and use new to dynamically allocate the buffer array. Now it’s well known that dynamic memory allocation is a no-no in embedded applications but there are a few ways it can be used safely. If you are only doing the allocation once there’s no chance of memory fragmentation.

Thank you pert. I will try new to dynamically allocate the sample[] array.

Another option might be to send the name of the config file to the library as part of the initialization process. Because of the unfortunate limitations of the Arduino IDE it may need to be a full path name for the config file.

...R

Another way to do this would be for the class to be developed in the right way (wishful thinking I know) with dynamic parameters like through template (for example)

As an example if you read the work from Benoît Blanchon on his cool JSON set of tools, and look at the memory model

Arduino JSON uses a preallocated memory pool to store the object tree, this is done by the StaticJsonBuffer class. Before using any function of the library you need to create a StaticJsonBuffer. Then you can use this instance to create arrays and objects, or parse a JSON string. StaticJsonBuffer has a template parameter that determines its capacity. For example, the following line create a StaticJsonBuffer with a capacity of 200 bytes:

  StaticJsonBuffer<[color=green][b]200[/b][/color]> jsonBuffer;

Robin2: Another option might be to send the name of the config file to the library as part of the initialization process. Because of the unfortunate limitations of the Arduino IDE it may need to be a full path name for the config file.

...R

That sounds like a good solution. Please point me to an example syntax.

wolfv: That sounds like a good solution. Please point me to an example syntax.

Sorry I haven't ever done it myself so I am making this up as I go along.

Perhaps a better way is to send the library the address if the string (char array) in which the name of the file is contained. This will give some idea of the code in the main program.

char configName[] = "/aa/bb/cc/config.txt";
Lib myLibrary(&configName);

I reckon I could get this working but I am not the right person to explain how to do it.

...R

Robin2,

I just realized, the config file needs to be included at pre-compile time for typedef to work:

typedef uint32_t read_pins_t;

As you mentioned, your method is during initialization (run-time).

J-M-L:
Another way to do this would be for the class to be developed in the right way (wishful thinking I know) with dynamic parameters like through template (for example)

As an example if you read the work from Benoît Blanchon on his cool JSON set of tools, and look at the memory model

  StaticJsonBuffer<[color=green][b]200[/b][/color]> jsonBuffer;

I can use “new” to dynamically allocate the sample array, as pert suggested.
But “typedef uint32_t read_pins_t;” needs to be included at pre-compile time.
Can dynamic parameters like through template do that?

pert:
The solution I’ve found to allow preprocessor macro configuration of the library from the sketch is to move all the library’s code that relies on the macro to the library header file and then define the macro in the sketch before the include of the library. However, I’m not sure if that will work for your specific application.

Templates.

template<size_t N>
class whatever
{
  public:
  int16_t samples[N];
};
#include <whatever.h>

whatever<15> accel_filter;

EDIT: Oh hey, look, someone already brought it up.

Thanks Jiggy-Ninja.

The typedef read_pins_t defined in the config file are used inside many implementation files, Row.cpp for example:

void Row::send(const uint8_t keyCount, const read_pins_t debouncedChanged)
{
    read_pins_t isFallingEdge;                  //bits, 1 means falling edge
    read_pins_t isRisingEdge;                   //bits, 1 means rising edge
    read_pins_t readPosition;                   //bits, active bit is 1
    ...
}

Is there a disadvantage to moving all such functions from the implementation files to the header files?

That template approach is pretty cool but the advantage of my dynamic allocation system is you can do things like this:

byte* samples;

void setup() {
  Serial.begin(9600);
  while (true) {
    Serial.print(F("Please enter buffer size: "));
    while (!Serial.available()) {}
    const int bufferSize = Serial.parseInt();
    Serial.println(bufferSize);
    Serial.print(F("Buffer allocation "));
    if (begin(bufferSize) == false) {
      Serial.println(F("failed"));
    }
    else {
      Serial.println(F("successful"));
      break;
    }
  }
}

void loop() {}

boolean begin(const int sampleCount) {
  samples = new byte[sampleCount];
  return samples != NULL;  //return success of the allocation
}

I also think it's more user friendly because any Arduino user is familiar with the function argument syntax, whereas the template syntax is probably unfamiliar to beginners.

pert:
That template approach is pretty cool but the advantage of my dynamic allocation system is you can do things like this:

byte* samples;

void setup() {
  Serial.begin(9600);
  while (true) {
    Serial.print(F("Please enter buffer size: "));
    while (!Serial.available()) {}
    const int bufferSize = Serial.parseInt();
    Serial.println(bufferSize);
    Serial.print(F("Buffer allocation "));
    if (begin(bufferSize) == false) {
      Serial.println(F(“failed”));
    }
    else {
      Serial.println(F(“successful”));
      break;
    }
  }
}

void loop() {}

boolean begin(const int sampleCount) {
  samples = new byte[sampleCount];
  return samples != NULL;  //return success of the allocation
}

What is the advantage of that?

I also think it’s more user friendly because any Arduino user is familiar with the function argument syntax, whereas the template syntax is probably unfamiliar to beginners.

Templates are awesome. They let me write a wrapping_integer class that can work with any integral type, including pointers. I also subclassed it to a more specialized wrapping_pointer class that defines the * and operators to make it more useful with a pointer type.

They’re also what lets the official EEPROM library read and write any data type with a single function name instead of having to juggle 6,000 individual functions for all the data types.

This can also be extended to PROGMEM. Here is an example PMEMRef class that uses the pgm functions behind the scenes and lets you read PROGMEM variables like normal variables:

namespace mrd
{
template<class T>
struct PMEMRef
{
	typedef T value_type;
	typedef const T* pointer_type;
	PMEMRef( pointer_type ptr ) : _ptr(ptr) {}
	
	operator value_type() const;
private:
	pointer_type _ptr;
}; // struct PMEMRef

template<class T>
PMEMRef<T>::operator typename PMEMRef<T>::value_type() const
{
	T return_value;
	/*PGM_P pgm_ptr = reinterpret_cast<PGM_P>(_ptr);
	uint8_t* loc_ptr = reinterpret_cast<uint8_t*>(&return_value);
	for( uint8_t i=0; i<sizeof(T); ++i )
	*(loc_ptr++) = pgm_read_byte(pgm_ptr++);*/
	memcpy_P( &return_value, _ptr, sizeof(T) );
	return return_value;
}
template<>
PMEMRef<uint8_t>::operator typename PMEMRef<uint8_t>::value_type() const
{
	return pgm_read_byte(_ptr);
}
template<>
PMEMRef< int8_t>::operator typename PMEMRef< int8_t>::value_type() const
{
	return pgm_read_byte(_ptr);
}
#include <PMEM.h>

const uint16_t test PROGMEM = 0xBEEF;

PMEMRef<uint16_t> progmem = &test;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  Serial.println(progmem, HEX);
}

void loop() {
  // put your main code here, to run repeatedly:

}

Jiggy <3 templates.

Jiggy-Ninja: What is the advantage of that?

Allowing the user to set the buffer at run time instead of compile time makes the library much more flexible. Is there a way to do that using the template system?

Jiggy-Ninja: Templates are awesome. They let me write a wrapping_integer class that can work with any integral type, including pointers. I also subclassed it to a more specialized wrapping_pointer class that defines the * and [] operators to make it more useful with a pointer type.

They're also what lets the official EEPROM library read and write any data type with a single function name instead of having to juggle 6,000 individual functions for all the data types.

No argument from me on that but those examples are a different application for templates, which I also find very useful.

Is there a way to do that using the template system?

See my post above #8

J-M-L: See my post above #8

I don't see how that technique can be used to set the buffer size at runtime.