Mathieu
A library in Arduino speak is not a library in C/C++ speak.
Only the Arduino core is treated like a real C/C++ library in the Arduino IDE and as such does not have the drawback of increasing your binary output with unused code.
There are some options you could take to fix your problem.
Change your code so the linker/compiler can do a better job. (requires good C/C++ knowledge)
Use a environment that allows you to use C/C++ libraries (requires knowledge of a C/C++ IDE )
Use defines to remove code. (requires more documentation and understanding of the library)
Split your library in several smaller libraries and only include the libraries needed in your sketch.
If you have many optional files the last is probably the most obvious choice
Best regards
Jantje
Note that even if you use C/C++ libraries you still have to seriously think about memory management.