How to save relevant library for the future

Repeatedly when I try to edit projects that I did, maybe some years ago, they no longer compile. I believe the reason is that the general newer library on my computer has changed. The errors I get has nothing to do with what I have written, but some deep down library routines. One possibility would be to store the entire Arduino software and libraries for each project I make, but that requires a lot of space.
Is it a way to store only the library parts that are pertinent for a specific project?

With the library manager of the IDE, you can access older library versions. So, when you create a project, simply embed the version of the library you've got loaded in a comment. Then, down the road, you can go fetch that library.
For libraries managed outside the IDE, I'd recommend saving the (usually) zip file from whatever source as part of your project structure.

Welcome to the forum

Your topic has been moved to the Project Guidance category of the forum as it does not relate directly to IDE 1.x

Would it really take much space to put the libraries for a project in the same folder as the sketch and use the local versions in the sketch ?

Alternatively, you could always copy the required functions from the library into your sketch

Of course, the proper way to do what you want would be to update your sketches to use the latest version of the libraries

1 Like

The problem is that when I use a standard function, that function will include other functions that again include other functions and so on. To collect all files used from the library is a near impossible task. Generally, because of all the conditional inclusions of library files, it can be really time consuming to find which definition is used for a variable, for instance. Other developing environment than Arduino sometimes has special functions for this. I have some times considered to make a program that does this, it is definitely possible, but time consuming.

Your best bet is, in my opinion, to create a portable install of IDE 1.x; one portable install for each project.

That way you have all relevant data (IDE, board packages, libraries and your sketch) In one place that you can easily backup.

Note:
IDE 2.x does not (yet) support portable installs.

2 Likes

My last experience is with a device that uses an e-ink display. It seems that the compiler finds errors in some underlying code to handle the display. There is no error messages with my calls to the functions, but it could be connected to altered definitions for the processor which is called by the display functions. So, maybe it would be enough to save the display library, as you propose. Maybe not. I only know that for me to try to debug the missing definitions in some files that is not directly part of my project, is comparable to rewrite the whole project.

This is a project I wrote and compiled successfully some years ago. On trying to compile it now, the first error I get is

... Arduino/libraries/timer1/var/pgmspace.h:89;53: error: variable '__c' must be const in order to be put into read-only section by means of '_attribute((progmem))' #define ATTR_PROGMEM attribute((progmem))

In this case the error seems to have occurred in the timer1 function, but I cannot debug it.

It is only one of several similar errors an it would require quite a lot of files to find the real culprit. So I have little hope of being able to fix this. My question was general. How can I avoid things like this in the future. The best way would be to have a command that made a complete library for the special project without files not used. Not having this, the next best could be to include all 3 part libraries in the project directory and hope that would be enough.

To me this sounds like a case of C++ rules being applied more tightly by either the IDE or the compiler. I remember seeing that happening when switching between MCU cores in the IDE. C++ can automatically correct/ignore a whole lot of minor issues, like a variable not being declared const when it should be.

To find out what is causing this problem (I don't know if I will be able to myself, others no doubt are better coders and know about such issues more intimately) you have to give information on basically your complete build environment: IDE version, Arduino core version, core settings (which board or MCU), the exact versions of the libraries used, your code, and probably some more things that I forgot about. Basically whatever is needed to reproduce this error.

This you describe is the main problem. You build a hardware, install Arduino, collect non standard libraries and write the code. After some time it works and you store everything you have done and think it is enough. But time goes, you get a new PC, install latest Arduino, an suddenly your program dont compile and the error messages make no sense. You have to start all over again. As you indicate there are lots of small settings you maybe never were aware of that also have to fit. So you have to be a real specialist to save a project so that it could be used later. I have for 20 years used other developing environments like Altera for instance, and never had problems like this. A good developing environment does things like that for you. Arduino, which is supposed to be low threshold, require much more detailed knowledge of the inner workings. That does not make sense.

For more than 30 years my job was to make HW and software using prosessors from various families. Microprocessors, signal prosessors, programmable devices. I had to use more than one type of hardware at the same time and was dependent on these interfaces to be logical and reliant. There was never a time to become wizzard for any of the environments. For my private projects Arduino is ingenious in that it has low cost interfaces and building blocks. It is easy to start up the first project. The problem is when the programs get more complicated, and the backward compatibily. I had hoped that some of the experts had seen this problem and made some hack to handle it like other environments have done. This not being the case it seems I must save the complete environment each time I have a finished product. Thanks anyway.

I was posting a general question. I know that I can solve the errors given enough time. The idea of asking a wizard I reserve for exeptional problems. I observe that this seems to be the preferred solution nowadays, but that is not the way to work for any kind of efficiency. And see what happened when I asked a general question to the Wizards. The error I submitted with timer1 was a very simple one. In my program I use that library in three lines only, just to start a timer that calls a routine each second. Then I include the timer1.h, and leave the rest to the library. That worked before, but now I get compilation errors in h-files in the timer1 library. Clearly not an error I did. I could fetch an old version of timer1, but since timer1 probably fetch other files from other libraries that is not given. The other errors are more complicated. Seems to me that the timer1 library has gone rotten. That is a serious problem. It should not happen. If you are all comfortable with this, fine. But if you want your project to last, you should take precautions.

man. 15. jan. 2024, 16:52 skrev Delta_G via Arduino Forum <notifications@arduino.discoursemail.com>:

I do it the wrong way, I put the libraries in the same folder as the sketch. If there are a lot of them I put a library folder in the sketch for just that sketch. As the libraries update those do not. If there is a memory leak it should show up in the first few months.

2 Likes

You seem to take this personally. There is no reason for that. For me this is a recurring problem. Programs that compiled some years ago, dont with new versions of Arduino. And generally, the errors shows up in library files. I was looking for someone with the same problem that has a solution. If you never have this problem, good for you. I am no expert in Arduino, and I understand that a free software with lots of 3 parties libraries can have problems like this.

man. 15. jan. 2024, 18:21 skrev Delta_G via Arduino Forum <notifications@arduino.discoursemail.com>:

InIf you really think you have the solution, that would be very interesting. When I have debugged these errors, it very fast gets difficult to trace. Say I use timer1 and copy that for later use. But timer1 must at least have information about the hardware used. That means it calls other files in the general library, and these files are selected based upon the hardware selected and maybe more conditions. In some of the environments I have used, I can point to a variable and jump to the file that defines it, the software taking care of all the conditional choises. How do you navigate like that in Arduino?

tir. 16. jan. 2024, 12:40 skrev Delta_G via Arduino Forum <notifications@arduino.discoursemail.com>:

I cannot agree to what you say about hardware. To set up a timer om a Mega 2560 if different from an ESP32. The interface on timer1 is identical for the two cases. Thus, the compiled must look up your board definition to select what to do. If you change board setting, you also select some of the library. I have 30 years with debugging software in a number of assembly languages, Algol, Fortran, Apl, VHDL, c,c++ , even using logic analyser... It's not fun anymore, just tedious work. Especially when somebody else wrote the code. My current project consists of 19 files plus library. Some thousands lines. Do you really want me to send it so you can debug it? I was looking for a way to avoid having to debug programs from scratch each time I need an old program. So far, no luck.

tir. 16. jan. 2024, 15:03 skrev Delta_G via Arduino Forum <notifications@arduino.discoursemail.com>:

Hi @janbording. If you don't mind using a command line interface, you might be interested in the "build profiles" feature of the official Arduino CLI tool:

https://arduino.github.io/arduino-cli/latest/sketch-project-file/#build-profiles

This allows you to define the versions of each of the platform (AKA "core") and library dependencies of a sketch in a metadata file and ensure those versions will be used when compiling and uploading it.

This leaves only the version of Arduino CLI in use as a variable in the build results. It would be reasonable to simply document which version of Arduino CLI the sketch was verified with. An alternative would be to just bundle Arduino CLI with the sketch (similar to the "portable IDE" approach some have suggested here). Unlike Arduino IDE, Arduino CLI is a lightweight single executable file, so bundling it with the sketch is less impactful.

Thank you. I'll look into it. Sometimes my problems could be solved if I could avoid a new compilation each time I want to program a board. Is there any possibility in Arduino to save .obj or .hex files or similar for direct download without compilation?

tir. 16. jan. 2024, 16:32 skrev ptillisch via Arduino Forum <notifications@arduino.discoursemail.com>:

Yes. I'll provide instructions:

If you are using Arduino IDE

  1. Select Sketch > Export Compiled Binary from the Arduino IDE menus.
    A compilation operation will start.
  2. Wait for the compilation to finish successfully.
  3. Select Sketch > Show Sketch Folder from the Arduino IDE menus.
    The folder of the sketch will open in your file manager.

The binary file will be in the sketch folder. If you are using Arduino IDE 1.x, it will be in the root of the sketch folder. If you are using Arduino IDE 2.x, it will be under the build subfolder.


You can not upload binary files via Arduino IDE. You can use the command line tool Arduino IDE uses for the upload.

Enable the Show verbose output during: ☐ upload option in the Arduino IDE preferences, perform any upload to the board, and then examine the contents of the "Output" panel at the bottom of the Arduino IDE window to learn the command required to upload a binary from the command line.


If you are using Arduino CLI

Compile the sketch using an arduino-cli compile --export-binaries command. The binary file will be found under the build subfolder of the sketch folder after the compilation finishes successfully.

You can later upload the binary file using an arduino-cli upload --input-dir or arduino-cli upload --input-file command:

https://arduino.github.io/arduino-cli/latest/commands/arduino-cli_upload/#options

Great! Thanks.

tir. 16. jan. 2024, 17:41 skrev ptillisch via Arduino Forum <notifications@arduino.discoursemail.com>: