Creating own libraries versus keeping code in main project cons and pros...

Hi all!
I have thought a long time what the consequenses are if i try to write code in an structured way also in my Arduino projects. Being a programmer for many years makes you know the power from different languages and different way to structure you code. Also the backdraws regarding memory usage and performance.

Sometimes classes works fine sometime a more straight forward way is better.
Sometimes a high maintenance factor is good.
Sometimes reusing variables due to memory cost is good.

WRONG POST
Look below...

Sometimes a high maintenance factor is good.

Cite a single example, please.

This isn't really a programming issue. It belongs somewhere else - like Bar Sport.

Chances are, if you're developing an application to run on an arduino, it's just going to be you and your soldering iron for company. Work the way you feel comfortable.

Personally, If I suddenly see the oportunity to simplify the code by creating a separate class, I'll just do it right there on the page, then test it in issolation. Once I'm happy it's behaving, I'll split it away into a library and get back to my project.

A word of warning, however. If you start defining classes, define prototypes for the classes and any functions that use them at the top of the page. Otherwise you may run into some misleading error messages when attempting to compile.

Hi all!
I have thought a long time what the consequenses are if i try to write code in an structured way also in my Arduino projects. Being a programmer for many years makes you know the power from different languages and different way to structure you code. Also the backdraws regarding memory usage and performance issues are there.

Sometimes classes works fine sometime a more straight forward way is better.
Sometimes a high maintenance factor is good.
Sometimes reusing variables due to memory cost is good.

The aspects are many...

In my Arduino project it would drive me crazy if i have 32 k of source code in my main project to handle instead of splitting in up into logical/physical units.

So what im doing is trying to design DC motor, encoder and a loopbackController classes/Libs where the contoller class is supposed to interact with motor and encoder class. By doing so I move a large amount of code avay from my sight in the project im working with.
Being a human thats nice...

What about being a machine?

My Questions are, by moving code to an external lib:

  • Will it affect performance?
  • Will it use more memory in ROM (Compile time)?
  • Will it use more memory in RAM?
  • Will it work to have dependencies i many levels?

BR

Svante

Thanks!

The first post was more of a misstake so its doubled... But i read your advice :slight_smile:

Thanks!

Please do not cross-post. This wastes time and resources as people attempt to answer your question on multiple threads.

Threads merged.

  • Moderator

You can edit posts, you know.

Maintenance isnt progamming... Sure :wink: Ive heard that... in a lot of context PaulS can you explain what you mean with this?

This isn't a programming question in the accepted meaning of the term on this forum. Programming questions are where you post a program (code) and ask a question about it.

I'll move this to Project Guidance.

PaulS:

Sometimes a high maintenance factor is good.

Cite a single example, please.

This isn’t really a programming issue. It belongs somewhere else - like Bar Sport.

So code qualiy → good mantenance det finns inte?

My Questions are, by moving code to an external lib:

  • Will it affect performance?
  • Will it use more memory in ROM (Compile time)?
  • Will it use more memory in RAM?
  • Will it work to have dependencies i many levels?
  • Will it affect performance?

No.

  • Will it use more memory in ROM (Compile time)?

No.

  • Will it use more memory in RAM?

No, I don't see why it would.

  • Will it work to have dependencies i many levels?

What do you mean?


Most of my larger projects use libraries. In one case I was communicating with a motor driver, so I developed a library for that. Then another library for serial comms via RS485. Then we use the Serial library. I see no objection to using libraries.

An alternative for a largish project is to add more tabs in the project and add .cpp and .h files as appropriate.

SvanteJ:
My Questions are, by moving code to an external lib:

  • Will it affect performance?
  • Will it use more memory in ROM (Compile time)?
  • Will it use more memory in RAM?
  • Will it work to have dependencies i many levels?

Write libraries if you intend on encapsulating complete functionality: links GPS, DHT11, GLCD, etc.
Otherwise, write functions and drop them into a separate .H tab in Arduini GUI.

LIBRARIES are linked in so really are functions. Classes have overhead, but also utility. Non-used flash memory is nonproductive but then the specific project may not require 100% of flash! SRAM is precious, so pass pointers to functions andbe kind to the "stack".

But, a microcontroller is not an enterprise server with an 80-90% utilization goal. It is a fixed $3 piece of sand. If your project is worthwhile and uses only 20% of thee uC resources, there is no need to be concerned. Write decent functions and if you find that a set of functions are always required for a particular need, these are candidates for a library.

Ray

First thing is if we are being pedantic, Arduino does not really use
libraries (ok the core code is a library) but all the others like LiquidCrystal, Wire, etc...
are not really libraries in the true sense. i.e. the code in the "library" is not compiled
into objects that are archived into an archive .a file that is linked against.
The IDE does not follow a traditional embedded build methodology.
It compiles the "library" modules to create .o files and then links in
the .o files rather than create a true library for it and linking against the library.
For this discussion, it doesn't really matter, since that is the way the IDE works.


I will definitely disagree with Nick about whether how code is structured
consumes more/less resources.

My Questions are, by moving code to an external lib:

  • Will it affect performance?
  • Will it use more memory in ROM (Compile time)?
  • Will it use more memory in RAM?
  • Will it work to have dependencies i many levels?

There are certain types of compiler and linker optimizations that can be done
depending on how the code is written, how the data is used & referenced, and how/where the source code lives.
Some of the optimizations can definitely affect performance, Flash, and RAM usage.

This can be particularly true for the AVR, which is 8 bit, Harvard Architecture, and
doesn't have the internal registers to handle pointers very well.

The compiler can be very smart when optimizing and the bigger the visibility the compiler
has, the deeper and better the optimizations can be.

For example if a variable is declared global and also static so that it is only known by
the compilation unit the compiler can do certain types of optimizations based on how
the code in that module uses it. When it is not static, the compiler can't do certain things
including eliminating data storage (RAM) for a variable that is never modified.

Other examples of things that are likely to be used when separating out functionality
into layers or libraries:

  • it takes less code and is faster to update a global variable vs a variable that
    is a member of class object - even if they are same type.

  • using pointers are costly on AVR when updating data members.
    i.e.
    foo.bar = 1;
    will generate way less code than
    foo->bar = 1;
    particularly if foo is a global data structure.

  • If you use virtual functions in your classes for clean layering and expandability
    you will pay the price in slight overhead in code size and performance vs having
    direct function calls.

  • When making the code available to others, having your own Arduino "complications" installation.
    When everything is part of the sketchbook directory, a single tar ball or zip image can be used to
    give the user everything they need and there is no having to install the additional "libraries".

This is not to say that layering and using an Arduino "library" is a bad thing.
Clean layering and module separation/libraries often helps the code in the project be much more maintainable
and re-usable in other projects.

Clean layering and module separation can also make it easier to track your changes with
your source control tool. (Surely you are using one like git, svn etc...?)
So you can easily see and track changes since the changes will be specific the modules
within the library vs having a single large file or grouping of files.

Having your code in an Arduino "library" can allow you to have multiple sketches
that use it which is often very useful for testing.

For the most part, I'd say if cycles and ram space are not super critical go
down a more organized path, which is likely to be a cleaner separation
into multiple modules and and potentially create an Arduino "library".
I say potentially since there is not much difference between a good module
separation and an Arduino "library" as far as a single sketch is concerned.
The benefit of an Arduino "library" is if you want to share the "library" code
across multiple sketches.

The best way to get a handle on code impact it is to starting looking at the
code generated by the compiler.
If you are on a *nix machine I posted a patch a while back for how to automatically create listing files
every time you build the code with the 1.5x IDE.
Sorry if you are using Windows, while it can also be done on Windows it is not as easy.

--- bill

bperrybap:
I will definitely disagree with Nick about whether how code is structured
consumes more/less resources.
...
The compiler can be very smart when optimizing and the bigger the visibility the compiler
has, the deeper and better the optimizations can be.

I acknowledge that the more the compiler sees the better, however the question was "Creating own libraries versus keeping code in main project". The question was not about whether to cram everything into a single compilation unit. "Own libraries" implies to me that the OP was thinking about migrating some .cpp and .h files from his project into a library, and would this be a bad thing?

I did a test of the Global Match example that comes with my Regexp library (itself quite a large piece of code).

Compiling in the "normal" way (ie. pulling in the library) I got:

Sketch uses 5,098 bytes (15%) of program storage space. Maximum is 32,256 bytes.
Global variables use 355 bytes (17%) of dynamic memory, leaving 1,693 bytes for local variables. Maximum is 2,048 bytes.

Copying and pasting the library code into the project (as a separate .h and .cpp file) gave exactly the same code size (and RAM size).

Then I copied and pasted the .cpp code into the main sketch, and recompiled (so we now have a single compilation unit). It still gave exactly the same object and RAM size.

Now you are welcome to make up a test case demonstrating that there is a difference by not using a library, but my point is that the difference, if any, won't be large.

And I have to say that if you are planning to have one monstrous file, then your project will be tedious to maintain.

Thanks for all good answers.
Just to make things clear.
I want this modularization for two reasons.
Maintenance & reuse. But it should not cost to much in the other end. I e ROM/RAM.
My first project is to build a loop back controlled dc motor system with a simple homemade encoder disc.
So my thought is to create four libs to start with encoder, dc motor, metrics and on top of these three the controller which interacts with these three and can be called from a sketch. Where you set speed behavior etc.

SvanteJ:
So my thought is to create four libs to start with encoder, dc motor, metrics and on top of these three the controller which interacts with these three and can be called from a sketch. Where you set speed behavior etc.

This makes me think there will be a lot of housekeeping code (four sets of it) that would be unnecessary if the code was just in functions - perhaps in a second .ino file

Certainly it will take more programmer time and I suspect it won't offer any compensaing benefits.

...R

Writing code in libs .cpp isn't that much harder than doing it in a sketch. Changing the content in a method is the same and the lib are probably suitable for other c++ editors. By breaking up the code in separate units it can be reused in other projects later. I can easily test it unit by unit. And for me it even helps to see things In different layers. F ex if I want to change the hardware I only have to care for rewriting libs on the level near the hardware. And all code will be kept in a version system so if I set this up right it will be easy to configure the environment. And also by writing things in libs I can use as many instances as I want in my code. If I have 2 dc motors I simply create 2 instances from the same lib.

We will se later what I think when I have tried it for a while...

Robin2:

SvanteJ:
So my thought is to create four libs to start with encoder, dc motor, metrics and on top of these three the controller which interacts with these three and can be called from a sketch. Where you set speed behavior etc.

This makes me think there will be a lot of housekeeping code (four sets of it) that would be unnecessary if the code was just in functions - perhaps in a second .ino file

Certainly it will take more programmer time and I suspect it won’t offer any compensating benefits.

…R

My thoughts, too.
I would suggest rearranging the existing code into something like the below, then making the final decision on a single library based upon what can be reused verbatim.

Files.jpg

SvanteJ:
By breaking up the code in separate units it can be reused in other projects later. I can easily test it unit by unit. And for me it even helps to see things In different layers.

I agree with this. But for me the separate units are functions or, perhaps a separate .ino file containing a few related functions.

I like the servo library because it has a verly limited scope and it works.

But I have read lots of posts here, and have experienced it myself, where more complex libraries don't work perfectly or are limited in some inconvenient way or have no documentation so I don't know what the limitations are. Then I run the risk of spending more time trying to "fix" the library than if I had ignored it and written my own code.

Some of these issues may be irrelevant with libraries written for the programmers own use - but not all of them. For example, what happens if a future project needs more functionality - then you have to go back and learn the workings of the library again which is almost certainly more trouble than adding a new function or modifying an existing one. And I just don't see the value in writing an all-bells-and-whistles library (to ensure it meets future needs) when all you need is a small amount of functionality.

Unfortunately comprehensive documentation and open-source code are almost mutually exclusive.

...R

SvanteJ:
<…>
And also by writing things in libs I can use as many instances as I want in my code. If I have 2 dc motors I simply create 2 instances from the same lib.

We will se later what I think when I have tried it for a while…

I just spent 3 months with the Cypress PSoC and Cypress disables the C++ option in their development environment (can be turned back on) and expressly states that development should be done in C for efficiency and stability. I thought it strange as I have worked with Arduino for over 3 years and I use C++ there; however, they have an interesting perspective.

Also, this professor of Computer Science has a similar take:
http://nrqm.ca/mechatronics-lab-guide/lab-guide-embedded-memory-management/

Frankly, this reminds me of a religious discussion and arguments never have winners. I say, do what the heck you want to do and in a year, report back to us on your success or failure. We all know that C++ libraries work with Arduino but some of us know that libraries can be a thorny issue.

If you are doing Arduino as a hobby and not doing paid-work, I see little value in constructing libraries just because you can. However, you articulated your thinking well; but, I have a difficult time agreeing that 4 separate libraries are warranted.

I risultati possono variare … batterie non incluse .

Ray

mrburnette:
I just spent 3 months with the Cypress PSoC and Cypress disables the C++ option in their development environment (can be turned back on) and expressly states that development should be done in C for efficiency and stability. I thought it strange as I have worked with Arduino for over 3 years and I use C++ there; however, they have an interesting perspective.

Yes, well if you don't mind reinventing the wheel all the time, especially with linked lists.

These days I would never go back from STL in more serious projects (not so much on the Arduino) because their lists, maps and vectors (in particular) along with the string class (lower-case "s") means that someone has "got right" all the gotchas in linked lists, like adding to the head, removing from the tail, etc.

And modern compilers do a pretty damn good job of turning them into fast efficient code.