Abominable documentation standards in the community - how to navigate?

I have often encountered this problem. There is a library that promises to do something that I need, would really be good to have. But there is almost no documentation on it. I can't even install the library in some cases.

I just had this problem with the Autotune PID library. They guy has literally no documentation on how to actually use the library. There is one blog post on some high level details of how it works, and that is it. There might be some example code, but I couldn't install it from the zip file so I can't even know. It just says there is no valid library. It's from 2012, maybe there's the problem. But the tutorials are for it, there are no more recent tutorials. I suppose people with better programming skills can jump in and use it without, but this seems to be really common.

It is baffling and infuriating that people do good work and spend a lot of time making these libraries and stuff, but then they don't bother to produce a short document that tells you something about how to use it?? Thus it ends up being useless, at least to some of us. I had this problem with the TMCstepper library, too. Why do so much work writing code and then not document it?

How do you go about using this code that's not very well documented? Why does this keep happening?

I had to give up. For some reason it and the arduino pid library are not in the arduino library system accessible through the ide, anyway, which is also baffling. I tried to get another autotune one working, sTune, but wasn't successful. I never found a solution, I have to keep trying tomorrow.

You can take advantage of free code, produced by volunteers. Some of it might actually work well and even be reasonably well documented.

You could also hire code writers to produce to your specification (see the Jobs and Paid Consultancy forum section), or look on line to buy professional packages with complete documentation.

Comb through the example sketches and header files.

Endemic problem in the tech industry in general :frowning:

Documentation is hard. Writing documentation aimed at people who really don't understand anything about what your code does is even harder.

2 Likes

Oh god…Don’t get me started on the PID Autotuning library. I’ve had a kiln controller project on the back burner for a while and when I was trying to figure out how to use that library I nearly threw my Arduino at the wall.

What I find is usually helpful however is to go to google and find anyone that has used a certain library in their project. That is, I’ll google in quotation marks something like “PID_AutoTune_v0.h”.

I studied this sketch using the Autotune library for instance;

However, since adafruit was heavily relying on this weird RGB LCD display in their sketch, which they were trying to sell me, I ended up not understanding how the autotuning worked and gave up :rofl: :rofl:. But maybe you’ll have some luck with it.

I also found this guy’s video and sketch which I haven't looked at too hard yet, but might be a lot easier to understand;

That being said, the autotuning library and autotuning in general is apparently not very accurate, and you’ll probably achieve more accuracy if you take the time to tune it yourself.

I confess that I haven’t actually tuned a PID myself, but I’m planning to use the instructions contained in a text file in the kiln controller project I’m building; Electric Kiln Controller | Arduino Project Hub

In addition, since the original PID library hasn’t been updated for a while, I’m also planning to switch to a library provided by Powerbroker2, who has made some really good libraries that I have used in the past, like SerialTransfer, so I’m sure it’s a good one; GitHub - PowerBroker2/ArduPID: PID library for Arduinos with greater accuracy than the legacy Arduino PID library

Also, if you're confident enough with code, I also find it helpful to just comb through the files contained in the library, like the .h and .cpp files, and figure out what the f*** is going on. Sounds really tedious, but if you've invested a lot of time into a project, it's sometimes the only way to move forward. Then if all else fails, you've got the paid consultancy/hiring option like jremington mentioned.

If the library is not accessible through the IDE, you just download the zip folder from the github page and put the master folder in the Arduino libraries folder on your PC.

I have written a lot of libraries (MajicDesigns (Marco Colli) · GitHub) and extensively document how they are used in online code and/or blog posts.

This issue been a bugbear of mine for a while because, despite extensively documenting code (using dOxygen) and examples, in most cases nobody reads anything and just sends a message or posts on the forum asking something that is already covered in the documentation. The 'load and go' mentality is rife.

I think there are a number of issues here:

  1. The hobby attracts a lot of newbies that have no idea what to do, even as basic coding. This is fair enough as it is the ethos of Arduino, but often the questions 'about the library' have nothing to do with the library but more like basic coding.
  2. Most libraries are created because the author had a problem to solve. Once the problem is solved they want to move on, so the docs never get done. Example code is often poor or non-existent as well.
  3. What level of documentation should be produced? Absolute novice level or do you assume the user has some idea and can join the dots. The amount of explanation varies a lot between these extremes.
  4. There are no 'expected' standards for where and how to document Arduino libraries - where do you put the help files (important if you expect someone to find them)? Do you set up a readme at the distribution point (eg, github) and how does the user know the link is there - most don't ever see the github site as they used the IDE Library Manager.

I have a rule to not use a third party library unless it has a good level of documentation (direct or through other's use) and I can see it has been actively supported by the author. I really don't have time to trawl through someone else's badly written code to work out what the parameters to a function are or what the workflow should be.

End of rant :slight_smile:

2 Likes

Hi, @marco_c
Will your libraries load with the Library Manager zip facility?

It seems a lot of noobs have problems when the library will not install from zip.

Tom... :grinning: :+1: :coffee: :australia:

Either I get it running or I throw it away. It is as easy as that.

If a poor documented library is my only alternative I will use it. If it doesn't work - i try to make it working or write my own code. I will not complain about others people work which they provided for free.

If I would need help with a 3rd party library, I would link the source I found, make a short demo example showing the problem, post it in code tags and would ask questions of how to solve the problem in a forum.

All my public libraries are registered so Library Manager will work. And yes, they do install from a zip downloaded from the 'tag' or 'release' section of the github repo.

:+1: :+1: :+1: :+1:
I go looking at the examples that come with the library to see if it will do what I need.

I get annoyed when some undocumented functions or parameters appear later, in that library, when I need them and have decided on another library that does what I need, but not as conveniently.

Tom...; :grinning: :coffee: :australia: :+1:

Yeah, I saw the powerbroker one, I was thinking of using it, but again without a nice document that tells you how to use it I can't make heads or tails of it usually. C arduino can be pretty hard to understand. I need to figure what these &s are doing in front of those variable again. I think it was pointer referencing and dereferecing, he is passing the memory location of a variable rather than the variable itsself.... The gains are also not clear. Sometimes the input to the pid loop is going to be 25000, a period in microseconds for instance, that varies by a factor of a thousand, sometimes it is a floating point that is less than 0.05 and varies by a tiny fraction of a percent. I am assuming the gain thinks of things in terms of integers, which has implications for tuning. I assume. All assumptions. Each assumption takes time to test, and if it's wrong can really take a lot of time to figure out, because I have to go back and test so many ones to figure out the one that was wrong.

On the brighter side, here for instance is an example of a guy that did a good job, I haven't read it fully yet, but at least he takes some time to tell you some things. A good general purpose library with lots of uses, clear advantages over the old, and a document that authoritatively tells you about it. ElectricRCAircraftGuy.com--RC, Arduino, Programming, & Electronics: Arduino micros() function with 0.5us precision - using my Timer2_Counter Library

That takes ten times as long, though, that's the problem. Maybe we should be paying people so they can afford to do better work. My complaint is the assymetry, how it takes time to produce code, and much less time to produce documentation, and yet people produce the code and no documentation. There is no better person than the original author. Why bother publishing code that is hardly useful when you could spend 5% more time and actually unlock it to 5 times as many people? Maybe we should pay some people in crypto to go around and document code that is good code but not documented. I have heard the old excuse that nobody is getting paid so don't expect anything to get done too many times in my life. There is a lot to do, a lot of value rides on these libraries and so on.

There was that situation recently with the guy that corrupted his code on purpose, that a huge number of software packages depended on, because he was tired of working for free when so many people clearly used his stuff so much. Not very specific I know, I don't remember the details. It's not the only example I vaguely remember.

Yeah, but if you google the name of the library you'll always see the github. I think the documentation should very much go with the github. All the stuff in one place.

It gives an error "no valid libraries in this zip file".

Then simply open the .zip and create the library folder structure manually using the contained .h and .cpp files.

Bottom line (for me anyway) is that writing code is fun and writing documentation isn't. And, it takes WAY more than 5% more time to do it right.

I've only posted on GitHub one of my personal libraries and it's gotten some traction: https://github.com/gfvalvo/NewEncoder.

I think I've done a decent job of documenting it in the README. I wrote it for my own use but am happy to answer any usage questions. HOWEVER, I'd expect anyone using it to have a reasonable understanding of the C / C++ language and I feel no obligation to provide a tutorial on such things as pointer referencing / de-referencing ('&' and '*'), scope specifiers, structs, references, enums, etc. There are already plenty of those tutorials online already and they do a better job than I could.

So, if you're unhappy with the documentation of a given library, you should contact the author or enter an 'issue' on GitHub. I'm sure they'd be happy to refund 100% of what you paid for it.

1 Like

Opinion:

The Arduino lib is a port of
https://www.mathworks.com/matlabcentral/fileexchange/4652-autotunerpid-toolkit

In the above link, some of the theory is explained.

But, IMO, open-source libraries are generally in two categories:

  • User written and shared,
  • Company written to support sale items (SparkFun, Adafruit, etc)

In the beginning, there was Arduino hardware and the Core, and the hardware and Core were good, but there were no libraries and writing all the source code was bad as it consumed great quantities of time and created much frustration.

A really big BRAVO for all the folks who have written libraries, provided examples, and done testing. For those elite who also documented your code, a well-deserved Thank-You.

Now, I have often written in this forum that libraries are a two-edged sword: the sharp edge can cut to the bone. Many folks use libraries and they work well and they give little thought to that. Others, use libraries, have issues, and they investigate and fix the code (or adapt to their specific need.) But, still others are not capable of 'fixing' a library: maybe inexperienced or maybe they have other options.

It is those who cannot 'fix' and have no options that become frustrated and create much bad press - so many in this group are cut & paste programmers. We (senior forum members) see these folks all the time and there is often no happy outcome for anyone.

So, if one wants to learn programming, they must understand all aspects of the sketch, including more than a black-box knowledge of used libraries. It is not intended that they should be able to rewrite the library from scratch; rather, one should understand the library methods and instantiation. This peels back some of the cloaking of the lib and exposes the working routines. At this point, a fair understanding exists which provides valuable insight into usage (and breakage, if things go wrong.)

My personal take on libraries is that they (generally) are tested & canned routines that get me through a project more easily and quicker. I put all libraries in my sketch folder (easy backup) and I routinely hack libraries, pruning them and often improving them for my use (by hardware architecture.) This way, the next project starts with a sketch folder subordinate to an architecture folder: /Arduino/MySketches/ESP32/ or
/Arduino/MySketches/UNO/ or
/Arduino.MySketches/Mega and so forth...

Therefore, if I need an ILI9341 display for an STM32 project, I copy those "personalized/hacked" files from the most recent project of that architecture. Then, I flesh out the code skeleton, include some display output in setup, and compile. I'm on my way and I am fully immune from the insanity of library updates! Easy backup using Zip from within the parent folder.

Ray

I think documentation takes a lot longer than an extra 5%. For my libraries it is more like 30-50% depending on the complexity of the library, and I document as I go along (ie, it is built in from the start).

Can you give us an example of a Library you have produced, how long it took to write and then how long the extensive documentation took ?