automatic generator of keywords.txt

[aster94]

I didn't wont to lost 5 minutes to create the keywords.txt of a library i was working on, so i lost 1 hour to write a python script that does this for me 

note that the keywords.txt is missing in the box on the left
video

If you want to test it you could find it here

[pert]

It seems like a very useful program. Thanks for sharing it.

However, if the keywords.txt in that library was generated from your script, it's not working correctly. You should not use multiple tabs to separate fields in keywords.txt. This will cause they keywords to be incorrectly colored. Each field of keywords.txt is separated by a single tab, if you use multiple tabs, this causes the default identifier to be used. Avoid your OCD need to make everything line up all "pretty". That's not what those tabs are for. This is a file that a computer is supposed to read. When you try to make it more pleasing to the human eye, you are destroying the formatting the computer needs.

I'm also curious what the story is with the random spaces in the file. Is your script adding those? Here's what the file looks like with indicators that show tabs and spaces:

Note that all your efforts to make it pretty with multiple tabs are for nothing when someone has a different tab size setting. This is why in the world of files written for humans, we always use spaces for alignment, even if you prefer tabs for indentation.

As someone who's diving deep into the world of keywords.txt, you will definitely want to carefully study the official specification:
https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#keywords

[aster94]

However, if the keywords.txt in that library was generated from your script, it's not working correctly
...
I'm also curious what the story is with the random spaces in the file. Is your script adding those?

nope, i manually made them before making the script, and that strange indent was generated by using two different program VScode (set as spaces) and nodepad++ (set as tab of 4) + github (that is set as 8 space tabs) this is the reason of a so strange indent

I should have updated the two keywords.txt before posting it ahahahah but i didn't wanted to make two more commit only for a file but now i did it

Each field of keywords.txt is separated by a single tab

I wasn't aware of it! I corrected the script and now it works as it should do, thanks for pointing this out!
You can check it here and here

Anaway the script has been made only for library like this, and won't work well in header-only library, but it could be fixed easily adding filtering in this for loop

[pert]

Way cool! It seems like it might be worth putting this script in a dedicated repository to make it easier for people to find and use.

I've been meaning to figure out a way to do an automated check for missing or extra keywords during CI builds or for use in this project of mine. To avoid reinventing the wheel, I've considered just using one of these keywords.txt generators and then comparing the generated file to the existing keywords.txt. Towards that end, I've collected a list of the keywords.txt generators I come across:
https://github.com/per1234/arduino-ci-script/issues/17
Maybe you'd be interested in seeing what the "competition" has done. I haven't actually tried any of them, since the check for missing/extra keywords hasn't climbed high enough on my "to do" list for arduino-ci-script yet.

[aster94]

This one, since it uses doxygen, would be the most accurate, the only problem for your ci script is that require an extra program to use it (doxygen) and maybe you wish to keep it as simple as possible.

My approach is almost the same as this but i didn't found it before 

[pert]

Thanks for the information! Ideally, I would do this purely in Bash using standard tools. Although it's mostly focused on use in Travis CI builds currently, I do want to make arduino-ci-script friendly to use locally with any OS. The user would likely prefer not to have to install a couple extra dependencies onto their computer.

On the other hand, it's probably quite tricky to deal with every corner case, so that makes me think the doxygen approach is worth considering.

I'm certain there are a ton of Arduino libraries out there with incomplete or outdated keywords.txt files. I know I've stumbled across some problems even with the keywords.txt files of official Arduino libraries without even trying. I understand why some people decide not to bother with keywords.txt, but if you're going to do it I think it's worth doing right.

[aster94]

In my opinion if you would like to have a program that works for 99.99% of the cases doxygen is the way to go. If I were you i would make the script download a fixed stable release (not master) from their github page then build it (they seems to have a good guide: https://github.com/doxygen/doxygen/blob/master/BUILD.txt). Then you should invest some times to refine a custom doxyfile and finally using KeywordsTxtGenerator or another script, even in bash shouldn't be too hard, you would be able to check for missing values.

[pert]

Thanks for the advice! I really need to make this project a priority. I already wrote a function that checks the format of keywords.txt, and that has been very useful. I think they would go well together. Our conversation at least got me thinking about it and I even gave bengtmartensson/KeywordsTxtGenerator a try just to verify this is a feasible approach.

[aster94]

Your CI script is a nice project
It is an overkill for people like me writing just a few libraries but every maker of an arduino core (and arduino people themself) should use it

[pert]

Thanks. The idea is to make it easy to do continuous integration testing of Arduino projects specifically so that people with only a few libraries would start doing CI.

When I started with the project, the process of using the Arduino IDE in a Travis CI build was more complicated than it is with the recent IDE versions. I wanted something that would allow me to always use the newest version of the Arduino IDE in my tests without having to go update 20 repositories every time there was a new IDE release. I also wanted to make it easy to test with multiple IDE versions to ensure backwards compatibility. I found there are issues with certain versions of the Arduino IDE that can cause a tremendous amount of confusion for someone trying to set up a CI build configuration from scratch using that version. The script attempts to handle all these issues seamlessly, without exposing the user to any of that complexity.

Over time, I've added more features to the script, each adding to the length of the documentation. For this reason, it may now seem a bit intimidating to a potential user. I think I could improve on this situation by providing links to demonstrations of the basic use of the script for a library and a sketch. I actually do have a link to the .travis.yml file of one of my libraries for this purpose, but I have added other tests to that file over time so that it no longer serves well as a simple demonstration of arduino-ci-script. I need to create some dedicated repositories for that purpose.

[jsmith2021]

Hello,

On the topic of automatic keyword file generation, I am working on an Arduino Class Generator which automatically creates a class body (.cpp), header (.h),  keywords (.txt), and example file (.ino) from an Arduino Sketch

The motivation for my project is that I found myself writing sketches to create the functionality I wanted, and then manually converting that sketch into a library for object orientated programming and sharing.

This process consists of parsing the sketch into the necessary information to make a library, like comments, variables, methods, and other included libraries

The included files demonstrate the project. Morse.ino is the original sketch, and MorseParsed is the intermediary file used to generate Morce.cpp, Morse.h, MorseExampleFile.ino, and MorseKeywords.txt

The github for the project is here
and I made a pull request to include it in the github IDE, the ClassGeneratorInIde picture shows what using the IDE would like like if my change is accepted

I also posted this on the forum here

Does anyone reading have thoughts on this project?

Thanks,
Jacob Smith