Library development and GitHub

[jimLee]

How are people developing libraries for Arduino setting up their development systems to work with GitHub?

I ask because, for a long time I've been fighting with the idea that the Arduino library folder is located inside the Arduino folder itself. I don't mind, but gitHub does.

This all begin with just making a repository for my Arduino folder holding my little sketch projects while ignoring libraries altogether. Pretty soon I needed to make repositories for my own library development. Then I ran into the issue that.. gitHub doesn't like repositories inside other repositories.

What to do?

Turns out that if you setup a local library repository outside your Arduino library folder. GitHub is fine with that. If you then move your private library folder into the Arduino library folder. Github asks where it is, you show it in the Arduino library folder, and everything is fine.

This worked for about ten years.

This method finally broke a couple days ago. Moving library repositories into the Arduino library folder causes gitHub to rename them all Arduino repositories. It's a mess.

So I ask. How do people setup their development systems for working on both Arduino sketches and Arduino libraries using gitHub?

Thanks!

[kenb4]

@jimLee

This method finally broke a couple days ago. Moving library repositories into the Arduino library folder causes gitHub to rename them all Arduino repositories.

Not sure what that means. Granted, it's been a bit over a week since I did a push/pull.

I have, each as a separate repo, some sketches, and some libraries

• ~/Arduino
  • first sketch repo
  • second sketch repo
  • not a repo
  • also not a repo
  • third sketch repo
  • libraries
    • ArduinoJson
    • SomeOtherLibrary installed by the IDE
    • my first library repo
    • my second library repo

From within ~/Arduino or libraries, I do a git clone to pull a sketch or library respectively, into a subdirectory. For something brand new, git init within that directory and then push; or pull from within that directory to get the latest from GitHub. That's all with the git command directly; I don't use anything else.

gitHub doesn't like repositories inside other repositories.

GitHub does support git submodules, but you might not even need that. You could have a "sketches" repository, with each in a subdirectory; and specifically add libraries to .gitignore. Then a separate repo for each library would be better, especially if you make those libraries public.

[sterretje]

I'm not a very experienced (nor active) git user so this might not help you. I keep things simple (for now) and add something to the name of the actual project.

Note:
I can't even remember exactly how I added the above repositories or even made a release for one of them.

[ptillisch]

Hi @jimLee.

@jimLee

the Arduino library folder is located inside the Arduino folder itself. I don't mind, but gitHub does.

GitHub should not "mind".

@jimLee

if you setup a local library repository outside your Arduino library folder. GitHub is fine with that. If you then move your private library folder into the Arduino library folder. Github asks where it is

GitHub is a web service. The web service doesn't know or care anything about the files on your computer. So it seems that you must be referring to something other than the GitHub web service, such as a local application.

Please provide a detailed description of what you mean by "GitHub". Are you perhaps referring to the GitHub Desktop local Git client application? Or are you referring to something else.

@jimLee

Moving library repositories into the Arduino library folder causes gitHub to rename them all Arduino repositories.

Please provide a detailed description of what you mean by "rename them all Arduino repositories".

@jimLee

How do people setup their development systems for working on both Arduino sketches and Arduino libraries using gitHub?

When it comes to a library, just initialize a Git repository in the root of the library folder:

Git - git-init Documentation

(you can perform the initialization via a Git client if you prefer working from a GUI instead of using Git directly from the command line)

---

For developing a sketch project, I do recommend a structure like this:

MySketch/
├── .git/
│   │
│   ...
├── MySketch/
│   ├── Additional.ino
│   ├── MySketch.ino
│   ...
├── LICENSE.txt
├── README.md
...

Note that I have placed the sketch inside a dedicated parent folder, and the root of the repository is that parent folder. The reason for placing the sketch in a subfolder of the repository of the repository is to facilitate the use of the sketch by people who obtain it via GitHub's GitHub's "Download ZIP" feature, or from the archive download links that GitHub automatically adds to the "Assets" section of the release page.

For someone contributing to the development of the project, a local copy of the project would be obtained by cloning the repository. However, for someone who only wants a copy of the sketch to use, and who might not use Git often if ever, just downloading a ZIP archive of the files will be much more convenient. When you use the "Download ZIP" feature, the ZIP package you get has a root folder named with the format <repository name>-<Git ref>, where <Git ref> is the Git ref for the revision of the repository at which the ZIP package was generated. This is a problem for repositories where the sketch is the root of the repository because the primary .ino file of the sketch must have a name matching the folder. So if you don't place the sketch in a subfolder of the repository, the ZIP package will contain an invalid sketch

When a user opens a sketch in Arduino IDE, if there is a mismatch between the folder and filename, it displays a dialog explaining that the sketch is not valid, and offering to move the file to an appropriately named folder:

It seems like clicking the "OK" button in that dialog would solve the problem, and in fact it does in the case of a sketch that only contains a single .ino file. However a sketch may contain multiple code files. This feature only moves the single .ino file the user selected when opening the sketch to the appropriately named new folder, leaving behind any other files that might have been present in the original path. That breaks multi-file sketches.

By storing the sketch in a subfolder of the repository, you ensure that the sketch will always have a correct folder name.

[ptillisch]

I think I see now. You have a structure like this:

Arduino/
├── .git/
│   │
│   ...
├── MySketch/
│   ├── .git/
│   │   │
│   │   ...
│   │
│   ...
├── OneMoreSketch/
│   │
│   ...
├── YetAnotherSketch/
│   │
│   ...
├── libraries/
│   ├── MyLibrary/
│   │   ├── .git/
│   │   │   │
│   │   │   ...
│   │   ...
│   ...
...

@jimLee

This all begin with just making a repository for my Arduino folder holding my little sketch projects

This is the cause of your problems. In general, I would recommend creating a separate repository for each project. However, if you do want to place a collection of sketches in a single repository as you have apparently done with your "Arduino" repository, just place those in a dedicated subfolder of your repository. For example, here you can see I moved the repository that was originally located in the Arduino folder to a subfolder named SomeSketches:

Arduino/
├── .git/
│   │
│   ...
├── MySketch/
│   ├── .git/
│   │   │
│   │   ...
│   │
│   ...
├── SomeSketches/
│   ├── .git/
│   │   │
│   │   ...
│   ├── OneMoreSketch/
│   │   │
│   │   ...
│   ├── YetAnotherSketch/
│   │   │
│   │   ...
│   │
│   ...
├── libraries/
│   ├── MyLibrary/
│   │   ├── .git/
│   │   │   │
│   │   │   ...
│   │   ...
│   ...
...

You can do this with a simple folder move operation in your file manager. You don't need to perform any Git operation on the repository that was originally located in Arduino. If you like, you can rename your GitHub repository to match the new folder name (in this example, Arduino -> SomeSketches), but there is no requirement to do so.

That SomeSketches repository can now coexist with the other project repositories that are located under the Arduino folder, without those other project repositories being treated as Git submodules of the catch-all sketch repository.