We are helping electronics educations.
Recently, one of our client asked if it's possible to clone all Arduino libraries into their local server.
Because they have low-speed connection and high networking cost compared to their management budget, they were having problems in this.
When an instructor teaches about 16*2 dot-matrix LCD control, hundreds of students are trying to download the same 1602 LCD library at the same time. And it causes the classroom internet connection bandwidth overtaking.
I have a quick research for possible solutions.
I found that all Arduino libraries are registered here:
I can clone all the repositories listed there recursively by some scripting.
I am gonna host them all in the school's local NAS server.
But I am not sure if it is possible to change Arduino's default library lockup target to the school's local GitLab and/or FTP server.
For PyPI and Ubuntu, we helped those schools to solve this problem already by cloning all the repository into their NAS server.
For example, there are already many local PyPI servers that are working out of the box!
We could just re-address our local PyPI server by using:
prepare all needed libraries for a lesson and download them as ZIPs.
Host the ZIPs in the local LAN
let the students import the ZIPs from the local LAN.
as alternative:
after the preparations by the teacher,
he should push all his used libraries to a file share
the students (or any cron, job, service...) should pull the libraries from the file share.
Something you should note is that the Arduino IDE Library Manager does not download the libraries from the repositories. Instead, it downloads them from archive files hosted on downloads.arduino.cc (which were generated from the repository tags).
The URL of the archive files is defined in the libraries[*].url field of library_index.json
You be interested in this hack Adafruit made to take advantage of the Arduino Boards Manager system to distribute their boards platforms after the initial release of the feature before the Arduino developers had added the formal support for 3rd party package index URLs:
An alternative solution would be to share the staging folder between all the student's Arduino IDE installations. The archive files downloaded when a library is installed via the Arduino IDE Library Manager or a boards platform installed via the Arduino IDE Boards Manager are cached in on the local machine. The IDE first checks to see if the file is present there (validating it via a checksum) before downloading. So if a single staging folder was shared, each file would only be downloaded once, after which it would be reused for any subsequent installations of that library or platform.
By default, the staging folder is here:
Windows
%LOCALAPPDATA%\Arduino15\staging\
Linux
~/.arduino15/staging/
macOS
~/Library/Arduino15/staging/
If you are using Arduino IDE 2.x, you can configure the location by changing the value of the directories.downloads key in the configuration file at this path:
Surely, this is an overkill.
Students only need libraries that match the class lesson plan; a smaller subset of the universe.
Are you running ArduinoIDE over the network? Is the NAS a mapped drive?
Hundreds?
Are the students harddisks being "wiped" between sessions? An entire semester of library code and IDE on the student workstation seems a more reasonable approach ... reimaging the occasional PC is not a network overload. Just initiate a cloning script stored on the mapped NAS.
Yes, because we want students to experience searching and installing via Arduino library manager by themselves, all the computers are cleaned up after the semister ends.
Thank you for the detailed answer, @ptillisch
I was thinking both GitLab and FTP as I realised that the Arduino library manager is using blob files, just like PyPI does.
The Adafruit's work seems interesting.
I'll dig it deeper.