CI Workflow - Library Load - Ex UNO R4 Wifi

I have posted on CI workflow on firmware but seems I am running into as many issues on library load.

Ex:

I have CI workflow that parses all .h files noted in sketch and loads library

Read in current files but ignore secrets.h as that is not a library but the SSID and passcode for site.

Ex:

#include <WiFiS3.h>
#include <ArduinoGraphics.h>
#include <ArduinoLEDMatrix.h>
#include <ArduinoHttpClient.h>
#include <ArduinoJson.h>
#include "/opt/shuffleboard/secrets.h"  // Explicit path to secrets.h

// Static variables

Ansible tasks:

- name: Read .ino file content
  ansible.builtin.slurp:
    src: "{{ ino_file_path }}"
  register: ino_file_content
  when: ino_file_stat.stat.exists

- name: Decode .ino content
  ansible.builtin.set_fact:
    ino_text: "{{ ino_file_content.content | b64decode | default('') }}"
  when: ino_file_stat.stat.exists and ino_file_content.content is defined

- name: Debug .ino content (truncated to 1000 characters)
  ansible.builtin.debug:
    var: ino_text | truncate(1000)
  when: ino_text is defined

- name: Extract valid Arduino library names
  ansible.builtin.set_fact:
    libraries: "{{ ino_text | regex_findall('#include\\s+[<\"]([^/][^>\"]+)\\.h[>\"]') | reject('search', '/') | reject('equalto', 'secrets') | list }}"
  when: ino_text is defined
  ignore_errors: true

- name: Debug extracted libraries
  ansible.builtin.debug:
    var: libraries
  when: libraries is defined

- name: Install Arduino libraries
  ansible.builtin.command: 
    cmd: arduino-cli lib install "{{ item }}"
  loop: "{{ libraries }}"
  when: libraries is defined and libraries | length > 0
  register: lib_install_result
  changed_when: lib_install_result.rc == 0
  failed_when: lib_install_result.rc != 0

- name: List installed Arduino libraries
  ansible.builtin.command: 
    cmd: arduino-cli lib list
  register: libraries_installed
  when: libraries is defined and libraries | length > 0

- name: Show library installation results
  ansible.builtin.debug:
    var: libraries_installed.stdout_lines
  when: libraries_installed is defined

Error:

TASK [arduino-library-install : Install Arduino libraries] *********************
failed: [triton] (item=WiFiS3) => {"ansible_loop_var": "item", "changed": false, "cmd": ["arduino-cli", "lib", "install", "WiFiS3"], "delta": "0:00:00.487988", "end": "2025-06-09 23:38:11.257589", "failed_when_result": true, "item": "WiFiS3", "msg": "non-zero return code", "rc": 1, "start": "2025-06-09 23:38:10.769601", "stderr": "Error installing WiFiS3: Library 'WiFiS3@latest' not found", "stderr_lines": ["Error installing WiFiS3: Library 'WiFiS3@latest' not found"], "stdout": "", "stdout_lines": []}
changed: [triton] => (item=ArduinoGraphics)
failed: [triton] (item=ArduinoLEDMatrix) => {"ansible_loop_var": "item", "changed": false, "cmd": ["arduino-cli", "lib", "install", "ArduinoLEDMatrix"], "delta": "0:00:00.490689", "end": "2025-06-09 23:38:13.188581", "failed_when_result": true, "item": "ArduinoLEDMatrix", "msg": "non-zero return code", "rc": 1, "start": "2025-06-09 23:38:12.697892", "stderr": "Error installing ArduinoLEDMatrix: Library 'ArduinoLEDMatrix@latest' not found", "stderr_lines": ["Error installing ArduinoLEDMatrix: Library 'ArduinoLEDMatrix@latest' not found"], "stdout": "", "stdout_lines": []}
changed: [triton] => (item=ArduinoHttpClient)
changed: [triton] => (item=ArduinoJson)

Yet same search and manual install in UI via Windows client works fine, so this is just orchestration issue on PI V on how to take a sketch and parse out library and install it. I think the gap is that manual search I can select from libraries that match best a string, vs my crude way in CI which must meet exact match.

I have to imagine others have had this issue of parsing a sketch to pre-stage up libraries needed.

Hi @penguinpages. arduino-cli lib install <library name> can only be used to install libraries that are distributed via the Arduino Library Manager system. It is expected that it will fail if you specify the name of a library that is not available via Library Manager.

There is no "WiFiS3" or "ArduinoLEDMatrix" in Library Manager. These specific libraries are "platform bundled" libraries, which are distributed as part of the "Arduino UNO R4 Boards" platform (machine identifier arduino:renesas_uno).

Even for libraries that are available for installation via Library Manager, your approach is lawed in that you are assuming that the header filename in the #include directive will always match the library name. Although this is a common practice, there is absolutely no guarantee of such a match as the library developers are free to use any header filename they like. In some cases, it wouldn't even make sense to do that because a library doesn't necessarily have a single primary header file.

The correct way to do this is to define the library dependencies of the sketch via a Build Profile in the Sketch Project File:

https://arduino.github.io/arduino-cli/latest/sketch-project-file/#build-profiles

You can then simply specify the profile name as an argument to the --profile flag in the arduino-cli compile command and Arduino CLI will take care of installing the dependencies for you.

I saw this page but was not sure if I had to go down this path. Also the workflow of use was not clear as are a few of the fields

Example: I have ten boards t01-t10

I am using ansible to manage CI flow so all variables I am trying to hold in ansible/inventories/production/hosts.yaml

Ex: t01 and t02

    arduino_boards:
      hosts:
        t01:
          SN: 24:EC:4A:23:65:A4
          ip_address: 172.16.100.121
          mac_address: 24:ec:4a:23:65:a4
          ansible_ssh_common_args: '-o StrictHostKeyChecking=no'
          fqbn: "arduino:renesas_uno:unor4wifi"
        t02:
          SN: 48:CA:43:5C:F2:94
          ip_address: 172.16.100.122
          mac_address: 48:ca:43:5c:f2:94
          ansible_ssh_common_args: '-o StrictHostKeyChecking=no'
          fqbn: "arduino:renesas_uno:unor4wifi"

Note, seems somewhere along way fqbn added ":" which is the same as MAC, which works, just change in process :slight_smile:

To address that each board may need different libraries for different builds I will need to create a "profile" file for each board and then part of the setup workflow use that to load libraries needed to deploy the sketch, but it would be nice to have one profile for the total project as most boards will use same libraries and updates of one in version / will likely be replicated by many of the ten.

ansible/roles/arduino-library-install/library_profile.yaml (vs each board having separate file) in its sketch folder. But open to pros/cons to this.

Issues

  1. What is the "platform" field and where do I get it for my board? Or is this a URL list of where to search for listed libraries when loading them up?

Google implies "platform" is for the board itself, such as its chipset(s): " The Arduino UNO R4 WiFi is based on the Arduino UNO R4 Board Package. It can be programmed using the Arduino IDE, Arduino Cloud Editor, or Arduino CLI. The Renesas RA4M1 microcontroller is the main processor, and it also includes an ESP32-S3 module for Wi-Fi and Bluetooth connectivity. The board is also compatible with the Arduino IoT Cloud. " But poking around in Arduinio IDE, still not sure what that field should be."

  1. Example library_profile.yaml is helpful in the web page but is there a workflow logic of how to extract "libraries" field from a tested library in IDE? Ex: WiFiS3

When I select install:

Downloading R4HttpClient@1.0.4
R4HttpClient@1.0.4
Installing R4HttpClient@1.0.4
Installed R4HttpClient@1.0.4

But that does not seem correct:

# Compilation of all the libraries used in the Arduino projects for each board.
# Each board t01-t10 will have its own profile to enable boards which will need different libraries.
# Documentation: https://arduino.github.io/arduino-cli/1.2/sketch-project-file/#build-profiles
# Command line testing: arduino-cli compile --profile /opt/shuffleboard/ansible/roles/library-profile/library_profile.yaml --fqbn arduino:renesas_uno:unor4wifi /opt/shuffleboard/ansible/roles/t01/arduino/t01.ino
# Updated 2023-10-20
profiles:

  t00:
    notes: This board is test board state, it is a Renesas Uno R4 WiFi board.
    fqbn: arduino:renesa  # Not sure if this reflects the board, or where to search for sub libraries.
      # - platform: attiny:avr (1.0.2)
      #   platform_index_url: https://raw.githubusercontent.com/damellis/attiny/ide-1.6.x-boards-manager/package_damellis_attiny_index.json
      # - platform: arduino:avr (1.8.3)
    libraries:
      - R4HttpClient@1.0.4 #include <WiFiS3.h> Renesas Uno R4 WiFi board wifi library
      - foo #include <ArduinoGraphics.h> Renesas Uno R4 WiFi board graphics library for LCD displays
      - foo #include <Arduino_LED_Matrix.h> Renesas Uno R4 WiFi board integrated LED matrix library
      - foo #include <ArduinoHttpClient.h> Renesas Uno R4 WiFi board HTTP client library for web requests. API PUT/GET/POST requests
      - foo #include <ArduinoJson.h> Renesas Uno R4 WiFi board JSON parsing library
      - foo #include "Wire.h" Renesas Uno R4 WiFi board I2C library for 16 channel relay boards.
      - foo #include <qrcode.h> Renesas Uno R4 WiFi board QR code library
    port: {{ address }} # serial port based on fact set from ansible task "get-serial"
    port_config:
      baudrate: 115200

  t01:
    notes: This board is test board state, it is a Renesas Uno R4 WiFi board.
    fqbn: arduino:renesa  # Not sure if this reflects the board, or where to search for sub libraries.
      # - platform: attiny:avr (1.0.2)
      #   platform_index_url: https://raw.githubusercontent.com/damellis/attiny/ide-1.6.x-boards-manager/package_damellis_attiny_index.json
      # - platform: arduino:avr (1.8.3)
    libraries:
      - foo #include <WiFiS3.h> Renesas Uno R4 WiFi board wifi library
      - foo #include <ArduinoGraphics.h> Renesas Uno R4 WiFi board graphics library for LCD displays
      - foo #include <Arduino_LED_Matrix.h> Renesas Uno R4 WiFi board integrated LED matrix library
      - foo #include <ArduinoHttpClient.h> Renesas Uno R4 WiFi board HTTP client library for web requests. API PUT/GET/POST requests
      - foo #include <ArduinoJson.h> Renesas Uno R4 WiFi board JSON parsing library
      - foo #include "Wire.h" Renesas Uno R4 WiFi board I2C library for 16 channel relay boards.
      - foo #include <qrcode.h> Renesas Uno R4 WiFi board QR code library
    port: {{ address }} # serial port based on fact set from ansible task "get-serial"
    port_config:
      baudrate: 115200

default_profile: t00  # Default profile to use when no profile is specified as test board.

Update on this thread.

I use as a tool AI such as GROK or ChatGPT or Copilot. I was really stuck on how to translate what I could do "pet" style in Arduinio UI to search and install library into a CI where I could know what fields to call a given installed library. I am still not certain how GROK figured it out, and need to learn that back process, as it was not clear on that as much as I neeed.

What I can say is the below workflow reviews my ten arduinios and I can set different libraries for each, and it will deploy them as the workflow calls each board ex; t01-t10

Please review and correct or provide ideas:

ansible/roles/arduino-library-install/tasks/main.yaml

---
- name: Define variables using facts
  ansible.builtin.set_fact:
    project_dir: "/opt/shuffleboard"
    ino_file_path: "{{ project_dir }}/ansible/roles/{{ tabletop_number }}/{{ board_id }}/{{ board_id }}.ino"

- name: Verify required facts are set
  ansible.builtin.fail:
    msg: "Required fact {{ item }} is not set"
  when: vars[item] | default('') == ''
  loop:
    - tabletop_number
    - board_id
    - address
    - fqbn

- name: Check if .ino file exists
  ansible.builtin.stat:
    path: "{{ ino_file_path }}"
  register: ino_file_stat

- name: Fail if .ino file is missing
  ansible.builtin.fail:
    msg: "Error: .ino file {{ ino_file_path }} does not exist"
  when: not ino_file_stat.stat.exists

- name: Read .ino file content
  ansible.builtin.slurp:
    src: "{{ ino_file_path }}"
  register: ino_file_content

- name: Decode .ino content
  ansible.builtin.set_fact:
    ino_text: "{{ ino_file_content.content | b64decode }}"

- name: Define library mappings
  ansible.builtin.set_fact:
    library_mappings:
      WiFiS3.h: WiFiS3@1.0.0
      ArduinoGraphics.h: ArduinoGraphics@1.0.0
      Arduino_LED_Matrix.h: Arduino_LED_Matrix@1.0.0
      ArduinoHttpClient.h: ArduinoHttpClient@0.5.0
      ArduinoJson.h: ArduinoJson@7.2.0
      qrcode.h: QRCode@0.0.1
      Wire.h: null  # Core library, no installation needed
      secrets.h: null  # Local file, no installation needed

- name: Extract included headers
  ansible.builtin.set_fact:
    included_headers: "{{ ino_text | regex_findall('#include\\s+[<\"]([^>\"]+)\\.h[>\"]') | list }}"

- name: Map headers to library names
  ansible.builtin.set_fact:
    libraries_to_install: "{{ included_headers | map('extract', library_mappings) | select('string') | list }}"

- name: Update Arduino CLI library index
  ansible.builtin.command:
    cmd: arduino-cli lib update-index
  changed_when: true

- name: Install Arduino libraries
  ansible.builtin.command:
    cmd: arduino-cli lib install "{{ item }}"
  loop: "{{ libraries_to_install }}"
  when: libraries_to_install | length > 0
  register: lib_install_result
  changed_when: lib_install_result.rc == 0
  failed_when: lib_install_result.rc != 0

- name: List installed Arduino libraries
  ansible.builtin.command:
    cmd: arduino-cli lib list
  register: libraries_installed

- name: Show library installation results
  ansible.builtin.debug:
    var: libraries_installed.stdout_lines

Which consumes example (shortened) ansible/roles/arduino-library-install/library_profile.yaml

# Compilation of libraries for Arduino projects for boards t01-t10.
# Each board has a distinct profile to support different library requirements.
# Documentation: https://arduino.github.io/arduino-cli/1.2/sketch-project-file/#build-profiles
# Command line testing: arduino-cli compile --profile /opt/shuffleboard/ansible/roles/arduino-library-install/library_profile.yaml --fqbn {{ fqbn }} /opt/shuffleboard/ansible/roles/{{ tabletop_number }}/{{ board_id }}/{{ board_id }}.ino
# Updated 2025-06-10
#
# Process to identify libraries for this file:
# 1. Extract #include directives from the .ino file (e.g., WiFiS3.h, PCF8575.h).
# 2. Check if the header is a core library (e.g., Arduino.h, Wire.h), local file (e.g., secrets.h), or external library.
# 3. For external libraries, use `arduino-cli lib search <keyword>` to find the library name (e.g., PCF8575 for PCF8575.h).
# 4. Verify the latest stable version with `arduino-cli lib search <library_name> --names` (e.g., PCF8575@1.0.2).
# 5. Confirm compatibility with the board (e.g., arduino:renesas_uno:unor4wifi) via documentation or testing.
# 6. List external libraries here with annotations; exclude core libraries and local files.

profiles:
  t01:
    notes: Sand board t01, Renesas Uno R4 WiFi. Baseline libraries plus additional libraries for relay, OLED, and stepper motor control.
    fqbn: "{{ fqbn }}"
    libraries:
      - WiFiS3@1.0.0        #include <WiFiS3.h> Renesas Uno R4 WiFi board wifi library
      - ArduinoGraphics@1.0.0  #include <ArduinoGraphics.h> Renesas Uno R4 WiFi board graphics library for LCD displays
      - Arduino_LED_Matrix@1.0.0  #include <Arduino_LED_Matrix.h> Renesas Uno R4 WiFi board integrated LED matrix library
      - ArduinoHttpClient@0.5.0  #include <ArduinoHttpClient.h> Renesas Uno R4 WiFi board HTTP client library for web requests. API PUT/GET/POST requests
      - ArduinoJson@7.2.0      #include <ArduinoJson.h> Renesas Uno R4 WiFi board JSON parsing library
      - QRCode@0.0.1          #include <qrcode.h> Renesas Uno R4 WiFi board QR code library
      - PCF8575@1.0.2         #include <PCF8575.h> Library for PCF8575 I2C 16-channel expander for relay boards
      - Adafruit GFX Library@1.11.10  #include <Adafruit_GFX.h> Graphics library for drawing on displays
      - Adafruit SSD1306@2.5.10  #include <Adafruit_SSD1306.h> Library for SSD1306-based OLED displays
      - Stepper@1.1.3          #include <Stepper.h> Library for controlling stepper motors
    port: "{{ address }}"
    port_config:
      baudrate: 115200

  t02:
    notes: Sand board t02, Renesas Uno R4 WiFi. Baseline libraries; add board-specific libraries as needed.
    fqbn: "{{ fqbn }}"
    libraries:
      - WiFiS3@1.0.0        #include <WiFiS3.h> Renesas Uno R4 WiFi board wifi library
      - ArduinoGraphics@1.0.0  #include <ArduinoGraphics.h> Renesas Uno R4 WiFi board graphics library for LCD displays
      - Arduino_LED_Matrix@1.0.0  #include <Arduino_LED_Matrix.h> Renesas Uno R4 WiFi board integrated LED matrix library
      - ArduinoHttpClient@0.5.0  #include <ArduinoHttpClient.h> Renesas Uno R4 WiFi board HTTP client library for web requests. API PUT/GET/POST requests
      - ArduinoJson@7.2.0      #include <ArduinoJson.h> Renesas Uno R4 WiFi board JSON parsing library
      - QRCode@0.0.1          #include <qrcode.h> Renesas Uno R4 WiFi board QR code library
      # Example: - Servo@1.2.0  #include <Servo.h> Library for servo motor control
    port: "{{ address }}"
    port_config:
      baudrate: 115200

<snip>

default_profile: t01  # Default profile for compilation

I can feed AI other libaries that I have built in my tool bag of tasks that I have gotten to work and it can spit back the fields but WHERE it is collecting this, and when I would use "URL" as shown in website as a needed field, I am not clear on and need to RTFM.

Feel free to use :slight_smile: And thanks for those on forum helping

There are alternatives. For example, we use a GitHub Actions-based CI system to perform automated "smoke test" compilations of all the Arduino firmware projects maintained by the Arduino company. We specify the dependencies in the GitHub Actions workflow instead of using build profiles. For example:

However, this approach is relatively simple for us because we have a GitHub Actions action that handles the dependencies installation, so we don't need to implement everything in the workflow.

I'm not sure what you mean by that. As far as I know (and I've had a special focus on the Arduino boards platform framework for over a decade), the FQBN has always had this format. For example, here you can see it in the documentation for the Arduino IDE 1.x command line interface circa 2013:

This is where you define the Arduino boards platform dependencies of the sketch.

You can use arduino-cli board search to determine the platform ID for a given board if you like.

For example:

$ arduino-cli board search "uno r4 wifi"

Board Name          FQBN                          Platform ID
Arduino UNO R4 WiFi arduino:renesas_uno:unor4wifi arduino:renesas_uno

If you already installed the platform via arduino-cli core install, then you will already know the ID.

No.

That is correct, though the next part is fairly nonsensical:

An Arduino platform (which I often refer to as a "boards platform" in order to be a bit more descriptive) is the collection of software that adds support to the Arduino development tools (Arduino CLI, Arduino IDE, Arduino Cloud Editor) for a given set of Arduino boards or microcontrollers.

The platform dependency of the UNO R4 WiFi board is "Arduino UNO R4 Boards", which is installed via the Arduino IDE Boards Manager. The machine identifier for that board is arduino:renesas_uno.

The machine identifier for a library is the library name, as shown in the Arduino IDE Library Manager.

If your sketch has a dependency on version 1.0.4 of the "R4HttpClient" library, then that would indeed be correct. However, the associated comment seems to imply that you think this is the library that provides the WiFiS3.h header file. That is false. The library that provides the WiFiS3.h is the "WiFiS3" library.

As I already explained in my previous reply, the "WiFiS3" library is a "platform bundled library". In this case, the library is bundled with the arduino:renesas_uno platform:

So you manage this specific dependency via the platforms mapping of a build profile.

I just don't think it is worth the effort to try to automatically determine the library dependencies of a sketch by parsing the #include directives. You are always going to need to manually baby such a system, as you are doing with the "library_mappings" system. Maintaining and babying the system is going to be more work than just defining the list of dependencies.

This would be an alternative approach to a build profile. It doesn't make sense to do this when you already have a build profile. When you use a build profile, Arduino CLI automatically installs the dependencies specified by the build profile. The sketch will be compiled using an isolated dependencies environment that consists only of the dependencies specified in the build profile, so the libraries you installed via arduino-cli lib install will be completely ignored when compiling the sketch.

You would be better off to just take the time to understand the system and do it yourself. It is really not very complicated and the work required to do it yourself will be less than the work of using the AI.

Coming back to this topic. Hopeful now with a bitt better understanding of workflow for CI

  1. Build manually as needed / update a ansible/roles/arduino-library-install/library_profile.yaml where this file defines all needed libraries to be consumed for deploying sketches.

  2. This ansible/roles/arduino-library-install/library_profile.yaml will include common / hosted libraries that are platform dependent (in my case roles such as Wifi, API, and some for LCD display common to all boards, some which are different for each arduinio deploy)

  3. To create common tasks or library of common actions, I will include local "libraries" which will built and then included. I can then make more complex arduinio jobs that leverage modules for sub tasks or actions.

ansible/roles/arduino-library-install/library_profile.yaml

base_profile:
  notes: Common libraries for all Renesas Uno R4 WiFi boards
  libraries:
    - WiFiS3@1.0.0
    - ArduinoJson@6.21.3
    # - Adafruit GFX Library@1.11.9
    # - Adafruit SSD1306@2.5.7
    # - Adafruit BusIO@1.16.0
    # - name: ShuffleboardLib
    #   path: ~/Arduino/libraries/ShuffleboardLib
profiles:
  t00:
    notes: t00 - Test Board and Template
    inherit: base_profile
    libraries: []
      # - PCF8575@1.0.2
      # - Stepper@1.1.3
  t01:
    notes: t01
    inherit: base_profile
    libraries: []
      # - PCF8575@1.0.2
      # - Stepper@1.1.3

custom libraries:

$ ls ansible/roles/sketch_library/
README.md  board_setup.cpp  board_setup.h  game_config.cpp  game_config.h  game_sequence.cpp  game_sequence.h  game_submit.cpp  game_submit.h  library.properties  wifi_connect.cpp  wifi_connect.h


Ex: Module/ custom library to setup wifi and display it on LED matrix

ansible/roles/sketch_library/wifi_connect.cpp

// Task: Implement WiFi connection and IP display
// Description: Connects to WiFi using provided SSID and password, validates connection,
// and scrolls the IP address on the LED matrix for Arduino R4 WiFi.

#include "cierto_connect.h"

bool connectToWiFi(const char* ssid, const char* pass, ArduinoLEDMatrix& matrix) {
  int status = WL_IDLE_STATUS;
  Serial.print("Connecting to SSID: ");
  Serial.println(ssid);
  while (status != WL_CONNECTED) {
    status = WiFi.begin(ssid, pass);
    delay(10000);
  }
  Serial.println("Connected to WiFi");

  IPAddress ip = WiFi.localIP();
  String ipStr = ipToString(ip);
  Serial.print("IP Address: ");
  Serial.println(ipStr);
  scrollText(matrix, ipStr);
  return true;
}

String ipToString(IPAddress ip) {
  return String(ip[0]) + "." + String(ip[1]) + "." + String(ip[2]) + "." + String(ip[3]);
}

void scrollText(ArduinoLEDMatrix& matrix, String text) {
  matrix.beginDraw();
  matrix.stroke(0xFFFFFFFF);
  matrix.textScrollSpeed(50);
  matrix.textFont(Font_5x7);
  matrix.beginText(0, 1, 0xFFFFFF);
  matrix.print("IP: " + text + " ");
  matrix.endText(SCROLL_LEFT);
  matrix.endDraw();
}

ansible/roles/sketch_library/wifi_connect.h

#ifndef WIFI_CONNECT_H
#define WIFI_CONNECT_H

// Task: Declare WiFi connection and IP display functions
// Description: Provides functions to connect to WiFi and display the IP address on the LED matrix
// for Arduino R4 WiFi, used in shuffleboard sketches.

#include <WiFiS3.h>
#include <ArduinoGraphics.h>
#include <Arduino_LED_Matrix.h>

bool connectToWiFi(const char* ssid, const char* pass, ArduinoLEDMatrix& matrix);
String ipToString(IPAddress ip);
void scrollText(ArduinoLEDMatrix& matrix, String text);

#endif

Questions:

  1. I am still struggling to get base libraries to load correct in CI workflow. You mentioned a use of github to take all libraries and validate this. I think what I stuggle with is I can get a board to work with UI and get it to push sketch fine, when when it comes to taking those libraries and know fields to put into something like library_profile.yaml it gets messy.

Ex: sketch works fine that notes:
#include <WiFiS3.h>
#include <ArduinoHttpClient.h>
#include <ArduinoJson.h>
#include <ArduinoGraphics.h>
#include <Arduino_LED_Matrix.h>

But to take that into library_profile.yaml

Do I put in "WiFiS3@1.04" or "R4HttpClient@1.0.4"

"ArduinoHttpClient" or "ArduinoHttpClient@0.6.1"

  1. I want to leverge this custom library logic to break apart complex or common tasks all the boards will use into modules/"custom libraries". But I think I need / should pull out the libraires they note into the upper level ansible/roles/arduino-library-install/library_profile.yaml such that the only thing in those libraries are just code for the task and all library staging and loading is done before. Is this correct?

For now, forget all about this ansible thing and focus your attention exclusively on build profiles. It doesn't make sense to involve all this complex ansible stuff in the matter when you don't even understand fundamentals of Arduino sketch dependencies.

Just work on being able to manually compile a sketch using a build profile. Once you are able to accomplish that, you can then start to work on using that knowledge in the context of your ansible thing.

Why don't you just have Arduino CLI generate the profile for you? Just compile the sketch without using a build profile and add the --dump-profile flag to the command:

arduino-cli compile --fqbn arduino:renesas_uno:unor4wifi --dump-profile

The command will output the build profile for the sketch you compiled, and then you only need to copy that into the sketch project file.

As I already explained multiple times, this is a platform bundled library. So it is provided by the platform dependency:

profiles:
  some-profile:
    platforms:
      - platform: arduino:renesas_uno (1.4.1)

Please provide a detailed description of what it is that makes you think your sketch has a dependency on the "R4HttpClient" library.

Is it only that you typed the "WiFiS3" keyword into Library Manager and saw that library in the search result???

Neither. The dependency specification format is <identifier> (<version>).

So if your sketch has a dependency on version 0.6.1 of the "ArduinoHttpClient" library, you would have this in your profile:

profiles:
  some-profile:
    libraries:
      - ArduinoHttpClient (0.6.1)