ESP8266 and SdFat/ESP8266SdFat and compiler library selection

Arduino ide 1.8.13
ESP8266 arduino core 2.7.4
Lolin D1 mini but fails for other platforms

ReadWrite example program from examples>ESP8266SdFat>ReadWrite

#include <SPI.h>
//#include <SD.h>
#include "SdFat.h"

using namespace sdfat;

SdFat SD;

#define SD_CS_PIN SS
File myFile;

void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

  Serial.print("Initializing SD card...");

  if (!SD.begin(SD_CS_PIN)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");

  // open the file. note that only one file can be open at a time,
  // so you have to close this one before opening another.
  myFile = SD.open("test.txt", FILE_WRITE);

  // if the file opened okay, write to it:
  if (myFile) {
    Serial.print("Writing to test.txt...");
    myFile.println("testing 1, 2, 3.");
    // close the file:
    myFile.close();
    Serial.println("done.");
  } else {
    // if the file didn't open, print an error:
    Serial.println("error opening test.txt");
  }

  // re-open the file for reading:
  myFile = SD.open("test.txt");
  if (myFile) {
    Serial.println("test.txt:");

    // read from the file until there's nothing else in it:
    while (myFile.available()) {
      Serial.write(myFile.read());
    }
    // close the file:
    myFile.close();
  } else {
    // if the file didn't open, print an error:
    Serial.println("error opening test.txt");
  }
}

void loop() {
  // nothing happens after setup
}

With SdFat in the Arduino user added libraries folder in the sketchbook the compiler uses it over ESP8266SdFat and the example program fails to compile with multiple errors related to the SdFat library.

Multiple libraries were found for “SdFat.h”
Used: C:\Users\myName\Documents\Arduino\libraries\SdFat
Not used: C:\Users\myName\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.7.4\libraries\ESP8266SdFat

If I rename SdFat to XXSdFat, the compiler makes the correct choice and the sketch compiles without error, like when I remove the SdFat.h from my libraries folder completely.

Multiple libraries were found for “SdFat.h”
Used: C:\Users\myName\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.7.4\libraries\ESP8266SdFat
Not used: C:\Users\myName\Documents\Arduino\libraries\XXSdFat

I have had this library choice error show up with several different sketches, and appears similar to the issue described in

Can anyone confirm this experience? Clearly there is a workaround, but if the basic example for the use of SdFat with the esp8266 won’t compile when SdFat.h is in the user libraries folders for the AVR boards it may be a bug worth fixing.

The explanation for the behavior is here:
https://arduino.github.io/arduino-cli/latest/sketch-build-process/#dependency-resolution

When multiple libraries contain a file matching an #include directive, the Arduino IDE has to make a decision as to which of them to use. The priority is based on several factors:

“Architecture Matching” (Sketch build process - Arduino CLI)

The architecture identifier for your Lolin D1 mini is “esp8266”.
The architecture compatibility defined in the metadata of the ESP8266SdFat library:

architectures=*

and for the SDFat library:

architectures=*

So the architecture matching is a tie and we must move on to the next factor:
“Folder Name Priority” (Sketch build process - Arduino CLI)
We can see the folder names from the output you shared:

Multiple libraries were found for “SdFat.h”
Used: C:\Users\myName\Documents\Arduino\libraries\SdFat
Not used: C:\Users\myName\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.7.4\libraries\ESP8266SdFat

So the SdFat library has a perfect folder name match, and thus gets priority over the mismatched ESP8266SdFat folder name.

So the Arduino build system is working just as is intended. The bug would be with the ESP8266SdFat library. Considering the library name, the wildcard architecture specification seems questionable. If that was changed to “esp8266” then the library would have an explicit architecture match and get priority.

Another option would be to provide a folder name match. This would be as simple as adding a wrapper header file to the ESP8266SdFat library:

ESP8266SdFat.h

#include "SdFat.h"

Thank you @pert. As usual you have been very helpful.

The architecture compatibility defined in the metadata of the ESP8266SdFat library:
ESP8266SdFat/library.properties at 0a46e4ebb2de585c5a64f981dbc2b2223a438984 · earlephilhower/ESP8266SdFat · GitHub

However, when I look at the actual library properties for the esp8266SdFat.h bundled with the esp8266 2.7.4 core in Arduino15

C:\Users\myName\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.7.4\libraries\ESP8266SdFat\library.properties

I see an older version than what you cited and it has the esp8266 specific architecture.

name=ESP8266SdFat
version=1.1.0
license=MIT
author=Bill Greiman <fat16lib@sbcglobal.net>
maintainer=Earle F. Philhower, III <earlephilhower@yahoo.com>
sentence=Fork of Bill Greiman's SdFat library <https://github.com/greiman/SdFat> with fixes for ESP8266 Arduino support.
paragraph=Minor tweak of Bill Greiman's SdFat FAT16/FAT32 file system for SD cards.  Modified to enclose all objects, constants, etc. in a unique namespace "sdfat" to make it easier to integrate with the ESP8266's own File class.
category=Data Storage
url=https://github.com/earlephilhower/ESP8266SdFat
repository=https://github.com/earlephilhower/ESP8266SdFat.git
architectures=esp8266
dot_a_linkage=true

I will absolve the build process and log a bug against ESP8266SdFat if the architectural specification is not relevant.

EDIT:

I think I now understand Architecture Matching a little better.

A library that is architecture compatible wins against a library that is not architecture compatible (see Architecture Matching)

Both the explicit architectures = ESP8266 in the esp8266SdFat and the generic architecture of SdFat = * are both architecture compatible and the build process carries on with criteria further down on the list.

Why does not a specific architecture compatability take precedence over a * compatability?

The bug would be with the ESP8266SdFat library. Considering the library name, the wildcard architecture specification seems questionable. If that was changed to “esp8266” then the library would have an explicit architecture match and get priority.

Based on this suggestion, the library author changed the architecture specification in a recent github commit.

name=ESP8266SdFat
version=2.0.2
license=MIT
author=Bill Greiman <fat16lib@sbcglobal.net>
maintainer=Bill Greiman <fat16lib@sbcglobal.net>
sentence=FAT16/FAT32/exFAT file system.
paragraph=FAT16/FAT32/exFAT file system.
category=Data Storage
url=https://github.com/greiman/SdFat
repository=https://github.com/greiman/SdFat.git
architectures=esp8266

I have been testing this proposed solution and placed the new properties file in the library in 2.7.4 and find that it does not solve the library selection issue when SdFat is present in the user library folder. As previously surmised, when both architectures are valid, the name match still prevails.

Multiple libraries were found for “SdFat.h”
Used: C:\Users\myName\Documents\Arduino\libraries\SdFat
Not used: C:\Users\myName\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.7.4\libraries\ESP8266SdFat

Another option would be to provide a folder name match. This would be as simple as adding a wrapper header file to the ESP8266SdFat library:

I’m not certain about the specifics of the wrapper header file solution, but if the name of the library folder in the esp core is changed from ESP8266SdFat to simple SdFat
and the architectural specification is left at architectures=esp8266 then the build process selects the correct library.

Multiple libraries were found for "SdFat.h"
 Used: C:\Users\myName\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.7.4\libraries\SdFat
 Not used: C:\Users\myName\Documents\Arduino\libraries\SdFat

I am unclear about any other consequences of this folder renaming and it certainly confuses the issue as to this library being a branch of the original SdFat.

It was recommended to leave this discussion here on the forum instead of back on git hub, so lets see where this leads.

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.