ESP32S-S3-DevkitC-1... Hard work

Projects General Guidance
21

Does anyone know of a decent library with examples that I can play about with?

I have binned the HORRIBLE ESP32 S3 and gone back to a standard ESP32 Wroom 1.

I have an SDcard connected, and an external SPIFFs memory IC (w25q128fv).

I have tried the following libraries...

SparkFun_SPI_SerialFlash - This reads the memory, but there are no other examples apart from reading the chip information.
Effortless_SPIFFS - No configuration options and no examples
Serialflash library - doesn't compile at all - LOADS of errors
Adafruit_SPIFlash - Doesn't compile
ESPflash - works, but no useful examples and cannot access external SPI (no idea how)
SPIMemory - None of the examples seem to work

Achieved:
Play an MP3 file from the onboard SPIFFS memory. Done that. All fine.
Play an MP3 file from the SD card. Done that. All fine.

Not achieved (due to not working out how to access the external SPI chip correctly)
Transfer MP3 files from the SD card to the external SPIFFS memory
Play an MP3 file from the external SPIFFS memory file.

All the libraries either don't have examples that work, or not enough information for me to deduce what the issue is. Or.. not enough examples.

The SparkFun_SPI_SerialFlash library works great, but there is only a read example.

The best set of examples are in the Serialflash library. But it throws up a pile of errors. Is it meant for Teensy's only?

For example, the Copy SD to Spiffs example throw up:

C:\Users\steve\Documents\Arduino\ESP32 Multi-function board with sound 2023\Prototype_board_v12\Prototype_board_v12.ino: In function 'void loop()':
Prototype_board_v12:225:28: error: 'filename' was not declared in this scope
     if (SerialFlash.create(filename, length)) {
                            ^~~~~~~~
C:\Users\steve\Documents\Arduino\ESP32 Multi-function board with sound 2023\Prototype_board_v12\Prototype_board_v12.ino:225:28: note: suggested alternative: 'basename'
     if (SerialFlash.create(filename, length)) {
                            ^~~~~~~~
                            basename
Prototype_board_v12:225:38: error: 'length' was not declared in this scope
     if (SerialFlash.create(filename, length)) {
                                      ^~~~~~
C:\Users\steve\Documents\Arduino\ESP32 Multi-function board with sound 2023\Prototype_board_v12\Prototype_board_v12.ino:225:38: note: suggested alternative: 'long'
     if (SerialFlash.create(filename, length)) {
                                      ^~~~~~
                                      long
Prototype_board_v12:234:15: error: 'f' was not declared in this scope
           n = f.read(buf, 256);
               ^
Prototype_board_v12:251:5: error: 'f' was not declared in this scope
     f.close();
     ^
C:\Users\steve\Documents\Arduino\ESP32 Multi-function board with sound 2023\Prototype_board_v12\Prototype_board_v12.ino:251:5: note: suggested alternative: 'fs'
     f.close();
     ^
     fs
'filename' was not declared in this scope
22

If this is the error message, then clearly 'filename' was not declared at all in your sketch,
something like

String filename = "testfile";

or

char *filename = "testfile";

may fix that already.
'length' should probably be of unsigned long type, and it's value should be the same as the size of the source file.
The rest are just cascading i guess, but without the actual code i can't be sure.

It could have served another purpose in the future.

23

I would have paid the postage to get it to me !

1 Like
24

I am sure the S3 will be amazing.... but at the moment, literally nothing seems to be friends with it, and I am not clever enough to re-write libraries to get it working.
I was designing a PCB and intended to use it, but... nope!

As for the SerialFlash errors. I did declare the missing variables, but it still failed.
I got to a point where I concluded it just wasn't going to work.

If I remember rightly...
File filename,
int length,
File F

It then had a problem with other items. f, ff, compareFiles etc

How come none of these are declared in the examples?

25

Yea... this is the CopyFromSD example...

I added:

File f;
File ff;
File filename;
File compareFiles;
int length;

But then all hell breaks loose... compareFiles seems to be used as 2 different things.

CopyFromSD:110:53: error: 'bool compareFiles(fs::File&, SerialFlashFile&)' redeclared as different kind of symbol
 bool compareFiles(File &file, SerialFlashFile &ffile) {
                                                     ^
C:\Users\steve\AppData\Local\Temp\arduino_modified_sketch_795764\CopyFromSD.ino:11:6: note: previous declaration 'fs::File compareFiles'
 File compareFiles;
      ^~~~~~~~~~~~
C:\Users\steve\AppData\Local\Temp\arduino_modified_sketch_795764\CopyFromSD.ino: In function 'void setup()':
CopyFromSD:56:31: error: no match for call to '(fs::File) (fs::File&, SerialFlashFile&)'
         if (compareFiles(f, ff) == true) {
                               ^
CopyFromSD:85:22: error: invalid conversion from 'char*' to 'uint8_t*' {aka 'unsigned char*'} [-fpermissive]
           n = f.read(buf, 256);
                      ^~~
In file included from C:\Users\steve\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.8\libraries\SD\src/SD.h:17,
                 from C:\Users\steve\AppData\Local\Temp\arduino_modified_sketch_795764\CopyFromSD.ino:2:
C:\Users\steve\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.8\libraries\FS\src/FS.h:60:26: note:   initializing argument 1 of 'size_t fs::File::read(uint8_t*, size_t)'
     size_t read(uint8_t* buf, size_t size);
                 ~~~~~~~~~^~~
C:\Users\steve\AppData\Local\Temp\arduino_modified_sketch_795764\CopyFromSD.ino: At global scope:
CopyFromSD:110:53: error: 'bool compareFiles(fs::File&, SerialFlashFile&)' redeclared as different kind of symbol
 bool compareFiles(File &file, SerialFlashFile &ffile) {
                                                     ^
C:\Users\steve\AppData\Local\Temp\arduino_modified_sketch_795764\CopyFromSD.ino:11:6: note: previous declaration 'fs::File compareFiles'
 File compareFiles;
      ^~~~~~~~~~~~
C:\Users\steve\AppData\Local\Temp\arduino_modified_sketch_795764\CopyFromSD.ino: In function 'bool compareFiles(fs::File&, SerialFlashFile&)':
CopyFromSD:118:15: error: invalid conversion from 'char*' to 'uint8_t*' {aka 'unsigned char*'} [-fpermissive]
     file.read(buf1, n);
               ^~~~
In file included from C:\Users\steve\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.8\libraries\SD\src/SD.h:17,
                 from C:\Users\steve\AppData\Local\Temp\arduino_modified_sketch_795764\CopyFromSD.ino:2:
C:\Users\steve\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.8\libraries\FS\src/FS.h:60:26: note:   initializing argument 1 of 'size_t fs::File::read(uint8_t*, size_t)'
     size_t read(uint8_t* buf, size_t size);
                 ~~~~~~~~~^~~
'bool compareFiles(fs::File&, SerialFlashFile&)' redeclared as different kind of symbol

Don't think this is a winner. Time to back out the door I think
It would help if it didn't take 3 minutes to compile each time :face_with_raised_eyebrow:

26

Well it doesn't make sense to what you added, let alone that the compiler will accept it. The 'name of a file (filename) is not of the type File of course
'f' does appear to be, but declaring it without reference to where (on which device) that file is will not help the compiler.
With the sketch

and at least a link to the library

There would be some way for us to also have a look.

The same here, mind you i don't have your hardware for testing, but sometimes the SRC of a library reveals many more options.

Well yes, without the details required, and with the understanding of the skill level you have, i think i feel the same way. Sorry i couldn't be of more help.

27

How do you link to a library? Find it on Github? I am just pulling them out of the Arudino IDE library.

I KNOW what I added makes no sense.... but I am stabbing in the dark

I checked the Sparkfun_SPI_SerialFlash library. It's all they have.

28
29

Basically, these libraries compile on anything except ESP32's

30

well the first thing that caught my eye in the comments was

 * Development of this library was funded by PJRC.COM, LLC by sales of Teensy.
 * Please support PJRC's efforts to develop open source software by purchasing
 * Teensy or other genuine PJRC products.

So basically they are written for teensy as a lot of his libraries are, although they may compile and work on others.
mind you in the Arduino reference it states

This library is compatible with all architectures so you should be able to use it on all the Arduino boards.

Saying that would have been enough.

With the assistance of google i found also
SPIMemory
which is more barebones than anything i think.
anyway i installed SerialFlash.h and opened 'CopyFromSD' and on a standard ESP32 dev board it compiles after i change a few 'char' declarations into 'uint8_t' They are basically the same thing, but clearly the 'char' on an ESP32 could also be something else, maybe like UTF16 or something.
But then it compiled.
And then i did for the ESP32S3 Dev Module, and it did take forever (first time i compiled something for that board) but it compiled as well, so i am just not sure where your compile errors are coming from

#include <SerialFlash.h>
#include <SD.h>
#include <SPI.h>

const int SDchipSelect = 4;    // Audio Shield has SD card CS on pin 10
const int FlashChipSelect = 6; // digital pin for flash chip CS pin
//const int FlashChipSelect = 21; // Arduino 101 built-in SPI Flash

void setup() {
  //uncomment these if using Teensy audio shield
  //SPI.setSCK(14);  // Audio shield has SCK on pin 14
  //SPI.setMOSI(7);  // Audio shield has MOSI on pin 7

  //uncomment these if you have other SPI chips connected
  //to keep them disabled while using only SerialFlash
  //pinMode(4, INPUT_PULLUP);
  //pinMode(10, INPUT_PULLUP);

  Serial.begin(9600);

  // wait up to 10 seconds for Arduino Serial Monitor
  unsigned long startMillis = millis();
  while (!Serial && (millis() - startMillis < 10000)) ;
  delay(100);
  Serial.println("Copy all files from SD Card to SPI Flash");

  if (!SD.begin(SDchipSelect)) {
    error("Unable to access SD card");
  }
  if (!SerialFlash.begin(FlashChipSelect)) {
    error("Unable to access SPI Flash chip");
  }

  int count = 0;
  File rootdir = SD.open("/");
  while (1) {
    // open a file from the SD card
    Serial.println();
    File f = rootdir.openNextFile();
    if (!f) break;
    const char *filename = f.name();
    Serial.print(filename);
    Serial.print("    ");
    unsigned long length = f.size();
    Serial.println(length);

    // check if this file is already on the Flash chip
    if (SerialFlash.exists(filename)) {
      Serial.println("  already exists on the Flash chip");
      SerialFlashFile ff = SerialFlash.open(filename);
      if (ff && ff.size() == f.size()) {
        Serial.println("  size is the same, comparing data...");
        if (compareFiles(f, ff) == true) {
          Serial.println("  files are identical :)");
          f.close();
          ff.close();
          continue;  // advance to next file
        } else {
          Serial.println("  files are different");
        }
      } else {
        Serial.print("  size is different, ");
        Serial.print(ff.size());
        Serial.println(" bytes");
      }
      // delete the copy on the Flash chip, if different
      Serial.println("  delete file from Flash chip");
      SerialFlash.remove(filename);
    }

    // create the file on the Flash chip and copy data
    if (SerialFlash.create(filename, length)) {
      SerialFlashFile ff = SerialFlash.open(filename);
      if (ff) {
        Serial.print("  copying");
        // copy data loop
        unsigned long count = 0;
        unsigned char dotcount = 9;
        while (count < length) {
          uint8_t buf[256];
          unsigned int n;
          n = f.read(buf, 256);
          ff.write(buf, n);
          count = count + n;
          Serial.print(".");
          if (++dotcount > 100) {
             Serial.println();
             dotcount = 0;
          }
        }
        ff.close();
        if (dotcount > 0) Serial.println();
      } else {
        Serial.println("  error opening freshly created file!");
      }
    } else {
      Serial.println("  unable to create file");
    }
    f.close();
  }
  rootdir.close();
  delay(10);
  Serial.println("Finished All Files");
}


bool compareFiles(File &file, SerialFlashFile &ffile) {
  file.seek(0);
  ffile.seek(0);
  unsigned long count = file.size();
  while (count > 0) {
    uint8_t buf1[128], buf2[128];
    unsigned long n = count;
    if (n > 128) n = 128;
    file.read(buf1, n);
    ffile.read(buf2, n);
    if (memcmp(buf1, buf2, n) != 0) return false; // differ
    count = count - n;
  }
  return true;  // all data identical
}


void loop() {
}

void error(const char *message) {
  while (1) {
    Serial.println(message);
    delay(2500);
  }
}

This compiles ! Whether it works or not is a different matter, i think this assumes SD & FLASH on the same SPI bus with a different CS pin, but i don't know. As said before i don't have your hardware.

31

Thank you for that!

That code does indeed complile. Doesn't work... but at least it compiles!

I will have look later to see what the issue is.

Everything is indeed on the same SPI bus, just different CS pins

ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1184
load:0x40078000,len:13220
ho 0 tail 12 room 4
load:0x40080400,len:3028
entry 0x400805e4
Copy all files from SD Card to SPI Flash

System Volume Information    0
  unable to create file

Test.mp3    10141906
  unable to create file

Goodbye.mp3    7920960
  unable to create file

Wavfile1.wav    95609350
  unable to create file

Wavfile2.wav    67496626
  unable to create file

Finished All Files
32

OK. I added SD card directory list and the SD card seems to be working fine.

It's failing to create the SPIFFS file

This bit has the issue....

    if (SerialFlash.create(filename, length)) {                                               // create the file on the Flash chip and copy data
      SerialFlashFile ff = SerialFlash.open(filename);
      if (ff) {
        Serial.print("  copying");
        // copy data loop
        unsigned long count = 0;
        unsigned char dotcount = 9;
        while (count < length) {
          uint8_t buf[256];
          unsigned int n;
          n = f.read(buf, 256);
          ff.write(buf, n);
          count = count + n;
          Serial.print(".");
          if (++dotcount > 100) {
            Serial.println();
            dotcount = 0;
          }
        }
        ff.close();
        if (dotcount > 0) Serial.println();
      } else {
        Serial.println("  error opening freshly created file!");
      }
    } else {
      Serial.println("  unable to create file");
    }
    f.close();
33

Hmm i always have the issue that the SD library doesn't disable and re-enable itself when not in use (eg it doesn't set the CS pin HIGH and LOW ) but that doesn't explain

if (!SerialFlash.begin(FlashChipSelect)) {
    error("Unable to access SPI Flash chip");
  }

That this condition is true, And chances are the SerialFlash also doesn't do that. I never got anything to work like that, but that makes no sense given this is an example included with the library.
Anyway the obvious thing to do first is to see if you can create a file on the Flash while the SD-card is not connected, to confirm that this is or isn't the issue.
Other option is of course that there is a '/' missing at the start of the filename, which is a required part of the filename, but not included in the return of

const char *filename = f.name();

Basically try and create a new file on the FLASH called something like

/ThisNewFile.txt

or something like that.

edit: even on a SD card the '/' before the start of the name is required or the file can not be created, so the example is a tad suspect to say the least, but with some modification it might work, If you can get it to work, it will be time to get in touch with the Author.

34

Thank you very much for your assistance.

I will indeed play around this evening.

It does pass the SPI check at the beginning, but still does so if I disconnect the CS pin of the external SPIFFs memory.
Therefore, it must be checking the internal SPIFFs memory on the EPS32 by default? (Even though the CS pin is stated at boot).

Hmm... this is all a little out of my depth, but I will tinker on.

If it cannot access the SPIFFs, then there is the first issue (although why does it pass the check?)

---- update -----

Doesn't matter what pin I set as the CS for the SPIFFS, it still passed the initialisation check

35

That is a bit confusing to me as well, maybe the SRC of the library will provide some clues.

Ah ! (from SerialFlashChip.cpp)

bool SerialFlashChip::begin(uint8_t pin)
{
	uint8_t id[5];
	uint8_t f;
	uint32_t size;

	cspin_basereg = PIN_TO_BASEREG(pin);
	cspin_bitmask = PIN_TO_BITMASK(pin);
	SPIPORT.begin();
	pinMode(pin, OUTPUT);
	CSRELEASE();
	readID(id);
	f = 0;
	size = capacity(id);
	if (size > 16777216) {
		// more than 16 Mbyte requires 32 bit addresses
		f |= FLAG_32BIT_ADDR;
		SPIPORT.beginTransaction(SPICONFIG);
		if (id[0] == ID0_SPANSION) {
			// spansion uses MSB of bank register
			CSASSERT();
			SPIPORT.transfer16(0x1780); // bank register write
			CSRELEASE();
		} else {
			// micron & winbond & macronix use command
			CSASSERT();
			SPIPORT.transfer(0x06); // write enable
			CSRELEASE();
			delayMicroseconds(1);
			CSASSERT();
			SPIPORT.transfer(0xB7); // enter 4 byte addr mode
			CSRELEASE();
		}
		SPIPORT.endTransaction();
		if (id[0] == ID0_MICRON) f |= FLAG_MULTI_DIE;
	}
	if (id[0] == ID0_SPANSION) {
		// Spansion has separate suspend commands
		f |= FLAG_DIFF_SUSPEND;
		if (!id[4]) {
			// Spansion chips with id[4] == 0 use 256K sectors
			f |= FLAG_256K_BLOCKS;
		}
	}
	if (id[0] == ID0_MICRON) {
		// Micron requires busy checks with a different command
		f |= FLAG_STATUS_CMD70; // TODO: all or just multi-die chips?
	}
	flags = f;
	readID(id);
	return true;
}

It always returns 'true' and never tests for anything, well that does explain that.

Ok so i think you should explicitly switch the CS pins.

if (!SD.begin(SDchipSelect)) {
    error("Unable to access SD card");
  }
  digitalWrite(SDchipSelect, HIGH);
  if (!SerialFlash.begin(FlashChipSelect)) {
    error("Unable to access SPI Flash chip");
  }
 digitalWrite(FlashChipSelect, HIGH);

and before any operation on either.

digitalWrite(SDchipSelect, LOW);
File f = rootdir.openNextFile();
    if (!f) break;
    const char *filename = f.name();
    Serial.print(filename);
    Serial.print("    ");
    unsigned long length = f.size();
    Serial.println(length);

    // check if this file is already on the Flash chip
    digitalWrite(SDchipSelect, HIGH);
    digitalWrite(FlashChipSelect, LOW);
    if (SerialFlash.exists(filename)) {

and so forth... Unless there is something in the library doing it, which for the SD library i am really there is not, then it should be done manually.
edit :

CSASSERT();
SPIPORT.transfer(0xB7); // enter 4 byte addr mode
CSRELEASE();

This does look like it is done in SerialFlash though
You can not have more than 1 SPI device enabled on the same bus.
Then there is the issue of the filename.
I did also read through the ReadMe of SerialFlash, and that did inform me of some interesting stuff as well, but no mention of the '/' anywhere, so that may not be it. (still that is also an absolute requirement for the SD library)

36

Well I just tried the 'EraseEverything' example for the flash memory in the examples.
That appeared to work.

Then I tried the RawHardwareTest in examples and that fails for some reason. It returns different values to what it wrote.
Also Serial.begin was missing from the Setup, so no wonder it didn't print any results..... do they check these goddam libraries!

Hmm... think I might be down a rabbit hole here.

What is the point of a check then if it tests for nothing? Brilliant.

I'll try manually switching the CS pins, but I think I may call it a day then. I am SO far away from achieving my goal that I just don't think I will get there.

I am surprised at the lack of support for the ESP32 still in these libraries.

37

OK. I don't think I am going to get this Teensy based library converted over.

Manually toggling the CS pins makes no difference and none of the other examples work correctly either. If I uncomment the SPI.setMOSI(MOSI), SPI.setMISO(MISO), SPI.setSCK(SCK) for instance, I get reams of errors declaring class SPIClass' has no member named 'setMOSI' etc.

Life's too short!

Thanks for all your help, but I think ESP32's are going back in the box of processors until the libraries catch up

38

Well it appears that the teensy support is actually funded by the sale of teensy boards, (chances are the teensy SD library does switch CS pins, but i know for sure that on an esp32 it doesn't) but there are just so many boards available. I think Espressif is supporting the core development, and for some other chip manufacturers is may also be true, but really those guys actually just sell chips, and do not really built boards at all, unlike Arduino & teensy. There are quite a few quirks with ESP32's and 8266's and even the standard firmware is all that they really come with. Quite a few libraries do work these MCU's though, and it is still a lot more than the new Arduino boards.

39

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.