ESP32 Loading a sequence of .jpg images from SPIFFS

Hello,

I'd like to do a startup screen animation for my trip computer, and for that purpose, I have uploaded a series of jpegs into SPIFFS memory. They're about 120 kB altogether.

Now, how do I correctly reference those images for display on the screen one-by-one? The jpeg files are named with ascending consecutive numbers from 1.jpg to 31.jpg.

I've tried using a char array, but this did not work:

for (int i = 1; i <= 31; i++) {

    String fileName = "/"+i+".jpg";
    char jpegs[filename.length()+1];
    fileName.toCharArray(jpegs, sizeof(jpegs);
    
    
    drawJpeg(jpegs, 0, 0);
    delay(80);

  }

(the rest of the code that is omitted here is basically a condensed version of the TFT_SPIFFS_Jpeg examples from the TFT_eSPI library).

How do I construct file name references that I can then use to retrieve the images one by one from SPIFFS using a for loop?

  • carguy

I think one problem you may be having is that using String concatenation ( + ) with different data types can lead to unexpected results when the String has not been initialized. Click below for more information on this.

I think the below code may work but it might not be the best solution. The String() function returns an instance of the String class which is concatenated with the constant strings and stored in the fileName object.

for (int i = 1; i <= 31; i++) {

    String fileName = "/"+ String(i) +".jpg";
    char jpegs[filename.length()+1];
    fileName.toCharArray(jpegs, sizeof(jpegs);
    
    
    drawJpeg(jpegs, 0, 0);
    delay(80);

  }

There is no need to create the filenames if you use the SPIFFS functions to iterate through the files in a directory

#include "FS.h"

void setup()
{
  Serial.begin(115200);
  SPIFFS.begin();
  Dir dir = SPIFFS.openDir("/");
  while (dir.next())
  {
    fileName = dir.fileName();
    Serial.print(dir.fileName());
  }
}

The problem is that I will probably be storing various different jpegs in SPIFFS which will have nothing to do with my startup screen animation. If I'd just go through all the files in the "flat" SPIFFS, it would lead to unwanted images being displayed. Also, there will be a number of small .mp3 files in SPIFFS, which the loop would probably also go through (?) and try to open them as jpegs unsuccessfully...

I've tried out @chadwr's code snippet, but now that that part of the code compiles, there is more trouble about. The sketch compiles and uploads, but the ESP32 panics on boot, giving this message:

Guru Meditation Error: Core  1 panic'ed (IntegerDivideByZero)
. Exception was unhandled.
Register dump:
PC      : 0x400d0cae  PS      : 0x00060730  A0      : 0x800d0e4c  A1      : 0x3ffcad60  
A2      : 0x3ffc2c04  A3      : 0x00000000  A4      : 0x00000000  A5      : 0x3f405286  
A6      : 0x00000000  A7      : 0xff000000  A8      : 0x800d3b60  A9      : 0x00000000  
A10     : 0x3ffc736c  A11     : 0xffffffff  A12     : 0x3ffc2c1c  A13     : 0x3f405286  
A14     : 0x00000072  A15     : 0xff000000  SAR     : 0x00000008  EXCCAUSE: 0x00000006  
EXCVADDR: 0x00000000  LBEG    : 0x4000c46c  LEND    : 0x4000c477  LCOUNT  : 0x00000000  

Backtrace: 0x400d0cae:0x3ffcad60 0x400d0e49:0x3ffcadb0 0x400d0f4b:0x3ffcadf0 0x400efa0f:0x3ffcae50

Here is my code as it is so far. It's basically a condensed version of the examples found in the TFT_eSPI library. Looking at it, coud you think of a reason why it does that? I don't see any division by zero in the code. And commenting out my loop section discussed in this thread has no effect on the problem.

#define FS_NO_GLOBALS
#define minimum(a,b)     (((a) < (b)) ? (a) : (b))
#include <FS.h>
#include <JPEGDecoder.h>
#include <SPI.h>
#include <TFT_eSPI.h>
TFT_eSPI tft = TFT_eSPI();

void jpegRender(int xpos, int ypos) {

  // retrieve infomration about the image
  uint16_t  *pImg;
  uint16_t mcu_w = JpegDec.MCUWidth;
  uint16_t mcu_h = JpegDec.MCUHeight;
  uint32_t max_x = JpegDec.width;
  uint32_t max_y = JpegDec.height;

  uint32_t min_w = minimum(mcu_w, max_x % mcu_w);
  uint32_t min_h = minimum(mcu_h, max_y % mcu_h);

  // save the current image block size
  uint32_t win_w = mcu_w;
  uint32_t win_h = mcu_h;
  max_x += xpos;
  max_y += ypos;

  // read each MCU block until there are no more
  while ( JpegDec.readSwappedBytes()) { // Swap byte order so the SPI buffer can be used

    // save a pointer to the image block
    pImg = JpegDec.pImage;

    // calculate where the image block should be drawn on the screen
    int mcu_x = JpegDec.MCUx * mcu_w + xpos;  // Calculate coordinates of top left corner of current MCU
    int mcu_y = JpegDec.MCUy * mcu_h + ypos;

    // check if the image block size needs to be changed for the right edge
    if (mcu_x + mcu_w <= max_x) win_w = mcu_w;
    else win_w = min_w;

    // check if the image block size needs to be changed for the bottom edge
    if (mcu_y + mcu_h <= max_y) win_h = mcu_h;
    else win_h = min_h;

    // copy pixels into a contiguous block
    if (win_w != mcu_w)
    {
      uint16_t *cImg;
      int p = 0;
      cImg = pImg + win_w;
      for (int h = 1; h < win_h; h++)
      {
        p += mcu_w;
        for (int w = 0; w < win_w; w++)
        {
          *cImg = *(pImg + w + p);
          cImg++;
        }
      }
    }

    // draw image MCU block only if it will fit on the screen
    if ( ( mcu_x + win_w) <= tft.width() && ( mcu_y + win_h) <= tft.height())
    {
      tft.pushRect(mcu_x, mcu_y, win_w, win_h, pImg);
    }

    else if ( ( mcu_y + win_h) >= tft.height()) JpegDec.abort();
  }
}

void drawJpeg(const char *filename, int xpos, int ypos) {

  // Open the named file (the Jpeg decoder library will close it after rendering image)
  fs::File jpegFile = SPIFFS.open( filename, "r");    // File handle reference for SPIFFS
  //  File jpegFile = SD.open( filename, FILE_READ);  // or, file handle reference for SD library

  if ( !jpegFile ) {
    Serial.print("ERROR: File \""); Serial.print(filename); Serial.println ("\" not found!");
    return;
  }

  // Use one of the three following methods to initialise the decoder:
  //boolean decoded = JpegDec.decodeFsFile(jpegFile); // Pass a SPIFFS file handle to the decoder,
  //boolean decoded = JpegDec.decodeSdFile(jpegFile); // or pass the SD file handle to the decoder,
  boolean decoded = JpegDec.decodeFsFile(filename);  // or pass the filename (leading / distinguishes SPIFFS files)
  // Note: the filename can be a String or character array type
  if (decoded)  jpegRender(xpos, ypos);
  else Serial.println("Jpeg file format not supported!");
}

void setup() {

  Serial.begin(115200);
  SPIFFS.begin(true);
  tft.begin();
  tft.setRotation(2);
  tft.fillScreen(TFT_WHITE);

  for (int i = 1; i <= 31; i++) {

    String fileName = "/" + String(i) + ".jpg";
    char jpegs[fileName.length() + 1];
    fileName.toCharArray(jpegs, sizeof(jpegs));
    drawJpeg(jpegs, 0, 0);
    delay(80);
  }
}
void loop() {
}

The problem is that I will probably be storing various different jpegs in SPIFFS which will have nothing to do with my startup screen animation. If I'd just go through all the files in the "flat" SPIFFS, it would lead to unwanted images being displayed.

So put the startup images in their own directory and iterate through that instead of the root directory.

#include "FS.h"

String fileName;

void setup()
{
  Serial.begin(115200);
  SPIFFS.begin();
  Dir dir = SPIFFS.openDir("/startup/");
  while (dir.next())
  {
    fileName = dir.fileName();
    Serial.println(dir.fileName());
  }
}

void loop()
{
}

It would be preferable to concatenate String with a String e.g.

    String fileName = "/" + String(i) + ".jpg";
    char jpegs[filename.length()+1];
    fileName.toCharArray(jpegs, sizeof(jpegs);

All the same, I am suspicious of creating a dynamic char array with unknown size (at compile time)

If the SPIFFS library wants a char[] it would be much simpler to just say:

    char buf[10];    // big enough for "/9999.jpg"
    sprintf(buf, "/%d.jpg", i);

David.

As I understand it, the SPIFFS file sytem is "flat", meaning it supports no multi-level directory structures. Every file is stored on the top level, and wanting to save a file as "/jpegs/5.jpg" will not result in the file "5.jpg" being stored in the sub-folder "/jpegs/", but a file named "jpegs/5.jpg" being stored on the top level.

Speaking of SPIFFS, do you know if it can - or should - be used to save settings before a device goes into power off/sleep mode? My trip computer will be wired up so that it can sense when the ignition is off (it will be connected to permanent +12V from the battery), and in that event, I would like it to save some settings. Also, I would like to store settings that concern the trip computer's general appearance.

About like this (just made this up off the top of my head):

### screen appearance

appearance.language = EN
appearance.miles_or_kph = miles
appearance.carType = MkI
appearance.startupAni = on
appearance.soundAlerts = on
appearance.voiceAlerts = off
appearance.dayAndNightMode = both
appearance.lightSensorDayNightThreshold = 550

### connected sensors

sensors.speedo = true
sensors.engineRPM = true
sensors.fuelInjection = true
sensors.fuelTank = true
sensors.coolantTemp = true
sensors.coolantLevel = false
sensors.oilTemp = true
sensors.coolantLevel = false
sensors.lampCheckFront = false
sensors.lampCheckRear = false
sensors.cabinTemp = true
sensors.outsideTemp = true

### data saved on ignition-off

data.last.MPG = 35.5
data.last.avgSpeed = 48
data.last.totalMiles = 264
data.last.tripDriveTime = 27
data.last.totalDriveTime = 382

### menu structures

screen.fuel1.component1 = currentCons
screen.fuel1.component2 = fuelTankLevel

...

Do the SPIFFS libraries have functions that would allow the ESP32 to extract the settings from a settings.txt file on system startup?

Every file is stored on the top level, and wanting to save a file as "/jpegs/5.jpg" will not result in the file "5.jpg" being stored in the sub-folder "/jpegs/", but a file named "jpegs/5.jpg" being stored on the top level.

That is my understanding too. Try the example that I posted. Create a directory under the data folder in the sketch directory and put a couple of files in it. Upload the sketch files to the ESP8266 as usual and compile/run the program.

I get back a list of files that includes the fake directory name but only those in that directory even if there are others in the root.

You can do most things in SPIFFS. e.g. write a small file. It is up to you if you want to read a file at startup.

Writing large files is really SLOW. Reading any size file is FAST.

I am sure that the Flash endurance will far outlive your racing car.
So you can easily write log files for a journey.

David.

david_prentice:
You can do most things in SPIFFS. e.g. write a small file. It is up to you if you want to read a file at startup.

Writing large files is really SLOW. Reading any size file is FAST.

The idea is to read the file once at startup, and from that retrieve parameters that will be used while the trip computer is in running mode.

"data.last" Parameters would only be written to the file when the engine is turned off (the plan is to then let the trip computer go into deep sleep about a minute or two after engine power off. During that time, it should have ample time to save a handful of parameters).

The trip computer will also have a "settings" submenu where you can do such things as select the system language, miles or kilometers mode, register the connected sensors, etc.

The original plan was to do all that in EEPROM, but having spent the last two days reading up a bit on SPIFFS, it seems more practically to use that for this purpose.

So how exactly do you read that kind of information from a txt file in SPIFFS?

david_prentice:
I am sure that the Flash endurance will far outlive your racing car.

Happy to hear that you consider the MGF a racing car... :smiley: ... there is much divided opinion about that... coincidentally, now is a good time to get one in the UK. Specimens in good nick command no more than £1,500... and prices will go up again in a few years...

All red cars with minimal legroom are racing cars.

My next door neighbours have a Triumph Spitfire.

You read and write files in the regular way. I suggest that you do some experiments on your PC with a regular PC Compiler. Any learning process is easier on the PC.

David.

The Spitfire is also a nice little car... but the MGF is better in nearly every respect. Again, if you've got £1,500 to spare, this is one of the best ways to spend it on a car.

I'm a little rusty with the whole fread/frwite thing... I used to use it some, back in my days as a web designer/programmer (mainly worked with PHP and MySQL though)... I know how to open and read files, but I can't really remember how you retrieve a particular line from a settings file as the one I posted above. Say you want to know what value data.last.totalDriveTime currently has... or you want to write to the file and change that setting's value...

Unix files are byte oriented. Seek to some specified position and read N bytes.

Alternatively, write a complete struct as a byte stream. Known as Serialize.
Then read the whole block back into the struct.

Fine in principle but you have to worry about field alignment.

Do some experiments.
Personally, I would have fixed length fields with ascii data. i.e. your file would be human readable.
Binary data is more efficient but you need to parse it before a a human can understand.

David.

Right now, I am still having trouble getting the "TFT_SPIFFS_Jpeg" sketch by Bodmer to work.

I have only very gently modified it for my purposes. It compiles, and on the serial monitor it even tells me that the images were rendered. No errors are reported.

But the screen stays dark completely.

I've checked if there's something wrong with my TFT display, but it works fine when I try my other sketches for it.

Here's the code:

//====================================================================================
//                                  Libraries
//====================================================================================
// Call up the SPIFFS FLASH filing system this is part of the ESP Core
#define FS_NO_GLOBALS
#include <FS.h>

// JPEG decoder library
#include <JPEGDecoder.h>

// SPI library, built into IDE
#include <SPI.h>

// Call up the TFT library
#include <TFT_eSPI.h> // Hardware-specific library for ESP8266
// The TFT control pins are set in the User_Setup.h file <<<<<<<<<<<<<<<<< NOTE!
// that can be found in the "src" folder of the library

// Invoke TFT library
TFT_eSPI tft = TFT_eSPI();


/*====================================================================================
  This sketch contains support functions to render the Jpeg images.

  Created by Bodmer 15th Jan 2017
  ==================================================================================*/

// Return the minimum of two values a and b
#define minimum(a,b)     (((a) < (b)) ? (a) : (b))

//====================================================================================
//   Opens the image file and prime the Jpeg decoder
//====================================================================================
void drawJpeg(const char *filename, int xpos, int ypos) {

  Serial.println("===========================");
  Serial.print("Drawing file: "); Serial.println(filename);
  Serial.println("===========================");

  // Open the named file (the Jpeg decoder library will close it after rendering image)
   File jpegFile = SPIFFS.open( filename, "r");    // File handle reference for SPIFFS
  //  File jpegFile = SD.open( filename, FILE_READ);  // or, file handle reference for SD library

  if ( !jpegFile ) {
    Serial.print("ERROR: File \""); Serial.print(filename); Serial.println ("\" not found!");
    return;
  }

  // Use one of the three following methods to initialise the decoder:
  boolean decoded = JpegDec.decodeFsFile(jpegFile); // Pass a SPIFFS file handle to the decoder,
  //boolean decoded = JpegDec.decodeSdFile(jpegFile); // or pass the SD file handle to the decoder,
  //boolean decoded = JpegDec.decodeFsFile(filename);  // or pass the filename (leading / distinguishes SPIFFS files)
  // Note: the filename can be a String or character array type
  if (decoded) {
    // print information about the image to the serial port
    jpegInfo();

    // render the image onto the screen at given coordinates
    jpegRender(xpos, ypos);
  }
  else {
    Serial.println("Jpeg file format not supported!");
  }
}

//====================================================================================
//   Decode and render the Jpeg image onto the TFT screen
//====================================================================================
void jpegRender(int xpos, int ypos) {

  // retrieve infomration about the image
  uint16_t  *pImg;
  uint16_t mcu_w = JpegDec.MCUWidth;
  uint16_t mcu_h = JpegDec.MCUHeight;
  uint32_t max_x = JpegDec.width;
  uint32_t max_y = JpegDec.height;

  // Jpeg images are draw as a set of image block (tiles) called Minimum Coding Units (MCUs)
  // Typically these MCUs are 16x16 pixel blocks
  // Determine the width and height of the right and bottom edge image blocks
  uint32_t min_w = minimum(mcu_w, max_x % mcu_w);
  uint32_t min_h = minimum(mcu_h, max_y % mcu_h);

  // save the current image block size
  uint32_t win_w = mcu_w;
  uint32_t win_h = mcu_h;

  // record the current time so we can measure how long it takes to draw an image
  uint32_t drawTime = millis();

  // save the coordinate of the right and bottom edges to assist image cropping
  // to the screen size
  max_x += xpos;
  max_y += ypos;

  // read each MCU block until there are no more
  while ( JpegDec.readSwappedBytes()) { // Swap byte order so the SPI buffer can be used

    // save a pointer to the image block
    pImg = JpegDec.pImage;

    // calculate where the image block should be drawn on the screen
    int mcu_x = JpegDec.MCUx * mcu_w + xpos;  // Calculate coordinates of top left corner of current MCU
    int mcu_y = JpegDec.MCUy * mcu_h + ypos;

    // check if the image block size needs to be changed for the right edge
    if (mcu_x + mcu_w <= max_x) win_w = mcu_w;
    else win_w = min_w;

    // check if the image block size needs to be changed for the bottom edge
    if (mcu_y + mcu_h <= max_y) win_h = mcu_h;
    else win_h = min_h;

    // copy pixels into a contiguous block
    if (win_w != mcu_w)
    {
      uint16_t *cImg;
      int p = 0;
      cImg = pImg + win_w;
      for (int h = 1; h < win_h; h++)
      {
        p += mcu_w;
        for (int w = 0; w < win_w; w++)
        {
          *cImg = *(pImg + w + p);
          cImg++;
        }
      }
    }

    // draw image MCU block only if it will fit on the screen
    if ( ( mcu_x + win_w) <= tft.width() && ( mcu_y + win_h) <= tft.height())
    {
      tft.pushRect(mcu_x, mcu_y, win_w, win_h, pImg);
    }

    else if ( ( mcu_y + win_h) >= tft.height()) JpegDec.abort();

  }

  // calculate how long it took to draw the image
  drawTime = millis() - drawTime; // Calculate the time it took

  // print the results to the serial port
  Serial.print  ("Total render time was    : "); Serial.print(drawTime); Serial.println(" ms");
  Serial.println("=====================================");

}

//====================================================================================
//   Print information decoded from the Jpeg image
//====================================================================================
void jpegInfo() {

  Serial.println("===============");
  Serial.println("JPEG image info");
  Serial.println("===============");
  Serial.print  ("Width      :"); Serial.println(JpegDec.width);
  Serial.print  ("Height     :"); Serial.println(JpegDec.height);
  Serial.print  ("Components :"); Serial.println(JpegDec.comps);
  Serial.print  ("MCU / row  :"); Serial.println(JpegDec.MCUSPerRow);
  Serial.print  ("MCU / col  :"); Serial.println(JpegDec.MCUSPerCol);
  Serial.print  ("Scan type  :"); Serial.println(JpegDec.scanType);
  Serial.print  ("MCU width  :"); Serial.println(JpegDec.MCUWidth);
  Serial.print  ("MCU height :"); Serial.println(JpegDec.MCUHeight);
  Serial.println("===============");
  Serial.println("");
}


void setup()
{
  Serial.begin(250000); // Used for messages and the C array generator

  delay(10);
  Serial.println("NodeMCU decoder test!");

  tft.init();
  tft.setRotation(2);  // 0 & 2 Portrait. 1 & 3 landscape
  tft.fillScreen(TFT_WHITE);

  if (!SPIFFS.begin()) {
    Serial.println("SPIFFS initialisation failed!");
    while (1) yield(); // Stay here twiddling thumbs waiting
  }
  Serial.println("\r\nInitialisation done.");
 
}

//====================================================================================
//                                    Loop
//====================================================================================
void loop()
{

  for (byte i = 1; i <= 31; i++) {

    char jpegs[10];    // big enough for "/9999.jpg"
    sprintf(jpegs, "/%d.jpg", i);

    drawJpeg(jpegs, 0, 0);
  }
}

I have enclosed the sketch folder and all my jpegs. Don't be shocked at them being 150 kb at the moment; I will probably try to pare them down to around 100KB as the project progresses.

TFT_SPIFFS_Jpeg-carguy.zip (151 KB)

Getting back to this part of the project, I'm still trying to figure out how to display images on the TFT screen that are loaded from SPIFFS.

At the moment, I've got a sequence of sixteen .jpeg images stored in SPIFFS, named d1.jpg to d16.jpg, which I want to display in sequence as the start-up/welcome screen. Here's my code so far:

#include <SPIFFS.h>
#include <SPI.h>
#include <TFT_eSPI.h>


void setup() {

  tft.init();
  tft.setRotation(2);
  tft.fillScreen(0xffff);

  TFT_eSPI tft = TFT_eSPI();

  for (byte iStart = 1; iStart <= 16; iStart++) {

    char aniImage[7];
    sprintf(aniImage, "/d%2d.jpg", iStart);

    tft.pushImage(0,  0, 128, 160, aniImage);
    delay(120);
  }

}

void loop() {}

This code does not compile. The error I get is

HeadUnitESP32:75: error: invalid conversion from 'char*' to 'uint8_t* {aka unsigned char*}' [-fpermissive]

     tft.pushImage(0,  0, 128, 160, aniImage);

                                            ^

In file included from D:\Arduino\HeadUnitESP32\HeadUnitESP32.ino:24:0:

C:\Users\Me\Documents\Arduino\libraries\TFT_eSPI-master/TFT_eSPI.h:429:12: note:   initializing argument 5 of 'void TFT_eSPI::pushImage(int32_t, int32_t, uint32_t, uint32_t, uint8_t*)'

   void     pushImage(int32_t x0, int32_t y0, uint32_t w, uint32_t h, uint8_t  *data);

            ^

How do I access the .jpeg images stored in SPIFFS and put them on the screen with pushImage?

shouldn't anyImage be a byte array of image bytes, not the name of the file?

well I've just realized that one big mistake in my code was that I am trying to display jpeg-encoded images, not just unencoded byte arrays. So it needs to happen using the JPEGDecoder library and the drawJpeg command.

That said, even with that, I still can't get it to work. It tells me that drawJpeg was not declared in this scope, although I've downloaded the latest JPEGDecoder library from Github.

I've tried making my code work by adapting one of Bodmer's example sketches, but I still get the error that

'drawJpeg' was not declared in this scope:

#define FS_NO_GLOBALS
#include <FS.h>

// JPEG decoder library
#include <JPEGDecoder.h>

// SPI library, built into IDE
#include <SPI.h>

// Call up the TFT library
#include <TFT_eSPI.h> // Hardware-specific library for ESP8266
// The TFT control pins are set in the User_Setup.h file <<<<<<<<<<<<<<<<< NOTE!
// that can be found in the "src" folder of the library

// Invoke TFT library
TFT_eSPI tft = TFT_eSPI();

//====================================================================================
//                                    Setup
//====================================================================================
void setup()
{
  Serial.begin(250000); // Used for messages and the C array generator

  delay(10);
  Serial.println("NodeMCU decoder test!");

  tft.begin();
  tft.setRotation(0);  // 0 & 2 Portrait. 1 & 3 landscape
  tft.fillScreen(TFT_BLACK);


}

//====================================================================================
//                                    Loop
//====================================================================================
void loop()
{
  // Note the / before the SPIFFS file name must be present, this means the file is in
  // the root directory of the SPIFFS, e.g. "/Tiger.jpg" for a file called "Tiger.jpg"

  tft.setRotation(0);  // portrait
  tft.fillScreen(random(0xFFFF));

  drawJpeg("/d1.jpg", 0, 0);
  
}
//====================================================================================

The JPEGDecoder library does not appear to have a drawJpeg() function, at least not in the version that I found. Nor do you have such a function in your code

Which example are you trying to adapt ?

It's the example sketch "TFT_SPIFFS_Jpeg.ino" from Bodmer's TFT_eSPI library. It can also be found here:

https://github.com/Bodmer/TFT_eSPI/blob/master/examples/160%20x%20128/TFT_SPIFFS_Jpeg/TFT_SPIFFS_Jpeg.ino

I just downloaded the latest versions of the JPEGDecoder and TFT_eSPI libraries today, but still no luck.

EDIT: I think I've found the problem. in the sketch folder for TFT_SPIFFS_Jpeg, there is a file called JPEG_functions. And in it, there is actually the function drawJpeg.

Compiling my code right now with drawJpeg function incorporated into it, I'll see in a minute or two if it works now.