New Jpeg decoder library - based on tjpgd

Other Hardware Displays
1

I have incorporated the Tiny JPEG Decompressor engine into an Arduino compatible library called TJpg_Decoder.

At the moment this has had limited testing and the examples use the TFT_eSPI library.

If anyone is willing to create the equivalent Adafruit_GFX examples then I can add them.

The library just needs one simple support callback function to be included in the sketch, this renders blocks of pixels to the screen and the implementation will thus be library dependent. The jpg decoder will call this function during the decompression process.

Due to the memory requirements the library is unsuitable for use with an UNO, a Mega might work but is likely to be very slow due the the maths involved in the decoding process. 32bit processors should run fine.

Typical speeds for decoding+rendering a 13kByte 240x320 image from Flash (PROGMEM) for the ESP8266 and ESP32:

ESP8266 80MHz 285ms, 160MHz 160ms, ESP32 SPI 115ms, 8bit parallel 95ms

From SPIFFS:

ESP8266 80MHz 291ms, 160MHz 167ms, ESP32 SPI 125ms, 8bit parallel 105ms

From SD card:

Untested...

2

I have now added one Adafruit_GFX compatible example "Flash_Jpg_GFX".

3

Here are some comparisons with Picojpeg and Tjpg (with my sketches):

/* show_tjpg_P:   render JPEG image from PROGMEM memory. tiger_jpg */

/* Any Arduino with 8kB SRAM can decode JPEGs
 * tiger.jpg is a typical 31kB 320x240 picture
 * NANO-EVERY  @ 16MHz     picojpeg: 3251ms  tjpgd: 4704ms (18kB angry)
 * MEGA2560    @ 16MHz     picojpeg: 5012ms  tjpgd: 6303ms
 * Xmega128a4u @ 62MHz     picojpeg:  806ms  tjpgd:
 * Xmega128a1  @ 46MHz     picojpeg: 1146ms  tjpgd: 1826ms
 * M0_Pro      @ 48MHz     picojpeg: 1160ms  tjpgd: 1061ms
 * Due         @ 84MHz     picojpeg:  715ms  tjpgd:  547ms
 * STM32F103   @ 72mHz     picojpeg:  808ms  tjpgd:  601ms
 * STM32F446   @180mHz     picojpeg:  222ms  tjpgd:  169ms -O1=165 -O2=167 -O3=180
 * STM32F767   @216MHz     picojpeg:  129ms  tjpgd:   92ms
 * ESP32       @240mHz     picojpeg:  224ms  tjpgd:  172ms
 * ESP8266     @ 80MHz     picojpeg:  584ms  tjpgd:  375ms
 * ESP8266     @160MHz     picojpeg:  325ms  tjpgd:  215ms
 * ESP8266     @160MHz     picojpeg:  239ms  tjpgd:  160ms (13kB panda)
 * Teensy3.2   @120mHz     picojpeg:  448ms  tjpgd:  331ms
 */

Chan's tjpgd is significantly faster on 32 bit MCUs.
My "typical" JPEG is 31kB. Good Resolution but is noticeably slower than your small 13kB JPEG.
I was impressed by the STM32767

I could not build your sketch on ESP8266. I will investigate tomorrow. (multiple SD.h)

My sketches use local tabs. It is much neater to organise as a class in a public Library.

David.

4

Thanks bodmer.

david_prentice: Can you share with us the image you used for the test?

PD: I would like to test the setup I have installed with an F767 core and a 7 "TFT FT813

5

I am sure that I have attached my sketches before now.

You can render any size JPEG that you like from SD (or SPIFFS)
You can use large arrays in PROGMEM with ARM or ESP
AVR targets are limited to < 32kB PROGMEM arrays.

It is surprising how good a 31kB JPEG can look.
Even the 18kB JPEGs look pretty good.
IMO, you can see the "compression" in 13kB JPEGs

Note that my figures were from PROGMEM. Not SD or SPIFFS
In practice JPEG decoding is compute intensive but a "smaller" JPEG will need less traffic from SD, SPIFFS or PROGMEM

David.

tiger.jpg

angry_16.jpg

woof.jpg

6

I can build and run Flash_Jpg on ESP32 straight out of the box.

I can build and run Flash_Jpg_GFX on ESP32

TJpg_Decoder.cpp fails on ESP8266 due to SD.h library conflicts.

I am using a Win10-64 PC with IDE v1.8.9
ESP8266 v2.4.2
ESP32 v1.0.2

The Boards Manager says that ESP8266 v2.5.2 and ESP32 v1.0.4 are available.

From memory, attempts to upgrade Cores gave problems.
What Core versions are you using ?

David.

7

The SD_Jpg example compiles OK using ESP32 with board package 1.0.4

The SD_Jpg example compiles OK for ESP8266 with board package 2.5.2

ESP8266 board package 2.5.1 is probably the oldest that will work with the example as in that release the SD + SPIFFS file system was harmonised.

I am using IDE 1.8.7 but doubt that matters.

The SD_Jpg example and SD support within the library is untested at the moment but I see nothing obviously wrong with the code.

There is an option to use LittleFS in the library with the ESP8266 but that is untested and will not be enabled until board package 2.6.0 is released.

8

I have updated this Laptop to ESP8266 v2.5.2

My Fork was out of step with your TFT_eSPI.
I deleted my Fork completely. And re-Forked.

My ESP8266 build problems were down to a missing cast in some conditional LOAD_RLE code.
This has been fixed in your current Beta.

I have it running on ESP8266 now.
I have it running on STM32 (disabling TJPGD_LOAD_SD_LIBRARY, TJPGD_LOAD_SPIFFS, enabling JD_TBLCLIP)

I will upgrade my Desktop to ESP32 v1.0.4 and ESP8266 v2.5.2
and try a selection of JPEGs.

David.

9

Hi David, thanks for testing the library. If there are any issues then post here or on Github.

The library has not been extensively tested but most of the work is done by the proven tjpgd code so hopefully there will not be many problems.

10

@Bodmer,

I have adapted your SPIFFS_Jpg.ino example to render any JPG that it finds on SPIFFS

I have upgraded my ESP8266 to v2.5.2
SPIFFS works differently on ESP8266 or ESP32 e.g. walking the Directory.

Your Library class certainly makes everything tidier.

David.

// Example for library:
// https://github.com/Bodmer/TJpg_Decoder

// This example if for an ESP8266 or ESP32, it renders a Jpeg file
// that is stored in a SPIFFS file. The test image is in the sketch
// "data" folder (press Ctrl+K to see it). You must upload the image
// to SPIFFS using the ESP8266 or ESP32 Arduino IDE upload menu option.

// Include the jpeg decoder library
#include <TJpg_Decoder.h>

// Include SPIFFS
#define FS_NO_GLOBALS
#include <FS.h>
#ifdef ESP32
#include "SPIFFS.h" // ESP32 only
#endif

// Include the TFT library https://github.com/Bodmer/TFT_eSPI
#include "SPI.h"
#include <TFT_eSPI.h>              // Hardware-specific library
TFT_eSPI tft = TFT_eSPI();         // Invoke custom library


// This next function will be called during decoding of the jpeg file to
// render each block to the TFT.  If you use a different TFT library
// you will need to adapt this function to suit.
bool tft_output(int16_t x, int16_t y, uint16_t w, uint16_t h, uint16_t* bitmap)
{
    // Stop further decoding as image is running off bottom of screen
    if ( y >= tft.height() ) return 0;

    // This function will clip the image block rendering automatically at the TFT boundaries
    tft.pushImage(x, y, w, h, bitmap);

    // This might work instead if you adapt the sketch to use the Adafruit_GFX library
    // tft.drawRGBBitmap(x, y, bitmap, w, h);

    // Return 1 to decode next block
    return 1;
}


void setup()
{
    Serial.begin(115200);
    Serial.println("\n\n Testing TJpg_Decoder library");

    // Initialise SPIFFS
    if (!SPIFFS.begin()) {
        Serial.println("SPIFFS initialisation failed!");
        while (1) yield(); // Stay here twiddling thumbs waiting
    }
    Serial.println("\r\nInitialisation done.");

    // Initialise the TFT
    tft.begin();
    tft.setTextColor(0xFFFF, 0x0000);
    tft.fillScreen(TFT_BLACK);
    tft.setSwapBytes(true); // We need to swap the colour bytes (endianess)

    // The jpeg image can be scaled by a factor of 1, 2, 4, or 8
    TJpgDec.setJpgScale(1);

    // The decoder must be given the exact name of the rendering function above
    TJpgDec.setCallback(tft_output);
}

void doSomething(const char *name)
{
    tft.fillScreen(TFT_RED);

    // Time recorded for test purposes
    uint32_t t = millis();
    // Get the width and height in pixels of the jpeg if you wish
    uint16_t w = 0, h = 0, scale;
    TJpgDec.getFsJpgSize(&w, &h, name); // Note name preceded with "/"
    tft.setRotation(w > h ? 1 : 0);
    for (scale = 1; scale <= 8; scale <<= 1) {
        if (w <= tft.width() * scale && h <= tft.height() * scale) break;
    }
    TJpgDec.setJpgScale(scale);
    // Draw the image, top left at 0,0
    TJpgDec.drawFsJpg(0, 0, name);
    // How much time did rendering take (ESP8266 80MHz 291ms, 160MHz 167ms, ESP32 SPI 125ms, 8bit parallel 105ms
    t = millis() - t;
    char buf[80];
    sprintf(buf, "%s %dx%d 1/%d %ld ms", name, w, h, scale, t);
    tft.setCursor(0, tft.height() - 8);
    tft.print(buf);
    Serial.println(buf);
    delay(2000);
}

//====================================================================================
//                                    Loop
//====================================================================================
#if defined(ESP32)
void loop()
{
    File directory = SPIFFS.open("/");
    File f;
    while (f = directory.openNextFile()) {
        String bum = f.name();
        const char *s = bum.c_str();
        if (strstr(s, ".jpg") != NULL) {
            doSomething(s);
        }
    }
}
#else   //old ESP8266 has different SPIFFS methods
void loop()
{
    fs::Dir directory = SPIFFS.openDir("/");
    while (directory.next()) {
        String bum = directory.fileName();
        const char *s = bum.c_str();
        if (strstr(s, ".jpg") != NULL) {
            doSomething(s);
        }
    }
}
#endif
11

Hi David, thanks for the sketch. Worked great on an ESP32 with one tweak (added (long int) to avoid error):

sprintf(buf, "%s %dx%d 1/%d %ld ms", name, w, h, scale, (long int)t);

I will add to the library at the next update.

12

Surely the correct statement should be:

sprintf(buf, "%s %ux%u 1/%u %lu ms", name, w, h, scale, t);

I did not get any warnings about the printf format arguments with ESP32 v1.0.4

Incidentally, SPIFFS changed when I upgraded from v1.0.2 to v1.0.4

David.

Edit. I enabled "All Warnings" and sure enough the Expressif compiler barfed at %ld

When I tried the "correct" format string, the Expressif compiler barfed at %lu

I have always understood that %u is default unsigned int for the compiler. e.g. uint16_t, uint32_t, uint64_t ...
But %lu was specifically uint32_t
And %llu was specifically uint64_t

My suggested format string should be ok for AVR, XMega, ARM, ESP, ...
Obviously this sketch is specifically for SPIFFS which means the target is always ESP.

Hey-ho. I am impressed that a Compiler checks variadic arguments.

ESP8266 2.5.2 gives Warning. ESP32 v1.0.4 gives Error for uint32_t with %lu
This should keep ESP happy. And any other Compilers.

sprintf(buf, "%s %ux%u 1/%u %lu ms", name, w, h, scale, (unsigned long)t);
13

The library has been updated on Github.

A new example "All_SPIFFS" has been added based on David's sketch above, plus minor changes to improve rendering performance with ESP32 and ESP8266

14

I finally got around to testing the SD_Jpg example that comes with the TJpg_Decoder library and it worked fine on an ESP32.

For an ESP32 board a performance comparison of using different storage media gave the following results for the included 13kbyte panda.jpg file:

FLASH: 106 ms
SPIFFS: 120 ms
SD Card: 180 ms

I do not have a setup that uses the SD_MMC library with the ESP32, but this would be expected to give results closer to the SPIFFS performance.

15

I have added a new example that will fetch a jpeg from the web using a URL and save it to SPIFFS, then render the image on the TFT.

16

@Bodmer,

Nice Example. Horrible picture!

David.

17

David, That was one of the test images... and the only one I found that was 240x320 in my imgur set! I've changed the picture to M81 (Bode's Galaxy) :slight_smile:

18

Nicer picture. But a bird, animal or human would be a more recognisable image.

What is your opinion of my comment in #11 ?
Is the Expressif compiler correct about %u and %lu ?

David.

19

@David, re. #11

I think the problem is that with the ESP32 the "int" "int32_t" and "long" are all 32 bits.

Each varaible type can only be allocated one print format macro, so "int" and "int32_t" have been treated as equivalent. A long in a program is a deliberate type cast so had to be given a "long" type print macro for portability.

This seem acceptable to a few boards I tried compiling for, Arduino boards do produce a warning:

void setup() {
Serial.begin(115200);
}

void loop() {

char buf[32];
uint16_t            a = 16; sprintf(buf, " %u", a);   Serial.println(buf);
unsigned short      b = 16; sprintf(buf, " %u", b);   Serial.println(buf);

uint32_t            c = 32; sprintf(buf, " %u", c);   Serial.println(buf); // Some boards gives warning, but compiles OK
unsigned int        d = 32; sprintf(buf, " %u", d);   Serial.println(buf);
unsigned long       e = 32; sprintf(buf, " %lu", e);  Serial.println(buf);

uint64_t            f = 64; sprintf(buf, " %llu", f); Serial.println(buf);
unsigned long long  g = 64; sprintf(buf, " %llu", g); Serial.println(buf);

int32_t            cc = 32; sprintf(buf, " %d", cc);  Serial.println(buf); // Some boards give warning
int                dd = 32; sprintf(buf, " %d", dd);  Serial.println(buf);
long               ee = 32; sprintf(buf, " %ld", ee); Serial.println(buf);

while(1) delay(0);
}
20

I have always understood that

%d is native int
%ld is int32_t
%lld is int64_t

It seems unambiguous for (signed) int
Less clear for unsigned int

If unsigned int follows the same style as native int
Which means that the (native 16-bit int) AVR compiler expects uint16_t for %u
And the (native 32-bit int) PC, ARM, ESP32, ESP8266 compilers expects uint32_t for %u

Likewise, a 64-bit int Compiler would expect uint64_t. And would probably push uint64_t for every variadic argument to printf() e.g %u, %lu, %llu

I have never used a 64-bit int compiler.

Of course the sizeof(int) is implementation dependent.
The implementation of printf() is probably more historic convention than strict language rules.

David.