Go Down

Topic: New Jpeg decoder library - based on tjpgd (Read 4139 times) previous topic - next topic

bodmer

I have incorporated the Tiny JPEG Decompressor engine into an Arduino compatible library called TJpg_Decoder.

At the moment this has had limited testing and the examples use the TFT_eSPI library.

If anyone is willing to create the equivalent  Adafruit_GFX examples then I can add them.

The library just needs one simple support callback function to be included in the sketch, this renders blocks of pixels to the screen and the implementation will thus be library dependent. The jpg decoder will call this function during the decompression process.

Due to the memory requirements the library is unsuitable for use with an UNO, a Mega might work but is likely to be very slow due the the maths involved in the decoding process. 32bit processors should run fine.

Typical speeds for decoding+rendering a 13kByte 240x320 image from Flash (PROGMEM) for the ESP8266 and ESP32:

ESP8266 80MHz 285ms, 160MHz 160ms, ESP32 SPI 115ms, 8bit parallel 95ms

From SPIFFS:

ESP8266 80MHz 291ms, 160MHz 167ms, ESP32 SPI 125ms, 8bit parallel 105ms

From SD card:

Untested...

bodmer

I have now added one Adafruit_GFX compatible example "Flash_Jpg_GFX".

david_prentice

#2
Oct 22, 2019, 12:27 am Last Edit: Oct 22, 2019, 12:27 am by david_prentice
Here are some comparisons with Picojpeg and Tjpg (with my sketches):

Code: [Select]

/* show_tjpg_P:   render JPEG image from PROGMEM memory. tiger_jpg */

/* Any Arduino with 8kB SRAM can decode JPEGs
 * tiger.jpg is a typical 31kB 320x240 picture
 * NANO-EVERY  @ 16MHz     picojpeg: 3251ms  tjpgd: 4704ms (18kB angry)
 * MEGA2560    @ 16MHz     picojpeg: 5012ms  tjpgd: 6303ms
 * Xmega128a4u @ 62MHz     picojpeg:  806ms  tjpgd:
 * Xmega128a1  @ 46MHz     picojpeg: 1146ms  tjpgd: 1826ms
 * M0_Pro      @ 48MHz     picojpeg: 1160ms  tjpgd: 1061ms
 * Due         @ 84MHz     picojpeg:  715ms  tjpgd:  547ms
 * STM32F103   @ 72mHz     picojpeg:  808ms  tjpgd:  601ms
 * STM32F446   @180mHz     picojpeg:  222ms  tjpgd:  169ms -O1=165 -O2=167 -O3=180
 * STM32F767   @216MHz     picojpeg:  129ms  tjpgd:   92ms
 * ESP32       @240mHz     picojpeg:  224ms  tjpgd:  172ms
 * ESP8266     @ 80MHz     picojpeg:  584ms  tjpgd:  375ms
 * ESP8266     @160MHz     picojpeg:  325ms  tjpgd:  215ms
 * ESP8266     @160MHz     picojpeg:  239ms  tjpgd:  160ms (13kB panda)
 * Teensy3.2   @120mHz     picojpeg:  448ms  tjpgd:  331ms
 */


Chan's tjpgd is significantly faster on 32 bit MCUs.
My "typical" JPEG is 31kB.    Good Resolution but is noticeably slower than your small 13kB JPEG.
I was impressed by the STM32767

I could not build your sketch on ESP8266.   I will investigate tomorrow.   (multiple SD.h)

My sketches use local tabs.   It is much neater to organise as a class in a public Library.

David.

TFTLCDCyg

#3
Oct 22, 2019, 12:53 am Last Edit: Oct 22, 2019, 12:54 am by TFTLCDCyg
Thanks bodmer.

david_prentice: Can you share with us the image you used for the test?

PD: I would like to test the setup I have installed with an F767 core and a 7 "TFT FT813
ft81xmania.com/comunidad/

david_prentice

I am sure that I have attached my sketches before now.

You can render any size JPEG that you like from SD (or SPIFFS)
You can use large arrays in PROGMEM with ARM or ESP
AVR targets are limited to < 32kB PROGMEM arrays.

It is surprising how good a 31kB JPEG can look.
Even the 18kB JPEGs look pretty good.
IMO,  you can see the "compression" in 13kB JPEGs

Note that my figures were from PROGMEM.   Not SD or SPIFFS
In practice JPEG decoding is compute intensive but a "smaller" JPEG will need less traffic from SD, SPIFFS or PROGMEM

David.

david_prentice

I can build and run Flash_Jpg on ESP32 straight out of the box.

I can build and run Flash_Jpg_GFX on ESP32

TJpg_Decoder.cpp fails on ESP8266 due to SD.h library conflicts.

I am using a Win10-64 PC with IDE v1.8.9
ESP8266 v2.4.2 
ESP32 v1.0.2

The Boards Manager says that ESP8266 v2.5.2 and ESP32 v1.0.4 are available.

From memory,  attempts to upgrade Cores gave problems.
What Core versions are you using ?

David.

bodmer

The SD_Jpg example compiles OK using ESP32 with board package 1.0.4

The SD_Jpg example compiles OK for ESP8266 with board package 2.5.2

ESP8266 board package 2.5.1 is probably the oldest that will work with the example as in that release the SD + SPIFFS file system was harmonised.

I am using IDE 1.8.7 but doubt that matters.

The SD_Jpg example and SD support within the library is untested at the moment but I see nothing obviously wrong with the code.

There is an option to use LittleFS in the library with the ESP8266 but that is untested and will not be enabled until board package 2.6.0 is released.

david_prentice

#7
Oct 23, 2019, 01:28 am Last Edit: Oct 23, 2019, 01:29 am by david_prentice
I have updated this Laptop to ESP8266 v2.5.2

My Fork was out of step with your TFT_eSPI.
I deleted my Fork completely.   And re-Forked.

My ESP8266 build problems were down to a missing cast in some conditional LOAD_RLE code.
This has been fixed in your current Beta.

I have it running on ESP8266 now.
I have it running on STM32 (disabling TJPGD_LOAD_SD_LIBRARY, TJPGD_LOAD_SPIFFS, enabling JD_TBLCLIP)

I will upgrade my Desktop to ESP32 v1.0.4 and ESP8266 v2.5.2
and try a selection of JPEGs.

David.

bodmer

Hi David, thanks for testing the library. If there are any issues then post here or on Github.

The library has not been extensively tested but most of the work is done by the proven tjpgd code so hopefully there will not be many problems.

david_prentice

@Bodmer,

I have adapted your SPIFFS_Jpg.ino example to render any JPG that it finds on SPIFFS

I have upgraded my ESP8266 to v2.5.2
SPIFFS works differently on ESP8266 or ESP32 e.g. walking the Directory.

Your Library class certainly makes everything tidier.

David.
Code: [Select]

// Example for library:
// https://github.com/Bodmer/TJpg_Decoder

// This example if for an ESP8266 or ESP32, it renders a Jpeg file
// that is stored in a SPIFFS file. The test image is in the sketch
// "data" folder (press Ctrl+K to see it). You must upload the image
// to SPIFFS using the ESP8266 or ESP32 Arduino IDE upload menu option.

// Include the jpeg decoder library
#include <TJpg_Decoder.h>

// Include SPIFFS
#define FS_NO_GLOBALS
#include <FS.h>
#ifdef ESP32
#include "SPIFFS.h" // ESP32 only
#endif

// Include the TFT library https://github.com/Bodmer/TFT_eSPI
#include "SPI.h"
#include <TFT_eSPI.h>              // Hardware-specific library
TFT_eSPI tft = TFT_eSPI();         // Invoke custom library


// This next function will be called during decoding of the jpeg file to
// render each block to the TFT.  If you use a different TFT library
// you will need to adapt this function to suit.
bool tft_output(int16_t x, int16_t y, uint16_t w, uint16_t h, uint16_t* bitmap)
{
    // Stop further decoding as image is running off bottom of screen
    if ( y >= tft.height() ) return 0;

    // This function will clip the image block rendering automatically at the TFT boundaries
    tft.pushImage(x, y, w, h, bitmap);

    // This might work instead if you adapt the sketch to use the Adafruit_GFX library
    // tft.drawRGBBitmap(x, y, bitmap, w, h);

    // Return 1 to decode next block
    return 1;
}


void setup()
{
    Serial.begin(115200);
    Serial.println("\n\n Testing TJpg_Decoder library");

    // Initialise SPIFFS
    if (!SPIFFS.begin()) {
        Serial.println("SPIFFS initialisation failed!");
        while (1) yield(); // Stay here twiddling thumbs waiting
    }
    Serial.println("\r\nInitialisation done.");

    // Initialise the TFT
    tft.begin();
    tft.setTextColor(0xFFFF, 0x0000);
    tft.fillScreen(TFT_BLACK);
    tft.setSwapBytes(true); // We need to swap the colour bytes (endianess)

    // The jpeg image can be scaled by a factor of 1, 2, 4, or 8
    TJpgDec.setJpgScale(1);

    // The decoder must be given the exact name of the rendering function above
    TJpgDec.setCallback(tft_output);
}

void doSomething(const char *name)
{
    tft.fillScreen(TFT_RED);

    // Time recorded for test purposes
    uint32_t t = millis();
    // Get the width and height in pixels of the jpeg if you wish
    uint16_t w = 0, h = 0, scale;
    TJpgDec.getFsJpgSize(&w, &h, name); // Note name preceded with "/"
    tft.setRotation(w > h ? 1 : 0);
    for (scale = 1; scale <= 8; scale <<= 1) {
        if (w <= tft.width() * scale && h <= tft.height() * scale) break;
    }
    TJpgDec.setJpgScale(scale);
    // Draw the image, top left at 0,0
    TJpgDec.drawFsJpg(0, 0, name);
    // How much time did rendering take (ESP8266 80MHz 291ms, 160MHz 167ms, ESP32 SPI 125ms, 8bit parallel 105ms
    t = millis() - t;
    char buf[80];
    sprintf(buf, "%s %dx%d 1/%d %ld ms", name, w, h, scale, t);
    tft.setCursor(0, tft.height() - 8);
    tft.print(buf);
    Serial.println(buf);
    delay(2000);
}

//====================================================================================
//                                    Loop
//====================================================================================
#if defined(ESP32)
void loop()
{
    File directory = SPIFFS.open("/");
    File f;
    while (f = directory.openNextFile()) {
        String bum = f.name();
        const char *s = bum.c_str();
        if (strstr(s, ".jpg") != NULL) {
            doSomething(s);
        }
    }
}
#else   //old ESP8266 has different SPIFFS methods
void loop()
{
    fs::Dir directory = SPIFFS.openDir("/");
    while (directory.next()) {
        String bum = directory.fileName();
        const char *s = bum.c_str();
        if (strstr(s, ".jpg") != NULL) {
            doSomething(s);
        }
    }
}
#endif

bodmer

Hi David, thanks for the sketch. Worked great on an ESP32 with one tweak (added (long int) to avoid error):
Code: [Select]

sprintf(buf, "%s %dx%d 1/%d %ld ms", name, w, h, scale, (long int)t);


I will add to the library at the next update.

david_prentice

#11
Oct 25, 2019, 10:46 am Last Edit: Oct 25, 2019, 01:14 pm by david_prentice
Surely the correct statement should be:
Code: [Select]

sprintf(buf, "%s %ux%u 1/%u %lu ms", name, w, h, scale, t);


I did not get any warnings about the printf format arguments with ESP32 v1.0.4

Incidentally,  SPIFFS changed when I upgraded from v1.0.2 to v1.0.4

David.

Edit.  I enabled "All Warnings" and sure enough the Expressif compiler barfed at %ld

When I tried the "correct" format string,  the Expressif compiler barfed at %lu

I have always understood that %u is default unsigned int for the compiler.   e.g. uint16_t, uint32_t, uint64_t ...
But %lu was specifically uint32_t
And %llu was specifically uint64_t

My suggested format string should be ok for AVR, XMega, ARM, ESP, ...
Obviously this sketch is specifically for SPIFFS which means the target is always ESP.

Hey-ho.   I am impressed that a Compiler checks variadic arguments.

ESP8266 2.5.2 gives Warning.  ESP32 v1.0.4 gives Error for uint32_t with %lu 
This should keep ESP happy.   And any other Compilers.
Code: [Select]

sprintf(buf, "%s %ux%u 1/%u %lu ms", name, w, h, scale, (unsigned long)t);

bodmer

The library has been updated on Github.

A new example "All_SPIFFS" has been added based on David's sketch above, plus minor changes to improve rendering performance with ESP32 and ESP8266

bodmer

#13
Oct 31, 2019, 11:58 pm Last Edit: Nov 01, 2019, 12:00 am by bodmer
I finally got around to testing the SD_Jpg example that comes with the TJpg_Decoder library and it worked fine on an ESP32.

For an ESP32 board a performance comparison of using different storage media gave the following results for the included 13kbyte panda.jpg file:

FLASH:    106 ms
SPIFFS:   120 ms
SD Card: 180 ms

I do not have a setup that uses the SD_MMC library with the ESP32, but this would be expected to give results closer to the SPIFFS performance.

bodmer

I have added a new example that will fetch a jpeg from the web using a URL and save it to SPIFFS, then render the image on the TFT.

Go Up