switching colors in PROGMEM stored icons

It just seems that by going 32-bit, there'd be a whole host of things I would not have to bother with in order to keep within the constraints of 8-bit AVRs. It would make many things a little more simple.

I've looked at the ESP32. Even the most modest variant with 4 MB flash memory would mean I could just dump all my images into PROGMEM as full 8-bit indexed bitmap files, and wouldn't have to extract the pixel information from them or anything like that. I could even store my full-screen warning icons in flash. Like this one which will be shown when an outside temperature below +5 °C is detected (using DS18B20 sensors):

Fully loaded ESP breakouts can be had from around €8, which is indeed about the same as the 1284P.

ESP32 or ESP8266 can even use a file system in Flash memory. So you can use standard BMP, JPEG, ... that you have designed and maintained on your PC.

All the same, a Mega128x or Mega2560 can use all of its memory for code.
The standard PROGMEM functions only work below 64kB.
You have to use PROGMEM_FAR functions to access all data.

In practice, you ensure that your functions use FAR addresses and any "big data" follows regular PROGMEM (that might not be in your control).

Here is a sketch that can run on an Arduino:

/*
    far_memx_cpp.cpp

    Created: 16/11/2017 10:37:14
    Author : David

    C++ project arranges program components:
    1. Vector table
    2. JMP Trampolines for code > W:64k (B:128k)
    3. PROGMEM data
    4. PROGMEM data in section .fini2
    5. executable code
    6. initialised data that is copied to SRAM
    7. anonymous data that is copied to SRAM
*/

//select suitable array size to illustrate address map
//#define BIGLY 19000    //should fit in a ATmega1284
#define BIGLY 21650    //some code < B:128k, some code with trampolines
//#define BIGLY 21780    //some trampolines in AS7
//#define BIGLY 32767   //biggest array size with GCC/G++

#if defined(ARDUINO)        //kludge for sketch
char g_buf[80];             //somewhere for sprintf output
#define initstdio()      Serial.begin(9600)
#define PRINTF(args ...) {sprintf(g_buf, args); Serial.print(g_buf);}
#define MAIN() void setup(void)
#else                       //regular AS7 C++ project
#include <avr/io.h>
#include <stdio.h>
#include <avr/pgmspace.h>

extern "C" void initstdio(void);
#define PRINTF(args ...) printf(args)
#define MAIN() int main(void)

#endif

#define ATTRIB __attribute__((section(".fini2")))
#define M0(x) pgm_get_far_address(x), #x

const uint8_t PROGMEM ATTRIB forename[BIGLY] = "David in section(.fini2)";
const uint8_t PROGMEM ATTRIB midname[BIGLY] = "Ness in section(.fini2)";
const uint8_t PROGMEM ATTRIB surname[BIGLY] = "Prentice in section(.fini2)";

const uint8_t PROGMEM arm[BIGLY] = "arm in PROGMEM";
const uint8_t PROGMEM leg[BIGLY] = "leg in PROGMEM";
const uint8_t PROGMEM ear[BIGLY] = "ear in PROGMEM";

uint_farptr_t names[10];
char named_data[] = "named_data copied to SRAM";

void showdata(uint_farptr_t ads, const char *name) //data are BYTE addresses
{
    PRINTF("B:%08lx %s[%d] = ", ads, name, BIGLY);
    char buf[40];
    strcpy_PF(buf, ads);      //copy far PROGMEM to SRAM
    PRINTF("\"%s\"\n", buf);  //only works with SRAM
}

void showfunc(uint_farptr_t ads, const char *name) //functions are PC word address
{
    PRINTF("W:%08lx %s() ", ads, name);
    if (ads >= 0x10000) {
        ads &= 0xFFFF;
        uint16_t jmp = pgm_read_word_far(ads * 2);
        uint16_t vec = pgm_read_word_far(ads * 2 + 2);
        ads = ((jmp & 3L) << 16) | vec;
        PRINTF("trampoline: JMP 0x%08lx ", ads);
    }
    PRINTF("(B:%08lx)\n", ads * 2);
}

void loop(void)
{
}

MAIN()
{
    // makes life convenient for AS7 Simulator
    names[0] = pgm_get_far_address(forename);
    names[1] = pgm_get_far_address(midname);
    names[2] = pgm_get_far_address(surname);
    names[3] = pgm_get_far_address(arm);
    names[4] = pgm_get_far_address(leg);
    names[5] = pgm_get_far_address(ear);
    names[6] = 0;
    names[7] = pgm_get_far_address(showfunc);
    names[8] = pgm_get_far_address(showdata);

    initstdio();

    PRINTF("Hello far_memx. BIGLY=%d. Adjust to illustrate\n", BIGLY);
    PRINTF("AVR PROGMEM functions only work < B:64k\n");
    PRINTF("AVR needs far addresses for byte data > B:64k\n");
    showdata(M0(forename));
    showdata(M0(midname));
    showdata(M0(surname));
    showdata(M0(arm));
    showdata(M0(leg));
    showdata(M0(ear));

    PRINTF("function addresses are PC words W:0..64k (B:0..128k)\n");
    PRINTF("mega2560 has more than 64k words\n");
    PRINTF("if function is > W:64k a vector is used\n");
    PRINTF("0x10076 is trampoline @ W:0x0076 (B:0x00EC)\n");

    showfunc(M0(showfunc));
    showfunc(M0(showdata));

    PRINTF("named_data is :\"%s\"\n", named_data);
    loop();
}

Note that was written for regular C++ project in AS7. It is easier and quicker to use the Simulator than uploading large HEX files with the Bootloader.

uartstdio.c (not needed for Arduino. Add to your AS7 project)

/*
 * uartstdio.c
 *
 * Created: 05-Oct-16 10:33:50
 *  Author: David Prentice
 */ 

#include <avr/io.h>
#include <stdio.h>

int uart_getchar(FILE *fp)
{
    while ((UCSR0A & (1<<RXC0)) == 0) ;
    return UDR0;
}

int uart_putchar(char c, FILE *fp)
{
//    if (data == '\n') uart_putchar('\r', fp);
    while ((UCSR0A & (1<<UDRE0)) == 0) ;
    UDR0 = c;
    return c;
}

void initstdio(void)
{
    static FILE mystdout = FDEV_SETUP_STREAM(uart_putchar, uart_getchar, _FDEV_SETUP_RW);
    stdin = stdout = &mystdout;
    UBRR0L = F_CPU/16/9600 - 1;
    UCSR0A = (0<<U2X0);
    UCSR0B = (1<<RXEN0)|(1<<TXEN0)|(0<<UCSZ02);
	
}

David.

wow, thanks for that. I am sure I will be able to put that code to use, regardless. I am going to have to look at it in more detail tonight or tomorrow, just checking in quickly here.

As you said, the ESP32 board would have the advantage of being able to just dump my BMPs "as-is" in 24-bit (althought I will probably still use 8-bit indexed) into PROGMEM on the 16 MB flash module that many of them come with. Thanks for the tip of looking at the ESP32 btw; I was unaware of it, and it seems like the board that most closely resembles my vision that I have had for the trip computer all along.

Moving the project to 32 bit now, I could even resurrect this crazy idea I had in the beginning of this project, of making animated screens and menus. Part of my job in advertising was creating Adobe Flash based web animation. So that's where I was coming from with that idea.

I've seen that there is a C library called GIFLIB over at sourceforge.

Do you think it could be ported to the ESP32 using the Arduino IDE, to show animated GIFs? Or would it be better (simpler) to just store the individual images of an animation in PROGMEM as indexed and run through them one by one? Both could be done without a problem using Adobe Flash, which lets you export animated gifs the same way as individual bitmaps for every frame.

I have never done animated GIFs or even regular GIFs. I am sure that the ESP32 (or ESP8266) will have plenty of grunt.

I have animated with whole frames of BMP. The ESP could do that pretty quick too.

I really don't like the Mega2560. But it is not too much of a problem i.e. functions with far arguments, adding section attribute to data. Just be realistic. PROGMEM facilities only work with data < B:64k
Note that things like sprintf_P() will be ok if you are careful. i.e. anonymous_P must be < B:64k
Regular sprintf(format, ...) is fine because the anonymous format string gets copied to SRAM.

David.

ok, but the ESP32 would store PROGMEM data in the onboard flash module, right?

The problem with GIFs is that even though they use indexed colors, with up to 256 different colors either in a shared color table or a separate color table for each picture, the pixel-specific information in them is not directly a particular color in the color table. IIRC, the LZW algorithm in the GIF file specification cuts longer pixel sequences of the same color by not iterating that same color repetitively for every single pixel, but saying "assign color #5D of the color table to the next ten pixels". Which is also why the GIF format is still superior in achievable file size to JPEG and PNG on the web whenever you've got images with few colors and larger contiguous areas of the same color to display. Probably not that relevant anymore these days with gigabit Internet connections, but oh well.

So you can't just parse a GIF file's pixels like you would the pixels of a BMP. You'd have to have a piece of software that could correctly parse LZW-compressed files and back translate them into correctly colored pixels on a screen. Apparently, the GIFLIB library does this; but I guess it has never been ported to Arduino (for obvious reasons) or the ESP32 and its predecessor.

Then again, with 16 MB of flash memory, you could store nearly any animation as a sequence of indexed bitmaps.

I was just reading the Wikipedia article on GIFs.

It looks as if you just need to decode the LZW when rendering a frame. From distant memory, LZW needs a fair bit of SRAM when decoding. Fine for ARM or ESP. Remember that CPM micros could do both ZIP and UNZIP. So processing power is not large even if RAM requirement is.

With a simple large rectangular BMP, you can easily extract a small window pane. e.g. if you store multiple icons or sprites in one file. It would be painful to extract a small pane from LZW stream becaus you need to decode the whole large image.

Has anyone done GIFs on an Arduino TFT?
The biggest attraction (for me) is to use standard animated GIF files.

David.

I've downloaded the whole GIFLIB library from the sourceforge link to have a look at it... apparently, the decoding routines for displaying GIFs on screen are stored in three files called "dgif_lib.c", gif_lib.h" and "gif_lib_private.h", which I have attached to this post.

The "juicy" bit seems to be in dgif_lib.c, about two thirds down, where the actual LZ decompression routine is shown.

Realistically, I will probably really do my animations as some sort of timed succession of individual BMPs shown on screen one after another.

dgif_lib.c (38.1 KB)

gif_lib.h (12.5 KB)

gif_lib_private.h (2.09 KB)

I see that GIFLIB was written in 1990. I bet that there is a more elegant implementation out there, somewhere.

I can remember UNZIP being a small utility on CPM, Amiga or AtariST.
From the Wiki article, the GIF header(s) are straightforward.

A TFT would be the ideal target. Set a Window and you display a stream of pixels as you "unzip".

Google "GIF Arduino" only gets LED Matrix displays. I can't believe that no one has rendered GIF to a TFT yet.

David.

Animated GIFs do tend to be quite large in Arduino terms. Even one animated arrow or bullet point like you had during the Web's infancy on ham-handedly composed Geocities web pages twenty years ago could run you ten kilobytes.

It is only in recent times that animated GIFs have had a revival on the web, for displaying short live-action video sequences, now that we've got all that vast bandwidth compared to back then.

I guess you could read them from SDcard. But having seen how long it takes one single 128x128 image to load from SD, even as a .raw file, the Arduino would probably be out of its depth displaying quick GIF sequences of just a few milliseconds per frame.

You can store quite a bit in the Mega2560 Flash memory. Zero, Due or Teensy have more memory. ESP has separate Flash chip. A GIF "video" involves lots of frames. A GIF animated logo should not be too big.
Surely the LZW compression is dramatic on block colour graphics.

As soon as you have an SD, storage is "unlimited".

A RAW file from SD can be quite fast even on an AVR. The Arduino SPI class could be improved.
ARM and ESP have fast SPI bus with DMA. The result is impressive.

David.

One thing I have just read is that there might be licensing issues still with GIF. That used to be a headache in the early days of web design, some more affordable image editing tools would not let you compose animated GIFs or even static one-frame GIFs because the rights to the format were owned by CompuServe (I think). Something about the licensing rights still appears to make open-source development surrounding the GIF format difficult.

See here:

https://www.gnu.org/philosophy/gif.en.html

Also, I've just read that APNG appears to be an alternative, and open standard of creating animated images, developed by Mozilla. There is a whole page on it on the Mozilla wiki page:

https://wiki.mozilla.org/APNG_Specification#Structure

If it is anything like what Mozilla have done to FireFox it will require 5GB RAM to even run.

All the same, it is worth sticking to common PC formats.

There has never been a problem with decoding a GIF. From your (20 year old) GNU link, it is GIF creation and encoding that was patented.

David.

To illustrate, this is what we might be talking about in terms of animation:

This was created out of Adobe Flash using the animated GIF export option. In its current form, it has some 700 KB as a global-palette GIF with 177 frames.

I should probably explain that I have now moved the project from a 128x128 to a 128x160 Adafruit TFT screen, simply to be able to do more with all the vast capabilities of the ESP32. Also, there will be different screen designs for my own MG F MkI on the one hand, and a friend's MGF MkII, in accordance with the slight touch ups of the interior that the car received at some point in mid-production. This welcome screen animation will be for my MkI. Yes, it might be a bit lengthy to sit through every time, so I'll put something in the code that it won't show every single time you turn the ignition. I might also shorten the final version by a few seconds. And the frame rate could be brought down to some 8 fps and you'd still have sufficiently smooth flowing animation. This is just a collection of ideas at this point.

The animated menus, on the other hand, might look something like this (now in the MkII design):

These "change-over" menus should occupy much less space. The entire 128x160 animation that you are seeing right here has 85 frames. I should be able to do these little "change-over" animations as successions of five or six 65x25 px 8-bit bitmaps. The frame rate and total number of frames could be pared down as well, and the human eye will still see what it wants to see, i.e. an animated carousel of menu icons.

Maybe they could be controlled by a function like

void rotateIconsCarousel(byte thisGroupMenu, byte nextGroupMenu, byte thisSubMenu, byte nextSubMenu, byte frameRate) {}

Anyway, I've just ordered an ESP32 off eBay with the 16 MB flash memory. It should arrive in about two weeks from China. With all these animations, even the 16 MB will probably end up being packed with graphics, but I will be much happier. It will also allow me to create any number of daytime and nighttime 16-bit color fonts to display numerical values.

Also, here's a very in-depth description of the GIF file specification:

http://giflib.sourceforge.net/whatsinagif/bits_and_bytes.html

Yes, it is a good article. Wikipedia is good too.

I am sure that someone has written the code for a microcontroller already. I am not sure whether I have the enthusiasm.

Anyway, your ESP32 will be able to animate uncompressed bitmaps. The biggest advantage of GIF is that all the frames are stored in a single file.

David.

david_prentice:
Anyway, your ESP32 will be able to animate uncompressed bitmaps. The biggest advantage of GIF is that all the frames are stored in a single file.

David.

Yes, and the file size. The actual welcome screen on the finished product will very probably not be 15 seconds long and contain 177 images, but it's just more handy to have it in a small compact file. Some 700 KB for the animation above, but as 128x160 x 177 frames as 8-bit BMP, we're looking at 3.5 MB if I'm not mistaken.

I've tried to find a GIF decoding library online that could be ported to the Arduino, but didn't manage to find anything so far. What you do see on web forums is people asking around if somebody has a good GIF library... :slight_smile:

I would love to write some code myself to decode and display GIF images. But my programming skills just aren't up to that.

ok call me crazy, but I have actually begun work on my own GIF decoder Arduino sketch.

If you really just see the GIF file format description as a "how-to" of how the bytes need to be decoded and analyzed, then it should be possible to just walk through all that, bit by bit (or rather: byte by byte :slight_smile: ), write code accordingly, and at the end hopefully display an image correctly.

Here's my code so far:

#include "logo.h"

void showGIF(uint16_t gifName, uint16_t width, uint16_t height, uint16_t xpos, uint16_t ypos) {

  boolean goodFile = false;
  char header[6];
  byte headerElement;

  // reading header
  for (byte i = 0; i <= 5; i++) {

    headerElement = pgm_read_byte(&MGlogo[i]);
    header[i] = headerElement;
  }

  if (String(header).startsWith("GIF89a")) {

    Serial.print("Header found: ");
    Serial.println(header);
    goodFile = true;
  }

  else Serial.println("Wrong file header found. Doing nothing.");

  if (goodFile) {
    //reading file width
    int fileWidth   = (pgm_read_byte(&MGlogo[6])) | (pgm_read_byte(&MGlogo[7]) << 8);
    int fileHeight  = (pgm_read_byte(&MGlogo[8])) | (pgm_read_byte(&MGlogo[9]) << 8);


    Serial.println("-------------\nImage information:\n");
    Serial.print("File dimensions: ");
    Serial.print(fileWidth);
    Serial.print(" x ");
    Serial.print(fileHeight);
    Serial.println(" px");

    // Reading Packed Field byte and expanding it
    byte packedField = pgm_read_byte(&MGlogo[10]);
    Serial.print("Packed Field: ");
    Serial.println(packedField, BIN);

    byte globalColorTableFlag = bitRead(packedField, 7);

    byte colorResolution = (bitRead(packedField, 6) <<2) | (bitRead(packedField, 5) << 1) | (bitRead(packedField, 4));

    byte sortFlag = (bitRead(packedField, 4) >> 0);

    byte globalColorTableSize = (bitRead(packedField, 0)) | (bitRead(packedField, 1) << 1) | (bitRead(packedField, 2) << 2);
    
    globalColorTableSize = pow(2, globalColorTableSize) + 1;

    Serial.print("Global color table: ");
    if (globalColorTableFlag == 1) Serial.println("Y");
    else Serial.println("N");

    Serial.print("Global color resolution: ");
    Serial.println(colorResolution);


    Serial.print("Color sorting: ");
    if (sortFlag == 1) Serial.println("decreasing");
    else Serial.println("increasing");

    Serial.print("Color table size: ");
    Serial.print(globalColorTableSize);
    Serial.println(" indexed colors");
  }
}

void setup()
{

  Serial.begin(115200);
  showGIF(MGlogo, 30, 30, 0, 0);

}

void loop()
{

  /* add main program code here */

}

I've attached all the files to this post.

A few hiccups so far; I can't do

pgm_read_byte(&gifName[i])

, it tells me "invalid types 'uint16_t {aka unsigned int}[byte {aka unsigned char}]' for array subscript". Not 100% sure what that's about.

Also, I'm still having trouble analyzing the "packed field" byte, which contains several bits of information. I get wrong values for color resolution (should be "8"), and the number of indexed colors (is 4 but should be 10).

According to sourceforge, the packed field byte looks like this:

My packed field byte I get from my code, when expanded, equals 10000100. So I am going to have to see why that is.

I may be wholely out of my depth here trying to write a GIF decoder function like this, but it kind of seems like a cool sub-project to take on while I will spend two weeks twiddling my thumbs waiting for my ESP32 module to arrive from China :slight_smile:

mglogo-zip2.zip (2.38 KB)

EDIT:

It kind of looks like somebody has already done the work and made a whole library for animated GIFs for the Teensy:

But weirdly, this library only works with GIFs up to 32x32 pixels. And it looks sort of bulky at first glance.

Shouldn't there be a leaner way to decode GIFs and display them on a TFT, even bigger ones than 32x32?

Maybe the approach to write my own GIF decoder isn't so silly after all.

The SmartMatrix library has probably got everything you need. But my brain hurts when I try to read it.

It is not even very clear what sort of LED matrix it is supposed to drive.
A raw matrix requires multiplexing, complex timing etc.

A TFT controller does all of the hard work. You just need to extract dimensions, palette etc from the header field(s) and extract the LZW compressed bitmap.

Incidentally, I guess that you are a Brit. No one else would want an MG.
You can buy ESP32 hardware from UK/EU vendors. Costs more but arrives in 24/48 hours.

David.

No, I'm actually German :slight_smile:

Funny story about me owning an MG, yeah... the F is a nice litte car though. Loads of fun to drive, and quite affordable these days. A lot of bang for your buck.

Anyway, looking at the GIF library, it kind of looks like it's so bulky because it tries to cover every possible case of what kind of data might be inside a GIF file. I am thinking if I keep the parameters of my own created GIF files within certain boundaries, I might not need much code in the first place. I also wouldn't need many of the if statements and error messages it can print out if certain criteria aren't met. I could make sure they are met in creating the files to begin with. So I could boil things down in that respect.

Also, this library only seems to accept files loaded from an SD card. Especially on the ESP32 with 16 MB of flash, I would like my images to all be in PROGMEM.

In terms of leanness, I would like a GIF decoding function to be more along the lines of the showBMP() function in the showBMP_kbv_Uno sketch. Using the LZW algorithm probably means significantly more code still than in showBMP(), I know.

Maybe I will be able to extract the actual LZW algorithm from that library and use it for my purposes. I'll probably have a closer look this weekend.