Arduino Forum

Using Arduino => Displays => Topic started by: rlightner on Oct 23, 2019, 03:31 am

Title: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: rlightner on Oct 23, 2019, 03:31 am
I've got a BMP (attached) that was generated with ImageMagick from a color png. the BMP displays just find using any app.

Infranview says colors are: 2   (1 BitsPerPixel) (BW)
Gimp says Color Space: Indexed color (2 colors)

I'm using the drawBitmapFromSpiffs function from

https://github.com/ZinggJM/GxEPD2/blob/master/examples/GxEPD2_Spiffs_Example/GxEPD2_Spiffs_Example.ino

and it just draws the screen a solid black. Am I doing something wrong here?

Thanks in advance!

Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: ZinggJM on Oct 23, 2019, 06:30 am
Please report what processor you use.

Your issue may be related to this post. (https://forum.arduino.cc/index.php?topic=641905.msg4345990#msg4345990)

I may find time to check this.

You could also have posted diagnostic output from the example.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: rlightner on Oct 23, 2019, 06:34 am
TTGO T5 V2.1 ESP32 2.7

https://github.com/lewisxhe/TTGO-EPaper-Series

bmp sig: 19778
File size: 6482
Creator Bytes: 0
Image Offset: 146
Header size: 124
Bit Depth: 1
Image size: 176x264
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: ZinggJM on Oct 23, 2019, 07:18 am
I see neither a success message nor a failure message in your post, should see: loaded in …

Before I spend time on setting up for test:

Do the other example bitmaps render correctly?

Could it be that your bmp has no palette? the code expects a palette with black and white entry also for b/w.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 23, 2019, 10:32 am
The "output5.bmp" is a regular BMP format file.   176x264  1BPP  i.e. 1-bit colour (monochrome).
It contains its own colour Palette.   You can edit the Palette in IrfanView.   Currently Index:0=WHITE, 1=BLACK

Most BMP programs on Arduino only render 24-bit colour BMP files.

I provide an example that will render BMP files with 1BPP, 4BPP, 8BPP, 16BPP, 24BPP

Although your image is only 1 bit deep,  the individual Palette colours can be chosen from a 16M Colour Palette (24-bits).
Obviously how accurate the colours are on your TFT depend on the TFT resolution (16-bits or 18-bits)

David.

OOPS.  I was replying to a TFT colour display question.   When you are using a monochrome display.
Since your EPaper device can only render BLACK or WHITE it will ignore most of the 24-bits in a Palette Entry.

Edit.  I looked at the drawBitmapFromSpiffs() function from #0
It seems to support all the same BMP formats that I do.
It understands Palettes.   It should display BLACK and WHITE properly.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: ZinggJM on Oct 23, 2019, 02:03 pm
@David,

How did you check the palette of this picture?

or is this wrong:

Code: [Select]
         file.seekSet(54); //palette is always @ 54


@rlightner,

the palette looks wrong, index 0 and 1 have 2 different colors, but neither black nor white:

palette values below are red green blue

Code: [Select]
Loading image 'betty_4.bmp'
File size: 26326
Image Offset: 118
Header size: 40
Bit Depth: 4
Image size: 204x252
0 0 0
128 0 0
0 128 0
128 128 0
0 0 128
128 0 128
0 128 128
128 128 128
192 192 192
255 0 0
0 255 0
255 255 0
0 0 255
255 0 255
0 255 255
255 255 255
_PowerOn : 36727
_Update_Full : 1527816
loaded in 2842 ms
_Update_Full : 1527948

Loading image 'bb4.bmp'
File size: 206
Image Offset: 62
Header size: 40
Bit Depth: 1
Image size: 34x18
0 0 0
255 255 255
_Update_Part : 556839
loaded in 899 ms
_Update_Full : 1528333

Loading image 'output5.bmp'
File size: 6466
Image Offset: 130
Header size: 108
Bit Depth: 1
Image size: 176x264
255 0 0
0 255 0
_Update_Part : 555762
loaded in 1590 ms
_Update_Full : 1528561
GxEPD2_SD_Example done


Jean-Marc

BTW: thanks to David:

Code: [Select]
// BMP handling code extracts taken from: https://github.com/prenticedavid/MCUFRIEND_kbv/tree/master/examples/showBMP_kbv_Uno

Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 23, 2019, 02:26 pm
The header size is always 54.
If there is a Palette,  it follows the Header.

From IrfanView:  output5.bmp has Palette 255:255:255 , 000:000:000  i.e. WHITE,BLACK

I have some x1 bitmaps e.g. truck.bmp

betty_4.bmp is a 4BPP image i.e. 16 colours.

You should be able to examine the Palette in IrfanView.
Likewise,  you can take a 24BPP colour JPG, PNG, BMP and reduce it to 8BPP (256 colours) quite well.

Reducing to 4BPP or 1BPP requires a bit of skill.    IrfanView does a pretty good job but 1BPP is very difficult to get the "best" look.

I suggest that you look very carefully at your Palette.   I would calculate the Palette once.    Then you render each pixel directly from the Palette.

David.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: rlightner on Oct 23, 2019, 02:34 pm
I'm using ImageMagick to created the bmp:


convert output.png -filter point -resize "176x264>" -dither FloydSteinberg -define dither:diffusion-amount=85% -monochrome -depth 1 output5.bmp


I'm going to read back over the messages above and try and figure out what I'm doing wrong.


Thanks again for the effort.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: ZinggJM on Oct 23, 2019, 02:56 pm
@David,

I just added one line of diagnostic output, to try to analyze:

Code: [Select]
       if (depth == 1) with_color = false;
        if (depth <= 8)
        {
          if (depth < 8) bitmask >>= depth;
          file.seekSet(54); //palette is always @ 54
          for (uint16_t pn = 0; pn < (1 << depth); pn++)
          {
            blue  = file.read();
            green = file.read();
            red   = file.read();
            file.read();
            Serial.print(red); Serial.print(" "); Serial.print(green); Serial.print(" "); Serial.println(blue);
            whitish = with_color ? ((red > 0x80) && (green > 0x80) && (blue > 0x80)) : ((red + green + blue) > 3 * 0x80); // whitish
            colored = (red > 0xF0) || ((green > 0xF0) && (blue > 0xF0)); // reddish or yellowish?
            if (0 == pn % 8) mono_palette_buffer[pn / 8] = 0;
            mono_palette_buffer[pn / 8] |= whitish << pn % 8;
            if (0 == pn % 8) color_palette_buffer[pn / 8] = 0;
            color_palette_buffer[pn / 8] |= colored << pn % 8;
          }
        }


Why don't I get 255 255 255 and 0 0 0, as I would expect from info of IrfanView?
but 255 0 0 and 0 255 0 instead?

Same values in IrfanView HEX view.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 23, 2019, 03:05 pm
My apologies.  I copied output5.bmp to an SD.   Then displayed via an ESP8266.

It shows as Yellow graphics on a Red background.

I will check my algorithm.

I changed the Palette in IrfanView to give Blue graphics on a Cyan background.  
This renders fine.

I will have to compare the file contents.  But this will be later this afternoon.

Quote
Why don't I get 255 255 255 and 0 0 0, as I would expect from info of IrfanView?
but 255 0 0 and 0 255 0 instead?
Ah-ha.  My display must be GREEN graphics on Red background.   Not Yellow.
So the algorithm is correct.   I must see why IrfanView is different to ImageMagick.

David.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 23, 2019, 03:58 pm
Yes,  the formats are different:    output6.bmp was generated by IrfanView Blue/Cyan
Code: [Select]

// file:output5.bmp ID  4D42
// size 6466
// offset       130
// width        176
// height       264
// planes       1
// depth        1
// compression  0
// pixelsize    6336
// colorsused   2
// colorsimportant      2

// file:output6.bmp ID  4D42
// size 6398
// offset       62
// width        176
// height       264
// planes       1
// depth        1
// compression  0
// pixelsize    6336
// colorsused   2

Note that ImageMagick generated output5.bmp has an extra "colorsimportant" field.
The actual bitmap rendered correctly from the respective "offset"

Code: [Select]

filespec: output5.bmp
000000 42 4D 42 19 00 00 00 00 00 00 82 00 00 00 6C 00 *BMB...........l.*
000010 00 00 B0 00 00 00 08 01 00 00 01 00 01 00 00 00 *..░.............*
000020 00 00 C0 18 00 00 00 00 00 00 00 00 00 00 02 00 *..└.............*
000030 00 00 02 00 00 00 00 00 FF 00 00 FF 00 00 FF 00 *................*
000040 00 00 00 00 00 FF 42 47 52 73 00 00 00 00 00 00 *......BGRs......*
000050 00 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 *.....@..........*
000060 00 40 00 00 00 00 00 00 00 00 00 00 00 40 00 00 *.@...........@..*
000070 00 00 00 00 00 00 00 00 00 00 FF FF FF 00 00 00 *................*
000080 00 00 *..*


Code: [Select]

filespec: output6.bmp
000000 42 4D FE 18 00 00 00 00 00 00 3E 00 00 00 28 00 *BM■.......>...(.*
000010 00 00 B0 00 00 00 08 01 00 00 01 00 01 00 00 00 *..░.............*
000020 00 00 C0 18 00 00 00 00 00 00 00 00 00 00 02 00 *..└.............*
000030 00 00 00 00 00 00 FF FF 00 00 FF 00 00 00 *..............*


You can see the colorsimportant field at offset 0x0032 means that the Palette uses a different format.  i.e. Palette is @ 0x007A [FF FF FF 00] [00 00 00 00] White-Black
The IrfanView Palette is @ 0x0036 [FF FF 00 00] [FF 00 00 00] Cyan-Blue

Here is a Wikipedia article that describes the BMP file format (https://en.wikipedia.org/wiki/BMP_file_format#DIB_header_(bitmap_information_header))

I suspect that ImageMagick can generate both formats.   I wonder whether it is worth the effort to support the extra
Quote
An OS/2 2.x OS22XBITMAPHEADER (BITMAPINFOHEADER2 in IBM's documentation) contains 24 additional bytes
David.

p.s. I have just been listening to an excellent play on the Radio about Nikolai Gogol (my hero)
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: ZinggJM on Oct 23, 2019, 04:45 pm
@David,

yes, I once had a look at that Wikipedia article, and decided it is too complicated for my practical use, and I stay with code that is widely used for Arduino libraries, e.g. the one I found in your library.

Do you have a practical suggestion how to deal with this for users of our libraries?

I downloaded ImageMagick, but I still need some time to learn how to use it.

Users like @rlightner and others could profit from a workaround.

Thank you.

Jean-Marc
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: rlightner on Oct 23, 2019, 04:51 pm
The reason why I'm doing it the way I am is that the file encoded this way is only 7 kb, but with the other way, comes out to around 138 kb. I'm trying to minimize the sending over the wire to the device. If at the end of the day I just need to take the hit, I will.

I do appreciate all the work you guys have given to the community. It's a great building block for others to create upon!
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 23, 2019, 05:12 pm
Quote
You can see the colorsimportant field at offset 0x0032 means that the Palette uses a different format.  
i.e. Palette is @ 0x007A [FF FF FF 00] [00 00 00 00] White-Black
The IrfanView Palette is @ 0x0036 [FF FF 00 00] [FF 00 00 00] Cyan-Blue
So you just have to read 0x32 and if 2 you seek to 0x007A instead of 0x0036
It should not be more than two lines.  e.g.
Code: [Select]


        if (bmpDepth <= PALETTEDEPTH) {   // these modes have separate palette
            bmpFile.seek(50); //colorsimportant
            bmpFile.seek(read32() ? 122 : 54); //important or regular palette


 

If there are ImageMagick users out there,  please could you try creating BMP with different colour depths.
IrfanView recognises the colorsimportant field but does not seem to generate it.

I will post the patch on the BMP examples on the MCUFRIEND_kbv Beta.

David.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: ZinggJM on Oct 23, 2019, 05:38 pm
@David, thank you, now I see it clearly enough; my brain was a bit tired, it seems.

@rlightner,

I will do the change tomorrow morning after a quick test, and put it on GitHub without creating a new version for now.

Jean-Marc
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 23, 2019, 05:41 pm
I have posted the change on GitHub e.g. https://github.com/prenticedavid/MCUFRIEND_kbv/tree/master/examples/showBMP_kbv_Uno

@rlightner,
you can edit the drawBitmapFromSpiffs() function in the sketch:

Code: [Select]

        if (depth <= 8)
        {
          if (depth < 8) bitmask >>= depth;
          //file.seek(54); //palette is always @ 54
            file.seek(50); //colorsimportant
            file.seek(read32() ? 122 : 54); //important or regular palette


Please test your existing file.

If all works ok,  I would appreciate it if you go back to ImageMagick
See if you can start with a colour JPG and reduce colours to 256-colour, 16-colour, 2-colour BMP files.

It would be nice if you posted the result.   e.g. attach the files in a ZIP

David.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: rlightner on Oct 23, 2019, 06:25 pm
Will do!
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 23, 2019, 07:06 pm
I have just installed ImageMagick on this Win10-64 PC

I am lost.   I can't even find how to set a Path to the program or even send a simple command

Code: [Select]

 "C:\Program Files\ImageMagick-7.0.8-Q16\magick.exe" tractor10.jpg -colors 256 tractor_8.bmp

I just get Parser errors.

David.

Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: rlightner on Oct 23, 2019, 07:14 pm
Should have put it under your "Program Files" directory if you installed the 64 bit version.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 23, 2019, 07:22 pm
I just downloaded the Windows installer.   I assume that it knows where it wants to store stuff.

IrfanView calls itself "v4.41 64-bit" but installed itself in "/Program Files" and not "/Program Files (x86)"

David.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: rlightner on Oct 23, 2019, 07:25 pm
Correct, if your OS is 64bit, 64bit apps get installed in "/Program Files" and 32bit go in the "/Program Files (x86)" folder.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 23, 2019, 08:30 pm
Woo-Hoo.   This Win10 PC defaults to PowerShell which is horrible.
I found cmd.exe and everything works nicely.

I have created some BMP files with different color depth.   I get different values in the colorsimportant field e.g. 2, 16, 256

But the result is horrible.

I have a lot of learning to do.

IrfanView tends to do this stuff for you e.g. if you ask it to reduce to 16 colours it makes the "best possible" Palette and the end result looks good.

David.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: rlightner on Oct 23, 2019, 11:25 pm
I'm executing this via a function in firebase, which has 'convert' available to the environment, so its not something I can do by hand.

this is the line that gets executed:

Code: [Select]

convert **from_filename.png** -filter point -resize "176x264>" -dither FloydSteinberg -define dither:diffusion-amount=85% -monochrome -colors 2 **output_filename.bmp**
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 23, 2019, 11:54 pm
I have made a safer calculation for MCUFRIEND_kbv examples:
Code: [Select]

        if (bmpDepth <= PALETTEDEPTH) {   // these modes have separate palette
            bmpFile.seek(bmpImageoffset - (4<<bmpDepth)); //54 for regular, diff for colorsimportant
            bitmask = 0xFF;


your edit would be:
Code: [Select]

        if (depth <= 8)
        {
          if (depth < 8) bitmask >>= depth;
            file.seek(imageOffset - (4<<depth)); //54 for regular, diff for colorsimportant
          //file.seek(54); //palette is always @ 54


Thanks for your sample command.   I am struggling with ImageMagick.  
I have created tractor_1.bmp tractor_4.bmp tractor_8.bmp with ImageMagick
and created tractor_11.bmp tractor_44.bmp tractor_88.bmp with IrfanView

ImageMagick creates bigger headers on all 3 files.   And creates a RLE bitmap for the 256-colour file.
My programs reject the RLE flag as an unsupported format.

I have attached a ZIP.

David.

Edit.  Added output6.bmp to ZIP.   This was generated by Irfanview with Blue graphics and Cyan background.
Jean-Marc's program should distinguish between the two colours i.e. Cyan->WHITE and Blue->BLACK

Code: [Select]

magick tractor10.jpg -depth 1 -colors 2 tractor_1.bmp
magick tractor10.jpg -depth 4 -colors 16 tractor_4.bmp
magick tractor10.jpg -depth 8 -colors 256 tractor_8.bmp
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: rlightner on Oct 24, 2019, 12:00 am
What command are you using with imagemagick to create them?
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 24, 2019, 10:46 am
Using the simple command produced rubbish BMP files.  
Most importantly the 256-colour BMP defaults to using RLE8 compression (which I don't support)
Not only does it produce RLE8 but the RLE8 files are bigger than uncompressed files!

I have improved my ImageMagick commands e.g.
Code: [Select]

magick convert tractor10.jpg -compress none -type palette -dither FloydSteinberg -colors 256 -depth 8 tractor_8.bmp

magick convert marilyn-monroe-9412123-1-402.jpg -resize 20% -compress none -type palette -dither FloydSteinberg -colors 2 -depth 1 marilyn_1.bmp


My original tractor JPEG is 276x182.   I do not change the size
My original Marilyn JPEG is 1200x1200.  I resize it to 240x240

If anything,  ImageMagick seems to produce a slightly better monochrome than IrfanView.

Although there is a lot to learn,  the command line ImageMagick is much more powerful than a GUI.

I don't know whether you own a colour TFT e.g. to view all the BMP in colour.
I would like to know whether Jean-Marc's sketch displays them on EPaper and how effectively.

You can obviously view the BMP files on the PC to see what they should look like.
256-colour Palette is very good.
16-colour Palette is surprisingly good.
2-colour Palette is as good as you can expect.

David.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: ZinggJM on Oct 24, 2019, 01:44 pm
Version 1.2.2 of my library GxEPD2 (https://github.com/ZinggJM/GxEPD2) is available.

- fixed BMP handling, e.g. for BMPs created by ImageMagick

@rlightner, @david_prentice, thank you for your contributions.

Jean-Marc
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: david_prentice on Oct 24, 2019, 04:22 pm
@Jean-Marc,

What is your opinion of color BMP rendered on a monochrome EPaper ?

If I build a monochrome Palette e.g.
Code: [Select]

            for (col = 0; col < n; col++) {
                pos = read32(bmpFile);    //map palette to 5-6-5
                uint8_t r = (pos & 0xFF0000) >> 16;
                uint8_t g = (pos & 0x00FF00) >> 8;
                uint8_t b = (pos & 0x0000FF) >> 0;
                palette[col] = (r + g + b < 0x0180) ? 0x0000 : 0xFFFF;  //monochrome
                //palette[col] = tft.color565(r, g, b);  //regular color
                //palette[col] = ((pos & 0x0000F8) >> 3) | ((pos & 0x00FC00) >> 5) | ((pos & 0xF80000) >> 8);
            }

The 16-color and 256-color BMPs look a bit bleached.   Adjusting the 0x0180 threshold helps but is not as good as ImageMagick or IrfanView.

I presume that EPaper users will often want to create good monochrome.

Incidentally,   a 16-Grayscale OLED looks pretty good.   Much like my surprise with 16-color BMP.

David.
Title: Re: 1 bit depth BMP created with ImageMagick all black using GxEPD2
Post by: ZinggJM on Oct 24, 2019, 05:23 pm
@David

Quote
What is your opinion of color BMP rendered on a monochrome EPaper ?
It is in any case a (bad) compromise. And it is even worse with 3-color e-paper, b/w/r or b/w/y.

I think it is preferable to have the user generate and adjust a b/w or 3-color BMP for his purpose.

But my BMP examples just should be able to render any BMP bitmap.
I didn't experiment with thresholds, and the code may even be wrong, at least earlier versions were.

I didn't pay much attention testing your tractor bitmaps so far, they looked more or less the same.
Maybe I take a second look tomorrow, and also at your bitmaps2, and on b/w/r e-paper.

Maybe I should mention that some SPI b/w e-paper displays can be used to show 4 grey levels, using a special waveform table and the fact that the "previous" and "new" buffer provide 2 bits per pixel, normally used for differential update. Parallel IF e-paper displays usually have 16 grey levels.

Jean-Marc