Go Down

Topic: screenshotToFat() for ILI9341_due library? (Read 909 times) previous topic - next topic

HermannSW

Nov 02, 2015, 08:23 pm Last Edit: Nov 02, 2015, 08:31 pm by HermannSW
Hi,

I successfully connected ILI9340 2.2" display and its SD card reader to 3.3V powered Arduino Nano, here is the wiring I used:
https://twitter.com/HermannSW/status/659307846438420480

I was able to run ILI9341_due library sdFatTftBitmap sample. I did remove the delays between the pictures read (in only 970ms!) from SDcard, looks nice:
https://twitter.com/HermannSW/status/658764112529367041

This demo proved that SD card and display can be used together because of the different CS pins.

I really like ILI9341_due library screenshotToConsole() function, and now I tried to add screenshotToFat() function to the library. Although the display content keeps constant during screenshot, the 320x240 SCRSHOT.565 from SD card does not show the display content. I added code to screenshotToFat() for sending same data written to Fat to serial console as well for comparison (without runlength encoding, no problem because that call will ago finally). The result of conversion of the hex console string to ppm looks the same as what ends up on SD card:


A soon as I comment out the SD card commands the data written to console is the correct picture (in 39sec).


I tried to disableCS/enableCS during writing to FAT but it did not work.

This is the complete diff (modified files attached):
Code: [Select]
$ cd libraries/ILI9341_due-master/
$ diff ILI9341_due.h.orig ILI9341_due.h
544a545
>         void screenshotToFat();
$
$ diff ILI9341_due.cpp.orig ILI9341_due.cpp
53a54,55
> #include <SdFat.h>
> #include <SdFatUtil.h>
965a968,1053
> void ILI9341_due::screenshotToFat()
> {
> uint8_t color[3];
>         uint16_t lcdbuffer;
> uint32_t t0, t1;
> SdFile   bmpFile;
>         SdFat sd;
>         const uint8_t chipSelect = 4; //SS;
>
>         char buf[0x36]={
>          0x42, 0x4d, 0x36, 0x84, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x36, 0x00, 0x00, 0x00, 0x28, 0x00,
>          0x00, 0x00, 0x40, 0x01, 0x00, 0x00, 0xf0, 0x00, 0x00, 0x00, 0x01, 0x00, 0x10, 0x00, 0x00, 0x00,
>          0x00, 0x00, 0x00, 0x84, 0x03, 0x00, 0xe9, 0x24, 0x00, 0x00, 0xe9, 0x24, 0x00, 0x00, 0x00, 0x00,
>          0x00, 0x00, 0x00, 0x00, 0x00, 0x00
>         };
>
> Serial.println();
> Serial.println(F("==== PIXEL DATA START ===="));
>
>         t0 = micros();
>
>         if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();
>
> if (!bmpFile.open("scrshot.565", O_CREAT | O_WRITE)) {
> Serial.println("File open failed.");
> return;
> }
> else {
> Serial.println("File opened.");
> }
>
>         bmpFile.write(buf, sizeof(buf));
>
> //        sd.card()->chipSelectHigh();
>
> beginTransaction();
> setAddr_cont(0, 0, _width, _height);
> writecommand_cont(ILI9341_RAMRD); // read from RAM
> readdata8_cont(); // dummy read, also sets DC high
>
> //        disableCS();
> //   enableCS();
> //#if SPI_MODE_NORMAL | SPI_MODE_DMA
> //   enableCS();
> //#endif
>
> for (uint32_t i = 1; i <= (uint32_t)_width*(uint32_t)_height; i++)
> {
> #if SPI_MODE_DMA
> read_cont(color, 3);
> #elif SPI_MODE_NORMAL | SPI_MODE_EXTENDED
> color[0] = read8_cont();
> color[1] = read8_cont();
> color[2] = read8_cont();
> #endif
>                 lcdbuffer = color565(color[0], color[1], color[2]);               
>
> //                sd.card()->chipSelectLow();
>
>                 bmpFile.write(&lcdbuffer, 2);
>
> printHex8(color, 3);
>
> //                sd.card()->chipSelectHigh();
>
> }
> #if 0
> for(uint32_t i=1; i<19; ++i) {
>                 lcdbuffer = color565(0,0,255);               
>                 bmpFile.write(&lcdbuffer, 2);
> }
> #endif
> disableCS();
> endTransaction();
>
> //        sd.card()->chipSelectLow();
>         bmpFile.close();
>
>         t1 = micros();
>
> Serial.println();
> Serial.println(F("==== PIXEL DATA END ===="));
> Serial.print(F("time [us]: "));
> Serial.println(t1-t0);       
> }
>
$


I also attached hex2toppm.c that converts the uncompress 320*240*6 length hex string from Serial console to .ppm picture.


My questions are:
  • Is screenshotToFat possible at all, or is there something preventing it?
  • What is missing/needs to be done differently in screenshotToFat()?



I really think that screenshotToFat() is useful.
And I will need its technique anyway since next on my list is to get JTRON OV7670 camera running and displaying photo on ILI9340 2.2" display.


Hermann.

HermannSW

Did a histogram of the colors in both pictures above. My idea was that disarranged screenshot might have the same pixels as the good screenshot just at different places. Since the colors are different something else happens that seems to break the 3-byte border between pixel data, at least the number of black pixels only increased by 89 compared top good screenshot. Not sure if this histogram can be helpful in diagnosing the disarrangement cause.

Code: [Select]
$ ppmhist disarrangement.ppm
   r     g     b    lum count 
 ----- ----- ----- ----- -------
     0     0     0     0   72169
   196   196   192   196    1685
   196   192   196   194    1131
   192   196   196   195    1119
   120   120   124   120     170
   120   124   120   122     105
     0   196   192   137      99
   196     0     0    59      97
   192   196     0   172      70
     0     0   196    22      69
   124   120   120   121      62
     0   120   124    85      10
   120     0     0    36       8
   124   120     0   107       3
     0     0   120    14       3
$
$ ppmhist good.ppm
   r     g     b    lum count 
 ----- ----- ----- ----- -------
     0     0     0     0   72080
   196   192   196   194    4176
   120   124   120   122     544
$


Hermann.

MarekB

Just from a quick look..

Code: [Select]
read_cont(color, 3);

keeps TFT CS low (tft spi active). So when you're doing

Code: [Select]
bmpFile.write(&lcdbuffer, 2);

you are effectively writing to SD and also to TFT. Try DisableCS before bmpFile.write and EnableCS after it.

HermannSW

Thanks for the explanation.

I found a way around the issue, which also avoids writing to TFT while writing to SD card.

Below diff does a "good" screenshot (wrong colors because of 565 format, and upsidedown).
It reads one line (320*3 bytes), converts that to 320*2 "565" 16bit lcdbuffer, and then write lcdbuffer to SD card.

This is photo of screen:


This is the SCRSHOT.565 file (saved as .jpg):



What's not that nice is that taking the screenshot takes 3 seconds:
Code: [Select]
==== PIXEL DATA START ====
File opened.

==== PIXEL DATA END ====
time [ms]: 3022



This is the diff of ILI9341_due.cpp (that file is attached as well):
Code: [Select]
$ diff ILI9341_due.cpp.orig ILI9341_due.cpp
965a966,1043
> #include <SdFat.h>
> #include <SdFatUtil.h>
>
> void ILI9341_due::screenshotToFat()
> {
>     uint8_t        color[3];
>     uint16_t       *lcdbuffer;
>     uint32_t       t0, t1;
>     SdFile         bmpFile;
>     SdFat          sd;
>     const uint8_t  chipSelect = 4; //SS;
>
>     char buf[0x36]={
>         0x42, 0x4d, 0x36, 0x84, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x36, 0x00, 0x00, 0x00, 0x28, 0x00,
>         0x00, 0x00, 0x40, 0x01, 0x00, 0x00, 0xf0, 0x00, 0x00, 0x00, 0x01, 0x00, 0x10, 0x00, 0x00, 0x00,
>         0x00, 0x00, 0x00, 0x84, 0x03, 0x00, 0xe9, 0x24, 0x00, 0x00, 0xe9, 0x24, 0x00, 0x00, 0x00, 0x00,
>         0x00, 0x00, 0x00, 0x00, 0x00, 0x00
>     };
>
>     Serial.println();
>     Serial.println(F("==== PIXEL DATA START ===="));
>
>     t0 = millis();
>
>     lcdbuffer = (uint16_t*) malloc(_width*2);   
>
>     if (lcdbuffer==NULL) {
>         Serial.println("Cannot allocate lcdbuffer");
>         return;
>     }
>
>     if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();
>
>     if (!bmpFile.open("scrshot.565", O_CREAT | O_WRITE)) {
>         Serial.println("File open failed.");
>         return;
>     }
>     else {
>         Serial.println("File opened.");
>     }
>
>     bmpFile.write(buf, sizeof(buf));
>
>     for (uint32_t i = 0; i < (uint32_t)_height; i++)
>     {
>         beginTransaction();
>         setAddr_cont(0, i, _width, 1);
>         writecommand_cont(ILI9341_RAMRD); // read from RAM
>         readdata8_cont(); // dummy read, also sets DC high
>
>         for (uint32_t j = 0; j < (uint32_t)_width; j++)
>         {
> #if SPI_MODE_DMA
>             read_cont(color, 3);
> #elif SPI_MODE_NORMAL | SPI_MODE_EXTENDED
>             color[0] = read8_cont();
>             color[1] = read8_cont();
>             color[2] = read8_cont();
> #endif
>             lcdbuffer[j] = color565(color[0], color[1], color[2]);               
>         }
>
>         disableCS();
>         endTransaction();
>
>         bmpFile.write(lcdbuffer, _width*2);
>     }
>
>     bmpFile.close();
>
>     t1 = millis();
>
>     Serial.println();
>     Serial.println(F("==== PIXEL DATA END ===="));
>     Serial.print(F("time [ms]: "));
>     Serial.println(t1-t0);       
> }
>
$


Hermann.

robtillaart

#4
Nov 03, 2015, 09:02 pm Last Edit: Nov 03, 2015, 09:12 pm by robtillaart
as the screen is not that big you might change some uint32_t ==> uint16_t

e.g.  for (uint32_t i = 0; i < (uint32_t)_height; i++)
       for (uint32_t j = 0; j < (uint32_t)_width; j++)

that might speed up a little ?


you might also inline this call as it is done for every pixel.

lcdbuffer[j] = color565(color[0], color[1], color[2]);

==>

lcdbuffer[j] = (((uint16_t) color[0]) & 0xF8) << 8 ) |
                   (((uint16_t) color[1] & 0xFC) << 4) |
                   (((uint16_t) color[2] & 0xF8) >> 3);  // please check the bitmath

give it a try
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

HermannSW

#5
Nov 04, 2015, 12:45 pm Last Edit: Nov 04, 2015, 01:10 pm by HermannSW
Hi,

color565 seems to be always inlined already, but the column type downgrade from uint32_t to uint16_t won 267ms.

I will keep the screenshot .565 file upsidedown because that is the format of the ILI9341_due library sdFatTftBitmap sample/demo.

Here is complete list of enhancement steps I did. Biggest difference (>10s) is on whether screenshot file was already present on SD card from previous screenshot or not. The numbers are starting with previously posted version, in milliseconds:
  • 13751  1st screenshot file write
  • 2898  repeated screenshot
  • 2622  move sd.begin() out (needs run once only)
  • 2621  remove "File opened" output to Serial
  • 2082  SPI_FULL_SPEED for SD card, possible because short SPI wires
  • 2078  rows i -> uint8_t
  • 1811  cols j -> unit16_t
  • 12565 1st screenshot file write
  • 1816  repeated screenshot
  • 1797  cols j -> uint8_t (up to _width/2, double code block)
  • 950   just read line from TFT, comment out write line to SD
  • 784   comment out bmp header SD card write as well


I moved everything into a single .inc file, no difference in ILI9341_due.cpp any more:
Code: [Select]
$ diff ILI9341_due.cpp.orig ILI9341_due.cpp
$
$ diff ILI9341_due.h.orig ILI9341_due.h
94a95,97
> #include <SdFat.h>
> #include <SdFatUtil.h>
>
544a548
> #include "ILI9341_due.inc"
$


All modified/new files are attached, you will have to decide whether you want blown code for going from .1811 to .1797 version.

The last changes (commenting out SD card code) show 0.8s as time needed to just do the TFT read and line prepare stuff. The SD card file write adds another second. While 1.8s is much faster than 3 seconds earlier, it is still not really "fast".

I did use (a 3.3V powered) Arduino Nano with 2KB SRAM. Allocating 2*320 bytes for a single TFT line via malloc works fine. But allocating 3*320 or two times 320*3/2 fails to allocate memory.

So currently I do not see how to speed up the 1.8s[=0.8s(TFT)+1s(SD)] screenshotToFat() time further -- on the other hand doing that in 1.8s with only 2KB of SRAM is perhaps more impressive than I currently feel ...

Last, but not least, here is screenshotToFat() generated screenshot from giraffe.565 displayed in sdFatTftBitmap demo:
Code: [Select]
$ diff giraffe.565 SCRSHOT.565
$


Hermann.


HermannSW

#6
Nov 06, 2015, 09:37 pm Last Edit: Nov 06, 2015, 09:39 pm by HermannSW
Okay, the wrongly colored display of the ILI9341_due demo files and of my screenshots on Linux didn't leave me alone. I wanted to know what is responsible for the wrong colors.

I started with a test on whether my 2.2" TFT display is really a 565rgb display with this code creating 16 single-bit-color rectangles:
Code: [Select]
tft.setRotation(iliRotation270);
for(int i=0; i<16; ++i)
{
    tft.fillRect((15-i)*20,0,20,240,(1<<i));
}


This photo is not perfect, but it shows 5 red rectangles on the left, 6 green in the middle and 5 blue on the right:



Next I looked up the BMP file format description, but that is not a simple one:
https://en.wikipedia.org/wiki/BMP_file_format

With a little further searching I was lucky to find "bmpsuite":
http://entropymine.com/jason/bmpsuite/bmpsuite/html/bmpsuite.html

The picture I needed for 565 TFT format was "g/rgb16-565.bmp".
"bmpsuite.c" does generate all the demo pictures, here the 565 part:
Code: [Select]
defaultbmp(c);
c->filename = "g/rgb16-565.bmp";
c->bpp = 16;
c->pal_entries = 0;
c->compression = BI_BITFIELDS;
c->bf_r = 0x0000f800; c->nbits_r = 5; c->bf_shift_r = 11;
c->bf_g = 0x000007e0; c->nbits_g = 6; c->bf_shift_g = 5;
c->bf_b = 0x0000001f; c->nbits_b = 5; c->bf_shift_b = 0;
c->bitfieldssize = 12;
set_calculated_fields(c);
if(!make_bmp_file(c)) goto done;


The difference to the ILI9341_due library is the compression setting.
While ILI9341_due does compression none (0x00), 565 really needs BI_BITFIELDS.

I did screenshotToFat() for above screen and got this stored on SD card:


So compression none with 16bit seems to be 555 -- that is the cause for the color problems.


I do want to use ILI9341_due library, and therefore will keep the 565 compression none format.
But I created a simple converter bash script, which can be used in Linux or Cygwin under WIndows:
Code: [Select]
$ cat ~/bin/correct.qvga
#!/bin/bash
echo -en "\x42\x4D\x42\x58\x02\x00\x00\x00\x00\x00\x42\x00\x00\x00\x28\x00"
echo -en "\x00\x00\x40\x01\x00\x00\xF0\x00\x00\x00\x01\x00\x10\x00\x03\x00"
echo -en "\x00\x00\x00\x58\x02\x00\x13\x0B\x00\x00\x13\x0B\x00\x00\x00\x00"
echo -en "\x00\x00\x00\x00\x00\x00\x00\xF8\x00\x00\xE0\x07\x00\x00\x1F\x00"
echo -en "\x00\x00"
for((y=320*2; y<240*320*2; y+=320*2))
do
  tail --bytes $y $1 | head --bytes 640
done
$


I used it to correct the giraffe.565 QVGA photo from ILI9341_due:
Code: [Select]
$ correct.qvga ILI9341_due-master/examples/sdFatTftBitmap/images/giraffe.565 > giraffe.565.bmp
$


Here are links to both files:
https://stamm-wilbrandt.de/en/forum/giraffe.565
https://stamm-wilbrandt.de/en/forum/giraffe.565.bmp

And here is the image further converted to jpeg for browser display:



Chrome, Firefox and Opera browsers can display .bmp file, both under Linux and under Windows.
Interestingly IE 11 cannot ... ;-)

Hermann.

robtillaart

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

MarekB

The difference to the ILI9341_due library is the compression setting.
While ILI9341_due does compression none (0x00), 565 really needs BI_BITFIELDS.

So compression none with 16bit seems to be 555 -- that is the cause for the color problems.

I do want to use ILI9341_due library, and therefore will keep the 565 compression none format.
Hi Hermann,

can you help me understand this a bit more. As far as I know ILI9341_due does not do any checks on nor writes any bitmap header (which if I understand correctly is the problem here). Or are you talking about sdFatTftBitmap example or BMP24toILI565 conversion tool? What ILI9341_due cares about is the pixel data, there can be anything in the header. sdFatTftBitmap example is doing a check for compression None but that's not necessary. If I remember correctly the only useful information extracted from the header is the width and height of the image.

HermannSW

#9
Nov 06, 2015, 11:32 pm Last Edit: Nov 06, 2015, 11:37 pm by HermannSW
Hi Marek,

you are right, ILI9341_due does not deal with BMP headers.
Just the "sdFatTftBitmap.ino" example does.
Besides some checks it does a fixed header skip:
"bmpFile.seekSet(54);   //skip header"

Both formats are correct, the compress=none from "examples/sdFatTftBitmap/images/*.565" files:
Code: [Select]
0x00: 42 4D
0x02: 36 84  03 00
0x06: 00 00    00 00
0x0A: 36 00  00 00
0x0E: 28 00  00 00
0x12: 40 01  00 00
0x16: F0 00  00 00
0x1A: 01 00    10 00
0x1E: 00 00  00 00 compres
0x22: 00 84  03 00 img siz
0x26: E9 24  00 00 x/m
0x2A: E9 24  00 00 y/m
0x2E: 00 00  00 00 colors in table
0x32: 00 00  00 00 important color co
0x36:


as well as the compress=BI_BITFIELDS I found in bmpsuite:
Code: [Select]
0x00: 42 4D
0x02: 42 40  00 00 file size
0x06: 00 00    00 00  rsvd1 rsvd2
0x0A: 42 00  00 00 offset pix
0x0E: 28 00  00 00 hdr size
0x12: 7F 00  00 00 width
0x16: 40 00  00 00 height
0x1A: 01 00    10 00  planes/bits per pix
0x1E: 03 00  00 00 compress
0x22: 00 40  00 00 img size
0x26: 13 0B  00 00 x/m
0x2A: 13 0B  00 00 y/m
0x2E: 00 00  00 00 colors in table
0x32: 00 00  00 00 important color cnt
0x36: 00 F8  00 00 red
0x3A: E0 07  00 00 green
0x3E: 1F 00  00 00 blue
0x42:



"bmpFile.seekSet(bmpImageoffset);" in "sdFatTftBitmap.ino" should allow both .bmp image formats to be read.

But I can live with .565 as is, the whole ILI9341_due library is really cool and fast.
Because of the DMA support of the library I even ordered a Due microcontroller two weeks ago ;-)  (for only 12€)
https://twitter.com/HermannSW/status/658677702573076480

Hermann.

HermannSW

#10
Nov 08, 2015, 02:19 pm Last Edit: Nov 08, 2015, 02:22 pm by HermannSW
Just for completeness, did screenshot of red/green/blue image created by this:
Code: [Select]
tft.setRotation(iliRotation270);
bmpDraw("giraffe.565", 0, 0);
tft.fillRect(  0,0,100,240,0xF800); // red
tft.fillRect(100,0,120,240,0x07E0); // green
tft.fillRect(220,0,100,240,0x001F); // blue
tft.screenshotToFat();



This is screenshot file from SD card:
https://stamm-wilbrandt.de/en/forum/rgb/SCRSHOT.565

This is corrected version by "correct.qvga" script:
https://stamm-wilbrandt.de/en/forum/rgb/SCRSHOT.565.bmp

Below is the picture converted further to .jpg for all browsers.

If you open the .565 image in Gimp and then use "Color Picker Tool" and click on the selected colors, you get these (wrong) HTML notation outputs:
  • red: f70000
  • green: 08ff00
  • blue: 0000ff


These are the (correct) values for the .565.bmp file:
  • red: ff0000
  • green: 00ff00
  • blue: 0000ff




Hermann.

Go Up
 


Please enter a valid email to subscribe

Confirm your email address

We need to confirm your email address.
To complete the subscription, please click the link in the email we just sent you.

Thank you for subscribing!

Arduino
via Egeo 16
Torino, 10131
Italy