GIGA with display shield and OV7670

I received by display shield yesterday, so I thought I would try a few experiments. Like I tried the GigaCameraDisplay example, which worked, which I mentioned over on the Teensy forum:
Teensy 4.1 or Arduino GIGA R1 | Teensy Forum (pjrc.com)

But did not like that the image was rotated 90 degrees and the, and was curious if the display libraries supported rotation of the display at all... So I did a quick and dirty GFX test:

REDIRECT_STDOUT_TO(Serial)

#include "Arduino_GigaDisplay_GFX.h"

GigaDisplay_GFX display;
#define GC9A01A_CYAN    0x07FF
#define GC9A01A_RED     0xf800
#define GC9A01A_BLUE    0x001F
#define GC9A01A_GREEN   0x07E0
#define GC9A01A_MAGENTA 0xF81F
#define GC9A01A_WHITE   0xffff
#define GC9A01A_BLACK   0x0000
#define GC9A01A_YELLOW  0xFFE0
uint16_t colors[] = {GC9A01A_RED, GC9A01A_BLUE, GC9A01A_GREEN, GC9A01A_RED, GC9A01A_BLUE, GC9A01A_BLACK};

void setup() {
  while (!Serial && millis() < 5000) { }
  Serial.begin(115200);
}
uint8_t rotation = 3;
void loop() {
  rotation = (rotation + 1 ) & 0x3;
  display.setRotation(rotation);
  printf("Rotation: %u, W:%u H:%u\n", rotation, display.width(), display.height());
  display.fillScreen(colors[rotation]);
  display.fillRect(120, 180, 120, 180, 0x8888);
  display.setTextSize(4);
  display.setCursor(display.width()/2, display.height() / 2);
  display.print(rotation);
  delay(1000);
}

Which worked.

So, I then made a modified version of the camera display software that uses GFX:

Ran into a few issues with the code, like:

g_camera_width = cam.getResolutionWidth();
g_camera_height = cam.getResolutionHeight();

Found that these two functions are in the header file, but are not implemented.
Thought about checking/issuing ISSUE or the like, but already something like 186 open issues and 26 open Pull Requests, so not sure if this library is being maintained or not...

Also ran into issue with the sketch include:
#include "SDRAM.h"
As one of the library header files I included also include it, and this header file is not protected:
i.e. it does not either have:
#pragma once
Or a check like:

#ifndef __SDRAM_H__
#define __SDRAM_H__
...
#endif

EDIT: - forgot to mention that, the camera did not work if I did:
fb.setBuffer((uint8_t *)SDRAM.malloc(640 * 480 * 2));
As the memory was not aligned to 32 byte boundary like it does internal to the library when it calls malloc... So I put in the alignment code:
(End Edit)

But got it to compile and run:

REDIRECT_STDOUT_TO(Serial)
#include "Arduino_GigaDisplay_GFX.h"
#define GC9A01A_CYAN 0x07FF
#define GC9A01A_RED 0xf800
#define GC9A01A_BLUE 0x001F
#define GC9A01A_GREEN 0x07E0
#define GC9A01A_MAGENTA 0xF81F
#define GC9A01A_WHITE 0xffff
#define GC9A01A_BLACK 0x0000
#define GC9A01A_YELLOW 0xFFE0
#define ALIGN_PTR(p,a)   ((p & (a-1)) ?(((uintptr_t)p + a) & ~(uintptr_t)(a-1)) : p)


#include "arducam_dvp.h"
//#include "SDRAM.h"

// This example only works with Greyscale cameras (due to the palette + resize&rotate algo)
//#define ARDUCAM_CAMERA_HM01B0

#include "OV7670/ov767x.h"
OV7670 ov767x;
//OV7675 ov767x;
Camera cam(ov767x);
#define IMAGE_MODE CAMERA_RGB565

// The buffer used to capture the frame
FrameBuffer fb;

// The buffer used to rotate and resize the frame
GigaDisplay_GFX display;
uint16_t g_camera_width;
uint16_t g_camera_height;

void blinkLED(uint32_t count = 0xFFFFFFFF) {
  pinMode(LED_BUILTIN, OUTPUT);
  while (count--) {
    digitalWrite(LED_BUILTIN, LOW);   // turn the LED on (HIGH is the voltage level)
    delay(50);                        // wait for a second
    digitalWrite(LED_BUILTIN, HIGH);  // turn the LED off by making the voltage LOW
    delay(50);                        // wait for a second
  }
}

void setup() {
  while (!Serial && millis() < 5000) {}
  Serial.begin(115200);

  // Init the cam QVGA, 30FPS
  SDRAM.begin();
  Serial.println("Before camera start"); Serial.flush();
  if (!cam.begin(CAMERA_R640x480 /*CAMERA_R320x240 */, IMAGE_MODE, 10)) {
    blinkLED();
  }

  Serial.println("Before setBuffer"); Serial.flush();

  uint8_t *fb_mem = (uint8_t *)SDRAM.malloc(640 * 480 * 2 + 32);
  fb.setBuffer((uint8_t *)ALIGN_PTR((uintptr_t)fb_mem, 32));
  printf("Frame buffer: %p\n", fb.getBuffer());

  // clear the display (gives a nice black background)
  Serial.println("Before setRotation"); Serial.flush();
  display.begin();
  display.setRotation(3);
  Serial.println("Before fillscreen"); Serial.flush();
  display.fillScreen(GC9A01A_BLUE);

  g_camera_width = 640; //320; //cam.getResolutionWidth();
  g_camera_height = 480; //240; //cam.getResolutionHeight();
  Serial.println("end setup"); Serial.flush();
}

#define HTONS(x) (((x >> 8) & 0x00FF) | ((x << 8) & 0xFF00))

void loop() {

  // Grab frame and write to another framebuffer
  if (cam.grabFrame(fb, 3000) == 0) {

    // We need to swap bytes.
    uint16_t *pixels = (uint16_t *)fb.getBuffer();
    for (int i = 0; i < g_camera_width * g_camera_height; i++) pixels[i] = HTONS(pixels[i]);
    display.drawRGBBitmap((display.width() - g_camera_width) / 2, (display.height() - g_camera_height) / 2, pixels, g_camera_width, g_camera_height);
  } else {
    blinkLED(20);
  }
}

Note: I did not like the camera orientation in different orientation from the display, so used an adapter I made a while ago for my Teensy 4.1 board which rotates it...

Now back to playing.... Maybe convert one of my SD Image viewers I have for some TFT displays...

1 Like

I have a version of image viewer ported over. So far it is really slow:
There is also an issue with the JPEG decoder with a red tint.


Where the image should look like:
image

The PNG is fine as well as my own decoding of BMP files

The USB SD Stick loading is really slow. The SDFat library is a bit better.

In case anyone wants a look at the Play in Progress:
tft_picture_view_sd_giga_shield-240203a.zip (13.9 KB)

As I mentioned on the SD display of bitmap thread...
There is currently temporary patch of the JPEGDEC library that is working.

3 Likes

Is this the line of code you changed for sdram buffer to run correctly?

#define ALIGN_PTR(p,a)   ((p & (a-1)) ?(((uintptr_t)p + a) & ~(uintptr_t)(a-1)) : p)

See this post:

https://forum.arduino.cc/t/giga-display-shield-using-giga-r1-sdram/1205930

Note: I have two different sketches running to the display shield:

The one that receives data from an OV7670 camera and shows them on the screen.
Current one:
GigaCameraDisplay_GFX-240204a.zip (1.4 KB)
This is the one that I had the ALIGN_PTR function. you mentioned. Where it was then used:

  uint8_t *fb_mem = (uint8_t *)SDRAM.malloc(640 * 480 * 2 + 32);
  fb.setBuffer((uint8_t *)ALIGN_PTR((uintptr_t)fb_mem, 32));
  printf("Frame buffer: %p\n", fb.getBuffer());

This was needed by the OV7670 camera code. If the buffer passed in was not properly aligned the camera functions failed.

For the one that shows a photo (many years old) showing two collies. It had the one line fix in
the JPEGDEC library you mentioned in the other forum thread:

#if !defined(NO_SIMD) && (defined(ARM_MATH_CM4) || defined(ARM_MATH_CM7))
//#define HAS_SIMD
#endif

Current version of that sketch
tft_picture_view_sd_giga_shield-240204a.zip (13.8 KB)

Which may have some of my other libraries included, most can be removed.
This sketch tries to open an USB memory stick and/or SD drive connection.
And then loops through the files on either of those two drives looking for JPEG, BMP, or PNG files and decodes and displays them.

Note: The SD Card is a whole lot faster than the USB stuff. If it finds both drives it defaults to USB, if you hit enter in the serial monitor, it will pause after the current image completes, if then enter a 2 and send, it switches to the SD...

I don't think I am using SDRAM in this sketch... Maybe one of the libraries might...

1 Like

I was attempting to read the screen and store it in an sdram buffer. Couldn't make it work.

Yes, already discovered SDFat being much faster.

Trying out the tft_picture_view_sd_giga_shield code you posted above. I've never used SDIO before. How is it's speed compared to SDFat? And how much difference does GIGA_digitalWriteFast.h make?

Should mention, the sketch is a quick conversion of one I did earlier that does the picture viewer on to an ILI9341 display using conversion of my Teensy ILI9341_t3n library. It has DMA updates to the screen and the like...

I also ran that same sketch on the Portenta H7 board I have. The HAT carrier that I have for it, has the SDIO SD adapter. Arduino has their own library for this, performance was not great.
More details about that over on the thread:

The digitalWriteFast is a lot faster, although in this case may not matter much, Not doing that much changing of IO pins. But it also has the digitalToggleFast call which I like...
More details in the thread:
digitalWriteFast for GIGA? - Hardware / GIGA R1 - Arduino Forum

1 Like

Follow on to some of the issues I mentioned in the first post:

Looks like this one has been fixed last week up on github:
add include guard for SDRAM.h · arduino/ArduinoCore-mbed@d209903 (github.com)

Camera: methods getResolutionWidth and getResolutionHeight are not implemented · Issue #837 · arduino/ArduinoCore-mbed (github.com)

I also opened up another Issue as potentially using one of the constructors could trash memory :
camera: FrameBuffer::FrameBuffer(int32_t x, int32_t y, int32_t bpp) might trash memory · Issue #838 · arduino/ArduinoCore-mbed (github.com)

1 Like

Thanks for the useful info.
Regarding your tft_picture_view_sd_giga_shield code, I'm trying to get it to start reading from a specific directory and draw the jpeg's therein. It just repeats the first jpeg only. Any help would be appreciated. All my SD cards are set up the same way and starting from root dir is not what I need.

Today I was playing around some more with the picture viewer and added an experiment that creates a GFX Canvas. I copied the Adafruit Canvas16 and have it do SDRAM malloc instead of normal malloc.

if when I pause the sketch, and hit c
it toggles on or off using the canvas. Where all of the decoding of the image goes to the canvas and then when done (and depending on timing), it then does one draw of the canvas to the screen, which makes the issues show up faster (from top to bottom)...
tft_picture_view_sd_giga_shield-240205a.zip (17.0 KB)

Wow. are you writing most of the code in this? Amazing but overwhelming. What would help the novices (me) would be some Giga + Display bare bones code that will save screen or a jpeg, etc, to sdram. Too hard to follow all the functions. I have a day job unrelated to this field. :upside_down_face:

In the code you will see things like:

#ifdef CS_SD
    if (!(g_devices_started & SD_DRIVE)) {
      Serial.println("calling SDBEGIN");
      if (SD.begin(SD_CONFIG)) {
        g_devices_started |= SD_DRIVE;
        tft.setCursor(1, 40);
        tft.setTextColor(RED, WHITE);
        tft.println("SD Started");
        if (!root_SD.open("/")) {
          tft.print("Failed to open root_SD directory");
        }
        Serial.print("SD Started");
        em = 0;
      }
    }
#endif

The line: if (!root_SD.open("/")) {
Starts at the root directory... If you instead do something like: "/images"
It would start at that directory name.

The code that enumerates for SDFat, looks something like:

        if (g_current_device == SD_DRIVE) {
        imageFile = root_SD.openNextFile();
        if (!imageFile) {
          if (did_rewind) break;  // only go around once.
          root_SD.rewindDirectory();
          did_rewind = true;
          continue;  // try again
        }
        // maybe should check file name quick and dirty
        name_len = imageFile.getName(file_name, sizeof(file_name));

This should enumerate that directory.
And I don't remember fully with SDFat, (I mostly use SD library on Teensy which is a wrapper of SDFat).
But Probably sufficient to use the FsFile object that is returned from the openNextFile.

If not, you might need to build the complete path name, and then:
do something like: SD.open(pathname, O_READ);
or...

That's basically what I tried, but would apparently rewind to the beginning of the directory I chose, and draw same jpeg over and over...

I would call the function like this:

void loop() {

  fetchDirFiles("Jpg800x480L");
}

Then:

void fetchDirFiles(char* DirName) {
  // don't process unless time elapsed or g_fast_mode
  // Timing may depend on which type of display we are using...
  // if it has logical frame buffer, maybe as soon as we display an image,
  // try to load the next one, and then wait until the image time to
  // tell display to update...
  if (!g_fast_mode && !g_stepMode && (!g_picture_loaded) && (emDisplayed < (uint32_t)g_display_image_time)) return;

  //---------------------------------------------------------------------------
  // Find the next file to read in.
  //---------------------------------------------------------------------------
  if (!g_picture_loaded) {
    bool did_rewind = false;
    uint8_t name_len;
    bool bmp_file = false;
    bool jpg_file = false;
    bool png_file = false;

    Serial.println("\nLoop looking for image file");

    // BUGBUG: this is not overly clean, but tries to allow two different
    // types of file systems...
    WrapperFile wpfImage;
    FsFile imageFile;
    struct dirent *dir_entry;

    char file_name[MAX_FILENAME_LEN];
    for (;;) {

        if (g_current_device == SD_DRIVE) {
          FsFile dirFile;
          if (!dirFile.open(DirName, O_READ)) {  
          //SD.chdir(); return;
          SD.errorHalt("Open directory failed");
         }

        imageFile.openNext(&dirFile);

The rest of code unchanged. Note the last six lines of code or so above. Beginning with FsFile dirFile;

I put

  fetchDirFiles("Jpg800x480L");

in void setup() instead of void loop(). Program stopped after drawing first jpeg. So instead of rewinding, it was looping.

Sorry not sure what that is...

In case any of this is interesting to anyone...
I put a copy of all of my Giga specific test sketches up onto a github project.
Will put a copy of GIGA specific libraries there as well.

But for example the TFT picture viewer is now up at:
Arduino_GIGA-stuff/sketches/tft_picture_view_sd_giga_shield at main · KurtE/Arduino_GIGA-stuff (github.com)

Figured it is a bit more convent than posting zip files.

But need to warn this lots of the stuff was simple test cases or the like. I need to weed through it at some point

1 Like

Thanks for sharing, KurtE. Will take a look this weekend.

BTW, What's the model of your camera? Looks much better than mine.

KurtE:

Figured out a way to read a specific directory in your sketch, 'tft_picture_view_sd_giga_shield' with minimal changes. Draws images from SD or USB.

Add this line before void setup():

char dirUSB[MAX_FILENAME_LEN];

Add this into void setup() at the end:

#ifdef CS_SD
 char* dirName = "/Jpg800x480L"; //Directory holding images
  root_SD.open(dirName); 
  strcpy(dirUSB, "/usb");
  strcat(dirUSB, dirName);   
  root_USB = opendir(dirUSB);
  strcat(dirUSB, "/");
  Serial.println(dirUSB);
#endif          

In void loop(), change line

strcpy(file_name, "/usb/");

to

strcpy(file_name, dirUSB);

I did have some code I did using MDMA/DCMI chaining on the Giga to rotate while blitting, I can't seem to find it now though!

I originally did it for LVGL but it didn't perform well on small areas, large areas it was good though.

I will have a better look tomorrow...

p.s. I have found some notes about general Arduino_GigaDisplay_GFX performance for just blitting where It shows a 8 times speedup using the dsi frame buffer directly and a 10 times speedup using DMA2D.

2 Likes

If you get a chance can you post your notes. Running into some issues with speed on the display shield in different rotations see: Anyway to speed up the display (GFX) - Hardware / GIGA Display Shield - Arduino Forum by @KurtE

1 Like