Arduino portenta h7 vision shield camera troubleshooting

Official Hardware Portenta Family Portenta H7
1

So I've been struggling to get the camera on this arduino portenta h7 vision shield to work for the past few days. I tried the arduino sketch and processing sketch code from this website: https://docs.arduino.cc/tutorials/portenta-vision-shield/getting-started-camera/

I keep getting the "connection timed out" message in the processing sketch terminal, which I see is printed in the draw function. I think that means my camera is unable to receive raw data from the serial port ? Is this an issue with the vision shield or something else I'm doing wrong ?

2

Hi @dzhahahah2 ,

The tutorial was developed using Processing version 3.5.4. Here is the download link:

Please don't forget to uncomment one of lines 34, 35 or 36 of the sketch 'CameraViewer.pde' depending on the OS you are using and indicate the port associated with the Portenta H7 on your computer.

Also, you can leverage the Portenta Vision Shield camera using OpenMV IDE as shown in this tutorial:

In OpenMV IDE > File > Examples > Camera you will find several other examples. :slight_smile:

1 Like
3

Hi @Hm_pro,

Thank you for your response. The blob detection code worked for me in OpenMV, I could see the camera on the vision shield working.

However, the arduino and processing sketch that I mentioned in the first post still don't work, so I think there is probably something I'm doing wrong with the code. I didn't edit the arduino sketch code but I edited the processing sketch code since I use windows. Here it is:

/*
  This sketch reads a raw Stream of RGB565 pixels
  from the Serial port and displays the frame on
  the window.
  Use with the Examples -> CameraCaptureRawBytes Arduino sketch.
  This example code is in the public domain.
*/

import processing.serial.*;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;

Serial myPort;

// must match resolution used in the sketch
final int cameraWidth = 320;
final int cameraHeight = 240;
final int cameraBytesPerPixel = 1;
final int cameraPixelCount = cameraWidth * cameraHeight;
final int bytesPerFrame = cameraPixelCount * cameraBytesPerPixel;

PImage myImage;
byte[] frameBuffer = new byte[bytesPerFrame];
int lastUpdate = 0;
boolean shouldRedraw = false;

void setup() {
  size(640, 480);

  // if you have only ONE serial port active
  //myPort = new Serial(this, Serial.list()[0], 921600);          // if you have only ONE serial port active

  // if you know the serial port name
  myPort = new Serial(this, "COM3", 921600);                    // Windows
  //myPort = new Serial(this, "/dev/ttyACM0", 921600);            // Linux
  //myPort = new Serial(this, "/dev/cu.usbmodem14401", 921600);     // Mac

  // wait for full frame of bytes
  myPort.buffer(bytesPerFrame);  

  myImage = createImage(cameraWidth, cameraHeight, ALPHA);
  
  // Let the Arduino sketch know we're ready to receive data
  myPort.write(1);
}

void draw() {
  // Time out after 1.5 seconds and ask for new data
  if(millis() - lastUpdate > 1500) {
    println("Connection timed out.");
    myPort.clear();
    myPort.write(1);
  }
  
  if(shouldRedraw){    
    PImage img = myImage.copy();
    img.resize(640, 480);
    image(img, 0, 0);
    shouldRedraw = false;
  }
}

void serialEvent(Serial myPort) {
  lastUpdate = millis();
  
  // read the received bytes
  myPort.readBytes(frameBuffer);

  // Access raw bytes via byte buffer  
  ByteBuffer bb = ByteBuffer.wrap(frameBuffer);
  
  /* 
    Ensure proper endianness of the data for > 8 bit values.
    When using > 8bit values uncomment the following line and
    adjust the translation to the pixel color. 
  */     
  //bb.order(ByteOrder.BIG_ENDIAN);

  int i = 0;

  while (bb.hasRemaining()) {
    // read 8-bit pixel
    byte pixelValue = bb.get();

    // set pixel color
    myImage.pixels[i++] = color(Byte.toUnsignedInt(pixelValue));    
  }
  
  myImage.updatePixels();
  
  // Ensures that the new image data is drawn in the next draw loop
  shouldRedraw = true;
  
  // Let the Arduino sketch know we received all pixels
  // and are ready for the next frame
  myPort.write(1);
}

When I ran the arduino sketch, I saw the port was COM3, so I used that for the processing sketch. Do you know what am I doing wrong ?

4

@dzhahahah2 thank you for your reply!

We will double check the tutorial.
Alternatively, you can test the Portenta Vision Shield camera via the CameraCaptureWebSerial sketch available in Arduino IDE > File > Examples > Camera.
The example shows how to capture images from the camera and send them over Web Serial.

After uploading the sketch to the Portenta H7:

  • Go to the following folder: Arduino15/packages/arduino/hardware/mbed_portenta/4.1.3/libraries/Camera/extras/WebSerialCamera.
  • Open the 'index.html' file in a browser (Chrome or Edge).
  • Finally click on connect, choose Envie M7, then connect.

You can also take a look at the comments included in the sketch: ArduinoCore-mbed/libraries/Camera/examples/CameraCaptureWebSerial/CameraCaptureWebSerial.ino at main · arduino/ArduinoCore-mbed · GitHub

5

Make sure to uncomment the correct library and class for your camera like it says in the tutorial and adjust line 9 as well with the correct camera in use. Mine uses the HM0360 so it will not work if you uncomment the one for HM01B0.