Using ESP32-CAM as standalone controller

Hello, I'm starting to learn ESP32-CAM (Ai Thinker) module.
Almost all the examples that are shared on the internet about the use of "ESP32-CAM" use computer media to process images generated from this ESP32-CAM.
I'm looking for an example of an ESP32-CAM program that purely processes images within the module itself. So the ESP32-CAM can stand alone as a controller without depending on other programs. Maybe start with detecting the presence / absent object.
Please show me the link perhaps you know using ESP32-CAM as standalone controller that programed with Arduino.

Thank you.


I know nothing about image processing HOWEVER I don't think you'll get far trying to run an instance of it on an ESP32.

I has been made a project with RPi about image processing.
to detect printing object

I have used Liner Regression for weather prediction and object detection on a ESP32. I have used KNN, Simple machine learning with Arduino KNN | Arduino Blog, to detect multiple objects with an ESP32. Tensorflow Lite is available for the ESP32.

There are large, open source libraries in C and C++ for just about any image processing you might want to do, OpenCV for example. All you have to do is incorporate the appropriate code into your ESP32-Cam application.

See GitHub - joachimBurket/esp32-opencv: Shrinked OpenCV for ESP32

Most people won't bother, though, because it is so much easier and faster to do the work on a PC.

1 Like

Thank you for information. But I still curious to using ESP32-cam with arduino IDE.

Thank you for the link. Almost site that I found are using ESP IDF or non arduino IDE that is my lack my knowledge :sweat_smile: .
I try installing ESP IDF but still not success.

I have a sketch here using Arduino IDE which may be of interest which shows how to capture a frame and then access the raw RGB data.

Hi thank you. I already try your code a few day ago and still in my list to use it especially esp32camdemo-greyscale.ino :+1:

I did use the Arduino IDE to do Linear Regression and KNN on a ESP32.

Sorry I mean using esp32-cam not other sensor. Maybe if you have a code please show me

Using LR to predict the next hours air pressure from a ESP32:

void fDoTrends( void *pvParameters )
  const int magicNumber = 96;
  double    values[2];
  int       lrCount = 0;
  float     lrData = 0.0f;
  float     DataPoints[magicNumber] = {0.0f};
  float     TimeStamps[magicNumber] = {0.0f};
  float     dpHigh = 702.0f;
  float     dpLow  = 683.0f;
  float     dpAtom = 0.0f;
  float     dpMean = 0.0f; //data point mean
  float     tsAtom = 0.0f;
  float     tsUnit = 0.0f;
  float     tsMean = 0.0f;
  bool      dpRecalculate = true;
  bool      FirstTimeMQTT = true;
  String    apInfo = "";
  apInfo.reserve( 150 );
  for (;;)
    if ( xQueueReceive(xQ_lrData, &lrData, portMAX_DELAY) == pdTRUE )
      apInfo.concat( String((float)xTaskGetTickCount() / 1000.0f) );
      apInfo.concat( "," );
      apInfo.concat( String(lrData) );
      apInfo.concat( ",0.0" );
      apInfo.concat( ",0.0" );
      if ( MQTTclient.connected() )
        xSemaphoreTake( sema_MQTT_KeepAlive, portMAX_DELAY );
        MQTTclient.publish( topicPressureInfo, apInfo.c_str() );
        vTaskDelay( 1 );
        xSemaphoreGive( sema_MQTT_KeepAlive );
      xQueueSend( xQ_pMessage, (void *) &apInfo, portMAX_DELAY ); // wait for queue space to become available
      apInfo = "";
      //find dpHigh and dpLow, collects historical high and low data points, used for data normalization
      if ( lrData > dpHigh )
        dpHigh = lrData;
        dpRecalculate = true;
      if ( lrData < dpLow )
        dpLow = lrData;
        dpRecalculate = true;
      if ( lrCount != magicNumber )
        DataPoints[lrCount] = lrData;
        TimeStamps[lrCount] = (float)xTaskGetTickCount() / 1000.0f;
        log_i( "lrCount %d TimeStamp %f lrData %f", lrCount, TimeStamps[lrCount], DataPoints[lrCount] );
      } else {
        //shift datapoints collected one place to the left
        for ( int i = 0; i < magicNumber; i++ )
          DataPoints[i] = DataPoints[i + 1];
          TimeStamps[i] = TimeStamps[i + 1];
        //insert new data points and time stamp (ts) at the end of the data arrays
        DataPoints[magicNumber - 1] = lrData;
        TimeStamps[magicNumber - 1] = (float)xTaskGetTickCount() / 1000.0f;
        lr.Reset(); //reset the LinearRegression Parameters
        // use dpHigh and dpLow to calculate data mean atom for normalization
        if ( dpRecalculate )
          dpAtom = 1.0f / (dpHigh - dpLow); // a new high or low data point has been found adjust mean dpAtom
          dpRecalculate = false;
        //timestamp mean is ts * (1 / ts_Firstcell - ts_Lastcell[magicNumber]). ts=time stamp
        tsAtom = 1.0f / (TimeStamps[magicNumber - 1] - TimeStamps[0]); // no need to do this part of the calculation every for loop ++
        for (int i = 0; i < magicNumber; i++)
          dpMean = (DataPoints[i] - dpLow) * dpAtom;
          tsMean = TimeStamps[i] * tsAtom;
          lr.Data( tsMean, dpMean ); // train lr
          //send to mqtt the first time
          if ( FirstTimeMQTT )
            apInfo.concat( String(TimeStamps[i]) );
            apInfo.concat( "," );
            apInfo.concat( String(DataPoints[i]) );
            apInfo.concat( "," );
            apInfo.concat(  String(tsMean) );
            apInfo.concat( "," );
            apInfo.concat( String(dpMean) );
            xQueueSend( xQ_pMessage, (void *) &apInfo, portMAX_DELAY ); // wait for queue space to become available
            apInfo = "";
        if ( !FirstTimeMQTT )
          apInfo.concat( String(TimeStamps[magicNumber - 1]) );
          apInfo.concat( "," );
          apInfo.concat( String(DataPoints[magicNumber - 1]) );
          apInfo.concat( "," );
          apInfo.concat( String(tsMean) );
          apInfo.concat( "," );
          apInfo.concat( String(dpMean) );
          xQueueSend( xQ_pMessage, (void *) &apInfo, portMAX_DELAY );
          apInfo = "";
        FirstTimeMQTT = false;
        lr.Parameters( values );
        tsUnit = TimeStamps[magicNumber - 1] - TimeStamps[magicNumber - 2]; //get the time stamp quantity
        tsUnit += TimeStamps[magicNumber - 1]; //get a future time
        tsUnit *= tsAtom; //setting time units to the same scale
        log_i( "Calculate next x using y = %f", lr. Calculate( tsUnit ) ); //calculate next datapoint using time stamp
        log_i( "Correlation: %f Values: Y=%f and *X + %f ", lr.Correlation(), values[0], values[1] ); // correlation is the strength and direction of the relationship
        //calculate datapoint for current time stamp, use current data point against calculated datapoint to get error magnatude and direction.
        //log_i( "lr.Error( x_pMessage.TimeStamp, x_pMessage.nDataPoint ) %f", lr.Error(x_pMessage.nTimeStamp, x_pMessage.nDataPoint) ); //
      log_i( "fDoTrends high watermark % d",  uxTaskGetStackHighWaterMark( NULL ) );
    } //if ( xQueueReceive(xQ_lrData, &lrData, portMAX_DELAY) == pdTRUE )
  } //for(;;)
  vTaskDelete ( NULL );
} //void fDoTrends( void *pvParameters )

Using this library GitHub - cubiwan/Regressino: Calculate potential, exponential, logarithmic, lineal and logistic regressio in Arduino.

Judging by the many answers you’ve posted, I may have the easiest question you’ve been asked. Can the ESP32-Cam detect a simple two color logo and flip a single gpio pin to indicate detected/not detected? Thanks for your time!

Yes, the ESP32CAM running KNN could detect colors.

Simple machine learning with Arduino KNN | Arduino Blog

Works well.

If you require more image processing, consider using a RPi.

1 Like

Thanks for the prompt reply! To be clear, the logo to be detected, would require both color and shape recognition. Is that possible? I’m considering the AI Thinker device.

The question can be answered easily by doing your own research.

I use KNN to detect thermal images that distinguish between humans and animals, upon detection of a human, do somestuff.

I use TensorFlow to do image processing on a RPi.

ESP32's and Arduino BLE 32 bit thingy can do ML. An RPi does ML really well.

Please feel free to do your own research.

1 Like

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.