ESP32 Google Vision Face Detection request fail

Hi, I am trying to submit an image on the web to Google Vision Face detection.
Submiting the image directly on the Google Vision page:

https://cloud.google.com/vision/docs/ocr?apix_params=%7B%22resource%22%3A%7B%22requests%22%3A%5B%7B%22features%22%3A%5B%7B%22type%22%3A%22FACE_DETECTION%22%7D%5D%2C%22image%22%3A%7B%22source%22%3A%7B%22imageUri%22%3A%22http%3A%2F%2Fwww.newdesignfile.com%2Fpostpic%2F2010%2F05%2Ffree-stock-photos-people_102217.jpg%22%7D%7D%2C%22imageContext%22%3A%7B%22languageHints%22%3A%5B%22pt%22%5D%7D%7D%5D%7D%7D

works OK.
Also using NodeRed on a raspberry Pi works ok.
But I can not make it work with ESP32.
It always responds on the console with:
HTTP Response code: -5
Error code: -5

My code is as below:

/*
  Rui Santos
  Complete project details at Complete project details at https://RandomNerdTutorials.com/esp32-http-get-post-arduino/

  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files.

  The above copyright notice and this permission notice shall be included in all
  copies or substantial portions of the Software.
*/

#include <WiFiClientSecure.h>
#include <HTTPClient.h>

const char* ssid = "mySSID";
const char* password = "myPass";

//Your Domain name with URL path or IP address with path
const char* serverName   = "https://vision.googleapis.com/v1/images:annotate?key=MyAPIKey";

String _request = "{\"requests\":[{\"image\":{\"source\":{\"imageUri\":\"http://www.newdesignfile.com/postpic/2010/05/free-stock-photos-people_102217.jpg\"}},\"features\":[{\"type\":\"FACE_DETECTION\",\"maxResults\":10}]}]}";


// the following variables are unsigned longs because the time, measured in
// milliseconds, will quickly become a bigger number than can be stored in an int.
unsigned long lastTime = 0;
unsigned long timerDelay = 15000;

void setup() {
  Serial.begin(115200);

  WiFi.begin(ssid, password);
  Serial.println("Connecting");
  while(WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("Connected to WiFi network with IP Address: ");
  Serial.println(WiFi.localIP());
 
}

void loop() {
  //Send an HTTP POST request every 10 minutes
  if ((millis() - lastTime) > timerDelay) {
    //Check WiFi connection status
    if(WiFi.status()== WL_CONNECTED){
      WiFiClient client;
      HTTPClient http;
    
      // Your Domain name with URL path or IP address with path
      http.begin(client, serverName);
      Serial.print("Conected to Server: ");
      Serial.println(serverName);
      
      //If you need an HTTP request with a content type: application/json, use the following:
      http.addHeader("Content-Type", "application/json");
      Serial.print("Posting request: ");    
      Serial.println(_request);
      int httpResponseCode = http.POST(_request); 
      
      Serial.print("HTTP Response code: ");
      Serial.println(httpResponseCode);

      String payload = "{}"; 

      if(httpResponseCode>0) {
        Serial.print("HTTP Response code: ");
        Serial.println(httpResponseCode);
        payload = http.getString();
        Serial.println(payload);        
      }else{
        Serial.print("Error code: ");
        Serial.println(httpResponseCode);
      }    
        
      // Free resources
      http.end();
    }else {
      Serial.println("WiFi Disconnected");
    }
    lastTime = millis();
  }
}

Any assistance welcome.
Thanks

Documentation:

When I use their "try it" example with your image URI in it, I get this result:

{
  "responses": [
    {
      "faceAnnotations": [
        {
          "boundingPoly": {
            "vertices": [
              {
                "x": 258,
                "y": 149
              },
              {
                "x": 2220,
                "y": 149
              },
              {
                "x": 2220,
                "y": 2431
              },
              {
                "x": 258,
                "y": 2431
              }
            ]
          },
          "fdBoundingPoly": {
            "vertices": [
              {
                "x": 437,
                "y": 671
              },
              {
                "x": 2022,
                "y": 671
              },
              {
                "x": 2022,
                "y": 2249
              },
              {
                "x": 437,
                "y": 2249
              }
            ]
          },
          "landmarks": [
            {
              "type": "LEFT_EYE",
              "position": {
                "x": 912.4723,
                "y": 1326.346,
                "z": 0.00011062622
              }
            },
            {
              "type": "RIGHT_EYE",
              "position": {
                "x": 1526.6857,
                "y": 1320.6454,
                "z": -1.4965897
              }
            },
            {
              "type": "LEFT_OF_LEFT_EYEBROW",
              "position": {
                "x": 716.0888,
                "y": 1169.8564,
                "z": 28.227432
              }
            },
            {
              "type": "RIGHT_OF_LEFT_EYEBROW",
              "position": {
                "x": 1093.6174,
                "y": 1220.1599,
                "z": -129.28482
              }
            },
            {
              "type": "LEFT_OF_RIGHT_EYEBROW",
              "position": {
                "x": 1362.4092,
                "y": 1204.37,
                "z": -130.85046
              }
            },
            {
              "type": "RIGHT_OF_RIGHT_EYEBROW",
              "position": {
                "x": 1714.0054,
                "y": 1162.9084,
                "z": 24.61473
              }
            },
            {
              "type": "MIDPOINT_BETWEEN_EYES",
              "position": {
                "x": 1226.4629,
                "y": 1341.3789,
                "z": -117.020584
              }
            },
            {
              "type": "NOSE_TIP",
              "position": {
                "x": 1229.9436,
                "y": 1693.3721,
                "z": -223.58087
              }
            },
            {
              "type": "UPPER_LIP",
              "position": {
                "x": 1234.3906,
                "y": 1874.5812,
                "z": -62.535545
              }
            },
            {
              "type": "LOWER_LIP",
              "position": {
                "x": 1250.9275,
                "y": 2081.3457,
                "z": 11.499863
              }
            },
            {
              "type": "MOUTH_LEFT",
              "position": {
                "x": 954.0944,
                "y": 1895.6145,
                "z": 102.50222
              }
            },
            {
              "type": "MOUTH_RIGHT",
              "position": {
                "x": 1521.7438,
                "y": 1894.9465,
                "z": 100.648766
              }
            },
            {
              "type": "MOUTH_CENTER",
              "position": {
                "x": 1243.1978,
                "y": 1966.7877,
                "z": -5.760807
              }
            },
            {
              "type": "NOSE_BOTTOM_RIGHT",
              "position": {
                "x": 1408.7616,
                "y": 1733.0752,
                "z": -11.283958
              }
            },
            {
              "type": "NOSE_BOTTOM_LEFT",
              "position": {
                "x": 1066.104,
                "y": 1723.5013,
                "z": -9.195057
              }
            },
            {
              "type": "NOSE_BOTTOM_CENTER",
              "position": {
                "x": 1236.0104,
                "y": 1779.2417,
                "z": -82.91817
              }
            },
            {
              "type": "LEFT_EYE_TOP_BOUNDARY",
              "position": {
                "x": 907.03656,
                "y": 1269.0936,
                "z": -40.569603
              }
            },
            {
              "type": "LEFT_EYE_RIGHT_CORNER",
              "position": {
                "x": 1023.96606,
                "y": 1341.1979,
                "z": 1.8108139
              }
            },
            {
              "type": "LEFT_EYE_BOTTOM_BOUNDARY",
              "position": {
                "x": 904.22015,
                "y": 1369.6213,
                "z": 1.1123695
              }
            },
            {
              "type": "LEFT_EYE_LEFT_CORNER",
              "position": {
                "x": 791.03687,
                "y": 1323.9991,
                "z": 51.21014
              }
            },
            {
              "type": "RIGHT_EYE_TOP_BOUNDARY",
              "position": {
                "x": 1530.0085,
                "y": 1262.1971,
                "z": -42.456882
              }
            },
            {
              "type": "RIGHT_EYE_RIGHT_CORNER",
              "position": {
                "x": 1644.8903,
                "y": 1320.5172,
                "z": 49.03029
              }
            },
            {
              "type": "RIGHT_EYE_BOTTOM_BOUNDARY",
              "position": {
                "x": 1532.0492,
                "y": 1367.053,
                "z": 0.018764496
              }
            },
            {
              "type": "RIGHT_EYE_LEFT_CORNER",
              "position": {
                "x": 1404.3707,
                "y": 1342.3015,
                "z": 0.95695496
              }
            },
            {
              "type": "LEFT_EYEBROW_UPPER_MIDPOINT",
              "position": {
                "x": 905.46796,
                "y": 1143.9843,
                "z": -95.198
              }
            },
            {
              "type": "RIGHT_EYEBROW_UPPER_MIDPOINT",
              "position": {
                "x": 1530.6193,
                "y": 1137.086,
                "z": -97.41205
              }
            },
            {
              "type": "LEFT_EAR_TRAGION",
              "position": {
                "x": 578.94385,
                "y": 1447.5095,
                "z": 728.64966
              }
            },
            {
              "type": "RIGHT_EAR_TRAGION",
              "position": {
                "x": 1911.881,
                "y": 1411.063,
                "z": 721.7596
              }
            },
            {
              "type": "FOREHEAD_GLABELLA",
              "position": {
                "x": 1227.5787,
                "y": 1215.9983,
                "z": -149.7467
              }
            },
            {
              "type": "CHIN_GNATHION",
              "position": {
                "x": 1260.499,
                "y": 2329.7454,
                "z": 135.59154
              }
            },
            {
              "type": "CHIN_LEFT_GONION",
              "position": {
                "x": 696.0957,
                "y": 1933.858,
                "z": 578.16125
              }
            },
            {
              "type": "CHIN_RIGHT_GONION",
              "position": {
                "x": 1797.679,
                "y": 1951.9707,
                "z": 573.6668
              }
            },
            {
              "type": "LEFT_CHEEK_CENTER",
              "position": {
                "x": 801.43207,
                "y": 1681.3684,
                "z": 116.79872
              }
            },
            {
              "type": "RIGHT_CHEEK_CENTER",
              "position": {
                "x": 1654.8096,
                "y": 1678.382,
                "z": 114.498856
              }
            }
          ],
          "rollAngle": -1.2034408,
          "panAngle": -0.1817784,
          "tiltAngle": -10.599325,
          "detectionConfidence": 0.92555153,
          "landmarkingConfidence": 0.7823187,
          "joyLikelihood": "VERY_LIKELY",
          "sorrowLikelihood": "VERY_UNLIKELY",
          "angerLikelihood": "VERY_UNLIKELY",
          "surpriseLikelihood": "VERY_UNLIKELY",
          "underExposedLikelihood": "VERY_UNLIKELY",
          "blurredLikelihood": "VERY_UNLIKELY",
          "headwearLikelihood": "VERY_UNLIKELY"
        }
      ]
    }
  ]
}

That is right, this not exactly what I am looking for but at least Google responds.
With my code above and submitting the same image's URL it returns error -5.
I am sure making some sort of mistake on what I am sending to google but still did not find what.
Thanks

The JSON in their example (with your image) is:

{
  "requests": [
    {
      "image": {
        "source": {
          "imageUri": "http://www.newdesignfile.com/postpic/2010/05/free-stock-photos-people_102217.jpg"
        }
       },
       "features": [
         {
           "maxResults": 10,
           "type": "FACE_DETECTION"
         }
       ]
    }
  ]
}

Put that in a file named "request.json". In that same directory, use the "curl" command to send it:

curl -v -X POST \ 
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://vision.googleapis.com/v1/images:annotate"

I don't have the 'gcloud' application so I end up with an Error 403: "Permission denied" since I don't have an API key but the logging of the messages might be informative:

zsh: command not found: gcloud
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 142.250.176.202...
* TCP_NODELAY set
* Connected to vision.googleapis.com (142.250.176.202) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=upload.video.google.com
*  start date: Aug 30 03:13:58 2021 GMT
*  expire date: Nov 22 03:13:57 2021 GMT
*  subjectAltName: host "vision.googleapis.com" matched cert's "*.googleapis.com"
*  issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1C3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7f8e71010a00)
> POST /v1/images:annotate HTTP/2
> Host: vision.googleapis.com
> User-Agent: curl/7.64.1
> Accept: */*
> Authorization: Bearer 
> Content-Type: application/json; charset=utf-8
> Content-Length: 298
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
* We are completely uploaded and fine
< HTTP/2 403 
< vary: X-Origin
< vary: Referer
< vary: Origin,Accept-Encoding
< content-type: application/json; charset=UTF-8
< date: Sun, 26 Sep 2021 00:25:39 GMT
< server: ESF
< cache-control: private
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< x-content-type-options: nosniff
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< accept-ranges: none
< 
{
  "error": {
    "code": 403,
    "message": "The request is missing a valid API key.",
    "status": "PERMISSION_DENIED"
  }
}
* Connection #0 to host vision.googleapis.com left intact
* Closing connection 0

You probably just need to figure out what goes in the "Authorization: Bearer" header.

Hi, let me understand: You are suggesting a PHP script that would be run against the image file and transfer the image to Google Vision?
Then Google vision would respond to the PHP script with json and then I, somehow, return this information to my ESP32? Thanks

No. I am suggesting you find out what is in a POST request that returns the correct data and then duplicate that POST request in your ESP8266 sketch.

OK, I have been trying something similar. I prepared a NodeRed web input node and changed the server on the code above to make the request to it. I noticed that the json data and the Google Vision Key arrive successfully. Still I could find no apparent reason for the problem.
In the process I noticed I am using WiFi.h and changed to WiFiSecure.h but the problem remains.
I will try your suggestion and see if I can understand what may be going wrong.
Thanks a lot.

Maybe there is documentation somewhere that says what the "-5" means. It's not a usual HTML code like "403" so I suspect it is saying there was a problem before your request even made it to the HTTP server.

Hi, in fact the problem was with the WiFiClientSecure.h
I eventually changed the lines:

      WiFiClientSecure client;   <--------- Key change to solve the problem.
      HTTPClient http;
      client.setInsecure();

and Google Vision responded with the same json string as you got above.
Thanks very much for your time and assistance it contributed for my research and resolution of the problem.

Regards
Paulo

Glad it worked out.

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.