Converting python https request to Esp32 Arduino

Using Arduino Programming Questions
1

this following python script works fine, how can I convert it to Esp32 Ardunio ?
I understand its not possible to call python script from Esp32 Ardunio .

import requests

data = {
    'To': '+1xxxxxx1785',
    'From': '+1xxxxx8954',
    'Body': 'new test msg from Twilio',
}

response = requests.post(
    'https://api.twilio.com/2010-04-01/Accounts/xxxxxxxxxxxxxxxxxxxxb91bc/Messages.json',
    data=data,
    auth=('xxxxxxxxxxxxxb91bc', 'xxxxxxxxxxxxxxxxxxxx6a9a'),
)
2

There are many tutorials on line like

3

I got this but I am not sure how to pass in the data and credentials and getting HTTP 400 error.
the working curl command and the arduino code is below
Note: I used the base64 encoded value for the credentials
xxxxxxxxxc9793bb91bc:xxxxxxefb8c99865a56a9a in https code.

--------------------------- CURL command ----------------------------------

curl 'https://api.twilio.com/2010-04-01/Accounts/xxxxxxxxxxxxx91bc/Messages.json' -X POST \
--data-urlencode 'To=+1xxxxx1785' \
--data-urlencode 'From=+1xxxxxxx8954' \
--data-urlencode 'Body=new test msg from Twilio' \
-u xxxxxxxxxc9793bb91bc:xxxxxxefb8c99865a56a9a

----------------------------- https api call ------------------------------

#include <HTTPClient.h>
#include <Arduino.h>
#if defined(ESP32)
  #include <WiFi.h>
#elif defined(ESP8266)
  #include <ESP8266WiFi.h>
#endif

const char WIFI_SSID[] = "Puma";        
const char WIFI_PASSWORD[] = "xxxxxxx1785"; 

String URL   = "https://api.twilio.com/2010-04-01/Accounts/AC0c8db51e8dc5730f8f797c9793bb91bc/Messages.json";         
String DATA   = "'To'='+1xxxx1785'&'From': '+1xxxxx8954'&'Body': 'new test msg from Twilio'";

void setup() {
    Serial.begin(115200);

  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
  Serial.println("Connecting");
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("Connected to WiFi network with IP Address: ");
  Serial.println(WiFi.localIP());

  HTTPClient http;

  http.begin(URL);
  http.addHeader("Content-Type", "application/x-www-form-urlencoded");
  http.addHeader("Authorization", "Basic xxxxxxYjUxZThkYzU3MzBmOGY3OTdjOTc5M2JiOTFiYzo4NzczNDFmMTI0YjE5MjlmZWZiOGM5OTg2NWE1NmE5YQ==");
  int httpCode = http.POST(DATA);

  // httpCode will be negative on error
  if (httpCode > 0) {
    // file found at server
    if (httpCode == HTTP_CODE_OK) {
      String payload = http.getString();
      Serial.println(payload);
    } else {
      // HTTP header has been send and Server response header has been handled
      Serial.printf("[HTTP] POST... code: %d\n", httpCode);
    }
  } else {
    Serial.printf("[HTTP] POST... failed, error: %s\n", http.errorToString(httpCode).c_str());
  }

  http.end();
}

void loop() {
}
4

I think you might've missed J-M-L's point. Something you might want to take note of is the https at the start of your URL (https not just http). Usually, this involves a bit more than standard http requests do, I'd recommend reading the link in @J-M-L's post to get an idea of what's required.

5

i am not finding HTTPS POST section in the document , I see HTTP POST though. can you please guide me ?
The " ESP32 HTTPS Requests without Certificate" is showing how to GET but I need the POST method.

6

You should get and print the payload if the httpCode is greater than zero. This includes success messages, and details for any non-OK code like 400. That will help you fix any problems, now and in the future after the code is deployed in the field. (No payload to print if the return value is negative.)

You don't need to manually encode the Authorization -- HTTPClient::setAuthorization will do that for you.

However, I don't see any automatic way of doing the data encoding. Glancing at what you have there, you should not be using the single-quote, two equal-signs are missing, and some characters are not percent-encoded.

  • the name of the first value is 'To' -- it should be just To
  • From and Body have the same problem, and they are using :-and-space instead of =. Without that equal-sign, you're actually creating a long-name variable with no value.
  • technically, any spaces should be encoded as a + sign, but it might work with spaces
  • however, this means that any time you really mean +, it must be percent-encoded

Since you have curl doing the --data-urlencode, you can grab it

$ curl http://httpbin.org/anything \
  --data-urlencode 'To=+1xxxxx1785' \
  --data-urlencode 'From=+1xxxxxxx8954' \
  --data-urlencode 'Body=new test msg from Twilio' \
  --trace-ascii reqres.txt
$ grep -A 2 'Send data' reqres.txt
=> Send data, 67 bytes (0x43)
0000: To=%2B1xxxxx1785&From=%2B1xxxxxxx8954&Body=new+test+msg+from+Twi
0040: lio

If you are going to be sending a lot of messages, you should of course write some code to do the encoding.

7

hi Ken
i made the modification as per your advise but I am now getting HTTP 401 . Also I don't understand what you mean by do not manually authenticate.

#include <HTTPClient.h>
#include <Arduino.h>
#if defined(ESP32)
  #include <WiFi.h>
#elif defined(ESP8266)
  #include <ESP8266WiFi.h>
#endif

const char WIFI_SSID[] = "Puma";        
const char WIFI_PASSWORD[] = "xxxxxx1785"; 

String URL   = "https://api.twilio.com/2010-04-01/Accounts/AC0c8db51e8dc5730f8f797c9793bb91bc/Messages.json";         
String DATA  = "To=%2B1xxxxx1785&From=%2B1xxxxx8954&Body=new+test+msg+from+Twilo";
             

void setup() {
    Serial.begin(115200);

  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
  Serial.println("Connecting");
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("Connected to WiFi network with IP Address: ");
  Serial.println(WiFi.localIP());

  HTTPClient http;

  http.begin(URL);
  http.addHeader("Content-Type", "application/x-www-form-urlencoded");
  http.addHeader("Authorization", "QUMwYzhkYjUxZThkYzU3MzBmOGY3OTdjOTc5M2JiOTFiYzo4NzczNDFmMTI0YjE5MjlmZWZiOGM5OTg2NWE1NmE5YQ==");
  int httpCode = http.POST(DATA);

  // httpCode will be negative on error
  if (httpCode > 0) {
    // file found at server
    if (httpCode == HTTP_CODE_OK) {
      String payload = http.getString();
      Serial.println(payload);
    } else {
      // HTTP header has been send and Server response header has been handled
      Serial.printf("[HTTP] POST... code: %d\n", httpCode);
    }
  } else {
    Serial.printf("[HTTP] POST... failed, error: %s\n", http.errorToString(httpCode).c_str());
  }

  http.end();
}

void loop() {
}
8

it worked . . i had a typo in Authorization , missed the "Basic " key worked.

thanks all for your help

9

Avoiding typos is one reason to use the built-in functions when available. Instead of

use

http.setAuthorization("xxxxxxxxxc9793bb91bc", "xxxxxxefb8c99865a56a9a");

BTW, base64 is encoding, not encrypting. It is trivial to reverse. One can tell by looking that the base64-encoded value you posted in your code above is not for the redacted credentials with the x's. If those are legitimate credentials, they are now comprised by posting on a public forum. Invalidate them if possible, and generate new ones.

10

thanks
how can we protect the credentials in Arduino code ?

11

don't post them in the forum :slight_smile:

if you need the plain text credentials within the code then it's hard to with usual Arduinos. In general you would not have a plain text representation of the secret.

Protecting "secrets" in a safe enough manner requires a Secure Enclave or equivalent and is not trivial. On an ESP32 your would need to use flash encryption, secure storage (NVS), hardware based cryptographic features, sign your firmware using ESP-IDF tools, leverage secure boot, and always use TLS/SSL to encrypt the data in transit.