ArduinoIoTCloud somehow blocking HTTPS but not HTTP

Hi all

I have been using Arduino for well over a year now, mainly focused on a single project which is stable (by which I mean it never crashes and does the job that is required of it) but which I enhance from time to time. My latest enhancement has it submitting HTTPS GET requests to a third-party server. I have struggled to make this work reliably and now have a repeatable way of recreating the problem.

What I observed is that in the first five-to-ten seconds that my sketch is running, my HTTPS GET requests work just fine. However, once ArduinoIoTCloud (which my sketch also uses) is up and running, the GET requests fail every time with a -1 error which appears to mean some kind of networking issue.

I decided to see what happens when I no longer execute ArduinoCloud.update() in my sketch. Obviously, the ArduinoIoTCloud functionality is lost (and I don't want that to happen) but, importantly, all my HTTPS GET requests succeed whenever executed by the sketch. I have tried using port 8443 instead of 443 (by modifying the underlying ESP8266HTTPClient.cpp library) but that did not solve the problem.

I am now at a loss for what is going on but it does appear to be some kind of clash between ArduinoIoTCloud and HTTPS.

As additional information, my sketch regularly uses HTTP GET to a different server and I have experienced no problems with that - and, therefore, no apparent clash with ArduinoIoTCloud.

This seems to me to be pretty fundamental in the sense that ArduinoIoTCloud surely should not preclude the reliable use of HTTPS and yet it appears to do precisely that.

Has anyone experienced something similar and, if so, how did you manage to fix it?

Below is a very stripped-down sketch which reproduces the problem. The "First message" is always successful and sometimes one or two the of "Test message(s)", depending on whether they're generated early enough after the sketch starts i.e. before it, apparently, clashes with ArduinoIoTCloud.

With my thanks to anyone who gives this their attention.

#include <Arduino_ConnectionHandler.h>
#include <ArduinoIoTCloud.h>
#include <ESP8266HTTPClient.h>

#define INTERVAL 1000

const char *macrodroidServer = "https://<URL hidden>";
unsigned long previousMillis;
bool cloudConnected;

WiFiConnectionHandler ArduinoIoTPreferredConnection("", "");

const char *DEVICE_LOGIN_NAME = "<hidden>";
const char *DEVICE_KEY = "<hidden>";

String startDateAndTime = ".";
String status = "";
float flowRate;
float totalLitres;
int runTimeSecs;
bool otaReady;

bool firstMessageSent;

void setup() {
  //initialise serial and wait for port to open:
  Serial.begin(115200);
  // wait up to 5 seconds for port to open
  for (const unsigned long serialBeginTime = millis(); millis() - serialBeginTime < 5000;) yield();

  initProperties();

  ArduinoCloud.addCallback(ArduinoIoTCloudEvent::CONNECT, doThisOnConnect);
  ArduinoIoTPreferredConnection = WiFiConnectionHandler("<SSID hidden>", "<PASS hidden>");
  ArduinoCloud.begin(ArduinoIoTPreferredConnection);
  setDebugMessageLevel(4);
  ArduinoCloud.printDebugInfo();

  previousMillis = millis();
  delay(1000);
}

void loop() {
  unsigned long currentMillis = millis();
  unsigned long sinceLastPassMillis = currentMillis - previousMillis;
  if (!firstMessageSent) {
    macrodroidNotification("First message title");
    firstMessageSent = true;
  }
  if (sinceLastPassMillis > INTERVAL) {
    int randNo = random(20);
    if (randNo < 2) macrodroidNotification("Test message title");

    updateStatusForArduinoCloud();
    ArduinoCloud.update();
    previousMillis = currentMillis;
  }
}

int urlEncode(char encodeMe[], char result[] = NULL) {
  char c;
  char code0;
  char code1;
  int charPos = 0;
  String s = encodeMe;
  for (int i = 0; i < s.length(); i++) {
    c = encodeMe[i];
    if (c == ' ') {
      if (result)
        result[charPos] = '+';
    } else if (isalnum(c)) {
      if (result)
        result[charPos] = c;
    } else if (result) {
      code1 = (c & 0xf) + '0';
      if ((c & 0xf) > 9) code1 = (c & 0xf) - 10 + 'A';
      c = (c >> 4) & 0xf;
      code0 = c + '0';
      if (c > 9) code0 = c - 10 + 'A';
      result[charPos] = '%';
      charPos++;
      result[charPos] = code0;
      charPos++;
      result[charPos] = code1;
    } else
      charPos += 2;
    charPos++;;
  }
  return charPos;
}

void initProperties() {
  ArduinoCloud.setBoardId(DEVICE_LOGIN_NAME);
  ArduinoCloud.setSecretDeviceKey(DEVICE_KEY);
  ArduinoCloud.addProperty(startDateAndTime, READ, ON_CHANGE, NULL);
  ArduinoCloud.addProperty(status, READ, ON_CHANGE, NULL);
  ArduinoCloud.addProperty(flowRate, READ, 1 * SECONDS, NULL);
  ArduinoCloud.addProperty(totalLitres, READ, 1 * SECONDS, NULL);
  ArduinoCloud.addProperty(runTimeSecs, READ, ON_CHANGE, NULL, 1);
  ArduinoCloud.addProperty(otaReady, READWRITE, ON_CHANGE, onOtaReadyChange);
}

void updateStatusForArduinoCloud() {
  status = "Status";
}

void macrodroidNotification(char title[]) {
  int titleEncodedSize = urlEncode(title);
  char titleEncoded[titleEncodedSize + 1];
  titleEncoded[titleEncodedSize] = '\0';
  urlEncode(title, titleEncoded);

  char body[] = "Test message body";
  int bodyEncodedSize = urlEncode(body);
  char bodyEncoded[bodyEncodedSize + 1];
  bodyEncoded[bodyEncodedSize] = '\0';
  urlEncode(body, bodyEncoded);

  HTTPClient https;
  https.setTimeout(2000);

  WiFiClientSecure client;
  client.setInsecure();

  char url[strlen(macrodroidServer) + titleEncodedSize + bodyEncodedSize + 25];
  strcpy(url, macrodroidServer);
  strcat(url, "?");
  strcat(url, "title=");
  strcat(url, titleEncoded);
  strcat(url, "&messagebody=");
  strcat(url, bodyEncoded);

  if (https.begin(client, url)) {
    Serial.print("Connected to Macrodroid with url ");
    Serial.print(url);
    Serial.print(" and result ");
    Serial.println(https.GET());
    https.end();
    client.stop();
  }
}

void doThisOnConnect() {
  cloudConnected = true;
}

void onOtaReadyChange()  {
  //
}

did you manage to get this solved, I have exactly the same issue.

thanks,

Piet.

Hi PIet. I didn't manage to solve it but did manage to implement a workaround whereby a third-party service (Sign In - ThingSpeak IoT) that works off HTTP and can generate a back-to-back HTTPS request that I was able to make use of.

At the same time, I have pressed Arduino's tech support on the matter and - after initially telling me to post in this forum - they seem to have got the point that something fundamental is going wrong that requires their attention. That was only on 1 Dec so I guess it's early days but I'm hopeful they will fix it in due course. I will also make them aware of your comment here today. Best of luck.

Hi Peter, thanks for the info, I'll have a look at ThingSpeak.

As a long time user of Blynk I am planning to migrate to a new platform in the coming months. ArduinoCloud looks promising and migrating from Blynk is straightforward. However https is a dealbreaker. I'l see how it goes before deciding.

regards,

piet.

Arduino support kindly referred me to Multiple Instances of WiFiClientSecure is not working. · Issue #8800 · esp8266/Arduino · GitHub. At the time of writing, I haven't been able to check whether it describes the underlying cause of my problem but it offers what seems to be a credible explanation (insufficient memory to support two simultaneous SSL sessions) as well as, happily, two possible solutions.