Non-blocking connection to a server

Projects Programming
21

If you need a non blocking Telegram library check this one:

By the way, since you have an ESP32, you can use FreeRTOS for a full multitask working.

22

Thank you I'll look into it.

However, it seems that using FreeRTOS is the only solution.

I was expecting that a library exists to make a request for a connection without waiting for it to be done before giving back the hand to the rest of the code

23

Hello everyone,

I'm trying to perform a connection to a server (here Telegram's server but it could be another one) using FreeRTOS.
The connection to the server could take time, especially when it is not reachable (no internet for eg.). So my goal was to check if the ESP32 is connected to the server, if not then we initiate the connection but with the Core 0.

Let me give you the sample of code:

#include <WiFiClientSecure.h>
WiFiClientSecure Serv_Telegram;

// Some parts of the code are not shown because not useful I think
// such as the certificat for the connection
// I made sure that the ESP is connected to the WiFi

SemaphoreHandle_t xMutex;

TaskHandle_t xHandleConnectTask = NULL;

void Task_ReconnexionTelegram(void *parameter) {
  for (;;)
  {
    ulTaskNotifyTake(pdTRUE, portMAX_DELAY);
    
    if (xSemaphoreTake(xMutex, portMAX_DELAY))
    {
      Serial.println("[Task] Connexion to Telegram");
      Serv_Telegram.stop();
      vTaskDelay(pdMS_TO_TICKS(200));
      Serv_Telegram.setInsecure();
      Serv_Telegram.connect(Serveur_Telegram, 443);
      Serial.println("[Task] End connexion to Telegram");

      xSemaphoreGive(xMutex);
    }
  }
}


bool ASYNC_Telegram_Init(void) {

  Serv_Telegram.setCACert(test_root_ca);

  xMutex = xSemaphoreCreateMutex();

  xSemaphoreGive(xMutex);

  // Task to connect from Core0
  xTaskCreatePinnedToCore(
    Task_ReconnexionTelegram,
    "ConnectTelegram",
    8192,
    NULL,
    1,
    &xHandleConnectTask,
    0
  );

  return true;
}

bool ASYNC_Connexion_Telegram(void)
{
  bool connected = false;

  if (xSemaphoreTake(xMutex, 10 / portTICK_PERIOD_MS))
  {
    Serial.println("Check Telegram State");

    connected = Serv_Telegram.connected();
    xSemaphoreGive(xMutex);

    if(!connected && xHandleConnectTask != NULL)
    {
      Serial.println("Connexion to Telegram");

      // Serv_Telegram.stop();
      // delay(200);
      // Serv_Telegram.setInsecure();
      // Serv_Telegram.connect(Serveur_Telegram, 443);

      xTaskNotifyGive(xHandleConnectTask);

      Serial.println("End connexion to Telegram");
    }
  }
  else
  {
    Serial.println("Connexion à Telegram en cours");
  }

  return connected;
}

So ASYNC_Telegram_Init() is called in the setup. Then, let's say that ASYNC_Connexion_Telegram() is called from the loop as long as it returns false.

If I uncomment the 4 lines in ASYNC_Connexion_Telegram() and comment xTaskNotifyGive(xHandleConnectTask);, the connection goes perfectly well.

However, like this, when I tried to run the connection calls from the task on Core0 it never connects; even worse it ends up causing a reset of the ESP32.

Do you have any ideas or tips to help figure this out please?

24

Is it related to your other post
Non-blocking connection to a server - #21 by cotestatnt ?

25

In a way it is yes because I'm talking about FreeRTOS.
But if you read the topic you sent you'll see that all they propose is getting the temperature data from another task.

Here I'm trying to do the opposite so I guess it deserve a topic by itself. Especially because it could also help some other people willing to do the same

So please don't close this topic I really need help to do it I don't understand what goes wrong

27

The first thing I'd try is to move it to Core 1.

Please include enough code so that it compiles and we can test it. I typically don't post coding advise unless I can at least compile any proposed solution and preferably test it to see if it resolves the issue you're seeing.

28

Why not reflect what you're apparently trying to do? Have two tasks: (1) read sensors, and (2) talk to Telegram. (One of them could the default main loop task.) Use a queue (or two) to communicate between them.

  • Took a bunch of readings, put (a pointer to) them them in the queue, and then start a another set
  • There's a new set of readings that I'm supposed to do something with in Telegram.
    • Oh, it's taking 20 seconds to connect. It doesn't matter. OK, send it.
    • No internet connection; keep this set around for a while and keep retrying. (At some point, will have to discard them, write to disk, etc. since more readings keep coming in.)
    • Done sending, hand the pointer back, so that it can be disposed of or reused
pert 29

I have merged your forum topics due to them having too much overlap on the same subject matter @Anthony_P.

In the future, please only create one topic for each distinct subject matter and be careful not to cause them to converge into parallel discussions.

The reason is that generating multiple forum topics on the same subject matter can waste the time of the people trying to help. Someone might spend a lot of time investigating and writing a detailed answer on one topic, without knowing that someone else already did the same in the other topic.

Thanks in advance for your cooperation.

1 Like
30

Hello gfvalvo, this is then a thing I did and it didn't change a thing.

However, by making a code that you could compile, I succeded in making this work.
It turns out I was using a function that, when the response time from the server is too long, it stops the communication and ask for a reconnection right away. This was so fast that I couldn't see that the connection was successful.

I'll then post a sample of code that can be compiled, and that works for me.
Thanks again for wanting to help me. A piece of advice for every new to Arduino: if you're whole code doesn't work, don't hesitate to start a new one with only the part you wanna try.

Again thanks to everyone.
Here is the working code:

#include <WiFi.h>

bool Connexion_WiFi(void) {
  WiFi.begin("SSID", "PASSWORD");

  return true;
}

bool Get_Status_WiFi(void) {
  if (WiFi.status() == WL_CONNECTED) {
    return true;
  } else {
    return false;
  }
}

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/queue.h"
#include "freertos/semphr.h"

#include <WiFiClientSecure.h>

WiFiClientSecure Serv_Telegram;

const char *test_root_ca = R"literal(
-----BEGIN CERTIFICATE-----
MIIDxTCCAq2gAwIBAgIBADANBgkqhkiG9w0BAQsFADCBgzELMAkGA1UEBhMCVVMx
EDAOBgNVBAgTB0FyaXpvbmExEzARBgNVBAcTClNjb3R0c2RhbGUxGjAYBgNVBAoT
EUdvRGFkZHkuY29tLCBJbmMuMTEwLwYDVQQDEyhHbyBEYWRkeSBSb290IENlcnRp
ZmljYXRlIEF1dGhvcml0eSAtIEcyMB4XDTA5MDkwMTAwMDAwMFoXDTM3MTIzMTIz
NTk1OVowgYMxCzAJBgNVBAYTAlVTMRAwDgYDVQQIEwdBcml6b25hMRMwEQYDVQQH
EwpTY290dHNkYWxlMRowGAYDVQQKExFHb0RhZGR5LmNvbSwgSW5jLjExMC8GA1UE
AxMoR28gRGFkZHkgUm9vdCBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkgLSBHMjCCASIw
DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9xYgjx+lk09xvJGKP3gElY6SKD
E6bFIEMBO4Tx5oVJnyfq9oQbTqC023CYxzIBsQU+B07u9PpPL1kwIuerGVZr4oAH
/PMWdYA5UXvl+TW2dE6pjYIT5LY/qQOD+qK+ihVqf94Lw7YZFAXK6sOoBJQ7Rnwy
DfMAZiLIjWltNowRGLfTshxgtDj6AozO091GB94KPutdfMh8+7ArU6SSYmlRJQVh
GkSBjCypQ5Yj36w6gZoOKcUcqeldHraenjAKOc7xiID7S13MMuyFYkMlNAJWJwGR
tDtwKj9useiciAF9n9T521NtYJ2/LOdYq7hfRvzOxBsDPAnrSTFcaUaz4EcCAwEA
AaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYE
FDqahQcQZyi27/a9BUFuIMGU2g/eMA0GCSqGSIb3DQEBCwUAA4IBAQCZ21151fmX
WWcDYfF+OwYxdS2hII5PZYe096acvNjpL9DbWu7PdIxztDhC2gV7+AJ1uP2lsdeu
9tfeE8tTEH6KRtGX+rcuKxGrkLAngPnon1rpN5+r5N9ss4UXnT3ZJE95kTXWXwTr
gIOrmgIttRD02JDHBHNA7XIloKmf7J6raBKZV8aPEjoJpL1E/QYVN8Gb5DKj7Tjo
2GTzLH4U/ALqn83/B2gX2yKQOC16jdFU8WnjXzPKej17CuPKf1855eJ1usV2GDPO
LPAvTK33sefOT6jEm0pUBsV/fdUID+Ic/n4XuKxe9tQWskMJDE32p2u0mYRlynqI
4uJEvlz36hz1
-----END CERTIFICATE-----
)literal";

#define Serveur_Telegram "https://api.telegram.org"

SemaphoreHandle_t xMutex;
TaskHandle_t xHandleConnectTask = NULL;

void Task_ReconnexionTelegram(void *parameter) {
  for (;;)
  {
    ulTaskNotifyTake(pdTRUE, portMAX_DELAY);
    
    if (xSemaphoreTake(xMutex, portMAX_DELAY))
    {
      Serial.println("[Task] Connecting to Telegram");
      Serv_Telegram.stop();
      vTaskDelay(pdMS_TO_TICKS(200));
      Serv_Telegram.setInsecure();  // Useful only if using the IP Address
      Serv_Telegram.connect(Serveur_Telegram, 443);
      Serial.println("[Task] End connecting to Telegram");

      xSemaphoreGive(xMutex);
    }
  }
}

bool ASYNC_Telegram_Init(void) {

  Serv_Telegram.setCACert(test_root_ca);

  xMutex = xSemaphoreCreateMutex();

  xSemaphoreGive(xMutex);

  // Task to connect from Core0
  xTaskCreatePinnedToCore(
    Task_ReconnexionTelegram,
    "ConnectTelegram",
    8192,
    NULL,
    1,
    &xHandleConnectTask,
    0
  );

  return true;
}

bool ASYNC_Connexion_Telegram(void)
{
  bool connected = false;

  if(!Get_Status_WiFi()) return false;

  if (xSemaphoreTake(xMutex, 10 / portTICK_PERIOD_MS))
  {
    Serial.println("Check Telegram State");

    connected = Serv_Telegram.connected();
    xSemaphoreGive(xMutex);

    if(!connected && xHandleConnectTask != NULL)
    {
      Serial.println("Connecting to Telegram");

      // Synchronous connection
      /*Serv_Telegram.stop();
      delay(200);
      Serv_Telegram.setInsecure();  // Useful only if using the IP Address
      Serv_Telegram.connect(Serveur_Telegram, 443);*/

      // Asynchronous connection
      xTaskNotifyGive(xHandleConnectTask);

      Serial.println("End connecting to Telegram");
    }
  }
  else
  {
    Serial.println("Task_ReconnexionTelegram in execution");
  }

  return connected;
}

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);

  pinMode(2, OUTPUT);   // Led ESP32

  Connexion_WiFi();

  while(!Get_Status_WiFi());

  Serial.println("Connected to WiFi");

  ASYNC_Telegram_Init();

}

void loop() {
  // put your main code here, to run repeatedly:
  while(!ASYNC_Connexion_Telegram())
  {
    digitalWrite(2, ((millis() / 200) & 0x01));   // Blinking the LED
  }
  Serial.println("Connected to Telegram");
  while(true);
}

.
EDIT:
As suggested by @cotestatnt

It might be a better idea to run the Task_ReconnexionTelegram() function from Core1 and not Core0. To do so, change the init function to this:

bool ASYNC_Telegram_Init(void) {

  Serv_Telegram.setCACert(test_root_ca);

  xMutex = xSemaphoreCreateMutex();

  xSemaphoreGive(xMutex);

  // Task to connect from Core0
  xTaskCreatePinnedToCore(
    Task_ReconnexionTelegram,
    "ConnectTelegram",
    8192,
    NULL,
    1,
    &xHandleConnectTask,
    1
  );

  return true;
}
31

Yes thank you

That's actually the next step I'm going to try to set up. Using the queue to send a message with the data I want.
The data will be collected from Core1 which will fullfill the queue and the Core0 will be in charge of emptying the queue by sending the message.

Thanks

32

I recommend against that. Core 0 is very sensitive to misuse as the WiFi system runs on it. You can easily cause it to crash. Just run both tasks on Core 1 ... that's the whole point of using a multi-tasking OS. I do something similar to your project but it's multi-task wrapper for the PubSubClient library. Three tasks all running on Core 1.

33

So you would recommand me to make those tasks running on Core1 and leaving Core0 as it is?

If yes, I'm not sure I'm getting the purpose of having two cores if I can only use Core1 to make things "in parralel"?

34

As I said, the WiFi (and BlueTooth) systems run on Core 0, thus running independently of the user code on Core 1. I would ask, why do you think you need to use two cores for your project?

35

The system I'm doing for example need to be connected to Telegram to exchange informations; and is also a Web server to display other data.

As soon as the server of Telegram is not reachable (because of not internet mostly), then the code is blocked by the instruction server.connect() over 2 minutes if server.setHandshakeTimeout(30) isn't called before (which still block the code for 30sec).

This said, it means that trying to access to the server become an almost impossible thing, even worse when you click on some buttons and you have to wait for 30sec that the changes are taken into account

I could also not try to reconnect everytime but only like once every 5min but reconnecting the sooner is the better

Here are some of the reasons

36

If you don't have an internet connection, how will the server work anyway?

Not good ones. The problems you're having can be overcome by properly partitioning the functions across different FreeRTOS tasks, all on the same core, and controlling their interactions via the facilities provided by the OS. That's the whole point of a multi-tasking OS.

37

Thanks to local network. My box could not have access to internet but I'll still be able to connect to the ESP on my local network. And even with internet, Telegram's servers could be down for a reason or another.

You are right.
Therefore I'm just trying to understand why it would be bad to do it on Core0, even if surely I can do it on Core1.
For what I understood so far, all of WiFi related activies are executed on Core0 right? So a server.connect() which requires to communicate over WiFi ends up being executed on Core0 no?

38

When you develop firmware for the ESP32, FreeRTOS runs under the hood regardless of whether you add additional tasks in your sketch.
The setup() and loop() functions themselves actually run in a dedicated task on core 1.

As already mentioned by @gfvalvo, core 0 is typically "reserved" for system functionalities such as the WiFi or Bluetooth stack.
If you use core 0 for tasks that block the MCU or are too resource-intensive, the result is that other "critical" tasks won’t have the resources they need to run, and the likelihood of a task watchdog reset becomes very high.
So, you can run your tasks on core 0, but it's not a good practice.

Core 1 is typically also referred to as the "APP CPU" (while core 0 is called "PRO CPU") precisely to highlight that it is the most suitable core for running user-defined tasks.

39

Okay thanks for the explanations.

Tho, still something I'm not sure I understood well

You confirmed that WiFi activities run on Core0.
So my question was, even if I call server.connect() from Core1, it will ask Core0 to do what it has to do since the communication with the server is by internet, meaning over WiFi.

What I think (and can be wrong that's why I'm asking) is the following:

  • You call server.connect() from Core1 (in a specific task or from the loop)
  • This awake a blocking task that tries to connect to the server. But since it needs WiFi it runs on Core0 (thanks to a Semaphore give or whatever).
  • When the task finishes, the Semaphore is freed and Core1 gets the result and continues from where it was.

I may be wrong from my point number 2 but if not, then from my point of view id doesn't change a thing to call server.connect() from Core1 or Core0, unless giving the opportunity to Core1 to do something else.

Of course, the priority of the task that calls server.connect() has the lower priority (1) so it shouldn't block Core0 from doing the rest it has to do no? Do you know the priority of the system functionnalities that run of Core0?

40

The connect() function in the Arduino library is designed to be blocking, but that doesn't mean the low-level software layer running in the background is also blocking.
Don’t confuse calls to high-level library methods with everything else that's happening under the hood (try compiling an empty Arduino sketch and see how much code is still included in the firmware to get an idea).

The library you're using relies on the ESP32 core APIs for Arduino, which in turn is built on top of ESP-IDF, the official framework from Espressif.

I’ve never looked into it in such detail, but I can assume that when you request a connection to a server, at a low level there's a FreeRTOS task that queues the request until the resource becomes available and then notify back the caller.
And in fact, if out of curiosity you take a look at the ESP-IDF examples for Wi-Fi, you'll notice that they are all "event-based" — a concept that is radically different from the typical "do-this-and-wait" approach used in the Arduino world.

In any case, CPU1 can safely call functions that will then be executed by CPU0, since both CPUs share the same memory.

41

Noted

Thank you a lot for your time.

Now I know I should limit the use of the Core0 especially for application that are designed to run for a long time, non-stop.