SSL - A buffer is too small to receive or write a message

Projects Programming
1

I have recently developed a project for an ESP8266 to read my Google calendar and display daily reminders. My project is based on the project by Andreas Spiess here:

^GitHub - SensorsIot/Reminder-with-Google-Calender: Reminder connects an ESP8266 to a Google Calender

I am instead using a TFT display to display the time and the reminders. The project works fine on the ESP8266, but I needed more GPIO pins so decided to port it to the ESP32. This has been relatively straightforward, except that the code that fetches the calendar returns this error (obtained with debug set to 'Verbose':

Fetching events from: 
https://script.google.com:/macros/s/AKfycbx7OxaWUoywhIxFxrDDcxPzzEB1Lw43SZ_WKfCN8oFVeWvXZPjwRaMrfdm21O63bH1H/exec
[ 43293][V][ssl_client.cpp:369] send_ssl_data(): Writing HTTP request with 150 bytes...
[ 43293][V][ssl_client.cpp:374] send_ssl_data(): Handling error -27136
[ 43299][E][ssl_client.cpp:37] _handle_error(): [send_ssl_data():375]: (-27136) SSL - A buffer is too small to receive or write a message
[ 43313][V][ssl_client.cpp:324] stop_ssl_socket(): Cleaning SSL connection.
Performed GET

Debug messages narrow the problem down to the following code:

  HTTPSRedirect gclient;
  gclient = new HTTPSRedirect(httpsPort);
  gclient->setInsecure();
  gclient->setPrintResponseBody(true);
  gclient->setContentTypeHeader("application/json");

  
  String calendarData = "";

  //Fetch Google Calendar events
  String url = String("/macros/s/") + GScriptIdRead + "/exec";
  Serial.println(F("Fetching events from: "));
  Serial.print("https://");
  Serial.print(googleHost);
  Serial.print(":");
  Serial.println(url);
  gclient->GET(url, googleHost);
  Serial.println(F("Performed GET"));
  calendarData = gclient->getResponseBody();
  Serial.println(calendarData);

I am using the HTTPSRedirect library by jbuszkie as did Andreas:

^GitHub - jbuszkie/HTTPSRedirect: Clone of https://github.com/electronicsguy/ESP8266.git - just the HTTPSRedirect

This handles HTTPS re-directs such as used by Google but under the hood it boils down to a GET request being passed to the built-in ESP32 ssl_client. The URL parameter is stored as a String type and passed as const String&. The SSL library send_ssl_data() method accepts the GET request as 'sslclient_context *ssl_client'. The type sslclient_context is a struct. The GET() method seems to create the request in an appropriate format, again stored as a String type, which is then somehow passed to the ssl_client.

The sslclient_context struct is defined in terms of mbedtls objects and here is where I get rather lost.

typedef struct sslclient_context {
    int socket;
    mbedtls_ssl_context ssl_ctx;
    mbedtls_ssl_config ssl_conf;

    mbedtls_ctr_drbg_context drbg_ctx;
    mbedtls_entropy_context entropy_ctx;

    mbedtls_x509_crt ca_cert;
    mbedtls_x509_crt client_cert;
    mbedtls_pk_context client_key;

    unsigned long handshake_timeout;
} sslclient_context;

SSL-Client.h:

SSL-Client.cpp

My question is, how can I increase the size of the buffer? Having looked at the source code, I can't see a specific size of buffer defined anywhere. Given how long some URL constructs can be, I find it rather odd that a URL buffer is limited to evidently less than 150 characters?

2

I think I might have found a possible answer:

^Project Configuration - ESP32 - — ESP-IDF Programming Guide latest documentation

It seems there is a setting for "variable buffer length" which by default is disabled. The my next question is, how do I enable this in my Arduino IDE project? There is no 'Component Config' option in the IDE, so I presume this must apply somehow to another programming environment. I had a look in the .arduino15 directory on my system and found an sdkconfig file in .arduino15/packages/esp32/hardware/esp32/2.0.4/tools/sdk/esp32

It has the following entry:

# CONFIG_MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH is not set

This at least confirms that it is disabled as the documentation indicates. The documentation also states that based on the settings configured in the sdkconfig file, an skdconfig.h file is generated in the build directory. I located and had a look in my build directory but couldn't find such a file in there.

I then found an example sdkconfig.h file online and following the format of that file tried adding the following to my .ino:

#define CONFIG_MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH 1

Unfortunately, this made no difference.

I imagine it ought to be possible to apply this setting somewhere, but how?

3

Ok, the .arduino15/packages/esp32/hardware/esp32/2.0.4/tools/sdk/esp32/include/mbedtls/mbedtls/include/mbedtls directory has a file called ssl.h which appears to contain most of the configuration options or the detection code for them. Amogst other things it contains the following:

#if defined(MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH)
    size_t in_buf_len;          /*!< length of input buffer           */
#endif

so I made a copy of the file, renamed it as a backup, then added the line:

#define MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH 1

to it prior to the above statement.

I then re-compiled and re-tested, but unfortunately it made no difference so the problem would appear to be elsewhere?

4

I have uploaded a minimal test version. After a lot of digging around in the SDK sources I am still none the wiser unfortunately.

Google-Minimal.zip (13.2 KB)