Sending video(.avi) and audio (.wav) files with arduino script from ESP32S3 via http post multipart/form-data to server

Hi, I can't find a method of sending .avi and .wav files generated on a ESP32 board and stored on the sd card via http post multipart/form-data created by a arduino script.

Board: XIAO ESP32S3 (Sense)

UseCase: I would like to use a Arduino script to record both video and audio with the board, store them temporarily on the sd card and then send them via http post multipart/form-data to a server that processes the data further. The two files together are smaller than 1 megabyte.

Problem: Recording a video as an .avi file, recording audio as a .wav file and saving both files on the sd card works fine without any problems. The structure of the post request is also valid and works as well. In addition, the request contains a json generated in the script (sending the json as part of the request already works). However, I do not know how to attach the .avi and .wav file to the request. Simply adding the files as a string does not work.

Question: How can I send .avi and .wav as part of the http post multipart/form-data request? Is there a way to attach the files read directly from the SD card to the request?

Code example: Below you can find a minimal code example (Arduino) that establishes a wifi connection, retrieves the two files from the SD card, generates a json and sets up the request. What is missing is the way the files are sent with the request.

I am happy to hear about any suggestions for solutions.

#include <WiFi.h>
#include <HTTPClient.h>
#include <ArduinoJson.h>

const char* ssid = "...";
const char* password = "...";

const char* serverAddress = "..."; 

const char* aviFilePath = "/video0.avi";
const char* wavFilePath = "/arduino_rec.wav";

//SD 
#include "FS.h"
#include "SD.h"
#include "SPI.h"
const int SD_PIN_CS = 21;
bool sd_sign = false;
File videoFile; 
File audioFile; 

void setup() {
  Serial.begin(115200);
  
  // Connect to Wi-Fi
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi...");
  }
  Serial.println("Connected to WiFi");

  //Setup SD Card 
  // Initialize the SD card
  if (!SD.begin(SD_PIN_CS)) {
    Serial.println("SD card initialization failed!");
    return;
  }

  uint8_t cardType = SD.cardType();

  // Determine if the type of SD card is available
  if(cardType == CARD_NONE){
    Serial.println("No SD card attached");
    return;
  }

  Serial.print("SD Card Type: ");
  if(cardType == CARD_MMC){
    Serial.println("MMC");
  } else if(cardType == CARD_SD){
    Serial.println("SDSC");
  } else if(cardType == CARD_SDHC){
    Serial.println("SDHC");
  } else {
    Serial.println("UNKNOWN");
  }
  
  sd_sign = true;
}

void loop() {
  if (SD.exists("/video0.avi") == 1 && SD.exists("/arduino_rec.wav") == 1) {

    videoFile = SD.open("/video0.avi", FILE_READ);
    audioFile = SD.open("/arduino_rec.wav", FILE_READ);

    if ((WiFi.status() == WL_CONNECTED)) { // Check Wi-Fi connection status

      StaticJsonDocument<96> movement;
      movement["start_date"] = "2024-03-08 16:03:10.210804";
      movement["end_date"] = "2024-03-08 16:04:10.210804";
      movement["audio"] = "audioKey";
      movement["video"] = "videoKey";
      
      String output; 

      serializeJson(movement, output);

      HTTPClient http;
      http.begin(serverAddress); 

      // Set headers for multipart/form-data
      String boundary = "----WebKitFormBoundary" + String(random(0xFFFFFF), HEX);
      http.addHeader("Content-Type", "multipart/form-data; boundary=" + boundary);

      String requestBody = "------" + boundary + "\r\n";
      requestBody += "Content-Disposition: form-data; name=\"json\"\r\n\r\n";
      requestBody += output; 
      requestBody += "\r\n";
      requestBody += "--" + boundary + "\r\n";
      requestBody += "Content-Disposition: form-data; name=\"videoKey\"; filename=\"video0.avi\"\r\n";
      requestBody += "Content-Type: video/avi\r\n\r\n";
      requestBody += videoFile.readString();
      requestBody += "\r\n";
      requestBody += "--" + boundary + "\r\n";
      requestBody += "Content-Disposition: form-data; name=\"audioKey\"; filename=\"arduino_rec.wav\"\r\n";
      requestBody += "Content-Type: audio/wav\r\n\r\n";
      requestBody += audioFile.readString();
      requestBody += "\r\n";
      requestBody += "--" + boundary + "--\r\n";

      // Calculate the content length
      int contentLength = requestBody.length();

      // Set the Content-Length header
      http.addHeader("Content-Length", String(contentLength));

      // Send the POST request
     int httpResponseCode = http.sendRequest("POST", requestBody);

      if (httpResponseCode == HTTP_CODE_OK) {
        Serial.println("Data sent successfully!");
      } else {
        Serial.print("Error sending data. HTTP code: ");
        Serial.println(httpResponseCode);
      }

      videoFile.close(); 
      audioFile.close(); 

      http.end();
      }
    } 
    delay(10000); 
  

try with ESPAsyncWebServer

from the documentation

Respond with content coming from a File

//Send index.htm with default content type
request->send(SPIFFS, "/index.htm");

//Send index.htm as text
request->send(SPIFFS, "/index.htm", "text/plain");

//Download index.htm
request->send(SPIFFS, "/index.htm", String(), true);

Respond with content coming from a File and extra headers

//Send index.htm with default content type
AsyncWebServerResponse *response = request->beginResponse(SPIFFS, "/index.htm");

//Send index.htm as text
AsyncWebServerResponse *response = request->beginResponse(SPIFFS, "/index.htm", "text/plain");

//Download index.htm
AsyncWebServerResponse *response = request->beginResponse(SPIFFS, "/index.htm", String(), true);

response->addHeader("Server","ESP Async Web Server");
request->send(response);

look at the other options too like Respond with content using a callback

Hi @J-M-L, thanks for your suggestions. However, my problem is not the general creation of the request (it works) but specifically how I can add an .avi and .wav that is stored on the sd card to the multipart request. Unfortunately, the suggested repository does not contain any suggestions regarding a corresponding http post multipart/form-data request. In addition, the repository only suggests how to send "text/plain" with the requests, but none for video or audio files.

Are there some other suggestions on how to use a arduino script to add the .avi and .wav file from the sd card to the http post multipart/form-data request?

I think I missed something in your original request

what you want is for the ESP32 to issue a POST request towards another HTTP server and send the file multipart/form-data.

it's not a download of the file content to a browser

right?

@J-M-L exactly, I have two files (one .avi and one .wav) stored on a sd card that is connected to the ESP32. I would like to send these two files as part of an http post multipart/form-data request from the ESP32 to another server, which then processes the files. The request setup itself already works, only adding the files is missing. Do you have any ideas on how I can implement this? So to add the .avi and .wav file from the sd card to the http post multipart/form-data request?

Are you familiar with the look of such post request?

Building manually a multi part request is a pain because of needing to provide the size of the POST request content size including separator and CR LF etc… so it’s all about counting bytes

Less than 1 Meg is not that big. Why do you want to do multi part , are you sending other form info?

the format looks something like that

POST /upload HTTP/1.1
Host: www.example.com
Content-Type: multipart/form-data; boundary=---------------------------1234567890
Content-Length: (total size of the HTTP request)

-----------------------------1234567890
Content-Disposition: form-data; name="text_field"

value_of_text_field
-----------------------------1234567890
Content-Disposition: form-data; name="file"; filename="example.bin"
Content-Type: application/octet-stream

(binary file content here)

-----------------------------1234567890--

if you send binary make sure the boundary string you use in not part of the binary (so don't make it too short - sometimes it's build with concatenating time and a UUID)

Hey @J-M-L

So I think I managed to construct the http post multipart/form-data request the right way (using correct separators) as I am receiving a 200 status code and the json is send and processed correctly. Surely I would have preferred to create the request with a library and not manually as I do now, but I have not found a library with which a corresponding http post multipart/form-data can be created.

You can see the structure of my request below:


----------WebKitFormBoundary7bd6fd
Content-Disposition: form-data; name="json"

{"start_date":"2024-03-08 16:03:10.210804","end_date":"2024-03-08 16:04:10.210804","audio":"audioKey","video":"videoKey"}

------WebKitFormBoundary7bd6fd

Content-Disposition: form-data; name="videoKey"; filename="video0.avi"

Content-Type: video/avi

------WebKitFormBoundary7bd6fd

Content-Disposition: form-data; name="audioKey"; filename="arduino_rec.wav"

Content-Type: audio/wav

------WebKitFormBoundary7bd6fd--

However, the corresponding files are missing because I don’t know how to add the .avi and .wav files read from the sd card to the post. Are there any further suggestions for this step?

The server to which I send the files from the ESP expects data as multipart/form-data, so that's why I want to use it. Potentially, the file size can also become larger, but the size of 1 Megabyte would be sufficient for the basic functionality.

Two interesting parts in WString.h (somewhere under your board /packages)

#ifdef BOARD_HAS_PSRAM
        enum { CAPACITY_MAX = 3145728 }; 
#else
        enum { CAPACITY_MAX = 65535 }; 
#endif 

Strings are 64KB max unless PSRAM is Enabled. This is an option (in the IDE, under the Tools menu) for some of my boards, but not others.

// if there's not enough memory for the concatenated value, the string
// will be left unchanged (but this isn't signalled in any way)
String & operator +=(const String &rhs) {

The first thing I tried with your example was to add some quotes

      requestBody += "videoFile.readString()";

and that came through as a separate part; so the structure is mostly right. If PSRAM is not enabled, then what happens is

  1. readString() grabs just under 64KB; it can't make the String bigger
    • Stream::readString does it one byte at a time, which is inefficient, especially since it has to dynamically resize the string as it slowly grows
  2. The resulting String is supposed to be added to requestBody, but because the result would also exceed the max, nothing happens
  3. The part is empty

HTTPClient.sendRequest has an overload to send a Stream, which is a base class of File. Instead of the JSON, you can add metadata in the request path and/or with query parameters, or even custom headers. That's definitely simpler (although I didn't try it) but that's just one file at a time.

Where's the fun in that?

To avoid building a giant string, just so that you can get the total length, use Chunked Transfer Encoding. You only need the length of each chunk. When sending a file, you are also reading the file in chunks. I don't see any support in HTTPClient, so it is done manually "over the wire" with the WiFiClient

  const char host[] = "10.0.0.231";  // This is where I ran a test server
  const uint16_t port = 8080;

  videoFile = SPIFFS.open(aviFilePath, FILE_READ);  // no SD handy
  audioFile = SPIFFS.open(wavFilePath, FILE_READ);

  StaticJsonDocument<96> movement;  // Deprecated in ArduinoJson v7
  movement["start_date"] = "2024-03-08 16:03:10.210804";
  movement["end_date"] = "2024-03-08 16:04:10.210804";
  movement["audio"] = "audioKey";
  movement["video"] = "videoKey";

  String json;
  serializeJson(movement, json);

  WiFiClient wifi;

  if (wifi.connect(host, port)) {

    wifi.println("POST /upload HTTP/1.1");
    wifi.print("Host: ");
    wifi.println(host);

    String boundary = "..--..--MyOwnBoundary" + String(random(0xEFFFFF) + 0x100000, HEX);
    wifi.println("Content-Type: multipart/form-data; boundary=" + boundary);
    wifi.println("Transfer-Encoding: chunked");
    wifi.println();  // end of request headers (will be more headers for each part)

    auto stringPart = [&](const String &name, const String &value) {
      // RFC 1342 §7.2.1 says CRLF before two hyphens is considered part of boundary
      String part = "\r\n--" + boundary + "\r\n";
      part += "Content-Disposition: form-data; name=\"";
      part += name;
      part += "\"\r\n\r\n";  // CRLF to end header, and CRLF before content
      part += value;

      // Serial.printf("part %s len %d\n%s\n", name, part.length(), part.c_str());
      wifi.println(part.length(), HEX);
      wifi.println(part);
    };

    auto filePart = [&](const String &name, File &file, const String &filename, const String &type) {
      String part = "\r\n--" + boundary + "\r\n";
      part += "Content-Disposition: form-data; name=\"";
      part += name;
      part += "\"; filename=\"";
      part += filename;
      part += "\"\r\nContent-Type: ";
      part += type;
      part += "\r\n\r\n";  // CRLF to end header, and CRLF before content

      wifi.println(part.length(), HEX);
      wifi.println(part);

      uint8_t buf[1 << 12];  // has to fit in stack; maybe use global buffer instead
      for (size_t r = file.read(buf, sizeof(buf)); r > 0;) {
        wifi.println(r, HEX);
        wifi.write(buf, r);
        wifi.println();
        Serial.printf("wrote %d of %s, now position %d\n", r, file.name(), file.position());
        r = file.read(buf, sizeof(buf));
      }
    };

    auto endPart = [&]() {
      String part = "\r\n--" + boundary + "--\r\n";  // -- on both ends marks terminating boundary

      wifi.println(part.length(), HEX);
      wifi.println(part);
      wifi.println(0);  // last chunk
      wifi.println();
    };

    stringPart("json", json);
    filePart("videoKey", videoFile, aviFilePath + 1, "video/avi");  // + 1 to skip leading slash
    filePart("audioKey", audioFile, wavFilePath + 1, "audio/wav");
    endPart();

    char buf[200];  // buffer big enough for expected response
    if (readFirstBlock(wifi, (uint8_t *)buf, sizeof(buf))) {
      Serial.printf("status: %d\n", parseStatusCode(buf));
    }

    videoFile.close();
    audioFile.close();

    wifi.stop();
  } else {
    Serial.println("Can't connect to HTTP server");
  }

At the end are calls to a couple of functions to read (the first part of) the response, and then to parse out the status code.

size_t readFirstBlock(WiFiClient &client, uint8_t *buf, size_t len) {
  size_t p = 0;
  size_t available = 0;
  // wait for first byte
  while (client.connected()) {
    if (available = client.available()) {
      break;
    }
    delay(1);
    p++;
  }
  if (!available) {
    Serial.println("not connected");
    return 0;
  }

  Serial.printf("waited %d for %d available\n", p, available);
  // expect payload to be a single contiguous block
  p = client.read(buf, std::min(available, len - 1));
  buf[p] = 0;
  Serial.printf("buf: %d-->\n%s\n<---\n", p, buf);
  return p;
}

int parseStatusCode(const char *data) {
  int code;
  if (sscanf(data, "HTTP/%*s %d", &code) < 1) {
    return 0;
  }
  return code >= 100 && code <= 599 ? code : 0;
}
1 Like

I think the text structure is important don't have empty lines in weird places, it should look like this

------WebKitFormBoundary7bd6fd
Content-Disposition: form-data; name="audioKey"; filename="arduino_rec.wav"
Content-Type: audio/wav
(binary file content here)

------WebKitFormBoundary7bd6fd--

You then dump all the bytes from your file as binary where (binary file content here) is listed. Assuming you have your SD configured correctly in setup, walking through the file could be done with something like this (typed here totally untested)

// ---------- content descriptor start ----------
client.println("------WebKitFormBoundary7bd6fd");
client.println("Content-Disposition: form-data; name=\"audioKey\"; filename=\"arduino_rec.wav\"");
client.println("Content-Type: audio/wav");
// ---------- content  ----------
// Open the file
File file = SD.open("arduino_rec.wav", FILE_READ);
if (file) {
  const size_t bufferSize = 256; // we use a buffer to read 256 bytes at a time
  uint8_t buffer[bufferSize];
  size_t bytesRead;

  while (file.available()) {
    bytesRead = file.read(buffer, sizeof(buffer));
    if (bytesRead > 0) client.write(buffer, bytesRead);
  }
  file.close();
} else {
  Serial.println("Error opening arduino_rec.wav");
}
// ---------- content descriptor end ----------
client.println("\r\n");
client.println("------WebKitFormBoundary7bd6fd");  // or "------WebKitFormBoundary7bd6fd--" if it's the last one

Hey @kenb4 I really like the information and conclusions you shared as well as the suggested approach, thank you very much. The code runs so far but unfortunately I get the status code 400 with the following output as a response from the server:

wrote 4096 of arduino_rec.wav, now position 626688
wrote 4096 of arduino_rec.wav, now position 630784
wrote 4096 of arduino_rec.wav, now position 634880
wrote 4096 of arduino_rec.wav, now position 638976
wrote 1068 of arduino_rec.wav, now position 640044
waited 0 for 407 available
buf: 199-->
HTTP/1.1 400 Bad Request
Server: nginx/1.15.8
Date: Thu, 14 Mar 2024 16:10:23 GMT
Content-Type: text/html
Content-Length: 255
Connection: close

<html>
<head><title>400 The plain HTTP request
<---
status: 400

Do you have any idea what the problem could be? I don't think it's the file size, as it's rather small and the server also accepts larger files.

do you have a way to see the request on the server side?

At the start of the response, there are 407 available bytes, and the Content-Length of the body is 255. The buffer is only

    char buf[200];  // buffer big enough for expected response

So making that at least 408 (don't forget the trailing null) will read the rest. The stack might be getting tight though; so better yet, to print the whole response regardless of size and return the status code, use instead

int statusFromResponse(WiFiClient &client) {
  size_t p = 0;
  size_t available = 0;
  // wait for first byte
  while (client.connected()) {
    if (available = client.available()) {
      break;
    }
    delay(1);
    p++;
  }
  if (!available) {
    Serial.println("not connected");
    return 0;
  }
  Serial.printf("waited %d for %d available --->\n", p, available);

  int code = -1;
  size_t t = 0;
  char buf[200];

  while (client.connected() && available) {
    p = client.read((uint8_t *)buf, std::min(available, sizeof(buf) - 1));
    if (t == 0) {
      sscanf(buf, "HTTP/%*s %d", &code);
    }
    t += p;
    buf[p] = 0;
    Serial.print(buf);
    available = client.available();
  }
  Serial.printf("<---\n%d total\n", t);
  return code;
}

and so the two function calls are replaced with a single one, with no buffer

    Serial.printf("status: %d\n", statusFromResponse(wifi));

If we're lucky, the remaining 200-ish bytes will have useful details.

1 Like

Hey @kenb4 thanks a lot for the further information. I got your point and implemented the code as suggested, but as far as I understand the logs do not contain any further information:

wrote 4096 of arduino_rec.wav, now position 622592
wrote 4096 of arduino_rec.wav, now position 626688
wrote 4096 of arduino_rec.wav, now position 630784
wrote 4096 of arduino_rec.wav, now position 634880
wrote 4096 of arduino_rec.wav, now position 638976
wrote 1068 of arduino_rec.wav, now position 640044
waited 0 for 407 available --->
HTTP/1.1 400 Bad Request
Server: nginx/1.15.8
Date: Fri, 15 Mar 2024 00:24:01 GMT
Content-Type: text/html
Content-Length: 255
Connection: close

<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<center>The plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx/1.15.8</center>
</body>
</html>
<---
407 total
status: 400

Unfortunately, I can only see the following log on the server side from which I cannot draw any more conclusions:

[15/Mar/2024:00:27:03 +0000] "POST /api/movement/999 HTTP/1.1" 400 255 "-" "-" "-"

@kenb4 do you see another possible cause of error with the logs or do you have an idea what else could be going wrong?

The full response says

<center>The plain HTTP request was sent to HTTPS port</center>

We actually got the first half of it in the <title> before.

Nginx is being thorough and completely disallows unencrypted plain HTTP requests on the HTTPS port. It could be "nice" and let those work, but then you might think everything is encrypted, when it is not; a false sense of security.

For a quick workaround, you can replace

  WiFiClient wifi;

with

  WiFiClientSecure wifi;
  wifi.setInsecure();

which says, "Use the secure client without setting up all the certificates etc", which at a minimum means that you are not verifying the identity of the target server. It could be spoofed. This option is also used when the server has "self-signed certificates", which cannot be verified.

Of course, you can also pursue properly setting up the secure connection. Or using the plain HTTP port, if the server has one active.

1 Like

Hey @kenb4 thanks a lot for your further suggestions and explanations, it worked, you see the corresponding response below:

wrote 4096 of arduino_rec.wav, now position 634880
wrote 4096 of arduino_rec.wav, now position 638976
wrote 1068 of arduino_rec.wav, now position 640044
waited 137 for 396 available --->
HTTP/1.1 200 OK
Server: nginx/1.15.8
Date: Fri, 15 Mar 2024 10:33:22 GMT
Content-Type: application/json
Content-Length: 46
Connection: keep-alive
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, baggage, sentry-trace
Access-Control-Allow-Methods: GET, PUT, PUSH, DELETE, OPTIONS
<---
396 total
status: 200

Is there anything else to consider with regard to the request? The files sent are currently quite small. What is the upper limit?

Is CAPACITY_MAX decisive? Do the sent files together have to be smaller than 3 mb or what is the best number to set the limit?

the comment was made in case you would consider making a huge String to send out in one go.

The proposed solution does not create first a String in memory so it does not apply

the costly piece is just this buffer used to dump the file in chunks rather than byte by byte. You could make it smaller if you want to be memory conscious for the stack.

there is another fixed size buffer to read the response

that's also on the stack, so same consideration applies.


Note that if that second buffer is too small to get the full answer loaded wifi.stop(); will be called but the client still has unread content in its buffer.

➜ I'm unsure what happens then but it's possible that the client won't be released from memory (was the case in some Ethernet implementation in the past). It might actually be necessary to empty the buffer anyway, so fill-up the buffer with p = client.read(buf, std::min(available, len - 1)); but then if the min was not available read and ignore the remaining bytes.

1 Like

I'd guess a web server has to handle clients disconnecting without reading everything. However, the follow-up function in post #12, statusFromResponse, does use a loop to read everything immediately available, regardless of the buffer size, after waiting for the first byte. It does not wait after that. In the 200 OK response in post #15, the headers declare

Content-Type: application/json
Content-Length: 46

but that JSON is not there. The last thing read was a header, and not even the blank line between the headers and the body. Apparently there was a stall, and nothing immediately available, so the reader quit.

I wrote something to handle that situation: if the headers declare a Content-Length (in the first contiguously available blocks), the reader will wait until it gets those bytes. It's a little much though. I can post it if you want to try it.

If you want to reduce the buffer size in the file chunker, For reading efficiency, I'd keep it a power of two, and no smaller than the page/sector size, which might be 512 (depending on the media).

You should try some plausibly large files.

1 Like

Hey @kenb4 and @J-M-L thank you for the further explanations. Especially post #8 and post #12 have led me to the solution of this issue, thank you very much.

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.