Encoding the text file using Base64 but not working on audio file

Hi

I want to encode my audio file using base64.
Here is the code:

#include <SPI.h>
#include <SD.h>
#include <base64.h>
#include "Arduino.h"
#include <FS.h>
#include "Wav.h"
#include "I2S.h"
#include <WiFi.h>
#include <HTTPClient.h>


const char* ssid = "name";
const char* password = "pass";
const char* serverName = "http://***.**.**.**:8000";



const char filename[] = "/soundMont.mp3";

File file;

String get_wifi_status(int status){
    switch(status){
        case WL_IDLE_STATUS:
        return "WL_IDLE_STATUS";
        case WL_SCAN_COMPLETED:
        return "WL_SCAN_COMPLETED";
        case WL_NO_SSID_AVAIL:
        return "WL_NO_SSID_AVAIL";
        case WL_CONNECT_FAILED:
        return "WL_CONNECT_FAILED";
        case WL_CONNECTION_LOST:
        return "WL_CONNECTION_LOST";
        case WL_CONNECTED:
        return "WL_CONNECTED";
        case WL_DISCONNECTED:
        return "WL_DISCONNECTED";
    }
}
  
void setup() {
  Serial.begin(115200);

  ////
  int status = WL_IDLE_STATUS;

  WiFi.begin(ssid, password);
  Serial.println("Connecting");
  Serial.println(get_wifi_status(status));
  while(WiFi.status() != WL_CONNECTED) {
    delay(500);
    // Serial.print(".");
    status = WiFi.status();
    Serial.println(get_wifi_status(status));
  }
  Serial.println("");
  Serial.print("Connected to WiFi network with IP Address: ");
  Serial.println(WiFi.localIP());
 ////

  Serial.println("Initializing SD card...");
  while (!SD.begin(SS)) {
    Serial.write('.');
    delay(1000);
  }
  Serial.println(F("\nWiring is correct and a card is present."));

  file = SD.open(filename);
  if (file) {
      //encoding base64
      unsigned int fileSize = file.size();         // Get the file size.
      uint8_t * pBuffer = (uint8_t *) malloc(fileSize);  // allocate memory for the file.
      if (pBuffer != nullptr) {       // if allocation worked

        file.read(pBuffer, fileSize);     // then read the whole file into SRAM.
        // >>>> this is where you do something with the buffer <<<<
        String toEncode =(char*)pBuffer;
        String encoded = base64::encode(toEncode);
        Serial.println(encoded.length());
        Serial.println(encoded);

      free(pBuffer);  // Free the memory that was used by the buffer.
      file.close();

      if(WiFi.status()== WL_CONNECTED){
      WiFiClient client;
      HTTPClient http;
    
      // Your Domain name with URL path or IP address with path
      http.begin(client, serverName);
     
      // If you need an HTTP request with a content type: text/plain
      http.addHeader("Content-Type", "text/plain");

      int httpResponseCode = http.POST("encoded");
     
      Serial.print("HTTP Response code: ");
      Serial.println(httpResponseCode);
        
      // Free resources
      http.end();
      }
    else {
      Serial.println("WiFi Disconnected");
    }

      } else {
        Serial.println("\Error: not enough memory to load audio");
      }
    } else {
      Serial.println("\Error: could not find audio");
    }
    
}

void loop() {}

when i read text file it encoded correctly and as expected. But when I encode an audio file with the same code that works perfectly with text file, it returned short encoded string ended with 2 equals sign as shown below:

Connected to WiFi network with IP Address: ***.**.**.*
Initializing SD card...

Wiring is correct and a card is present.
12
UklGRuRXAQ==
HTTP Response code: -1

Why that's happened?
How to solve this issue?
I want to simply transfer the file (audio file) to server even without encoding. this is my goal

as asked in the other thread (why did you open a new one)

and the wav file is not text (probably told that to you a few times already), it's binary data so why do you do

this will stop at the first null byte...(and the wav buffer is not null terminated)

I thought it might be different topic to be asked, but hello again :slight_smile: .

How large is the file

it's 88 kb.

and the wav file is not text (probably told that to you a few times already), it's binary data so why do you do

Sorry I just realized that, kindly help me with that, I'm new in this field.
thank you so much!

are you using this Basic/Base64.h at master · esp8266/Basic · GitHub ?

you need to find out what length will be the encoded version, for that you call base64_enc_len(fileSize) ➜ you need to build an char output buffer that has this size plus one byte if you want to add the trailing null char

then you call base64_encode(outputBUffer, (char*) pBuffer, fileSize) and add the trailing null char if you want to

I used GitHub - Densaugeo/base64_arduino: Base64 encoder/decoder for arduino repo

Did you read the Readme for the library? It starts from example just for your case:

Binary to base64 example:

unsigned char binary[] = {133, 244, 117, 206, 178, 195};
unsigned char base64[9]; // 8 bytes for output + 1 for null terminator

unsigned int base64_length = encode_base64(binary, 6, base64);

printf("%d\n", base64_length); // Prints "8"
printf((char *) base64); // Prints "hfR1zrLD"

it's the same principle

  • you build your binary buffer
  • you calculate the length of base64 string needed for a given number of binary bytes (including a trailing null) and allocate a buffer of that size
  • you encode and add the trailing null (if the library does not do it)

Hello to you all,

I stuck in this issue and I feel it's easy to solve but cuz I'm a bit stressed, can you help.

here what I wrote:

file = SD.open(filename);
  if (file) {
      //encoding base64
      
      unsigned int fileSize = file.size();// Get the file size.
      //char* BinaryBuffer = (char*) malloc(fileSize);  // allocate memory for the file.
      uint8_t * BinaryBuffer = (uint8_t *) malloc(fileSize);  // allocate memory for the file.

      if (BinaryBuffer != nullptr) { // if allocation worked
        file.read(BinaryBuffer, fileSize);     // then read the whole file into SRAM.
        
        unsigned int base64len = encode_base64_length(fileSize);
        unsigned char base64Buffer[base64len+1];

        // >>>> this is where you do something with the buffer <<<<
        unsigned int base64_length = encode_base64((unsigned char)BinaryBuffer, sizeof(BinaryBuffer), base64Buffer);

        printf("%d\n", base64_length); // Prints "8"
        printf((char *) base64Buffer); // Prints "hfR1zrLD"

the error:

cast from 'uint8_t*' {aka 'unsigned char*'} to 'unsigned char' loses precision [-fpermissive]

I'm forced to use uint8_t* to use

file.read(BinaryBuffer, fileSize);

to read the whole file into SRAM.

so I faced the error in

unsigned int base64_length = encode_base64((unsigned char)BinaryBuffer, sizeof(BinaryBuffer), base64Buffer);

as they expected unsigned char
Kindly, I need your support, highly appreciated.

The type of your BinaryBuffer is pointer to unsigned char, not the unsigned char.
The function encode_base64 need a pointer too, so you have not cast it to unsigned char:

unsigned int base64_length = encode_base64(BinaryBuffer, sizeof(BinaryBuffer), base64Buffer);

@b707 @J-M-L
Thank you all, based on your replies and the library I used, I came up with this code:

```cpp
#include <SPI.h>
#include <SD.h>
//#include <base64.h>
#include "base64.hpp"
#include "Arduino.h"
#include <FS.h>
#include "Wav.h"
#include "I2S.h"
#include <WiFi.h>
#include <HTTPClient.h>

const char* ssid = "name";
const char* password = "pass";
const char* serverName = "http://xxx.xx.xx.xx:8000";

File file;
  
void setup() {
  Serial.begin(115200);

  Serial.println("Initializing SD card...");
  while (!SD.begin(SS)) {
    Serial.write('.');
    delay(1000);
  }
  Serial.println(F("\nWiring is correct and a card is present."));

  file = SD.open(filename);
  if (file) {

      //encoding base64
      
      //Note1: you build your binary buffer

      unsigned int fileSize = file.size();// Get the file size.
      uint8_t * BinaryBuffer = (uint8_t *) malloc(fileSize);  // allocate memory for the file.
      if (BinaryBuffer != nullptr) { // if allocation worked
        file.read(BinaryBuffer, fileSize);     // then read the whole file into SRAM.
          
        //Note2:you calculate the length of base64 string needed for a given number of binary bytes (including a trailing null) and allocate a buffer of that size
       
        unsigned int base64len = encode_base64_length(fileSize);
        unsigned char base64Buffer[base64len+1];
  
        //Note3: you encode and add the trailing null (if the library does not do it) * I think the lib do it, kindly correct me if I'm wrong

        unsigned int base64_length = encode_base64(BinaryBuffer, sizeof(BinaryBuffer), base64Buffer);
        printf("%d\n", base64_length); 
        printf((char *) base64Buffer); 

      free(BinaryBuffer);  // Free the memory that was used by the buffer.
      free(base64Buffer);
      file.close();

     
      } else {
        Serial.println("\Error: not enough memory to load audio");
      }
    } else {
      Serial.println("\Error: could not find audio");
    }
    
}

void loop() {}

Did I apply the instruction correctly? is it what you meant?

I recommend you use malloc() for this buffer as well. That way you won't overflow the stack if there isn't enough stack space.

        //Note2: you calculate the length of base64 string needed for a given number of binary bytes (including a trailing null) and allocate a buffer of that size
       
        unsigned int base64len = encode_base64_length(fileSize);

        unsigned char* base64Buffer = malloc(base64len+1);
        if (base64Buffer != NULL)
        {
          //Note3: you encode and add the trailing null (if the library does not do it) * I think the lib do it, kindly correct me if I'm wrong

          unsigned int base64_length = encode_base64(BinaryBuffer, sizeof(BinaryBuffer), base64Buffer);
          printf("%d\n", base64_length); 
          print((char *)base64Buffer); 
          free(base64Buffer);
        }
       else 
       {
        Serial.println("\Error: not enough memory to convert");
      }
      free(BinaryBuffer);  // Free the memory that was used by the buffer.

Kind of like robbing Peter to pay Paul. Stack Space + Heap Space = FIXED

The malloc way has significant advantage: in case of memory lacking malloc just return nothing and don't crash the entire program. Of course, base64 array won't be encoded, but the rest of routine still works.

Not on every architectures and you might have additional constraints like On esp32 the task running the main loop has a fixed stack size.

So, code at post 10 matched what you mentioned here:

The only thing i need to do is fixing what John mentioned right?

True. But if were my project on an ESP32, I'd use PSRAM for both buffers.

and you'd be right :slight_smile:

How can I do that?

In a similar way to using space obtained with malloc(), assuming your ESP32 has PSRAM:

#include "Arduino.h"

#ifndef CONFIG_SPIRAM
#error "PSRAM Required"
#endif

void setup() {
	uint8_t *ptr = static_cast<uint8_t*>(ps_malloc(100));
	if (ptr == nullptr) {
		Serial.println("PSRAM Allocation Failed");
	} else {
		Serial.println("PSRAM Allocated");
		// do stuff with data pointed to by 'ptr'
		free(ptr);
	}
}

void loop() {
}

Great, why you choose to use PSRAM?