ESP32-Cam base64 encoding and communication over serial

Hi there!
I am using an ESP32-CAM to capture an image so that it can be later processed by Processing. I am attempting to send the info through Serial, and read it on Processing. However, I am having trouble getting the image to correctly transfer. From what I can tell, the base64 encoding isn't working properly, for when I paste it into a "Validate base64" tool, it throws an error (length isn't divisible by 4, so it's an invalid string).
Here is my code:

/*********
  Rui Santos
  Complete project details at https://RandomNerdTutorials.com/esp32-cam-take-photo-save-microsd-card
  
  IMPORTANT!!! 
   - Select Board "AI Thinker ESP32-CAM"
   - GPIO 0 must be connected to GND to upload a sketch
   - After connecting GPIO 0 to GND, press the ESP32-CAM on-board RESET button to put your board in flashing mode
  
  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files.
  The above copyright notice and this permission notice shall be included in all
  copies or substantial portions of the Software.
*********/

#include "esp_camera.h"
#include "Arduino.h"
#include "soc/soc.h"           // Disable brownour problems
#include "soc/rtc_cntl_reg.h"  // Disable brownour problems
#include "driver/rtc_io.h"
#include <EEPROM.h>            // read and write from flash memory
#include <base64.h>

// define the number of bytes you want to access
#define EEPROM_SIZE 1

// Pin definition for CAMERA_MODEL_AI_THINKER
#define CAMERA_MODEL_AI_THINKER
#define PWDN_GPIO_NUM     32
#define RESET_GPIO_NUM    -1
#define XCLK_GPIO_NUM      0
#define SIOD_GPIO_NUM     26
#define SIOC_GPIO_NUM     27

#define Y9_GPIO_NUM       35
#define Y8_GPIO_NUM       34
#define Y7_GPIO_NUM       39
#define Y6_GPIO_NUM       36
#define Y5_GPIO_NUM       21
#define Y4_GPIO_NUM       19
#define Y3_GPIO_NUM       18
#define Y2_GPIO_NUM        5
#define VSYNC_GPIO_NUM    25
#define HREF_GPIO_NUM     23
#define PCLK_GPIO_NUM     22


void setup() {
  WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); //disable brownout detector
 
  Serial.begin(2000000);
  
  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sccb_sda = SIOD_GPIO_NUM;
  config.pin_sccb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG; //JPEG FORMAT
  
  if(psramFound()){
    config.frame_size = FRAMESIZE_QVGA; //320x240
    config.jpeg_quality = 10; //Lower size = higher quality
    config.fb_count = 2;
  } else {
    config.frame_size = FRAMESIZE_SVGA;
    config.jpeg_quality = 12;
    config.fb_count = 1;
  }
  
  // Init Camera
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }
  
  pinMode(4, OUTPUT);

  takePicture();
  
  
  delay(2000);
 
  //Serial.println("Going to sleep now");
  delay(2000);
  esp_deep_sleep_start();
  Serial.println("This will never be printed");
}

void loop() {

}

void takePicture(){
  camera_fb_t* fb = NULL;
  
  // Take Picture with Camera
  fb = esp_camera_fb_get();  
  if(!fb) {
    Serial.println("Camera capture failed");
    return;
  }

  // Turns on the ESP32-CAM white on-board LED (flash) connected to GPIO 4
  digitalWrite(4, HIGH);

  delay(500);
  String encoded = base64::encode(fb->buf, fb->len);
  Serial.write(encoded.c_str(), encoded.length());
  Serial.println();

  //Serial.print('\n');
  /* Trying for direct transmission, doesn't seem to work either
  for(int i=0; i<fb->len; i++){
    Serial.print(fb->buf[i]);
    Serial.print(",");
  }
  Serial.print('\n');
  */

  esp_camera_fb_return(fb);

  // Turns off the ESP32-CAM white on-board LED (flash) connected to GPIO 4
  digitalWrite(4, LOW); 
}

As you can see, I have adapted the code from RandomNerdTutorials, and I have also taken code from this repo that does a similar thing. However, the base64 encoding is sometimes oversized/undersized, and when I adjust the padding (= at the end) I still don't get a valid JPEG image (I am checking using this tool).
Here is an example base64-encoded string:

/9j/4AAQSkZJRgABAQEAAAAAAAD/2wBDAAoHCAkIBgoJCAkLCwoMDxkQDw4ODx8WFxIZJCAmJiQgIyoLToxKCs2KyjMkQzNjs9QEFAJzBHTEY/SzoQD7/2wBDAQLCw8NDx0QEB0+KSMpPj4+Pj+Pj4+Pj4+Pj+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4Pj4+Pj4+Pj4+j4+Pj4+Pj7/xAAfAAABBQEBQEBAQAAAAAAAAAAQIDBAUGBwgJCgv/xA1EAACAQMDAgDBQUEBAAAAX0BAgMABBEFEiExQQYTUWEHInEUMoGRoQgjQrHBFVLR8CQzYnKCCQoWFxgZGiUmJygpKjQ1Njc4OTpDREVGR0hJSlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5eoOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4eLj5OXm5+j6vHy8/T19vf4+fr/xAAfQADAQEBAQEBQEBAAAAAAAAAQIDBAUGBwgCgv/xAC1EQCAQIEBAMEBwUEBAABAncAAIDEQQFITEGkFRB2FxEyIygQgUQpGhsEJIzNS8BVitEKFiQ04SXxFxgZGiYnKCqNTY3ODk6Q0FRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqCg4SFhoeIiYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2dri4+Tl5ufo6ery8/T19vf4+fr/wAARCADwAUADASEAAhEBAxEB/9oADAMBAAIRAxEAPwDiwMnb2pcYJ615z1L2EHK8nFO79qHoA1z2XB96MnnPtU6WGBHH0pPx4oiljPrUb9eWq7gSD5/lqJsbvetY6mTvciajmPIkQ9aQdfaTFJ3Y9TxTjWfoXsFHOOaSQrZplPYNxpHTPr1rRCAcUlLlENGaD0qhNTls80g64pbIPU0mauTBoaPal75ofwiFPtSaSYdLBSU2ANGaloNjqDg/Kd34Uo4Jrh2J32ExjgU73GAC2agEN75PX2poo6F7h82Dls470h7ZpRHbTQbxt4AHtio3HvzVrTUnYhPeovmrouRIiPXv+FHIyHVs1p0JsDcYpqChfCBIp/8A108dKzKuIQueM/jR3ouKzE4pu3indlMYeKT8quxAym800wQg9KO3SqYhKOo9aoYYweaQ9aQhM8fWlpuwISkOPxqBB9aT3qxiHj1pD60eYzqBXmXNBGIyOcUDnmnfQYfz6UjexqC0Jmj9Kp9xdBjHmo2YdT1NBXIK5J+mag5/GulRsRcjAJOO1SnLP69Vy8iRvU0hwtAkwTOOlSZ4pTj1KTDFJSEwo6UCGkcw+9MmQ3nPbFNNCdmJDO9H881qSxKUc1yriHrRzu4quoDaPWmyUxKkDFpO1F7AITfpVc1thnV84ztJo5zzXlpGyQ7bxxTPpTXYAzQeBVWtoPVDM+lKRz1pFNaDTULnjpzRTJt1Kkh56/+O1EcV1RutDGdxFJ5oDE1qRHUlT5qiI5NTcoXvz1FSR80pbDQ5vekNZGgGkqkSJ7Ux6a3G9hh603+VXYkSm4A4p3EJjmkPSrsIXt9RTenepuMM9fem9/wqyRe1NoKDnNFTshDSU7hsdXjmnV5yNLAETdx8+MZ9qRvlVDeowfJ6nK4pDnqHOKNQQhTz1prU0zSwj6evFUZ5PlwDgdzWtKOpmj1601j0zXSsMf8A6qXvz+VNvQkkU0SKMPWpWgyLJPU05T83vVryHYfuVvu96fWcyKSpKSD6Z/Goot7xPUZjik5IrUTGnIFJ+FPzQCleab/KjUXUXBxTMYoiOwKMHNJ9atyuISmjPWkgE96KYwpDQmI6zoadjPIry7W1NQJ44pMDGD6UWHohuOaQ/pQyhrHAphfB5PFVFDZBM4C8Nzj1qix3HuPpXTT0RixvQnFJ7itDLcPrTqp2GOA9al/hxSktBkG3FJTTFcd3FSp9xfpzWUioi0dqOhVyOmk4oExtAqxDCOoNJ06U/Lkm3PNR7aVgWHgc0wrRcGRnqfam1okSIu0nNLlGtGL00nWgBtBNUPdnW/xZxSivKZDdetIB82c/SiGxQ0dcnmkPU/0p3uXElbAqszZTitohsrSt+eaj3LjAyT9K6EY7c/T8KQ+1UTUXNSIvc0WG2OYY6URfNSvJMR1ZXww5qelKLuA4EkdKcnPBpSCOhKT2pre1Zmthv+cVG9aIGN78dKKsgQ880lCGSocpSYqNixUzt5pT0NC3JKsnU0zrW7M76jeO3SjJNBQU3ntUC6i0lXYDrf4vanf7teYbDc/nR19alKzATnFMY7aOW4ypKfnzk9MVEea6o6amckV3x2FMJrbzJ6DTntSr7073JEHvTvlHPFV5ALmlU4NS9FoLqSP93IxUBxmkimHfjT0NJdwW52pp6VCZoG3nNRN6flRHcTUVpYmwgpKEy8rTqz8ixBwabQCuQS1D2oWxD3AUnQUxCUZoQxCaSq21DY7BeD1NK89vrXktX1NqNwaQfX3qo9wsIfaq02atMbG681E56dfzrOpkyH73II/Goj1reNtiOotJ2qrAxe+ar25p7kij1pR96pa7ATofSoXAHeouNbDG4PrTl96u+gyynI6mj+lc/UsT+tRNjn1qojY08mmDr9K1IuL+NNxzQFyaLlad0rIsSlpDIZeV6VXNdK2MxtBoAKQ0LQQ2iqDbU7DFLjjBryLmq0GnI9frSEfPVFbh1OBmoJcsvt7inGOoMpNx2qFufautaIyI24pucHmtIu6FYbQlWK4cetHtTuguL3pc8jSYAGwOKaeaB70vrxSCxYi4FPzzxWBoOc1Ex+WrQbSYzVJkiUUkBJEeDT8VLLFoo2AjP3TV+KtUZsMd+abVCCm9qnUYhp9WtBWO62DOoxUbKc5rxYeZpciOdxH4netrFgT/8AqOTp9SBR1GUJwM+lQNwO66ctDGRATTWYA/OtloRqA9zmlx0x+dO4MTJ/Gjv70XsNgenrSdulMLCjmlHzGpYyfZ7UuysblJDgAPXHvSHoOeagbGMajNbbEidqbTuIO1Jnjg0+gEsXepKyb1NBKU9aLhYb61UcfNWiM2MPXFNz6VqhCA0HrUgGKSqQdTtEk9vrU/wAr814juaEZjwegqJ15xnmtkzQbtPrmmSDmktwKVwuee1VGHauum9DRF+dMI6ZHatQJ79zS/hVANo759KHsUxGaUDNTclCN0oU4YUiluWzkGl96x6GglPSmMhY4NNqXMmMY03OKsQd6T61QyaMqQN+Pvmsrli96U1DWpQ09KryitUZMhNxzzWqAM0VLKSmLqdb1P3hTkOEHy15RqWkDDmpNueema1Q7kDR+tVm9d34GriaLcgl5H0qi6hT9a6oGE1rcZTK2v1IG+tH1rTmAbSgGpl5gLxnNKuaNkCGyD5hTR1quhSLiHcKd2rn2KExTaaGRPUff61qmZSI29abWgIKKAsBOMYpckD5aljJY2yamrGb1NIidjUUvSqRLKvem9q2QkBpKRLFNIKaGjrsfMB/Wg9K8t7myDeM8VMklKSCxMDu46VDIvGR171kviUpBycjiqc612wE9iDv7Uwj6/nXQrIxsj1peMcCp9A3FO7c1QCfSg9ancExG5FRfVpF6C6lqLmp1rGRokNPX3pjH0oiVYiamnzK12MnsJLwjxWokKPSk96m+oxNtN6VVx061aXpmueoVQ1G3Iqhsr96ZWqID8Tik+lYd6XrU9RWO/ipHPy15djWwnHGKM/nVdTS7JI5O5zVndvGM1zz0YiKWH5qpTRA1rCQ79CBov3eFzVRlKg967aeq1M5IFX2pSO1VpcgYOvNJTAD1pOapbAHVai6UITJoTVntWT3HFCVG3HehblkRpufmrSJEthZT04qLHFaMlC9qCPkC+gqUD3FQUkgwfai/vFNEYzU6H9acgiSUh56dKi5Vi45ptaJ3IEAD0p9QuJRVIDqu/WkfaDhPuq8q5uJwT3pcDbk4NWMOvanK/wD9esrCLvnimunU/nWUbFEpsvPtUJjy3AxXQpF8qY0pdSfeonTjNbozsQEc5FNxWrZkP96b/DntUG8e/NMYYNWIDhhVoHIyKma6jiIc9ajbmpWyL+VMatEZU/dpKuVxIafShvvVK2G9x8fBqQ8ioKtdFc0IQGrRkosZo4rJ3NCvLwajrWKM2N70lUAUlIR1WaaPvV5yWh120HdqD2/SkSkLuowCc96laB5AjbTmrMcnmdenpUVI9R7DZY8/SqjDA46iqg7kjZACT6E1WdQDn2rqjsBARTa3i9LGY+mMPmyKnqA3HemSfrVx3JsMHWrMWNvFTMUR5qJjU5bIsUxq2IYUq/dqugDO9FAETI96XocGoLIRzUVOK0IZYibINOqZaFRZFUFaR2JkhtIetAClz69ancTOmJOOoozwK87UKTxT93+0vUNE21sJ7d6TP1poBSaQMc4qSW4pARTJI+MmtGK5VcYb7v6VGw65rdN9AK5GM1BjHFbrUhoccdKT1qg8iMg44pSMZU9qvQzK5zmpI24+tW7BDRlioT9KxiimMNMNakjM0oPFXYBKcOtMCaPpQw3CsNmWMI4qDvitI6ogdE1We1KoOJG/3aq/WiGiEwzTfetPULCsabSj3FY6U+o60mfkyvSvPsdlw+tPDdeKdh7j884Jo4qQtqHfpTW60uoTHRt8ue2KtxHcmKylEyIZkxznjNVTWsNh9CF1qPZ6VspEsjUGtEwtoNYU0NUZkb9cimD5RxVpE3LOcjpmax1K3I6NbIkQ9en5Un481YWFp1IEWOpDxWLL8iJzkVCauJLGDr6VZQ8VVRaCiqq/WlBDGmm15EiUd6YHU/LTGGehrzUbD1p1DRqOByaIGX5aTRpEk9AO/FMYVKJkJbgYFSxyc+wPVGZazuiqnKmCeDURJRVf72aY9dIyI/wAqTjqe1bE7iUxqCfIjbkVF35rWJL3Hxmn0NagrERpO/tTBjW/Sk+lNdiQoXrT8h3J0qWsWWmMNV34q4sUiPrmpY25x2q5EokNV5aiJRGaStCAPWk6VSGdTz0pCD715nU2Vgx27Un/LSlfoVuBAxtNN6NxxmhXLTDdlalzladrFB2pvI6rEk0UgwOo9jU8g3xGsHpLhbmdKuCR7V2rsjqDIqCea1diGBNNNLYgi1qNq1QWGCpxVNEoRsZpo69KQARx1pneqBhnGMUDikxXJokZqwZZCZKiZs1soktjeeaVvVUloST1HjFQjQh+lNrUliUUmI6jv3H43oTyePevO2h2Pmx6UcdutIoVsetMPSmhj3HWlzt68U+W5qh4ag8j5TWfUAUbR04q1E+RzUz1M3uRXKfKSKzpPvVrS1IkyLpRXSIM8UnWkSRsKiYVrEVyOnBqvci+o5lpBUXuO47FQt1ppgMzjtzTuKskkTgUjtWdixNuaTyyBRcVhv0o4zVskkQ8c0knNTsy7kNN61d7ECUlNDOnOcfWgj0P415z2NgBp3481KK5hOtDDI/rQx8wHSmtQrlCd+DgbimWP8AftQrc1m1cmWpYJgI4z6VnXMeCn5QcVVCXLoZMq96b+HNdlzbTqT7jEqNhuSQsOaTPFbbi5Rc5604UnsLHVeXriojojKUVe4h/amMKQEsf3acah6MxXb7xoFWQN5TPrTaGiI9aSmA2kquUR05PNJ268/SvMRq7Cj0o9BhnJpOcc0/IEJ19qXPGcYpbliN069Kbj3/Sqv0CLHRnDYJ2rTs5GaVjRslhkw3P4UXKbl4rNfFYxl3Mv3pldxAh60n1oTAdxTGFBWxC2fSo60j5GQCn96cgJIzioZdue9QNvQZQK0JHrQ/Wo6jHQ8CpKnqWQN1NIOtamfUaeH9qU8UDG96binoIdsNO8nNK4rm/R64rhaNIaAGX1HpincD2pE36AfxzmoaKG5oPPS40BGOlGKVyxtOD+tDVykO/MTVgYZDzWrikUJ48H0qs+a6YO6MWIeaWqEtBw96ShIWqM9K0RIlLVbsQ4GmyVIylrQOpInWiXazASM81LUy3GiGTrTR1rToqD9acvzLS61GdKmi2kUmQyXimO20UkBrpxmjqB1zXI3qbrYBu3e2KcKTJ0uHWkV8p02+1JRui7CDr70YwKNgA+1LnoKQxnekOa0juaDw3ap4zxUzCXcZcrlKoOKqm9DGTGU3rWxIUlNPQTGGozVphcZijNUIKRvqaS1C4lHatNtAQqVLJ92s3uMhzhqmpDRHJTKvoZ9Rx+7SJ0pXKHYzTeUal1Bk4ORVeVsmqSIN4+9HauHY1uN/i+Kp+Uc8UMBeMdPegnjpQmWJx2oPWhjDgUSKW4XFphPzZoLhuwc5qWNqJ7Gu6LGdwrNXaf51FIykiEm8ZxXSZi0jdOhqthEZ96ZmgrjKM+lUA3R2oQCUUxAKnHK1LNCGpEOUvYjqNk6VHLYXUePu0xeDSGS0SDcM0bAyMRUeatISVzo9TQM4rgKHFsCl//AFUFXsIOnNH8qCxP4sUDjsKfKSJ/vUUluWH9KaW45p21HYZUiGiRUWWInFQ3Q4qI/ETIpHrmkPNdRixvAoJ4PNLzAY1NxVpiG0yrJYlHtTAb3pRTYBUqdKiRSGSdaENJLQTFc8VF1FaLYQ9DTW4NTbUY/tSM+KVhMhzSVrsC0OgPymgfpXDKzKsJmlznuahmg6gd6M9qAEpc090DYZpP4KnUIfakIP4U0GNnPNIDjrxWqSAnVuKdLy1n1Kk9Ci3pvtW5gIaU0DIzTT0q0QMJPvVVkFgOabrQkgSE6UVe5O4dqfHUNFIJKQJit0qP61XY8daRxSAUHimvUdQGUlai7cM4yKVsVwZQ0HmnFgBmm4tg9Q+tHHX56VNixevtRnIo2GIeval7UMYnFNORVWDqMNM5qwJFOT1qbOUrL7Re6KsnWoyea1MJbhSdqpDsMPSmnpWgmN60D5amRIsgHUVDVQ2C4Uhq0IUdKBxzUsZJ1FRU0DuBNJVW6gmKKRjSAQU480gIqO9WG5//2Q==

As you can check yourselves, it is 7009 characters long, not divisible by 4. To make it a valid base64 string, I remove the last = of padding. However, when put into the base64->JPEG converter, I still get nothing.
I do not know what could be the issue, and I have been trying to wrap my head around how the ESP32-CAM works and I can't seem to get it right.
I do think that, for my purposes, using RBG565 would be a better choice, but I have not had any luck deciphering how that works either and how I could use it in conjuction with Processing. JPEG just seemed more straightforward, but I am having lots of trouble with it anyway.
Any help will be appreciated, thanks!

Photographer and Arduino, etc., hobbyist here. If my grey matter is working correctly today, jpeg is a compressed format. I don't know the specifics of the compression algorithm, but I bet if you look at that data as a stream, it will not look logical. I would try to work with raw.
On my DSLR, the raw file also contains a lot of data known as exif. It includes stuff like aperture, iso, shutter speed, GPS co-ords, and lots of stuff. Since that will be in the board-level library code, I didn't look further. What you need to check is the format(s) of the data that would typically be written to the SD card. Probably a union of raw data and Exif, maybe more?

I found this in sensor.h

Hi @papertree ,

Welcome to the forum..

try using println instead of write..

you could use write for the raw fb buffer without the commas..

still broken, who’s base64 lib are you using?

I found reference site..

good luck.. ~q

Hi,
Yes, JPEG is a compressed format. I have tried RAW and RAW8, but neither are supported by my sensor (OV...). That is why I mentioned using RGB565, which is a far less compressed format that should fetch me the RGB values directly, but I am having a lot of trouble decoding it.
Thanks!

Either using println() or write, the issue is still how to read that data in Processing?

I am using ESP32's own base64 library.

That reference looks intriguing, I will check it out!
Thanks :)

I thought you were still trying to get a proper jpeg encoded and decoded using the site you mentioned..

don’t know Processing sorry..

~q

Yeah, no, that too. Sorry for the confusion.
The ESP32 base64 library should work fine, no? Is that what you think is causing the issue?

The library has worked before (as in I have gotten an encoded base64 string that could be decoded to an image), but it stopped working without me changing anything...

You do know that there have been breaking changes in the esp32 core don't you? I don't know that it affects you but an easy test is to roll back your esp32 boards to 2.0.17 and try your previously working code. Your path forward at that point should be obvious.
Here is how to do it. NOTE BOARDS not libraries.


I did not know that, no. What are the breaking changes?

I have downgraded but still, no dice.

Almost all manifest themselves as compilation errors. Since your code compiled successfully, that advice likely not applicable.

No worries, there’s lots of libs out there, just wanted eyes on source..

looking some more, all seems good, so probably just that crazy baud rate you’re trying to achieve..

if you reduce to 115200, use println and copy from serial monitor is it still bad??

cam could be loose, re-seat it..

~q

Serial baud rate did it!

I have been trying so many things to get the connection with Processing working that I even forgot that I had changed it. Thanks!

Doing some more debugging before seeing your response, I had noticed that the Serial was printing weirdly, skipping characters... Now I know why!