Bitmap array on SD card to unsigned char array

Hello,

I have a file on an SD card formatted like this:

0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,
0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,

and so on (8000 lines or so).

I'm using a WaveShare 4.2 inch 4 tone greyscale e-paper display. To display images, it required unsigned char arrays in this format:

const unsigned char gImage_cutdown = {
0XFE,0XAF,0XEA,0XFE,0XAF,0XEA,0XFE,0XAF,0XEA,0XFE,0XAF,0XEA,0XFE,0XAF,0XEA,0XFE,
0XAF,0XEA,0XFE,0XAF,0XEA,0X00,0X00,0X00,0X00,0X05,0X55,0XAA,0XAA,0X95,0XA9,0X5A,
...........
}

By having const unsigned char arrays compiled into the code, I can show them on the display. I want be able to load files from my SD card, write the contents to unsigned char arrays and then show the result on the e-paper display.

(I'm hoping that "const" isn't going to trip me up here, but I'll cross that bridge later)

I'm using the standard SPI interface for SD. (FS.h, SD,h, SPI,h). Not sure if it matters, but I'm using an ESP32.
I know the SPI interface is working as I'm able to write the file contents to the serial monitor. I've also been able to make Strings with the contents, but not been able to re-create the arrays.

/* Declares */
unsigned char SDarray[30000]; //Declare unsigned char array with max possible size
String SDstr; // for storing each byte while processing

//Modified SD Card read function. The default function just prints to serial monitor.
void readFile(fs::FS &fs, const char * path){
    Serial.printf("Reading file: %s\n", path);

    File file = fs.open(path);
    if(!file){
        Serial.println("Failed to open file for reading");
        return;
    }
int SDparse = 0; // new integer to manage array creation
    Serial.print("Read from file: ");
    while(file.available()){
        //Serial.write(file.read());
        char c = file.read(); //This is where the major changes start. I record each character into char c
        if (c == ','){ //comma indicates end of a byte has been reached 
          SDarray[SDparse] = SDstr(); //SDstr was fully assembled on previous loop, already contains something like 0xFF
          SDparse++; //this integer incremented so that the next full byte gets written to the next place in the array
          SDstr = ""; //empty the SDstr String ready to create the next one
        }
        else if (c == '\n') {
          continue; //if newline character, ignore and continue on. 
        }
        else
        {
        SDstr.concat(c); //this goes about assembling the individual characters into something like 0xFF or 0xA1
        }
    }
    file.close();
}

Later in the script, the readFile() function is called out:

  readFile(SD, "/octopus.c"); //should put the bitmap array into the unsigned char array: SDarray[]
  Paint_Clear(WHITE);
  Paint_DrawImage(SDarray,16,0,768,300);//This is the Waveshare function that adds the bitmap array to the "paint area" before displaying it
  EPD_4IN2_4GrayDisplay(BlackImage);
  DEV_Delay_ms(10000);

So of course, assuming the way I'm trying to do this isn't complete off, the line giving me trouble is this one:

SDarray[SDparse] = SDstr();

because you can't just put a String into an unsigned char array and expect that to work. I need it to somehow look at the contents of SDstr (eg/ 0x02) and actually interpret that literally(?).

Does anyone know how to do that? Or have a completely different way of doing what I'm trying to achieve? Thanks in advance.

byte image[IMAGE_SIZE];
char buffer[20];
int index = 0;

  while (file.available())
  {
    int count = file.readBytesUntil(buffer, ',');
    buffer[count] = '\0'; // Add null terminator
    image[index++] = strtoul(buffer, 0); // Convert hex constant to binary
  }

Hi,

Thanks for the help.
I tried your code and got a "no matching function for call" error against file.readBytesUntil.
I see lots of help online for serial.readBytesUntil but not for file.readBytesUntil. Do I need a specific library for the function?

Hello

If possible, it would be better to store binary files on the SD card instead of text files, so you just read every bytes of a file and store them in the array directly, no need to watch for , or to convert hex strings, and it will be much faster... :slight_smile:

Class File is a class derived from Stream so the file.readBytesUntil(char *, char) function should work. If you have either parameter type wrong you could get that error. Would you please format the sketch (Tools->Auto Format), copy with the proper tags (Edit -> Copy for Forum), and paste the sketch into a reply here?

Also, when reporting error messages, it helps if you quote them verbatim. Just copy and paste all of the messages from the box below your sketch. Click in the messages, type Ctrl-A / Command-A to select all, Ctrl-C / Command-C to copy the selected text, and Ctrl-V / Command-V to paste them in a reply.

Ok, here is the full code:

// The SD card is using VSPI, the display is using HSPI


/* Includes ------------------------------------------------------------------*/
#include "DEV_Config.h"
#include "EPD.h"
#include "GUI_Paint.h"
#include "imagedata.h"
#include <stdio.h>
#include <stdlib.h>
#include "FS.h" //for SD card
#include "SD.h" //for SD card
#include "SPI.h" //for SD card
//#include "DigiFrame_Functions.h"

/* Declares -------------------------------------------------------------------*/
//Create a new image cache
UBYTE *BlackImage;
/* you have to edit the startup_stm32fxxx.s file and set a big enough heap size */
UWORD Imagesize = ((EPD_4IN2_WIDTH % 8 == 0) ? (EPD_4IN2_WIDTH / 8 ) : (EPD_4IN2_WIDTH / 8 + 1)) * EPD_4IN2_HEIGHT;

unsigned char SDarray[30000];
byte image[30000];
char SDhold;
String SDstr;
char buffer[20];


/* Entry point ----------------------------------------------------------------*/

// Initialize SD card
void initSDCard() {
  if (!SD.begin()) {
    Serial.println("Card Mount Failed");
    return;
  }
  uint8_t cardType = SD.cardType();

  if (cardType == CARD_NONE) {
    Serial.println("No SD card attached");
    return;
  }
  Serial.print("SD Card Type: ");
  if (cardType == CARD_MMC) {
    Serial.println("MMC");
  } else if (cardType == CARD_SD) {
    Serial.println("SDSC");
  } else if (cardType == CARD_SDHC) {
    Serial.println("SDHC");
  } else {
    Serial.println("UNKNOWN");
  }
  uint64_t cardSize = SD.cardSize() / (1024 * 1024);
  Serial.printf("SD Card Size: %lluMB\n", cardSize);
}

//Modified SD Card read function
void readFile(fs::FS &fs, const char * path) {
  Serial.printf("Reading file: %s\n", path);

  File file = fs.open(path);
  if (!file) {
    Serial.println("Failed to open file for reading");
    return;
  }
  int index = 0;
  Serial.print("Read from file: ");
  while (file.available()) {
    //Serial.write(file.read());
    int count = file.readBytesUntil(',', buffer);
    buffer[count] = '\0'; // Add null terminator
    image[index++] = strtoul(buffer, 0); // Convert hex constant to binary


    // char c = file.read();

    /*
      if (c == ','){ //comma indicates end of a byte has been reached
      buffer[count] = '\0'; // Add null terminator
      //SDarray[SDparse] = SDstr();
      //Serial.print(SDstr);
      SDparse++;
      //SDstr = "";
      }
      else if (c == '\n') {
      continue; //if newline character, ignore and continue on.
      }
      else
      {
      //SDstr.concat(c);
      }
      count++;
    */
    //file.read(SDraw, 150000);
  }
  file.close();
  //   Serial.print(SDraw);
  //SDarray = unsigned char(SDstr);
}


void setup()
{
  Serial.begin(115200);
  initSDCard();

  printf("EPD_4IN2_test Demo\r\n");
  DEV_Module_Init();

  printf("e-Paper Init and Clear...\r\n");
  EPD_4IN2_Init();
  EPD_4IN2_Clear();
  DEV_Delay_ms(500);

  if ((BlackImage = (UBYTE *)malloc(Imagesize)) == NULL) {
    printf("Failed to apply for black memory...\r\n");
    while (1);
  }
  printf("Paint_NewImage\r\n");
  Paint_NewImage(BlackImage, EPD_4IN2_WIDTH, EPD_4IN2_HEIGHT, 0, WHITE);
  Paint_SetScale(4); //I don&t understand this but seems to be necessary

#if 1 //Welcome Screen
  Paint_Clear(WHITE);
  Paint_DrawString_EN(32, 140, "DIGIFRAME", &Font20, WHITE, BLACK);//Font 20 each char 14 pxl wide 24char = 336
  Paint_DrawString_EN(116, 170, "(Name Subject to Change)", &Font12, WHITE, BLACK);//Font 12 each char 7 pxl wide 24char = 168
  EPD_4IN2_4GrayDisplay(BlackImage);
  DEV_Delay_ms(8000);
#endif



}

/* The main loop -------------------------------------------------------------*/
void loop()
{
#if 1 //from SD card
  readFile(SD, "/octopus.c");
  Paint_Clear(WHITE);
  Paint_DrawImage(image, 16, 0, 768, 300);
  EPD_4IN2_4GrayDisplay(BlackImage);
  DEV_Delay_ms(10000);
#endif


#if 1 //CutDown
  Paint_Clear(WHITE);
  Paint_DrawImage(gImage_cutdown, 16, 0, 768, 300);
  EPD_4IN2_4GrayDisplay(BlackImage);
  DEV_Delay_ms(10000);
#endif

}

Here is the full error text:

In file included from C:\Users\danie\OneDrive\My Documents\Arduino\GB_Camera_DigiFrame\GB_Camera_DigiFrame.ino:6:
C:\Program Files (x86)\Arduino\hardware\expressif\esp32\libraries\esp32-waveshare-epd\src/EPD.h:1:18: warning: extra tokens at end of #ifndef directive
#ifndef _utility/EPD_H
^
C:\Program Files (x86)\Arduino\hardware\expressif\esp32\libraries\esp32-waveshare-epd\src/EPD.h:2:18: warning: ISO C++11 requires whitespace after the macro name
#define _utility/EPD_H
^
C:\Users\danie\OneDrive\My Documents\Arduino\GB_Camera_DigiFrame\GB_Camera_DigiFrame.ino: In function 'void readFile(fs::FS&, const char*)':
GB_Camera_DigiFrame:70:51: error: no matching function for call to 'fs::File::readBytesUntil(char, char [20])'
int count = file.readBytesUntil(',',buffer);
^
In file included from C:\Program Files (x86)\Arduino\hardware\expressif\esp32\cores\esp32/Arduino.h:160,
from sketch\GB_Camera_DigiFrame.ino.cpp:1:
C:\Program Files (x86)\Arduino\hardware\expressif\esp32\cores\esp32/Stream.h:109:12: note: candidate: 'size_t Stream::readBytesUntil(char, char*, size_t)'
size_t readBytesUntil(char terminator, char *buffer, size_t length); // as readBytes with terminator character
^~~~~~~~~~~~~~
C:\Program Files (x86)\Arduino\hardware\expressif\esp32\cores\esp32/Stream.h:109:12: note: candidate expects 3 arguments, 2 provided
C:\Program Files (x86)\Arduino\hardware\expressif\esp32\cores\esp32/Stream.h:110:12: note: candidate: 'size_t Stream::readBytesUntil(char, uint8_t*, size_t)'
size_t readBytesUntil(char terminator, uint8_t *buffer, size_t length)
^~~~~~~~~~~~~~
C:\Program Files (x86)\Arduino\hardware\expressif\esp32\cores\esp32/Stream.h:110:12: note: candidate expects 3 arguments, 2 provided
GB_Camera_DigiFrame:72:43: error: too few arguments to function 'long unsigned int strtoul(const char*, char**, int)'
image[index++] = strtoul(buffer, 0); // Convert hex constant to binary
^
In file included from c:\program files (x86)\arduino\hardware\expressif\esp32\tools\xtensa-esp32-elf\xtensa-esp32-elf\include\c++\8.4.0\cstdlib:75,
from c:\program files (x86)\arduino\hardware\expressif\esp32\tools\xtensa-esp32-elf\xtensa-esp32-elf\include\c++\8.4.0\stdlib.h:36,
from C:\Program Files (x86)\Arduino\hardware\expressif\esp32/tools/sdk/esp32/include/newlib/platform_include/assert.h:21,
from c:\program files (x86)\arduino\hardware\expressif\esp32\tools\xtensa-esp32-elf\xtensa-esp32-elf\sys-include\sys\reent.h:503,
from C:\Program Files (x86)\Arduino\hardware\expressif\esp32/tools/sdk/esp32/include/newlib/platform_include/sys/reent.h:17,
from c:\program files (x86)\arduino\hardware\expressif\esp32\tools\xtensa-esp32-elf\xtensa-esp32-elf\sys-include\stdio.h:60,
from C:\Program Files (x86)\Arduino\hardware\expressif\esp32\cores\esp32/Arduino.h:27,
from sketch\GB_Camera_DigiFrame.ino.cpp:1:
c:\program files (x86)\arduino\hardware\expressif\esp32\tools\xtensa-esp32-elf\xtensa-esp32-elf\sys-include\stdlib.h:185:15: note: declared here
unsigned long strtoul (const char *__restrict __n, char **__restrict __end_PTR, int __base);
^~~~~~~
Multiple libraries were found for "SD.h"
Used: C:\Program Files (x86)\Arduino\hardware\expressif\esp32\libraries\SD
Not used: C:\Program Files (x86)\Arduino\libraries\SD
exit status 1
no matching function for call to 'fs::File::readBytesUntil(char, char [20])'

Thanks

Hi guix,

That sounds interesting. I'm currently using "Image2LCD" to convert bitmap images to the format that the WaveShare screen likes. If I can somehow convert that data to a binary file, is there a simple function to read that into an unsigned char array?

Yes. file.readBytes(imageBuffer, numberOfBytes);

The solution to this error is given in the error message :

candidate: 'size_t Stream::readBytesUntil(char, char*, size_t)'
size_t readBytesUntil(char terminator, char *buffer, size_t length); // as readBytes with terminator character

So you can use : file.readBytesUntil( ',' , buffer, 20 );


I have tried Image2LCD, you can choose to export binary (.bin) file instead of .c file :slight_smile:

excellent, that seems to have fixed the readbytesuntil issue!

I just had to add a char ptr* in my decalres and then tweak the strtoul line to be like this:

image[index++] = strtoul(buffer, &ptr, 0); // Convert hex constant to binary

now it works perfectly!

I will also have a go with binary files too. thank you both for all the help.