Trying to convert utf-8 to other charset to use a hebrew liquidCrystal

i have an LCD display with a hebrew character set
i'd like to have a function that converts hardcoded utf-8 string to that charachters
from what i can see it's just a difference of 16 between the code sets
so in utf -8 the char i'd like to convert has cod 144 , the target code is 160
i understand that for utf-8 i need to use wchar and not char
problem is i don't see any strlen functions that handle wchar
so i could something like this.

char* convert(const wchar_t * buffer){
    size_t size=strlen(buffer);
    
    for (size_t i = 1; i < size; i=i+1)
    {
      byte b=buffer[i];
      Serial.println(b);    
      byte result;
      if (b>=144 && b<=170){
         result=b+16;  
      }else{
        result=b;
      }
      
      disp->print(0,0,"%c",(char)result);
      disp->print(0,1,"%d",b);
      delay(1000);
    }
    
}

UTF-8 is a multibyte code with one or more bytes per character. So it's not related to wchar.

For proper decoding of a UTF-8 char you extract the bits of subsequent bytes until the next start code. In Windows there exists a function MultiByteToWideChar() for that purpose, dunno of general C++ support..

Next problem: your LCD display most probably only supports 8 bit characters, i.e. you have to convert the previously obtained wchar to a single char of the display charset. Consult the data sheet of the display or the related driver chip for the supported character codes.

Probably you don't have to deal with UTF-8 but only with two different 8 bit charsets.

actually i've already consulted their data sheet, that's why i know the code ranges from 160 to 187

also i guess i'll have to start using visual studio 2019, getting tired of not having any real debug and refactor abilities in vs code (as a java dev, auto refactor are bread and butter on the ide)

Do you mean UTF-16 for the wchar_t input?

if i understand i need to use wchar for UTF-U as well

I have a LCD library for private usage which converts UTF-8 characters to the character set in the LCD ROM.

I know nothing about Hebrew, but I tried to implement a converter for that display.

https://werner.rothschopf.net/microcontroller/202012_arduino_liquid_crystal_language_en.htm

You can try the example 5036_Convert_Hebrew as a starter.

the last parameter is the converter you handover to the constructor

LiquidCrystal_PCF8574 lcd(addr, cols, rows, convert_SPLC780D1_015A)

if you prefer to write your own converter, that's also possible.

1 Like

nice, it does seem to display hebrew chars, i can use that as a basis, i was about to start going over on how to do debugging in arduino platform io

i just need to handle the RTL stuff..

does your library handle string reversal? that's the thing i've been trying do.
hebrew is a right to left language.
i've been trying to do this, but even trying to debug this in normal console c is proving exceedingly difficult because hebrew doesn't even show up on a console in windows.

that's something what should be done by the LCD driver on HW level.

if you add

  lcd.rightToLeft();  
  lcd.setCursor(15, 0);                 // set cursor to start position

you'll get a right to left.
You have to call this again after a lcd.clear()

oh thanks!, i was sort googling this ,and everyone were saying you need to look right justified, are you saying it's a problem with the LCD?
you might wanna add this as a comment to the samples.
i have a class which acts as an "LCD Wrapper" i'm assuming i'm gonna have to re-implement the coordinates system.
btw is there a way i can have this library in platformio dependency management ?

no, I want to express, that this "right to left" is part of the functionality of the hd44780 compatible functions. It should work with the SPLC780D1 also. I just don't have a SPLC780D1 around currently, only a hd44780.
Can you please give it a try?

/*******************************************************************************
   Support of Cyrillic characters on liquid crystal display
   5036_Convert_Hebrew

   This sketch includes several LCD hardware:
   - Support LCDs with SPLC780D1 015A ROM (sold as "Hebrew" but also contains some Cyrillic) - https://s.click.aliexpress.com/e/_DBc5znh

   You can switch through the examples by pressing enter in the serial monitor.

   by noiasca
   2022-12-26 right to left
   2022-12-22 compiles but no specific LCD available to test

 *******************************************************************************/

#include <NoiascaLiquidCrystal.h>      // download library from https://werner.rothschopf.net/202009_arduino_liquid_crystal_intro.htm 

const uint8_t cols = 16;               // columns/characters per row
const uint8_t rows = 2;                // how many rows
const uint8_t csPin = 10;              // the CS pin for the MCP23S08 IC
const uint8_t addr = 0x3F;             // set the I2C LCD address to 0x3F or 0x27
const uint8_t rs = 8;                  // parallel LCD
const uint8_t en = 9;                  // parallel LCD
const uint8_t d4 = 4;                  // parallel LCD
const uint8_t d5 = 5;                  // parallel LCD
const uint8_t d6 = 6;                  // parallel LCD
const uint8_t d7 = 7;                  // parallel LCD
const uint8_t bl = 10;                 // parallel LCD set to 255 if no backlight is used

//#include <NoiascaHW/lcd_4bit.h>                          // parallel interface, 4bit
//LiquidCrystal_4bit lcd(rs, en, d4, d5, d6, d7, bl, cols, rows, convert_SPLC780D1_015A);  // create lcd object with special converter

#include <Wire.h>
#include <NoiascaHW/lcd_PCF8574.h>                         // I2C
LiquidCrystal_PCF8574 lcd(addr, cols, rows, convert_SPLC780D1_015A); // create lcd object I2C most common addresses are 0x3F or 0x27

//#include <NoiascaHW/lcd_spi.h>                           // SPI MCP23S08 Adapter
//LiquidCrystal_SPI lcd(0x40, csPin, cols, rows, convert_SPLC780D1_015A);  // create lcd object  - with special characters

// the structure of the content information
struct Content
{
  const char lang[3];                  // iso language code
  const char *output1;                 // just a sentence to be displayed
  const char *output2;                 // special characters used in that language (as far as I found them on wikipedia)
};

// Demo content to be printed on the LCD
Content content[] = {
  {"he", "Hebrew 1", "טיךכלםמן"},
  {"he", "Hebrew 2", "אבגדהוזח"},
  {"he", "Hebrew 3", "נסעףפץצקרשת"},
  //{"ru", "Добрый день", "А Б В Г Д"},            // Dobryy den' - hello
  //{"ru", "доброе утро", "Е Ё Ж З И"},            // dobroye utro - good morning
  //{"ru", "Добрый вечер", "Й К Л М Н"},           // Dobryy vecher - good evening
  //{"ua", "доброго вечора", "О П Р С Т"},         // dobroho vechora - good evening
  //{"bu", "добър вечер", "У Ф Х Ц Ч"},            // dobŭr vecher - good evening
  //{"sb", "добро вече", "Ш Щ Ъ Ы Ь"},             // dobro veče - good evening
  //{"mz", "добро попладне", "Э Ю Я"},             // dobro popladne - good evening
  //{"de", "Latin", "ä ö ü ß "},                   // some latin letters
  //{"sm", "Symbols", "µ °C"},
};

// for Hebrew use this instead of a lcd.clear()
void myClear()
{
  lcd.clear();
  lcd.rightToLeft();
  //lcd.autoscroll();
  lcd.setCursor(cols - 1, 0);               // set cursor to start position
}

void setup()
{
  Serial.begin(115200);
  Serial.println(F("+++++++++++++++"));

  //SPI.begin();                       // start SPI if needed
  Wire.begin();                        // start the I2C/Wire Library if needed

  lcd.begin();                         // initialize the lcd
  lcd.backlight();                     // turn on backlight
  lcd.setCursor(0,0);
  showBegin();
}

void loop()
{
  doDemo();
}

void doDemo()
{
  static byte i = 0;
  if (Serial.available() > 0)
  {
    Serial.read();
    Serial.println(content[i].lang);
    lcd.clear(); //myClear();
    lcd.print(content[i].lang);
    lcd.print(": ");
    lcd.print(content[i].output1);
    lcd.rightToLeft();
    lcd.setCursor(cols - 1, 1);
    lcd.print(content[i].output2);
    i++;
    i = i % (sizeof(content) / sizeof(content[0]));
  }
}

void showBegin()
{
  Serial.println(F("convert Hebrew characters"));
  lcd.setCursor(0, 0);
  lcd.print("Hebrew/Cyrillic");
  lcd.setCursor(0, 1);
}

// example print the second row right to left for Hebrew
void showLanguage(size_t i)
{
  Serial.println(content[i].lang);
  lcd.clear();
  lcd.leftToRight();
  lcd.print(content[i].lang);
  lcd.print(": ");
  lcd.print(content[i].output1);
  delay(1000);
  lcd.rightToLeft();
  lcd.setCursor(cols - 1, 1);
  lcd.print(content[i].output2);
}

I'm not working with platformio, hence I don't know what's needed for that.

Further more, is there any chance you could provide me a picture of the LCD showing one row in Hebrew with my sketch for my homepage?

edit: code updated (Latin left to right, Hebrew right to left)

perhaps i have something different..

what is the result from the sketch in #12 after you pressed on or two times the send button?

oh i already did, i had to modify the sample plus re-arranged the method order so it would work with platform.io (and i also included the link to aliexpress display, and modified the address and lcd size.
the project also includes 3 images i too.

NoiascaLiquidCrystalHebrewUsage.zip (1.1 MB)

but in the end it works now?

oh yes, as soon as you said right to left i've applied it. i thought i said that, i was just surprised about the driver part, i'm not familiar with all fine details of the hardware.

fine.
Let's upload one picture to the forum:
lcd_heb_2

you can (and you should) mark one post as solution if you think it's closed now.

Furthermore - if you edit your first post title you can shift the whole thread into "Using Arduino / Displays" which is more appropriate imho.

thanks, now i'm gonna start the never ending of "of why isn't this working, or working as it should, going back to the basics, write and re-write, that will never end :wink: "

how come println gives me extra chinese chars?

#include <Arduino.h>
#include <ezLED.h>
#include <OneButton.h>
#include "Utils.h"
#include <Wire.h>
#include <RTClib.h>
//#include <LiquidCrystal_I2C.h>
 #include <NoiascaLiquidCrystal.h>      // use the library downloaded from https://werner.rothschopf.net/202009_arduino_liquid_crystal_intro.htm
 #include <NoiascaHW/lcd_PCF8574.h>
//  #include <NoiascaHW/lcd_i2c.h>         // I2C Expander

const int RED_LED = 9;
const int BLUE_LED = 10;
const int GREEN_LED = 11;
const int YELLOW_LED = 12;

ezLED red_led(RED_LED);
ezLED blue_led(BLUE_LED);
ezLED green_led(GREEN_LED);
ezLED yellow_led(YELLOW_LED);

const int RED_BUTTUN = 2;
const int GREEN_BUTTUN = 4;
const int BLUE_BUTTUN = 8;

OneButton red_button(RED_BUTTUN, false);
OneButton green_button(GREEN_BUTTUN, false);
OneButton blue_button(BLUE_BUTTUN, false);

const int YELLOW_BUTTUN = 6;
OneButton yellow_button(YELLOW_BUTTUN, false);

LiquidCrystal_PCF8574 _lcd(0x27, 20, 4, convert_SPLC780D1_015A);
RTC_DS3231 _rtc;
void initLCD()
{    
    _lcd.init();
    _lcd.begin();
    _lcd.backlight();
    _lcd.setCursor(0, 0);

    _lcd.clear();
}

void initRTC()
{
    if (!_rtc.begin())
    {
        while (1)
            delay(10);
    }
    if (_rtc.lostPower())
    {
        _rtc.adjust(DateTime(F(__DATE__), F(__TIME__)));
    }
}
void initButLed()
{

    red_button.attachClick([]()
                           {
            debug("RedButton");
            red_led.toggle(); });
    yellow_button.attachClick([]()
                              {
        debug("YellowButton");
        yellow_led.toggle(); });
    green_button.attachClick([]()
                             {
        debug("GreenButton");
        green_led.toggle(); });
    blue_button.attachClick([]()
                            {
        debug("blue");
        blue_led.toggle(); });
}
void setup()
{
    /// debug_init();
    Serial.begin(115200);
    // pinMode(LED_BUILTIN, OUTPUT);

    Wire.begin();
    initLCD();
    initRTC();
    initButLed();
}

void onTick()
{
    red_led.loop();
    red_button.tick();

    yellow_led.loop();
    yellow_button.tick();

    green_led.loop();
    green_button.tick();

    blue_led.loop();
    blue_button.tick();
}

void loop()
{
    delay(1000);
    onTick();

    Serial.println("Hello");
    // _lcd.setCursor(0, 0);
    //_lcd.println("Hello");
    
    char fmt[] = "hh:mm:ss";
    ///_lcd.clear();
    _lcd.setCursor(0, 0);
    
   //_lcd.rightToLeft();
   _lcd.clear();
    _lcd.println("Hello");     
     //_lcd.print(_rtc.now().toString(fmt));
}

additionally calling LeftToRight still flips the word (in english) , and splits it to different lines like so
H
olle