Deriving a modified Keybaord.h / Keyboard.cpp that supports modifier-keys Ctrl & Alt problems to understand what the existing code does

Hi everybody,

I'm working on a project with a USB-keyboard-emulator. German keyboards need to use the modifier-key "AltGr" which is the modifier-combination Ctrl-Alt to create the keypresses for characters like

{ [ ] } \ @ € ~

The standard Keyboard.h / Keyboard.cpp-library is designed for US-Keyboards. For US-keyboards it is sufficient to use the Shift-Key as the modifier-key. The code inside the Keyboard.cpp uses the fact that printable standard-characters are all below value 127 which means in a byte the MSB (Most Significant Bit) is unused for the character-representation.

The author of the Keyboard-library has chosen to use the MSB for the Shift-Key. He defined a constant

#define SHIFT 0x80

which is the value with only the MSB beeing a 1 in binary

87654321
10000000  // in hex $80

In the used ascii-table characters that require a pressed shift-key are encoded through binary "OR"-ing the MSB
example the colon ":"

	0x33|SHIFT,      // :

value 0x33 | 0x80 (aka the constant "SHIFT")

the code that does the transformation from the ASCII-Code to the bytes that must be set for the USB-key-press-"message" decodes the MSB in this way

size_t Keyboard_::press(uint8_t k) 
{
	uint8_t i;
	if (k >= 136) {			// it's a non-printing key (not a modifier)
		k = k - 136;
	} else if (k >= 128) {	// it's a modifier key
		_keyReport.modifiers |= (1<<(k-128));
		k = 0;
	} else {				// it's a printing key
		k = pgm_read_byte(_asciimap + k);
		if (!k) {
			setWriteError();
			return 0;
		}
		if (k & 0x80) {						// it's a capital letter or other character reached with shift
			_keyReport.modifiers |= 0x02;	// the left shift modifier
			k &= 0x7F;
		}
	}

sophisticated code I guess with most compact version of determining the eventually nescessary Shift-key-modifier. To hard to understand for me.

The technique of "OR"-ing the MSB into the ASCII-code does not work if in summary three modifier-keys (Shift, Ctrl, Alt) shall be used.

This means for an ASCII-table that includes keypresses like Ctrl-Alt-q which creates the "@"-symbol on a german keyboard the ASCII-table must be changed to hold the modifier in an extra byte.
For the ESP32 bluetooth-keyboard-library this is done in this way

/* German keyboard (as HID standard) */
#define KEYMAP_SIZE (127)
const KEYMAP keymap[KEYMAP_SIZE] = {
   {0x00, 0 }, /* 0  */
   {0x00, 0 }, /* 1  */
...
   {0x14, KEY_CTRL_ALT }, /* 64   @ */
...

just as a second byte.

I'm struggling on how to modify the existing code of Keyboard.cpp
to setup the USB-keyboard-report-message in the functions

size_t Keyboard_::press(uint8_t k) 

// and

size_t Keyboard_::release(uint8_t k) 

it makes use of a function

		k = pgm_read_byte(_asciimap + k);

I have no idea where to lookup this function

And I have no idea what mechanism makes it possible to use the function "print" on the level of the users code so the demo-code can be

      Keyboard.print("Hello World");

My main goal is to create a derived version of the Keyboard-library
something named like KeyboardGer.h / KeyboardGER.cpp that can be used as a drop-in replacement for the original Keyboard-library.
This shall enable to write code for german keyboards (and shall be used as a starting point for easy adapting the KeyboardGER-library for other non-US-keyboard-layouts like french, italian or whatever else language-specific keyboards.

here is the original code from the file keyboard.cpp

/*
  Keyboard.cpp

  Copyright (c) 2015, Arduino LLC
  Original code (pre-library): Copyright (c) 2011, Peter Barrett

  This library is free software; you can redistribute it and/or
  modify it under the terms of the GNU Lesser General Public
  License as published by the Free Software Foundation; either
  version 2.1 of the License, or (at your option) any later version.

  This library is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
  Lesser General Public License for more details.

  You should have received a copy of the GNU Lesser General Public
  License along with this library; if not, write to the Free Software
  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
*/

#include "Keyboard.h"

#if defined(_USING_HID)

//================================================================================
//================================================================================
//	Keyboard

static const uint8_t _hidReportDescriptor[] PROGMEM = {

  //  Keyboard
    0x05, 0x01,                    // USAGE_PAGE (Generic Desktop)  // 47
    0x09, 0x06,                    // USAGE (Keyboard)
    0xa1, 0x01,                    // COLLECTION (Application)
    0x85, 0x02,                    //   REPORT_ID (2)
    0x05, 0x07,                    //   USAGE_PAGE (Keyboard)
   
  0x19, 0xe0,                    //   USAGE_MINIMUM (Keyboard LeftControl)
    0x29, 0xe7,                    //   USAGE_MAXIMUM (Keyboard Right GUI)
    0x15, 0x00,                    //   LOGICAL_MINIMUM (0)
    0x25, 0x01,                    //   LOGICAL_MAXIMUM (1)
    0x75, 0x01,                    //   REPORT_SIZE (1)
    
  0x95, 0x08,                    //   REPORT_COUNT (8)
    0x81, 0x02,                    //   INPUT (Data,Var,Abs)
    0x95, 0x01,                    //   REPORT_COUNT (1)
    0x75, 0x08,                    //   REPORT_SIZE (8)
    0x81, 0x03,                    //   INPUT (Cnst,Var,Abs)
    
  0x95, 0x06,                    //   REPORT_COUNT (6)
    0x75, 0x08,                    //   REPORT_SIZE (8)
    0x15, 0x00,                    //   LOGICAL_MINIMUM (0)
    0x25, 0x73,                    //   LOGICAL_MAXIMUM (115)
    0x05, 0x07,                    //   USAGE_PAGE (Keyboard)
    
  0x19, 0x00,                    //   USAGE_MINIMUM (Reserved (no event indicated))
    0x29, 0x73,                    //   USAGE_MAXIMUM (Keyboard Application)
    0x81, 0x00,                    //   INPUT (Data,Ary,Abs)
    0xc0,                          // END_COLLECTION
};

Keyboard_::Keyboard_(void) 
{
	static HIDSubDescriptor node(_hidReportDescriptor, sizeof(_hidReportDescriptor));
	HID().AppendDescriptor(&node);
}

void Keyboard_::begin(void)
{
}

void Keyboard_::end(void)
{
}

void Keyboard_::sendReport(KeyReport* keys)
{
	HID().SendReport(2,keys,sizeof(KeyReport));
}

extern
const uint8_t _asciimap[128] PROGMEM;

#define SHIFT 0x80
const uint8_t _asciimap[128] =
{
	0x00,             // NUL
	0x00,             // SOH
	0x00,             // STX
	0x00,             // ETX
	0x00,             // EOT
	0x00,             // ENQ
	0x00,             // ACK  
	0x00,             // BEL
	0x2a,			// BS	Backspace
	0x2b,			// TAB	Tab
	0x28,			// LF	Enter
	0x00,             // VT 
	0x00,             // FF 
	0x00,             // CR 
	0x00,             // SO 
	0x00,             // SI 
	0x00,             // DEL
	0x00,             // DC1
	0x00,             // DC2
	0x00,             // DC3
	0x00,             // DC4
	0x00,             // NAK
	0x00,             // SYN
	0x00,             // ETB
	0x00,             // CAN
	0x00,             // EM 
	0x00,             // SUB
	0x00,             // ESC
	0x00,             // FS 
	0x00,             // GS 
	0x00,             // RS 
	0x00,             // US 
	0x2c,		   //  ' '
	0x1e|SHIFT,	   // !
	0x34|SHIFT,	   // "
	0x20|SHIFT,    // #
	0x21|SHIFT,    // $
	0x22|SHIFT,    // %
	0x24|SHIFT,    // &
	0x34,          // '
	0x26|SHIFT,    // (
	0x27|SHIFT,    // )
	0x25|SHIFT,    // *
	0x2e|SHIFT,    // +
	0x36,          // ,
	0x2d,          // -
	0x37,          // .
	0x38,          // /
	0x27,          // 0
	0x1e,          // 1
	0x1f,          // 2
	0x20,          // 3
	0x21,          // 4
	0x22,          // 5
	0x23,          // 6
	0x24,          // 7
	0x25,          // 8
	0x26,          // 9
	0x33|SHIFT,      // :
	0x33,          // ;
	0x36|SHIFT,      // <
	0x2e,          // =
	0x37|SHIFT,      // >
	0x38|SHIFT,      // ?
	0x1f|SHIFT,      // @
	0x04|SHIFT,      // A
	0x05|SHIFT,      // B
	0x06|SHIFT,      // C
	0x07|SHIFT,      // D
	0x08|SHIFT,      // E
	0x09|SHIFT,      // F
	0x0a|SHIFT,      // G
	0x0b|SHIFT,      // H
	0x0c|SHIFT,      // I
	0x0d|SHIFT,      // J
	0x0e|SHIFT,      // K
	0x0f|SHIFT,      // L
	0x10|SHIFT,      // M
	0x11|SHIFT,      // N
	0x12|SHIFT,      // O
	0x13|SHIFT,      // P
	0x14|SHIFT,      // Q
	0x15|SHIFT,      // R
	0x16|SHIFT,      // S
	0x17|SHIFT,      // T
	0x18|SHIFT,      // U
	0x19|SHIFT,      // V
	0x1a|SHIFT,      // W
	0x1b|SHIFT,      // X
	0x1c|SHIFT,      // Y
	0x1d|SHIFT,      // Z
	0x2f,          // [
	0x31,          // bslash
	0x30,          // ]
	0x23|SHIFT,    // ^
	0x2d|SHIFT,    // _
	0x35,          // `
	0x04,          // a
	0x05,          // b
	0x06,          // c
	0x07,          // d
	0x08,          // e
	0x09,          // f
	0x0a,          // g
	0x0b,          // h
	0x0c,          // i
	0x0d,          // j
	0x0e,          // k
	0x0f,          // l
	0x10,          // m
	0x11,          // n
	0x12,          // o
	0x13,          // p
	0x14,          // q
	0x15,          // r
	0x16,          // s
	0x17,          // t
	0x18,          // u
	0x19,          // v
	0x1a,          // w
	0x1b,          // x
	0x1c,          // y
	0x1d,          // z
	0x2f|SHIFT,    // {
	0x31|SHIFT,    // |
	0x30|SHIFT,    // }
	0x35|SHIFT,    // ~
	0				// DEL
};


uint8_t USBPutChar(uint8_t c);

// press() adds the specified key (printing, non-printing, or modifier)
// to the persistent key report and sends the report.  Because of the way 
// USB HID works, the host acts like the key remains pressed until we 
// call release(), releaseAll(), or otherwise clear the report and resend.
size_t Keyboard_::press(uint8_t k) 
{
	uint8_t i;
	if (k >= 136) {			// it's a non-printing key (not a modifier)
		k = k - 136;
	} else if (k >= 128) {	// it's a modifier key
		_keyReport.modifiers |= (1<<(k-128));
		k = 0;
	} else {				// it's a printing key
		k = pgm_read_byte(_asciimap + k);
		if (!k) {
			setWriteError();
			return 0;
		}
		if (k & 0x80) {						// it's a capital letter or other character reached with shift
			_keyReport.modifiers |= 0x02;	// the left shift modifier
			k &= 0x7F;
		}
	}
	
	// Add k to the key report only if it's not already present
	// and if there is an empty slot.
	if (_keyReport.keys[0] != k && _keyReport.keys[1] != k && 
		_keyReport.keys[2] != k && _keyReport.keys[3] != k &&
		_keyReport.keys[4] != k && _keyReport.keys[5] != k) {
		
		for (i=0; i<6; i++) {
			if (_keyReport.keys[i] == 0x00) {
				_keyReport.keys[i] = k;
				break;
			}
		}
		if (i == 6) {
			setWriteError();
			return 0;
		}	
	}
	sendReport(&_keyReport);
	return 1;
}

// release() takes the specified key out of the persistent key report and
// sends the report.  This tells the OS the key is no longer pressed and that
// it shouldn't be repeated any more.
size_t Keyboard_::release(uint8_t k) 
{
	uint8_t i;
	if (k >= 136) {			// it's a non-printing key (not a modifier)
		k = k - 136;
	} else if (k >= 128) {	// it's a modifier key
		_keyReport.modifiers &= ~(1<<(k-128));
		k = 0;
	} else {				// it's a printing key
		k = pgm_read_byte(_asciimap + k);
		if (!k) {
			return 0;
		}
		if (k & 0x80) {							// it's a capital letter or other character reached with shift
			_keyReport.modifiers &= ~(0x02);	// the left shift modifier
			k &= 0x7F;
		}
	}
	
	// Test the key report to see if k is present.  Clear it if it exists.
	// Check all positions in case the key is present more than once (which it shouldn't be)
	for (i=0; i<6; i++) {
		if (0 != k && _keyReport.keys[i] == k) {
			_keyReport.keys[i] = 0x00;
		}
	}

	sendReport(&_keyReport);
	return 1;
}

void Keyboard_::releaseAll(void)
{
	_keyReport.keys[0] = 0;
	_keyReport.keys[1] = 0;	
	_keyReport.keys[2] = 0;
	_keyReport.keys[3] = 0;	
	_keyReport.keys[4] = 0;
	_keyReport.keys[5] = 0;	
	_keyReport.modifiers = 0;
	sendReport(&_keyReport);
}

size_t Keyboard_::write(uint8_t c)
{
	uint8_t p = press(c);  // Keydown
	release(c);            // Keyup
	return p;              // just return the result of press() since release() almost always returns 1
}

size_t Keyboard_::write(const uint8_t *buffer, size_t size) {
	size_t n = 0;
	while (size--) {
		if (*buffer != '\r') {
			if (write(*buffer)) {
			  n++;
			} else {
			  break;
			}
		}
		buffer++;
	}
	return n;
}

Keyboard_ Keyboard;

#endif

best regards Stefan

P.S. I have found this library KeyboardAzertyFr

But with looking inside this library I found that the author did not modify the code to be able to use the modifiers Ctrl and Alt. The characters are just marked with a comment "Todo"

Here is the documentation for the avr-libc macro:
https://www.nongnu.org/avr-libc/user-manual/group__avr__pgmspace.html#ga349c2c134daa01aeea62256f1fd7489c

Here it is:
https://github.com/arduino-libraries/Keyboard/blob/1.0.2/src/Keyboard.h#L95
By inheriting the Arduino core library's Print class, it is only necessary for the Keyboard library to define the application-specific write function and Print provides all the general-purpose print and println overloads:
https://github.com/arduino/ArduinoCore-API/blob/1.2.0/api/Print.cpp

This is the part that does the SHIFT state.

The lookup table only does the first 128 ASCII characters so the 8th bit of the table is available for the SHIFT flag. What I would do is change the table to use 'uint16_t' instead of uint8_t. Then you can use the entire ISO Latin 1 character set and have 8 bits left over for flags such as SHIFT, CTRL, ALT, etc. Then you would need pgm_read_word() to get each entry from the table.

1 Like

Hi in0,
Hi John,

@in0 thank you very much for pointing me to the doc of pgm_read
ah! pgm program-memory read.

@johnwasser
thank you very much for pointing me to this line of code.

In the meantime I started to transfer the code from the ESP32 bluetooth-keyboard HIDKeyboardTypes.h-library into the keyboard.h-lib.
I will have to think about it what makes more sense to me.
two byte-values or one word-value with bitwise "OR"-ing.

best regards Stefan

For fun, I scraped the Microsoft page layout for the German keyboard. This is what I got. It's missing the control characters (like Tab, CR, LF, Spacebar...) and it doesn't handle the three Deadkey accent keys, but it's a start.

const uint16_t NONE  = 0x0000;
const uint16_t SHIFT = 0x0100;
const uint16_t ALTGR = 0x0200;

const uint16_t kbdgr[] PROGMEM =
{
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 
30 | SHIFT, // Exclamation Mark
31 | SHIFT, // Quotation Mark
40 | NONE, // Number Sign
33 | SHIFT, // Dollar Sign
34 | SHIFT, // Percent Sign
35 | SHIFT, // Ampersand
40 | SHIFT, // Apostrophe
37 | SHIFT, // Left Parenthesis
38 | SHIFT, // Right Parenthesis
48 | SHIFT, // Asterisk
48 | NONE, // Plus Sign
54 | NONE, // Comma
56 | NONE, // Hyphen-Minus
55 | NONE, // Full Stop
36 | SHIFT, // Solidus
39 | NONE, // Digit Zero
30 | NONE, // Digit One
31 | NONE, // Digit Two
32 | NONE, // Digit Three
33 | NONE, // Digit Four
34 | NONE, // Digit Five
35 | NONE, // Digit Six
36 | NONE, // Digit Seven
37 | NONE, // Digit Eight
38 | NONE, // Digit Nine
55 | SHIFT, // Colon
54 | SHIFT, // Semicolon
100 | NONE, // Less-Than Sign
39 | SHIFT, // Equals Sign
100 | SHIFT, // Greater-Than Sign
45 | SHIFT, // Question Mark
20 | ALTGR, // Commercial At
4 | SHIFT, // Latin Capital Letter A
5 | SHIFT, // Latin Capital Letter B
6 | SHIFT, // Latin Capital Letter C
7 | SHIFT, // Latin Capital Letter D
8 | SHIFT, // Latin Capital Letter E
9 | SHIFT, // Latin Capital Letter F
10 | SHIFT, // Latin Capital Letter G
11 | SHIFT, // Latin Capital Letter H
12 | SHIFT, // Latin Capital Letter I
13 | SHIFT, // Latin Capital Letter J
14 | SHIFT, // Latin Capital Letter K
15 | SHIFT, // Latin Capital Letter L
16 | SHIFT, // Latin Capital Letter M
17 | SHIFT, // Latin Capital Letter N
18 | SHIFT, // Latin Capital Letter O
19 | SHIFT, // Latin Capital Letter P
20 | SHIFT, // Latin Capital Letter Q
21 | SHIFT, // Latin Capital Letter R
22 | SHIFT, // Latin Capital Letter S
23 | SHIFT, // Latin Capital Letter T
160 | SHIFT, // Latin Capital Letter U
25 | SHIFT, // Latin Capital Letter V
26 | SHIFT, // Latin Capital Letter W
27 | SHIFT, // Latin Capital Letter X
29 | SHIFT, // Latin Capital Letter Y
28 | SHIFT, // Latin Capital Letter Z
37 | ALTGR, // Left Square Bracket
45 | ALTGR, // Reverse Solidus
38 | ALTGR, // Right Square Bracket
53 | NONE, // Circumflex Accent (DeadKey : ^ â Â ê Ê î Î ô Ô û Û)
56 | SHIFT, // Low Line
46 | SHIFT, // Grave Accent (DeadKey : ` à À è È ì Ì ò Ò ù Ù)
4 | NONE, // Latin Small Letter A
5 | NONE, // Latin Small Letter B
6 | NONE, // Latin Small Letter C
7 | NONE, // Latin Small Letter D
8 | NONE, // Latin Small Letter E
9 | NONE, // Latin Small Letter F
10 | NONE, // Latin Small Letter G
11 | NONE, // Latin Small Letter H
12 | NONE, // Latin Small Letter I
13 | NONE, // Latin Small Letter J
14 | NONE, // Latin Small Letter K
15 | NONE, // Latin Small Letter L
16 | NONE, // Latin Small Letter M
17 | NONE, // Latin Small Letter N
18 | NONE, // Latin Small Letter O
19 | NONE, // Latin Small Letter P
20 | NONE, // Latin Small Letter Q
21 | NONE, // Latin Small Letter R
22 | NONE, // Latin Small Letter S
23 | NONE, // Latin Small Letter T
160 | NONE, // Latin Small Letter U
25 | NONE, // Latin Small Letter V
26 | NONE, // Latin Small Letter W
27 | NONE, // Latin Small Letter X
29 | NONE, // Latin Small Letter Y
28 | NONE, // Latin Small Letter Z
36 | ALTGR, // Left Curly Bracket
100 | ALTGR, // Vertical Line
39 | ALTGR, // Right Curly Bracket
48 | ALTGR, // Tilde
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 
0, 
0, 
0, 
0, 
0, 
0, 
0, 
32 | SHIFT, // Section Sign
0, 
0, 
0, 
0, 
0, 
0, 
0, 
0, 
53 | SHIFT, // Degree Sign
0, 
31 | ALTGR, // Superscript Two
32 | ALTGR, // Superscript Three
46 | NONE, // Acute Accent (DeadKey : ´ á Á é É í Í ó Ó ú Ú ý Ý)
16 | ALTGR, // Micro Sign
0, 
0, 
0, 
0, 
0, 
0, 
0, 
0, 
0, 
0, 
0, 
0, 
0, 
0, 
52 | SHIFT, // Latin Capital Letter A With Diaeresis
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 
51 | SHIFT, // Latin Capital Letter O With Diaeresis
0, 
0, 
0, 
0, 
0, 
47 | SHIFT, // Latin Capital Letter U With Diaeresis
0, 
0, 
45 | NONE, // Latin Small Letter Sharp S (German)
0, 
0, 
0, 
0, 
52 | NONE, // Latin Small Letter A With Diaeresis
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 
51 | NONE, // Latin Small Letter O With Diaeresis
0, 
0, 
0, 
0, 
0, 
47 | NONE, // Latin Small Letter U With Diaeresis
0, 
0, 
0,
};