Hello kind person who might read this.
Lately I've tried to tinker with an ESP32-S3 and it's usb HID capabilities. I tried to make a little keyboard. However I struggle on the Arduino IDE to press specific keys ( the modifier keys like Shift, win, Alt, Altgr,Fn) in order to make macros and overall type on the keyboard.
My code uses the ESP32's hardware library package (
C:\Users\User\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.0.3\libraries )
Here is the Arduino's page of this library's modifiers:
This library is great to send simple characters in ASCII ( The ascii table can be found in the Adafruit TinyUSB's libraries in the file "HID.h" )
Here the matrix itself :
#define HID_ASCII_TO_KEYCODE \
{0, 0 }, /* 0x00 Null */ \
{0, 0 }, /* 0x01 */ \
{0, 0 }, /* 0x02 */ \
{0, 0 }, /* 0x03 */ \
{0, 0 }, /* 0x04 */ \
{0, 0 }, /* 0x05 */ \
{0, 0 }, /* 0x06 */ \
{0, 0 }, /* 0x07 */ \
{0, HID_KEY_BACKSPACE }, /* 0x08 Backspace */ \
{0, HID_KEY_TAB }, /* 0x09 Tab */ \
{0, HID_KEY_ENTER }, /* 0x0A Line Feed */ \
{0, 0 }, /* 0x0B */ \
{0, 0 }, /* 0x0C */ \
{0, HID_KEY_ENTER }, /* 0x0D CR */ \
{0, 0 }, /* 0x0E */ \
{0, 0 }, /* 0x0F */ \
{0, 0 }, /* 0x10 */ \
{0, 0 }, /* 0x11 */ \
{0, 0 }, /* 0x12 */ \
{0, 0 }, /* 0x13 */ \
{0, 0 }, /* 0x14 */ \
{0, 0 }, /* 0x15 */ \
{0, 0 }, /* 0x16 */ \
{0, 0 }, /* 0x17 */ \
{0, 0 }, /* 0x18 */ \
{0, 0 }, /* 0x19 */ \
{0, 0 }, /* 0x1A */ \
{0, HID_KEY_ESCAPE }, /* 0x1B Escape */ \
{0, 0 }, /* 0x1C */ \
{0, 0 }, /* 0x1D */ \
{0, 0 }, /* 0x1E */ \
{0, 0 }, /* 0x1F */ \
\
{0, HID_KEY_SPACE }, /* 0x20 */ \
{1, HID_KEY_1 }, /* 0x21 ! */ \
{1, HID_KEY_APOSTROPHE }, /* 0x22 " */ \
{1, HID_KEY_3 }, /* 0x23 # */ \
{1, HID_KEY_4 }, /* 0x24 $ */ \
{1, HID_KEY_5 }, /* 0x25 % */ \
{1, HID_KEY_7 }, /* 0x26 & */ \
{0, HID_KEY_APOSTROPHE }, /* 0x27 ' */ \
{1, HID_KEY_9 }, /* 0x28 ( */ \
{1, HID_KEY_0 }, /* 0x29 ) */ \
{1, HID_KEY_8 }, /* 0x2A * */ \
{1, HID_KEY_EQUAL }, /* 0x2B + */ \
{0, HID_KEY_COMMA }, /* 0x2C , */ \
{0, HID_KEY_MINUS }, /* 0x2D - */ \
{0, HID_KEY_PERIOD }, /* 0x2E . */ \
{0, HID_KEY_SLASH }, /* 0x2F / */ \
{0, HID_KEY_0 }, /* 0x30 0 */ \
{0, HID_KEY_1 }, /* 0x31 1 */ \
{0, HID_KEY_2 }, /* 0x32 2 */ \
{0, HID_KEY_3 }, /* 0x33 3 */ \
{0, HID_KEY_4 }, /* 0x34 4 */ \
{0, HID_KEY_5 }, /* 0x35 5 */ \
{0, HID_KEY_6 }, /* 0x36 6 */ \
{0, HID_KEY_7 }, /* 0x37 7 */ \
{0, HID_KEY_8 }, /* 0x38 8 */ \
{0, HID_KEY_9 }, /* 0x39 9 */ \
{1, HID_KEY_SEMICOLON }, /* 0x3A : */ \
{0, HID_KEY_SEMICOLON }, /* 0x3B ; */ \
{1, HID_KEY_COMMA }, /* 0x3C < */ \
{0, HID_KEY_EQUAL }, /* 0x3D = */ \
{1, HID_KEY_PERIOD }, /* 0x3E > */ \
{1, HID_KEY_SLASH }, /* 0x3F ? */ \
\
{1, HID_KEY_2 }, /* 0x40 @ */ \
{1, HID_KEY_A }, /* 0x41 A */ \
{1, HID_KEY_B }, /* 0x42 B */ \
{1, HID_KEY_C }, /* 0x43 C */ \
{1, HID_KEY_D }, /* 0x44 D */ \
{1, HID_KEY_E }, /* 0x45 E */ \
{1, HID_KEY_F }, /* 0x46 F */ \
{1, HID_KEY_G }, /* 0x47 G */ \
{1, HID_KEY_H }, /* 0x48 H */ \
{1, HID_KEY_I }, /* 0x49 I */ \
{1, HID_KEY_J }, /* 0x4A J */ \
{1, HID_KEY_K }, /* 0x4B K */ \
{1, HID_KEY_L }, /* 0x4C L */ \
{1, HID_KEY_M }, /* 0x4D M */ \
{1, HID_KEY_N }, /* 0x4E N */ \
{1, HID_KEY_O }, /* 0x4F O */ \
{1, HID_KEY_P }, /* 0x50 P */ \
{1, HID_KEY_Q }, /* 0x51 Q */ \
{1, HID_KEY_R }, /* 0x52 R */ \
{1, HID_KEY_S }, /* 0x53 S */ \
{1, HID_KEY_T }, /* 0x55 T */ \
{1, HID_KEY_U }, /* 0x55 U */ \
{1, HID_KEY_V }, /* 0x56 V */ \
{1, HID_KEY_W }, /* 0x57 W */ \
{1, HID_KEY_X }, /* 0x58 X */ \
{1, HID_KEY_Y }, /* 0x59 Y */ \
{1, HID_KEY_Z }, /* 0x5A Z */ \
{0, HID_KEY_BRACKET_LEFT }, /* 0x5B [ */ \
{0, HID_KEY_BACKSLASH }, /* 0x5C '\' */ \
{0, HID_KEY_BRACKET_RIGHT }, /* 0x5D ] */ \
{1, HID_KEY_6 }, /* 0x5E ^ */ \
{1, HID_KEY_MINUS }, /* 0x5F _ */ \
\
{0, HID_KEY_GRAVE }, /* 0x60`
*/ \
{0, HID_KEY_A }, /* 0x61 a */ \
{0, HID_KEY_B }, /* 0x62 b */ \
{0, HID_KEY_C }, /* 0x63 c */ \
{0, HID_KEY_D }, /* 0x66 d */ \
{0, HID_KEY_E }, /* 0x65 e */ \
{0, HID_KEY_F }, /* 0x66 f */ \
{0, HID_KEY_G }, /* 0x67 g */ \
{0, HID_KEY_H }, /* 0x68 h */ \
{0, HID_KEY_I }, /* 0x69 i */ \
{0, HID_KEY_J }, /* 0x6A j */ \
{0, HID_KEY_K }, /* 0x6B k */ \
{0, HID_KEY_L }, /* 0x6C l */ \
{0, HID_KEY_M }, /* 0x6D m */ \
{0, HID_KEY_N }, /* 0x6E n */ \
{0, HID_KEY_O }, /* 0x6F o */ \
{0, HID_KEY_P }, /* 0x70 p */ \
{0, HID_KEY_Q }, /* 0x71 q */ \
{0, HID_KEY_R }, /* 0x72 r */ \
{0, HID_KEY_S }, /* 0x73 s */ \
{0, HID_KEY_T }, /* 0x75 t */ \
{0, HID_KEY_U }, /* 0x75 u */ \
{0, HID_KEY_V }, /* 0x76 v */ \
{0, HID_KEY_W }, /* 0x77 w */ \
{0, HID_KEY_X }, /* 0x78 x */ \
{0, HID_KEY_Y }, /* 0x79 y */ \
{0, HID_KEY_Z }, /* 0x7A z */ \
{1, HID_KEY_BRACKET_LEFT }, /* 0x7B { */ \
{1, HID_KEY_BACKSLASH }, /* 0x7C | */ \
{1, HID_KEY_BRACKET_RIGHT }, /* 0x7D } */ \
{1, HID_KEY_GRAVE }, /* 0x7E ~ */ \
{0, HID_KEY_DELETE } /* 0x7F Delete */ \
This converts an ASCII hex keycode like 0x61 into HID_Key_A with the 0 in front as the modifier key in action
If I understand well, When I send the command Keyboard.press(0x61) , the USBHIDKeyboard library goes to this, and sees what is the equivalence to "0x61" which is "{0,HID_KEY_A}"
HID_Key_A corresponds to the HID Keycode "0x04"
Then, the method press() here in Keyboard.cpp
size_t Keyboard_::press(uint8_t k)
{
uint8_t i;
if (k >= 136) { // it's a non-printing key (not a modifier)
k = k - 136;
} else if (k >= 128) { // it's a modifier key
_keyReport.modifiers |= (1<<(k-128));
k = 0;
} else { // it's a printing key
k = pgm_read_byte(_asciimap + k);
if (!k) {
setWriteError();
return 0;
}
if ((k & ALT_GR) == ALT_GR) {
_keyReport.modifiers |= 0x40; // AltGr = right Alt
k &= 0x3F;
} else if ((k & SHIFT) == SHIFT) {
_keyReport.modifiers |= 0x02; // the left shift modifier
k &= 0x7F;
}
if (k == ISO_REPLACEMENT) {
k = ISO_KEY;
}
}
// Add k to the key report only if it's not already present
// and if there is an empty slot.
if (_keyReport.keys[0] != k && _keyReport.keys[1] != k &&
_keyReport.keys[2] != k && _keyReport.keys[3] != k &&
_keyReport.keys[4] != k && _keyReport.keys[5] != k) {
for (i=0; i<6; i++) {
if (_keyReport.keys[i] == 0x00) {
_keyReport.keys[i] = k;
break;
}
}
if (i == 6) {
setWriteError();
return 0;
}
}
sendReport(&_keyReport);
return 1;
}
will add the keycode with the adequate modifier to the next keyReport, in order to send it to the computer.
This code works for everything, EXCEPT what I want: the modifier keys.
If I try to use the modifier key KEY_LEFT_SHIFT which is defined by 0x81, the press isn't actually registered properly.
Example of my code ( cropped from the setup():
Keyboard.press(0x81); // press shift
Keyboard.press(0x61); //press the "a" key
delay(50);
Keyboard.release(0x61); // release the "a"key
delay(100);
Keyboard.release(0x81); //release the shift key
However, what this does is
- on boot-> it presses on the "shift" key once and locks it in "pressed"
- each time, it presses the 0x61 key , no problem.
- releases the 0x61 key , all good
- doesn't release the shift key, it stays locked for the rest of the program (EDIT :However I can deactivate it with a press on my real keyboard)
- refuses to press the shift key anymore on any loop() iteration
0x61 still is pressed each time, that's not a problem, it's just the modifier key that has it's own mind.
THINGS TO NOTE:
- If I release the shift key BEFORE releasing the 0x61 key, it works almost like intended ( presses shift each time) , HOWEVER like that I'm unable to press the shift key longer than the 0x61 key.
- If I use the method releaseAll() , it also releases the Shift key, but then everything is released at the same time. Still not what I want.
How could I make my program in order to let it press the shift key, press and release a bunch of other keys, then release my shift key ? (without caplock as a solution)
Also if possible, I would like to use the raw keycodes for usb-HID ( like 0x04 for 'a', 0x05 for 'b') instead of an ASCII conversion that takes some added time to process (like 0x61 for 'a', 0x62 for 'b').
Because yet, the program converts from ASCII to a HID keycode , which is used in a macro , then sends it to the computer. That's really not efficient.
Sorry if my english is quite poor, It's not my mother tongue.
Thank you in advance if you can help me 