Converting serial input to keystrokes

Projects General Guidance
1

I need a way to convert serial input from an Arduino Uno through an usb port to keystrokes. This is for a makeshift dance mat I made to play Stepmania on, so the requirements are that I need to be able to press four different buttons- doesn't matter which, any key is fine, as well as mouse buttons, joystick commands etc., anything the game can map. I need to be able to control when the button is pressed and when it is released: this is important; simply pressing and instantly releasing a key isn't good enough, nor is rapidly pressing a key over and over again, I need to be able to simulate holding a key down exactly as if I was to hold it down on a real keyboard.

I know that the Leonardo can act as a keyboard, but I'd rather not have to buy one of those. My computer does not have a ps2 port, so I cant use any ps2 keyboard emulation solutions.

Things I've tried already:

There's a program called AAC Keys which converts serial commands to keystrokes, it works well but is incapable of holding down keys. The same goes for the SerialKeys utility in Windows NT, and anything else that uses the GIDEI protocol.

I've tried Java Robot class in Processing, but it also seems unable to hold a key down. One person says at Silicon Republic: 08/05/10 that robot works great for holding keys down, but the best I was able to get out of it was pressing a key rapidly with a loop, which isn't good enough. There are both keypress and keyrelease functions, but they seem to only work for things like holding down shift so you can capitalize a letter or pressing ctrl-alt-delete, but not actually holding down keys for an extended period of time.

I've read that Gobetwino won't work either, but haven't tried it myself.

There are some promising looking instructions at http://mitchtech.net/arduino-usb-hid-keyboard/ for turning an UNO into a hid device with a firmware update, but I can't get any of the flashing software to work. I tried Atmel Flip, first on an old Xp system, where it simply says "could not open usb device" after I select the device and on a Windows 7 system where it freezes up whenever I try to select the input option. I tried the dfu programmer on a laptop running Linux Mint, but it says that there is no device on the COM. On all three systems I can upload code to the arduino fine, I know the COM is open, I've tried bridging the reset and ground pins before flashing, and I've tried both the UNO R3 SMD (the one with the ATmega16U2) and an older UNO with 8U2, after soldering a 10k resistor to the back per the instructions.

It looks to me like the latter is the best option, but as you can see I'm having no luck with flashing the firmware. Flashing firmware sounds like a somewhat commonplace operation, so I was hoping someone here would have some advice on how I can get that to work, or give me some other ideas on how I can solve this problem.

2

squareballer:
I need a way to convert serial input from an Arduino Uno through an usb port to keystrokes.

Gobetwino lets you send characters, but I don't believe it lets you generate key up / key down events.

Your assumption that you need to deal with the serial stream may be faulty. It's possible to make an Arduino UNO present itself as a keyboard which means you would have total control over the keyboard events it generates. I suspect it would also be possible to make it emulate a mouse using the same technology. You say you've tried that without success. I haven't tried it myself but I read articles explaining how to do it and I'm confident it's possible.

Your last option would be to write an application on the PC which received commands from the Arduino serial channel and used the SendInput() Windows API to generate the HID events locally. The code to do that is relatively simple if you're comfortable coding Windows applications, but very different to anything you'll have seen in Processing or Wiring.

3

Using the Windows API sounds like a good idea. I haven't had any luck with it so far, however. I don't know anything about writing Windows programs but python has a win32api module, so I tried using that. As a test program, I wrote something that executes this line at regular intervals:

win32api.keybd_event(ord('D'), 0, 0 | 0, 1)

which in theory uses the keybd_event function of win32api to press the 'D' key. I also tried it with other keys. What happens is that it does send input to programs like Notepad, I can use it to type, or to capitalize letters by holding down Shift, but it doesn't work for anything else-- I can't use it for key mapping, and programs like Stepmania do not recognize the input. This is a problem I didn't have with any of my other attempted solutions, however, it does seem to be able to hold down keys as I wanted it to. I'm not sure why, maybe its like the difference between, say, copying and pasting a letter vs typing it?

You said I should use the SendInput function, but I can't figure out how it works from the MSDN reference page (I've never used C++). I read that SendInput calls the keybd_event function anyway for typing, and hoped I could use that instead, and since I can't get keybd_event to work I'm a bit skeptical that SendInput would be any better.

I'm going to keep trying to flash the Arduino firmware somehow, maybe ill use different computers etc. I might try taking apart an old usb keyboard if I think I could use that, or perhaps I can find a ps2 to usb converter and write a program to emulate a ps2 keyboard, which is supposedly easier.

4

If you use SendInput then you need to deal with virtual key mappings and key up/down events - it is not as simple as sending a sequence of ascii characters. While the code to implement this is not complicated, it's not trivial and I'm not about to write it for you.

You seem to have ruled out the Arduino as HID approach. Given that you (presumably) already know how to develop and Arduino sketch and given that quite a few people have posted tutorials showing how to use them to emulate an HID device, I would have thought that this was still a viable option.