Wearable Computer Interface/3d navigator

Wearable Keyboard/Mouse

It should be possible to make wearable gloves that perform everything that is required to interface with your computer.

The general idea is to use gloves with flex bending sensors, gyroscopes, and accelerometers to detect gestures, keystrokes, mouse pointing, clicking, scrolling, desired 3d movement in games/virtual reality etc.

Keystrokes, for example can be detected when the accelerometers detect a finger striking a hard surface and differentiated by the bend sensors the mechanical/rotation/acceleration dynamics. The position/acceleration data of the hand will be different depending on if a pinky finger or thumb hit something. This, however, multiple gyro/acceleration sensors may be necessary.

During the act of hitting something, the processor should be able to distinguish between the keys depending on the sensor inputs.

Computer Learning

Over time, each persons hand dynamics can, will and should change. The system could be setup so each person gets to completely define static and dynamic input that correlates with each gesture/letter. Doing this might allow one person to use sign language, another to type as with a stenograph, while yet another uses a completely new coding. It should be possible to add word gestures, etc as well.

Mouse

The mouse pointer should be easy if the rotation sensor can be mapped directly to the pointer.

Although tapping the thumb to different fingers should be detectable, for click and drag it may be necessary to add push buttons to the side of the index finger.

If two acceleration/rotation sensors are used, it may be possible to detect when the finger and thumb stop touching.

Finally, as another whole possibility, one could use shoes like the booglie shoes to detect mouse clicks.

Optional Camera/Head Sensor

Adding a camera and/or head sensor may prove necessary or useful. With a screen, this device could become a full computing system. A camera may be necessary for pointing, alignment, or add other new abilities to the device.

New Gestures

New gestures, and in fact, a whole new way of communicating with computers may develop. It might be useful to integrate a 3d tone to the whole development such that people learn to navigate a 3d world through the gloves.

Feedback

The gloves do need a way to communicate back to the user. Sound, light, vibration are all possible. What works best will need to be researched.

The Gloves Themselves

The gloves must be fingerless, preferably with an uncovered palm as well. For initial testing, it may prove useful to find some sun-protecting fingerless gloves.

Initial list of required Gestures

Gloves Activate/Deactivate - It's going to be especially import

Mouse Activate/Deactivate

Keyboard Activate/Deactivate

Forward, Back, Up, Down, Left, Right, Rotate.

Shoot, Enter, Backspace/Undo, Zoom/Unzoom, Scroll.

Modes

As a general rule, the device will be much more intuitive to use with fewer "modes."

Having a scroll mode, typing mode, mousing mode, 3d mode, etc will take users that much longer to learn to use the gloves.

Discovering the natural body movements for each action will make the gloves more intuitive to use. Note, during development of these gloves, we may discover that we need additional sensors, for example, rotation of the head, switches or metal conductors to show when to fingers are touching.

Carpel Tunnel

Carpel Tunnel is a huge risk with these gloves if the gestures/use requires exagerated unusual movement. If the gestures are picked with care, these gloves may reduce uncomfortable movement and actually help.

Wires

Although a prototype may involve tons of wires running everywhere... the actual devices need to each be self-contained. A pair of gloves that require running wires up and down your shirt is just not a possibility. Two gloves, each with their own processor, battery, etc is viable. It may work best, for example to have a phone/device mount on non-dominant glove, while the dominant glove has more sensors like the 3 mouse buttons.

Challenges

Gloves Activate/Deactivate - It's going to be especially important to find some gestures to activate and deactivate the gloves that are not common. It will probably be something easy but unnatural like touching your thumb to the pinky finger.

Detecting unique gestures is going to take some work. It will be especially important to have software that helps users define gestures that are all unique and easily differentiated.

Initial Technologies TODO list:

Glove prototype design/production

Machine learning algorithm

Keyboard/Mouse emulator software - Is it possible to just use a standard BLE keyboard receiver?

3d movement game simulator

Initialization/Training/Tuning Software

This idea was inspired by:

http://vimeo.com/90252137

Anyone have any advice, thoughts, additions?

Thanks

You can eliminate the need for gyro and accelerometers on the gloves if you use motion tracking software. You could then also have hand gestures to do various things like scroll and zoom.

Think of the movie Minority Report and the way they interact with the computer.

Where does the Arduino fit into this?

A good first step would be to build a pair of the gloves based on the ones shown in that video: http://theglovesproject.com/category/diy/