Hey! I'm working on a project where I want to basically control the rotation of a sphere using my hand. Sort of like Minority Report meets Google Earth. However the project is turning out to be seem more complicated than I originally thought so I wanted to see what help I could get.
Originally I was thinking of using the ez430 Chronos Wristwatch from TI and using its accelerometer to control my sphere. But the more I got into that idea the more I realized how hard this must be.
I thought I could track a hand's position just by using an accelerometer but I've heard that in order to get actual position data you have to take a double integral of the accelerometer data which causes errors (drift) to come up exponentially over time.
Then I read about using an IMU which pairs a gyro with an accelerometer to get more accurate readings, but is that overkill?
Essentially I'd like to create the illusion for a user that when they move their hand around a spherical model would spin around, like a virtual 'globe' that you can physically manipulate to see different sides of. Does anyone have some good ideas for an electronics interface that can help me do this?
It certainly isn't easy. Are you just wanting to measure the orientation of the hand, or the position as well?
It is definitely possible to get yaw/pitch/roll information using the sensors you mentioned, of course, but you have likely found that the math/algorithms to integrate the readings properly is fairly daunting. Get used to it; none of the methods for what you are looking to achieve are simple, math-wise...
You could try a "waldo" device - many such systems have been made since the 1950s or earlier, to basically measure the position of all the joints of the arm and hand, in order to guide remote controlled arms for various uses (nuclear research and maintenance was a big one). Back in the 1960s, Ralph Mosher, working with General Electric, developed a device called the "GE Handyman":
You might also know of him better for the Hardiman suit (also developed at GE):
Basically, you build an exoskeleton to measure joint angles (via any number of methods - the easiest for the hobbyist would likely be using potentiometers and/or resistive flex sensors; digital encoder devices might also be a possibility), then translate those angles (using inverse kinematics) into position and orientation information. This forum member is doing something real similar to this (to control a humanoid robot is the ultimate goal):
You might also want to check out similar devices and designs used in motion capture rigs, as well as virtual reality research; they all follow the same idea as presented, though (though different kinds of sensors may be used, of course).
Another method can be done with ultrasonics, which is how the Nintendo PowerGlove and a old Logitech headtracker system worked. Optics can also be used - similar to the wiimote...
An advanced method, though, is magnetic tracking - these are systems that use pulsed magnetics to determine position and orientation of a sensor. Two companies pretty much "locked up" the market in the 1980s and 1990s: Polhemus and Ascension:
Don't bother asking about prices - you likely can't afford these systems (but then again, who am I to question your income - maybe you are wealthy?). Fortunately, there exists on the market (well, in beta currently) a solution that seem similar:
Oh - one other device worth mentioning: Sony PSMove...
Well I'm only looking for position information and definitely am not looking to build an exoskeleton arm for this This system will be used in public spaces to let random strangers control imagery on a planetarium, so the system needs to be very simple and have a low footprint; telling little Johnny to strap a scary metal exoskeleton to his arm is probably not a great idea, lol.
Here's the thing that gets me; although deriving explicit positional data from an accelerometer is a bad idea I have seen several projects where an accelerometer is used to control a 2D mouse with good accuracy. Trouble is I haven't found the source code to actually do this sort of thing, though my hunch involves inferring positional data using either acceleration or velocity data from the accelerometer, smoothing the data out using a Kalman filter and then modifying the X-Y coordinates of the mouse using that data.
Is THAT feasible? I think even if its impossible I'm likely still going to have to try to get an accelerometer to work for this purpose as I've proposed this as an undergraduate research project for my university...kinda nailed my own coffin there, lol.
Trouble is I haven't found the source code to actually do this sort of thing, though my hunch involves inferring positional data using either acceleration or velocity data from the accelerometer, smoothing the data out using a Kalman filter and then modifying the X-Y coordinates of the mouse using that data.
Something like that could work; the only problem is that you are only getting acceleration data, and not true x/y positional values - I imagine that error might accumulate quickly, but it is certainly worth trying. I can't find the link, but I remember running across a company that was selling 3D position tracking sensors that connected to a PC via USB (and sold for $99.00 US) - they seemed to do something like you're describing, so it is certainly possible (but finding an example that works the same might not be - for the reason that it is a very marketable product).
One method you might try, though not wireless, is to feed a self-retracting line through a hollow shaft/handle analog joystick (using gimbaled x/y potentiometers). On the axle of the line retraction mechanism, you can mount a multi-turn potentiometer (maybe with gear reduction), or a quadrature-encoder device. Then, by using these three sensors and a bit of geometry/trig math, you can calculate the 3D position of the end of the line on the retraction mechanism; mount a small ball on the end of the line to grab.
Alternatively, you could also mount the end of the line to your wrist, and then attach a joystick handle, or a "wired glove"...
You might also be able to get yaw/pitch/roll info from a device like this one:
Something to know about the device I described - there was a company that at one time sold such a device to hook up to the PC; I doubt that there is a current patent on it, but it might be worth searching for if you are concerned. I don't know if it would be an issue for the project you are planning to do or not. For personal use, it shouldn't be an issue, and you might come up with improvements (as long as you're not selling the item, it shouldn't be a problem - but then again, I am not a patent attorney!).
I remember seeing a video by someone who basically did this kind of thing very much on the cheap using a disassembled good old-fashioned ball mouse attaching sliders across the mouse rollers.
Accelerometer + Gyro to sense movement
Ultrasonic to sense distance
User stands in front of sensor about 5 feet away, hands at side. When the reach out in front the sensor "sees" their hand and they are now "touching" the sphere. If they move right/left/up/down the sphere moves. When the retract their hand they are no longer "touching" the sphere. Movements of the hand are ignored.
Not sure if the sensor in front of the person is sensitive enough to see the outstretched hand as opposed to the body so maybe the sensor off to the side to create a horizontal plane would be better. Or maybe a laser to the side to create a more accurate plane.
How about going for one of those wireless Wii nunchuck controllers, and using the nunchuck interfacing code? You've definately got your accelerometer and wireless taken care of..
as for location, I've always wondered if one could put a rotating photosensor on each axis and use an IR emitter.. intersection of the angles of highest reading gives location.. but I wonder about how accurate that might be. On the same idea, you could also use ultrasonic and measure delay, giving relative poisition.
the last of your posts would be do-able by simply placing a row of photodetectors and a light opposite, and simply use the shadow of the hand on the photodetectors. One row x, one column y, and if you want to be fancy, one more for a z axis. Again, positional accuracy not as important as detecting relative motion...
Location is painful, but the wiichuck should be good for the rest...
also, is absolute position even required? If using accelerometer, wouldn''t RELATIVE (aka motion) position be the only important factor? Your hand motion spins the globe, but the motion is the important part, not where you made that motion. Accelerometer can handle the motion detection and therefore relative position pretty nicely.
Ha I found it! This guy is really amazing. VR hand sensing using rollers from a ball mouse after about 2 mins. But watch the whole video. Lots of good stuff.
Hmm, now this system is getting a bit complex. I will continue thinking about using different sensors, but I really had my heart set on accelerometers; I mean its obviously been proven that one can make good use of an accelerometer. You can play games on the Wii in which you can control objects in 3D like I'm attempting to do, and the Chronos wristwatch comes with sample software to control a 2D mouse.
Just a few minutes ago I had a flash insight that I feel pretty good about. Newton's Second Law of Motion: F=ma! I can use the accelerometer data directly to calculate 3D force vectors, then apply those vectors to an on-screen object, like my sphere
as for location, I've always wondered if one could put a rotating photosensor on each axis and use an IR emitter.. intersection of the angles of highest reading gives location..
Actually, something like this was done waaay back in the 1960s by Ivan Sutherland, who created the "Sword of Damocles" early VR/AR system:
If you do the research, you'll find a tantalizing reference to a 3D position sensing system he came up with (but I have yet to find little else other than the singular reference - and I can't even find that, anymore!) which used (IIRC) a photodetector mounted on a wand or such, and a bright light which was mounted behind a disc with a rotating slit (powered by a motor). Three of these were mounted orthogonally (one per "wall"), and the angle of rotation was monitored; where the three "beams" of light hit the sensor on the wand, the 3D position reading was calculated.
but I wonder about how accurate that might be.
I tend to doubt it would be very accurate, but it may not be all that bad - perhaps with laser line projectors, you might be able to get centimeter-level positional accuracy (with any other light source, though, I would expect better than 5cm).
I honestly don't know how accurate Sutherland's system was, as I haven't found anything about it other than that single small reference to it. I suspect that if you could physically view the archives at whatever university he was at during that time (whether Utah or MIT), you would probably find it.
Or - if you could get in contact with him - he may remember something about it (good luck finding a working email address - I have yet to find one - that, or he might use a whitelist).
I've decided to see if I can track down Sutherland's system (I also sent a request to Oracle for assistance in contacting him - we'll see where that goes!) - here are links I found:
(click the "pdf" link on the right to download a PDF of the article)
Now - the interesting thing - this paper was titled "Twinkle box: a three-dimensional computer input device" - and was authored by Robert P. Burton, and Ivan E. Sutherland; Sutherland was secondary on there, and a lot of what I have seen seems to give Burton the credit for the invention, so I think I was wrong there...?
Anyhow - here's a link to Burton's bio page (and if you scroll down toward the bottom, you'll see info on the above paper, too):
If you mounted two at right angles, you could get 3D tracking.
The hand itself probably doesn't have enough contrast with the background to be tracked. But you could do something really cheap and unobtrusive, like, say, modifying one of those dollar-store keychain flashlights by replacing the LED with an IR emitter. Or maybe one of the novelty flashlights that's worn as a ring. That would give you a clear point source that should track well, doesn't scare the bejeebers out of people like a complex harness, and won't give you a heart attack if someone drops or steals it ;D
Like I said I think I am going to go with a ez430 Chronos wristwatch and use its embedded accelerometer to influence the position of an object floating in a 3D sphere. The accelerometer data will be plugged into Newton's 2nd law (F=ma) and that force will then be applied to a floating object.
I don't have the money or time to try those other more elaborate ideas
This system will be used in public spaces to let random strangers control imagery on a planetarium, so the system needs to be very simple and have a low footprint; telling little Johnny to strap a scary metal exoskeleton to his arm is probably not a great idea, lol.