MotionCapture/Input Device - ShadowCoat

I am working on a wearable input device for controlling telerobots, wearable computers or just regular computers.

The basic idea is that all movement of the upper body should be captured. This can then be used for 1 to 1 control of humanoid robots or simply for organic mouse control. Other options would be software which recognises the activity of the person wearing the input tool and reacts accordingly (--> wearable computing).

I have decided to use a electro-mechanical device for the motion capturing. I am using stretch sensors analogue to key muscles of the human body. So when for example, the biceps is flexed thus moving the arm and shortening its length, the corresponding stretch sensor becomes relaxed. By smart sensor positioning the main body movements should be able to be discreetly recorded. (well. to be honest, I am still a bit lost with how to best record shoulder motion, but I'm confident I'll get it done.)

Anyway. The stretch sensors simply change conductivity according to stretch. I use voltage dividers and my arduino mega for reading the analogue signals and then process the signals in processing.

Here, take a look at the "device":

You can find general information on my blog: http://fkeel.blogspot.com/
If you want more detailed information on how to practically use stretch sensors in this type of application check out this post: Building Stuff: Motion Capturing with Stretch Sensor

If you want to see the ShadowCoat in action, I have several videos uploaded to youtube. The two most recent ones are here:

Here I am demonstrating how it could be used to control a curser
http://www.youtube.com/watch?v=qx6TJ0_ZEXk

And here is just a general demo

Anyway thanks for checking out my project - I would really appreciate any feedback, comments, ideas, links to similar projects or anything you can think of

Cheers

p.

edit: ahhh yes. I have posted about this before, but I felt like there has been so much progress and changes in the way I am approaching this that it deserved a new thread. The old thread is here:

http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1285891851

As a new member of the forum (my first post) I was looking for some interesting projects and then I saw your project. I was directly attracted to it because it resembles my project I want to do. I am also convinced that tele-operated humanoid robots have a great potential and can be used for many things. So I just want to say that you must keep up the good work!

My approach to tele-operation of humanoid robots is more mechanical but I like your idea of using stretch sensors. But for now it is all air and ideas (just bought my arduino and sensors/motors) so I was happy when I discovered your project.

share some of your air

I am not 100% happy with my stretch sensor solution - there are lot of little problems which I keep ignoring, hoping they will go away... I have fiddled around with some mechanical solutions but was not really happy with my results (you can find info on this on my blog, if your interested...).

I guess I should stop shying away from inertial measurements ... I'm gonna experiment a bit with accelerometers in the near future...

but yeah: how would you approach this problem? what did you have in mind?

Interesting that you're using sensors strapped directly to your body...

Did you think about potentially using an exoskeleton instead? Similar to the robot things in Avatar I suppose.

Not the best picture, but if you've seen the film you'll get it :-*

Now that wouldn't be cheap ;D

It would be cool to make one though...

Or just have direct control of a robosapien or something from these sensors.

Ok fkeel

I've uploaded a pdf on my webspace (see link below) that should explain the idea behind my setup (did it quickly cause I'm having exams, it's pretty busy). I really believe in it so thats why I will try to build a proof of concept. It has been in my head for years now (tried a lot of different sensing and controlling setups) so I'm looking forward to start. The pdf explains how I will tackle the shoulder joint. All other joints will use the same concept. It is some basic explanation that should make it clear I hope. Once my website is up and running and when I will have some physical results (hopefully after the exams and during vacations) I will surely show them on this forum. Thanks for asking by the way.

http://users.skynet.be/fb040380/TOHR%20concept%20part%20one.pdf

I looked up your NAO project but what are your other mechanical setups you have tried (didn't find them directly) to control it?

@ Funky Diver: in fact my inspiration came from the Halo video games (Avatar gave me a push too). I tried to make a wearable exoskeleton but it was too complicated. A remote controlled system minimizes complexity and puts the human operator out of danger. I know its all big and complicated but you must start somewhere and Arduino gave me a superb opportunity.

@ funky diver
take a look at this here: http://www.animazoo.com/index.php/gypsy-7

the reason why I do not want an exoskeleton is that I want my device to be unobtrusive and subtle. imagine someone wearing a exoskeleton going to a business meeting. might be a bit strange, wouldnt it? instead I want it to be as natural as possible, so that the user doesnt even really feel its there.

@ mowcius
Yeah, thats my basic idea. I plan on using it to control the nao robot Aldebaran | Humanoid and programmable robots its sort of similar to the robosapian.

@fons
wow. you really have put quite some thought into this. anyway your mechanical set up is much more elaborate than anything i have ever tried. i am however exhousted and tired and must say there are parts I dont understand yet.

What material would you use for the connecters to the rotary encoder?
(I am just wondering how massive this would have to be...)

I will have to come back to this at a later point and take a closer look.

thanks for sharing though.

I posted my attempts at a mechanical solution sometime in september last year: Building Stuff: September 2010

(its nothing as elaborate as what you have in mind though, and well... i ended up abandoning it.)

I have been messing round a bit with friendly plastic as well, this may interest you for prototyping material:

Fair does... although I'm intrigued why you would want to control something whilst you're in a business meeting?

Don't get me wrong, I can see an application for it, I just fail to see the application at the present moment for what you're doing right now

I can think of many applications,, but nothing for discreetly controlling a robot or mouse pointer.

Monitoring a persons muscle balance / movement ... yes, I get that. But not this at the moment :-?

@ fkeel: You had some nice results with your mechanical approach (that video you controlling 3 servos is just the same I had in mind for my shoulder joint project). I understand that you would like to keep it 'low-profile'.
My torso vest would consist of a stiff plate in front (chest plate) and a stiff plate on your back (where the mechanical arms would be mounted on) sewn in some rough synthetic textile. The front and rear part are then connected to each other with a shoelace setup. Those mechanical arms would be metal tubes of no more than a centimeter across (where the pivots are bolted on). The arm mount consists of two metal rings that are attached to each other with a metal bar (where you mount the other end of the mechanical arm). It is not very practical for quick usage...

That plastic material is interesting indeed. I can make the stiff parts for the torso vest with this for fast prototyping. It will be trail and error style engineering eheh.

I guess that Animazoo product is what you would like to build?

Cool project and implementation - thanks for sharing.

I have done voice control and touch screen systems and have found I get tired when talking to the computer or using touch screen too much.

I like the usage of an arm twist for action - that could be done while the elbows are still on the chair.

I personally spent a ton of time working out a great system and syntax for building voice controlled automation systems .. and then never the voice control and it is rare that I use touch screen as well due to workload (as odd as that sounds).

You may want to consider the "work out" aspect of the suit (good or bad) when working on your design.