Ambitious Project: A Talking House

Okay, so I haven’t had much sleep in the past couple days so any sort of grammatical errors within this post is probably due to pure exhaustion. I apologize. Also, the idea just came to me this morning so things may be a bit… fragmented (I apologize for that as well).

Okay, so I want a talking house.Well, it’s going to be a talking apartment really but same similarity. The different ideas I’ve come up with utilizes a variety of projects so I wasn’t really certain where to post my plans so I decided to just go to you guys.

Here’s the jist of my proposal:

What I want is a series of automated messages that are released from my computer upon a variety of events. For example, say I walk through the main door and trip an IR beam prompting a message akin to “Welcome to the bridge captain. How may I serve you.” I’ve thought of different ways to accomplish this but I’m hoping for some kind of efficient way of accomplishing this from the Arduino community. Maybe use RFID tags instead of an IR beam? I was thinking of using ZigBee and an Arduino to transmit the message to my computer and then have the computer use a TTS application like Festival to speak-a my language. Perhaps I can have a script that plays a pre-recorded audio file?

That’s just the first part though, I want to speak to my machine also. I run Linux (Ubuntu) and I hope to use a voice recognition application (julius) to satisfy my needs. Also, does anyone know a way for me to use multiple microphones to input audio into my Linux box? Since I will be speaking aloud at a good distance (~10-15 ft.) away from my computer I am thinking about running a second microphone across the room and hiding the wire under the rug. Maybe there are ‘industrial’ microphones that can pick up sound better then your average desktop computer mics? (I’ll have to look into that ::slight_smile: )

Basically, what I want to implement is an elaborate speech recognition ‘hotkey’ system that just spits out verbal confirmations. If successful, I want to continue building upon the system (adding the 'Sudo make me a sandwich robot machine, having the computer sing me a song etc…). Maybe I’m just being silly, but having a real-life HAL welcoming me to my ‘Stardeck’ sounds uber-cool!

Also important to note, I have very little experience working with electronics and hardware; more of a programmer really (and a shoddy one at that). Being eighteen my skill-set isn’t exactly what you would call ‘flushed-out’ so, please, speak to me like an idiot so I can understand what is being said. I’m hoping that this project can be broken down into steps that will teach me about Arduino and other various projects.

Oh, and another thing – this project will probably take a long time to do and there will be lots of learning involved. If you know of any, may I please get some recommendations on some books that pertain to the subject at hand? I would be much obliged.

Thanks for reading through this massive post and I would appreciate any sort of responses.

[Edit: I would be willing to use Windows to accomplish my goals if needed since I’m probably going to be using a dedicated box anyways. Also, I guess you could say I’m trying to emulate an A.I here. Sort of like the computer chick from Portal, HAL, or the computer home chick from Eureka.]

The bonus of using linux is, you could write your own hardware drivers.
This could potentially solve some issues for you… in regards to the multiple microphones for instance, having the driver sync their signals.

The bonus of using linux is, you could write your own hardware drivers.
This could potentially solve some issues for you… in regards to the multiple microphones for instance, having the driver sync their signals.

Hmm, sounds a bit tedious and very time consuming but doable in the long run granted my ambition doesn’t sputter out by then. At the moment I’m looking into the Dragon NaturallySpeaking product line It looks promising as far as audio input and it’s probably more mature for audio related input then what GNU/Linux can offer (as far as I know). And I think I’m going to have pre-recorded phrases instead of TTS (if I can find someone to voice it for me). I don’t think I’m going to go with Linux unfortunately. I just found out that microphone arrays are already supported in Windows. Everything I need is already provided in a Windows environment.

Thanks for your reply.

I love the idea! I don’t think I can be much help, other than to say don’t knock your skillset for being 18. I don’t think I’ll ever be as good a coder as I was when I was 18 and I certainly don’t have the imagination I did.

You could use passive infrared (PIR) sensors to track which room you’re in. It only detects movements but you have limited entry and exit points to each room. If you stop moving then it knows which room you’re in because you haven’t gone into an adjacent room (but you’d need sensors on exits which you probably want anyway).

Then, when it knows which room you’re in it only has to monitor that room. Therefore, you don’t need your PC to monitor a mic per room - you could have the arduino circuit switch (using relays) so that there is only ever one active microphone.

I think PC mics are at the lowest end of the market, they’re dirt cheap so probably far from idea for what you want. You can pay a bucketload of cash for them, so maybe speak to someone who knows their stuff about them or see if you can borrow a couple to test out.

Edit: You have to have HAL saying “I’m sorry Dave, I’m afraid I can’t do that” for your error response.

If I were going to do something like this for an apartment, I would actually seriously consider something like X-10; as in, doing it over the powerline, but in a more robust way than X-10. You would have modules that would plug into the wall, to control lamps and such, but also have the microphone built in, and perhaps even motion sensors.

But all of that is likely beyond your capabilities at the moment (for that matter, there would be a learning curve for me as well!).

If you don’t mind having wires running everywhere, perhaps you could do everything over daisy-chained Cat5e cable, run around the perimeter of the apartment? In all the apartments I’ve lived in, it wasn’t too hard to pull the carpet up off the edge of the nailboard that is next to the floor molding, and tuck it under there. Whatever you do, you -don’t- want the cabling to run under traffic areas, because it could abrade and potentially start a fire (and invalidate your renter’s insurance - you do have renter’s insurance, right?).

The biggest issue (well, besides the software) will be the microphones - multiple microphones, run into a small mixer console (to mix down to input to the PC), are going to have multipath echo and such that you would have to somehow filter out, so keep that in mind.

For commanding the system with voice recognition, make sure you have a command word to “wake up” the system, and have it “acknowledge” that it is ready for the command (like on ST, the command was something like “COMPUTER!”, then it would give a little squiggle-beep to signify it was ready, then you could speak the command).

Finally - good luck with the project; if you get something working even half-way decently, you could probably gain a market for installation in rich geek homes (?) or something. A project like this is a big thing, and successes will be hard won, but the technology is certainly - well, almost - there. The hardest part will be the voice recognition to word translation to command parsing - researchers have been working on these kinds of systems for decades…


Thank you cr0sh and cojam for your replies.

I would have to say that the X-10 would also be too expensive for a poor college student. A lot of the ‘extra’ features will only be addressed once I get a simple recognition system going. But, here is another question: What about multiple people in the room? That’s why I was thinking RFID tags – so I could have a unique identifier for myself. Is there a way to get RFID tags and passive IR beams to work in tandem? Who knows, maybe I would be able to set up a mock ‘security system’ where if someone other then me enters the apartment then a little robot will shoot marshmallows at them or something.

[Edit: The speech recognition and command parsing would be handled by the Dragon software (it’s in the link I posted a couple posts above this one) and it seems more then capable for handling commands.]

the computer home chick from Eureka

You’d better be sure you stick to emulating an A.I.: if it really becomes self-aware, it might lock you in and force you to watch sensitivity-training videos. ;D

One question that comes to mind is how many rooms the PC needs to “listen” in: it sounds like you currently have a very small, perhaps even one-room, apartment. The best microphone setup for your home would be very different from the best one for mine, where normal conversation levels at one end of the house are completely inaudible at the other. A system that works for you now could fail badly if you moved to a place with a different size/shape.


Also, I guess you could say I’m trying to emulate an A.I here

I mentioned the emulation part…

I’m moving to an apartment very similar to the one I’m staying in now and the rooms will roughly be of the same quantity (3). What I’m going to be working on now is just proof of concept stuff and experimenting. I’m moving in a few months and I am 100% sure that I’m not going to have anything near a fully workable system by then. Till then I’m just going to be toying around with RFID tags and passive IR beams. Try to get something semi-working… :slight_smile: I’ve ordered some resistors, capacitors etc. and an Arduino Pro to mess around with.

Thanks for your reply.

Oh, and anyone know how to get RFID and IR beams to work in tandem? I was thinking that a tag kept in my wallet would distinguish who I am while the IR beams would track where I’m going. I’m not quite sure how to get them to interface with my machine though… :C

I mentioned the emulation part…

Artificial intelligence is the process of programming a computer to do what a human can do, at which point, upon success, it ceases to be considered requiring intelligence.

Alright, I know I mangled that quote, but something real similar was quoted by somebody “famous” in the AI community long ago, mainly referring to the progress made in both chess playing programs and expert system knowledge bases during the 1970s-1980s.

Imagine you managed to create a computer system that could talk with you, reason with you, come up with concepts and “thoughts” out of thin air - in other words was a machine that could arguably pass a full, unlimited Turing Test.

Would you consider it intelligent? Sentient? Or a mere program? Furthermore, what might that imply about the human mind?

True artificial intelligence goes way beyond simple programming; it is an idea which has vexed mankind in its philosophical implications for millennia.

/this is Bar Sport, right?


RFID is either very limited range (you have to hold the tag to the reader) or expensive.

I read an article on wifi tags (I was looking into medical note tracking in hospitals), which would probably be much better for you and you may be able to measure signal strength on an arduino - in which case if you have two then you could work out roughly where you are. You’d probably need to map it - the stuff in your flat will mean it’s not as linear as you’d expect.

Just a point. When the Mac first had voice recognition way back in the early 90’s I thought great the future had arrived. Then after only one day of talking to my computer I felt that this was the naffest thing in the world to do and I turned it off and hung up my microphone. I think mouse and keyboard are a suitable low key way to interact with the computer.

“Open a letter for Mat, Hal”

“Formatting you hard drive Dave”

I think mouse and keyboard are a suitable low key way to interact with the computer.

Indeed, but the original poster wants to interract with his house, not with his computer. You don’t use a mouse and keyboard to operate computer controlled devices such as your washing machine or your car, why should you with your house?

I think the limited range of commands you’d need will probably suit voice control fine, after all voice-controlled dialing has been around on mobile phones for years.

I really like the idea, I might give it a go myself at some point. I’ve ordered some secure smartcards (multi-section password protected ones) and a couple of readers. I reckon that on my door would work well with a voice controlled house :slight_smile:

First off I do audio installs for a living. I am also a heavy linux user.

Dragon is a great product. I use it for dictation and some computer control. I do not feel it will let you do this type of project. It has to be aware of the program you are using and has to have some built in commands. Also running on windows would be a HUGE drawback. You would lose much of your access to low level scripting and other bits of open source (free) code to do what you want.

As for the audio portion there are LOTS of options. Depending on budget. Got a $2k to throw at an audio DSP? I can set you up with a box that has auto-mixing, contact closure and RS232 and TCP/IP control. Oh and it has audio file storage and playback. I also know of some microphones that look like a wall plate. I think that is all over kill but it is an option.

Linux can support multiple microphones. Just get a couple low end USB interfaces or a multi input recording interface. It also has basic voice recognition. I think you found one of the projects already. Take a look at a phone PBX replacement project called asterisk for some ideas how to implement it. They use it in a phone menu system. With that project you could phone in commands from out side your house!

Linux also offers some other projects including motion sensing and image recognition via web cam. The best part is it is very easy to have one part talk to another. Want the camera to trigger listening for your command? Easy. Not so much if at all under Windows. For interfacing with the real world you have several options. I am just now learning Arduino. Phidgest also have some great options. Thing Arduino may be lower cost in the long run.

Draw up a long term plan and then break it down to smaller parts.

I saw an ad a few weeks back in MAKE magazine about a board that supported voice commands (it was limited to a certain number). Can’t remember the brand, I will look it up for you at home.

Might be an interesting alternative to a Linux box running Dragon. Although Dragon is (probably) a lot more versatile.

Is the room where the command is issued important? Otherwise you could just mix the microphone signals to one input with an audio mixer (you could interface electret mics to an Arduino to determine from which room the sound is made).

Best solution is probably to just use the USB devices tjbaudio is talking about.

Can’t find the ad. Maybe someone here knows which product I saw.

Today, by coincidence, I stumbled upon the product I was referring to, it’s called the VRBot: