Hmm I'll have to study up on that. I'm certainly not opposed to some complex tech.
My rough plan for the time being is to use some electret mics to LM386's in to the analog pin on an wifi connected arduino, datastream straight off the pin to a socket on my node.js server on a big box(16 xeon cores and 96 gb ram) assemble into .wav, and then the tough part. All audio output (whole home audio system) is routed through that box. It will be recording all the time both output and the arduinos. Sox will invert the waveform combine the tracks and maybe with a whole lot of fiddling and probably luck it will remove enough of the output for sphinx to understand my voice.
Oh if anyone has some hints on turning a stream of analog samples into a wav file that would save me some research.