Is there a way to filter the signal that servo moves only when significant characters are said ?
Yes.
You can filter either in software or hardware. In hardware you need an envelope follower before the A/D converter. Envelope detector - Wikipedia
Or you can do the same thing in software by following the values and making note of the biggest one. After a certain time use the biggest sample you have to set the jaw servo value.
A bit like this, which was done with lights.