Using Goertzel protocol for DTMF and other decoding

Other Hardware Audio
1

@6v6gt, I found your 2019 rework of the Goertzel code, and am working through it. But in addition to the standard eight DTMF frequency pairs, I would like to be able to detect other call-progress tones, such as (in the US):

dial tone - 350 + 440

ringing  - 440 + 480

busy -  480 + 620

Is there any reason to believe that these relatively low frequencies can't be detected as well as the DMTF tones?

Also, it appears that your code would let me do these in a separate instance so I wouldn't be wasting time looking for the DMTF values. Is that correct?

I'll be using an Ooma Telo VOIP box to make a call. As far as I can tell, the Telo is just like a central office, with the same frequencies and behaviour.

2

No, it is a general purpose algorithm. However, the Goertzel algorithm is known for being finicky and often needs individual optimization (sample length, sample rate, center frequency) for reliable detection of certain tones in a new setup.

A web search will find a number of examples and papers discussing the known weaknesses, along with proposed improvements to the algorithm.

Here is one place to start.

3

Thanks very much. Hopefully that Kevin Banks paper will explain the method to me, but the EE stuff (FFT and such) is well above my hobbyist pay grade. The latest re-work of the method I've found is by @6v6gt:

https://forum.arduino.cc/index.php?topic=121540.msg4305445#msg4305445

I'm hoping it will be obvious as to how I add the lower frequencies to his code.

4

@ShermanP
Is the MT8870 an option,

5

I've just managed to get a few moments to look at this.
I can't say definitely if it will work with these lower frequencies but you can try.

Are you able to set up a test which can generate these three combinations:

If so, then I'd suggest a "small" hack to the sketch dtmf_v0_08P.ino at the link you found just to detect these combinations, crammed into a 3x3 matrix.

//row objects
Goertzel x_goertzel[3] = {
  Goertzel( 440.0 ),
  Goertzel( 480.0 ),
  Goertzel( 1700.0 )  //dummy
  } ;

//column objects
Goertzel y_goertzel[3] = {
  Goertzel( 350.0 ),
  Goertzel( 620.0 ),
  Goertzel( 480.0 ) 
  } ;

const char decoder[3][3] = {
  { 'D' , 'U', 'U' } ,   // 'D' = dial, 'U' = unknown
  { 'U' , 'B', 'U' } ,   // 'B' = busy
  { 'R' , 'U', 'U' } ,   // 'R' = ring
  } ;

and this in the loop has now only to scan a 3x3 matrix instead of a 4x4:
for ( int i = 0 ; i < 4 ; i++ ) { //change to 3 instead of 4

In the mean time I can haul the whole thing out again, look at it in more detail and, hopefully, work out a way of integrating both the standard DTMF sequences and the extended sequences of Dial, Busy and Ring in a single sketch without too much redundancy.

6

You don't need to know about those things. However, you DO have to follow the rules in the Kevin Banks paper. Especially important is to find a sample rate and sample size N such that

the target frequencies should be integer multiples of sample_rate/N.

The importance of that choice cannot be overstated. If you don't follow it, the algorithm will fail.

7

I don't think so. As far as I can tell, it is limited to the DTMF frequencies, and I also need the call progress frequencies for dial tone, ringing, etc. I can't find anything in the datasheet about rolling your own frequencies.

8

Yes I saw that, but they won't be exact multiples. I mean, the frequencies are really oddball, I guess so their harmonics won't ever meet up. And I think @6v6gt's code fixes the sample rate at 8000. But I have more studying to do. Anyway, I assume he tested his version, so it should work for me.

9

I don't have a signal generator, but I can tap into my phone line to get the dial tone and ringing sounds. I don't know what I would dial to get a busy tone these days. Everything seems to have call waiting.

But I do have a question about the input signal. If I have the resistor divider producing 2.5V, and feed the input from the phone line through a capacitor, do I need the composite signal to be pretty much rail to rail? It would seem that would need to be the case to get the best results.

Yes that's what I was thinking. I just wanted to be sure the frequencies I need aren't too low to work with the algorithm for some reason.

Well, please don't do that on my account. I was thinking of just using different instances depending on what frequencies I'm looking for, and even making the array dimensions part of the instance. I basically have two situations that need decoding - DTMF and call progress - but even the DTMF may be limited to just a couple keys.

Thanks very much for responding, and thanks very much for the 2019 post. It's just very cool that this works at all.

10

If you haven't "done the math", you can always hope for good luck.

11

I'm going to have to rework what I'm doing to get audio out of the phone line. I tried something with the transformer, but even with an LM386 set at 20db, I'm only getting about 1V P-P, which isn't even enough to decode DTMF. And unfortunately, dial tone is at a much lower level. So I need to figure out, or find somewhere, a better interface.

12

Did you put a scope across the input to the Analog pin to check the waveform ?
You have to be careful if you simply amplify a signal which has a DC offset in it because you also amplify the offset.

I have just done a quick test with the DTMF code I published and fed it from the following circuit which I use for amplifying telephony level signals, designed to amplify only the AC component of the signal. The irrelevant parts of the circuit are marked. It appeared to work fine.

I used this DTMF tone generator Online Tone Generator - A free and simple way to generate DTMF dial tones. and connected the PC audio output to socket marked RJ11 (effectively the input side of the transformer via a 470nF cap and a 1k resistor)

image
image1152Ă—721 77.5 KB

13

Thanks very much for taking the time to do that.

I was trying to use a circuit I used back in the day for recording a phone conversation. It worked fine for that, but it had many protections built in, and was just designed for mic-level anyway, so it just wouldn't produce enough juice. I did look at it on the scope, and it was only around 100mV peak-to-peak for DTMF, and dial tone was almost nothing. So I'm redoing the circuit.

I understand the problem with DC offset, and I did feed the output of the LM386 through a capacitor, but the LM386's output is nominally always midway between the two rails, regardless of gain. So I was hoping to eliminate both the capacitor and the bias resistors on A0. But I would need to adjust the idle ADC value in the code to what it actually is rather than assume 512. Well, I can play with that either way and see which is better. Actually, I'm not sure the LM386 is the right part. A regular opamp may be better for this.

Edit: I meant 128 for 8-bit conversion, not 512.

Edit2: Could you explain the U3A opamp circuit? And then how is SIGNAL connected to A0?

14

The U3A subcircuit is an audio amplifier with high pass filter, fixed DC input/output offset of Vcc*82/(100+82) or 2.25V, and variable gain, ranging from less than 1 to about 150, depending on the setting of RV1.

In the following implementation of the circuit, the gain is about 30 and the frequency response rolls off below 1 kHz, to unity gain at 30Hz.

Capture
Capture384Ă—184 9.83 KB

15

It sounds like your scheme for integrating the amplifier is OK. Wrong, of course, would have been to amplify the signal after the DC offset was applied. But it is now clear what you did,

Fortunately for me @jremington has already analysed it but I did find the circuit somewhere and adapted it. Here is one: https://electronics.stackexchange.com/questions/157046/how-to-amplify-a-small-ac-no-dc-offset-wave-with-an-op-amp-powered-from-a-0-5v

Signal is connected directly to the analog input.

If you are having trouble with normally telephony signal voltage levels and conventional DTMF frequencies, I can get the scope out and say at what level my installation fails at.

EDIT
I use that circuit for the analysis of ASFK signalling on the telephone line and ring detection. There is also an ATTINY1614 (not shown) which performs the demodulation.

16

Ok, so since the R1 - C1 series is effectively out of the circuit for DC, the opamp just becomes a voltage follower for the DC voltage produced by the upstream bias resistors. But they do affect the gain of the AC component, so you can amplify just the AC without affecting the DC offset. That's pretty neat. And with a big enough capacitor, frequencies down to 300-400 Hz wouldn't be attenuated that much.

But do you actually get a reading of 128 at A0 when there's no AC signal?

It seems to me that my LM386 might still work, depending on how stable the output DC level stays. And since its inputs are already referenced to ground, I wonder if I could connect the transformer secondary directly to the + and - inputs, and get more differential gain. I think I can actually test that. The datasheet says it can take inputs down to -0.4V, which is probably good enough.

I need to do more testing of the signals my Ooma gives me, and get them to A0 looking right. And I'll see if the LM386 will work.

17

Near enough to work in the intended application and, as my tests have shown, also for the DTMF decoder. I can't recall the exact reason for the imbalance in the resistor divider but I seem to remember that no signal but with noise caused a problem for the demodulator because it was continually crossing the 128 midpoint, probably for its activity indicator led. What I also remember is that it tolerates significant clipping.

18

Ok, I got DTMF working. Here's the schematic:

Telephone Interface
Telephone Interface1467Ă—839 53.8 KB

The LM386 turned out to be a good choice. Its inputs are already referenced to ground, so I was able to simply connect the transformer secondary leads to the LM386 inputs. And its output is automatically Vcc/2, so perfect to feed to A0 directly. No capacitors or biasing transitors needed. And as it turned out, the default 20db amplification is a bit too much, so I added a 680R resistor across the secondary so it wouldn't clip, at least on dial tone.

Since this device will only call out, the ring detector circuit is just to try to prevent it from inadvertently answering the phone when it wants to dial out. However, it does another useful function. If I pick up any extension phone on the network, the line voltage will drop from 48V to about 7V. That's enough to generate a small spike in the optocoupler, which in turn is enough to trigger an interrupt on the Arduino. I can then have it listen for a special key sequence I might press, that would not be part of a dialed number, like "star, pound, star, pound", to have it perform a function without having to go to the device itself and press a button. This circuit is on the NC terminal of the relay, so it's connected to the line most of the time.

Everything else is on the NO terminal, so it's only connected to the line when the device needs to make a call, and at the same time all the ring detector stuff is disconnected from the line. Basically its just the audio path through the transformer to the LM386, and finally to the Arudino. But tapping off of that is the circuit that sinks enough DC current to take the line off-hook. That needs a FWBR because the polarity of tip and ring is not always certain.

I was able to test DTMF by calling in from my cell phone and pressing the keys on that phone. Everything worked perfectly:

DTMF Decoder V0_08P
9   (2,2)   failed/rejected matches=2
1   (0,0)   failed/rejected matches=1
1   (0,0)   failed/rejected matches=1
1   (0,0)   failed/rejected matches=3
1   (0,0)   failed/rejected matches=2
8   (1,2)   failed/rejected matches=2
9   (2,2)   failed/rejected matches=2
#   (2,3)   failed/rejected matches=1
#   (2,3)   failed/rejected matches=3
#   (2,3)   failed/rejected matches=1
#   (2,3)   failed/rejected matches=1
*   (0,3)   failed/rejected matches=2
*   (0,3)   failed/rejected matches=2
*   (0,3)   failed/rejected matches=1

I dialed "918#*" from the cell phone, and it got all of them. I need to see what the failed/rejected is all about since it didn't fail. Also, it needs debouncing - one result per keypress.

Next I need to get dialtone and the other call progress tones working. Dialtone is almost 4V peak-to-peak at A0, so there should be no problem if I have the right frequencies. But Wikipedia has an audio file of the US standard 350/440 dialtone, and it doesn't quite sound like my Ooma dialtone. So I may have to alter the code to sequence through all the frequencies in that range to see which ones work.

Thanks very much for everyone's help, particularly @6v6gt for the code and the circuit suggestions.

I'll probably post this as a project when I get it finished.

Edit: Oops. I need a diode across the relay coil.

19

It looks like you are making good progress.

The term "failed/rejected matches" is a bit misleading. It is actually the number of tries to get a valid result. The minimum will be the value of the parameter validationRepeats which is set at 1 in the example code but can be played with. So failed/rejected matches=1 in this case is actually good and should better have generated something like an OK message . Anything higher than 1 indicates it was struggling.

The jitter on the the output is somewhat alarming though. Maybe the keys repeat with a short break if held in for more than Xms. Maybe a scope picture when say the "#" key is held in for a number of seconds will show something.

20

So if I hold down the # key, I should ideally get a string of # results, all with "1" as the number of repeats? Well I will experiment more, and also see what it looks like on the scope. But since the DTMF is coming from a cell phone at the other end of the conversation, there might be some noise or distortion that's giving it some trouble.

I should also say that I will need to rework the transformer secondary circuit to bring in the outgoing DTMF to make a call - through an R-2R network. Not sure how I will do that.