Go Down

Topic: Esp8266 Arduino Telephone Caller ID system with anti-spam feature (Read 7931 times) previous topic - next topic

6v6gt

This is an ESP8266 based device which interprets PSTN telephone caller ID information and displays it on a screen, supplemented with information drawn from a mySQL based directory. There is an optional feature to suppress the telephone ring for calls identified as spam. Full spam protection allows only callers who are specifically registered in the directory to cause the phone to ring. Spam calls land silently, after a preset period, onto the voice mail. You must (or should) have voice mail if you use anti-spam features on your telephone connection.






The situation here is that black lists are not now very useful because the worst telephone spammers are now spoofing numbers, including those of valid telephone subscribers in the same network. At home, I now  implement a white list, the strongest level of spam protection.

The design uses a software demodulator based on the Arduino APRSlib (www.unsigned.io). For signal conditioning, it uses a separate ADC chip (the inbuilt one is not fast enough) and a rail to rail op amp which also handles ring detection. It does not rely on any telephone specific chips, which are now anyway obsolete and difficult to obtain.

Call (and telephone book) data is maintained on a MySQL database (LAMP server). This server can be local to your lan or an internet service. It would have been possible also to do all this in flash memory but number matching, joining tables, formatting output etc. is much easier in SQL and PHP, and mySQL anyway has basic admin tools for browsing and maintaining tables. The device works also in offline mode and then can display incoming calls, but then there is no call storage/retrieval or directory look up and the spam suppression in not available.

Ring suppression is implemented with a simple relay and activated only during ringing in a confirmed spam case. It is fail safe with normally closed contacts so that during power failures, crashes etc. the standard telephone functions are available.

The main user interface is a 240 x 320 tft touch screen (SPI). This is used for displaying and paging through call history and setting some configuration parameters. Other parameters relevant to the online mode of operation are set through a Web browser with the ESP8266 running its own access point.  

Even if you are not interested the telephone application, the design contains some features which may be interesting for other purposes, for example the connection to a MySQL data base for storing and retrieving data, A light weight and easy to use navigation bar class for the touch screen and a simple 2 point calibration of the touch screen on set up to map it to the screen.

The main documentation (including a schematic) is a PDF file (which may, because of its size, is in the next post in this thread).
The zip file contains the ESP8266 software and the PHP scripts for accessing the database.

This is probably not a good project for an absolute beginner. Some knowledge of MySQL and PHP would also be useful as could be the availability of an oscilloscope.

This is a new development of something I published here on July last year.


Disclaimer: Before connecting any device (such as this) to your telephone system, ensure that doing so does not conflict with any rules of your local telephone operator. Understand that you use such a device at your own risk.


Edit 1.
I've captured an audio sample of the AFSK encoded telephone caller id. It is in the zip file Telephone_Caller_ID-AFSK_sample.zip below. You hear it as a 1 second burst between the low frequency ring tones. It is in MDMF format and contains the number 0800807111 date 2203 and time 0851. I have used such samples for testing the demodulator via a PC sound card.

6v6gt

The documentation, including a schematic is attached here.
It is too big to fit in the first post.

alezarate


dimitrios_leonardos

Congratulations,excellent and usefull job.I wonder if it would be possible for me to get a device.If you are intersested please contact me at dimitriosleonardos@yahoo.com,to arrange it...

6v6gt

Congratulations,excellent and usefull job.I wonder if it would be possible for me to get a device.If you are intersested please contact me at dimitriosleonardos@yahoo.com,to arrange it...
OK. Thanks. I guess you have understood that this is not a stand alone system. It needs also a server somewhere with Apache/PHP/MySQL for storing the calls and maintaining the telephone book (This could be a network service or even a Rapsberry Pi in the WLAN)

The design documentation is complete so someone could build it based on that alone. I won't do this for you but there are people on this site who do. Simply put post in the Gigs and Collaborations  section of the forum. I would, however, be prepared to give some limited support to an Arduino developer who is also familiar with ESP8266 and PHP/MySQL if it comes to troubleshooting etc.

I also have a later version of the software which includes a "Message Waiting" indicator and screen button to accept a current call attempt, even if it is identified as spam. I will publish this soon here. I have also a partial development on a Raspberry Pi, where the demodulator and parser are offloaded to an ATMEGA328P, but that some way from publication.

jimLee

This is pretty close to what I wanted to build. I thought it would be cool to put a little blue ice cube icon on my phone. So when a SPAM call came in I could click it and "ice" the call. Then everyone connected to the service would have that call blocked as well.

Until this AM when my Daughter got a SPAM call spoofing the number from the cell phone I'm currently building! I guess they are hacking into iPhones and reading their address lists for numbers?

Spooky world.

-jim lee
PNW Ardiuno & Maker club
1012 9Th Street, Anacortes, WA 98221 (Around the back of building)

rocketmanrc

I built a version of this and it works great! I really appreciate all the work that has gone into the development of this project and it was amazing to find something that did most of what I needed for what I wanted to do. No more spam calls at night for me :)

In my case I did not use a display and it was fairly easy to remove the display dependent parts from the code.

The only issue I had was with the ESP8266 crashing in the timer interrupt handler. I resolved this by editing SPI.cpp in the Arduino platform code and marking all methods in there with the ICACHE_RAM_ATTR. I've attached my modified file in case anyone else runs into this.

Cheers,

Rick

6v6gt

I'm pleased that you had some success with this project. I didn't experience the SPI issue which you referred to but I did experience similar problems with internal functions in a timer call back chain not being forced to remain in ram. That was solved at the latest by core 2.4.0. Anyway, someone may well benefit from your troubleshooting.

I'm now adding the latest production version of this application v1_12 which has the following features:
1. A Message Waiting Indicator
2. An Accept Spam Call on screen button, allowing the user to accept a spam call during the period the ringing is suppressed.
The PHP code, database format, documentation (apart from the supplement below) etc. remain the same as in the original version published earlier in the thread.

Also included are some examples of the format of message waiting notifications (MDMF). However, you can also use the following tool to visualize the various Caller Id and message formats and even generate the audio signals: http://forum.arduino.cc/index.php?topic=599571.0

I'm currently documenting a simpler system aimed more at beginners who can take the basic core consisting of the demodulator and parser, and incrementally add their own functionality to it without having to touch the core logic. It is ATmega328P based. Hopefully, it will appear soon.


rocketmanrc

The message waiting feature is the icing on the cake for me on this project. Because we mostly had telemarketing/spam calls on our landline we largely ignored the phone and sometimes we didn't notice that we had voice mail for days. That problem will be solved now!

This really is a great project and I know how much work has gone into it. Thanks again 6v6gt!

witkov63210

I love to see the interest in this subject. Personally, I think we need telemarketers a little less than fleas, mosquitoes, and politicians.
It is my ambition to put together a system that can be given away as a promotional device like ball point pens or refrigerator magnets. so far, I have a system that works but, it would make a very expensive refrigerator magnet at hobbyist prices.
It is completely stand-alone, it uses the Mega2560, the HT9032 receiver, the Seeed 240X320 TFT display/SD, the Osepp RTC/SD, and the Emic2 text to speech talker. I have incorporated a couple features that have not been mentioned so far.
1.) Since I have two SD cards, I use one to hold the message played for the blocked caller. I do not simply block the call and let it go at that. To reduce the likelihood of a repeat call I play the triple tone followed by 'The number you have reached has been disconnected or is no longer in service.' I use the the Simple Audio which requires audio stored on a clean SD.
2.) Since the telephone companies are encouraging telemarketers to spoof their CIDs, I have gotten calls from numbers from power company, doctors offices, IRS, insurance company, I have even received calls from myself. The problem with simply blocking these calls is that I may be blocking legitimate calls. My reasoning says that a telemarketer won't stick with a particular spoof number for an extended amount of time. By simply scanning the time stamp on the call records I can purge the blocked calls after a settable amount of time.
I can do searches, printouts create/edit white lists, and other features that have been mentioned here. My system works quite well but I am not good at packaging so it is not impressive to look at. Also, I have been an amateur programmer since 1975. My code is not pretty either. I have been thinking about incorporating the components unto a single circuit board to avoid the high hobbyist prices but not until I reach a plateau in the development.
Did I mention that I am not fond of telemarketers? For several years I have been asking who is behind the push to eliminate credit card interest and why it is so important to them that they must ring my bell 5 or 10 times a day. I have tried many times to see where they are going with their presentation. So far the only conclusion is that they simply do not want me to answer my phone. Why???  Who??? Why cant calls be traceable, like emails? Did I say I don't like telemarketers?

Adrianotiger

I made the same project with an Arduino nano and an ESP8266 3-4 years ago.

The HT9032D is a fantastic component for it. Since I didn't have much memory, I was just sending the caller ID to my database and got the answer directly from whitepages service.

This year, after I saw that you can buy an ESP32 wroom with 16MB of flash for just 5$, I begun to see what is possible to do with this microcontroller.

I am still developing it, but this is the base schematic for it:


- Hook-Up resistor with optocoupler. Just put it to an output to answer the call.
- 600:600 audio transformator. Connecting it to ADC and DAC you can listen and also play waves*! And you can also detect DTMF signals without any external device.
- The HT9032D for caller ID
- Ringing detection

My idea is that if someone calls you, the ESP32 will ask if the number is valid. If it is valid, it will leave it ringing (maybe in future, after 8PM it will answer automatically).
If not valid, the module will hook up and play a sound or text.
Do you know ESP8266SAM? It is a text-to-speach engine and works perfectly on the ESP.

So you can say: "press 1 if you want send a message to Adriano" and you can listen for DTMF audio. You can play also the "Number not valid" tone.

Note: The optocoupler is important, since you have over 90V AC on the Tip and Ring bus when it is ringing.

6v6gt

The HT9032D chip (no longer stocked by Mouser, Digikey etc. but still available from some EBAY retailers) can certainly reduce the programming effort required for these Telephone System applications because there is then no need for a software AFSK demodulator. I used a different philosophy and took an established software demodulator originally intended for Amateur Packet Radio Service applications as a basis. With quite a small hardware footprint, I also achieved quite pleasing results.

jimLee

PNW Ardiuno & Maker club
1012 9Th Street, Anacortes, WA 98221 (Around the back of building)

Go Up