Websockets with Arduino

Hello, I need to use websockets for real-time communication with my Arduino device connected with WI-Fi. I will access it through Wi-Fi from my laptop. Have you used any websocket library capable for ESP8266-12E? I also found that some people use MQTT. Do I need to use MQTT with websockets? Any advice/suggestion?

#update: I run the “server” code from here:
WebSockets
and although is compiled and downloaded correctly, when I try to access the WIFI ESP8266 from the browser in the laptop, I can’t… Any idea what is wrong?

I also found that some people use MQTT. Do I need to use MQTT with websockets? Any advice/suggestion?

MQTT is a message broker. The use cases are not the same as for Websockets. With MQTT you need a broker (p.e. Mosquitto) on a server.
You cannot use MQTT with websockets (at least no without additional programs on a webserver).

and although is compiled and downloaded correctly, when I try to access the WIFI ESP8266 from the browser in the laptop, I can't... Any idea what is wrong?

You probably don't have the websocket support in your web application. I got the impression you have no clue what Websockets are and what they used for. So you should start by describing in detail what you're trying to achieve and not how you think it should be implemented.

Correct If I am wrong, but MQTT is being used on IoT in cases when you want to access an IoT device and this can support many clients (through the broker).

Websockets are used in browser when you want to have real-time interaction with server, such as live playing games, so that you don't need to refresh the page.

I have some switches connected to an ESP8266-12E and want to access it through my laptop and on real-time when the switches are ON or OFF to get the information instantly.

Moreover, the ideal would be to run the above solution on a 4G modem with Arduino MEGA 2560, I do not how...at the moment.

I achieved sending messages from my browser to the ESP8266 and get response. How do I connect MQTT on this is the problem now? I mean, should I find a website to connect to the MQTT over websockets ? I am searching it...

Here is an example of MQTT

(not using some freebie on-line service.. which is an option)

Many people use a RaspberryPi to install the MQTT framework on.. this makes it your server (broker)

Your clients (Arduinos or whatever) are clients.

This is often referred to as a SUB/PUB type of approach..

where 1 to many clients 'subscribe' to the topic.. and are instantly notified whenever something happens/updates and this change sis then PUBLISHED to all subscribed clients.

MQTT sounds like what you want.

otherwise.. you are stuck having to poll over and over checking some reference/end point to see if there is a change... which is a waste of resources.

You have another aspect of this project I believe as well..

You want this updated real-time.. where?

On some webpage? Is this 'webpage' hosted on the ESP itself?

  • You have manual switches connected to an ESP module
  • You want to load a webpage? (again hosted where?) that shows the state(s) of these switches..

is this the summary of your project?

As you write it, I do not need a Raspberry Pi as a broker. Can I have the end device working both as end-device + broker? I need to have the end device, which is the ESP-8266-12E (or the Arduino MEGA 2560 + 4G/LTE shield) and the client or clients, which are laptops/desktops/smartphones that will hit the IP address and get information of the switches in real-time (meaning without the need to refresh the pages and waiting. Yes, it is a good idea to store the website on the ESP8266-12E (alternatively on Arduino MEGA + 4G/LTE shield) and update it with the values of the switches (ON/OFF). Optical switches. Yes that's the summary! Can this work?? And how do I make it?

It sounds like your approach is a 'polling' approach.

Which to me sounds like:

Your approach:

  • You call your friend..
  • his mom answers,
  • you ask if 'friend' is home..
  • (mom says 'no)..
  • you hang up.. and
  • call right back and
  • ask again if 'friend' is home
  • (again mom says 'no')..
  • you again hang up and
  • call right back.. (rinse/repeat)

Does this sound efficient to you? You spend all your time 'checking' not even knowing if there is a change.

or MQTT approach:

  • You call your friend..
  • his mom answers,
  • you ask if 'friend' is home..
  • (mom says 'no).....
    ***** but then says.. I will have him call you as soon as he arrives.

I would think you would want scenario 'B' here.... especially if you plan on expanding this.
but to each their own.

When you 'load' the IP address.. you ARE loading/refreshing the page.... so I'm not following what you mean.

If you want to go your way...

Have your ESP that has the switches connected to it.. send/update a database using a PHP script (or whatever)...

(simply sending param data in the URL to a PHP script)

And all your other 'clients'... can load this data however you want.

"and the client or clients, which are laptops/desktops/smartphones that will hit the IP address and get information of the switches in real-time"

Question: How/what/where are you expecting this data to look like?

If your clients 'hit an ip address' what does that mean to YOU?

Thank you for your answer. Let me explain...
8 Switches connected to the Arduino/ESP-8266-12E (does not matter at this point)

The user puts the IP address of the and device (Arduino/ESP-8266-12E) on his browser, and watches on the browser on real-time the data below...

timestamp_1: SW1-> OFF, SW2 -> ON, SW3 -> OFF, SW4 -> OFF, SW5 -> ON, SW6 -> OFF, SW7 ->OFF, SW8 -> ON
(after 5 seconds)
timestamp_2: SW1-> ON, SW2 -> ON, SW3 -> ON, SW4 -> OFF, SW5 -> OFF, SW6 -> OFF, SW7 ->ON SW8 -> OFF
(after 5 seconds)
timestamp_3: SW1-> OFF, SW2 -> OFF, SW3 -> ON, SW4 -> ON, SW5 -> OFF, SW6 -> ON, SW7 ->ON SW8 -> ON
....

Then what you are wanting to do it a very inefficient approach IMHO..

but..

Have your 'Arduino/ESP' module (with the connected switches) update a database (using whatever server side scripting language you are comfortable with..such as PHP for example) every time a switch is 'changed'..

The rest has nothing to do with anything 'Arduino' related..

You have your 'webpage' (hosted....where? you never said).. that polls this database to see if any updates have been made.

The 'real-time' is now on the 'webpage' your want your clients to access..

You could use some javascript/jquery (AJAX call to a php script) to continually check for updates.. or updates the page every 'xx' seconds... up to you.

If the webpage is hosted ON the ESP with the buttons.. you might not need a database.. as you can get the data right there)

I have 6 ESP32's connecting to a RPi Broker, running a Python program that directs communications between the ESP32's and my websites database.

I can, for instance, using my web site, look at the current humidity in the bedroom, see it is too low, turn up the humidity with a slider on my web page, click the update button, and in moments the humidifier clicks on in the bedroom.

Nice project!.. (I understand MQTT and how to use it for projects like this…but not exactly the same thing I dont think)

Question:

If you are ‘looking at your web site’… and viewing the current humidity in the bedroom.

  • This is what, represented by a number I’m guessing?
  • When you load the page… it displays the current ‘value’… yes?.. what happens if you let the page sit open? (not adjusting temp with the slider/sending data back to anything)… does that numbers/value update on its own?

If yes… how are you doing the ‘web page updates’?

Thats what he is asking for…

Not just to see the ‘humidity value’ once a page is loaded… but to keep the page open… and constantly see the ‘updates’ in real time… without re-loading/re-freshing the page…

xl97:
Then what you are wanting to do it a very inefficient approach IMHO..

but..

Have your 'Arduino/ESP' module (with the connected switches) update a database (using whatever server side scripting language you are comfortable with..such as PHP for example) every time a switch is 'changed'..

The rest has nothing to do with anything 'Arduino' related..

You have your 'webpage' (hosted....where? you never said).. that polls this database to see if any updates have been made.

The 'real-time' is now on the 'webpage' your want your clients to access..

You could use some javascript/jquery (AJAX call to a php script) to continually check for updates.. or updates the page every 'xx' seconds... up to you.

If the webpage is hosted ON the ESP with the buttons.. you might not need a database.. as you can get the data right there)

So, If I understand correctly it is useless to use websockets or MQTT, I just write a script in javascript/jquery and I upload it to the Arduino/ESP8266 and I program the script to be updated every 5 sec or when it senses different levels on the switches and run it? And this solution supports multiple clients (laptops/desktops/smartphone) accessing the webpage (stored on the Arduino/ESP8266) simultaneously + that they just put the IP address in the browser and it works. Am I correct?

Useless.....not at all.

It is the CORRECT approach. (not sure how that can be said any more clearly)

More or less that is correct.... except I dont think you are grasping that what the 'clients' see on this webpage is just a read only type of thing.. (just viewing some webpage you output...static value of the states of the switches)

I'm not sure how you think javascript is going to detect anything from these switches? (maybe "I'm" missing something here?)

Your ESP will have a webpage..
when it parses/displays itself.. it will display the switches states... thats its.

for this.. the javascript is ONLY used to automatically refresh the page.. (which would then re-load and display any updated switch states)

In your 'HTML' for building the page.. you include the states of these switches..by URL param or whatever..

This tutorials uses a button to change an LED state.. but it also talks about using JS to auto-refresh a page:

http://www.martyncurrey.com/esp8266-and-the-arduino-ide-part-8-auto-update-webpage/

As a first approach before using MQTT, websockets and stuff like this...I am following this tutorial: http://dweet.io/ and I have managed to send the data from Arduino MEGA 2560 + 4G shield on my laptop by browsing the link it says inside the tutorial, but what I get is something like this:

{
  "this": "succeeded",
  "by": "dweeting",
  "the": "dweet",
  "with": {
    "thing": "my-thing-name",
    "created": "2014-01-15T17:28:42.556Z",
    "content": {
      "hello": "world",
      "foo": "bar"
    }
  }
}

that has nothing to do with what I programmed the Arduino to send!!! Any idea, what is wrong ? Am I missing something?

I dont see any tutorial.

I only see some 'messaging' site.

Dont know what it is...never heard of it.

Not sure what it has to do with your project either.

xl97:
I dont see any tutorial.

I only see some 'messaging' site.

Dont know what it is...never heard of it.

Not sure what it has to do with your project either.

Somewhere I have to post the data from the Arduino... :slight_smile: Especially now that I use 4G/LTE module. In the WIFI testing I managed to send HTTP data...

Let me explain, when I use WIFI module, it prints me on the Serial the IP address, I hit it on the browser, I access the WIFI, everything brilliant!

When I put the HTML code to the Arduino MEGA 2560 + 4G/LTE shield and I need to read it from the webbrowser of the laptop, I cannot find out the IP address of the 4G modem and I think it needs reverse tunnel to access it (from experiments I have done with mobile phone).

I am building firstly the HTTP part to see that there is access/response, then I will use MQTT/websockets....

First:
Where is this 'web page' hosted?

Second:
You state this webpage is hosted on an ESP module... yes or no?

Third:
What is the IP address of this ESP module that is hosting the webpage?

xl97:
First:
Where is this ‘web page’ hosted?

Second:
You state this webpage is hosted on an ESP module… yes or no?

Third:
What is the IP address of this ESP module that is hosting the webpage?

Ok, the ESP8266-12E works, concerning a simple webpage HTML/javascript code, that is stored inside the ESP8266-12E, forget about it. Yes, the IP address that each time gets from the home router/DSL modem.
As far as the Arduino MEGA + 4G/LTE shield, I have stored the webpage inside Arduino. But I cannot access it because I don’t know what is its IP address. (AT commands’ .ino code does not work) For this reason I try to post the webpage (that I have stored inside to Arduino) to http://dweet.io in order to get informed of the status of the switches. Any better idea??

Forget about it? WTF? Where is this webpage HOSTED!

Just answer the questions above.

"Yes, the IP address that each time gets from the home router/DSL modem."

  • What does that even mean? Is that a sentence/statement/question?

No clue about your 'shield' thing...

but you if its similar to an ethernet shield.. then I wold assume you gave it an IP address within your local LAN ip range... no?

Your 'dweet' thing.. is more or less an MQTT broker.. (although I dont know how it alerts any subscribed clients).

I'm not going to worry about that... as you should need it.

So you do NOT want to host the webpage on an ESP module any more.. but instead.. you want to host it on the 4G LTE shield??

Which I have no clue about it.. this lets you configure it as a web server??

Isn't this just Arduino acting as Webserver? You don't need MQTT or Websocket or any fancy doodleedoo.

If I'm reading things correctly, you want to have an Arduino collecting data from sensor, and then serve it to a PC client? If so, this is just an Arduino webserver case. You "create" a HTML page in an arduino, and when a client connects to the Arduino's IP address, it sends the HTML page.

I'm still confused about your hardware choices, tho. Do you have a 4G/LTE + Mega or ESP8266-12E?

Edit: I just re-read this part:

alex5678:
Thank you for your answer. Let me explain...
8 Switches connected to the Arduino/ESP-8266-12E (does not matter at this point)

The user puts the IP address of the and device (Arduino/ESP-8266-12E) on his browser, and watches on the browser on real-time the data below...

timestamp_1: SW1-> OFF, SW2 -> ON, SW3 -> OFF, SW4 -> OFF, SW5 -> ON, SW6 -> OFF, SW7 ->OFF, SW8 -> ON
(after 5 seconds)
timestamp_2: SW1-> ON, SW2 -> ON, SW3 -> ON, SW4 -> OFF, SW5 -> OFF, SW6 -> OFF, SW7 ->ON SW8 -> OFF
(after 5 seconds)
timestamp_3: SW1-> OFF, SW2 -> OFF, SW3 -> ON, SW4 -> ON, SW5 -> OFF, SW6 -> ON, SW7 ->ON SW8 -> ON
....

Apparently you want some sort of history? If you do, you definitely need a database. And if you do that, you need completely different configuration than the one I proposed above.

@linearity64

I 100% believe you are correct

If the ESP is hosting the page.. any time a 'client' accesses it.. it should be able to display the current switch state(s).

He would just need to add something to the HTML markup (JS) that refreshes the page every '5' seconds (or whatever).. because one of the main requests was.. real-time updates.. without having to submit or refresh..etc..

Dont know anything about the 4G/LTE shield.. nor if it can act as a webserver or not?

  • Doesnt need MQTT if the nothing is subscribed or changes based on updates ('clients' are more of a web page/server client.. than a MQTT pub/sub client)..

  • Doesnt need the odd website.. that sorta acts an online MQTT service (dweet.io)

  • Doesnt need a database (unless what you brought up is true.. he wants to see a 'history' of switch states?)

Simple ESP hosted webpage.. that has the switches connected to it. once a 'web page' request is made.. it reads button states.. and displays on the page. (I even posted a link on how to parse the HTML 'variable/placeholders' with the switch states)..