Posting Arduino data to PHP for real-time processing

Projects General Guidance
1

As above, I'll try to describe it in a sequence diagram:

  1. Arduino submits a POST request to a PHP page using WiFi (An unique user ID)

  2. PHP page receives the data and then requests data from MySQL. (Select queries will be made with reference to the unique user ID)

  3. Following that, the MySQL data will be used to load several scripts (API calls), like Twitter and Facebook feed and whatnot.

  4. All the data will be dynamically displayed on the index page in real-time. (A notice board)

I do know that the coding might involve jQuery and AJAX, but what I do not understand is how the data should flow and if I'm violating any programming conventions or not.

My question is how do I receive the data and do the aforementioned points in sequence in PHP?

Thank you

2

hahaashton:
To sum it up, the Arduino will send data over whenever there's an input from the sensor (loop), and the PHP page will constantly poll for data and do the aforementioned stuff.

I find this sentence confusing.

First, you have not told us how the Arduino sends data to the server. Perhaps it is connected to it by USB, or maybe it is a WiFi connection. They provide different opportunities for the server software.

Assuming you are connecting the Arduino to the server using WiFi ...

The idea of the Arduino sending data periodically is appropriate.

But I don't know what the PHP page will be "constantly polling" - it can't be polliing the Arduino.

...R

3

I apologize for not being explicit enough, but yes I'll be sending data over to the PHP server through the use of WiFi.

I'll give a brief example:

Let's say the data that's being passed to the PHP server is a unique user ID that's used to retrieve a Twitter username from the database.

Upon retrieving the username, it will be used to load their Twitter/Facebook feed. This will then be displayed on the index page.

From the point of receiving data on the Arduino to the displaying of Twitter/Facebook feed, it has to be seamless and quick.

4

hahaashton:
From the point of receiving data on the Arduino to the displaying of Twitter/Facebook feed, it has to be seamless and quick.

That sounds like a PHP problem rather than an Arduino problem. I don't know PHP (I got cheesed off with it years ago) - I have used Ruby and now prefer Python.

You have not said where the index page exists - presumably that is also produced on the server by PHP.

...R

5

This is what I understand :

  1. The Arduino sends data to a remote server using Wi-Fi
  2. The server performs computations
  3. If someone is watching the website at that time, the page will change instantly to reflect the computations, without the need for the user to refresh the page

What is the maximum delay you want between the Arduino's request and the page change ?

6

mathers:
3. If someone is watching the website at that time, the page will change instantly to reflect the computations, without the need for the user to refresh the page

Then the page the user is looking at must ask the server to update the page. The update cannot be initiated by the server.

There is a technique called "long polling" which is easy to use. It just means that the page makes a request to the server but the server will not respond until new data arrives (or perhaps until some timeout expires). Because this happens in the background it gives the viewer the impression of an immediate update.

My EzScrn demo uses the techique.

The alternative to long polling is for the webpage to request an update at regular intervals. Then the server can respond immediately (in the traditional way) with whatever data it has at that time. However this probably means a lot of unnecessary calls to the server.

In saying all this I am assuming you know how to use Javascript and Ajax to make requests to the server to update the data on a page without needing to reload the whole page.

...R

7

Might MQTT be a solution? - Scotty

8

Yes, mathers and Robin2, both of you are right. The "immediate update" has to be done within a matter of 1 to 3 seconds. The shorter the better of course!

However, I'm really new to PHP that's why I need step by step guidance as to how to approach my idea. (I do java programming for the most part)

The big problem that I'm facing now is actually is the executing of one function after another.

Let's say after I've made a successful query from the database, I wanna use the Twitter username in the function to call for Twitter's API and then display the results on the index page

9

HTTP Plugin for MySQL:

Old method:

HTTP Client->Web Service (e.g. PHP script)->Apache (Web server)->Mysql

New HTTP Plugin for MySQL method:

HTTP Client->HTTP Plugin->Mysql

HTTP Plugin in same thread of Mysql server and stanard API plus HTTP Plugin wrote at C, a way faster and cleaner.

10

sonnyyu:
HTTP Plugin in same thread of Mysql server and stanard API plus HTTP Plugin wrote at C, a way faster and cleaner.

Nah .. Python.

Getting a response from a server in "1 to 3 seconds" does not require any fancy coding. Most server code will respond in small fraction of a second.

I am assuming, of course, that you will only have one (or a few) users viewing the web page that shows the Arduino data. If you are thinking of a web project for thousands of users that is well beyond the scope of this Forum.

...R

11

Robin2:
Then the page the user is looking at must ask the server to update the page. The update cannot be initiated by the server.

Yes it can - if you use web sockets.

Basically, the page would establish a web socket connection with the server; this necessitates the need to set up a web socket server in PHP (presumably what the OP wants to do - personally, I would use Node.js) - here's one of many tutorials:

Creating Real Time Applications with PHP and WebSockets

So - you would have this index.php page sitting on a server; the Arduino would make a request to it via WiFi (personally, I would make it a GET request, rather than a POST - unless there is some special reason a POST request is wanted).

The PHP program would use that data to go out and query facebook, twitter, etc - another tutorial:

Build Your First Twitter App Using PHP in 8 Easy Steps

Some processing is done, then the page would send the data via the opened websocket to the leaderboard browser.

Now - you would either need to make this index.php page smart, and have it somehow know when it is talking to the Arduino vs when it is establishing a websocket connection to the leaderboard browser - or you would need two separate page (and somehow communicate the websocket and session information between them) - but ultimately both things can be easily done.

The leaderboard page would be basically a pure DHTML and AJAX style page served up by the server; the web browser would render the output, and establish the websocket connection with the server; this socket stays open, waiting for receive events (when the server sends data over the open websocket).

Your response time will likely be most dependent on how fast you can get the data from your feeds and parse it; the websocket connection itself will be really fast (I once created a simple demo where I served up a WebGL cube on a browser, and established a websocket connection back to the server for rotation information; then I had my browser on my phone connect and get a page for sending the orientation sensor values over a websocket - the server just basically piped the data thru to the WebGL page - it worked extremely well - in theory, I could have possibly done away with the server, but there were firewalls in the way preventing that).

Hopefully - that all helps.

12

cr0sh:

The update cannot be initiated by the server.

Yes it can - if you use web sockets.

I have not tried to use web sockets - I have not had anything I need to do that would require that level of complexity.

However I'm guessing that web sockets can only work with the permission/collaboration of the browser PC. You can't write a web server program that can send data to my PC without my permission (I hope).

...R