Controlling Arduino from an exernal (nice) webpage

Projects General Guidance
1

Dear friends, I’m new in the amazing Arduino world, and I’m learning a lot from you all!!!

But finally I’m here because I want to control an Arduino+Ethernet board from a standard hosted webpage (not Arduino hosted web, not using Cosm, not using Teleduino…) I’d want something like this:

Arduino <--> My (nice) web page hosted in an Internet server <--> User

So, the Arduino would run a code to communicate with the web (interface) and there would be simply functions (EG void setAlarmTime(int hour, int minutes); ) so that the web page could make the Arduino to execute them passing the appropriate parameters provided by the user via the said web. There also would be functions like int getRoomTemperature(); to send info from the Arduino to the web. The user would log in the web page, so then he/she can access to a menu where he/she could manage the things controlled by the Arduino.

I’m been looking for stc like this but I’ve seen anything solving it this way, just the well-known method (Serving limited web pages from arduino, cosm…) that’s because I’m asking for help (maybe I didn’t search well) because I’m pretty lost in this subject and I hope you could give me some clues.

THANK YOU IN ADVANCE!!!!

2

So, the Arduino would run a code to communicate with the web (interface)

There is a server somewhere. It has been contacted by your browser, and has supplied the browser with some data to render. How do you then expect the Arduino to get that rendered data and make sense of it?

You have a fundamental misunderstanding of client/server architecture and the role that web browsers (clients) play in that scenario.

3

Because of the client server issues PaulS mentions, you can't do precisely what you're asking.

However, since all you want is a web site that the user can control/monitor the arduino's outputs and sensors, there are other ways to do it. If you want to stay strictly web, create a page for the user where they can observe readings and press buttons to tell the arduino what to do. Have the web server store these instructions and create another page for the arduino to poll periodically to send its readings and pick instructions up.

If you want the arduino to respond faster, write something on the web server that communicates with it over UDP or TCP.

4

I think you have two options.

One is to have the client contact the Arduino. Your pretty web site would serve out a page that contained script and HTML that causes the browser to access the Arduino when required. For example, you could use AJAX-based approaches, although there are plenty of other ways available.

The other is to have the web server contact the Arduino. In this approach your web app or server-side scripting would contact the Arduino (for example using HTTP) set send commands to it or retrieve status data from it. Conceptually this is very similar to having the web server access a database as a result of an HTTP request.

5

Hey, thank you for ur fast repply!!

Huh! well I think I am even more lost than I thought!! :S

PauS; I was thinking my web page could send a packet to the Arduino, then the Arduino would execute a function depending in what the web sent. Is that not possible?

wildbill; more accurately let’s say I want my web page be able to call Arduino functions (previously coded on the Arduino) , not necessarily just monitor the inputs directly (as COSM). When you say “create another page for the arduino to poll periodically to send its readings and pick instructions up” do you mean a page stored in the Arduino memory?

So for example, if I want to know the room temperature, the user would go to my web page and would click a button, then the web should sent a TCP/UDP packet to the Arduino in order this to execute my getRoomTemperature() function. Do I understood well? And how can I do this (The TCP or UDP communication between the web and the arduino), do you know where can I find some examples?

THANK YOU!!

6

jmdiscovery:
And how can I do this (The TCP or UDP communication between the web and the arduino), do you know where can I find some examples?

I've suggested two approaches that are open to you. Either would work. Having your web app contact the Arduino would give you more control over the load on the Arduino, which might be an important factor.

7

Here is an example of the second type. You would need to have an apache web server or some such on your PC that is connected to the arduino.
http://schmidtcds.com/2011/01/arduino-home-automation-part-2/

8

It can certainly be done. Did you google it? Here's one simple example:

http://bildr.org/2011/06/arduino-ethernet-pin-control/

In this case the arduino acts like a web server, and does stuff when a certain URL is requested. Following this example, all you'd have to do is create a web page on your remote web server that builds the needed command links and either sends the request for the client or displays the link to the client for them to click. There are other examples floating around showing different ways to accomplish what you want. You just need to get a better idea of exactly how you want it to work.

9

laadams85:
Here is an example of the second type. You would need to have an apache web server or some such on your PC that is connected to the arduino.
http://schmidtcds.com/2011/01/arduino-home-automation-part-2/

hi- :slight_smile:

having tried/attempted this myself.. I'll chime in on my results..

I did not/do not have an ethernet enabled Arduino.. (to be clear)...

so my approach had nothing to do with hosting a page on the Arduino.. or having the Arduino 'connected' to the internet (per se`)

My question(s) to myself were if I could LOCALLY send data to my Arduino to control some components.... using PHP/HTML...etc

I created my 'web interface' (this consisted of HTML a FLASH GUI/Interface (just to make it fancy.. regular HTML form buttons would work the same)..

this is my 'front end'.... but gives me "NOTHING" to communicate to the Arduino with.. so I needed a 3rd party/intermediate script/language to grab my data.. and push/send it over to the Arduino..

I chose PHP... because my background is more web based than anything else....

I have a Windows?PC platform.. and installed WAMP server (Which is webserver that installs Apache, MySQL and PHP..etc on your machine)

I have my Arduino connected to my PC (correct board/port set)
the sketch uploaded to it is just a simple sketch that grabs the incoming serial data, parses in.. and changes the R, G, B PWM analog pins to whatever values were sent over.

The flash QUI grabs the values/data that are entered.. sends this data to the php script.. that in turns send this data to the comm port that the Arduino is listening to...

this all has worked out fine (link to project & source files in footer)...

HOWEVER...

1.) this is only accessible from your LOCAL COMPUTER running WAMP server..... if you want this to be accessible from the OUTSIDE world (internet).. then you would need to go to your ROUTER, log-in and forward any HTTP traffic to the IP/computer that is running your WAMP server.

2.) Being on a "WINDOWS" platform... I am in the understanding that this type of approach/communication is only '1-way'... the Arduino can NOT data back and have the PHP script grab it..and update the 'page'... I believe this to be a PHP 'thing'.. related to Windows.. I have read using LINUX.. Arduino >> PHP will work fine..

just to discuss another approach...

here is original thread:
http://arduino.cc/forum/index.php/topic,124394.0.html

good luck!

10

Being on a "WINDOWS" platform... I am in the understanding that this type of approach/communication is only '1-way'... the Arduino can NOT data back and have the PHP script grab it..and update the 'page'... I believe this to be a PHP 'thing'.. related to Windows.. I have read using LINUX.. Arduino >> PHP will work fine..

I can't imagine that PHP can't do serial communications both ways. If not you should be able to use python. I used java to do 2-way serial communications fairly easily.

11

just what I've read (a few times).. :slight_smile:

figured Id pass it along..

If there 'is' a way.. please let me know/post the results.. (it'd be fun to make a simple web app and some relays to test some basic home automation projects)

I have ONLY tried/done FLASH/PHP to Arduino

12

I figured out a way to do this. Its certainly not robust yet, mostly cause I'm not familiar with websites and php, but this is what I did. Download serproxy onto your machine. Modify the cfg file

# Config file for serproxy
# See serproxy's README file for documentation

# Transform newlines coming from the serial port into nils
# true (e.g. if using Flash) or false
newlines_to_nils=true

# Comm ports used
comm_ports=1,2,3,4

# Default settings
comm_baud=9600
comm_databits=8
comm_stopbits=1
comm_parity=none

# Idle time out in seconds
timeout=300

# Port 1 settings (ttyS0)
net_port1=5331

# Port 2 settings (ttyS1)
net_port2=5332

# Port 3 settings (ttyS2)
net_port3=5333

# Port 4 settings (ttyS3)
net_port4=5334

Then make your create your php file

<?php
$fp = fsockopen("localhost", 5334, $errno, $errstr, 30);
if(!$fp){
    echo "$errstr ($errno)";
} else{
    fwrite($fp,"Hello Arduino");
    $endline=FALSE;
    $string="";
    $startline=FALSE;
    while($startline==FALSE){
        $character=fgets($fp,2);
        if(strstr($character,"%")){
            $startline=TRUE;
        }
    }
    while($endline==FALSE){
        $character=fgets($fp,2);
        if(strstr($string, "#")) {
            $endline=TRUE;
        }else if($endline==FALSE){
            $string.=$character;
        }      
        
    }
    echo $string;
    fclose($fp);
}


?>

And a test sketch for arduino

int count = 0;
String readString;

void setup()
{
  Serial.begin(9600);
  Serial.println("Completed setup");  
}

void loop()
{
  readString="";
  if(Serial.available()){
    while(Serial.available()){
      Serial.read();
      char c = Serial.read();
      readString+=c;
      delay(10);
    }     
    count=count + 1;
    Serial.print("%Hello Website: ");
    Serial.print(count,DEC);
    Serial.println("#");    
  }
}

I really wanted to use the proper fgets procedure but was having troubles. Mostly feof won't find the end of file with the serial port is still open. Anybody with more experience with this stuff should be able to figure it out. Hope this helps

while (!feof($fp)) {
    echo fgets($fp, 128);
}

EDIT: Slightly better test code

13

But finally I’m here because I want to control an Arduino+Ethernet board from a standard hosted webpage (not Arduino hosted web, not using Cosm, not using Teleduino…) I’d want something like this:

Arduino <--> My (nice) web page hosted in an Internet server <--> User

Pretty simple. You just put full url links to the arduino in the web page that is hosted on a different server somewhere. The below page is served from a main server account, but the video and pan/tilt control functions are on a pc at the house. Look at the page source to see the setup.

http://web.comporium.net/~shb/wc2000-PT-script.htm

14

Hey!

Thanks to all your opinions I think I'm starting shaping the architecture. It could go something like this: an Ethernet enabled Arduino connected (directly w/ static IP) to the Internet, then, it should run a (small) "web" on the Arduino acting just as interface to another (big) web hosted on the Internet which would send commands/receive status to/from the (small) Arduino web. (based on the second option from PeterH) so:

[Arduno functions <-> (small arduino web)] <-> INTERNET<-> (big web on an Internet hosting) <-> User

Do you think this architecture is appropriate/plausible?

If so, the second point is (I don’t have web background :S) which protocol/language should I choose in order to get the desired BI-directional communication? Java, PHP, AJAX, JSON… How should I do it? Any examples this way?

Thank you for your great support!!!

PS I want to connect the Arduino to the Interner without having any PC/server on the middle.

15

Bitlash Commander might be worth a look for your application. The code is available on GitHub here: GitHub - billroy/bitlash-commander: Web-based dashboard builder for Arduino.

There's a sketch for the Arduino side that comes with Bitlash (http://bitlash.net) called "ethernetcommander.ino". It knows how to receive and execute Bitlash commands over the 'net, and how to send notifications upstream to the server. You can customize the sketch for your application.

Then there's the Bitlash Commander server. It talks to the Arduino over usbserial or the 'net. The server serves web pages which are control panels whose controls and indicators talk to the Arduino. There's a graphical editor for the control panels, or you can roll your own in HTML/Javascript.

Commander is a node.js application that uses JSON to communicate with the Arduino and for real-time updates to the web browser content.

Good luck with your project,

-br

Edit: There's a test server (with no Arduino connected) you can play with here: http://bitlash-commander.herokuapp.com

1280×600 26.8 KB

16

I found some code in the feof manual page to handle feof blocking.

<?php
function safe_feof($fp, &$start = NULL) {
 $start = microtime(true);

 return feof($fp);
}

/* Assuming $fp is previously opened by fsockopen() */

$start = NULL;
$timeout = ini_get('default_socket_timeout');

while(!safe_feof($fp, $start) && (microtime(true) - $start) < $timeout)
{
 /* Handle */
}
?>
17

Heyy!

I’ve working on it, and until now I’ve only implemented the Zoomcat approach (communication over the URL), but this one only works in one way, so I can send commands from my hosted webpage to the Arduino. I would like to do something with AJAX so I could have bi directional communication. I was thinking to implement a web client on the Arduino that would connect to my hosted web page, and thru Javascript handle the request in some way, does someone know how to implement this?

Other way could be using UDP, but this have I big inconvenience because I nedd to open ports in wherever I want to connect the Arduino, and it cant not be hadeled with javasrcipt, maybe TCP would be much better…

Thank you for your support!

18

If I understand you correctly, I believe what you wish to do can be done.
Firstly, did you take a look at BitLash as suggested by BillRoy, as it might be a good and suitable solution for you?

I have spent the past months learning and developing a system where by my Arduino monitors my renewable energy system and makes it availible for viewing on the web.
What I do, is have the Arduino use a http POST, to send data to my hosted site where a php script either stores the data in a SQL database or in a file.
Then, when a web browser points to the host site, another php script retrieves the data from sql database or file and again uses the POST method to send it back to the client, the javascript on your computer.
You need to know what structure you wish to use to transfer the data, I use json format.
Javascript then displays it in the way you wish on your screen.

Then, for the other way, to send data back to the Arduino, you can use a reverse path, that is, using a server between you and the Arduino, or you can send directly back to the Arduino, which is what I have been doing at present. If you use a hosted site, the you can use php or whatever language of you choice and is available on your host site, eg. perl or python, to communicate data back to your Arduino. If you decide to go from browser to Arduino, then you more likely write in javascript or a cleint-side language to send the data back to your Arduino. Either way, your Arduino needs to be able to accept requests, so it needs to be set up as a server, I use http not UDP and use the WebDuino library to provide this functionality.

Data that I want to go back to the Arduino, I hide and I make it so this function is not available from the web-page normally, as in my case, it is not something I would normally want anybody to be able to do from the webpage. But you can make possible from the web page by using an authorisation system.

I am no expert, and still learning myself, with a lot of reading and trying things out and help from other clever programmers e.g. http://www.cs.helsinki.fi/u/ljlukkar/iot/.
The web page is basically all writen in javascript and uses backbone.js as the MVC (model view controller) to keep a lot of the ajax code tidy and give it better structure.
To have an idea, you could look at my devolpments in my signature.

19

does someone know how to implement this?

  • As a server: accepts Bitlash commands in the HTTP POST body, returns the text of whatever Bitlash prints out whilst executing the commands

  • As a client: POSTs asynchronous updates to the Bitlash Commander server or perhaps an Apache/PHP server as rockwallaby has laid out.

You asked about Ajax. Ajax is downstream of all the above, in the line that connects the browser(s) to the server. Your server listens for AJAX requests (another type of POST, usually) and returns update data to the javascript running in your browser. That javascript updates the web page it's running in by manipulating the DOM.

Commander, for exaple, uses Ajax to distribute control value updates to all the connected browsers.

-br

20

Hey!,

  • Yes, of course I've cheking BitLash, it’s really powerful, I was trying to code some example using the ethernetcomander.ino What I’d like to do is to be able to call my own functions coded in the Arduino as I’ve seen I’d just need to customize the runWebserver function in order to call my own functions, great! :slight_smile: Now, where I need some more help is in the Web part, billroy, do you have some simple example of a web to send/receive commands/info to/from the Arduino (I look at the herokuapp panels but they are quite complex for the basic needs of my webpage..)

  • rockwallaby, reading you I feel like what you say is exactly what I think I want!! But how did you implement all this?! :slight_smile: I’ve seen your website but everything related to the Arduino/web part will be written soon (u hope) hehe :D, can’t wait! Will you advance some of the code pls...?

I’m learning a lot here, THANK YOU for your support and patient!!