How to control LED using Arduino as a webclient? [solved]

Simple client test code. If desired you can capture or analyse the returned string of data for what you are looking for in the returned data.

//zoomkat 9-22-12
//simple client test
//for use with IDE 1.0.1
//with DNS, DHCP, and Host
//open serial monitor and send an e to test
//for use with W5100 based ethernet shields

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address

char serverName[] = "web.comporium.net"; // zoomkat's test web page server
EthernetClient client;

//////////////////////

void setup(){

  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }

  Serial.begin(9600); 
  Serial.println("Better client test 9/22/12"); // so I can keep track of what is loaded
  Serial.println("Send an e in serial monitor to test"); // what to do to test
}

void loop(){
  // check for serial input
  if (Serial.available() > 0) //if something in serial buffer
  {
    byte inChar; // sets inChar as a byte
    inChar = Serial.read(); //gets byte from buffer
    if(inChar == 'e') // checks to see byte is an e
    {
      sendGET(); // call sendGET function below when byte is an e
    }
  }  
} 

//////////////////////////

void sendGET() //client function to send/receive GET request data.
{
  if (client.connect(serverName, 80)) {  //starts client connection, checks for connection
    Serial.println("connected");
    client.println("GET /~shb/arduino.txt HTTP/1.0"); //download text
    client.println("Host: web.comporium.net");
    client.println(); //end of get request
  } 
  else {
    Serial.println("connection failed"); //error message if no client connect
    Serial.println();
  }

  while(client.connected() && !client.available()) delay(1); //waits for data
  while (client.connected() || client.available()) { //connected or data available
    char c = client.read(); //gets byte from ethernet buffer
    Serial.print(c); //prints byte to serial monitor 
  }

  Serial.println();
  Serial.println("disconnecting.");
  Serial.println("==================");
  Serial.println();
  client.stop(); //stop client

}
1 Like

Thanks a lot for your reply zoomkat. I tried your code and it worked by receiving the HTTP header information, but I still don't understand how to interact with the variable in the webserver, and that is where my real problem is. :frowning:
So, the idea is that the wifi Shield connects to the webserver(which it does), and just waits until someone presses the button on the website which is hosted in the webserver. At that moment, the value of the variable changes. Since the wifi shield is permanently connected and waiting for incoming data, it receives the variable name and a value for that variable. According to this, the Arduino turns on or off a LED, and replies back to the webserver information, so that in case somebody else goes to the website knows the status of the LED.
Therefore, I think my rpoblem is I don't know how to write the commands for this to work out. I have used the GET and POST commands trying to access the variable created in the PHP segmente that takes cares of, but I get nothing, and the same goes on how to send information to the webserver in order to change the value of the variable.
Again, any help will be greatly appreciated! :slight_smile:

So, the idea is that the wifi Shield connects to the webserver(which it does), and just waits until someone presses the button on the website which is hosted in the webserver.

That is not how web servers and web clients normally work.

I have something similar, yet very different running that accomplishes much the same thing. I have a web server running on the arduino and it is contacted directly to report status and change the state of various things. For example, it knows if my garage door is open or closed and presents a button to change the state of the door. If the user clicks the button, it opens(closes) the door and reports back that it did. This is a much easier approach than the interaction you describe.

What Zoomkat said.

A more traditional web approach would be to have the wi-fi shield listening as an HTTP client and have the web server POST an update to the wi-fi shield when the buttons are clicked. External IP addressing and firewall issues you will run into on this path are well-covered in the forum if you search a bit.

If you want a persistent connection look into "telnet". But it would require special code on the server side.

-br

Maybe this will help?

Thanks again for your response zoomkat. I guess I'll have to investigate a little more, I thought that's how it worked.

draythomp, the functionality I'm looking for is exactly the same, but the approach is different, as I don't want the Arduino to act as a webserver, only as a client and depending on what the webserver sends it respond accordingly.

billroy, that's how I thought it could work (using the traditional approach you mention). I want the Wi-fi Shield to be an HTTP client who just listens until a there's a POST update from the webserver when somebody clicks on the button that's on the webpage, and of course the Arduino turns on or off a LED and POST a message back to the webserver server letting it know it turned on or off the LED. The issues you mention regarding IP addressing and firewall I already had encountered them, and had done some IP Forwarding, but my ISP has the ports blocked, I tried using a DNS service but had the same results. That's why I am dealing with a free web hosting service and taking the approach that the wifi shield only acts an HTTP client and not a webserver. I had already made a program to do what I want in my LAN and it worked fine, but I want this to be part of the WAN, so I can access it on the road and know the status of the LEDs and later on from analog sensors XD. I'll dig some info on the "telnet" stuff.

Kong, I am looking at the link right now, it looks very interesting, but it uses the Arduino as a webserver.

Thanks everybody for all your feedback and suggestions. It seems that I am not really seeing the big picture :blush: or am confused about some stuff and I am just complicating my life.

Again, all suggestions/comments are more than welcome . :wink:

The functionality you are after would in theory require websockets, but you will probably have do with polling.

On Arduino side you will have a loop that will keep making HTTP GET reguests to a url, lets say 192.168.0.11/ledstatus.php. The "192.168.0.11" is the IP address of your web server.

The ledstatus.php shoud serve some kind of responce. In your case it's enough to respond with a number 0 or 1 for on and off state. Read and parse the responce on Arduino and set the led on or off accordingly. The ledstatus.php should read the state from a txt file. It could be ledstatus.txt. Test the ledstastus.php with your browser to make it sure it works the way it should be.

On the server running on the ip 192.168.0.11 you will have php page containing a html form, lets say index.php. The form should have two radio buttons with options: "on" and "off", and button to submit the form. When the index.php page loads set the correct radio button selected according to value saved to a ledstatus.txt using php. Point the form action to formhandler.php.

The formhandler.php should first read the form parameter then save it to the ledstatus.txt file and then point the browser back to the index.php. You can also combine the formhandler.php and index.php into one file if you first check if the form parameter has been set and save the state if it is found.

Somebody jump in here and correct me if I'm wrong, but:

The traditional way web clients and servers work is that the server does work for the client at the clients request. So, the client sends something, the server does some work and provides back and update; like google does for search. The client traditionally doesn't do work for the server. This means that the client initiates the conversation by sending an HTTP request to the server. This is the problem you're trying to work around.

I see an easy way to do this, just set the arduino up as a client and have it periodically send a request to the server to see if something needs to be done. This is what the other poster is talking about when he mentioned polling. See, nothing happens unless the client sends a request that the server can....well...serve. Clients don't have a continuous connection to the server, that's a different protocol (also mentioned by another poster)

Unless I read it wrong, you want to reverse the process and have the client be the servant. That can be done, but only if the client intiates the conversation. Sort of: "Hey, server, is there any work to be done?" "Yep client, turn the light on." That kind of thing. Also, this is a little arduino with very limited capabilities when compared to a web server with all the supporting software they carry. So, you don't have php, java, or any of that stuff to do work for you. If you want to get a variable, you have to do it yourself in the arduino code. This is normally done by constructing a string that indicates what you did and sending it back to the server.

Does this help?

draythomp:
Also, this is a little arduino with very limited capabilities when compared to a web server with all the supporting software they carry. So, you don't have php, java, or any of that stuff to do work for you. If you want to get a variable, you have to do it yourself in the arduino code. This is normally done by constructing a string that indicates what you did and sending it back to the server.

Actually you can do quite a lot if you have an Arduino Mega 2560. I have been working on this lately and wrote a quick seminar paper for school about the subject. You can read the online version here http://www.cs.helsinki.fi/u/ljlukkar/iot/ but as it has been already two months since I wrote that the examples are out dated. How ever I have updated all the examples to GitHub to work with the current versions of the libraries.

All of the following examples also contain a link to dummy UI where you can try out how the UI works. If you are using Chrome open up the developer tools and pay attention what happens on the Network tab. None of these should be thought as real "projects" but instead of just demos. I haven't actually tested do even some of them work properly on all situations and the code is crappy.

The most simple one:
https://github.com/lasselukkari/Todo

Most advanced:
https://github.com/lasselukkari/Arduino-RC-Timer-Switch/

Random:
https://github.com/lasselukkari/Weather-Station
https://github.com/lasselukkari/PinToggle

PS. I just realized I spent many hours reading your blog after stumbling across it while googling yesterday.

There is also another option.
I have been working on using a router to do the web work and send serial commands to the Arduino.
This has the benefit that controlling the Arduino is done in exactly the same way over the serial monitor as with the web. Except for that the web is sending the serial commands in your place.
The nice thing about this solution is that I have written a parser that creates a HTML page based on info provided by Arduino. So changing the sketch on the Arduino changes the web interface eventhough Arduino does not service the web interface 8)
I succeeded in uploading a new hex file from a HTML page without having to reset the Arduino as well.
So this goes for a fully remote controllable Arduino.
The following demo is not impressive at all but it shows a hacked Linksys router which is connected to Arduino and has 1 tcpIP cable for the webcam that recorded the video. So this setup is controlled completely wireless ]:smiley:
The Arduino is doing a ++ on Field_11 and that is why you see the Value of Field_11 jumping up and down (it is a overrunning integer)
The field led_13 is used to set the led when the field is changed.
I don't know how to embed video so here is the direct link to my web server
http://www.baeyens.it/Media/Arduino%20web%20interface.wmv
The icons indicate the edit-ability of the field. This is because some fields can not be changed and some fields can be stored in eeprom.
I hope to do some writeup on this solution some day.
Best regards
Jantje

Actually, it's possible to get an arduino to do almost anything on the web. Ever since plugins became possible (yes, I know it was years and years ago) we could simply 'include' source from an external source. Just using an iframe we can put an entire web page on the browser with only a few bytes on the arduino. Heck, we could put almost everything on a web server, and a little src=http://somebigserver/mypage to get it on the user's browser.

I use this little trick to get cool gauges and really big graphs on my little arduino web page.

Don't you just love distributed processing?

draythomp:
Actually, it's possible to get an arduino to do almost anything on the web. Ever since plugins became possible (yes, I know it was years and years ago) we could simply 'include' source from an external source. Just using an iframe we can put an entire web page on the browser with only a few bytes on the arduino. Heck, we could put almost everything on a web server, and a little src=http://somebigserver/mypage to get it on the user's browser.

I use this little trick to get cool gauges and really big graphs on my little arduino web page.

I'm not sure was this comment for me. If it was, did you notice on my solution Arduino is doing a full CRUD (create, read, update delete) using a JSON Restinterface pretty much atomatically? It is not just loading files from the Internet to the browser.

Here is your code. I had to install new LAMP set up and while making sure everything works I wrote the example for you:

Arduino sketch modified from Ethernet WebClient example:

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = { 
  0x00, 0xAA, 0xBB, 0xCC, 0xDE, 0x02 };
IPAddress server(192,168,0,12); // Your webserver IP

// Initialize the Ethernet client library
// with the IP address and port of the server 
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

void setup() {
  // start the serial library:
  Serial.begin(9600);
  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    for(;;)
      ;
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);

}

void loop()
{
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  if (client.connect(server, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /led/ledstatus.php HTTP/1.0");
    client.println();
    delay(1000);
    char c;
    // go trough the responce. as the value we are looking for is in the end of the responce after this loop char c will contain the last char of the respoce, 0 or 1
    while (client.available()) {
      c = client.read();
    }
  
    // if char c converted to an integer is somethning else than 0
    if (atoi(&c)) {
          Serial.println("Led should be on");
          //turn the led on here
        } 
        else{
           Serial.println("Led should be off");
           //turn the led off here
        } 
   
  } 
  else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
  }
  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println("disconnecting.");
    client.stop();
  }
  delay(5000);
}

index.php

<?
if (isset($_POST['status'])) {
    $fh = fopen("ledstatus.txt", 'w') or die("can't open file");
    $on = $_POST['status'];
    fwrite($fh, $on);
    fclose($fh);

} else {
    $fh = fopen("ledstatus.txt", 'r');
    $on = fread($fh, 1);
    fclose($fh);
}
?>
<!DOCTYPE HTML>
<html>
<body>
<form action="index.php" method="post">
    <label> <input type="radio" name="status" <?if ($on) {echo 'checked = "checked"';} ?> value="1">On</label>
    <label> <input type="radio" name="status" <?if (!$on) {echo 'checked = "checked"';} ?> value="0">Off</label>
    <input type="submit" value="Submit">
</form>
</body>
</html>

ledstatus.php

<?
    $fh = fopen("ledstatus.txt", 'r');
    echo fread($fh, 1);
    fclose($fh);
?>

That's the kind of thing I was talking about, get the server to do the work. I didn't see where the arduino actually turns the light on and off, but that would be relatively easy to do.

What I was talking about was doing something similar where you don't have control of the server. You can put files on a file serving cloud machine, but the server belongs to someone else. It's still possible to do huge amounts of work using the viewer's browser.

However, I'm stepping way aside from the OPs original question and hijacking the thread.

:blush: Sorry for the late response. :blush:

Illukkari from your posts and comments, you're right on the point of where I would like to get at. Thank you very much for the code. I will try the code and come back with some comments.

Jantje that's another neat approach I've read about. However, my idea is to automate my house as much as possible, so it won't be so easy to make the serial connection.

draythomp My idea is that the server and client(s) con communicate with each other, being the internet server the one who controls the arduino wifi clients. As Illukkari mentioned the best approach is using websockets. From what I've read polling causes an increase of the use of bandwidth which could affect other users, and it's not in real time so, it is still an option but not the best. As for hijacking the thread, don't worry, it is a constructive discussion, from were we all learn. I am new to Arduino and HTML programming so it has been very interesting for me.

Cheers! XD

You're absolutely right in your concerns about real time and traffic. The traffic concern is a bit over rated though. The kind of traffic most automation work uses is far, far less than the traffic from a video download, so the little bit one does for automation doesn't matter. And, polling does put the action somewhat behind the event that needs to be acted upon. This CAN be a problem in some circumstances, so I understand your concern.

There is a websocket library out there for the arduino that could be used as a starting point for what you want to do. If it turns out that using a web interface is unreliable or too much trouble, you can go back to the idea of a serial connection and use any of the little radios that are talked about on this forum as a serial connection between devices.

Have fun.

ardufan:
From what I've read polling causes an increase of the use of bandwidth which could affect other users, and it's not in real time so, it is still an option but not the best.

You are polling the server with a 8 bit 16 MHz machine. Even if you would have hundreds of them looping as fast as they can the web server will not be your bottleneck. If it is, drop the HTTP or even the TCP because the headers are most of the messaging.

YESSSSSS !! :%

llukkari You're the man!!

Thank you everybody for all your help. Now...to practice a little more.

P.S.: draythomp thanks also for your suggestions. :slight_smile: