Arduino as web server/client?

Hi ill first explain what im trying to acheive and then where im at. So what im trying to do is to have an arduino sat somwhere remote where there is no net connection. Therefore I have an arduino+ethernet shield attactched to a GSM router which is connected to my openVPN giving me a duplex (bi-directional) socket/tunnel (without the need for fixed IP sim card) to the arduino and all connected openVPN clients. The arduino is to remotly log data via rs232/485 (not got there yet) and send these to a php page (via a get/post) which stores them into a mySql DB, but only when asked to...

So what i really want is the arduino sat there..waiting, for me to be able to goto a webpage and view data live (as live as it can be, only a few bytes), then set it to log every n number of mins, which would then tell the arduino to do this untill i log back in and tell it to stop?

So far i have tried using websockets, but from what pylon (on here) and i have found out is that GSM networks use a proxy that doesnt yet seem to suport websockets, so thats currently a no go.

So now im thinking of running the arduino as a webserver, ive done this easily using the standard library and the webduino, works fine with the GSM and openVPN. When i go to the IP (openVPN ip) it loads the page, so that works nice (no fixed IPs needed ETC), but what if i want to leave the arduino logging data to a server? It then needs to become a client to do get requests?

So how do i go from server to client to log, but when its logging i need to be able to gain access to stop it logging?

Any ideas?

Thanks.

Andy

If you already have a mySQL database set up, you might want to use the Arduino as a web client, connecting periodically to the php/mySQL server. This is the code I have tested to save variables in the mySQL database, both for the Arduino and the server.
http://arduino.cc/forum/index.php/topic,124289.0.html

You could use the database as a relay. You could have a field in a mySQL table that determines what to send to the Arduino when it "phones home". The Arduino would parse the response for commands from the database that were entered by your webpage (not the same page as the Arduino uses).

I thought about doing it this way. which to me sounds like a "polling" system. Things that crop to mind are:

What heppens when its not loggin? It still means it needs to keep polling the php page to see if any flags have been changed, surely this causes fustrating delays (between me setting flag and waiting for arduinos next connection) and massive waste on bandwidth?

Example bad scenario: i decide i dont need it logging for 3 months, but the arduino still has to connect to php which has to query the DB, return & check data, dispose of.... every 1 - 2mins for 3 months?

Surely there must be a more efficient way?

As a server it would sit there doing nothing, waiting for a connection to come in...

Thanks
Andy

Welcome to the Wonderful World of Servers! This code has been tested well.

edit: This comes with its own bag of worms. Unless you use some type of security to "log in", anyone could enter stuff there. And there are some people on the internet would do that just to see what happens.

Could you explain more about the post you have sent me to? Ive looked through the code but fail to see how it helps?

I have servers all over the shop, setting up the server isnt a problem as thats my job.

The arduino is the newish bit to me, im confused to how i can have it as a server but for it to also do http requests.

Thanks
Andy

It will do both, but it requires some timing in the program. It has challenges due to the lack of sockets in the w5100 (4). If the server is using all four sockets for inbound requests, you will not be able to connect to a server as a client. No socket available error.

If you want to see how it works, here is a sketch that is a web server and sends email as a client. You would need to change the email send to a html page request.

I see, i have it working and it deffo seems to be doing what i want.

Will have to have a good read through the code and get my head around it.

When a client connects, is that 1 socket used up??

Thanks
Andy

What may help you understand it is the local declarations of

Client client;

One declaration is in loop, the other is in sendEmail.

You can also use two global declarations, like

Client sclient;
Client cclient;

Then use those in the appropriate places.

Ok, but im still a little confused to when 1 of the 4 sockets are used up?

If i set the arduino going and then i connenct using my browser it downloads the page into my browser. Is the socket still connect and being used or once the page finshes loading does the socket become free?

thanks
Andy

When a client connects to the server, one socket.
When another client connects before the server has finished with the previous, another socket.
When you connect to a server as a client, another socket.
That is three sockets used. One remaining.

edit: My bad. I forgot the second question.
As each connection is closed properly, the socket is freed up for another connection. This applies to both server clients and client clients.

This is why I already have the library code ready for the w5200. It has 8 sockets.

Morning, thanks for the help, i understand that now.

One thing im still unsure of is using the code you led me to here: Server on Arduino Ethernet fails - #27 by SurferTim - Networking, Protocols, and Devices - Arduino Forum

how does that act a server and client. For instance, I can see how that can see me connect as a client and understand a command i give. But what i i want to send it a command and then the arduino to process that and start sending data to another page on a server, not back to the client?

Essentially what im trying todo is to be able to send a command to the arduino, the arduino sets a flag and starts sending data to a URL, NOT back to the client.

Thanks
Andy

That code has no client section, just server. This is the server/client code.

Some web client/server test code that I was tinkering with. You test the client function using the serial monitor, and use the serial monitor to see the get request sent to the server from the server web page.

//zoomkat 7-03-12, combined client and server
//simple button GET with iframe code
//for use with IDE 1.0
//open serial monitor and send an g to test client and
//see what the arduino client/server receives
//web page buttons make pin 5 high/low
//use the ' in html instead of " to prevent having to escape the "
//address will look like http://192.168.1.102:84 when submited
//for use with W5100 based ethernet shields
//note that the below bug fix may be required
// http://code.google.com/p/arduino/issues/detail?id=605

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //assign arduino mac address
byte ip[] = {192, 168, 1, 102 }; // ip in lan assigned to arduino
byte gateway[] = {192, 168, 1, 1 }; // internet access via router
byte subnet[] = {255, 255, 255, 0 }; //subnet mask
EthernetServer server(84); //server port arduino server will use
EthernetClient client;
char serverName[] = "web.comporium.net"; // (DNS) zoomkat's test web page server
//byte serverName[] = { 208, 104, 2, 86 }; // (IP) zoomkat web page server IP address

String readString; //used by server to capture GET request 

//////////////////////

void setup(){

  pinMode(5, OUTPUT); //pin selected to control
  pinMode(6, OUTPUT); //pin selected to control
  pinMode(7, OUTPUT); //pin selected to control
  pinMode(8, OUTPUT); //pin selected to control

  //pinMode(5, OUTPUT); //pin 5 selected to control
  Ethernet.begin(mac,ip,gateway,gateway,subnet); 
  server.begin();
  Serial.begin(9600); 
  Serial.println("server/client 1.0 test 7/03/12"); // keep track of what is loaded
  Serial.println("Send an g in serial monitor to test client"); // what to do to test client
}

void loop(){
  // check for serial input
  if (Serial.available() > 0) 
  {
    byte inChar;
    inChar = Serial.read();
    if(inChar == 'g')
    {
      sendGET(); // call client sendGET function
    }
  }  

  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();

        //read char by char HTTP request
        if (readString.length() < 100) {

          //store characters to string 
          readString += c; 
          //Serial.print(c);
        } 

        //if HTTP request has ended
        if (c == '\n') {

          ///////////////
          Serial.print(readString); //print to serial monitor for debuging 

            //now output HTML data header
          if(readString.indexOf('?') >=0) { //don't send new page
            client.println(F("HTTP/1.1 204 Zoomkat"));
            client.println();
            client.println();  
          }
          else {   
            client.println(F("HTTP/1.1 200 OK")); //send new page on browser request
            client.println(F("Content-Type: text/html"));
            client.println();

            client.println(F("<HTML>"));
            client.println(F("<HEAD>"));
            client.println(F("<TITLE>Arduino GET test page</TITLE>"));
            client.println(F("</HEAD>"));
            client.println(F("<BODY>"));

            client.println(F("<H1>Zoomkat's simple Arduino 1.0 button</H1>"));

            // DIY buttons
            client.println(F("Pin4"));
            client.println(F("<a href=/?on2 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off3 target=inlineframe>OFF</a>

")); 

            client.println(F("Pin5"));
            client.println(F("<a href=/?on4 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off5 target=inlineframe>OFF</a>

")); 

            client.println(F("Pin6"));
            client.println(F("<a href=/?on6 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off7 target=inlineframe>OFF</a>

")); 

            client.println(F("Pin7"));
            client.println(F("<a href=/?on8 target=inlineframe>ON</a>")); 
            client.println(F("<a href=/?off9 target=inlineframe>OFF</a>

")); 

            client.println(F("Pins"));
            client.println(F("&nbsp;<a href=/?off2468 target=inlineframe>ALL ON</a>")); 
            client.println(F("&nbsp;<a href=/?off3579 target=inlineframe>ALL OFF</a>

")); 

            
            
                      // mousedown buttons
          client.println(F("<input type=button value=ON onmousedown=location.href='/?on4;' target=inlineframe>")); 
          client.println(F("<input type=button value=OFF onmousedown=location.href='/?off5;' target=inlineframe>"));        
          client.println(F("&nbsp;<input type=button value='ALL OFF' onmousedown=location.href='/?off3579;' target=inlineframe>

"));        
                   
          // mousedown radio buttons
          client.println(F("<input type=radio onmousedown=location.href='/?on6;' target=inlineframe>ON</>")); 
          client.println(F("<input type=radio onmousedown=location.href='/?off7; target=inlineframe'>OFF</>")); 
          client.println(F("&nbsp;<input type=radio onmousedown=location.href='/?off3579;' target=inlineframe>ALL OFF</>

"));    
   
          
          // custom buttons
          client.print(F("<input type=submit value=ON target=inlineframe style=width:100px;height:45px onClick=location.href='/?on8;'>"));
          client.print(F("<input type=submit value=OFF target=inlineframe style=width:100px;height:45px onClick=location.href='/?off9;' target=inlineframe>"));
          client.print(F("&nbsp;<input type=submit value='ALL OFF' target=inlineframe style=width:100px;height:45px onClick=location.href='/?off3579;' target=inlineframe>"));

            
            client.println(F("<IFRAME name=inlineframe style='display:none'>"));          
            client.println(F("</IFRAME>"));

            client.println(F("</BODY>"));
            client.println(F("</HTML>"));
          }

          delay(1);
          //stopping client
          client.stop();

          ///////////////////// control arduino pin
          if(readString.indexOf('2') >0)//checks for 2
          {
            digitalWrite(5, HIGH);    // set pin 5 high
            Serial.println("Led 5 On");
            Serial.println();
          }
          if(readString.indexOf('3') >0)//checks for 3
          {
            digitalWrite(5, LOW);    // set pin 5 low
            Serial.println("Led 5 Off");
            Serial.println();
          }
          if(readString.indexOf('4') >0)//checks for 4
          {
            digitalWrite(6, HIGH);    // set pin 6 high
            Serial.println("Led 6 On");
            Serial.println();
          }
          if(readString.indexOf('5') >0)//checks for 5
          {
            digitalWrite(6, LOW);    // set pin 6 low
            Serial.println("Led 6 Off");
            Serial.println();
          }
          if(readString.indexOf('6') >0)//checks for 6
          {
            digitalWrite(7, HIGH);    // set pin 7 high
            Serial.println("Led 7 On");
            Serial.println();
          }
          if(readString.indexOf('7') >0)//checks for 7
          {
            digitalWrite(7, LOW);    // set pin 7 low
            Serial.println("Led 7 Off");
            Serial.println();
          }     
          if(readString.indexOf('8') >0)//checks for 8
          {
            digitalWrite(8, HIGH);    // set pin 8 high
            Serial.println("Led 8 On");
            Serial.println();
          }
          if(readString.indexOf('9') >0)//checks for 9
          {
            digitalWrite(8, LOW);    // set pin 8 low
            Serial.println("Led 8 Off");
            Serial.println();
          }         

          //clearing string for next read
          readString="";

        }
      }
    }
  }
} 

//////////////////////////
void sendGET() //client function to send and receive GET data from external server.
{
  if (client.connect(serverName, 80)) {
    Serial.println("connected");
    client.println("GET /~shb/arduino.txt HTTP/1.0");
    client.println();
  } 
  else {
    Serial.println("connection failed");
    Serial.println();
  }

  while(client.connected() && !client.available()) delay(1); //waits for data
  while (client.connected() || client.available()) { //connected or data available
    char c = client.read();
    Serial.print(c);
  }

  Serial.println();
  Serial.println("disconnecting.");
  Serial.println("==================");
  Serial.println();
  client.stop();

}