using GET function to poll server repeatedly

Hi,

I’m attempting to modify the Wifly_Webclient code to poll my server repeatedly to check for an updated value. The goal is to turn on/off a switch using my phone. For example “1” will represent the on position and “0” will represent the off position.

Currently, the code seems to print a the variable “c” value once and then retains that in memory and does not update. Could anyone make a suggestion how I can modify the code in order to poll the server?

Thanks

Andrew

// (Based on Ethernet's WebClient Example)

#include "WiFly.h"


#include "Credentials.h"


byte server[] = { 66, 96, xxx, xx }; // Google

//Client client(server, 80);

Client client("xxxxxxxxx.com", 80);

void setup() {

  Serial.begin(9600);

  WiFly.begin();

  if (!WiFly.join(ssid, passphrase)) {
    Serial.println("Association failed.");
    while (1) {
      // Hang on failure.
    }
    
  }
    Serial.println("Can u reach me ?");
    Serial.print("IP: ");
    Serial.println(WiFly.ip());
  

  Serial.println("connecting...");

//  if (client.connect()) {
  //  Serial.println("connected");
   // client.println("GET http://xxxxxxxxxxxxxx.com/testFile.txt");
    //client.println();
//  } 
  //else {
   // Serial.println("connection failed");
  }

char c = 'f';
int a = 0;
void loop() {
        
     while(a<5){
     client.connect();
     client.println("GET http://xxxxxxxxxx.com/testFile.txt\r\n");
     delay(5000);
     client.println();
     c = client.read();
     Serial.print(c);
     Serial.println();
     a=1+a;
     Serial.print(a);
     client.stop();
    }

    client.stop();    
  //  Serial.println("disconnecting.");    
  }

 // if (!client.connected()) {
  //  Serial.println();
 //   Serial.println("disconnecting.");
 //   client.stop();
 //   for(;;)
 //     ;
//  }

Edit: Moderator edited to put code into code box. You can go back to your post, click “Modify”, highlight the code, click the # icon, then click on “Save”. Coding Badly.

When posting code, please use the # icon on the editor’s toolbar.
You can go back to your post, click “modify”, highlight the code, then click the # icon, then click on “save”. That way it looks neater and it’s easier to read and you don’t get smiley faces.

You are never resetting the value of your counter A. So it is always greater than 5 and will not enter the loop to update. Add the a=0 should fix the problem.
Another pointer that may or may not be of help to you that makes code look neater is that the following statements all evaluate equally.

a=a+1;
a+=1;
a++;
++a;
int a = 0;
void loop() {
       
     while(a<5){
     client.connect();
     client.println("GET http://xxxxxxxxxx.com/testFile.txt\r\n");
     delay(5000);
     client.println();
     c = client.read();
     Serial.print(c);
     Serial.println();
     a=1+a;
     Serial.print(a);
     client.stop();
    }
// ---------------------------> Added Below <----------------------------
a=0;
// ------------------------------------------------------------------------
    client.stop();   
  }

Thanks for suggesting to use Code Brackets, that does look much neater ;). Please forgive my ignorance though, I've tried preview with brackets and I can't seem to get the code to appear that way. How did you post it like that?

The counter seems to be functioning correctly. I'm mainly having trouble with the variable "c." Either the value printed is incorrect or it saves a previous value that seems to have been stored in memory. I must have structured the code wrong somehow, is the order of operations incorrect?

If you want code you need to place the code inside of this statement [c ode]INSERT CODE HERE[/c ode]. I added the spaces so they don't get turned into code, just remove the spaces. C will only be updated 5 times, then the program will not work anymore. Is that what you are observing? If you have made any changes, could you post the updated code.

The only thing I tried was redefining “c” at the end of the while loop. Below is what I get as an output:

Can u reach me ?
IP: 192.168.1.104
connecting...
z
1*
2C
3L
4O
5

My text file used to have the letter “z” inside it. That’s why I believe that the “c” variable is not being updated due to an error in my code. The strange thing is that sometimes the code seems to work correctly, but just for the first loop I will get a correct response. The remaining four loops will have gibberish like in the example above.

void loop() {
       
     while(a<5){
     client.connect();
     client.println("GET http://xxxxxxxxxx.com/testFile.txt\r\n");
     delay(5000);
     client.println();
     c = client.read();
     Serial.print(c);
     Serial.println();
     a=1+a;
     Serial.print(a);
     client.stop();
     c='f';
    }
// ---------------------------> Added Below <----------------------------
a=0;
// ------------------------------------------------------------------------
    client.stop();   
  }

You need to change this stuff:

client.println("GET http://xxxxxxxxxx.com/testFile.txt\r\n");

Including the http://xxxxxx.com part is only allowed when making the request to a proxy server. You can change it to either HTTP/0.9 spec:

client.println("GET /testFile.txt");
client.println("");

Or if name-based virtual hosts are used you need to use HTTP/1.0 spec:

client.println("GET /testFile.txt HTTP/1.0");
client.println("Host: xxxxxxxxxxx.com");
client.println("");

Granted you mentioned you were seeing your "z" which means the way you had it might have been working, but if you use the HTTP/1.0 method you'll get a more debuggable response from the server.

You might also want to look into Wireshark (for Windows) or tcpdump (Mac or Unix) so that you can watch the network traffic. They are great tools to learn.

You are assuming that each call to the server returns a single letter. That is unlikely to be a valid assumption. You need to read everything that the server returns (until client.connected() is false AND client.available() is 0) in a while loop.

Then, when you make a second call, in your existing while loop, you will get another response, possibly different, depending on what the server is doing.

Chagrin, I’ll try your mods tonight.

Paul, I’m going to experiment with the following code. Do you think that the following while loop will do the trick?

void loop() {

     while(a<5){
     client.connect();
     client.println("GET http://xxxxxxxxxx.com/testFile.txt\r\n" (http://xxxxxxxxxx.com/testFile.txt%5Cr%5Cn);
     delay(5000);
	while(client.available){
   	client.println();
    	c = client.read();
        Serial.print(c);
        Serial.println();
	}
     a=1+a;
     Serial.print(a);
     client.stop();
    }
}

You are assuming that client.connect() succeeds, when sending data to the client. Not a good assumption. You should also change the while to:

while(client.connected())
{
  while(client.available() > 0)
  {
     // Read the client reply.
  }
}

The client may be sending data slower than you can read it, so client.available() may become 0 before the server is done sending data. But, as long as the client is connected, you should keep looking for data. Only when client.connected() returns false AND client.available() returns 0 has the entire response from the server been read.

Thanks Paul, your reply makes a lot of sense.

Is there any need for client.stop() if client.connected() and client.available() return false and 0 respectively?

This is what I have now:

void loop() {

     while(a<5){
     client.connect();
     client.println("GET http://xxxxxxxxxx.com/testFile.txt\r\n");
     delay(5000);
          while(client.connected())
          {
	while(client.available() > 0)
             {
   	client.println();
    	c = client.read();
             Serial.print(c);
             Serial.println();
	}
          }
     a=1+a;
     Serial.print(a);
     client.stop();
    }
}

Is there any need for client.stop()

void Client::stop() {
  if (_sock == MAX_SOCK_NUM)
    return;

  // attempt to close the connection gracefully (send a FIN to other side)
  disconnect(_sock);
  unsigned long start = millis();

  // wait a second for the connection to close
  while (status() != SnSR::CLOSED && millis() - start < 1000)
    delay(1);

  // if it hasn't closed, close it forcefully
  if (status() != SnSR::CLOSED)
    close(_sock);

  EthernetClass::_server_port[_sock] = 0;
  _sock = MAX_SOCK_NUM;
}

I’d say, yes, it’s a good idea to call client.stop() when you are done with a client connection.

That is a good way of putting it :).

Seems like the code is closer to working. However, I've noticed that I'm getting the message CLOS at the end of each connection. I tried to turn off this message by using set comm close 0. Now when I run my program it hangs after the first byte that is transferred. I set it back to CLOS. Do you know if it's possible for the code to execute with no message returned?

Also, do you think it really matters if there is other text printed? Ultimately, I will have my Arduino perform a task when it receives a command such as 0xx000001.

Do you know if it's possible for the code to execute with no message returned?

Are you asking if it is possible for the server to properly handle your GET request, and not return any information? If that is what you are asking, you need to look at the script that the server is executing. If it were my script, then it would return some text every time. Some value or "What the heck are you asking me to do?" or something.

However, I've noticed that I'm getting the message CLOS at the end of each connection.

Is that a problem?

Also, do you think it really matters if there is other text printed?

Not to me it doesn't. It shouldn't matter to your program, either. There must be something in the output that lets you identify the information of interest. There can be any other output that is ignored.

Ultimately, I will have my Arduino perform a task when it receives a command such as 0xx000001.

Hopefully reject it, as that is not a valid hex value (x is not a valid hex digit).

Are you asking if it is possible for the server to properly handle your GET request, and not return any information? If that is what you are asking, you need to look at the script that the server is executing. If it were my script, then it would return some text every time. Some value or “What the heck are you asking me to do?” or something.

I was wondering if it’s is possible for the server to return my information in the text file and not return the CLOS message. When I tried to stop the CLOS message the code was hanging.

I have another strange problem. I’ve been troubleshooting my code and haven’t been able to get the CLOS message to be on a separate line. Then it dawned on me that the connection is closing before the text file is read and the value that is return for “c” is what has been stored in memory previously (I changed the text file contents from “b” to “x”).

Why do you think the connection is closing before the file is read?

This is what I’m seeing in the terminal:

?
b
1
?
*CLOS*b
2
?
*CLOS*b
3
?
*CLOS*b
4
?
*CLOS*b
5

Here is the code I’ve been troubleshooting with:

void loop() {

     while(a<5){
     client.connect();
     client.println("GET http://xxxxxxxxxx.com/testFile.txt");
     Serial.print('?');
     Serial.println();
     delay(5000);
          if(client.connected())
          {
	    while(client.available() > 0)
            {
   	   // client.println();  
            c = client.read();
            Serial.print(c);
	  }
            }
          else {
          Serial.println("connection failed");
          }
     Serial.println();
     a=1+a;
     Serial.print(a);
     Serial.println();
     client.stop();
    }
}

Why do you think the connection is closing before the file is read?

I have no idea what your server is doing. Maybe it doesn't like being polled so often by the same client. Maybe there is some kind of connection maintained, and the file is not rewound.

You should, I would think, be calling some script on the server, not trying to get a text file from the server. The script can check for the file in use, and try again, if it is. The server can't.

On final thought. You should check that client.connected() returns true BEFORE you try to send the GET command to the server.

Below are the typical headers returned by an apache http server to my arduino with the w5100 ethernet shield (bottom is the client code). Have you gotten something like this just doing a single get request with your wifly shield?

HTTP/1.1 200 OK
Date: Thu, 14 Jul 2011 23:52:46 GMT
Server: Apache
Last-Modified: Sat, 13 Nov 2010 16:31:40 GMT
Accept-Ranges: bytes
Content-Length: 51
Connection: close
Content-Type: text/plain; charset=UTF-8

//zoomkat 11-13-10
//simple ethernet client test code
//for use with IDE 0021 and W5100 ethernet shield
//modify the arduino lan ip address as needed
//open serial monitor to see what the arduino receives
//push the shield reset button to run client again

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 192, 168, 1, 102 }; // Arduino IP address
byte server[] = { 208, 104, 2, 86 }; // zoomkat's web site

Client client(server, 80);

void setup()
{
  Ethernet.begin(mac, ip);
  Serial.begin(9600);
  Serial.println("starting simple arduino client test");
  Serial.println();

  delay(1000);

  Serial.println("connecting...");

  if (client.connect()) {
    Serial.println("connected");
    client.println("GET /~shb/arduino.txt HTTP/1.0");
    client.println();
  } else {
    Serial.println("connection failed");
  }
}

void loop()
{
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    Serial.println("==================================");
    Serial.println("");
    client.stop();
    for(;;);
  }
}

Paul,

Using your comment, I created a script on my server that reads the text file. Everything seems to work properly now! Here is the code I’m using:

// (Based on Ethernet's WebClient Example)

#include "WiFly.h"


#include "Credentials.h"


byte server[] = { 66, 96, xxx, xx }; // Google

//Client client(server, 80);

Client client("xxxxxxxxxxxxxx.com", 80);

void setup() {

  Serial.begin(9600);

  WiFly.begin();

  if (!WiFly.join(ssid, passphrase)) {
    Serial.println("Association failed.");
    while (1) {
      // Hang on failure.
    }
    
  }
    Serial.println("Can u reach me ?");
    Serial.print("IP: ");
    Serial.println(WiFly.ip());
  

  Serial.println("connecting...");

  }
char a;
char c;

void loop() {
for(;;){
     client.connect();
     client.println("GET http://xxxxxxxxxxxxxxxx.com/newfile.php");
     delay(1000);
          if(client.connected())
          {
            Serial.println();
            Serial.println("connected");
	    while(client.available() > 0)
            {
             c = client.read();
            Serial.print(c);
            }
          }
     Serial.println();
     client.stop();
     Serial.println("disconnecting.");
     delay(2500);
    }
}

The code will poll the script file on my server indefinitely. Below is the PHP script that I’m using:

<?php
$homepage = file_get_contents('testFile.txt');
echo $homepage;
?>

When I change the value inside the text file, it will show up on my terminal. Now I will write code the Arduino to accept commands from the server. I’m still puzzled as to why I receive CLOS prior to every subsequent request from the first. For example, say I had “1000” written in the text file. The first poll would received “1000.” But every other poll afterwards would return “CLOS1000”. This was not helpful because Arduino would not know how to interpret a command with "CLOS at the beginning. So I read this post: http://forum.sparkfun.com/viewtopic.php?f=13&t=21326. They recommended using the command set comm close 0 in SpiUartTerminal Tool AND reset the device. I did not reset my Arduino after the last time I changed this setting and that is why it didn’t work.

zoomcat,

I have gotten a reply like the one you referenced from a single request. However, I didn’t think it was necessary every time I polled the server so I deleted “HTTP/1.0” from the end of my GET request.

Thanks for your help.

However, I didn't think it was necessary every time I polled the server so I deleted "HTTP/1.0" from the end of my GET request.

Servers generally suffer from Alzheimer's. You introduce yourself to the server, and make a request. You come back 10 seconds later, the server has no idea who you are.

Every request must be complete.