Inconsistent HTTP Connnection

SurferTim: So after a little fiddling I got the code working and was able to access the webpage fine. But things seem to work fine on the local side of things. It's when I try to reach out and grab pages that things get wonky (i.e. failing to connect multiple times, only half the page downloading, etc.)

zoomkat: I don't believe I even have a card installed at the moment. I'll double check that when I get home. As for the server and gateway IP addresses, I've double checked them and things seem fine. My network uses 10.0.1.*, would this cause issues?

Can you explain “reach out and grab pages”? I thought this was server code. Clients “grab pages”.

My network uses 10.0.1.*, would this cause issues?

That is a localnet ip range. It will be good as a server ip only within the localnet or the local router. From the internet, you would need to translate your public ip to the internal localnet ip.

Uh oh, looks like we got crossed up. I"m trying to set the Arduino up as a client, not a server.

That was my bad.

Which code are you using now? Are you still trying your original code? Or zoomkat's?

I haven't had a chance to work on this project for the past few days, but I tried using my original code (with a few modifications) as well as zoomkat's. Both seem to give me mixed results as far as connectivity. With zoomkat's, sometimes it connects and downloads a page immediately after I press 'e', other times it takes around 20 seconds before it says connection failed.

I originally had things set up in a for loop so that it would keep trying over and over again. This seemed to provide okay results, but also seemed like a work-around with addressing the actual problem.

This loads www.google.com every 10 seconds. Don’t do that to Google too long. Besides, it is a big page and takes a while to display at 9600 baud. Change the server ip to yours.

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192,168,2,2);
IPAddress gateway(192, 168, 2, 1);
IPAddress subnet(255, 255, 255, 0);

IPAddress server(74,125,227,16); // Google

EthernetClient client;
int totalCount = 0;
int loopCount = 0;

void setup() {
  Serial.begin(9600);

  pinMode(4,OUTPUT);
  digitalWrite(4,HIGH);

  Ethernet.begin(mac, ip, gateway, gateway, subnet);
  delay(2000);
  Serial.println("Ready");
}

void loop()
{
  if(loopCount < 10)
  {
    delay(1000);
  }
  else
  {
    loopCount = 0;

    if(!getPage(server,"/")) Serial.print("Fail ");
    else Serial.print("Pass ");
    totalCount++;
    Serial.println(totalCount,DEC);
  }    

  loopCount++;
}

byte getPage(IPAddress ipBuf,char *page)
{
  int inChar;
  char outBuf[64];

  Serial.print("connecting...");

  if(client.connect(ipBuf,80))
  {
    Serial.println("connected");

    sprintf(outBuf,"GET %s HTTP/1.0\r\n\r\n",page);
    client.write(outBuf);
  } 
  else
  {
    Serial.println("failed");
    return 0;
  }

  while(client.connected())
  {
    while(client.available())
    {
      inChar = client.read();
      Serial.write(inChar);
    }
  }

  Serial.println();

  Serial.println("disconnecting.");
  client.stop();

  return 1;
}

Hmmm, well that seems to work so far. I've only had a few minutes this morning to play, but I haven't gotten any failed attempts (except for when I forgot to change the IP).

So what's different? Why is this one connecting fine whereas the others weren't?

jamgood96: Hmmm, well that seems to work so far. I've only had a few minutes this morning to play, but I haven't gotten any failed attempts (except for when I forgot to change the IP).

So what's different? Why is this one connecting fine whereas the others weren't?

The others were not mine. :)

edit: There are some reasons mine works better. I know how tcp works. If you look at the getPage() function, you will notice the "while(client.connected()) while(client.available())" loops.

If you are downloading a big page (like Google's home page), you will get several packets. That loop will check client.available(), and for a while, it will be zero. Then it will be about 1400. Your goal is to empty that rx buffer as fast as you can. Get client.available() to zero. Then it may be zero again for a few checks, then it will be 1400 again (another packet).

The client.connected() will return true until you have emptied the rx buffer of the final packet, so as long as client.available() returns non-zero, you are connected.

Yeah, that's exactly what I was thinking it did... XD

Thanks so much! I'll keep you posted if I run into errors with this, but so far so good.

The others were not mine.

I can't beleve this code is yours! It doesn't contain the magic pixie dust (below) that makes all things work! :) I notice the code does have a fat 2000ms delay that has been used in older code.

void setup()
{
   Serial.begin(9600);

   // disable the SD SPI interface
   pinMode(4,OUTPUT);
   digitalWrite(4,HIGH);

   // rest of your setup
}

zoomkat:

The others were not mine.

I can't beleve this code is yours! It doesn't contain the magic pixie dust (below) that makes all things work! :) I notice the code does have a fat 2000ms delay that has been used in older code.

void setup()
{
   Serial.begin(9600);

   // disable the SD SPI interface    pinMode(4,OUTPUT);    digitalWrite(4,HIGH);

   // rest of your setup }

You did not pay attention, because it does have the magic pixie dust! :)

You did not pay attention, because it does have the magic pixie dust!

Oh, my bad! Just as a test, I removed the pixie dust from your code like below, and ate some salted peanuts instead. The code made 17 successful passes, so salted peanuts work too! :slight_smile:

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress server(74,125,227,16); // Google

EthernetClient client;
int totalCount = 0;
int loopCount = 0;

void setup() {
  Serial.begin(9600);

  //pinMode(4,OUTPUT);
  //digitalWrite(4,HIGH);

  Ethernet.begin(mac);
  delay(2000);
  Serial.println("Ready");
}

I used to think that too. Here is an old thread that we both were involved in. Read reply #28. http://arduino.cc/forum/index.php/topic,75324.0.html I don't think salted peanuts would have helped there. The magic pixie dust did.

The OP's question was why mine was working and the others weren't. This is part of the reason why.

edit: If you want to read the posts where I was being you and eating salted peanuts, read replies 7,20,and 26.

One question? How can we be sure that the program will never stop on the line

while(client.connected()

If the client connection does not end we will never go out of the while loop

cantore: One question? How can we be sure that the program will never stop on the line

while(client.connected()

If the client connection does not end we will never go out of the while loop

Good question. There is the possibility of a lockup there if the connection breaks. Here is the thread with the discussion and the patch for a 10 second timeout. http://arduino.cc/forum/index.php/topic,102879.msg778413.html#msg778413

thanks

SurferTim:
This loads www.google.com every 10 seconds. Don’t do that to Google too long. Besides, it is a big page and takes a while to display at 9600 baud. Change the server ip to yours.

#include <SPI.h>

#include <Ethernet.h>

byte mac = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192,168,2,2);
IPAddress gateway(192, 168, 2, 1);
IPAddress subnet(255, 255, 255, 0);

IPAddress server(74,125,227,16); // Google

EthernetClient client;
int totalCount = 0;
int loopCount = 0;

void setup() {
  Serial.begin(9600);

pinMode(4,OUTPUT);
  digitalWrite(4,HIGH);

Ethernet.begin(mac, ip, gateway, gateway, subnet);
  delay(2000);
  Serial.println(“Ready”);
}

void loop()
{
  if(loopCount < 10)
  {
    delay(1000);
  }
  else
  {
    loopCount = 0;

if(!getPage(server,"/")) Serial.print("Fail ");
    else Serial.print("Pass ");
    totalCount++;
    Serial.println(totalCount,DEC);
  }

loopCount++;
}

byte getPage(IPAddress ipBuf,char *page)
{
  int inChar;
  char outBuf[64];

Serial.print(“connecting…”);

if(client.connect(ipBuf,80))
  {
    Serial.println(“connected”);

sprintf(outBuf,“GET %s HTTP/1.0\r\n\r\n”,page);
    client.write(outBuf);
  }
  else
  {
    Serial.println(“failed”);
    return 0;
  }

while(client.connected())
  {
    while(client.available())
    {
      inChar = client.read();
      Serial.write(inChar);
    }
  }

Serial.println();

Serial.println(“disconnecting.”);
  client.stop();

return 1;
}

The server I’m trying to reach is in a virtual hosting environment so in addition to the ip address, I need to specify the host I want to connect to. Do you have any advice on where that would be included if one wanted to make that tweak?

It is sent on a line by itself after the GET, followed by a double CR-LF.

  if(client.connect(ipBuf,80))
  {
    Serial.println("connected");
    // Insert it here. 
    sprintf(outBuf,"GET %s HTTP/1.0\r\nHost: www.mydomain.com\r\n\r\n",page);
    client.write(outBuf);
  }

edit: Insure outBuf has sufficient memory allocated to hold all that.

SurferTim: It is sent on a line by itself after the GET, followed by a double CR-LF.

  if(client.connect(ipBuf,80))
  {
    Serial.println("connected");
    // Insert it here. 
    sprintf(outBuf,"GET %s HTTP/1.0\r\nHost: www.mydomain.com\r\n\r\n",page);
    client.write(outBuf);
  }

edit: Insure outBuf has sufficient memory allocated to hold all that.

Thank you! Have to head to a Memorial Day event but will hopefully get to tinker with that upon return this evening!

This format is a bit different that you will see in examples. The reason I use this is it is easier to pass GET variables attached to the page. The sprintf function makes it easy to format a character array.

You could use the standard three client.println() calls to do that, but bear in mind each call to client.print(), client.println, or client.write() is sent in its own packet.

client.println("GET / HTTP/1.0");
client.println("Host: www.mydomain.com");
client.println();

Why send three packets? The client.write(outBuf) sends all that in one packet. Clean.