why won't ethernet stay connected for more than a few hours?

scary · October 13, 2012, 3:10am

Hi, I’ve blown $100 on an Freetronics EtherTen with the hope of using it to log sensor measurements at university. I’ve been testing it out at home wired into my router. The system just doesn’t work reliably and I’m wondering why. I’ve run it with the basic Cosm sketch for a while and it falls over after a day or so. I’ve tried logging to Google Documents and every second POST request gets through for a while before the connection just hangs (you tell me why every second post gets through). I’ve spent several days of my time looking for fixes on the net and this post seems to hint at some sort of software bug which makes the WizNet chip hang. I’ve also found that one of changes to IDE v1.0.1 was the addition of the Ethernet.maintain() class, which renews the temporary IP address assigned to the device. Expiry of a temporary IP address might explain why my Cosm sketch falls over after a period although, as I recall, the sketch falls over after varying lengths of time. If it fell over after a fixed period that would be a good indication that I was experiencing a buffer overflow, expiry of a temporary IP or other finite limit.

Note that I can post data to Cosm without issue so my code and connection are fine. I was literally using the example code without any alterations beyond changing the account key and number of the analogue port read. My problem is that the EtherTen hangs after a day and you have to reset the power to get it working again (it starts immediately without issue after that). Pushing reset does nothing. In the serial output you can see successful posts going through and then one time the server response is -3 and then for every POST attempt after that the server returns -1.

Today I upgraded to IDE v1.0.1 in the hope that this might provide a fix but no such luck. I’ve been trying to log to Google again with the code below. See the serial output below that for an example of what happens.

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress server(74,125,39,104); // Google

// Initialize the Ethernet client library
// with the IP address and port of the server 
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;
String formkey = "dGE5cUlvV2d3eHRhMmpyaEltQmVXNkE6MQ";



void setup(){
 // start the serial library:
  Serial.begin(57600);
  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    for(;;)
      ;
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  if (client.connect(server, 80)) {
    Serial.println("...connected!");
}
}



void loop(){

  // detect state of connection 
  Serial.print("checking connection . . . ");
  if (client.connect(server, 80)) {
    Serial.println(". . . connected!");
    Serial.println("Posting . . . ");
    
    String data;
  data+="";
  data+="entry.0.single=";
  data+=analogRead(A2);
  data+="&submit=Submit";

    client.print("POST /formResponse?formkey=");
    client.print(formkey);
    client.println("&ifq HTTP/1.1");
    client.println("Host: spreadsheets.google.com");
    client.println("Content-Type: application/x-www-form-urlencoded");
    client.println("Connection: close");
    client.print("Content-Length: ");
    client.println(data.length());
    client.println();
    client.print(data);
    client.println();

    Serial.print("POST /formResponse?formkey=");
    Serial.print(formkey);
    Serial.println("&ifq HTTP/1.1");
    Serial.println("Host: spreadsheets.google.com");
    Serial.println("Content-Type: application/x-www-form-urlencoded");
    Serial.println("Connection: close");
    Serial.print("Content-Length: ");
    Serial.println(data.length());
    Serial.println();
    Serial.print(data);
    Serial.println();
    Serial.println("Posted!");
    Serial.print("client returned: ");

if (client.available()) {
    char returned = client.read();
    Serial.println(returned);
}

else {
  Serial.println("client unavailable!");
}
}



else { 
   Serial.println(". . . not connected");
   
// check if connection is available
  Serial.print("checking connection availability . . . ");
 if (client.available()) {
    char c = client.read();
    Serial.print("client returned: ");
    Serial.println(c);
  }
  else {
        Serial.print("client unavailable");
        delay(10);
        client.flush();
        delay(10);
        client.stop();
        
  }
}

    
  delay(1000);
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
        delay(10);
        client.flush();
        delay(10);
        client.stop();
  }

    Serial.println();
        Serial.println();
  delay(20000);
}

Here is some output from serial monitor which shows how the connection falls over. Here the connection had run for a couple of hours before this point.

checking connection . . . . . . connected!
Posting . . .
POST /formResponse?formkey=dGE5cUlvV2d3eHRhMmpyaEltQmVXNkE6MQ&ifq HTTP/1.1
Host: spreadsheets.google.com
Content-Type: application/x-www-form-urlencoded
Connection: close
Content-Length: 32

entry.0.single=143&submit=Submit
Posted!
client returned: client unavailable!

checking connection . . . . . . not connected
checking connection availability . . . client returned: H

disconnecting.

checking connection . . . . . . connected!
Posting . . .
POST /formResponse?formkey=dGE5cUlvV2d3eHRhMmpyaEltQmVXNkE6MQ&ifq HTTP/1.1
Host: spreadsheets.google.com
Content-Type: application/x-www-form-urlencoded
Connection: close
Content-Length: 32

entry.0.single=143&submit=Submit
Posted!
client returned: client unavailable!

checking connection . . . . . . not connected
checking connection availability . . . client returned: H

disconnecting.

checking connection . . . . . . connected!
Posting . . .
POST /formResponse?formkey=dGE5cUlvV2d3eHRhMmpyaEltQmVXNkE6MQ&ifq HTTP/1.1
Host: spreadsheets.google.com
Content-Type: application/x-www-form-urlencoded
Connection: close
Content-Length: 32

entry.0.single=143&submit=Submit
Posted!
client returned: client unavailable!

checking connection . . . . . . not connected
checking connection availability . . . client returned: H

checking connection . . . . . . not connected
checking connection availability . . . client returned: T

checking connection . . . . . . not connected
checking connection availability . . . client returned: T

checking connection . . . . . . not connected
checking connection availability . . . client returned: P

The single letters returned from each failed POST eventually spell out: “HTTP/1.1 200 OKContent-Type: text/html; charset=utf-8X-Robots-Tag: noindex, nofollow, nosnippetDate: Sat, 13 Oct 2012 02:19:58 GMTExpires: Sat, 13 Oct 2012 02:19:58 GMTCache-Control: private, max-age=0X-Content-Typ”.

Is there some known bug with ethernet that causes it to fall over like this? Do WiFi shields experience the same problem or is this unique to ethernet? I’ve got a lot of uses for this technology if it proves reliable.

system · October 13, 2012, 5:46am

   String data;
  data+="";
  data+="entry.0.single=";
  data+=analogRead(A2);
  data+="&submit=Submit";

I suspect your problem is here. There is a known issue with the String class and the way it manages memory, that will result in you running out of SRAM. I recommend you get rid of the above, and just Client.print() the elements separately.

scary · October 13, 2012, 8:09am

Thanks for the reply, I've amended the code and I'll see how it runs overnight.

scary · October 13, 2012, 9:25am

This hasn't fixed it. It logged for 30 minutes before I had to do a mains reset then for about another 45 minutes before the same thing happened again. Thanks anyway.

SurferTim · October 13, 2012, 11:27am

Try something like this. I modified the code to read the server response until the server closes the connection, then closes this end. That should allow the socket to close properly and avoid running out of sockets.

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress server(74,125,39,104); // Google

EthernetClient client;
String formkey = "MyFormKey";

void setup(){
 // start the serial library:
  Serial.begin(57600);
  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    for(;;)
      ;
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
}

void loop(){

  // detect state of connection 
  Serial.print("checking connection . . . ");
  if (client.connect(server, 80)) {
    Serial.println("connected!");
    Serial.println("Posting . . . ");
    
    String data;
  data="entry.0.single=";
  data+=analogRead(A2);
  data+="&submit=Submit";

    client.print("POST /formResponse?formkey=");
    client.print(formkey);
    client.println("&ifq HTTP/1.1");
    client.println("Host: spreadsheets.google.com");
    client.println("Content-Type: application/x-www-form-urlencoded");
    client.println("Connection: close");
    client.print("Content-Length: ");
    client.println(data.length());
    client.println();
    client.print(data);

    Serial.print("POST /formResponse?formkey=");
    Serial.print(formkey);
    Serial.println("&ifq HTTP/1.1");
    Serial.println("Host: spreadsheets.google.com");
    Serial.println("Content-Type: application/x-www-form-urlencoded");
    Serial.println("Connection: close");
    Serial.print("Content-Length: ");
    Serial.println(data.length());
    Serial.println();
    Serial.print(data);
    Serial.println();
    Serial.println("Posted!");

    Serial.println("client returned: ");

    while(client.connected()) {
      while(client.available()) {
        Serial.write(client.read());
      }
    }
    client.stop();
    Serial.println();
    Serial.println("disconnected");
  }
  else {
    Serial.println("failed");
  }

  delay(20000);
}

My apology in advance for any typos or errors.

edit: I forgot the “failed” message print, so I added it. The code above does not include a timeout, and may lock up in the “while(client.connected())” loop if the connection breaks (fails, not closes, like pulling the cat5 cable out of the shield). If this works for you, let me know and I will post the timeout modification.

…and I removed that form key from my post, in the event that is your real form key.

scary · October 14, 2012, 7:27am

Hi SurferTim, I’m really grateful for your response. Your code has been running all afternoon with only one failure and I’m putting that one down to the power cut! (New Zealand is a 3rd world country by utility standards.) This is the longest I think I’ve seen any Google Docs code run and I’m super-impressed with its functionality. If it stays running for the next few days I totally owe you 50 beers!

Cheers,
Le Scare

SurferTim · October 14, 2012, 10:39am

Good deal! Here is the timeout code for that.

    int timeoutCount = 0;

    while(client.connected()) {
      while(client.available()) {
        timeoutCount = 0;
        Serial.write(client.read());
      }

      timeoutCount++;
      if(timeoutCount > 10000) {
        client.stop();
        Serial.println("timeout");
      }
      delay(1);      
    }
    client.stop();

This will prevent the w5100 from locking up if there is a connection break. It will close the connection if no packets received for 10 seconds.

edit: If you plan on using dhcp for the ip, you should call Ethernet.maintain() about once an hour to keep the ip lease active. I highly recommend using a static ip in the shield for long term connections. That way the router can power cycle and not affect the shield ip.

cantore · October 22, 2012, 1:01pm

In the version with the timeout is it ok that you call two times the client.stop if the timeout is reached?

SurferTim · October 22, 2012, 1:05pm

cantore:
In the version with the timeout is it ok that you call two times the client.stop if the timeout is reached?

That is ok the last I checked. If the connection is already stopped/closed, the stop function just returns.

big_jose · November 14, 2012, 9:09pm

Hi guys,

I'm having the same problem with my Arduino ethernet. Is there any one with the solution for this problem?

SurferTim · November 14, 2012, 9:12pm

I combined all the fixes I could into a demo sketch in the playground. http://www.arduino.cc/playground/Code/WebClient Have you tried that?

big_jose · November 14, 2012, 9:19pm

I will try it now. Thanks

system · May 6, 2021, 10:39am