Arduino as Web Client (via ethernet) connection issues after some hours

Hello to Everybody !

I am using the ATMega 2560 with Ethernet shield. My goal is to send several string to a remote server that are processed for my application.

Typically the process start without problem and each minute a send a string:
typical string is:
“SK=xxxxxxx&I=111111&G=222222&V=33333333&CHECK=1234567&S=99”.

After some hours of sending this type of string the connection to the server fails ( else part of the code posted below…).
If I reset the board, the process restart without problems.
The remote server and internet connection is all the times OK.

I post some piece of my function:

short sendGET(struct s_table *sensor_list,short i) //client function to send data from external server.
{
//String WebString = “”;
String dataToSend = “”;

if (client.connect(RemoteServer, 778)) {

dataToSend = “SK=”+sensor_list.idString+"&I="+String(sensor_list.IRead)+"&G="+String(sensor_list.GRead)+"&V="+String(sensor_list.VRead)+"&CHECK="+String(sensor_list.CheckSum)+"&S="+String(sensor_list*.Sens_OK);*

* Serial.print("DATA TO SEND: ");Serial.println(dataToSend); //24.01.2015 TOGLIERE DOPO !!!*

dataToCheck = “SK=”+sensor_list.idString+"&I="+String(sensor_list.IRead)+"&G="+String(sensor_list.GRead)+"&V="+String(sensor_list*.VRead);*

* client.print(“GET /?”); *
* client.println(dataToSend);*

* client.println();*
* client.flush();*
* client.stop();*
* return true;*

* }*
* else {*
* Serial.println(“connection failed”);*
* Serial.println();*
* client.flush();*
* delay(100);*
* client.stop(); *
* return SRV_CONNECTION_FAILED;
_ }
}*

Are there some optional command to use in the ethernet library to avoid this bug?
Are you experiencing this issues?
Thank you
Domenico_

If you are getting a “connection failed” message, it is probably due to your w5100 running out of sockets. It isn’t a bug in the library, but it is in your code.

You are not reading the response from the server, and not waiting for it to close the connection. That can cause the socket to fail to release the connection and that socket will no longer be available.

Here is a function you can use to check the socket status. I recommend calling it after each connection attempt. If none of the sockets have a status of 0x0, then any subsequent connection attempt will fail.

#include <utility/w5100.h>

byte socketStat[MAX_SOCK_NUM];

void ShowSockStatus()
{
  for (int i = 0; i < MAX_SOCK_NUM; i++) {
    Serial.print(F("Socket#"));
    Serial.print(i);
    uint8_t s = W5100.readSnSR(i);
    socketStat[i] = s;
    Serial.print(F(":0x"));
    Serial.print(s,16);
    Serial.print(F(" "));
    Serial.print(W5100.readSnPORT(i));
    Serial.print(F(" D:"));
    uint8_t dip[4];
    W5100.readSnDIPR(i, dip);
    for (int j=0; j<4; j++) {
      Serial.print(dip[j],10);
      if (j<3) Serial.print(".");
    }
    Serial.print(F("("));
    Serial.print(W5100.readSnDPORT(i));
    Serial.println(F(")"));
  }
}

The status codes are as follows
0x0 = available
0x14 = waiting for connection
0x17 = connected to server
0x1C = connected waiting for close
0x22 = UDP socket

edit: Did you try the web client code in the playground that I posted a link to in the other thread? It has fault tolerance and error checking that prevent the freeze and socket loss problem.
http://playground.arduino.cc/Code/WebClient

Thank you for this answer; I will follow your two suggestion. I will update them after the tests.

The connection shall fail only because of home internet fails or broken cable but not from a bug in my code. Thanks again

The connection shall fail only because of home internet fails or broken cable but not from a bug in my code.

If your code fails at any time and doesn't recover gracefully, there is a bug in your code. I intentionally break connections, reboot routers, write server test code that stalls, etc, and my code keeps on going. So should yours. A failure to connect to a server due to outside hardware fails is ok, as long as your code goes back to working when the problem is resolved.

Hello SurferTim
I indroduced your function void ShowSockStatus() in to my code (to which it is possible to have bug)

The function was introduced after the connection attempt.

Well, differently from other times, even if now I am printing the Socket report, I am not displaying any connection fail.

I attach the report of the 17 hour of correct working, that did not happen first.
I remark that except this adding nothing has changed in my code.

As it is the first time I am using the void ShowSockStatus(), looking the Logs report is there some message that can make a doubt that a socket will not be available and will no more have connection to remote server?
for example I look continously increase the variable s = W5100.readSnSR(i) relevant to Socket 0 0x17 = connected to server
and going ahead I look other new socket at same server reference IP:

Socket#0:0x17 2932 D:93.57.69.235(778)
Socket#1:0x17 2614 D:93.57.69.235(778)
Socket#2:0x17 2809 D:93.57.69.235(778)
Socket#3:0x14 778 D:93.57.69.235(778)

next connection attempt..

Socket#0:0x17 2933 D:93.57.69.235(778)
Socket#1:0x17 2614 D:93.57.69.235(778)
Socket#2:0x17 2809 D:93.57.69.235(778)
Socket#3:0x14 778 D:93.57.69.235(778)

Thank you again wait for you comment

logs.TXT (464 KB)

SG.c (1.38 KB)

This shows the w5100 has no available sockets. Any subsequent connection attempts at this point will fail. Socket#0:0x17 2933 D:93.57.69.235(778) Socket#1:0x17 2614 D:93.57.69.235(778) Socket#2:0x17 2809 D:93.57.69.235(778) Socket#3:0x14 778 D:93.57.69.235(778)

You have a bug in your code somewhere.

edit: I suspect this is it:

              client.print("GET /?");    
              client.println(dataToSend);
                           
              
              client.println();
// right here
              client.stop(); 
              return true;

Hello All Guys, Hello SurferTim
after some tests, I still having the connection failed after awhile.

Just to have a better solution I took your web client program http://playground.arduino.cc/Code/WebClient and readapted to my server to transmit some datas.
Basically it is the same program.
I tested from yesterday, but again after 12 hours the test failed,
Please find annexed my program ( you can test directly) and the logs report.

i have read the response from server without printing on serial ( shall I do it??) and put delays before stopping as according to the original program.

Still failed connection.

/*
   Web client sketch for IDE v1.0.1 and w5100/w5200
   Uses GET method.
   Posted October 2012 by SurferTim
   Last modified September 15, 2013
*/
#include <Time.h>
#include <SPI.h>
#include <Ethernet.h>

// this must be unique
byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

// change to your network settings
IPAddress ip(192,168,1,177);
IPAddress gateway(192, 168, 1, 1);
IPAddress subnet(255, 255, 255, 0);

// change to your server
IPAddress server(93,57,69,235); // Google

//Change to your domain name for virtual servers
char serverName[] = "HYDRA SERVER";
// If no domain name, use the ip address above
// char serverName[] = "74.125.227.16";

// change to your server's port
int serverPort = 778;

EthernetClient client;
int totalCount = 0;
char pageAdd[64];

// set this to the number of milliseconds delay
// this is 30 seconds
#define delayMillis 30000UL

unsigned long thisMillis = 0;
unsigned long lastMillis = 0;

void setup() {
  Serial.begin(9600);

  // disable SD SPI
  pinMode(4,OUTPUT);
  digitalWrite(4,HIGH);

  // Start ethernet
  Serial.println("Starting ethernet...");
  Ethernet.begin(mac, ip, gateway, gateway, subnet);

  // If using dhcp, comment out the line above
  // and uncomment the next 2 lines plus the Ethernet.maintain call in loop

  // if(!Ethernet.begin(mac)) Serial.println(F("failed"));
  // else Serial.println(F("ok"));

  Serial.println(Ethernet.localIP());

  delay(2000);
  Serial.println("Ready");
}

void loop()
{
  // If using dhcp to get an IP, uncomment the next line
  // Ethernet.maintain();
  String dataToSend = "/?SK=DGtest03&I=1&G="+String(random(0,1023))+"&V="+String(random(0,1023))+"&CHECK="+String(random(999999))+"&S=1";
  char rr[64];
  thisMillis = millis();

  if(thisMillis - lastMillis > delayMillis)
  {
    lastMillis = thisMillis;

    // Modify next line to load different page
    // or pass values to server
    dataToSend.toCharArray(rr,64);
   
    sprintf(pageAdd,rr,totalCount);
    printTime();

    // sprintf(pageAdd,"/arduino.php?test=%u",totalCount);

    if(!getPage(server,serverPort,pageAdd)) Serial.print("Fail ");
    else Serial.print("Pass ");
    totalCount++;
    Serial.println(totalCount,DEC);
    
    Serial.println();
  }    
}

byte getPage(IPAddress ipBuf,int thisPort, char *page)
{
    
  int inChar;
  char outBuf[128];

  Serial.print("connecting...");

  if(client.connect(ipBuf,thisPort) == 1)
  {
    Serial.println("connected");
    
    sprintf(outBuf,"GET %s HTTP/1.1",page);
    Serial.println(outBuf);
    client.println(outBuf);
    sprintf(outBuf,"Host: %s",serverName);
    client.println(outBuf);
    client.println("Connection: close\r\n");
    
  }
  else
  {
    Serial.println("failed");
    return 0;
  }

  // connectLoop controls the hardware fail timeout
  int connectLoop = 0;

  while(client.connected())
  {
    while(client.available())
    {
      inChar = client.read();
      //Serial.write(inChar);
      // set connectLoop to zero if a packet arrives
      connectLoop = 0;
    }

    connectLoop++;

    // if more than 10000 milliseconds since the last packet
    if(connectLoop > 10000)
    {
      // then close the connection from this end.
      Serial.println();
      Serial.println("Timeout");
      client.stop();
    }
    // this is a delay for the connectLoop timing
    delay(1);
  }

  //Serial.println();

  Serial.println("disconnecting.");
  // close client end
  client.stop();

  return 1;
}

void printTime(){
          
          time_t t = now();
          //Serial.println("System Date/Time:");         
          Serial.print(day());
          Serial.print(".");
          Serial.print(month());
          Serial.print(".");
          Serial.print(year());
          Serial.print(" ");
  
          Serial.print(hour());
          Serial.print("h ");
          Serial.print(minute());
          Serial.print("m ");
          Serial.print(second());
          Serial.print("s ----  ");
          
          //Serial.print("secondi dall'inizio (1.1.1970) = ");
          //Serial.println(t);
          //Serial.print("ore dall'inizio (1.1.1970) = ");
          //t = t / 3600;
          //Serial.println(t);
}

Any Idea more?
it is important and unfortunately this is blocking my job…

Ciao Domenico

SRV_logs.TXT (245 KB)

I don't use the String data type.

Have you tried my code connecting to another server like Google without any modifications?

What type server are you connecting to?

I had to use String data type to adapt the code to my project, I hope this cannot be the cause of the fail.

The server type information are released from the GET:

connecting...connected HTTP/1.1 200 OK Date: Wed, 11 Mar 2015 21:11:53 GMT Server: Apache/2.4.6 (Ubuntu) X-Powered-By: PHP/5.5.3-1ubuntu2.6 Vary: Accept-Encoding Content-Length: 79 Connection: close

Now I am trying the same code to the Google ( use port 80 because the 778 fails ) and wait for some hours to see if the connection fails again. Thank you..for now.

I use Apache to test my client code. My client code ran for several days without losing the ability to connect, and I tortured it. I unplugged the CAT5 cable from the arduino during downloads, rebooted the router, crashed the server, and everything I could think of, and no crash. It would fail to connect if the router was rebooting, and throw a "timeout" message if I disconnected the cable during a page download or crashed the server, but would go right back to working when the cable was plugged in or the router or server finished rebooting.

Why is your Apache server listening on port 778?

edit: I bet I can do what you are doing without using the String data type. I haven't needed it so far.

Have you checked the Apache server's logs? Maybe there is something there that will help you debug this.

Hello Tim,

After some days of stress and continuous test on Google server, I can say that the code is working fine on my Arduino.
Over 8500 connections have been realized without problems.
Now again I test on my server at the same conditions ( only change the listening port to 778, according top what my server administrator indicated to me) and I expect failure in 12-13 hours.

At moment, I would like to display you the logs answer from Google and from my server. Is it possible that the page sent from my server is not correctly closed? here you are the two logs:

GOOGLE LOGS:

####################TEST SERVER GOOGLE########################

Starting ethernet...
192.168.1.177
Ready
GET /?SK=DGtest03&I=1&G=7&V=514&CHECK=298385&S=1 HTTP/1.1
HTTP/1.1 302 Found
Location: http://www.google.fr/?gws_rd=cr&ei=G4wEVfGMM5GvogSoxoI4
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Set-Cookie: PREF=ID=4577a0e653c55fb6:FF=0:TM=1426361371:LM=1426361371:S=7gEHRwP7qcMSQ0mc; expires=Mon, 13-Mar-2017 19:29:31 GMT; path=/; domain=.google.com
Set-Cookie: NID=67=Bmsw8sc7hty5_9k2YtZ6uxKWJWggRDoPfHaoLJw7fR1sqW7-c-va8zf4w5DnaLj03mVjHoqkF0eFpMzGHQ98HCo6rRMyXvZ01-lNWMkpGpbFSixdXcRkHbN7IxvsjLas; expires=Sun, 13-Sep-2015 19:29:31 GMT; path=/; domain=.google.com; HttpOnly
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Date: Sat, 14 Mar 2015 19:29:31 GMT
Server: gws
Content-Length: 256
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic,p=0.5
Connection: close

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.fr/?gws_rd=cr&amp;ei=G4wEVfGMM5GvogSoxoI4">here</A>.
</BODY></HTML>
disconnecting.
Pass 1

MY SERVER LOGS:

############################################################################
Starting ethernet...
192.168.1.177
Ready
connecting...connected
GET /?SK=DGtest03&I=1&G=914&V=543&CHECK=635129&S=1 HTTP/1.1

HTTP/1.1 200 OK
Date: Sat, 14 Mar 2015 19:33:23 GMT
Server: Apache/2.4.6 (Ubuntu)
X-Powered-By: PHP/5.5.3-1ubuntu2.6
Vary: Accept-Encoding
Content-Length: 462
Connection: close
Content-Type: text/html



Riscontrato contatore con ID DGtest03 - Aggiornamento ROWS.



query update UPDATE  `telemetria`.`volture_new` SET  `I` =  I + '1',`G` =  '914',`V` =  '543', `checksum` =  '635129',`S` =  '1' , `timestamp` = now() WHERE  `volture_new`.`SK` ='DGtest03' 



Aggiornamento del contatore DGtest03 Avvenuto Correttamente

 <div id="ack" style="visibility: hidden;">lenght=27 RECEIVED#SK=DGtest03&I=1&G=914&V=543</div>disconnecting.
Pass 2

it seems that the html code does not send correctly a page to Arduino ( it is missing …).

Do you think is it this the reason to close wrongly a socket and generate future fails of connections?

Thank you in advance.
Domenico

It appears something is wrong with the page on your server. I have no problems using port 80 for localnet traffic.

I get the same results with my Apache server as you get with Google. Days and days of uninterrupted service.