Problem with multpile (fast) client-requests (Eth)

Mark,

Looks like you've found the solution. I was having the same exact issue. Putting in a delay is not reliable. Instead, looping until the connection ends works like a charm. Thanks!

Paul

This post deals with the same problem in a slightly different way: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1235991468/14#14

Looping until !connected() can still cause problems, because of the way the Ethernet library determines connection status.

I'm having similar problems... did you ever figure out how to fix this?

Here's some more information about this problem:
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1238295170

In addition to the loop to check whether the connection has been closed, I noticed that a few well placed delay() statements are necessary. Otherwise, after 4 or 5 connections the ethernet board 'locks up'. More specifically, the tx LED on the ethernet board goes nuts but othewise all communications cease. Here is my code after I put in the delay statements. Works fine now. Disclaimer: I am very new to Arduino.

#include <Ethernet.h>

byte mac[] = {
0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = {
x, x, x, x };
byte server[] = {
y, y, y, y };
Client client(server, 86);

int sen02=0;

void setup()
{
Ethernet.begin(mac, ip);
pinMode(2, INPUT);
delay(1000);
}

void loop()
{

if(digitalRead(2)!=sen02){
sen02=digitalRead(2);
client.connect();
delay(1000);
if(client.connected(){
client.println("GET /HA/Event.php?Sen=2&Sta=1&Des=Text1 HTTP/1.0") ;
delay(500);
client.println();
delay(500);
while(client.connected()) {
client.flush();
client.stop();
delay(1000);
}
}

}

}

Here's a fix (at least a work-around) to the multiple connect problem. The problem seems to be in how the TCP stack on the Wiznet chip is handling disconnects. It looks like it's failing to ACK the final FIN when the connection is closing. This leaves the socket in the SOCK_FIN_WAIT state from the client (Wiznet chip), and in the TIME_WAIT state on the server. Eventually these states timeout after about 30 seconds and the socket moves to a SOCK_CLOSED state.

So as you initiate each connection and subsequently close it (or it gets closed by the webserver), the one of the four available sockets for connections gets "stuck" waiting for the TCP connection to close (because of the FIN/ACK issue). After four connections in less than about 30 seconds, all of the sockets are in use (stuck waiting) and no more connections can be established. As each stock socket times out, then another connection can be made.

So the work-around is to change the Client connect() function to consider a socket in the SOCK_FIN_WAIT state available. This gets around the 30 second delay while the state times out. This solution is not ideal as it's not very nice to the server you connected to since it will end up with a bunch of TIME_WAIT connections that will eventually timeout. This could be bad if you make a lot of connections very quickly. It's what's happening now anyway, but you can only generate 4 at a time so it's not so bad on the server. The real fix is probably needed in the Wiznet TCP/IP stack on the chip - maybe there's an updated firmware??

Here's the revised connect() function in Client.cpp. Note that there are a couple of other enhancements to:

  1. More intelligently choose the available socket instead of always picking the last one.
  2. Bugfix for the _srcport overflowing and causing the connection to come from a privileged port.
uint8_t Client::connect() {

  uint8_t socketStatus;
  uint8_t i=0;

  _sock = 255;

  while ((i < MAX_SOCK_NUM) && (_sock == 255)) {
    socketStatus = getSn_SR(i);
    if ((socketStatus == SOCK_CLOSED) || (socketStatus == SOCK_FIN_WAIT)) {
      _sock = i;
    } else {
      i++;
    }
  }
  
  if (_sock == 255)
    return 0;
    
  _srcport++;

  if (_srcport > 64511)
    _srcport = 0;
    
  socket(_sock, Sn_MR_TCP, 1024 + _srcport, 0);
  
  if (!::connect(_sock, _ip, _port))
    return 0;
    
  while (status() != SOCK_ESTABLISHED) {
    if (status() == SOCK_CLOSED)
      return 0;
  }
  
  return 1;
}

Also, if we're going to consider the SOCK_FIN_WAIT state to be disconnected and eligible for new connections, the we probably need to update the connected() function as well:

uint8_t Client::connected() {
  uint8_t s = status();
  return !(s == SOCK_LISTEN || s == SOCK_CLOSED || (s == SOCK_CLOSE_WAIT && !available()) || (s == SOCK_FIN_WAIT));
}

With these changes I was able to make unlimited client connections to a webserver.

That's exactly what I was seeing, my fix was just to actually wait until the socket was closed. It still let me get a few requests per second.

Great work! The Wiznet chip is amazing, it's bad to see so much of its power go to waste. Wiznet supplies code for DHCP, which might be interesting to get running sometime.

It looks like you guys fixed the problem!

I'm using code based on the example posted by NJPaul and the client.cpp modifications suggested by etracer, great work!

Oh and this patch is meant for the standard Ethernet library, it doesn't work with Ethernet2 yet.

Will this patch find it's way to the official distribution?

I have applied the patches by etracer and used the example code by NJPaul, nothing has changed. I still get the same SYN, RST in wireshark session logs.

Did you remember to delete the Client.o file in the library directory? If not, the the library didn't recompile.

Yes, I did, but no luck again. I'll try again sometime available.

Oh and this patch is meant for the standard Ethernet library, it doesn't work with Ethernet2 yet.

Will this patch find it's way to the official distribution?

I've only done a casual look at the Ethernet2 library, but from what I saw the Client::connect() code looked identical to the "official" library.

I'll be writing up this patch and submitting it to the developer's list. There are a couple of other bugs and enhancements that I'm still looking into.

  1. The client can get into an unrecoverable state if the sketch opens connections and then fails to call client.stop() before issuing another client.connet(). After the fourth connect all sockets will be in use preventing any more connections. The insidious part is that there's no way to release the orphaned connections. Easily fixed by having the connect() return a fail result if there is already and active connection.

  2. There are some situations where the client.stop() fails to disconnect and leaves an "ESTABLISHED" connection to the server. I'm still researching this.

I have applied the patches by etracer and used the example code by NJPaul, nothing has changed. I still get the same SYN, RST in wireshark session logs.

SYN and RST are not bad things. The SYN is the initial state to start the TCP session. If the server is issuing you a RST then you're likely trying to connect to a non-existent service or you're firewalled, etc.

The patches have nothing to do with the ability to connect as that wasn't broken. It's the ability to open repeated connections that was the issue.

I think you have another configuration issue.

When anyone is looking into this issue, have they gone back to the original datasheets to see what's "supposed" to happen according to WIZNet?

I seem to recall the datasheets had some state flow diagrams in them.

There's also this: http://www.wizwiki.net/tc/regina/entry/W5100-Hardware-Design-Guide-2-MCU-Porting

--Phil.

Just a quick note that I think I've found the root cause of the incomplete disconnects that result in subsequent failed connects.

The client.stop() function is improperly terminating the connection. I'll post more here later after I finalize testing.

I've posted a completely revised Client portion of the Ethernet library. It includes many fixes and seems to resolve any connection issues or situations where a connection fails to completely close.

See here: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1238640832/0#0

Looks good. I will use it and let you know how it goes, and if I come up with any more fixes. I was thinking of putting in some functions to make it easy to do UDP as well.

I'm using the shield to log remote temperatures, pressures and humidity, UDP would be marvelous for logging.

etracer,

I'm using arduino 17
do you know if any of these fixes are included in ver17?

I'd hate replace it with an older fix
Thanks
Jeff

Yes, all of the client fixes I submitted are included in Arduino 0017.

Thanks
I've been doing so much reading about these problems it was starting to get hard to tell if it applied to 17 or not.

I am trying to send a very small amount of info to a php script on my webserver once every second and i am getting some intermittent failures

can you post a simple example that uses the libraries that are built into arduino 17 that will send an HTTP GET once per second?

I'm just trying to see what kinds of defensive coding you are using to deal with these problems

Thanks in advance
Jeff