Pages: [1] 2   Go Down
Author Topic: Problem with multpile (fast) client-requests (Eth)  (Read 4773 times)
0 Members and 1 Guest are viewing this topic.
0
Offline Offline
Newbie
*
Karma: 0
Posts: 8
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hello!

I'm new to Arduino, and want to do the following.
I want simple board with a button, when i press the button i want it to do a http-request, and display contents on the mac using serial connection.

This shouldn't be too hard..

I thought I'd best approach this problem one sub-problem at the time.
My first step was to create the http-request and serial display.

I found the code below, and understand what it does..
I inserted the delay(500); pieces myself.

Problem with code below is that sometimes it can't connect to the server, I don't understand why cause it really is up. When I increase the amount of delay it seems to go better. But I want to be able to push the button (in the final) project multpile times per second....

This should be possible, shouldn't is? Could the problem be serverside? If anyone can help, it would help me a lot.

Thanks!

Code:
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 10,0,0,7 };
byte server[] = {10,0,0,1};


Client client(server, 80);

enum CLIENT_STATE
{  
      STEP1, STEP2, STEP3, STEP4
};

static CLIENT_STATE client_state;

void setup()
{
  Ethernet.begin(mac, ip);
  Serial.begin(9600);
  
  delay(1000);
}

void loop()
{
  
  if (client_state==STEP1)
  {
    Serial.println("connecting...");
  
  if (client.connect()) {
    Serial.println("connected");
    
    client.println("GET /index.html HTTP/1.0");
    client.println("Host: localhost");
    client.println("User-Agent: AVR ethernet");
    client.println("Accept: text/html");

    client.println();
  

    client_state=STEP2;
    delay(500);
  } else {
    Serial.println("connection failed");
    delay(500);
    client_state=STEP1;
  }
  }
  
    if (client_state==STEP2)
  {
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
    client_state=STEP2;
  }
  else if (!client.connected())
  {    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
    delay(500);
      client_state=STEP1;
    }

  }
}
Logged

St. Louis MO
Offline Offline
Jr. Member
**
Karma: 0
Posts: 83
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I've been having a similar problem, but not related to the speed of requests.

I have mine set up to pull a web page every 60 sec, and I have an LED indicate when there was a connection failure. Maybe I should log the failures so I know for sure how often they happen, but it seems to have a fair number of them throughout the day.

The server is of course up and responding, but I'm not sure what to look at in my Arduino to see what the problem is.
Logged

- Jerry Adlersfluegel

0
Offline Offline
Newbie
*
Karma: 0
Posts: 8
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I've rewritten the code. Now it is only the essence...
I do a request every second with the doRequest function... It works a couple of times, but then it suddenly can't connect anymore (i keep getting "Can't get a connection!").

After a while it suddenly can connect again, and i get results from te server again. Most of the time it works once or twice, and then the procees repeats... (can't get a connection, and after a while CAN get a connection.)

Does anyone has an idea where the problem could be?
Code:
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 10, 0, 0, 12 };
byte server[] = {10,0,0,1 }; // local ip

Client client(server, 80);

void setup()
{
  Ethernet.begin(mac, ip);
  Serial.begin(9600);
}

void loop()
{
  doRequest();
  delay(1000);
}

void doRequest(){
    
  Serial.println("doRequest Function");
  while(!client.connect()) {
    Serial.println("Can't get a connection!");
  }
  Serial.println("(Finally) got a connection, so send get-request.....");
   client.println("GET /index.html HTTP/1.0");
   client.println();
  Serial.println("Reading incoming data");
  incoming();
  client.stop();
  Serial.println('socket stop');
}

void incoming(){
  while (client.available()) {
    char c = client.read();
    Serial.print(c);
  }
  
  if (!client.connected()) {
    Serial.println("Disconnected.");
  }
}

Logged

London, Ontario, Canada
Offline Offline
Jr. Member
**
Karma: 0
Posts: 90
Arduino rocks
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Any chance the connection is not fully stopped between tries?

Instead of relying on a delay(), try checking the status of the connection.

something like so:

while ( client.connected )
 {
 client.stop();
 Serial.println('stopping the connection... ');
 }

Serial.println('socket stop');


I have not played with the ethernet lib yet. Does the status change to disconnected right away or does it takes a few moments depending on the response of the remote machine?
Does the Stopping....  message always print exactly once?

In vb6, I have found that it takes time for the connection to fully stop after I issue the disconnect request. I have not yet played with my ethernet shield. Ideally you would say  while ( !client.disconnected ) but I have not verified the types of 'states' that the library provides.

example:  'not connected' is not the same as 'disconnected' because we could be in transition of states.

Logged

0
Offline Offline
Newbie
*
Karma: 0
Posts: 3
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Mark,

 Looks like you've found the solution. I was having the same exact issue. Putting in a delay is not reliable. Instead, looping until the connection ends works like a charm. Thanks!

Paul

Logged

Pullman, WA
Offline Offline
Edison Member
*
Karma: 11
Posts: 1245
Arduino Ninja
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

This post deals with the same problem in a slightly different way: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1235991468/14#14

Looping until !connected() can still cause problems, because of the way the Ethernet library determines connection status.
Logged

Wearable Arduino-compatible LED matrix glasses: https://www.kickstarter.com/projects/macetech/rgb-led-shades

0
Offline Offline
Newbie
*
Karma: 0
Posts: 23
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I'm having similar problems... did you ever figure out how to fix this?

Here's some more information about this problem:
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1238295170
Logged

0
Offline Offline
Newbie
*
Karma: 0
Posts: 3
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

In addition to the loop to check whether the connection has been closed, I noticed that a few well placed delay() statements are necessary. Otherwise, after 4 or 5 connections the ethernet board 'locks up'. More specifically, the tx LED on the ethernet board goes nuts but othewise all communications cease. Here is my code after I put in the delay statements. Works fine now. Disclaimer: I am very new to Arduino.

#include <Ethernet.h>

byte mac[] = {
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = {
  x, x, x, x };
byte server[] = {
  y, y, y, y };
Client client(server, 86);


int sen02=0;

void setup()
{
  Ethernet.begin(mac, ip);
  pinMode(2, INPUT);
  delay(1000);
}

void loop()
{

if(digitalRead(2)!=sen02){
sen02=digitalRead(2);
  client.connect();
  delay(1000);
  if(client.connected(){
    client.println("GET /HA/Event.php?Sen=2&Sta=1&Des=Text1 HTTP/1.0") ;
    delay(500);
    client.println();
    delay(500);
    while(client.connected()) {
      client.flush();
      client.stop();
      delay(1000);
      }
    }
  
}

}
Logged

Indiana
Offline Offline
Full Member
***
Karma: 1
Posts: 234
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Here's a fix (at least a work-around) to the multiple connect problem. The problem seems to be in how the TCP stack on the Wiznet chip is handling disconnects. It looks like it's failing to ACK the final FIN when the connection is closing. This leaves the socket in the SOCK_FIN_WAIT state from the client (Wiznet chip), and in the TIME_WAIT state on the server. Eventually these states timeout after about 30 seconds and the socket moves to a SOCK_CLOSED state.

So as you initiate each connection and subsequently close it (or it gets closed by the webserver), the one of the four available sockets for connections gets "stuck" waiting for the TCP connection to close (because of the FIN/ACK issue). After four connections in less than about 30 seconds, all of the sockets are in use (stuck waiting) and no more connections can be established. As each stock socket times out, then another connection can be made.

So the work-around is to change the Client connect() function to consider a socket in the SOCK_FIN_WAIT state available. This gets around the 30 second delay while the state times out. This solution is not ideal as it's not very nice to the server you connected to since it will end up with a bunch of TIME_WAIT connections that will eventually timeout. This could be bad if you make a lot of connections very quickly. It's what's happening now anyway, but you can only generate 4 at a time so it's not so bad on the server. The real fix is probably needed in the Wiznet TCP/IP stack on the chip - maybe there's an updated firmware??

Here's the revised connect() function in Client.cpp. Note that there are a couple of other enhancements to:
1. More intelligently choose the available socket instead of always picking the last one.
2. Bugfix for the _srcport overflowing and causing the connection to come from a privileged port.

Code:
uint8_t Client::connect() {

  uint8_t socketStatus;
  uint8_t i=0;

  _sock = 255;

  while ((i < MAX_SOCK_NUM) && (_sock == 255)) {
    socketStatus = getSn_SR(i);
    if ((socketStatus == SOCK_CLOSED) || (socketStatus == SOCK_FIN_WAIT)) {
      _sock = i;
    } else {
      i++;
    }
  }
  
  if (_sock == 255)
    return 0;
    
  _srcport++;

  if (_srcport > 64511)
    _srcport = 0;
    
  socket(_sock, Sn_MR_TCP, 1024 + _srcport, 0);
  
  if (!::connect(_sock, _ip, _port))
    return 0;
    
  while (status() != SOCK_ESTABLISHED) {
    if (status() == SOCK_CLOSED)
      return 0;
  }
  
  return 1;
}

Also, if we're going to consider the SOCK_FIN_WAIT state to be disconnected and eligible for new connections, the we probably need to update the connected() function as well:

Code:
uint8_t Client::connected() {
  uint8_t s = status();
  return !(s == SOCK_LISTEN || s == SOCK_CLOSED || (s == SOCK_CLOSE_WAIT && !available()) || (s == SOCK_FIN_WAIT));
}

With these changes I was able to make unlimited client connections to a webserver.
Logged

Pullman, WA
Offline Offline
Edison Member
*
Karma: 11
Posts: 1245
Arduino Ninja
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

That's exactly what I was seeing, my fix was just to actually wait until the socket was closed. It still let me get a few requests per second.

Great work! The Wiznet chip is amazing, it's bad to see so much of its power go to waste. Wiznet supplies code for DHCP, which might be interesting to get running sometime.
Logged

Wearable Arduino-compatible LED matrix glasses: https://www.kickstarter.com/projects/macetech/rgb-led-shades

0
Offline Offline
Newbie
*
Karma: 0
Posts: 23
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

It looks like you guys fixed the problem!

I'm using code based on the example posted by NJPaul and the client.cpp modifications suggested by etracer, great work!

Oh and this patch is meant for the standard Ethernet library, it doesn't work with Ethernet2 yet.

Will this patch find it's way to the official distribution?

Logged

0
Offline Offline
Newbie
*
Karma: 0
Posts: 5
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I have applied the patches by etracer and used the example code by NJPaul, nothing has changed. I still get the same SYN, RST in wireshark session logs.
Logged

Indiana
Offline Offline
Full Member
***
Karma: 1
Posts: 234
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Did you remember to delete the Client.o file in the library directory? If not, the the library didn't recompile.
Logged

0
Offline Offline
Newbie
*
Karma: 0
Posts: 5
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Yes, I did, but no luck again. I'll try again sometime available.
Logged

Indiana
Offline Offline
Full Member
***
Karma: 1
Posts: 234
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
Oh and this patch is meant for the standard Ethernet library, it doesn't work with Ethernet2 yet.

Will this patch find it's way to the official distribution?

I've only done a casual look at the Ethernet2 library, but from what I saw the Client::connect() code looked identical to the "official" library.

I'll be writing up this patch and submitting it to the developer's list. There are a couple of other bugs and enhancements that I'm still looking into.

1. The client can get into an unrecoverable state if the sketch opens connections and then fails to call client.stop() before issuing another client.connet(). After the fourth connect all sockets will be in use preventing any more connections. The insidious part is that there's no way to release the orphaned connections. Easily fixed by having the connect() return a fail result if there is already and active connection.

2. There are some situations where the client.stop() fails to disconnect and leaves an "ESTABLISHED" connection to the server. I'm still researching this.
Logged

Pages: [1] 2   Go Up
Jump to: