Go Down

Topic: new library UIPEthernet: uIP adapted to Arduino for ENC28J60 (Read 53466 times) previous topic - next topic

ntruchsess

new release v1.08

Using this new release will improve stability by a lot as it fixes a crash of enc28j60-chip that happened when the library tried to send a zero-length packet.

TCP-socket-connections are more responsive now. Outgoing data is send aprox. 10ms after last call to Client.write(). This will noticably decrease the responsetime of request->response type applications (like webserver).

ntruchsess

new release UIPEthernet v1.09

the new tcp fast polling of clients introduced in release 1.08 resulted in spurious invalid packets with checksum errors and invalid data, that is fixed now.
I also managed to save 54 bytes of flash in comparism to 1.08

tigers

Hi, thanks for the great work, the library is working great for me.

I've just one problem: I want to use the arduino both as a server (command webpage with form sending data with POST method) and as a client to send data to emoncms.org via a GET method. From my testing everything works great as long as the 2 things don't happen simultaneously, in which case everything hangs: the server no longer responds and the client no longer connects. I think it's due to the limits of the ENC28J60, since with the w5100 the code works fine.

Since it'd be no problem for me if the send part is delayed while the server is connected to another client I'm looking for a way to prevent that. I see on Github (and trtied myself) that I can't use if (client.available()) since it may return false after a client.stop() even when there still are incoming or outgoing data.
I tried a client.flush() right after the client.stop() and things seemed to get better but still there's a 50-50 chance that a new client connection to the server will hang everything.

Is there a way I can prevent the sending client to connect while there is another client open being served by the server part of the sketch? Or maybe I'm doing something wrong somewhere else?

I'd post the entire code but it's too large for the forum so:
the server part:

Code: [Select]

//WEB SERVER
  client = server.available();
  if (client) {
    while (client.connected()) {
        if (client.available()) {
        Serial.print(F("client OK - free RAM: "));
        Serial.println(freeRam());
        // GET, POST, or HEAD
        memset(buffer,0, sizeof(buffer)); // clear the buffer
        if(client.readBytesUntil('/', buffer,sizeof(buffer))){
          Serial.println(F("check 5"));
          if(strcmp(buffer,"POST ") == 0){ //strcomp restituisce 0 se le 2 stringhe sono uguali
            client.find("\n\r"); // skip to the body
            // find string starting with "pin", stop on first blank line
            // the POST parameters expected in the form pinDx=Y
            // where x is the pin number and Y is 0 for LOW and 1 for HIGH
           
            //findUntil cerca la stringa il primo parametro fino a che raggiunge la stringa a secondo parametro
            //così cerca pinD fino a che lo trova OPPURE finché raggiunge una nuova riga (cioè alla fine dei parametri in ingresso via POST
           
            Serial.println(F("Handling POST header"));
           
            while(client.findUntil("pinD", "\n\r")){
              Serial.print(F("pinD parameter found - free RAM: "));
              Serial.println(freeRam());
              int pin = client.parseInt();       // the pin number
              int val = client.parseInt();       // 0 or 1
             
              if (pin == 999) { //pin 999 è in realtà un setpoint
                setpoint = val;
                chkTermostato();
                Serial.print(F("check 2 - val= "));
                Serial.println(val);
              }//END if pin==999
             
              /*else { //se non è pin999 allora è un comando per il pin corrispondente
                //pinMode(pin, OUTPUT);
                //digitalWrite(pin, val);
              }//END ELSE*/
              Serial.println(F("check 4"));
              client.flush();
              break;
            }//END while client.findUntil
          //dopo aver letto i dati serve un flush o l'ENC28J60 si blocca? (troppi client aperti? Memory leak?)
          //suggerito dal creatore della libreria UIPEthernet su: http://forum.arduino.cc/index.php/topic,221395.0.html
          //client.flush();
          Serial.println(F("check 1"));
          //client.find("\n\r");
          //client.stop();
          }//END if(strcmp(buffer,"POST ") == 0)
         
          sendHeader(client,"Termostato remoto");

        THE CODE TO DISPLAY THE WEBPAGE IS HERE 


          client.println(F("</body></html>"));
          client.stop();
          Serial.print(F("client CLOSED 1 - free RAM: "));
          Serial.println(freeRam());
        }//END if(client.readBytesUntil
        Serial.println(F("check 6"));
        break;
      }//END if client.available
    }//END while client.connected
    // give the web browser time to receive the data
    delay(1);
    client.stop();
    client.flush();


and the client part (here the client instance is named "client", but I have a version in which it was named "client2" - same result):

Code: [Select]

//SEND DATA TO EMONCMS
void sendData()
{
  // if there's a successful connection:
    client = server.available();
    if (client.connect(server2, 80)) {
        Serial.println(F("Connecting to EmonCMS..."));
        // send the HTTP GET request:
        client.print(F("GET /api/post?apikey="));
        client.print(apikey);
        if (node > 0) {
          client.print(F("&node="));
          client.print(node);
        } //END IF node > 0
        client.print(F("&json={TOliviero"));
        client.print(":");
        client.print(DHT.temperature);
        client.print(",");
        client.print(F("HROliviero"));
        client.print(":");
        client.print(DHT.humidity);
        client.println(F("} HTTP/1.1"));
        client.println(F("Host:emoncms.org"));
        client.println(F("User-Agent: Arduino-ethernet"));
        client.println(F("Connection: close"));
        client.println();
        //Aggiunta per tentativo debug ENC
        client.stop();
        client.flush();   
    } //END IF client.connect
  else {
    /*
    // if you couldn't make a connection:
    Serial.println(F("Connection failed"));
    Serial.println(F("Disconnecting from EmonCMS..."));
    */
    client.stop();
    client.flush();
  }//END ELSE
}//END sendData()

ntruchsess

#183
Oct 13, 2014, 04:32 pm Last Edit: Oct 13, 2014, 04:36 pm by ntruchsess Reason: 1
- Don't call connect on a client-instance that is returned by server.available(). Just declare a separate client instance and use that. Server.available will not return this separate client-instance as it has a different local port than the Server-instance got assigned by Server.begin(<port>);

- no need to call flush() after stop().

- no need to use delay() to give some time to the remote side - you should call stop() right away, the library will send outstanding data after having called stop().

- be aware that client.connect(server,port) may return values < 0 if 'server' is a string and not type IPAddress, so better use 'if (client.connect(...) > 0).

- by calling stop() the client-instance is invalidated and cannot be used for anything else than initiating a new connection by calling 'connect()'. This implies all unread incoming data is discarded the moment you call stop() (and client.available() just returns 0 after calling stop(), write returns -1).

- make sure every execution of 'loop' does call into UIPEthernet. If you are not calling e.g. server.available() or client.available()  in every loop anyway, you may use Ethernet.maintain() for that. This ensures packets (e.g. icmp or arp) are processed even when no intentional traffic via client-interface is transfered.

regards,

Norbert

tigers

Hi, thank you very very much for the help.
Following your advice I modified the code as follows.
I couldn't add Ethernet.maintain() because it adds a whole 4k (!) to the sketch size and it didn't fit the memory, but I have the clien = server.available() called at every loop so it should be fine, right?
Also the server IP  is an array of byte so I left the =0 condition for now.

Things got a little better (the hangs occur less often) but the problem is still there: at random in 5 or so minutes of trying the code hangs somewhere and everything stops responding. I put the exact same code on a MEGA+w5100 and it works flawlessly even if I keep hammering the server with refreshes and POST requests while it is sending data as a client.

So if you can confirm that there are no more conceptual errors in my code I starting to think of some sort of hardware limit of the ENC28J60 (I am not thinking of the Arduino since I see no memory leak and there should be about 0.5/0.6 Kb of SRAM free) when trying to serve both clients at the same time.
And I'm back to my original question: is it possible to determine if the communication for a client is completely ended after client.stop()? I can accept to miss a data send  while serving the web page.

Thank you for your help, hope I can give back some time...

WEB SERVER CODE:
Code: [Select]

//WEB SERVER
  //Ethernet.maintain();
 
  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
        if (client.available()) {
        // GET, POST, or HEAD
        memset(buffer,0, sizeof(buffer)); // clear the buffer
        if(client.readBytesUntil('/', buffer,sizeof(buffer))){
          if(strcmp(buffer,"POST ") == 0){ //strcomp restituisce 0 se le 2 stringhe sono uguali
            client.find("\n\r"); // skip to the body
         
            while(client.findUntil("pinD", "\n\r")){
              int pin = client.parseInt();       // the pin number
              int val = client.parseInt();       // 0 or 1
             
              if (pin == 999) { //pin 999 è in realtà un setpoint
                setpoint = val;
                chkTermostato();
              }//END if pin==999
             
              /*else { //se non è pin999 allora è un comando per il pin corrispondente
                //pinMode(pin, OUTPUT);
                //digitalWrite(pin, val);
              }//END ELSE*/
              break;
            }//END while client.findUntil
          //dopo aver letto i dati serve un flush o l'ENC28J60 si blocca? (troppi client aperti? Memory leak?)
          //suggerito dal creatore della libreria UIPEthernet su: http://forum.arduino.cc/index.php/topic,221395.0.html
         
          Serial.println(F("check 1"));
         
          }//END if(strcmp(buffer,"POST ") == 0)
         
          sendHeader(client,"Termostato remoto");
...           
HERE IS THE CODE FOR THE PAGE
...

          client.println(F("</body></html>"));
          client.stop();
     
        break;
      }//END if client.available
    }//END while client.connected

    client.stop();
  }//END if (client)

THE CLIENT:
Code: [Select]

//SEND DATA TO EMONCMS
void sendData()
{
  // if there's a successful connection:
    if (client2.connect(server2, 80)) {
        Serial.println(F("Connecting to EmonCMS..."));
        // send the HTTP GET request:
        client2.print(F("GET /api/post?apikey="));
        client2.print(apikey);
        if (node > 0) {
          client2.print(F("&node="));
          client2.print(node);
        } //END IF node > 0
        client2.print(F("&json={TOliviero"));
        client2.print(":");
        client2.print(DHT.temperature);
        client2.print(",");
        client2.print(F("HROliviero"));
        client2.print(":");
        client2.print(DHT.humidity);
        client2.println(F("} HTTP/1.1"));
        client2.println(F("Host:emoncms.org"));
        client2.println(F("User-Agent: Arduino-ethernet"));
        client2.println(F("Connection: close"));
        client2.println();
       
        client2.stop();
       
    } //END IF client2.connect
  else {
    /*
    // if you couldn't make a connection:
    Serial.println(F("Connection failed"));
    Serial.println(F("Disconnecting from EmonCMS..."));
    */
    client2.stop();
   
  }//END ELSE
}//END sendData()


ntruchsess

#185
Oct 14, 2014, 06:25 pm Last Edit: Oct 14, 2014, 06:27 pm by ntruchsess Reason: 1
Ethernet.maintain() will pull in the dhcp-related code. You can avoid this by setting #define UIP_CONF_UDP 0 in utility/uipethernet-conf.h. In terms of triggering package-processing calling server.available() is equivalent (without pulling the dhcp-code).

To answer your question: When stop is called the connection is marked as 'to be closed' and will be processed by the library until either all outgoing data is being delivered or the remote side closes the socket. As long there is undelivered outgoing data associated with this client client.connected() will return 1. But you cannot send further outgoing data on a client after calling stop(). You also do not need to wait until outgoing data is delivered as this is done by the library 'under the hoods' (e.g. while calling server.available() or even 'client.write()'). Having sayed this your 'server'-code seems to be fine as the 'break'-statement will leave the while-loop after response is finished (and the second call to client.stop() will cause no harm).

Regarding the hang: Are you on Version 1.09 that I released few days ago? I've found and fixed a bug that caused the enc28j60-chip to fail.





tigers

OK, thanks to your help I finally got it. Thank you again.

I modified the code as follows, and everything works fine now - I just have some delay connecting to the server while a sending is in progress, but 2/3 tries in that case will solve the issue - this is acceptable for my use. But now the server has been stable and responsive in all its functionality for almost 2 days now.

I also upgraded to 1.09 as you suggested but the upgrade alone did not solve the problem so I must assume that, at least in my particular configuration and with my code, having a client connecting to a remote server while another is connecting to the arduino web server will block the ENC28J60. Hope this will help someone.

THE SERVER PART - I added a check to see if the client2 that posts the data on EMONCMS is connected and if so wait until it has done sending:
Code: [Select]
//WEB SERVER
  //Ethernet.maintain();
  client = server.available(); 
  if (client) {
    if (client2.connected()){
      while (client2.connected()); //aspetta che client2 abbia finito per non sovrapporsi e sovraccaricare l'ENC28J60
    }
    while (client.connected()) {
        if (client.available()) {
       ... ETC - DO THE WEB SERVER STUFF


CLIENT - this is easier since I don't mind skipping data so if the main client is connected to the server I just skip - no need to wait until all data has been sent:
Code: [Select]

if (!client.connected()) {
      sendData();
      Serial.println("Finish sending to EMON"); 
    }//END IF
    else {
        Serial.println("EMON sending skipped"); 
    }//END ELSE


Eli81

Hi Norbert,

First, thank you for the wonderful library and your effort to make it such.

I'm having a little issue in that server.available() only returns 1 if there is data available to be read from the client. Did you do this to emulate the Ethernet library? I would consider this a bug - it doesn't make very much sense that the client would have to send data before the server even knows someone is there.

Anyway, in the Ethernet library, this is easy to fix:

Code: [Select]
EthernetClient EthernetServer::available()
{
  accept();

  for (int sock = 0; sock < MAX_SOCK_NUM; sock++) {
    EthernetClient client(sock);
    if (EthernetClass::_server_port[sock] == _port &&
        (client.status() == SnSR::ESTABLISHED ||
         client.status() == SnSR::CLOSE_WAIT)) {
      //if (client.available()) { // comment the if statement out, and it will return 1 just if a client is connected, as would be expected.
        // XXX: don't always pick the lowest numbered socket.
        return client;
     // }
    }
  }


But I'm a terrible n00b and can't figure out how to change your code to have the same effect. Any help would be greatly appreciated!

Thanks

SurferTim

I'm having a little issue in that server.available() only returns 1 if there is data available to be read from the client. Did you do this to emulate the Ethernet library? I would consider this a bug - it doesn't make very much sense that the client would have to send data before the server even knows someone is there.
Why would you consider this a bug? You are asking the W5100 or ENC28J60 if any of the connected clients have sent data (characters) to the server. If you want to use that function you have recommended, you should rename it to something like server.connected().

Eli81

I'm having a little issue in that server.available() only returns 1 if there is data available to be read from the client. Did you do this to emulate the Ethernet library? I would consider this a bug - it doesn't make very much sense that the client would have to send data before the server even knows someone is there.
Why would you consider this a bug? You are asking the W5100 or ENC28J60 if any of the connected clients have sent data (characters) to the server. If you want to use that function you have recommended, you should rename it to something like server.connected().

Fair enough. Maybe I should just rephrase the question all together.

Is there a way to know that a client has connected without said client having to send data first?  Certainly this can't be expected behavior? I should be able to send data over my open connection as soon as it is opened. No?

SurferTim

I wish there was a simple way. I can use the w5100.h and w5100.cpp files to check for connected clients that have not sent anything, but I haven't experimented with the enc28j60 library yet. I don't have one of those ethernet shields to experiment with.



Eli81

Like I said, the modification that I showed to EthernetServer.cpp effectively does this. It makes server.available() return 1 just if a client is connected, even if there is no data to be read from the client.

This allows something like a telnet server to work, where the client connects and the server just starts spitting out data. That's what I'm trying to do.

Think of it as an Arduino BBS. :) The server spits out data as long as the connection is open, and I can send responses at any time as well. Works great with the W5100.

But I would really like to do this with the ENC28J60 and UIPEthernet library.

SurferTim

It appears your code will return the first socket with a client connected. I use a function that returns an array that shows the status of all sockets. That way I can see if there is more than one client connected.


Go Up