Problems with Time consumption of Ethernet / Ethernet2 Server (html)

Hello Folks,

I do use Arduino Uno Wifi Rev.2 mit megaAVR4809 (but dont use the Wifi) with an Ethernet2 Shield with W5500 chipset. Basically I made a Dali Master Interface for 64 Lamp slaves which is controlled by a simple Website with Ajax script. The Website is on MemoryCard in the shield.

Well - basically it’s working fine. Up to this moment, where aprox. 2-8 times a workday the ethernet interface seems to hang, any further changing of szenario, reloading the page, etc. is not possible anymore.
Later I tried to program the watchdog, which has a maximum time of 8.2s before calling the ether_loop().

The same occured with a Mega2560 and Ethernet Board with 5100 chipset.

Now, every time when the ethernet server hangs, the watchdog restarts, lights blink on/off, and the last szenario is lost. The problem is even more often, when the router has port open to control lights from outside. :o
latest version of Library I use here is Ethernet V2.0 (for all common chipsets).

After reading many post here, me thinks that serveral people having similar problems, but not exakt same behavior.

Of course, without any application, the library works always fine. Very sometimes you want to use your embedded controller for some purpose, anyhow.

Please, does anybody know about the problem ?
How much time does the ethernet server functions need maximum in the loop ?
What can occure, when client connected or even if not, when server loop exceeds 8200ms ?
What else did I not recognize ?

Thank you for supporting me in advance.

/*****************************************************
 ***       AJAX web server for Dali commands       ***
 ***  Written by Ing. Rudolf Mörth in March 2018   ***
 ***                                               ***
 *** EhternetServer.ino  V1.4  27.4.2018           ***
 *****************************************************

 Description:  Arduino web server that allows DALI master
               commands to be send to DALI slaves,
               so performs a DALI Gateway.
               The web page is stored on the micro SD card.
  
 Hardware:     Arduino Uno and official Arduino Ethernet 2
               shield. Should work with other Arduinos and
               compatible Ethernet shields.
               2Gb micro SD card formatted FAT16.
                
 Circuit:      Ethernet shield attached to pins 10, 11, 12, 13
               switch to SD-card with pin 4

 Software:     Developed using Arduino 1.8.5 software
               Should be compatible with Arduino 1.0 +
               SD card contains web page called index.htm
  
 *****************************************************
 
 References:   - Based on Ajax I/O example from Starting
                 Electronics:
 http://startingelectronics.org/tutorials/arduino/ethernet-shield-web-server-tutorial/SD-card-IO/
                  
 Date:         13 April 2015
 
 Author:       W.A. Smith, http://startingelectronics.org
  
 *****************************************************
 */
 
#include <SPI.h>
#include <Ethernet.h>   // Requires Library 2.0 or higher for W5500 chip
//#include <Ethernet2.h>// else use Ethernet 2 Library
#include <SD.h>

#define MSGMODE false   // true->Message and Test mode aktivieren

// Web server config
#define WebPort 80
#define SD_Pin     4    // SPI Slave select SD-card
#define SPI_SS     10   // SPI Slave select Ethernet shield
#define SPI_MOSI   11   // SPI MOSI
#define SPI_MISO   12   // SPI MISO
#define SPI_SCK    13   // SPI SCK
// size of buffer used to capture HTTP requests
#define REQ_BUF_SZ 127  // orig. 60 wg. System hängt...
                        // funktioniert mit 60 oder 127, nicht mit 256/1024 ???

// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = {
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED
};
IPAddress ip(192, 168, 101, 189);
IPAddress gateway(192, 168, 101, 251);
IPAddress subnet(255, 255, 255, 0);

// Data for webserver
File webFile;                     // the web page file on the SD card
char HTTP_req[REQ_BUF_SZ] = {0};  // buffered HTTP request stored as null terminated string
char req_index = 0;               // index into HTTP_req buffer

// Initialize the Ethernet server library
EthernetServer server(WebPort);
// XML response function
void XML_response( EthernetClient cl );


// Setup part
void Ether_setup() {

  // disable Ethernet chip
  pinMode(SPI_SS, OUTPUT);
  digitalWrite(SPI_SS, HIGH);

  // initialize SD card
  Serial.println(F("Initializing SD card..."));
  if (!SD.begin(SD_Pin)) {
    Serial.println(F("ERROR - SD card initialization failed!"));
    return;    // init failed
  }
  Serial.println(F("SUCCESS - SD card initialized."));
  // check for index.htm file
  if ( !SD.exists(F("index.htm")) ) {
    Serial.println(F("ERROR - Can't find index.htm file on SD-card !"));
    return;  // can't find index file
  }
  Serial.println(F("SUCCESS - Found index.htm file."));

  // start the Ethernet connection and the server:
  Ethernet.begin(mac, ip, gateway, subnet);
  
  server.begin();
  // Print server info
  Serial.print(F("Webserver is at "));
  Serial.print(Ethernet.localIP());
  Serial.print(F(":"));
  Serial.println(WebPort);

}


void Ether_loop() {

  EthernetClient client = server.available();  // try to get client
  char *webfname;
  const char deffname[] = "index.htm";

  if (client) {  // got client?
    boolean currentLineIsBlank = true;
    while (client.connected()) {
      if (client.available()) {   // client data available to read
        char c = client.read(); // read 1 byte (character) from client
        // limit the size of the stored received HTTP request
        // buffer first part of HTTP request in HTTP_req array (string)
        // leave last element in array as 0 to null terminate string (REQ_BUF_SZ - 1)
        if (req_index < (REQ_BUF_SZ - 1)) {
          HTTP_req[req_index] = c;          // save HTTP request character
          req_index++;
        }
        // last line of client request is blank and ends with \n
        // respond to client only after last line received
        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response header
          client.println(F("HTTP/1.1 200 OK"));
          // remainder of header follows below, depending on if
          // web page or XML page is requested
          // Ajax request - send XML file
          if (StrContains(HTTP_req, "ajax_inputs")) {
            // send rest of HTTP header
            client.println(F("Content-Type: text/xml"));
            client.println(F("Connection: keep-alive"));
            client.println();
            // Check input for commands
            SetCommands();
            // send XML file containing input states
            XML_response(client);
          }
          else if ( StrContains(HTTP_req, "GET /") )
          {
            // web page request maybe
            webfname = doGetRequest( HTTP_req, "GET /" );
            if (webfname == NULL)
              webfname = deffname;
            // send rest of HTTP header
            client.println(F("Content-Type: text/html"));
            client.println(F("Connection: keep-alive"));
            client.println();
            // send web page
            Serial.print("Datei:<");
            Serial.print(webfname);
            Serial.println(">");
            webFile = SD.open(webfname);        // open web page file
            if (webFile) {
              while(webFile.available()) {
                client.write(webFile.read()); // send web page to client
              }
              webFile.close();
            }
          }
          // display received HTTP request on serial port
          if( MSGMODE )   {
            Serial.println(HTTP_req);
          }
          // reset buffer index and all buffer elements to 0
          req_index = 0;
          StrClear(HTTP_req, REQ_BUF_SZ);
          currentLineIsBlank = true;
          client.flush();   // added cause of hanging of prob. long requests from browser ?
          break;
        }
        // every line of text received from the client ends with \r\n
        if (c == '\n') {
          // last character on line of received text
          // starting new line with next character read
          currentLineIsBlank = true;
        } 
        else if (c != '\r') {
          // a text character was received from client
          currentLineIsBlank = false;
        }
      } // end if (client.available())
    } // end while (client.connected())
    scheduler.delay(50);     // give the web browser time to receive the data
    client.stop(); // close the connection
  } // end if (client)
    
  // ToDo actions

}

EthernetServerDali.ino (12.2 KB)

my project with web server, telnet server, permanent Blynk and modbus tcp connection and some additional TCP requests inbetween doesn't have any problems with Ethernet library 2.0.0 and W5500 shield