Go Down

Topic: Web server refresh problem (Read 9256 times) previous topic - next topic

YemSalat

#15
Aug 15, 2015, 03:17 am Last Edit: Aug 15, 2015, 07:16 am by YemSalat
Forgot to add, since the library code was updated, its methods should be called in a slightly different way:
Now since the code is updated - you can just cycle through sockets 'manually' and pick the right one:
Code: [Select]

 for (int sock = 0; sock < MAX_SOCK_NUM; sock++) {
    EthernetClient sclient = server.available_(sock);
    serverWorks(sclient);
  }

Your function:
Code: [Select]

void serverWorks(EthernetClient sclient) {

}

SurferTim

#16
Aug 15, 2015, 03:22 am Last Edit: Aug 15, 2015, 03:27 am by SurferTim
Perhaps I should just quote the library code from EthernetClient.cpp:
Code: [Select]

EthernetClient EthernetServer::available() {
  accept();

  for (int sock = 0; sock < MAX_SOCK_NUM; sock++) {
    EthernetClient client(sock);
    if (EthernetClass::_server_port[sock] == _port &&
        (client.status() == SnSR::ESTABLISHED ||
         client.status() == SnSR::CLOSE_WAIT)) {
      if (client.available()) {
        // XXX: don't always pick the lowest numbered socket.
        return client;
      }
    }
  }
  return EthernetClient(MAX_SOCK_NUM);
}

Notice this?
// XXX: don't always pick the lowest numbered socket
And that causes a problem how? edit: It picks the lowest socket that is active and has characters available in the rx queue.

Quote
As you may see it is still not perfect (3s delay on style.css) but still much better then original.
I don't see that delay unless the server must serve a really large html file that takes 3 seconds to download.

I get that kind of crappy performance if I don't send the files in large packets. That was the point I was making.

YemSalat

#17
Aug 15, 2015, 03:52 am Last Edit: Aug 15, 2015, 03:53 am by YemSalat
And that causes a problem how? edit: It picks the lowest socket that is active and has characters available in the rx queue.
It cycles through sockets and if it finds one - it will return the client and ignore the rest.

The solution that I posted is rather 'brute force' and I don't like that it transfers part of the library logic to the code.

To be honest I personally do not completely understand what the problem is yet. I've been looking for various explanations for the past couple days. I thought that the problem is sockets not closing properly, but that does not seem to be the issue.
I don't know why but sometimes the socket will just hang for 3 seconds (default TCP timeout?)

There is a discussion realted to this here: http://forum.arduino.cc/index.php?topic=154157.0 (its in Italian)


Quote
I don't see that delay unless the server must serve a really large html file that takes 3 seconds to download.

I get that kind of crappy performance if I don't send the files in large packets. That was the point I was making.
Can you please post a screenshot of your browser devtools network tab when loading a web page with some files in it?

nickgammon

#18
Aug 15, 2015, 03:57 am Last Edit: Aug 15, 2015, 04:00 am by Nick Gammon
Complete modified EthernetServer.cpp:

Code: [Select]

#include «w5100.h»
#include «socket.h»
extern «C» {
...



Your posted code has some weird quotes in it.
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

nickgammon

My personal experience has been that the official Ethernet card works fine, and a cheap eBay clone keeps displaying the symptoms of not connecting more often than not. In fact, altering the library as suggested did not help in this case.

So I would say: "you get what you pay for".
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

nickgammon

#20
Aug 15, 2015, 04:01 am Last Edit: Aug 15, 2015, 04:02 am by Nick Gammon
Forgot to add, since the library code was updated, its methods should be called in a slightly different way:
Now since the code is updated - you can just cycle through sockets 'manually' and pick the right one:
Code: [Select]

 for (int sock = 0; sock < MAX_SOCK_NUM; sock++) {
    EthernetClient sclient = server.available_(sock);
    serverWorks(sclient);
  }


I tried:

Code: [Select]
EthernetClient client;
 for (int sock = 0; sock < MAX_SOCK_NUM; sock++) {
    client = server.available_(sock);
    if (client)
      break;
  }

  if (!client)
    return;


I'm not sure if that is OK or not.
Please post technical questions on the forum, not by personal message. Thanks!

More info: http://www.gammon.com.au/electronics

YemSalat

Your posted code has some weird quotes in it.
Oops, sorry, was copy-pasting from an online code highlighter, not sure why it changed the quotes (will fix in a second)

I will post my own code that I did tests with later (not on my laptop at the moment)

YemSalat

My personal experience has been that the official Ethernet card works fine, and a cheap eBay clone keeps displaying the symptoms of not connecting more often than not.
Could be the case, I got mine on ebay for about $6-7

SurferTim

#23
Aug 15, 2015, 05:04 am Last Edit: Aug 15, 2015, 05:05 am by SurferTim
I thought that the problem is sockets not closing properly, but that does not seem to be the issue.
That is exactly what I found. Sockets not closing properly.

Can you please post a screenshot of your browser devtools network tab when loading a web page with some files in it?
Why? My serial debug display shows the request and response to every file the web browser requests. That includes html, css, javascript, and two images (one jpg and the other favicon.ico) all on one page. Not only that, but the web browser displays the page exactly as it should.

YemSalat

#24
Aug 15, 2015, 05:46 am Last Edit: Aug 15, 2015, 07:02 am by YemSalat
Why? My serial debug display shows the request and response to every file the web browser requests. That includes html, css, javascript, and two images (one jpg and the other favicon.ico) all on one page.
Browser devtools will give a better visual representation of how requests are fired / executed.

I think on both Chrome and Firefox the shortcut to launch devtools is Ctrl+Shift+i
Then you can just switch to the 'Network' tab and reload the page.

Quote
Not only that, but the web browser displays the page exactly as it should.
Why wouldn't it? Web browsers don't care at what speed / order they get the files as long as they get all of them in the end. We are talking about delays here.

zoomkat

Some simple server iframe web page test code that shows the arduino server can supply small pages fairly fast.

Code: [Select]

// zoomkat's meta refresh data frame test page 5/25/13
// use http://192.168.1.102:84 in your brouser for main page
// http://192.168.1.102:84/data static data page
// http://192.168.1.102:84/datastart meta refresh data page
// for use with W5100 based ethernet shields
// set the refresh rate to 0 for fastest update
// use STOP for single data updates

#include <SPI.h>
#include <Ethernet.h>

const int analogInPin0 = A0;
const int analogInPin1 = A1;
const int analogInPin2 = A2;
const int analogInPin3 = A3;
const int analogInPin4 = A4;
const int analogInPin5 = A5;

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address
byte ip[] = { 192, 168, 1, 102 }; // arduino ip in lan
byte gateway[] = { 192, 168, 1, 1 }; // internet access via router
byte subnet[] = { 255, 255, 255, 0 }; //subnet mask
EthernetServer server(84); //server port
unsigned long int x=0; //set refresh counter to 0
String readString;

//////////////////////

void setup(){
  Serial.begin(9600);
    // disable SD SPI if memory card in the uSD slot
  pinMode(4,OUTPUT);
  digitalWrite(4,HIGH);

  Ethernet.begin(mac, ip, gateway, gateway, subnet);
  server.begin();
  Serial.println("meta refresh data frame test 5/25/13"); // so I can keep track of what is loaded
}

void loop(){
  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
         if (readString.length() < 100) {
          readString += c;
         }
        //check if HTTP request has ended
        if (c == '\n') {

          //check get atring received
          Serial.println(readString);

          //output HTML data header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println();

          //generate data page
          if(readString.indexOf("data") >0) {  //checks for "data" page
            x=x+1; //page upload counter
            client.print("<HTML><HEAD>");
            //meta-refresh page every 1 seconds if "datastart" page
            if(readString.indexOf("datastart") >0) client.print("<meta http-equiv='refresh' content='1'>");
            //meta-refresh 0 for fast data
            if(readString.indexOf("datafast") >0) client.print("<meta http-equiv='refresh' content='0'>");
            client.print("<title>Zoomkat's meta-refresh test</title></head><BODY><br>");
            client.print("page refresh number: ");
            client.print(x); //current refresh count
            client.print("<br><br>");
           
              //output the value of each analog input pin
            client.print("analog input0 is: ");
            client.print(analogRead(analogInPin0));
           
            client.print("<br>analog input1 is: ");
            client.print(analogRead(analogInPin1));
                       
            client.print("<br>analog input2 is: ");
            client.print(analogRead(analogInPin2));
           
            client.print("<br>analog input3 is: ");
            client.print(analogRead(analogInPin3));
                                   
            client.print("<br>analog input4 is: ");
            client.print(analogRead(analogInPin4));
           
            client.print("<br>analog input5 is: ");
            client.print(analogRead(analogInPin5));
            client.println("<br></BODY></HTML>");
           }
          //generate main page with iframe
          else
          {
            client.print("<HTML><HEAD><TITLE>Zoomkat's frame refresh test</TITLE></HEAD>");
            client.print("Zoomkat's Arduino frame meta refresh test 5/25/13");
            client.print("<BR><BR>Arduino analog input data frame:<BR>");
            client.print("&nbsp;&nbsp;<a href='http://192.168.1.102:84/datastart' target='DataBox' title=''yy''>META-REFRESH</a>");
            client.print("&nbsp;&nbsp;&nbsp;&nbsp;<a href='http://192.168.1.102:84/data' target='DataBox' title=''xx''>SINGLE-STOP</a>");
            client.print("&nbsp;&nbsp;&nbsp;&nbsp;<a href='http://192.168.1.102:84/datafast' target='DataBox' title=''zz''>FAST-DATA</a><BR>");
            client.print("<iframe src='http://192.168.1.102:84/data' width='350' height='250' name='DataBox'>");
            client.print("</iframe><BR></HTML>");
          }
          delay(1);
          //stopping client
          client.stop();
          //clearing string for next read
          readString="";
        }
      }
    }
  }
}

Google forum search: Use Google Search box in upper right side of this page.
Why I like my 2005 Rio Yellow Honda S2000  https://www.youtube.com/watch?v=pWjMvrkUqX0

YemSalat

Can you please post a screenshot of your browser devtools network tab when you load that page on your arduino? (I posted instructions on how to do that in my previous comment)

jurs

Can you please post a screenshot of your browser devtools network tab when you load that page on your arduino? (I posted instructions on how to do that in my previous comment)
I've tried zoomkat's code with Arduino 1.6.5 and the default Ethernet library and could not find any problems with time gaps in the handling of HTTP requests.

If the webserver gets under heavy stress and cannot establish every connection that is requested, the webbrowser (Google Chrome) seems to create a priority scheduling for the files requested.

In case of zoomkat's code, the webbrowser will always try to retrieve two files at highest possible speed:
- datafast
- favicon.ico

But it seems to be the webbrowser that delays requests to favicon.ico

In the picture included you can see: If the webbrowser is able to establish a connection to the Arduino webserver, it will always try to handle 'datafast' in shortest possible time.

When scrolling down (not visible in the image) I can see, that the response time to 'datafast' always stay short, while the response time to 'favicon.ico' is increasing more and more: 3s, 6s. 9s, 12s, up to more than 30 seconds!

So for me its the webbrowser that creates delays for certain files, possibly the decision is made by file extension. Or 'favicon.ico' has a low priority by default, because it is a special file that the browser requests.

The gap in the timeline chart when 'favicon.ico' is delayed for 3 seconds for the first time, is not existing. When scrolling down the timeline chart I can see that all the time is used: It is used to fulfil 'datafast' requests, which seem to have a higher priority to the Google Chrome webbrowser, while the webserver cannot fulfil the requests fast enough.

I cannot detect a problem with the Ethernet library and the Arduino webserver seems to handle requests all the time, without any gaps in the timeline.

YemSalat

#28
Aug 15, 2015, 01:20 pm Last Edit: Aug 15, 2015, 01:31 pm by YemSalat
Thanks for the test, but it is not very representative, there should really be more files in there in order to test it properly (sorry I should have looked through zoomkat's code better)

It is also better if the page is loaded just once, not constantly refreshed, so it doesn't affect the results. (And disable cahce of course)

I have not noticed difference when testing between Firefox/Chrome.


jurs

Thanks for the test, but it is not very representative, there should really be more files in there in order to test it properly (sorry I should have looked through zoomkat's code better)
So you do not want a "stress test" under heavy load of the webserver, but you just want the "total loading time of a web page consisting of many different files", for a one time total loading?

It is also better if the page is loaded just once, not constantly refreshed
Do you have a special "test suite of files" for testing, that you can post as a file attachment?

I could then put the files on a SD card and possibly test with the SD card webserver code that SurferTim has linked.

Go Up