Go Down

Topic: Web server refresh problem (Read 9237 times) previous topic - next topic

SurferTim

#30
Aug 15, 2015, 02:02 pm Last Edit: Aug 15, 2015, 02:28 pm by SurferTim
I attached the screenshots from three downloads from my SD card using the method you suggested. Do you see a problem?
index.htm (home page)
image (soccer pic page)
waterfall (background image loaded by the css file)

edit: Is there a way to show these images on the forum without hosting them on a remote server?

Here is my server code online. I'll leave it here for a limited time only. I need this Mega2560 for testing tomorrow. Do your own testing if you doubt me.

I thought I would also include the serial output from my server code from the playground.
Quote
Starting SD..ok
192.168.2.2
Ready

Client request #1: GET / HTTP/1.1
file = /
file type =
method = GET
params =
protocol = HTTP/1.1
Home page SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #2: GET /defcss.css HTTP/1.1
file = /DEFCSS.CSS
file type = CSS
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #3: GET /testjs.js HTTP/1.1
file = /TESTJS.JS
file type = JS
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #4: GET /favicon.ico HTTP/1.1
file = /FAVICON.ICO
file type = ICO
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #5: GET /soccer.htm HTTP/1.1
file = /SOCCER.HTM
file type = HTM
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #6: GET /img/wolver.JPG HTTP/1.1
file = /IMG/WOLVER.JPG
file type = JPG
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #7: GET /favicon.ico HTTP/1.1
file = /FAVICON.ICO
file type = ICO
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #8: GET /favicon.ico HTTP/1.1
file = /FAVICON.ICO
file type = ICO
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #9: GET /testhtml.htm HTTP/1.1
file = /TESTHTML.HTM
file type = HTM
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #10: GET /mycss.css HTTP/1.1
file = /MYCSS.CSS
file type = CSS
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #11: GET /IMG/WATERFAL.JPG HTTP/1.1
file = /IMG/WATERFAL.JPG
file type = JPG
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

Client request #12: GET /favicon.ico HTTP/1.1
file = /FAVICON.ICO
file type = ICO
method = GET
params =
protocol = HTTP/1.1
SD file
filename format ok
SRAM = 6688
file found..opened..send..closed
disconnected

YemSalat

#31
Aug 15, 2015, 02:57 pm Last Edit: Aug 15, 2015, 03:02 pm by YemSalat
So you do not want a "stress test" under heavy load of the webserver, but you just want the "total loading time of a web page consisting of many different files", for a one time total loading?
Yes, I only want to test the total loading time of the page.

Do you have a special "test suite of files" for testing, that you can post as a file attachment?

I could then put the files on a SD card and possibly test with the SD card webserver code that SurferTim has linked.
I do, please find attached, and thanks heaps for doing that. Can you please also post which Ethernet board you are using (is it the official one?)

I attached the screenshots from three downloads from my SD card using the method you suggested. Do you see a problem?
Can you post screenshots running your own code please, without modifications to Ethernet library.

[Edit] updated attached files to include favicon.ico

jurs

Here is my server code online. I'll leave it here for a limited time only. I need this Mega2560 for testing tomorrow. Do your own testing if you doubt me.
Measured times loading from your webserver across the Internet is much longer as it seems.

When loading "soccer.htm" the loading times are:
soccer.htm ==> 540 ms
wolver.jpg ==> 3.13 seconds
favicon.ico ==> 526 ms

In addition, Google Chrome draws a file "webfont.js" from one of the Google servers with status code "304 not changed", but this has nothing to with your server and seems to be Chrome-specific.

While the file requests are slow, there is no time gap between drawing the files. After each file is complete, the next request is handled immediately, until the whole file is loaded.

The file request for "webfont.js" from one of the Google servers does not need any additional time as that request is handled in multitasking.

SurferTim

Did you not read my previous post? I have my server code running online. I have no mods to the ethernet library. Do your own testing.
Here is my server code running online for today only.

@jurs: Thanks for running that test.

YemSalat

Did you not read my previous post? I have my server code running online. I have no mods to the ethernet library. Do your own testing.
Here is my server code running online for today only.

My bad, will test now.

SurferTim

Go ahead. Fire away. Somebody just hit the server pretty hard. Downloaded every file a few times in a row. No problems so far. All sockets still available.

YemSalat

@SurferTim, can you please put a bit more files in there? They don't need to be big, maybe just through in a couple more js/css and a small image.

So far I am getting the following results (images are clickable):

index:


waterfall:



PS Thanks heaps for doing this, I'll only be happy if I am screwing up somewhere else in my code and the library is indeed fine

SurferTim

#37
Aug 15, 2015, 03:19 pm Last Edit: Aug 15, 2015, 03:19 pm by SurferTim
I can't add anything right now. Too much traffic on the server.

Bear in mind this is an Arduino, not a Linux/Apache box. It has 4 sockets. If you add too many files, you will have problems, especially if the files are large. It can service only one file request at a time and accepts connections for only three other requests. All other connection requests may time out if the current connections not serviced quickly.

jurs

[Edit] updated attached files to include favicon.ico
Updated files? Hmmmm.

I've downloaded and tested the files as you posted them initially.

I had to rename "index.html" to "index.htm" to be FAT-compatible.
(SD library for FAT file system doesn't support 4-letter file extensions.)

The picture is the result created
- compiling with Arduino 1.0.5
- SurferTims webserver code http://playground.arduino.cc/Code/WebServerST
- Google Chrome as a webbrowser.

Total loading time of the page is about 3.6 seconds.

It seems that after all 4 socket connections are established for the files index.htm, base.css, style.css and pic1.jpg (which are loaded one after the other), the Chrome webbrowser fails to establish a fifth connection for script.js. So finally there is a time gap where no data is downloaded, as the webbrowser seems to do a retry for the failed connection only after about 3 seconds of waiting. Then the rest of the page loads.

Compiled with Arduino 1.6.5 things seem to be very similar. But after some reloading I could also get a different sequence of loading files in some cases. Total loading time is the same, even if the sequence of files loaded is a bit different sometimes: index.htm, base.css, style.css, script.js, time gap, pic1.jpg, pic2.jpg

YemSalat

I can't add anything right now. Too much traffic on the server.
I understand, thanks for putting what you have online!

Bear in mind this is an Arduino, not a Linux/Apache box. It has 4 sockets. If you add too many files, you will have problems, especially if the files are large. It can service only one file request at a time and accepts connections for only three other requests. All other connection requests may time out if the current connections not serviced quickly.
I am aware of that, its just that I am getting flaky performance even on the first few connections sometimes.
I am using a cheap ethernet board though, which can very well be the cause of my issues.

jurs

With the "updated" file suite including 'favicon.ico' things seem to become errastic.

SurferTim's webserver code compiled with Arduino 1.6.5 seems to create different loading times for the webpage.
- best case loading time: ca. 3.6 seconds
- worst case loading time: ca. 11 seconds
- worst of all cases: pic1.jpg and pic2.jpg create a "fail" and do not load at all

I don't know how this errastic behaviour come when just 'favicon.ico' is added to the webpage.

Here are the pictures of a "fast loading" and a "slow loading":

YemSalat

#41
Aug 15, 2015, 04:53 pm Last Edit: Aug 15, 2015, 04:55 pm by YemSalat
Thanks for the test @jurs!

I am quite baffled as well, I have never tested without the favicon myself to be honest.
Can you please try adding another image or css file to the html and removing the favicon.ico file? It would be good to know if it is the number of files or favicon.ico specifically that is screwing things up.

SurferTim

Four sockets is all it has. Any attempt to download more than 4 files and you might get fails.

Anyone tried my server code with a w5200 or w5500? They have 8 sockets.

YemSalat

Four sockets is all it has. Any attempt to download more than 4 files and you might get fails.
Yeah, but the way it fails should not differ so much depending on +/- 1 file if there are already more then 4.

jurs

I am quite baffled as well, I have never tested without the favicon myself to be honest.
Can you please try adding another image or css file to the html and removing the favicon.ico file? It would be good to know if it is the number of files or favicon.ico specifically that is screwing things up.
I don't think that it is caused by the 'favicon.ico' file itself, but by the number of files.

I think that the webbrowser must have some logic that considers
- the number of missed or stalled connections
- the frequency of missed or stalled connections

If the number or frequency of missed or stalled connections is increasing, I suppose that the webbrowser is reacting in two ways to avoid overloading the webserver with requests:
- reduce the number of connections established to the destination server at the same time
- increase the time between retries after a connection failure
Both counter measurements will increase the total loading time of a page with many files.

Google's corporate motto is "Don't be evil", so if a connection failure occurs, the Chrome webbrowser increases retry time after a failure and reduces the number of requests to a website in a given time. So that other users at the same time may also be able to get some of their HTTP requests fullfilled.

I can do some further testing with even more files, but I suppose, if you want things to get better and faster served, you need to help the web browser caching files that do not change often. So you would have to create a webserver code that
- evaluates cache-headers in the browser request
- sends cache-headers with the server response
So that the webbrowser gets enabled to cache files, to reduce web traffic and not make pictures load again and again and again with their full size.

Go Up