Pages: [1]   Go Down
Author Topic: Ethernet W5100 handling of xml files  (Read 2177 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Newbie
*
Karma: 0
Posts: 24
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

If have a strange problem with the handling of XML files in the Chrome browser

My hardware setup is a Mega2560 with an W5100 ethernet shield (incuding mini SD Card).

I'm using the Arduino 1.0.1 IDE and its Ethernet library.

I've programmed the Mega to run as a web server with the web server files stored on the SD Card. There are HTML, IMAGES, JS and XML files stored on the SD card. The Arduino also generates certain xml itself and sends these via the ethernet w.5100 server functions.


My Arduino web server, pages and xml files work perfectly when using the IE 9, FIREFOX 16.0.1 AND Sarafi 5.1.7 browsers.

But for some reason my XML files don't work correctly in Chrome 22.0.


The strange problem is

If the xml files are read from the SD Card they work in Chrome. But the XML files generated directly by the Arduino code itself do work at all  in Chrome.

There must be some difference in the content of the xml files depending on whether the file comes from the SD Card or from the Arduino.
 
I have looked at the xml data using an HTTP debugger and I can see that the xml files are being requested by Chrome and sent to Chrome.

There is no obvious difference between the xml data from the SD Card or from the Arduino.


below is a snippet of code I am using to generate one xml file on the Arduino.



Quote
client.println("HTTP/1.1 200 OK");
client.println("Content-Length: 110");
client.println("Content-Type: application/xml");
client.println("Server: Arduino");
client.println();
client.println("<?xml version=\"1.0\"?>");
client.println("<RTC>");
client.print("<time>");
char datestr[24];
DateTime t = RTC.now();
char daysInWord[3];
DayInStr(daysInWord,day_of_week(t.day(), t.month(), t.year()));
client.print(daysInWord);
client.print(" ");
sprintf(datestr,"%02d-%02d-%04d  %02d:%02d:%02d",t.day(),t.month(),t.year(),t.hour(),t.minute(),t.second());
client.print(datestr);
client.println("</time>");
client.print("<NTPStatus>");
if (NTP_Status) {
  client.print("Ok");
  }
  else {
  client.print("Failed");
}
client.println("</NTPStatus>");
client.println("</RTC>");
client.println();


And this is an extract of the code I'm using to read and send XML files from the SD Card.

Quote

    while ((c = dataFile.read()) >= 0) {
      buf[bufindex++]smiley-cry(char)c);  //fill buffer
      if(bufindex >= maxbyte)  //empty buffer when maximum length is reached.
      {
        client.write(buf, maxbyte);
        bytecount = bytecount+maxbyte;
        bufindex=0;
      }
    }

Can anyone help me understand what could be different between an xml file from the SD Card and an XML file generated by the Arduino?

Remember everything works in IE, Firefox and Safari. This problem only happens in Chrome?????  But Why???
« Last Edit: October 19, 2012, 04:39:40 am by Ziggy2012 » Logged

Miramar Beach, Florida
Offline Offline
Faraday Member
**
Karma: 146
Posts: 6026
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Maybe Chrome doesn't like the xml file sent in that many packets? This sends one packet of data with maxbyte number of characters.
Code:
        client.write(buf, maxbyte);

I commented the packets in this code.
Code:
// this is a packet
client.println("HTTP/1.1 200 OK");
// this is another packet
client.println("Content-Length: 110");
// this is a third packet
client.println("Content-Type: application/xml");
// this is a fourth
client.println("Server: Arduino");

// ... and this next packet sends only a cr/lf
client.println();
// ...and another
client.println("<?xml version=\"1.0\"?>");
// ... and another
client.println("<RTC>");
// ...and another
client.print("<time>");
char datestr[24];
DateTime t = RTC.now();
char daysInWord[3];
DayInStr(daysInWord,day_of_week(t.day(), t.month(), t.year()));
// ...and another
client.print(daysInWord);

// ...and this is my favorite. One space sent in its own packet
client.print(" ");

sprintf(datestr,"%02d-%02d-%04d  %02d:%02d:%02d",t.day(),t.month(),t.year(),t.hour(),t.minute(),t.second());
client.print(datestr);
client.println("</time>");
client.print("<NTPStatus>");
if (NTP_Status) {
  client.print("Ok");
  }
  else {
  client.print("Failed");
}
client.println("</NTPStatus>");
client.println("</RTC>");
client.println();
Logged

0
Offline Offline
Tesla Member
***
Karma: 144
Posts: 9609
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I think the problem is the smiley face in the code.
Logged

Consider the daffodil. And while you're doing that, I'll be over here, looking through your stuff.   smiley-cool

Offline Offline
Newbie
*
Karma: 0
Posts: 24
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote

Maybe Chrome doesn't like the xml file sent in that many packets?


zoomkat

Thanks for responding.

I agree this could be the problem with Chrome. Chrome is definitely layer 2 aware. There may be a bug in the handling of multiple packets in xml files.

I have already looked at the packets and xml file data using Wireshark and I could see the client.println's were generating more packets. I could not see any other difference in the xml data.

I know my code example using client.println's isn't pretty. (and the smiley is some garbage added by the forum coding??)

Anyway it's worth a try to recode using a buffer and 

client.write(buf, maxbyte);

to check if Chrome works even though this approach uses lots of Mega 2560 ram.

Z
Logged

Miramar Beach, Florida
Offline Offline
Faraday Member
**
Karma: 146
Posts: 6026
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

You don't need to send it all in one packet. Just tighten it up a bit.
Code:
client.println("HTTP/1.1 200 OK\r\nContent-Length: 110\r\nContent-Type: application/xml");
client.println("Server: Arduino\r\n");
client.print("<?xml version=\"1.0\"?>\r\n<RTC>\r\n<time>");
char datestr[24];
DateTime t = RTC.now();
char daysInWord[3];
DayInStr(daysInWord,day_of_week(t.day(), t.month(), t.year()));
client.print(daysInWord);
// I put the space in the sprintf below
sprintf(datestr," %02d-%02d-%04d  %02d:%02d:%02d",t.day(),t.month(),t.year(),t.hour(),t.minute(),t.second());
client.print(datestr);
client.println("</time>");
client.print("<NTPStatus>");
If there are errors, I'm sure you can figure out where I messed up.

edit: If you use the "F" function, you can avoid putting the constant strings into SRAM.
« Last Edit: October 19, 2012, 01:50:17 pm by SurferTim » Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 614
Posts: 49365
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Code:
char daysInWord[3];
DayInStr(daysInWord,day_of_week(t.day(), t.month(), t.year()));
Does the DayInStr function only put two characters and a NULL in the array? It must not put more than two, and it MUST put the NULL in order for the string to be passed to a function that expects a string, like client.print().
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 24
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Thanks for all the suggestions.

I now understand what the "issue" with Chrome 22.0 is and it is easily solved.

Hopefully this information will help somebody else in the future.

If you add the line

Content-Length:  "length of data content in bytes"

in the HTTP response header,  the byte length of the XML file MUST BE ABSOLUTELY CORRECT or Chrome rejects the xml file with a script error

NETWORK_ERROR: XMLHttpRequest Exception 101

The 101 error is associated with Chrome file access permissions and is completely misleading in this case. It is a wrong Content-Length value which causes the error.

IE 9, FireFox and Safari  only complain if the Content-Length value is not larger than the file. The value does not have to be correct.

I have also seen that Chrome only wants the Content-Length to be accurate for XML files. It does not care about .htm, .js etc.



The fix is easy.

Don't put the Content-Length in the HTTP header at all.  IE 9, FIREFOX 16.0.1 AND Sarafi 5.1.7 browsers don't care and nor does Chrome 22.  So far I haven't found any situation where the Content-Length has to be in the HTTP header.



« Last Edit: October 19, 2012, 08:25:27 pm by Ziggy2012 » Logged

0
Offline Offline
Tesla Member
***
Karma: 144
Posts: 9609
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
So far I haven't found any situation where the Content-Length has to be in the HTTP header.

A client post request might require the content length.
Logged

Consider the daffodil. And while you're doing that, I'll be over here, looking through your stuff.   smiley-cool

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 614
Posts: 49365
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
A client post request might require the content length.
Also, when sending binary data, such as an image file, you must specify the content length.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 24
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
Also, when sending binary data, such as an image file, you must specify the content length

IE, Firefox, Safari and Chrome accept and render gif, jpg and png image files without the Content-Length being included in the HTTP header. Do you have an example of an image file which does not work??

Z
Logged

Nice, France
Offline Offline
Full Member
***
Karma: 11
Posts: 237
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
A client post request might require the content length.
Also, when sending binary data, such as an image file, you must specify the content length.

That is an incorrrect assertion. Besides, if it were true - application/xml is also binary data.

Looking at the relevant specification:

Quote
Applications SHOULD use this field to indicate the transfer-length of the message-body, unless this is prohibited by the rules in section 4.4.

Any Content-Length greater than or equal to zero is a valid value. Section 4.4 describes how to determine the length of a message-body if a Content-Length is not given.

and the reason for this is also stated:

Quote
In HTTP, it SHOULD be sent whenever the message's length can be determined prior to being transferred,

or in other words, if content is being generated on the fly and buffering the entire content just to compute the length would be inconvenient (because, for example, your server is on a low-powered machine with little memory) then it is OK not to send it.

Its not ok, though, to just guess at a random value and send that smiley
« Last Edit: October 21, 2012, 09:20:44 am by Nantonos » Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 614
Posts: 49365
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
if content is being generated on the fly
You seem to think that the Arduino is capable of generating image data on the fly. Psst, I've got news for you.
Logged

Nice, France
Offline Offline
Full Member
***
Karma: 11
Posts: 237
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
if content is being generated on the fly
You seem to think that the Arduino is capable of generating image data on the fly. Psst, I've got news for you.

a) the example from the OP is generating binary content on the fly
b) I have an optical flowsensor right here that generates image data on the fly (albeit only 32x32 greyscale)
c) is it really so hard to say "didn't know that, guess I was wrong, thanks"?
« Last Edit: October 22, 2012, 02:23:11 pm by Nantonos » Logged

Pages: [1]   Go Up
Jump to: