Ethernet W5100 handling of xml files

If have a strange problem with the handling of XML files in the Chrome browser

My hardware setup is a Mega2560 with an W5100 ethernet shield (incuding mini SD Card).

I'm using the Arduino 1.0.1 IDE and its Ethernet library.

I've programmed the Mega to run as a web server with the web server files stored on the SD Card. There are HTML, IMAGES, JS and XML files stored on the SD card. The Arduino also generates certain xml itself and sends these via the ethernet w.5100 server functions.

My Arduino web server, pages and xml files work perfectly when using the IE 9, FIREFOX 16.0.1 AND Sarafi 5.1.7 browsers.

But for some reason my XML files don't work correctly in Chrome 22.0.

The strange problem is

If the xml files are read from the SD Card they work in Chrome. But the XML files generated directly by the Arduino code itself do work at all in Chrome.

There must be some difference in the content of the xml files depending on whether the file comes from the SD Card or from the Arduino.

I have looked at the xml data using an HTTP debugger and I can see that the xml files are being requested by Chrome and sent to Chrome.

There is no obvious difference between the xml data from the SD Card or from the Arduino.

below is a snippet of code I am using to generate one xml file on the Arduino.

client.println("HTTP/1.1 200 OK");
client.println("Content-Length: 110");
client.println("Content-Type: application/xml");
client.println("Server: Arduino");
client.println();
client.println("<?xml version=\"1.0\"?>");
client.println("");
client.print("");
char datestr[24];
DateTime t = RTC.now();
char daysInWord[3];
DayInStr(daysInWord,day_of_week(t.day(), t.month(), t.year()));
client.print(daysInWord);
client.print(" ");
sprintf(datestr,"%02d-%02d-%04d %02d:%02d:%02d",t.day(),t.month(),t.year(),t.hour(),t.minute(),t.second());
client.print(datestr);
client.println("");
client.print("");
if (NTP_Status) {
client.print("Ok");
}
else {
client.print("Failed");
}
client.println("");
client.println("");
client.println();

And this is an extract of the code I'm using to read and send XML files from the SD Card.

while ((c = dataFile.read()) >= 0) {
buf[bufindex++]=((char)c); //fill buffer
if(bufindex >= maxbyte) //empty buffer when maximum length is reached.
{
client.write(buf, maxbyte);
bytecount = bytecount+maxbyte;
bufindex=0;
}
}

Can anyone help me understand what could be different between an xml file from the SD Card and an XML file generated by the Arduino?

Remember everything works in IE, Firefox and Safari. This problem only happens in Chrome????? But Why???

Maybe Chrome doesn't like the xml file sent in that many packets? This sends one packet of data with maxbyte number of characters.

        client.write(buf, maxbyte);

I commented the packets in this code.

// this is a packet
client.println("HTTP/1.1 200 OK");
// this is another packet
client.println("Content-Length: 110");
// this is a third packet
client.println("Content-Type: application/xml");
// this is a fourth
client.println("Server: Arduino");

// ... and this next packet sends only a cr/lf
client.println();
// ...and another
client.println("<?xml version=\"1.0\"?>");
// ... and another
client.println("<RTC>");
// ...and another
client.print("<time>");
char datestr[24];
DateTime t = RTC.now();
char daysInWord[3];
DayInStr(daysInWord,day_of_week(t.day(), t.month(), t.year()));
// ...and another
client.print(daysInWord);

// ...and this is my favorite. One space sent in its own packet
client.print(" ");

sprintf(datestr,"%02d-%02d-%04d  %02d:%02d:%02d",t.day(),t.month(),t.year(),t.hour(),t.minute(),t.second());
client.print(datestr); 
client.println("</time>");
client.print("<NTPStatus>");
if (NTP_Status) {
  client.print("Ok");
  }
  else {
  client.print("Failed");
}
client.println("</NTPStatus>");
client.println("</RTC>");
client.println();

I think the problem is the smiley face in the code.

Maybe Chrome doesn't like the xml file sent in that many packets?

zoomkat

Thanks for responding.

I agree this could be the problem with Chrome. Chrome is definitely layer 2 aware. There may be a bug in the handling of multiple packets in xml files.

I have already looked at the packets and xml file data using Wireshark and I could see the client.println's were generating more packets. I could not see any other difference in the xml data.

I know my code example using client.println's isn't pretty. (and the smiley is some garbage added by the forum coding??)

Anyway it's worth a try to recode using a buffer and

client.write(buf, maxbyte);

to check if Chrome works even though this approach uses lots of Mega 2560 ram.

Z

You don't need to send it all in one packet. Just tighten it up a bit.

client.println("HTTP/1.1 200 OK\r\nContent-Length: 110\r\nContent-Type: application/xml");
client.println("Server: Arduino\r\n");
client.print("<?xml version=\"1.0\"?>\r\n<RTC>\r\n<time>");
char datestr[24];
DateTime t = RTC.now();
char daysInWord[3];
DayInStr(daysInWord,day_of_week(t.day(), t.month(), t.year()));
client.print(daysInWord);
// I put the space in the sprintf below
sprintf(datestr," %02d-%02d-%04d  %02d:%02d:%02d",t.day(),t.month(),t.year(),t.hour(),t.minute(),t.second());
client.print(datestr); 
client.println("</time>");
client.print("<NTPStatus>");

If there are errors, I'm sure you can figure out where I messed up.

edit: If you use the "F" function, you can avoid putting the constant strings into SRAM.

char daysInWord[3];
DayInStr(daysInWord,day_of_week(t.day(), t.month(), t.year()));

Does the DayInStr function only put two characters and a NULL in the array? It must not put more than two, and it MUST put the NULL in order for the string to be passed to a function that expects a string, like client.print().

Thanks for all the suggestions.

I now understand what the "issue" with Chrome 22.0 is and it is easily solved.

Hopefully this information will help somebody else in the future.

If you add the line

Content-Length: "length of data content in bytes"

in the HTTP response header, the byte length of the XML file MUST BE ABSOLUTELY CORRECT or Chrome rejects the xml file with a script error

NETWORK_ERROR: XMLHttpRequest Exception 101

The 101 error is associated with Chrome file access permissions and is completely misleading in this case. It is a wrong Content-Length value which causes the error.

IE 9, FireFox and Safari only complain if the Content-Length value is not larger than the file. The value does not have to be correct.

I have also seen that Chrome only wants the Content-Length to be accurate for XML files. It does not care about .htm, .js etc.

The fix is easy.

Don't put the Content-Length in the HTTP header at all. IE 9, FIREFOX 16.0.1 AND Sarafi 5.1.7 browsers don't care and nor does Chrome 22. So far I haven't found any situation where the Content-Length has to be in the HTTP header.

So far I haven't found any situation where the Content-Length has to be in the HTTP header.

A client post request might require the content length.

A client post request might require the content length.

Also, when sending binary data, such as an image file, you must specify the content length.

Also, when sending binary data, such as an image file, you must specify the content length

IE, Firefox, Safari and Chrome accept and render gif, jpg and png image files without the Content-Length being included in the HTTP header. Do you have an example of an image file which does not work??

Z

PaulS:

A client post request might require the content length.

Also, when sending binary data, such as an image file, you must specify the content length.

That is an incorrrect assertion. Besides, if it were true - application/xml is also binary data.

Looking at the relevant specification:

Applications SHOULD use this field to indicate the transfer-length of the message-body, unless this is prohibited by the rules in section 4.4.

Any Content-Length greater than or equal to zero is a valid value. Section 4.4 describes how to determine the length of a message-body if a Content-Length is not given.

and the reason for this is also stated:

In HTTP, it SHOULD be sent whenever the message's length can be determined prior to being transferred,

or in other words, if content is being generated on the fly and buffering the entire content just to compute the length would be inconvenient (because, for example, your server is on a low-powered machine with little memory) then it is OK not to send it.

Its not ok, though, to just guess at a random value and send that

if content is being generated on the fly

You seem to think that the Arduino is capable of generating image data on the fly. Psst, I've got news for you.

PaulS:

if content is being generated on the fly

You seem to think that the Arduino is capable of generating image data on the fly. Psst, I've got news for you.

a) the example from the OP is generating binary content on the fly
b) I have an optical flowsensor right here that generates image data on the fly (albeit only 32x32 greyscale)
c) is it really so hard to say "didn't know that, guess I was wrong, thanks"?