bug in Ethernet lib ? (Arduino mega, w5100)

Hello to all,

this is my first post (as I have the arduino only since one week now).

I have a Arduino Mega with ethernet shield and arduino-0022 SW, but the Webserver-Example does not work.
The web browser keeps waiting for an answer, that does not come.

I changed the example to echo whatever comes in at the arduino:

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 192,168,0, 111 };
  
// Initialize the Ethernet server library
// with the IP address and port you want to use 
// (port 80 is default for HTTP):
Server server(80);

void setup()
{
  // start the Ethernet connection and the server:
  Ethernet.begin(mac, ip);
  server.begin();
  delay(1000);
  blink();
}

void loop()
{
  // listen for incoming clients
  Client client = server.available();
  if (client) {
    blink();
          client.println("HTTP/1.0 200 OK");
          client.println("Content-Type: text/plain");
          client.println();
  
    // an http request ends with a blank line
    boolean currentLineIsBlank = true;
    int i = 0;
    while (client.connected()) {
      
      if (client.available()) {
        blink2();
        char c = client.read();
        client.print(c);
        i++;
        if (i > 2000) { 
          break;
        }
        // if you've gotten to the end of the line (received a newline
        // character) and the line is blank, the http request has ended,
        // so you can send a reply
        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response header
          //client.println("HTTP/1.0 200 OK");
          //client.println("Content-Type: text/html");
          client.println();

          // output the value of each analog input pin
          for (int analogChannel = 0; analogChannel < 6; analogChannel++) {
            client.print("analog input ");
            client.print(analogChannel);
            client.print(" is ");
            client.print(analogRead(analogChannel));
            client.println("
");
          }
          break;
        }
        if (c == '\n') {
          // you're starting a new line
          currentLineIsBlank = true;
        } 
        else if (c != '\r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }
    }
    // give the web browser time to receive the data
    delay(1);
    // close the connection:
    client.stop();
  }
}

void blink() {
      digitalWrite(13, HIGH);   // set the LED on
      delay(200);              // wait for a second
      digitalWrite(13, LOW);    // set the LED off
      delay(500);              // wait for a second
}
void blink2() {
      digitalWrite(13, HIGH);   // set the LED on
      delay(60);              // wait for a second
      digitalWrite(13, LOW);    // set the LED off
      delay(60);              // wait for a second
}

And this is, what comes in in the browser window now:

GET /index.html HTTP/1.1
Host: 192.168.0.111
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.16) Gecko/20110107 Iceweasel/3.5.16 (like Firefox/3.5.16)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language:GET /index.html HTTP/1.1
Host: 192.168.0.111
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.16) Gecko/20110107 Iceweasel/3.5.16 (like Firefox/3.5.16)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language:GET /index.html HTTP/1.1
Host: 192.168.0.111
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.16) Gecko/20110107 Iceweasel/3.5.16 (like Firefox/3.5.16)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language:GET /index.html HTTP/1.1
[...and.so.on.and.so.on...]

Conclusion 1: The example software never "sees" an empty line, and so never starts to send the HTTP response.

Conclusion 2: My first thought was, that the web browser is re-sending its request over and over due some socket state problem or so, but then I realized, that the reading from internal read buffer in the w5100 chip is always starting to read from the beginning.

(I noticed in W5100Class::recv_data_processing
the lines
ptr += len;
writeSnRX_RD(s, ptr);
which should actually do it.)

My main questions for now are:

  1. Why is nobody else reporting problems with the Ethernet library or Server example?
  2. Can anybody confirm this problem?

(Who knows, maybe it turns out to be a somehow broken ethernet shield after all.)

Thanks in advance for any suggestions.

Below is some server test code I use that might help you do some trouble shooting.

//zoomkat 12-18-10
//routerbot code
//for use with IDE 0021
//open serial monitor to see what the arduino receives
// 

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address
byte ip[] = { 192, 168, 1, 102 }; // ip in lan
byte gateway[] = { 192, 168, 1, 1 }; // internet access via router
byte subnet[] = { 255, 255, 255, 0 }; //subnet mask
Server server(84); //server port

String readString; 
 
 //////////////////////

void setup(){

//start Ethernet
Ethernet.begin(mac, ip, gateway, subnet);
server.begin();

//enable serial data print 
Serial.begin(9600); 
Serial.println("servertest1"); // so I can keep track of what is loaded
}

void loop(){
// Create a client connection
Client client = server.available();
if (client) {
while (client.connected()) {
if (client.available()) {
char c = client.read();

//read char by char HTTP request
if (readString.length() < 100) {

//store characters to string 
readString += c; 
Serial.print(c);
} 

//if HTTP request has ended
if (c == '\n') {

///////////////
Serial.println(readString);

  //now output HTML data header
  client.println("HTTP/1.1 204 Zoomkat");
  client.println();
  client.println();
  delay(1);
  //stopping client
client.stop();

/////////////////////
//clearing string for next read
readString="";
  
}}}}}

Hi Edison,

echoing to the serial could be another idea.

However, your code would not exploit the problem, as it appears only after reading in more that 253 (+/- ?) byte from the socket. (I just counted the bytes from one "GET ..." to the next repetition.)

Can you please check the Webserver example with either your code changed to echo more (I guess it needs more than replacing " < 100" with "< 300", due to string length restrictions), or just test the example I posted?

I dare to get a confirmation on this errorenous behaviour, to know where to continue searching (sw or hw).

Thank you,
Thomas

Hello,

I think I found the error(s) now by myself:

In w5100.cpp we have the method W5100Class::getRXReceivedSize. This method can tell, if and how much data is available. However, this method is only used in client (and udp), but in the client it only checks, if data is available, not using the actul byte count. This is probably where the whole problem starts.

W5100Class::recv_data_processing just reads the wanted number of bytes (len) from the chip, not asking, if they are valid (came in).

In socket.cpp the comment for recv states:

"It continues to wait for data as much as the application wants to receive."

Actually it does not wait. It just returns this number of bytes, getting them from the chip receive buffer, not asking if they were received or not.

Additionally, the commented behaviour by itself is not what you might expect from a recv call to a socket. The linux manpage states:

... return the length of the message on successful completion."

In other words: At least when you want to use a keep-alive connection, you do not know, how much will come in next time. So you don't want to tell it, how long (for how many bytes) it shall wait. Instead, you want it to get the info, how much it got.

After so much bug complaining, I fell the need to point out,
that THE WHOLE CODE IS REALLY GOOD STYLE PROGRAMMING, FUN TO READ AND EASY TO UNDERSTAND, and it's a pleasure to fix it. Thanks to the author(s).

One last word: I need the buggy behaviour being confirmed, otherwise I won't send a fix :wink:

The below site has code that seems similar to what you want to do. I think you can pot your own code in brackets by opening the post to modify, high light the code and then click the # on the tool bar, then save.

Thanks zoomcat, I will have a close look at this link.

Hello,

I still need a confirmation, that the webserver example
Examples > Ethernet > WebServer from arduino SW version 22

is not working, if you request with a "standard browser" like IE, firefox or so. The behaviour is, that the web browser waits "forever".

Please, can anybody confirm this, or tell me, that it is working for him!

Thanks for cooperation

Modifying the code for my lan, your code produces this after a really long wait.

GET /hgfds HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/x-ms-application, application/x-ms-xbap, application/vnd.ms-xpsdocument, application/xaml+xml, /
Accept-Language: en-us
UA-CPU: x86
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; (R1 1.5); .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)
Host: 192.168.1.102:84
Connection: Keep-Alive

analog input 0 is 318

analog input 1 is 246

analog input 2 is 222

analog input 3 is 198

analog input 4 is 164

analog input 5 is 184

Thanks again zoomcat for your investigation.

Looks like the behaviour is different at your place. Please let me ask further questions:

  1. The "very long delay" may be caused by the blink2() calls in my program, in combination
    with a different behaviour of your browser (not echoing each char immediately, as it does for me).
    => Please test again without the blink/blink2-calls.

  2. What happens at your place, when you run the original WebServer example?

  3. Maybe I should come back to suspecting the HW. When I am back home, I will see, if I
    can find any revision number or so off my Ethernet Shield, and compare it
    with anything from yours.

Thanks for sticking to this issue.

hi. i couldn't help to notice that we are having kind of similar problems....I think.

I've installed and run the program on this site

and it works fine when not connected to anything.

But when I connect an LED or whatever the page hangs infinitely.....

@Thomas33: I have a problem just like that. What did you do to correct it?

I get hung in the "while(client.available())" loop forever. The bytes available returned never gets smaller than 1024.

Dude! If you can come up with the solution, I will nominate you for "Arduino Linux King", and I don't think anyone here would challenge it.

EDIT: If you want to see what I have found so far:
http://arduino.cc/forum/index.php/topic,68624.0.html
Please excuse the references to "B.S." My bad.
That was prior to understanding the situation here.

@Thomas33: I'm sorry. You did not make it to "Arduino Linux King". Found it on my own. See the link above.

For what is worth, I can confirm this bug when using an Arduino Uno.
If I use Firefox or Konqueror, I get no answer. If I telnet port 80 or use wget, I get the correct answer.

Just tested compiling sketch from a windoze VM, and it works.
So it seems it's a compiler bug.
I reread instructions/prerequisites and in fact in Mandriva 2010.2 there are:
cross-avr-gcc-c++-4.4.3-7mnb2
cross-avr-binutils-2.20.51.0.4-1mnb2
avr-libc-1.6.8-1mdv2010.1
cross-avr-gcc-cpp-4.4.3-7mnb2
avrdude-5.10-1mdv2010.1
cross-avr-gcc-4.4.3-7mnb2

Could be a good thing to pin it down and patch but I think I'm not skilled enough for this task.

Hi SurferTim,

SurferTim:
@Thomas33: I'm sorry. You did not make it to "Arduino Linux King". Found it on my own. See the link above.

No, not me - but you rock!!! Thanks very much for all your investigation. You really pushed the issue, until something happend. Great!

I lost any hope in April, and just came back today to check for any progress. I haven't tested your fixes by now, but it sounds good I I look forward in confidence.

===================

Do you have any comment about the non-standard recv call, as I wrote about in

As long as this call works different than a standard recv, it will be impossible to use standard SW (well, due to the kind of difference it will mostly work, but sometimes fail).

@Thomas33: Glad you are back. I thought you had disappeared. You had done the initial research that helped me find the problem. It would be a shame if you had not been here to enjoy the rewards of your effort.

I don't know about the non-standard recv. My Mega2560 and the Ethernet shield are working great now. The Client::available() function was to blame. The fix corrected that problem. I could not understand why the read function was also affected, until I found that the read function calls the available function before reading.

I have added another routine to the ethernet library. This time to check the transmit buffer. Check reply#3 on this thread.
http://arduino.cc/forum/index.php/topic,72027.0.html

Now I don't have to guess if the client was sent everything, or if the connection has stalled.

Thomas33:
Hello,

I think I found the error(s) now by myself:

In w5100.cpp we have the method W5100Class::getRXReceivedSize. This method can tell, if and how much data is available. However, this method is only used in client (and udp), but in the client it only checks, if data is available, not using the actul byte count. This is probably where the whole problem starts.

The "Client" class in the Ethernet library only has a function to read a single byte so this doesn't matter.

Thomas33:
2.
W5100Class::recv_data_processing just reads the wanted number of bytes (len) from the chip, not asking, if they are valid (came in).

"Client" will never ask for more than one byte when it calls that function...

The "Client" class in the Ethernet library only has a function to read a single byte so this doesn't matter.

Actually, it does. That is the line that guided me to the fix. It was the getRXReceivedSize function that ended up containing the bug.

Edit: My apology to Thomas33. You did good.
At first, when you did not post the solution, I thought it was B.S.
Then, when I found the solution and it was exactly where you suspected, I thought "That S.O.B. did find it!". That is why I said "found it on my own". I thought you had found it.
Now the story comes out. Thanks for your effort. That line above is what did it. :slight_smile:

Ups - I nearly missed the follow-ups, as they went to page [2] ...

SurferTim:
@Thomas33: Glad you are back. I thought you had disappeared. You had done the initial research that helped me find the problem. It would be a shame if you had not been here to enjoy the rewards of your effort.

Indeed, the was a big dust layer on the arduino board :slight_smile:

@fungus:

"Client" will never ask for more than one byte when it calls that function...

Yes, this is right. And therefore it is not a bug, if using "Client" is the way to go. But the recv is different to other recv behaviours, and it may be confusing, as the name of the function was obviously taken from other implementations, but not the behaviour. But I dont want to demand on any changes, I just wanted to put some attention to it, in case, someone is willing to investigate.

@surferTim: Thanks for your plaudit :slight_smile: However, I still beliefe, that you did the main work :slight_smile:

You may have seen, that I opened another issue right away, maybe you have a look at it too:
http://arduino.cc/forum/index.php/topic,73170.0.html