Ethernet shield problems! Time to update library?

I've been testing various programs and I noticed that the Arduino hangs for about 35 seconds when using the Arduino to load a small status message from an Apache/PHP webserver.

A lot of people are running into problems with the Ethernet shield and/or Ethernet library, here are some topics and possible solutions:

Reducing ethernet traffic by copying new client files to ethernet library directory: Learning About Arduino Ethernet - LearningAboutArduinoEthernet

Using client.status() to find out what's wrong:
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1235991468

Modding the Ethernet shield to fix cold start hardware bug:
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1225354009/29#29

Client.stop() not working properly:
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1235494406

Use Ethernet2 library:
http://code.google.com/p/tinkerit/source/browse/#svn/trunk/Ethernet2%20library/Ethernet2

Ethernet2 library topic:
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1233499187

All topics have various solutions for the problems that are discussed, but I think the nasty 35 second hang bug still isn't fixed. Here's the code that prints client.status() in an attempt to find out what causes the Arduino to hang:

#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 192, 168, 1, 66 }; // Arduino IP
//byte server[] = { 64, 233, 187, 99 }; // Google
byte server[] = { 192, 168, 1, 17 }; // desktop pc running wampserver 2.0 and a simple php script that echoes whatever value is supplied in the querystring

// int receivingData = 0;
int isConnected = 0;
int timer = 0;
Client client(server, 80);

void connectMe(){
client.stop();
delay(500);
Serial.println("connecting...");

if (client.connect()) {
Serial.println("connected");
isConnected = 1;
client.println("GET /?rfidkey=12345678 HTTP/1.0");
client.println();
Serial.print("client.status = ");
Serial.println(client.status(),DEC);
} else {
Serial.println("connection failed!");
Serial.print("client.status = ");
Serial.println(client.status(),DEC);
}
}

void disconnectMe(){
Serial.println();
Serial.println("disconnecting...");
isConnected = 0;
client.flush();
client.stop();
if(!client.connected()){
Serial.println("disconnect error!");
}else{
Serial.println("disconnected");
Serial.print("client.status = ");
Serial.println(client.status(),DEC);
}
}

void setup(){
Ethernet.begin(mac, ip);
Serial.begin(9600);
delay(1000);
connectMe();
}

void loop(){
if (client.available() && client.connected() && isConnected == 1) {
char c = client.read();
Serial.print(c);
} else if (!client.connected()){
if (isConnected == 1){
disconnectMe();
}
delay(1000);
Serial.print("client.status = ");
Serial.println(client.status(),DEC);
}
}

And here's a copy of the serial monitor window:

connecting... connected client.status = 28 HTTP/1.1 200 OK Date: Sun, 29 Mar 2009 02:28:55 GMT Server: Apache/2.2.11 (Win32) PHP/5.2.9-1 X-Powered-By: PHP/5.2.9-1 Content-Length: 8 Connection: close Content-Type: text/html

12345678
disconnecting...
disconnected
client.status = 24
client.status = 24 <AFTER THIS IT HANGS FOR 35 SEC>

client.status = 0 <35 SECONDS LATER: RETURNED TO LOOP>
client.status = 0
client.status = 0
client.status = 0
client.status = 0
client.status = 0

connecting... <HANGS FOR 35 SECONDS>
connection failed!

client.status = 0
client.status = 0
client.status = 0
client.status = 0

connecting...
connection failed!
client.status = 23

The results are pretty random.. sometimes I can do sequential requests within a minute, but most of the time it just cannot connect :-/ I think is has to do with the return value of client.status() but I'm not sure... I've tried this code on an Deumilanove with both a 168 and 328. Running arduino 15 on XP. Tried various webservers, but all run Apache.. finally installed WAMP on my local pc to make sure the problem isn't caused by routers/webhosts...

So, Arduino DEV team or any other Arduino pro's, how can we solve this problem? It would be very helpful to update the Wiki or reference.. to save other people the problems that most users encounter when trying to load something over http... Time to add solutions from various topics to Arduino 0016 ?

Edit: another thing I noticed is that client.connect, client.stop and client.connected aren't color coded in the IDE...
Edit 2: similar problem described here: http://forums.ladyada.net/viewtopic.php?f=31&t=10192

My guess is that the 35 second delay is the timeout for a socket. Your first "hang" is likely a situation in which client.connected() returns true but isConnected is 0. In this case, neither condition in your loop() is satisfied, so nothing is printed. Then, after the socket times out, client.connected() returns 0 and you get the client.status = 0 messages. Note that the code prints "disconnect error!" when client.connected() returns false (implying that the connection was successfully closed). The case that you identify as "board crashed" is also likely a situation in which client.connected() returns true (as it would if client.status() were 23, which is SOCK_ESTABLISHED) but isConnected is 0. The second 35 second hang seems to be an attempt to connect that times out; this is not unusual (although it does beg the question of why the connection fails).

There seem to be two issues. One, the Client.connect() function appears to be returning 0 in cases in which the connection was successfully established (your last attempt). This is probably a bug in the library. The other is the failure to connect (your second attempt). This could be a bug in the library or simply a network failure or other error. But it seems like it would be good to limit the timeout.

The W5100 code provided by Wiznet (found in the Utility folder in the Ethernet library) includes setRTR and setRCR functions. These set the timeout and the retry count, respectively. I can't find that these values are set anywhere, though I might have missed them. They default to a 200ms timeout and 8 retries. However, that's only 1.6 seconds. If a timeout is triggered, all those registers do is set a Timeout flag in Sn_IR. I know socket.c at least looks at it, though I haven't traced how it works or if it actually ends the connection if a timeout occurs.

Oh hey, they'll be at ESC this week. I may stop by their booth and see if an engineer is there and has any tips.

@mellis: Thanks for your reply! I get the same long delay when I just modify a few numbers in the Google search code example. The code I posted isn't 100% done, it's just one of 1000 solutions that I tried to find out what causes the hang...

The strange thing is that I don't get this delay when I try the Google example over and over again.. but when I connect to an Apache webserver the problem suddenly appears. Could this have something to do with the way connections are handled? I found out that Google runs GWS, google web server, a modified version of Apache.

I tried the ethernet library, I tried updating the client.cpp & client.h files, I tried the ethernet2 libary, I tried looping connection.stop(), but this bug persists.. Can anyone with an Ethernet shield try the example code above with a different webserver? Does it hang the first time? Does it hang after a hard reset?

@macegr: that would be great.. I noticed the led on pin13 going dim during these hangs, which means that it's flashing at a high rate.. so there's definitely some SPI stuff going on as well.. could this be the cause?

@Mellis: What would be the best way to get these bugs fixed a.s.a.p? I bought a bunch of the official Arduino Ethernet shields for a project, but now we're stuck due to this problem.. We need to send the ID of an RFID tag to a webserver every time a tag has been scanned, any suggestions for possible workarounds? Any help is greatly appreciated!

I have a similar problem try to read a slot counter+lap timer with a webserver

Our threat for details at: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1235245502/10#10

The fastest way to get these fixed would be if you can find the solution. For the problem of the connect() function returning 0 even though the connection succeeds, take a look at the Client::connect() code in ARDUINO/hardware/libraries/Ethernet/Client.cpp. The first question is which "return 0" is being called:

  if (!::connect(_sock, _ip, _port))
    return 0;

or:

  while (status() != SOCK_ESTABLISHED) {
    if (status() == SOCK_CLOSED)
      return 0;
  }

The second question is why. Maybe try adding a small delay between the call to connect() and the start of the while-loop? Or try looking for an intermediate status (SOCK_INIT) before starting that loop?

I'm not sure what to suggest for why the connection sometimes fails. It may be because when you reset the board, the connection is started from the same source port again. You could try editing Client::connect() to use a random _srcport.

Also, it appears that the disconnect (i.e. Client::stop()) sometimes fails. Here, it would be good to check the status() of the client after calling stop(). Be sure there's no data available().

Does that give you a place to start?

Here's my research and a fix (work-around) for the multiple client connect issue.

http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1235494406/8#8

I'm not sure what to suggest for why the connection sometimes fails. It may be because when you reset the board, the connection is started from the same source port again. You could try editing Client::connect() to use a random _srcport.

I have watched the connection by tshark, it does connect from a different port at each reset. When using the local echo example the connection looks like this;

  3  17.126960  192.168.0.2 -> 192.168.0.3  ICMP Echo (ping) request
  4  17.127116  192.168.0.3 -> 192.168.0.2  ICMP Echo (ping) reply
 11  34.911229  192.168.0.2 -> 192.168.0.3  TCP 36588 > telnet [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=230615 TSER=0 WS=6
 12  34.911382  192.168.0.3 -> 192.168.0.2  TCP telnet > 36588 [RST] Seq=1 Win=0 Len=0

It does respond to PING requests, but it does not respond to any telnet requests.

When using the client example, the tshark output looks like this;

72 520.957368 de:ad:be:ef:fe:ed -> Broadcast    ARP Who has 192.168.0.2?  Tell 192.168.0.3
 73 520.957410 SamsungE_b0:47:e1 -> de:ad:be:ef:fe:ed ARP 192.168.0.2 is at 00:13:77:b0:47:e1
 74 520.957554  192.168.0.3 -> 192.168.0.2  TCP [TCP Port numbers reused] blackjack > http [SYN] Seq=0 Win=2048 Len=0 MSS=1460
 75 520.957605  192.168.0.2 -> 192.168.0.3  TCP http > blackjack [SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1460
 76 520.957755  192.168.0.3 -> 192.168.0.2  TCP blackjack > http [ACK] Seq=1 Ack=1 Win=2048 Len=0
 77 520.995661  192.168.0.3 -> 192.168.0.2  TCP blackjack > http [FIN, ACK] Seq=1 Ack=1 Win=2048 Len=0
 78 520.995811  192.168.0.2 -> 192.168.0.3  TCP http > blackjack [FIN, ACK] Seq=1 Ack=2 Win=5840 Len=0
 79 520.995959  192.168.0.3 -> 192.168.0.2  TCP blackjack > http [ACK] Seq=2 Ack=2 Win=2048 Len=0

But when I check the Apache web server logs, there is nothing in there.

etracer came up with a workaround that works for me:

http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1235494406/8#8

Thanks again!

I talked to our US Wiznet rep at ESC yesterday, but didn't get too far...they rotate someone from Korea every so often and it had just happened, he was only able to agree with everything I was saying :slight_smile:

I'll send an email, C is the universal language!

Hi, so I recently received the most excellent Arduino Mega and I'm having a bit of trouble interfacing it with the ethernet shield like I had hoped I could do. I can't seem to connect with it at all, though I can get a nice display of blinking LEDs while trying.

I have an older arduino NG.C board that I haven't yet tested the shield with (a friend is borrowing it) so my entire problem could be related to it.

My question stems from the fact that the ethernet shield is based on SPI and that the pins for the Mega for SPI seem to be different than the other boards. Could this be the cause of my problems?

Specifically, I will point to these two links:

and

For the mega, it says SPI is located on pins 50-53 and for the diecimila (pretty close to my Rev C I think) it says that SPI is located on pins 10-13.

Thank you in advance for any help you may provide.

Yeah, the SPI pins moving is going to be a deal breaker for the Ethernet shield, all existing Ethernet shields will have to be re-designed to work with the Mega.

all existing Ethernet shields will have to be re-designed to work with the Mega.

Or the SPI part of the code, which is conveniently abstracted already, can be rewritten to bit-bang the SPI for the MEGA.

-j

Because the Ethernet library needs to be even slower :frowning:

But lose the advantage of being hardware-based SPI in the process.

--Phil.

I've posted a completely revised Client portion of the Ethernet library. It includes many fixes and seems to resolve any connection issues or situations where a connection fails to completely close.

See here: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1238640832/0#0

Yeah, the SPI pins moving is going to be a deal breaker for the Ethernet shield, all existing Ethernet shields will have to be re-designed to work with the Mega.

Well, it sucks that I'm the one to find this out. I think a lot of folks are going to be disappointed that the ethernet shield isn't pin compatible with the Mega.

Would it be possible to move the pins physically to resolve this? The SPI connections on the mega are not under the ethernet shield so I would think that cutting off the header pins for 10-13 and then running wires from the headers to the SPI connection would resolve this issue. Would that require any modification to the Ethernet library to take care of the pin assignments?

:slight_smile:

-j

Smartprojects have this warning in their product description:

It has 53 IO (including 4 HW UARTs, 14 PWMs, I2C bus) and 16 Analog Input pins.
It can mount and use most of the Arduino Shields (not the ETH actual version).

http://www.smartprj.com/catalog/index.php?main_page=product_info&cPath=1&products_id=37

Perhaps this warning should be added to the Arduino Mega product page too...