W5100/Ethernet cannot use socket as UDP after using it as TCP

Does seem odd that it would fail for a DNS request that only came once every 5 seconds. That's pretty far from a DoS. Anyway, I changed my code to use 4.2.2.1 public DNS. I posted it on the other thread...

I am concerned about DoS when publishing my public ip, and not because of this code. Here is my serial monitor display for a short run. It will go on for hours like this.

Start!
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 0 D:0.0.0.0(0)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
xx.xx.58.116
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 1024 D:xx.xx.58.115(53)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
xx.xx.58.116
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 1036 D:xx.xx.58.115(53)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
xx.xx.58.116
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 1031 D:xx.xx.58.115(53)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x17 80 D:192.168.1.254(3691)
Socket#1:0x17 80 D:192.168.1.254(3692)
Socket#2:0x14 80 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.2.2
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

client disconnected
xx.xx.58.116
Socket#0:0x0 1026 D:xx.xx.58.115(53)
Socket#1:0x17 80 D:192.168.1.254(3692)
Socket#2:0x14 80 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x14 80 D:xx.xx.58.115(53)
Socket#1:0x17 80 D:192.168.1.254(3692)
Socket#2:0x17 80 D:192.168.1.254(3693)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.2.2
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

client disconnected
xx.xx.58.116
Socket#0:0x14 80 D:xx.xx.58.115(53)
Socket#1:0x0 1038 D:xx.xx.58.115(53)
Socket#2:0x17 80 D:192.168.1.254(3693)
Socket#3:0x0 0 D:0.0.0.0(0)
xx.xx.58.116
Socket#0:0x14 80 D:xx.xx.58.115(53)
Socket#1:0x0 1034 D:xx.xx.58.115(53)
Socket#2:0x0 80 D:192.168.1.254(3693)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x17 80 D:192.168.1.254(3694)
Socket#1:0x17 80 D:192.168.1.254(3695)
Socket#2:0x14 80 D:192.168.1.254(3693)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.2.2
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

client disconnected
xx.xx.58.116
Socket#0:0x0 1029 D:xx.xx.58.115(53)
Socket#1:0x17 80 D:192.168.1.254(3695)
Socket#2:0x14 80 D:192.168.1.254(3693)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x14 80 D:xx.xx.58.115(53)
Socket#1:0x17 80 D:192.168.1.254(3695)
Socket#2:0x17 80 D:192.168.1.254(3698)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.2.2
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

client disconnected
xx.xx.58.116
Socket#0:0x14 80 D:xx.xx.58.115(53)
Socket#1:0x0 1024 D:xx.xx.58.115(53)
Socket#2:0x17 80 D:192.168.1.254(3698)
Socket#3:0x0 0 D:0.0.0.0(0)
a
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
xx.xx.58.116
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 1024 D:xx.xx.58.115(53)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
xx.xx.58.116
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 1036 D:xx.xx.58.115(53)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
xx.xx.58.116
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 1031 D:xx.xx.58.115(53)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x17 80 D:192.168.1.254(3691)
Socket#1:0x17 80 D:192.168.1.254(3692)
Socket#2:0x14 80 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.2.2
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

client disconnected
xx.xx.58.116
Socket#0:0x0 1026 D:xx.xx.58.115(53)
Socket#1:0x17 80 D:192.168.1.254(3692)
Socket#2:0x14 80 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x14 80 D:xx.xx.58.115(53)
Socket#1:0x17 80 D:192.168.1.254(3692)
Socket#2:0x17 80 D:192.168.1.254(3693)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.2.2
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

client disconnected
xx.xx.58.116
Socket#0:0x14 80 D:xx.xx.58.115(53)
Socket#1:0x0 1038 D:xx.xx.58.115(53)
Socket#2:0x17 80 D:192.168.1.254(3693)
Socket#3:0x0 0 D:0.0.0.0(0)
xx.xx.58.116
Socket#0:0x14 80 D:xx.xx.58.115(53)
Socket#1:0x0 1034 D:xx.xx.58.115(53)
Socket#2:0x0 80 D:192.168.1.254(3693)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x17 80 D:192.168.1.254(3694)
Socket#1:0x17 80 D:192.168.1.254(3695)
Socket#2:0x14 80 D:192.168.1.254(3693)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.2.2
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

client disconnected
xx.xx.58.116
Socket#0:0x0 1029 D:xx.xx.58.115(53)
Socket#1:0x17 80 D:192.168.1.254(3695)
Socket#2:0x14 80 D:192.168.1.254(3693)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x14 80 D:xx.xx.58.115(53)
Socket#1:0x17 80 D:192.168.1.254(3695)
Socket#2:0x17 80 D:192.168.1.254(3698)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.2.2
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

client disconnected
xx.xx.58.116
Socket#0:0x14 80 D:xx.xx.58.115(53)
Socket#1:0x0 1024 D:xx.xx.58.115(53)
Socket#2:0x17 80 D:192.168.1.254(3698)
Socket#3:0x0 0 D:0.0.0.0(0)

my 2 cents;-

UDP is a lightweight protocol that by design doesn't handle things like packet sequencing. It does not guarantee that your packets will arrive in order (It does not even guarantee that your packets will arrive at all.) .
Say send command set voltage=1 v, then set voltage=4 v, You might get 0v or 1v or 4v if default is 0v. TCP is a better choice if you want robust packet delivery and sequencing.
If you're limited to using UDP you would have to develop a method of identifying the out of sequence packets and resequencing them or request to resend, but for Arduino is tough task.

a lot people implement UDP with Arduino, I mean neither OP nor this thread please hold fire.

I guess either they drink too much Italian coffee or I drink too much, might be both. :stuck_out_tongue:

@sonnyyu: That is a good thought, but NTP, DNS, and DHCP use UDP. You really can't avoid it.

SurferTim:
@sonnyyu: That is a good thought, but NTP, DNS, and DHCP use UDP. You really can't avoid it.

I mean neither OP nor this thread please hold fire.

I'm not firing at you. :slight_smile: I'm just stating a fact. I am not fond of UDP either as a reliable transfer, but we are subjected to it through theses services.

Thanks, that's 100% definitive. You are not having the same issue as I am, although I have no idea why...

One thing I just found out was that if the TCP request comes from the same machine as the UDP destination goes to (i.e. if you run a WGET from the same router that's hosting your DNS server) the problem never happens (for me). It's almost like it's a IP->MAC arp issue. Odd. I'm going to delve into this a bit more before I bother anyone. Looking up W5100 and ARP I did come up with this: http://forums.parallax.com/showthread.php/129515-W5100-Sn_SR-undocumented-misfeature-bug Not sure if that's related though... I'll get to the bottom of this at some point...

Maybe you are right. Maybe the delay waiting for the response from the dns server is not long enough. That could cause the dns fail.

edit: Your test is still running on my Arduino. It has not missed a beat since I started it.
Just so you know...

Wow, OK. Still not to the bottom of this, but I just had a pretty big discovery. When I get the DNS failure, the packet actually goes to the wrong machine! In my example I posted the DNS server was outside my network, so it should have been sent to the gateway MAC address. However, with a packet capture I found that it actually ended up going to the same machine that my TCP request came from!!!

That's exactly why this isn't working right. For some reason the W5100 gets confused and doesn't update the IP->MAC port for that socket. I'm going to delve into this a bit more to see if I can get to the bottom of what's going on...

Some background information about NTP, DNS, and DHCP use UDP;-

DNS works on both TCP and UDP

DHCP works on UDP only

Its use is in a typical local area network (LAN). The packet sequencing is never problem.

NTP

It used work both TCP and UDP [RFC0793] , UDP/TCP Port 123 was previously assigned by IANA for NTP. The new [RFC5905] drop TCP support. Here is the reason;- "Reliable message delivery such as TCP [RFC0793] can actually make the delivered NTP packet less reliable since retries would increase the delay value and other errors." obviously implement correctly UDP client is needed to take care the issue.

Now the question is "What Arduino NTP client does ?"

That could be it. I just tried the same test using a remote dns server and my domain, and it crashed like the Hindenberg. :frowning:

Now I am back to a localnet dns server and it is ok.

@sonnyyu: No question there. Arduino NTP uses UDP.

SurferTim:
That could be it. I just tried the same test using a remote dns server and my domain, and it crashed like the Hindenberg. :frowning:

Now I am back to a localnet dns server and it is ok.

@sonnyyu: No question there. Arduino NTP uses UDP.

My fault, the complete question is;

What Arduino NTP client does to take care reliable NTP packet deliver?

This might hijack the thread, I might start new thread about it.

It does nothing for "reliable". It waits 1 second. If no response, it bails.

And the remote server thing just went out the window. I forgot to remove my SD card again, and you code does not disable it. It is working fine with my domain and a remote dns server.

I have also hit this bug along with this thread in searching for an answer.

I can add a couple of things I have found trying to fix this.

It appears to be any protocol TCP then UDP or TCP then ICMP are the 2 bugs I have encountered. If the Wiznet socket is used for TCP it can't do ICMP reliably either to a different IP then TCP was connected with. I haven't proven it out for 100% sure but I do believe this bug is true for UDP first then ICMP on the same socket.

I have also had it ping the wrong IP (always the GW IP) I believe this is a known bug that is in the latest Wiznet errata stating that you need to leave the GW and mask set to zeros until you open connections this prevents the ARP request returning the GW MAC as a default.

Multiple other bugs in the Arduino Ethernet library. DNS returning positive results with the wrong IP if the lookup failed. Ethernet client connections that only work through a GW but not on the same subnet.

Arduino Due is @ 192.168.20.151 with a valid unique MAC Address

socket 0 -> TCP with 192.168.20.2 ->close TCP -> ICMP Ping to 192.168.20.254(works once then fails)
socket 1 -> TCP with 192.168.20.2 - >open,close,open close TCP always works
socket 2 - > TCP with 192.168.20.2 always works
socket 3 -> ICMP Ping to 192.168.20.254 always works never has TCP on it.

Here is something additional that leans towards an ARP table problem.

socket 0 -> TCP with 192.168.20.2 ->close TCP -> ICMP Ping to 192.168.20.2(Works fine)
socket 1 -> TCP with 192.168.20.2 - >open,close,open close TCP always works
socket 2 - > TCP with 192.168.20.2 always works
socket 3 -> ICMP Ping to 192.168.20.254 always works never has TCP on it.

Its like all 4 sockets share the same ARP table that is too short to stay updated for multiple connections on multiple sockets.

If TCP and ICMP are trying to talk to the same IP and of course MAC address all works no failures.

I tried watching the states that show the MAC address has been resolved.

I think a TCP/IP stack that just uses the MAC Raw mode is the only true solution. I need TCP/UDP/ICMP/Multicast UDP to function at any given time on any given open Wiznet socket.

Tom

Tom, I never did get a solution to this W5100 bug. I ended up implementing a workaround that just kept re-trying on failures. I agree 100% that this is an ARP issue that seems to manifest itself when you switch from TCP to UDP on the same socket.

Wish I could be of more help.

So it looks like the Wiznet errata (http://www.wiznet.co.kr/Admin_Root/UpLoad_Files/BoardFiles/3150Aplus_5100_errata_en_v2.4.pdf) does show a fix for this problem. However if I look in the Arduino libraries which I'm using (Arduino 1.0.5) I'm not able to find the implementation of this fix. Has anyone ever tried to implement this fix ? At first sight it shouldn't be it too difficult as it involves just a few fixes in libraries/Ethernet/utility/socket.cpp.

FYI: I ended up on this thread as I'm having problems with UDP and TCP connections, but I'm not yet sure if the problem in this thread is the same as my problem. I have not debugged it that far yet...

Hey all,

I know the last post here is way over a year ago, but I had comparable problems with the most current Arduino IDE 1.6.5 and I wanted to know if anyone found a solution ?! Or was there fixes done in Ethernet Library or does someone has done it by it's own?

I have added an NTP query to my code and Ethernet became unstable and/r blocked sockets appeared. The NTP query was using UDP as already mentioned here ...

For now I created a TCP-UDP-Tunnel using socat in my local network and changed the NTP code to use TCP and anything is stable again ... yes know that TCP is not the best and a TCP-UDP-tunnel also needs time but I don't need the time microsecond-exact :wink:

I haven't experienced a problem using UDP and TCP. What do you mean by this?

I have added an NTP query to my code and Ethernet became unstable and/r blocked sockets appeared. The NTP query was using UDP as already mentioned here ...

How did you determine the blocked sockets? How did it become unstable?

I think I found the problem sending UDP packets after TCP in the same socket. The problem, as stated above in this forum , is that if you are sending the UDP packet fails in the ARP request and in the register SnDHAR remains the MAC address of the previous request. To remedy the situation, do the following :

  1. open the socket : Udp.start (port)
  2. record to the register SnDIPR broadcast address :
    uint8_t ip[6]={0xff,0xff,0xff,0xff}; W5100.writeSnDIPR(sock, ip);
  3. use the command send dummy packet Udp.endPacket();
  4. Next is usually to prepare a pack for recording : Udp.startPacket(server, serverport)
  5. to record the desired Udp.write(data,len)
  6. sent packet: Udp.endPacket();
  7. close the Udp socket.stop();

PS. to find out the socket for paragraph 2. you can , for example, to change the EthernetUDP class , adding to the function uint8_t getsocket (){return _sock;},
or you can create a inherited class EthernetUDPMy: public EthernetUDP, and where to add the function getsocket().

class  EthernetUDPMy : public EthernetUDP {
	public:
		EthernetUDPMy::EthernetUDPMy() : EthernetUDP () {};
		uint8_t getsocket() {return _sock;}

};
EthernetUDPMy Udp;



//.......

	if (Udp.getsocket()==MAX_SOCK_NUM) {
		Serial.println("Udp Open Socket");
		if (!Udp.begin(localUdpPort)) return 0;
		clearArpUdp(Udp.getsocket());	
		Udp.endPacket();
	}
	if (!Udp.beginPacket(serverstr, port)) {Udp.stop(); return 0;}
//....
	rez=Udp.write(data, len);
	int isSent=Udp.endPacket();
	if (rez!=len || !isSent) {
		Serial.println("Error sent UDP");
		Udp.stop();
		rez=0;
	}





void clearArpUdp(SOCKET sock){
	if (sock>=MAX_SOCK_NUM) return;
	SPI.beginTransaction(SPI_ETHERNET_SETTINGS);
	uint8_t c[6]={0xff,0xff,0xff,0xff};
	W5100.writeSnDIPR(sock, c);
	SPI.endTransaction();
}