Show Posts
Pages: [1] 2
1  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: November 21, 2013, 02:06:59 pm
Tom,  I never did get a solution to this W5100 bug.  I ended up implementing a workaround that just kept re-trying on failures.  I agree 100% that this is an ARP issue that seems to manifest itself when you switch from TCP to UDP on the same socket.

Wish I could be of more help. 
2  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 28, 2013, 12:27:07 pm
Wow,  OK.  Still not to the bottom of this, but I just had a pretty big discovery.  When I get the DNS failure, the packet actually goes to the wrong machine!  In my example I posted the DNS server was outside my network, so it should have been sent to the gateway MAC address.  However, with a packet capture I found that it actually ended up going to the same machine that my TCP request came from!!!

That's exactly why this isn't working right.  For some reason the W5100 gets confused and doesn't update the IP->MAC port for that socket.  I'm going to delve into this a bit more to see if I can get to the bottom of what's going on...
3  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 28, 2013, 12:05:57 pm
Thanks,  that's 100% definitive.  You are not having the same issue as I am, although I have no idea why...

One thing I just found out was that if the TCP request comes from the same machine as the UDP destination goes to (i.e. if you run a WGET from the same router that's hosting your DNS server) the problem never happens (for me).  It's almost like it's a IP->MAC arp issue.  Odd.  I'm going to delve into this a bit more before I bother anyone.  Looking up W5100 and ARP I did come up with this:  http://forums.parallax.com/showthread.php/129515-W5100-Sn_SR-undocumented-misfeature-bug  Not sure if that's related though...  I'll get to the bottom of this at some point...

4  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 28, 2013, 11:41:24 am
Does seem odd that it would fail for a DNS request that only came once every 5 seconds.  That's pretty far from a DoS.  Anyway, I changed my code to use 4.2.2.1 public DNS.  I posted it on the other thread...
5  Development / Suggestions for the Arduino Project / Re: BUG: W5100 cannot use socket as UDP after using it as TCP on: April 28, 2013, 11:39:27 am
I am still having this bug.  Upon significant back and forth, we have not been able to reproduce it on Tim's setup (thanks Tim!), but it's very firmly repeatable on mine.  I'd love to see if someone else could give it a shot.

Here's the easiest to use code:

Code:
#include <SPI.h>
#include <Ethernet.h>
#include <EthernetUdp.h>
#include <utility/w5100.h>
#include "Dns.h"

void ShowSockStatus()
{
for (int i = 0; i < MAX_SOCK_NUM; i++) {
Serial.print("Socket#");
Serial.print(i);
uint8_t s = W5100.readSnSR(i);
Serial.print(":0x");
Serial.print(s,16);
Serial.print(" ");
Serial.print(W5100.readSnPORT(i));
Serial.print(" D:");
uint8_t dip[4];
W5100.readSnDIPR(i, dip);
for (int j=0; j<4; j++) {
Serial.print(dip[j],10);
if (j<3) Serial.print(".");
}
Serial.print("(");
Serial.print(W5100.readSnDPORT(i));
Serial.println(")");
}
}

byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

EthernetServer server(80);
IPAddress dnsserver(4,2,2,1);
IPAddress ip(192,168,10,20);
IPAddress gateway(192,168,10, 2);
IPAddress subnet(255, 255, 255, 0);

long timer;

void DoDNS()
{
DNSClient dns;
IPAddress remote_addr;
dns.begin(dnsserver);
if (dns.getHostByName("www.yahoo.com", remote_addr) == 1)
{
for (int i=0; i<4; i++) {
Serial.print(remote_addr[i]);
if (i!=3)
Serial.print('.');
else
Serial.println("");
}
}
else
Serial.println("DNS FAIL!");
ShowSockStatus();
}

void setup() {
Serial.begin(9600);
Ethernet.begin(mac, ip, dnsserver, gateway, subnet);
delay(500);

Serial.println("Start!");
DoDNS();
server.begin();
timer = millis();
}


void loop()
{
//Check if a web client has attached.
EthernetClient client = server.available();
if (client) {
Serial.println("new Client");
ShowSockStatus();
boolean currentLineIsBlank = true;
while (client.connected()) {
if (client.available()) {
char c = client.read();
Serial.print(c);
if (c == '\n' && currentLineIsBlank) {
client.stop();
}
if (c == '\n') {
// you're starting a new line
currentLineIsBlank = true;
}
else if (c != '\r') {
// you've gotten a character on the current line
currentLineIsBlank = false;
}
}
}
// give the web browser time to receive the data
delay(1);
// close the connection:
client.stop();
Serial.println("client disconnected");
}

if ((millis() - timer) > 5000) {
DoDNS();
timer = millis();
}
}

 

If you start this, let it do a couple of DNS lookups and then browse to the Arduino with a web browser (or wget) you should start getting DNS failure messages.

Here's an output:

Code:
Start!
98.139.183.24
Socket#0:0x0 1039 D:4.2.2.1(53)
Socket#1:0x0 0 D:0.0.0.0(0)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x17 80 D:192.168.10.1(58210)
Socket#1:0x14 80 D:0.0.0.0(0)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.10.20
User-Agent: Wget

client disconnected
DNS FAIL!
Socket#0:0x0 1032 D:4.2.2.1(53)
Socket#1:0x14 80 D:0.0.0.0(0)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
DNS FAIL!
Socket#0:0x0 1037 D:4.2.2.1(53)
Socket#1:0x14 80 D:0.0.0.0(0)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)

Note that the full secnario to get it to fail is for a specific socket (in this run above, it's Socket 0) goes from UDP transaction to TCP transaction and then you try and use it for UDP again.  In that scenario it won't work (for me)...
6  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 28, 2013, 11:09:30 am
Thanks Tim.  I should have mentioned the SD card thing.  I didn't include any initialization code to turn of the SD card.

I guess I'd love to see your output if you have it available...

I'm beginning to think that your setup is just not exhibiting the same things that mine is.  Maybe there's a batch of 5100s that have bugs?  Maybe it's my version of the Ethernet shield.  (I've got a couple that I bought off eBay).  I don't know.  I've used two different shields with three different boards (2 megas and 1 Uno).  I've got no idea why it seems so repeatable on my setup with very varied scenarios, but we can't seem to get yours to exhibit the same thing.

7  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 26, 2013, 06:00:58 pm
Are you using the code I put in the message?  There shouldn't be any NTP packets going across because there's nothing NTP related in that code. 

It's a reliable DNS server.  If I never browse to the web port on the Arduino then it'll sit there and poll forever with no issues.  In fact it's been up polling for the past 3 hours and has not had a single failure.  As soon as I browse to we web port though, it fails (well, I actually have to browse twice for it to happen).
8  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 26, 2013, 02:05:13 pm
Here's a much simpler example using the built in DNS library:

Code:
#include <SPI.h>
#include <Ethernet.h>
#include <EthernetUdp.h>
#include <utility/w5100.h>
#include "Dns.h"

void ShowSockStatus()
{
for (int i = 0; i < MAX_SOCK_NUM; i++) {
Serial.print("Socket#");
Serial.print(i);
uint8_t s = W5100.readSnSR(i);
Serial.print(":0x");
Serial.print(s,16);
Serial.print(" ");
Serial.print(W5100.readSnPORT(i));
Serial.print(" D:");
uint8_t dip[4];
W5100.readSnDIPR(i, dip);
for (int j=0; j<4; j++) {
Serial.print(dip[j],10);
if (j<3) Serial.print(".");
}
Serial.print("(");
Serial.print(W5100.readSnDPORT(i));
Serial.println(")");
}
}

byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

EthernetServer server(80);
IPAddress dnsserver(192,168,10,1);
IPAddress ip(192,168,10,20);
IPAddress gateway(192,168,10, 2);
IPAddress subnet(255, 255, 255, 0);

long timer;

void setup() {
Serial.begin(9600);
Ethernet.begin(mac, ip, dnsserver, gateway, subnet);
delay(500);

Serial.println("Start!");
server.begin();
ShowSockStatus();
timer = millis();
}

void loop()
{
//Check if a web client has attached.
EthernetClient client = server.available();
if (client) {
Serial.println("new Client");
ShowSockStatus();
boolean currentLineIsBlank = true;
while (client.connected()) {
if (client.available()) {
char c = client.read();
Serial.print(c);
if (c == '\n' && currentLineIsBlank) {
client.stop();
}
if (c == '\n') {
// you're starting a new line
currentLineIsBlank = true;
}
else if (c != '\r') {
// you've gotten a character on the current line
currentLineIsBlank = false;
}
}
}
// give the web browser time to receive the data
delay(1);
// close the connection:
client.stop();
Serial.println("client disconnected");
}

if ((millis() - timer) > 5000) {
DNSClient dns;
IPAddress remote_addr;
dns.begin(dnsserver);
if (dns.getHostByName("www.yahoo.com", remote_addr) == 1)
{
for (int i=0; i<4; i++) {
Serial.print(remote_addr[i]);
if (i!=3)
Serial.print('.');
else
Serial.println("");
}
}
else
Serial.println("DNS FAIL!");
timer = millis();
ShowSockStatus();
}
}

And the output

Code:
Start!
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 0 D:0.0.0.0(0)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
98.139.183.24
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 1030 D:192.168.10.1(53)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
98.138.253.109
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 1024 D:192.168.10.1(53)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
98.138.253.109
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 1024 D:192.168.10.1(53)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x17 80 D:192.168.10.213(11298)
Socket#1:0x17 80 D:192.168.10.213(11299)
Socket#2:0x14 80 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.10.20
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

client disconnected
98.139.183.24
Socket#0:0x0 1035 D:192.168.10.1(53)
Socket#1:0x17 80 D:192.168.10.213(11299)
Socket#2:0x14 80 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
98.138.253.109
Socket#0:0x0 1029 D:192.168.10.1(53)
Socket#1:0x17 80 D:192.168.10.213(11299)
Socket#2:0x14 80 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
98.139.183.24
Socket#0:0x0 1039 D:192.168.10.1(53)
Socket#1:0x0 80 D:192.168.10.213(11299)
Socket#2:0x14 80 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
new Client
Socket#0:0x17 80 D:192.168.10.213(11312)
Socket#1:0x14 80 D:192.168.10.213(11299)
Socket#2:0x17 80 D:192.168.10.213(11311)
Socket#3:0x0 0 D:0.0.0.0(0)
GET / HTTP/1.1
Host: 192.168.10.20
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

client disconnected
98.138.253.109
Socket#0:0x17 80 D:192.168.10.213(11312)
Socket#1:0x14 80 D:192.168.10.213(11299)
Socket#2:0x0 1033 D:192.168.10.1(53)
Socket#3:0x0 0 D:0.0.0.0(0)
98.139.183.24
Socket#0:0x17 80 D:192.168.10.213(11312)
Socket#1:0x14 80 D:192.168.10.213(11299)
Socket#2:0x0 1028 D:192.168.10.1(53)
Socket#3:0x0 0 D:0.0.0.0(0)
98.138.253.109
Socket#0:0x17 80 D:192.168.10.213(11312)
Socket#1:0x14 80 D:192.168.10.213(11299)
Socket#2:0x0 1039 D:192.168.10.1(53)
Socket#3:0x0 0 D:0.0.0.0(0)
DNS FAIL!
Socket#0:0x0 1034 D:192.168.10.1(53)
Socket#1:0x14 80 D:192.168.10.213(11299)
Socket#2:0x0 1039 D:192.168.10.1(53)
Socket#3:0x0 0 D:0.0.0.0(0)
DNS FAIL!
Socket#0:0x0 1033 D:192.168.10.1(53)
Socket#1:0x14 80 D:192.168.10.213(11299)
Socket#2:0x0 1039 D:192.168.10.1(53)
Socket#3:0x0 0 D:0.0.0.0(0)


Notice how it works right up till it tries to reuse socket #0 to do the DNS (which has previously been used as UDP/DNS and then TCP).  Then it fails...
9  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 26, 2013, 01:40:42 pm
You actually bring up a fantastic point with your DNS example.   When you call EthernetClient::connect(char * host, uint8_t port), it does exactly what I do with NTP.  It dynamically creates a UDP socket for DNS, connects to a server, and then releases that socket.  That should work just fine, except that this bug exists.  I'm certain that I can create a scenario with a http server that has a few connects() in it that will start failing on DNS resolution because of the bug.  There would be no solution either, because you can't manually specify a socket for the DNS server to use.  It just dynamically allocates one from the unused pool.
10  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 26, 2013, 01:10:47 pm
It sounds like this guy is having exactly the same problem as I am, but he doesn't know it.  He thinks he's running out of sockets, but in actuality the issue is a bug somewhere that won't allow you to reuse a socket that was previously allocated as a TCP socket as a UDP socket.

Your workaround is find for many scenarios, but not mine.  I'm trying to build a simple web server.  One that syncs time via UDP periodically and also includes a TFTP server so you can upload content to the SD card.  TFTP uses 1 UDP socket for the server, and one for the client if it connects.  So, to avoid the bug that I've identified here, I'd have to dedicate one socket to the NTP client, one socket to the TFTP server, one socket to the TFTP client, and then I'd have one remaining socket to use as a my listening socket for HTTP.  Of course that means I don't have a socket to use if someone actually connected to my HTTP server.

In other words, I'm sunk unless I can figure out how to resolve this bug. (or upgrade to the 5200, which seems like a waste)

Incidentally, another workaround is to actually reinitialize the W5100 after every TCP client disconnects.  (i.e. call Ethernet.begin() again)  Of course that means you will also have to go reattach your HTTP server and UDP servers.  Super kludgy, but it may be the only way around this bug.
11  Using Arduino / Networking, Protocols, and Devices / Re: Arduino + Ethernet - webserver is SLOW on: April 26, 2013, 12:51:17 pm
It's a sample.  It doesn't parse the request from the browser at all.  It only sends the one file.  It's a stretch to call it a web server I agree, but it's a sample. 
12  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 26, 2013, 12:45:46 pm
OK.  I see now why yours is working.  You dedicate a socket to the UDP connection that never is freed.  So you never reuse a socket that was previously used as a UDP socket as a TCP socket and vice versa.  Thus you avoid the issue.  In my version the call to CheckTime will create the UDP socket and then tear it down, freeing the  socket for use by other clients. 

What you've got is a workaround for the problem I'm having, but the problem I'm having is still a bug.  The W5100 only has 4 sockets available, and dedicating one of them full time to an NTP client when you only need to use that socket once every few days, and you only need it for about 500ms can be wasteful.  Creating the UDP socket and tearing it down should be a reasonable scenario, but it doesn't work.
13  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 26, 2013, 12:27:53 pm
I guess I don't understand how your UDP connection got setup on socket 0 and the TCP listening socket is on socket 1?  In the code it calls server.begin() before we've done anything with UDP, so that should use socket 0, and the first showSockStatus() gets called before we've ever done anything with UDP, but your first message shows a UDP socket listening on port 8888 as socket 0.  My first message looks like this:

Quote
Start!
Socket#0:0x14 80 D:0.0.0.0(0)
Socket#1:0x0 0 D:0.0.0.0(0)
Socket#2:0x0 0 D:0.0.0.0(0)
Socket#3:0x0 0 D:0.0.0.0(0)
 
14  Using Arduino / Networking, Protocols, and Devices / Re: Arduino + Ethernet - webserver is SLOW on: April 26, 2013, 12:03:38 pm
FYI:  This section is crazy inefficient

Code:
                    if (webFile) {
                        while(webFile.available()) {
                            client.write(webFile.read()); // send web page to client
                        }

The reason is because each call to client.write() will send a full IP packet with only the one character in it.  That's insane overhead.  To make things a-m-a-z-i-n-g-l-y faster, read blocks of data from the file and then write the blocks to the buffer.

Incidentally, make your block size 512 bytes.  Why?  Well, when you read data with the SD library it'll read into it's own internal cache buffer and then copy that data to your passed in buffer.  That's an extra memcpy step that you don't really need.  If you make your buffer exactly 512 bytes, it'll bypass the internal cache, and write data directly from the card to your buffer.

512 isn't the most efficient for the TCP side of things because you have a max of about 1400 bytes in a packet, but it's not that much slower.  If you want something really efficient, create two contiguous 512 byte buffers and then send them both.  But that's a lot more overhead, so I wouldn't bother.

Incidentally, The W5100 has a 2k write buffer just sitting there not being used.  Seems stupid that you have to allocate a 512 byte buffer in RAM to get around this issue when the library could handle all this for you without using any RAM.  That's exactly what happens with the UDP library.  Unfortunatly, the powers that be have decided that this is not a high priority...
15  Using Arduino / Networking, Protocols, and Devices / Re: W5100/Ethernet cannot use socket as UDP after using it as TCP on: April 26, 2013, 11:53:22 am
Dude.  Thanks a ton.  I appreciate immensely what you're doing here.  Unfortunately that's not exactly the same use case.  You only refreshed the browser once.  Notice that in my example after the first time I refreshed I still got responses back from the NTP, but after the second time I refreshed, the NTP stopped. Seems odd, but here's my explanation:

It seems that it's actually related to transition from UDP to a listening socket->connected transition, to being used for UDP again.  When you refresh the first time, you're using socket 0 (which has never been used for UDP) to do the TCP, and the listening port goes to socket 1.  The UDP time server then happens on socket 0 which hasn't been used for UDP *before* TCP.  If you wait about 10 seconds and then refresh, it'll use socket 1 to do the TCP conn, and then listen back on socket 0.  At that point the UDP will take place on socket 1 (which has done the UDP to TCP transition) and it will fail.

Ugh.  this is too complicated a scenario here.  Sorry sorry sorry.  I'm wondering if you can do 1 more test:
1.  Start the code.
2.  wait about 15 seconds.
3.  refresh your browser.
4.  wait about 15 seconds.
5.  refresh your browser again.

Incidentally, I tried to make a simpler scenario by writing some code that connects to itself, but apparently the W5100 won't connect to itself.
Pages: [1] 2