EthernetShield + Webserver on a NAS in my LAN

Hey there!

I ran into a weird problem:

I played around with arduino, the etherned shield a a lcd module. All seems all to be wired correctly - at least everythings works fine in some cases. I followed the EthernetClient example that came with the Ethernet class to have the arduino connect to a webserver. Everything seems to work fine as long as the webserver that I'm trying to reach is outside of my home network. But connecting to the webserver that is running on my Buffalo Linkstation NAS always fails. The code I use:

#include <LiquidCrystal.h>
#include <Ethernet.h>
#include <SPI.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xDF, 0xAE, 0xED };  
LiquidCrystal lcd(7, 6, 2, 3, 4, 5);
byte server[] = { 192, 168, 1, 125 }; // Webserver on my NAS running on port 81
EthernetClient client;

void setup() {
  lcd.begin(20, 4);
  Ethernet.begin(mac);
  lcd.print("connecting ...");
  delay(1000); 
  lcd.setCursor(0,1);
  if (client.connect(server, 81)) {
    lcd.print("connected");
    client.println("GET /tools/lcd.php HTTP/1.1");
    client.println();
  } else {
    lcd.print("connection failed");
  }
}

void loop() {
  if (client.available()) {
    char c = client.read();
    lcd.setCursor(0,2);
    lcd.print("SUCCESS");
    for(;;)
      ;
  }

  if (!client.connected()) {
    lcd.setCursor(0,2);
    lcd.print("disconnecting.");
    client.stop();
    for(;;)
      ;
  }
}

What happens is: Both if-statements in the main loop() are false at first - after some time, the second one becomes true. So the client seems to connect successfully but never becomes "available" - whatever that is supposed to mean ... As soon as I change the server to 74.125.132.94 (which is www.google.de) everything works as expected.

Has anyone an idea on this? I'm kinda stuck ...

Thanks and regards
Jürgen

Have you tried accessing your NAS manually (without a browser)?

This is done by the telnet command. Open a command line and enter:

telnet 192.168.1.125 81

When connected enter the following string:

GET /tools/lcd.php HTTP/1.0

Take care to hit the return key twice after that first line. If you get any response, you should change the protocol version to 1.0 too in your code. If you don't get a response, the problem lies in your NAS being a bit picky about how a request is sent. You tell your server that you're using HTTP 1.1 but effectively your sending the simplest form of a version 1.0 request.

Hey, thanks for the reply!

Seems like there's something wrong with the webserver's configuration?!?

If I telnet onto the NAS width HTTP/1.0 I receive the following response:

telnet 192.168.1.125 81
Trying 192.168.1.125...
Connected to 192.168.1.125.
Escape character is '^]'.
GET /tools/lcd.php HTTP/1.0

HTTP/1.0 200 OK
X-Powered-By: PHP/5.2.10-pl0-gentoo
Content-type: text/html
Content-Length: 39
Connection: close
Date: Mon, 16 Jul 2012 13:21:19 GMT
Server: lighttpd/1.4.23

While telnetting with HTTP/1.1 gives the following result:

telnet 192.168.1.125 81
Trying 192.168.1.125...
Connected to 192.168.1.125.
Escape character is '^]'.
GET /tools/lcd.php HTTP/1.1

HTTP/1.1 400 Bad Request
Content-Type: text/html
Content-Length: 349
Connection: close
Date: Mon, 16 Jul 2012 13:22:12 GMT
Server: lighttpd/1.4.23

But switching from HTTP/1.1 to HTTP/1.0 in my Arduino script has no effect - still the client never becomes "available".

Do I have to tell my NAS to serve the webpage in HTTP/1.0-mode? How would I do that?

Thanks for your help so far - it seems like we're getting closer to the problem!

Do I have to tell my NAS to serve the webpage in HTTP/1.0-mode? How would I do that?

You do that by saying so on the request line. Your server is correct (although a bit picky) in not accepting the single line request when asked for the HTTP/1.1 protocol. Just leave the HTTP/1.0 in your code and that part should be fine.

client.available() does not mean that the client is available but that the client has received some characters from the server (it returns the number of characters available for you to read).

Do you get a connected on the LCD?

Yes - on the LCD I get (I changed the request to HTTP/1.0):

connecting ...
connected

and after a while:

connecting ...
connected
disconnecting.

I'd start sniffing on the network. Do you have the possibility to sniff on that network (using a tool like WireShark)? You have to get the difference between the telnet on your host and the request from the Arduino. Smart switches usually allow to define a monitoring port where you get all the traffic.

If you cannot do that, describe how your network is constructed (every device connected, like your PC, router, switch, NAS, etc.).

It seems like I can't get any useful output out of Wireshark - maybe because of the lack of an intelligent switch. Or maybe because this programm confuses me ...

All I could see was that my chrome browser uses "GET /tools/lcd.php HTTP/1.1" to reach the webserver on the NAS.

I my network there live: a router (Speedport W700), a 19"-switch (the manufacturer is sempre) an Arduino with ethernet shield, a windows pc, a mac mini, a RaspberryPi running Debian Squeeze, a small WindowsXP/Ubuntu dual boot machine, a netbook (WinXP), a laptop (WinVista), two iPhones, an iPad, an iPod Touch and a Xbox. Not to forget the Buffalo LinkStation NAS running a lighttpd/1.4.23 webserver. So it's a bit crowded here. What exactly do you need to know about the connected devices?

I didn't need all devices connected to your network, but all in the path Arduino - NAS - PC. So I'd guess, this is Arduino, switch, router and PC. Correct? Is the Arduino connected to the switch or the router? What about the NAS? And where is your PC connected? Who is distributing the IPs (DHCP)? The router? The NAS? What IP does your Arduino get?

The reason for asking this: I think your problem lies in the network and not in the programming of the Arduino. You would have got anything back from the NAS if the Arduino had reached it.

Thanks for your patience with me - so here are a few more details:

The Arduino is - like the PC - directly connected to the switch. The switch is connected to the router which provides DHCP for the whole network - but with a lot of exceptions: The NAS has a static ip, the pc has a static ip. On the Arduino side I tried both - static and DHCP - with no success. DNS is done by the router (which just passes through outside DNS information, I think), but that shouldn't be of any interest as we only deal with (static) ips.

The Arduino usually gets 192.168.1.11, the pc ...246, the NAS ...125. The router has 192.168.1.1 (just like DNS and gateway).

As I mentioned earlier: When using the ip of a google server (and changing the GET-request to something like "GET /search?q=test HTTP/1.1") everything seems to work fine.

Something for you to try: turn the Arduino off, set up the PC to have the IP 192.168.1.11 (the one the Arduino had) and access the NAS again by using the telnet command (and not using a browser!) described in one of the last posts. Does that work?

Because the Google page works I'm almost sure that it has to do with your network setup or a configuration issue on the NAS. It's quite special that your NAS is listening on the port 81 and not 80. Have you configured that or is this the default?

Good idea!

Telnetting from the .11 address works perfectly fine (when using HTTP/1.0 ... HTTP/1.1 results in a bad request).

The port 81 thing is due to the NAS' admin interface which has a second webserver listening on port 80 (a behaviour that cannot be changed as far as I can tell).

I had also thought of a port issue earlier, but changing the Arduino code to fetch the admin page on port 80 also doesn't lead to a positiv result. Is there any other piece of code that I could try to check if the Arduino and the NAS are capable of talking to each other? The NAS not only provides samba shares but also a MySQL database as well as a Twonky UPnP streaming server.

I'm almost out of ideas. I guess it's definitely time to sniff. Can you organise a hub to connect three devices (NAS, Arduino, PC) over a non-switching network device (or alternatively connect your PC to a monitoring port of your switch if that exists). You must have a look at what's going on the network. Is this possible? Maybe a friend can lend a hub to you?

Hmmm, I'll try to borrow a hub from someone ... or maybe I could just connect the arduino to my laptop's ethernet port and bridge the connection over to my wifi ...

Another funny thing I experienced earlier today:

//client.println("GET /tools/lcd.php?key=1257 HTTP/1.0");
client.println("HTTP/1.0");

This actually led to a "SUCCESS" on my lcd screen - which means (as far as I can tell): There's something wrong with the GET-request (and/or the server answering the request). The downside: I cannot reproduce this little success.

What I'm going to try now: Set up another webserver and see if I'm able to connect to that one.

Just a quick update: I installed a (Xampp-)Webserver on my PC - and everything works as expected ...

Gnaaaaaaaaaaa, seems like the webserver on my NAS is the problem - that's too bad because I have no access to its configuration files.

I GOT IT (at least I think so :wink: - it was just another case of "You have to know what to search for!". Or in other words: The lightttp server is (as you already discovered) a bit picky and needs to be fed on a request with manual line break/carriage return: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1268643262

So changing the request from

client.println("GET /tools/lcd.php?key=1257 HTTP/1.0");
client.println("");

to

client.print("GET /tools/lcd.php?key=1257 HTTP/1.0\r\n\r\n");

did the trick.

Thank you so much for your patience and help, pylon!

Both implementations should produce the exact same output to the server. The println() method adds a carriage return AND newline to the supplied string:

size_t Print::println(void)
{
  size_t n = print('\r');
  n += print('\n');
  return n;
}

So I don't see the difference...

What version of the IDE are you using?

I use 1.0.1 but the problem (as far as I understand it) is not the request but the server handling it. It seems like println() sends data and line break in seperate packets (whatever "packet" means in this case) whereas print("SOMETHINGSOMETHING\r\n") sends everything in just one packet. Both methods should be fine but the lightttp server seems to get hickups from requests spanning over several packets.

It seems like println() sends data and line break in seperate packets

This is indeed the case.

whereas print("SOMETHINGSOMETHING\r\n") sends everything in just one packet.

Also true.