WAN only loads partial web page from Ethernet shield

Hi,

I am running a Arduino Mega 2560 with Ethernet shield (Wiznet chip).

I am using the following code : I have removed some of the HTML to make the posting more concise.

When I access the Arduino from the browser on the LAN via 192.168.1.50 the whole page always loads, very fast. No issues.

When I access the Arduino from the browser on the WAN, ie outside my home, only half the web page loads and the page is not functional (as the page load nearly always fails or loads fully so slowly as to be ineffective).

I am using port forwarding on the modem router and do not believe the modem router/ gateway is the issue because I have other services successfully port forwarded to the outside world.

I think the issue may be between the two rows of forward slashes in the code below, though not being a c programmer I am not sure.

Anyone any ideas ?

:slight_smile:

Jasz

#include <SPI.h>
#include <Ethernet.h>
#include <RCSwitch.h>

RCSwitch mySwitch = RCSwitch();

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };   //physical mac address
byte ip[] = { 192, 168, 1, 50 };                       // ip in lan (that's what you need to use in your browser
byte gateway[] = { 192, 168, 1, 1 };                   // internet access via router
byte subnet[] = { 255, 255, 255, 0 };                  //subnet mask
EthernetServer server(80);                             //server port     
String readString;

const int Pin_9 = 9;

   
void setup() {

  Ethernet.begin(mac, ip, subnet, gateway);
  server.begin();

  pinMode(Pin_9, OUTPUT); 
  digitalWrite(Pin_9, LOW);
  
  mySwitch.enableTransmit(10);
  mySwitch.setPulseLength(87);  
  mySwitch.setRepeatTransmit(96);
  
}


void loop() {


//////////////////////////////////////////////////

  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {   
      if (client.available()) {
     
       char c = client.read();
 
     if (readString.length() < 100) {
 
         readString += c;
        Serial.print(c);
    
     }
 
        if (c == '\n') { 
   
//////////////////////////////////////////////////
      
          client.println("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">");
          client.println("<html xmlns=\"http://www.w3.org/1999/xhtml\">");
 
           
//         client.println("HTTP/1.1 200 OK");
//         client.println("Content-Type: text/html");
           client.println("<html>");
           client.println("<head>");
           client.println("<meta name=\"viewport\" content=\"width=device-width\">");
           client.println("<title></title>");         
           client.println("</head>");
           client.println("<body>");
           client.println("<center>");
           client.println("
");  
           client.println("
"); 
           client.println("</center>");
           client.println("</body>");
           client.println("</html>");

          
           delay(1);
           //stopping client
           client.stop();
           //controls the Arduino if you press the buttons


           //clearing string for next read
            readString="";  
           
           
   //       client.println("<meta http-equiv=\"refresh\">");
          
          
                      
         }
       }
    }
}
}

Try combining some of those client.println calls. Maybe too many packets?

Thanks SurferTim,

I will begin combining these client.println() statements. There are an awful lot of them though (lots HTML).

  1. How many packets is too many and how do I assess text length content of a client.println(" ... html here .. "); statement in it's contribution to packet size?

  2. Is there any advantage in moving the html to a SD card. Would this reduce the requirement to consider packet size?

  3. Any idea why the page is fully server up, and very swiftly, on the LAN, and not successfully on the WAN?

Thank you

Jasz

The packet length is about 1400 bytes maximum. I try to send at least 32 bytes per packet, but that is not always possible.

Is there any problem with stopping the client immediately (or within a couple milliseconds) after sending data? Does the Ethernet library flush the output (ie, wait for it to be sent and received by the other side) before closing the connection?

A LAN is much faster (lower latency) and generally has lower rates of packet loss than a WAN, so that tiny millisecond may be long enough on a LAN but not long enough on a WAN.

Below is some test code you might try to see if it works from outside your network (with appropriate port forwarding on your router).

// zoomkat's meta refresh data frame test page 5/25/13
// use http://192.168.1.102:84 in your brouser for main page
// http://192.168.1.102:84/data static data page
// http://192.168.1.102:84/datastart meta refresh data page
// for use with W5100 based ethernet shields
// set the refresh rate to 0 for fastest update
// use STOP for single data updates

#include <SPI.h>
#include <Ethernet.h>

const int analogInPin0 = A0;
const int analogInPin1 = A1;
const int analogInPin2 = A2;
const int analogInPin3 = A3;
const int analogInPin4 = A4;
const int analogInPin5 = A5;

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address
byte ip[] = { 192, 168, 1, 102 }; // arduino ip in lan
byte gateway[] = { 192, 168, 1, 1 }; // internet access via router
byte subnet[] = { 255, 255, 255, 0 }; //subnet mask
EthernetServer server(84); //server port
unsigned long int x=0; //set refresh counter to 0
String readString; 

//////////////////////

void setup(){
  Serial.begin(9600);
    // disable SD SPI if memory card in the uSD slot
  pinMode(4,OUTPUT);
  digitalWrite(4,HIGH);

  Ethernet.begin(mac, ip, gateway, gateway, subnet);
  server.begin();
  Serial.println("meta refresh data frame test 5/25/13"); // so I can keep track of what is loaded
}

void loop(){
  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
         if (readString.length() < 100) {
          readString += c; 
         } 
        //check if HTTP request has ended
        if (c == '\n') {

          //check get atring received
          Serial.println(readString);

          //output HTML data header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println();

          //generate data page
          if(readString.indexOf("data") >0) {  //checks for "data" page
            x=x+1; //page upload counter
            client.print("<HTML><HEAD>");
            //meta-refresh page every 1 seconds if "datastart" page
            if(readString.indexOf("datastart") >0) client.print("<meta http-equiv='refresh' content='1'>"); 
            //meta-refresh 0 for fast data
            if(readString.indexOf("datafast") >0) client.print("<meta http-equiv='refresh' content='0'>"); 
            client.print("<title>Zoomkat's meta-refresh test</title></head><BODY>
");
            client.print("page refresh number: ");
            client.print(x); //current refresh count
            client.print("

");
            
              //output the value of each analog input pin
            client.print("analog input0 is: ");
            client.print(analogRead(analogInPin0));
            
            client.print("
analog input1 is: ");
            client.print(analogRead(analogInPin1));
                        
            client.print("
analog input2 is: ");
            client.print(analogRead(analogInPin2));
            
            client.print("
analog input3 is: ");
            client.print(analogRead(analogInPin3));
                                    
            client.print("
analog input4 is: ");
            client.print(analogRead(analogInPin4));
            
            client.print("
analog input5 is: ");
            client.print(analogRead(analogInPin5));
            client.println("
</BODY></HTML>");
           }
          //generate main page with iframe
          else
          {
            client.print("<HTML><HEAD><TITLE>Zoomkat's frame refresh test</TITLE></HEAD>");
            client.print("Zoomkat's Arduino frame meta refresh test 5/25/13");
            client.print("

Arduino analog input data frame:
");
            client.print("&nbsp;&nbsp;<a href='http://192.168.1.102:84/datastart' target='DataBox' title=''yy''>META-REFRESH</a>");
            client.print("&nbsp;&nbsp;&nbsp;&nbsp;<a href='http://192.168.1.102:84/data' target='DataBox' title=''xx''>SINGLE-STOP</a>");
            client.print("&nbsp;&nbsp;&nbsp;&nbsp;<a href='http://192.168.1.102:84/datafast' target='DataBox' title=''zz''>FAST-DATA</a>
");
            client.print("<iframe src='http://192.168.1.102:84/data' width='350' height='250' name='DataBox'>");
            client.print("</iframe>
</HTML>");
          }
          delay(1);
          //stopping client
          client.stop();
          //clearing string for next read
          readString="";
        }
      }
    }
  }
}

christop: Is there any problem with stopping the client immediately (or within a couple milliseconds) after sending data? Does the Ethernet library flush the output (ie, wait for it to be sent and received by the other side) before closing the connection?

If you are talking about the server code, there is no problem calling client.stop() immediately after sending the last packet. The close will not be sent until the last packet sent prior to the client.stop() is received and read by the client.

christop: A LAN is much faster (lower latency) and generally has lower rates of packet loss than a WAN, so that tiny millisecond may be long enough on a LAN but not long enough on a WAN.

That tiny millisecond delay doesn't need to be there. The server code will not send a close until it gets an ack for the last packet.

Hi Guys, Thank you for the many and valued reponses. They solved the problem.

Hi SurferTim : I combined as many of the

client.println(".. HTML here..");

statements as I could and the page now loads with no issues from the WAN side. I have approximately halved my client.println html statements this seems to have reduced the number of packets required of the arduino. Thank you :)

Hi Christop :

"Is there any problem with stopping the client immediately (or within a couple milliseconds) after sending data? Does the Ethernet library flush the output (ie, wait for it to be sent and received by the other side) before closing the connection?"

I do not know how to stop the client browser within a couple of milliseconds after the data call to the arduino server is made. I guess I would need to be able to write some code to do that and I am no coding expert. In fact my coding skils are basic to say the least. So I do not know if the arduino server flushes the output (ie, waits for the server side data to be sent and received by the browser) before closing the connection? The page loads about half way on the browser and then hangs. Sometimes it eventually completes. Reducing the number of client.println html statements by half solved the browser page load issue for some reason I can't say I really understand. Thank you :)

Hi Zoomkat : Thank you for your

"meta refresh data frame test page 5/25/13"

I tried this and got superb results on the LAN (obviously). On the LAN when clicking on "FAST" I got 155 page refreshes in 30 seconds. I tried this using my laptop connected to my phones mobile data network (HDPSA?) and could not get one page re-fresh. I am eager to try this from someone elses home computer(broadband) ie WAN senario. The good news is that on my slow mobile data connection my arduino web page with half the number of client.println html statements / consolidated client.println html statements now loads very fast.

:) :)