HTML/CSS/Java Webpage w Arduino Mega & Wifi Shield

Hello,

I have put together a really cool project that can turn on a LED strip and control a stepper motor (to open and close my blinds) from a webpage.

My current web code is wrapped in a client.print() function. This was very tedious and will continue to be since I am now looking to add a web stream to the page.

My question is, "Is there an easier way to create a webpage hosted on an Arduino?"

A snipit of my existing code is below:

client.print("<!DOCTYPE html>");
            client.print("<html lang=\"en\">");
            client.print("<HTML><body>");

            client.print("<head>");
            client.print("<meta charset=\"utf-8\">");
            client.print("<title>James Hayek</title>");
            client.print("<meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">");

            client.print("<link rel=\"stylesheet\" href=\"//googledrive.com/host/0BxNGritBJp1zUHJROUVPa1F6MkE\">");  
            client.print("</head>");
            client.print("<body class=\"php\">");
            client.print("<div id=\"container\">");
            client.print("  <div id=\"main\" role=\"main\" class=\"hellobox\">");
            client.print("    <header><a href=\"http://jameshayek.com\"><b>JamesHayek.com</b></a></header>");
            client.print("    <h1>James Hayek</h1>");
            client.print("    <h2>Arduino web hosted RGB light and vertical blind controls</h2>");
            client.print("  </div>");
            client.print("<nav><ul>");
            client.print("<table  CELLPADDING=\"4\" CELLSPACING=\"3\" ALIGN=\"CENTER\">");

            // the content of the HTTP response follows the header:
            client.print("<TH COLSPAN=\"2\">");
            
            client.print("<tr>");
            client.print("<td><FONT size = +2><a href=\"/L\"> Lights: On </a></FONT></td>");
            client.print("<td><FONT size = +2><a href=\"/K\"> Blinds: Open </a></FONT></td>");
            client.print("<td><FONT size = +2><a href=\"/50\"> Lights: 50% </a></FONT></td>");
            client.print("<td><FONT size = +2><a href=\"/H\"> Scanner</a></FONT></td>");
            client.print("<td><FONT size = +2><a href=\"/LS\">Test Scanner</a></FONT></td>");
            client.print("</tr>");

You can save arduino memory by using the F() macro like below for the static lines of html text.

            client.print(F("<HTML><HEAD><TITLE>Zoomkat's frame refresh test</TITLE></HEAD>"
            "Zoomkat's Arduino frame meta refresh test 8/17/13"
            "

Arduino analog input data frame:
"
            "&nbsp;&nbsp;<a href='/datastart' target='DataBox' title=''yy''>META-REFRESH</a>"
            "&nbsp;&nbsp;&nbsp;&nbsp;<a href='/data' target='DataBox' title=''xx''>SINGLE-STOP</a>"
            "&nbsp;&nbsp;&nbsp;&nbsp;<a href='/datafast' target='DataBox' title=''zz''>FAST-DATA</a>
"
            "<iframe src='/data' width='350' height='250' name='DataBox'>"
            "</iframe>
</HTML>"));

If you are using external resources, make the most of it. ( googledrive / web hosting )

The Arduino only need send a few lines at most if you can link in a javascript, or iframe resource.

Javascript can control the whole site. Interaction after the initial html page only needs to be ajax requests for data. All the visual elements can be externally loaded.

As for the tediousness of writing the code in client.print calls, C++ allows a few strange things, take this for instance:

client.print(
    "This is line 1 "
    "This is line 2 "
    "This is line 3 "
    "however, they are all actually "
    "on the same line"
);

Its still produces a single line, but as its HTML, it does not matter. The code possibly looks neater though.

Just remember for plain text, you'll need to add a space to the end of each string or at the start of a new string, Otherwisetheresultcouldlooklikethis.

zoomkat How many lines can I feed the F() macro before it no longer works works? As an example, I have t break up one line of HTML code into several ones when using the setup I have with client.print() functions.

pYro_65 I will try the C++ code as you posed, but again, last time I used multiple lines I had to break it up into smaller chunks.

Is there a way to host an Index.html file on the SD card and just call that for serving the webpage? (Similar to a real web server) This way I can write code in C++ for the Arduino and store an HTML file (Along with CSS/Java files)

Is there a way to host an Index.html file on the SD card and just call that for serving the webpage? (Similar to a real web server) This way I can write code in C++ for the Arduino and store an HTML file (Along with CSS/Java files)

Below is some servo slider test code that is served from an SD card.

//zoomkat 2/26/13
//SD server slider test code
//open serial monitor to see what the arduino receives
//browser address will look like http://192.168.1.102:84/servosld.htm when submited
//for use with W5100 based ethernet shields
//put the servosld.htm, slider.js, bluev_sl.gif,
//and bluev_bg.gif on the SD card
//download flies at:
// http://web.comporium.net/~shb/pix/servosld.htm
// http://web.comporium.net/~shb/pix/slider.js
// http://web.comporium.net/~shb/pix/bluev_bg.gif
// http://web.comporium.net/~shb/pix/bluev_sl.gif
// 

#include <SD.h>
#include <SPI.h>
#include <Ethernet.h>

#include <Servo.h> 
Servo myservoa, myservob, myservoc, myservod;  // create servo object to control a servo 
Servo myservoe, myservof, myservog; // myservoh not used due to lack of another free pin
String readString, pos;

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address
byte ip[] = { 192, 168, 1, 102 }; // ip in lan
byte gateway[] = { 192, 168, 1, 1 }; // internet access via router
byte subnet[] = { 255, 255, 255, 0 }; //subnet mask
EthernetServer server(84); //server port

//////////////////////

void setup(){

  Serial.begin(9600);

  // disable w5100 while setting up SD
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);
  Serial.print("Starting SD..");
  if(!SD.begin(4)) Serial.println("failed");
  else Serial.println("ok");

  Ethernet.begin(mac, ip, gateway, gateway, subnet);

  //delay(2000);
  server.begin();
  Serial.println("Ready");

  myservoa.attach(2);  //the pin for the servoa control
  myservob.attach(3);  //the pin for the servob control
  myservoc.attach(5);  //the pin for the servoc control
  myservod.attach(6);  //the pin for the servod control 
  myservoe.attach(7);  //the pin for the servoa control
  myservof.attach(8);  //the pin for the servob control
  myservog.attach(9);  //the pin for the servoc control
  //myservoh.attach(10);  //the pin for the servod control 

}

void loop(){
  // Create a client connection
  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();

        //read char by char HTTP request
        if (readString.length() < 100) {
          //store characters to string 
          readString += c; 
          //Serial.print(c);
        } 
        //if HTTP request has ended
        if (c == '\n') {

          ///////////////
          Serial.println(readString); //print to serial monitor for debuging

            //select proper header for file to be sent to browser

          client.println("HTTP/1.1 200 OK"); //send new page
          if(readString.indexOf("servosld") >=0) {
            client.println("Content-Type: text/html");
            client.println(); 
          }

          if(readString.indexOf("slider") >=0) {
            client.println("Content-Type: application/x-javascript");
            client.println(); 
          }

          if(readString.indexOf("bluev") >=0) {
            client.println("Content-Type: image/gif");
            client.println(); 
          }

          //select file to send to browser
          if(readString.indexOf("servosld") >=0) {
            File myFile = SD.open("SERVOSLD.HTM");
            if (myFile) {

              byte clientBuf[64];
              int clientCount = 0;              

              while (myFile.available()) 
              {
                clientBuf[clientCount] = myFile.read();
                clientCount++;

                if(clientCount > 63)
                {
                  client.write(clientBuf,64);
                  clientCount = 0;
                }                
              }
              if(clientCount > 0) client.write(clientBuf,clientCount);            
              myFile.close();
            }
          }

          if(readString.indexOf("slider") >=0) {
            File myFile = SD.open("slider.js");
            if (myFile) {

              byte clientBuf[64];
              int clientCount = 0;              

              while (myFile.available()) 
              {
                clientBuf[clientCount] = myFile.read();
                clientCount++;

                if(clientCount > 63)
                {
                  client.write(clientBuf,64);
                  clientCount = 0;
                }                
              }
              if(clientCount > 0) client.write(clientBuf,clientCount); 
              myFile.close();
            }
          }

          if(readString.indexOf("bluev_sl") >=0) {
            File myFile = SD.open("bluev_sl.gif");
            if (myFile) {

              byte clientBuf[64];
              int clientCount = 0;              

              while (myFile.available()) 
              {
                clientBuf[clientCount] = myFile.read();
                clientCount++;

                if(clientCount > 63)
                {
                  client.write(clientBuf,64);
                  clientCount = 0;
                }                
              }
              if(clientCount > 0) client.write(clientBuf,clientCount); 
              myFile.close();
            }
          }

          if(readString.indexOf("bluev_bg") >=0) {
            File myFile = SD.open("bluev_bg.gif");
            if (myFile) {

              byte clientBuf[64];
              int clientCount = 0;              

              while (myFile.available()) 
              {
                clientBuf[clientCount] = myFile.read();
                clientCount++;

                if(clientCount > 63)
                {
                  client.write(clientBuf,64);
                  clientCount = 0;
                }                
              }
              if(clientCount > 0) client.write(clientBuf,clientCount); 
              myFile.close();
            }
          }

          delay(1);
          //stopping client
          client.stop();

          //process GET string request from client and and position servo

          pos = readString.substring(8, 12); //get the first four characters         
          //Serial.println(pos);
          int n = pos.toInt();  //convert readString into a number   
          Serial.println(n); 
          Serial.println();

          if(readString.indexOf("?0") >0) myservoa.writeMicroseconds(n);
          if(readString.indexOf("?1") >0) myservob.writeMicroseconds(n);
          if(readString.indexOf("?2") >0) myservoc.writeMicroseconds(n);
          if(readString.indexOf("?3") >0) myservod.writeMicroseconds(n);
          if(readString.indexOf("?4") >0) myservoe.writeMicroseconds(n);
          if(readString.indexOf("?5") >0) myservof.writeMicroseconds(n);
          if(readString.indexOf("?6") >0) myservog.writeMicroseconds(n);
          //only seven servo pins, so back to myservoa for testing
          if(readString.indexOf("?7") >0) myservoa.writeMicroseconds(n);

          //clearing string for next read
          readString="";
          pos="";
        }
      }
    }
  } 
}

You can adapt my SD based server code to a wifi shield also. Like zoomkat's code above is written for an ethernet shield, but easily adapted to the wifi library.
http://playground.arduino.cc/Code/WebServerST

If you request the default page (just the ip address), it will upload index.htm to the client.

However, the wifi shield's firmware has some bugs with the server section. If more than one client tries to connect and download files, there is a probability that the files downloaded will be incorrect or corrupted.

Thanks for the help, I found more information and now have this coding:

/*
  WiFi Web Server Hosted on SD card
 
  
 created 13 July 2010
 by dlf (Metodo2 srl)
 modified 31 May 2012
 by Tom Igoe, James Hayek
 
 */

#include <SPI.h>
#include <WiFi.h>
#include <SD.h>



File webFile;

char ssid[] = "EnGeniusE27B20";      // your network SSID (name) 
//char pass[] = "secretPassword";   // your network password
int keyIndex = 0;                 // your network key Index number (needed only for WEP)

int status = WL_IDLE_STATUS;

WiFiServer server(81);

void setup() {
  //Initialize serial and wait for port to open:
  Serial.begin(9600); 
  while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }

  // check for the presence of the shield:
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial.println("WiFi shield not present"); 
    // don't continue:
    while(true);
  } 

  // attempt to connect to Wifi network:
  while ( status != WL_CONNECTED) { 
    Serial.print("Attempting to connect to SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network. Change this line if using open or WEP network:    
    status = WiFi.begin(ssid);

    // wait 10 seconds for connection:
    delay(10000);
  } 
  server.begin();
  // you're connected now, so print out the status:
  printWifiStatus();


  // initialize SD card
  Serial.println("Initializing SD card...");
  if (!SD.begin(4)) {
    Serial.println("ERROR - SD card initialization failed!");
    return;    // init failed
  }
  Serial.println("SUCCESS - SD card initialized.");
  // check for index.htm file
  if (!SD.exists("index.htm")) {
    Serial.println("ERROR - Can't find index.htm file!");
    return;  // can't find index file
  }
  Serial.println("SUCCESS - Found index.htm file.");
}


void loop() {
  // listen for incoming clients
  WiFiClient client = server.available();
  if (client) {
    Serial.println("new client");
    // an http request ends with a blank line
    boolean currentLineIsBlank = true;
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        Serial.write(c);
        // if you've gotten to the end of the line (received a newline
        // character) and the line is blank, the http request has ended,
        // so you can send a reply
        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connection: close");
          client.println(); 
          // send web page
          webFile = SD.open("index.htm");        // open web page file
          if (webFile) {
            while(webFile.available()) {
              client.write(webFile.read()); // send web page to client
            }
            webFile.close();       
          }           
          break;
        }


           // every line of text received from the client ends with \r\n

        if (c == '\n') {
          // you're starting a new line
          currentLineIsBlank = true;
        } 
        else if (c != '\r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }
    }
    // give the web browser time to receive the data
    delay(1);

    // close the connection:
    client.stop();
    Serial.println("client disonnected");
  }
}


void printWifiStatus() {
  // print the SSID of the network you're attached to:
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // print your WiFi shield's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);

  // print the received signal strength:
  long rssi = WiFi.RSSI();
  Serial.print("signal strength (RSSI):");
  Serial.print(rssi);
  Serial.println(" dBm");
}

A webpage I host on the SD card is found and it serves a simple text page. Only one problem, it takes forever to load if it loads, is there any reason for this?

As a follow up:

Below is my HTML

<!DOCTYPE html>
<html>

	<head>
			<title>James Hayek's Title</title>
			<link rel="stylesheet" type="text/css" href="main.css"/> 
<!--			<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script> -->
			
			<header>
			<headRunner>
				<h1><p align="center">Welcome!</p></h1>


				<p align="center">Here you can find information about myself including academics, 
				art & design, personal projects and EE endeavours.
				</p>
			</headRunner>
			</header>
			<divStyle class="btn2">Show Header</divStyle>
			<divStyle class="btn1">Hide Header</divStyle>
	</head>
		
	<body id="background">
			<div class="buttons" id="resume">
Resume</div>
			<div class="buttons"id="portfolio">
Portfolio</div> 
			<div class="buttons" id="curricular">
<a href="http://www.youtube.com/jameshayek"> Youtube Channel </a></div> 
			<div class="buttons" id="technical">
Technical Work</div> 
			<div align="center">
			


			
			<div id="information">	
				


				


				


				


				<h2><p align="center">I am currently pursuing a degree in Electrical Engineering at NJIT. 
				Here you can find updates on projects in and outside of school. My Youtube channel showcases
				most of my personal projects and my Google+ page documents my personal life. Feel free to use the links
				above to learn a little more about me. Feel free to email me with any questions on any projects posted on my site.</p></h2>


			</div>	
			
			
			

	
			</div>
			
			<div id="mainGroup">
			<div class='main'>
				<div class='activator'>
					<div class="img"><img src="https://lh5.googleusercontent.com/-Q1-A_8ogsHM/Ub58o0etPvI/AAAAAAAAAC4/_-3VmOcPVlQ/s763-no/madScientist.jpg" height="168" width="168"></img></div>  
					<div class="content">My goal is to obtain an excellent education while aiming to obtain a fulfilling career to utilize my knowledge in Mathematics, Robotics, Science and Engineering.
					Aside from working hard in school, I also work hard physically. Running three to four miles three times a week, I look to 
					maintain a healthy lifestyle so I can continue to strive for mental and physical Zen.</div>
				</div>
			</div>
			</div>
			
			
			
			







<!--
			
			<script>	
				$(document).ready(function(){

				$(".btn1").click(function(){
					$("headRunner").slideUp();
				});
				});

				$(document).ready(function(){

				$(".btn2").click(function(){
					$("headRunner").slideDown();
				});
				});
				
				$(document).ready(function() {

				$("#resume").mouseover(function() {
    
					$("#resume").fadeTo("fast",1);   
    
				});

				$("#resume").mouseleave(function() {
				
					$("#resume").fadeTo("fast", 0.25);
				
				});
				
				
				
				$("#portfolio").mouseover(function() {
    
					$("#portfolio").fadeTo("fast",1);   
    
				});

				$("#portfolio").mouseleave(function() {
				
					$("#portfolio").fadeTo("fast", 0.25);
				
				});				
				
				
				
			
				$("#curricular").mouseover(function() {
    
					$("#curricular").fadeTo("fast",1);   
    
				});

				$("#curricular").mouseleave(function() {
				
					$("#curricular").fadeTo("fast", 0.25);
				
				});
				
				
			

				$("#technical").mouseover(function() {
    
					$("#technical").fadeTo("fast",1);   
    
				});

				$("#technical").mouseleave(function() {
				
					$("#technical").fadeTo("fast", 0.25);
				
				});
				
				
				$('.buttons').hover(function() {
				$(this).animate({
					height: '140px'
					}, 300); 
					}, 
					
					function() {
				$(this).animate({
					height: '80px'
					}, 300);
				});
				
				
				
				
				
				
				
				
				
				
				$('.main').hover(function() {
				$(this).animate({
				width: '500px'
				}, 300);
				}, function() {
				$(this).animate({
				width: '200px'
				}, 300);
				});






				
				
				});


			</script> -->

			
	</body>

</html>

and here is the CSS

headRunner {
    position: absolute;
    margin: -10px 0 0 -8px;
    height: 140px;
    background-color: #8F9E8B;
    width: 100%;
    color: #FFFFFF;
    font-family: Verdana, Arial, Sans-Serif;
	z-index: -10;
	clear:both;	
}

.btn1, .btn2 {
	float: right;
	padding: 10px;
	width:6em;
	color: white;
	text-align:center;
	width: 100%
	border-radius: 50px;
	background: #5B7876;
}

.btn1:hover, .btn2:hover {
	background: #000000;
}


divStyle {
	width: 500px;
	background-color: blue;
	border-radius: 5px;

}

.buttons {
    height: 80px;
		-moz-border-radius: 40px;
		-webkit-border-radius: 40px;
    width: 80px;

    border-radius: 2000px;
	border: 1px solid #292929;
    background-color: #292929;
    text-align: center;
    color: #FFFFFF;
    font-family: Verdana, Arial, Sans-Serif;
    opacity: 0.25;
	float: left;
	margin-right:1em;
	font-size: small;
}

#information {

	float:center;
}

#background {
	background-color: #292929;
}

p {
	color:white;
}




.main {
    height: 200px;
    width: 200px;
    border: 1px solid white;
    overflow: hidden;
}
.main .activator {
    height: 200px;
    width: 200px;
    float: left;
    display: block;
}
.img {
    border: 1px solid gray;
    text-align: center;
    margin: 15px;
    height: 168px;
    width: 168px;
    float: left;
}
.content {
    display: none;
    margin: 15px;
	color:white;
}
.activator:hover {
    width: 500px;
}
.activator:hover .content {
    display: block;
}
.activator:hover .img {
    float: left;
}

#mainGroup {
	clear:both;
}

Is there a reason why it loads so slow? In fact, in most incidents it didn't load at all. I had to press escape and sometimes it serves the information it pulled from the SD card.

The wifi shield firmware has some pretty serious bugs. I reported them several months ago, and I have seen no action taken on any of them.

The major problem is the wifi shield server firmware uses only one socket, and uses it poorly. It apparently doesn't stop listening for requests when it gets one, so if another request from the same or another client arrives, the requests can become corrupted, returning corrupted or incorrect files to the client(s).

It also has a weird delay of up to a couple seconds every few seconds in the upload to a client if the file is large.

So you're telling me SurferTim that my Arduino sketch, and HTML and CSS code is clean and not generating errors?

To everyone, Would an Xbee and SD shield be a better setup?

zoomkat or pYro_65, have you ever had any issues serving a page via the Wifi shield and SD reader? (Slow page loads, hangs, freezes, etc)

My original code loaded quite fine. It was just cumbersome to program a nice looking webpage. Would there be any reason the code in my original post allowed the server to send the data fine, but not the data pulled from the SD card?

I noticed even after the page loads, I still have the "working/busy" graphic rotating in my browser tab. Right now the webpage has nothing but text. jameshayek.com is the site, I have it redirecting to the IP of the Arduino.

One thing you can do to speed up loading from SD:

          if (webFile) {
            while(webFile.available()) {
              client.write(webFile.read()); // send web page to client
            }
            webFile.close();       
          }

You read one character at a time from the file, and send that one character in a packet to the client. That packet is up to 512 bytes long, and includes room for a LOT more payload. Wasting time sending thousands of packets that have to be acknowledged or resent, and reassembled in the correct order, is NOT the way to manage fast page loading.

Read an array of characters from the file. The method that does that returns the number of bytes read. Use that value in the call the client.write() to write an array of bytes, where you tell it the number of bytes in the array.

Your code may be fine, and the problem is caused by the wifi firmware. I use an ethernet shield connected to a router with wireless capability for remote access, but I have other devices connected to the ethernet ports on that router also, so that offsets the cost of the router a little.

You may want to check into other wifi devices, like the CC3000. I haven't used it, but others on the forum have.

PaulS is correct (again). Take a look at zoomkat's code using the SD card. It uses a 64 byte buffer to send packets about 4 times faster than single byte packets. It doesn't do away with the 2 second delays tho, just fewer of them.

edit: The router solution also provides other advantages. The radio I use on the router is a 24db radio with a 12db antenna, giving that unit a range capability not possible with a standard client radio/antenna. At a minimum (with line-of-sight) a few miles.

If I need longer range here in the U.S., I can use a-mode (5.8GHz), and within certain limitations, I can transmit at 30db (1000mw) with no limit on the antenna gain, as long as the antenna has a horizontal pattern less than 35 degrees. This gives me a line-of-sight range of several miles.

As surfertim said, he had problems with the Wifi, which I saw myself when accessing his Arduino.

Try buffer the output like PaulS mentioned so no tiny packets are sent.

Here is a simple class I wrote to do just this: Ethernet shield server test (new code Uno ok) - #71 by pYro_65 - Networking, Protocols, and Devices - Arduino Forum

Used with a large buffer you may see a great improvement in reliability, as multiple lines can end up being sent in one packet.
As it buffers the data until full you need to call buffer.flush(); when you have finished sending the text.