Uno R3 + Ethernet shield = Connection Freeze

Hi!

Please excuse my bad english.

Last week I bought an Uno R3 along with the Ethernet shield, for my small home automation project.

My thought is that I will serve my pages along with some css from the SD card to the client. It's very little html and css so it shouldn't be that much problem doing it this way. I have managed to get the server running and send page from SD card to the client, with CSS, so thats fine. The problem is that I only can do one request to the server, directly after sketch have been uploaded, after that single request the server stops responding, well that's not exactly true I have pinged the server alot and it always responds, so it's better to say that it stops serving me pages.

I have also tried uploading the web server example sketch to the server, but it also works just for a few request, and that sketch prints to the client directly without loading any file from the sd card. So I'm starting to get really confused, and in need of a bit help.

Of course I've googled around a lot, and this problem seem's to be common, but I havn't found any solutions to my problem.

I can also add that my components is bought from sparkfun, so they should be genuine Arduino cards, in case that should matter.

This sketch downloads Google home page every 10 seconds.

How does that do?

edit: If you want to see how the w5100 and SD work together, take a look at my FTP client code. Maybe that will help. It uses 2 sockets on the w5100 and a file on the SD simultaneously.

Hi, it seems to work pretty good actually, no failures at all... I still don't understand why it doesn't work as a server.

You mean your code? Maybe if you posted it, we could take a look and help you out.

No I meant the get google code, my doesn’t work a bit. But here comes my code.

/*
 * Home Automation Project
 */

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>
#include <TextFinder.h>


byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192,168,1, 100);
IPAddress gateway(192,168,1,1);
IPAddress subnet(255,255,255,0);

int sdCS = 4; //SD cards chipselect
int sdSS = 10;//SD cards slave select

EthernetServer server(80);

const int inputLength = 16;             // length of the file requested
const int typeLength = 6;               // length of GET or POST
const long tempCheckInterval = 10000;   // time between checks (in ms)

char inputString[inputLength];          // for input from the browser
char requestTypeString[typeLength];     // what type of request: GET or POST
int nextChar = 0;                       // index counter for requestTypeString
const int fileStringLength = 16;        // length of the file requested
char fileString[fileStringLength];      // for input from the browser

void setup() {  
   // Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }

  pinMode(sdSS, OUTPUT);
  digitalWrite(sdSS, HIGH);

  Serial.print("Initializing SD card...");
  if (!SD.begin(sdCS)) {
    Serial.println("initialization failed!");
  }
  else {
    Serial.println(F("initialization done."));
  }

  delay(1000);

  Ethernet.begin(mac, ip, gateway, subnet);

  Serial.println(Ethernet.localIP());
  server.begin();
}


void loop() {
  String fileName = "";         // filename the client requests
  char inChar = 0;              // incoming character from client
  int requestType = 0;          // what type of request (GET or POST);
  int requestedFileLength = 0;  // length of the filename they asked for
  
  EthernetClient client = server.available();
  
  if (client) {

    TextFinder finder(client );
    
    Serial.println("new client");
    
    while (client.connected()) {
      if (client.available()) {
        // look for whatever comes before the /. It should be GET or POST:
        if(finder.getString("","/", requestTypeString,typeLength)){
          // Do something different for GET or POST:
          if(String(requestTypeString) == "GET " ) {
            requestType = 1;
          }
          else if(String(requestTypeString) == "POST ") {
            requestType = 2;
          }
          
          requestedFileLength = finder.getString("", " ",
          fileString, fileStringLength);

          switch (requestType) {
          case 1: // GET
            // Just send file
            break;
          case 2: //POST
            //TODO: handle posts
            break;
          }  
          //Serve requested page
          if (requestedFileLength < 2) {
            sendFile(client, "index.htm");
          }
          // otherwise send whatever file they asked for:
          else {
            sendFile(client, fileString);
          }     
        }

        delay(5);

        Serial.println("Closing the connection");
        client.stop();
      }
    }
  }
}

void sendFile(EthernetClient client, char file[]) {
  String outputString = "";   

  File page = SD.open(file);
  
  if (page) {
    sendHttpHeader(client, 200);

    while (page.available()) {

      char thisChar = page.read();
      outputString += thisChar;

      if (thisChar == '\n') {
        client.print(outputString);
        outputString = "";
      }
    }

    if (outputString != "") {
      client.print(outputString);
    }

    page.close();
  }
  else {
    sendHttpHeader(client, 404);
  }
}

void sendHttpHeader(EthernetClient client, int error) {
  client.print("HTTP/1.1 ");
  switch(error) {
  case 200: //OK
    client.println("200 OK");
    client.println("Content-Type: text/html");
    break;
  case 404: // file not found
    client.println("404 Not Found");
    break;
  }
  
  client.println();
}

Try this. Change the network settings to yours. It send a page with a form. Upload the code, then open the serial monitor. Use a web browser to download the page. I put the SD stuff there too, but it doesn’t read anything from it yet. Do that after you get the server working.

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip( 192,168,2,2 );
IPAddress gateway( 192,168,2,1 );
IPAddress subnet( 255,255,255,0 );
IPAddress mydns( 192,168,2,1 );

EthernetServer server(80);

void setup()
{
  Serial.begin(9600);

  // disable w5100 while setting up SD
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);

  Serial.print("Starting SD..");
  if(!SD.begin(4)) Serial.println("failed");
  else Serial.println("ok");
  
  Ethernet.begin(mac, ip, mydns, gateway, subnet);
  digitalWrite(10,HIGH);

  delay(2000);
  server.begin();
  Serial.println("Ready");
}

void loop()
{
  EthernetClient client = server.available();
  if(client) {
    boolean currentLineIsBlank = true;
    boolean currentLineIsGet = true;
    int tCount = 0;
    char tBuf[64];
    int r,t;
    char *pch;
    
    Serial.print("Client request: ");
    
    while (client.connected()) {
      while(client.available()) {
        char c = client.read();

        if(currentLineIsGet && tCount < 63)
        {
          tBuf[tCount] = c;
          tCount++;
          tBuf[tCount] = 0;          
        }

        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response
          Serial.println(tBuf);
          Serial.print("POST data: ");
          while(client.available()) Serial.write(client.read());
          Serial.println();

          pch = strtok(tBuf,"?");

          while(pch != NULL)
          {
            if(strncmp(pch,"t=",2) == 0)
            {
              t = atoi(pch+2);
              Serial.print("t=");
              Serial.println(t,DEC);             
            }

            if(strncmp(pch,"r=",2) == 0)
            {
              r = atoi(pch+2);
              Serial.print("r=");              
              Serial.println(r,DEC);
            }

            pch = strtok(NULL,"& ");
          }
          Serial.println("Sending response");
          client.write("HTTP/1.0 200 OK\r\nContent-Type: text/html\r\n\r\n<html>");

          client.write("<head><script type=\"text/javascript\">");
          client.write("function show_alert() {alert(\"This is an alert\");}");
          client.write("</script></head");
          client.write("<body><H1>TEST</H1>");
          client.write("<form method=GET onSubmit=\"show_alert()\">T: <input type=text name=t>
");
          client.write("R: <input type=text name=r>
<input type=submit></form>");
          client.write("</body></html>\r\n\r\n");
          client.stop();
        }
        else if (c == '\n') {
          currentLineIsBlank = true;
          currentLineIsGet = false;
        } 
        else if (c != '\r') {
          currentLineIsBlank = false;
        }
      }
    }
    Serial.println("done");
  }
}

edit: I reorganized the serial output to make more sense.

I tried your submitted code and it also seems to work very good, even after a while going and and many requests.
Why is your examples working so good and my isn’t, even the WebServer example that comes with the Arduino IDE isn’t working.
Do you know, why your code working like a charm and my doesn’t?

That is an easy question with a complex answer. This is not the first time I have been asked that. :)

First, the order of the setup function is important. Before initializing any SPI device, you must disable other SPI devices. Note:

My code disables the w5100 before calling SD.begin().

SD.begin() returns with its SPI SS disabled (HIGH), so nothing needs to be done after the call.

Ethernet.begin() returns with its SPI SS enabled (LOW), so I disable it after the call. I think this is a bug, but I have yet to report it to the Arduino crew.

Then I try to follow standard operating procedures (RFCs) when receiving the request, and sending the response.

It also helped that I had to debug the ethernet library when I first joined this forum a year ago. That gave me a lot of insight to how the library code and the w5100 hardware worked together.

Okey so there are some differences there!
I tried modifying your previously submitted code, cause it relly worked like a charm, with send file so I can send file from sd card to client.
But I don’t know if this is a proper way of doing it, because now it loads the page without CSS and it never stops loading the page. So is the way of
reading file from sd card incorrect or am I doing some other wrong in sending page, here’s code.

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip( 192,168,1,100 );
IPAddress gateway( 192,168,1,1 );
IPAddress subnet( 255,255,255,0 );
IPAddress mydns( 192,168,1,1 );

EthernetServer server(80);

void setup()
{
  Serial.begin(9600);

  // disable w5100 while setting up SD
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);

  Serial.print("Starting SD..");
  if(!SD.begin(4)) Serial.println("failed");
  else Serial.println("ok");
  
  Ethernet.begin(mac, ip, mydns, gateway, subnet);
  digitalWrite(10,HIGH);

  delay(2000);
  server.begin();
  Serial.println("Ready");
}

void loop()
{
  EthernetClient client = server.available();
  if(client) {
    boolean currentLineIsBlank = true;
    boolean currentLineIsGet = true;
    int tCount = 0;
    char tBuf[64];
    int r,t;
    char *pch;
    
    int page; //Flag to decide which page to serve
    
    Serial.print("Client request: ");
    
    while (client.connected()) {
      while(client.available()) {
        char c = client.read();

        if(currentLineIsGet && tCount < 63)
        {
          tBuf[tCount] = c;
          tCount++;
          tBuf[tCount] = 0;          
        }

        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response
          Serial.println(tBuf);
          Serial.print("POST data: ");
          while(client.available()) Serial.write(client.read());
          Serial.println();
       
          pch = strtok(tBuf,"/");

          while(pch != NULL)
          {

            if(strncmp(pch,"admin.htm",9) == 0)
            {
              Serial.println("admin.htm");  
              page = 1;            
            }
            
            else if(strncmp(pch,"log.htm",7) == 0)
            {
              Serial.println("log.htm");  
              page = 2;            
            }
            
            //Special case when the user connects without any parameter
            else 
            {
               Serial.println("index.htm");  
               page = 0;             
            }
            
            pch = strtok(NULL,"& ");
          }

          Serial.println("Sending response");

          switch(page) 
          {
            case 0: 
              sendFile(client, "index.htm");
              break;
            case 1:
              sendFile(client, "admin.htm");
              break;
            case 2:
              sendFile(client, "log.htm");
              break;
          }
        }
        else if (c == '\n') {
          currentLineIsBlank = true;
          currentLineIsGet = false;
        } 
        else if (c != '\r') {
          currentLineIsBlank = false;
        }
      }
    }
    Serial.println("done");
  }
}

// send the file that was requested:
void sendFile(EthernetClient client, char file[]) {
  String outputString = ""; // a String to get each line of the file
  // open the file for reading:
  File page = SD.open(file);
  if (page) {
    // send an OK header:
    sendHttpHeader(client, 200);
    // read from the file until there's nothing else in it:
    while (page.available()) {
      // add the current char to the output string:
      char thisChar = page.read();
      outputString += thisChar;
      // when you get a newline, send out and clear outputString:
      if (thisChar == '\n') {
        client.print(outputString);
        outputString = "";
      }
    }
    // if the file does not end with a newline, send the last line:
    if (outputString != "") {
      client.print(outputString);
    }
    // close the file:
    page.close();
  }
  else {
    // if the file didn't open:
    sendHttpHeader(client, 404);
  }
}

void sendHttpHeader(EthernetClient client, int error) {
  client.print(F("HTTP/1.1 "));
  switch(error) {
  case 200: // OK
    client.println(F("200 OK"));
    client.println(F("Content-Type: text/html"));
    break;
  case 404: // file not found
    client.println(F("404 Not Found"));
    break;
  }
  // response header ends with an extra linefeed:
  client.println();
}

Please fix the code posted in your last reply to remove all the commented out stuff. The Arduino doesn't see that code. We shouldn't have to, either.

How big is the file on the SD? You are building a String object that size. Maybe you are running out of SRAM?

    while (page.available()) {
      // add the current char to the output string:
      char thisChar = page.read();
      // if the opened file is large (>1K or so on an Uno), it will run out of SRAM here:
      outputString += thisChar;
      // when you get a newline, send out and clear outputString:
      if (thisChar == '\n') {
        client.print(outputString);
        outputString = "";
      }
    }

Oooops! The index.htm page is 4kB, is there any way to solve this problem? Maybe it's more clever to build up the site programmatically, but that could also be problem with RAM, or?

I'm use to program Java and Android, this is my first microcontroller project ever so I'am very new to this field, and not used to take care to any RAM limitations :blush:

Edit: is it really possible to run out of RAM in this case, because the outputstring gets declared as empty when it reads in a new line character or am I wrong? I mean it would have to read in pretty many characters before buffer reach 1kB.

    while (page.available()) {
      // add the current char to the output string:
      char thisChar = page.read();
      outputString += thisChar;
      if (thisChar == '\n') {
        client.print(outputString);
        outputString = ""; //declared as empty here, preventing buffer from reaching >1kB
      }

You are moving in exactly the right direction. The SD is the cure for limited memory. I use character arrays rather than String objects. As an example snippet:

char outBuf[64];
int outCount = 0;

while(page.available()) {
  outBuf[outCount] = page.read();
  outCount++;

  if(outCount > 63)
  {
    client.write(outBuf,64);
    outCount = 0;
  }
}

if(outCount > 0) client.write(outBuf,outCount);

client.stop();

The code uses a 64 byte array to store part of the file at a time. When it fills up, it sends what it has, and resets the index to the array and starts again.

edit: I forgot to increment outCount after the save to the array. Fixed now.

It seems to definetily be a RAM problem tested your latest code snippet but that resulted in alot of garbage in output, and thats not accepted. So I tested printing characters directly to client from like this client.print((char)page.read()), need to do a typecast otherwise the characters numerical value will be printed.

This solution actually seems to work pretty good, just have to wait and see.

  File page = SD.open(file);
  
  if (page) {
    sendHttpHeader(client, 200);
    while (page.available()) {
      client.print((char)page.read());
    }  
    page.close();
  }
  else {
    sendHttpHeader(client, 404);
  }

I did not debug the snippet. Your modification seems ok, but takes a lot of network resources to send the response. Every time you call any of these... client. print() client.println() client.write() ...you are sending a packet per call. So in your example, the 4K file will take 4K packets to send. Considering you are sending one character with a 20 byte IP header and a 20 byte TCP header, that is a lot of overhead. I like to send at least 64 characters per packet.

Thank you for your advice! But now at least I'm on the right track :), so next step is to optmize the printing to client to reduce network load!

BTW, I tried that snippet, and it required the array to be of type byte.

byte outBuf[64];

Otherwise, it did fine. I tested it on my FTP client code. I was looking for a reason to improve that code anyway. It had the same network-wasting send as yours.