Uno R3 - Ethernet shield and memory problem

Hi guys! I'm working on my home automation project! my goal is to control some relays through the I/O pins and get a temperature reading from pin A0 from a web browser. However that goal seem's very far off at this point.
Here comes a small description off how it should work. I have the htm and css on an sd card, the files are loaded on client requests, the data to set the pin modes get passed through POST values, I use TextFinder to parse the request and if I find a POST I take proper action.
I have got the server running pretty good, it reads files from sd cards and passes them over to client, I have even managed to get the posts working, and i can correctly set the pinmodes.
However I have stumbled over some major problems that I don't understand.
Problem 1: after I added the function checkPINPOST, the card seems to get some sort of memory problem, because after each request to the server it loads files on random from SD card, runs the setup function on random, cutting string reading, well it does all kinds of strange things. but as soon as I delete some code It all runs fine, why is that the sketch is about 25778 bytes(of a 32256 bytes maximum) so it shouldn't lack any memory or?
Problem 2: For the application to be user friendly the webpage should tell the user if any pin is on or off, my plan to do this is to have some specified strings in the html ex. $pin1, and my sendfile function reads carachers into a buffer and then after each character read I use the function string.endsWith("$pin1"), and if it's found replace $pin1 with on or off. But this doesn't seem to work, when I implement this algorithm, it refuses to load the page, seems like the code is going into some loop of eternity, even though it's impossible. Is this a correct way of achieving this or are there any other more efficient ways?

A post all my code for you to look into, and any help is needed. Starting to pull my hair of with those memory problems, or what it is.

Thanks!

PS! To run the code just copy a htm file to root of SD card and it shold run fine, or good enough for you to test and debug...

Here comes the code!

/*
 *  Project.: SwitchIt - A Tiny Ethernet Controller
 *  File....: SwitchIt.ino
 *  Date....: 2012-07-13
 *  Author..: Robert N
 */

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>
#include <TextFinder.h>


byte mac[] = { 
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192,168,1, 100);
IPAddress gateway(192,168,1,1);
IPAddress subnet(255,255,255,0);
IPAddress mydns(192,168,1,1);

//Array of strings holding Pins status
String status[7] = {
  "Av","Av","Av","Av","Av","Av","Av"};

int port = 86;

EthernetServer server(port);

const int inputLength = 25;   // length of the file requested *ORIGINAL:16
const int typeLength = 6;     // length of GET or POST

char requestTypeString[typeLength];  // what type of request: GET or POST
const int fileStringLength = 25;     // length of the file requested *ORIGINAL:16
char fileString[fileStringLength];   // for input from the browser

void setup() {  
  Serial.begin(9600);

  // disable w5100 while setting up SD
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);

  Serial.println("SwitchIt - A Tiny Ethernet Controller");
  Serial.println("---------------------------------------------");
  Serial.println("Initializing server!");
  Serial.print("Starting SD.... ");
  if(!SD.begin(4)) Serial.println("Failed!");
  else Serial.println("Ok!");

  Ethernet.begin(mac,ip,mydns,gateway,subnet);
  digitalWrite(10,HIGH);

  /*
   * Setup Pins
   */
  pinMode(2,OUTPUT);
  pinMode(3,OUTPUT);
  pinMode(5,OUTPUT);
  pinMode(6,OUTPUT);
  pinMode(7,OUTPUT);
  pinMode(8,OUTPUT);
  pinMode(9,OUTPUT);

  Serial.println("Pins and pinmodes has been set!");

  delay(2000);
  server.begin();
  Serial.println("Server Up and Running");
  Serial.print("Ip: ");
  Serial.println(Ethernet.localIP());
  Serial.print("Subnet: ");
  Serial.println(subnet);
  Serial.print("Gateway: ");
  Serial.println(gateway);
  Serial.print("DNS: ");
  Serial.println(mydns);

  Serial.println("=============================================");
  Serial.println("");
}


void loop() {
  char inChar = 0; // incoming character from client
  int requestType = 0; // what type of request (GET or POST);
  int requestedFileLength = 0; // length of the filename they asked for

  // listen for incoming clients:
  EthernetClient client = server.available();
  if (client) {
    // make an instance of TextFinder to look for stuff from the client:
    TextFinder finder(client );
    Serial.print("Client request: ");

    while (client.connected()) {
      if (client.available()) {
        // look for whatever comes before the /. It should be GET or POST:
        if(finder.getString("","/", requestTypeString,typeLength)){
          // Do something different for GET or POST:
          if(String(requestTypeString) == "GET " ) {
            requestType = 1;
          }
          else if(String(requestTypeString) == "POST ") {
            requestType = 2;
          }
          // gather what comes after the / into an array,
          // it's the filename the client wants:
          requestedFileLength = finder.getString("", " ",fileString, fileStringLength);

          if(strcmp(fileString, "") != 0)
          {
            Serial.println(fileString);
          }
          else 
          {
            Serial.println("/");
          }

          // now you're done with the GET/POST line, process what you got:
          switch (requestType) {
          case 1: // GET
            // do nothing with GET except send the file, below 
            break;
          case 2: //POST
            Serial.print("POST ");
            // skip the rest of the header,
            // which ends with newline and carriage return:
            finder.find("\n\r");
            
            //Check for I/O PIN POST values
            checkPINPOST(finder);
          }  

          Serial.print("Sending response -> ");
          // whether it's GET or POST, give them the string they asked for.
          // if there's nothing after the /,
          // then the client wants the index:
          if (requestedFileLength < 2) {
            sendFile(client, "index.htm");
          }
          // otherwise send whatever file they asked for:
          else {
            sendFile(client, fileString);
          }     
        }
        delay(100);
        Serial.println("");
        client.stop();
      }
    }
  }
}

void sendFile(EthernetClient client, char file[]) { 

  int i = 0;
  char ch;
  String outputString = "";
  
  File page = SD.open(file);
  
  if (page) {
    sendHttpHeader(client, 200);
    while (page.available()) {
      ch = (char)page.read();
      outputString += ch;
      i++;
      
      if(i == 20) {
        client.print(outputString);
        outputString = "";
        i = 0;
      }
    }
    
    client.print(outputString);
    outputString = "";
    
    page.close();
  }
  else {
    sendHttpHeader(client, 404);
  }
}

void sendHttpHeader(EthernetClient client, int error) {
  client.print("HTTP/1.1 ");
  switch(error) {
  case 200: // OK
    client.println("200 OK");
    client.println("Content-Type: text/html");
    Serial.println("200 - File sent!");
    break;
  case 404: // file not found
    client.println("404 Not Found");
    Serial.println("404 File Not Found!");
    break;
  }
  // response header ends with an extra linefeed:
  client.println();
}

/*
 * Function to check if any PIN POST values exists
 * if so, set POSTed PIN to HIGH or LOW
 */
void checkPINPOST(TextFinder finder) {
  int POSTvalue = 0;
  int PIN = 0;
  //If pin control POST found
  if(finder.find("p")) {
    PIN = finder.getValue('p');
    POSTvalue = finder.getValue('=');

    switch(PIN) {
      case(2):
      if(POSTvalue == 1) {
        Serial.println("p2=1");
        digitalWrite(2,HIGH);
        status[0] = "På";
      } 
      else {
        Serial.println("p2=0");
        digitalWrite(2,LOW);
        status[0] = "Av";    
      }
      break;
      case(3):
      if(POSTvalue == 1) {
        Serial.println("p3=1");
        digitalWrite(3,HIGH);
        status[1] = "På";
      } 
      else {
        Serial.println("p3=0");
        digitalWrite(3,LOW);  
        status[1] = "Av";  
      }
      break;
      case(5):
      if(POSTvalue == 1) {
        Serial.println("p5=1");
        digitalWrite(5,HIGH);
        status[2] = "På";
      } 
      else {
        Serial.println("p5=0");
        digitalWrite(5,LOW); 
        status[2] = "Av";   
      }
      break;
      case(6):
      if(POSTvalue == 1) {
        Serial.println("p6=1");
        digitalWrite(6,HIGH);
        status[3] = "På";
      } 
      else {
        Serial.println("p6=0");
        digitalWrite(6,LOW); 
        status[3] = "Av"; 
      }
      break;
      case(7):
      if(POSTvalue == 1) {
        Serial.println("p7=1");
        digitalWrite(7,HIGH);
        status[4] = "På";
      } 
      else {
        Serial.println("p7=0");
        digitalWrite(7,LOW); 
        status[4] = "Av";   
      }
      break;
      case(8):
      if(POSTvalue == 1) {
        Serial.println("p8=1");
        digitalWrite(8,HIGH);
        status[5] = "På";
      } 
      else {
        Serial.println("p8=0");
        digitalWrite(8,LOW);
        status[5] = "Av";   
      }
      break;
      case(9):
      if(POSTvalue == 1) {
        Serial.println("p9=1");
        digitalWrite(9,HIGH);
        status[6] = "På";
      } 
      else {
        Serial.println("p9=0");
        digitalWrite(9,LOW);
        status[6] = "Av";   
      }                      
      break;               
    }

  }
}

You have a lot of character literals, all consuming SRAM. I suggest you look at the F() macro.

You can pass flash-memory based strings to Serial.print() by wrapping them with F(). For example :
Serial.print(F(“Hello World”))

It also works with Client.print().

The whole block:

switch(PIN) {
      case(2):
      if(POSTvalue == 1) {
        Serial.println("p2=1");
        digitalWrite(2,HIGH);
        status[0] = "På";
      } 
      else {
        Serial.println("p2=0");
        digitalWrite(2,LOW);
        status[0] = "Av";    
      }
      break;
      case(3):
      if(POSTvalue == 1) {
        Serial.println("p3=1");
        digitalWrite(3,HIGH);
        status[1] = "På";
      } 
      else {
        Serial.println("p3=0");
        digitalWrite(3,LOW);  
        status[1] = "Av";  
      }
      break;
      case(5):
      if(POSTvalue == 1) {
        Serial.println("p5=1");
        digitalWrite(5,HIGH);
        status[2] = "På";
      } 
      else {
        Serial.println("p5=0");
        digitalWrite(5,LOW); 
        status[2] = "Av";   
      }
      break;
      case(6):
      if(POSTvalue == 1) {
        Serial.println("p6=1");
        digitalWrite(6,HIGH);
        status[3] = "På";
      } 
      else {
        Serial.println("p6=0");
        digitalWrite(6,LOW); 
        status[3] = "Av"; 
      }
      break;
      case(7):
      if(POSTvalue == 1) {
        Serial.println("p7=1");
        digitalWrite(7,HIGH);
        status[4] = "På";
      } 
      else {
        Serial.println("p7=0");
        digitalWrite(7,LOW); 
        status[4] = "Av";   
      }
      break;
      case(8):
      if(POSTvalue == 1) {
        Serial.println("p8=1");
        digitalWrite(8,HIGH);
        status[5] = "På";
      } 
      else {
        Serial.println("p8=0");
        digitalWrite(8,LOW);
        status[5] = "Av";   
      }
      break;
      case(9):
      if(POSTvalue == 1) {
        Serial.println("p9=1");
        digitalWrite(9,HIGH);
        status[6] = "På";
      } 
      else {
        Serial.println("p9=0");
        digitalWrite(9,LOW);
        status[6] = "Av";   
      }                      
      break;               
    }

can be written much shorter, also saving RAM:

if(POSTvalue == 1) {
        Serial.print("p");
        Serial.print(PIN);
        Serial.println("=1");
        digitalWrite(PIN,HIGH);
        status[PIN-(PIN < 5 ? 2 : 3)] = "På";
      } 
      else {
        Serial.print("p");
        Serial.print(PIN);
        Serial.println("=0");
        digitalWrite(PIN,LOW);
        status[PIN-(PIN < 5 ? 2 : 3)] = "Av";    
      }
String status[7] = {
  "Av","Av","Av","Av","Av","Av","Av"};

In a memory constrained situation? I don't think so.

Thanks for all advices.
I have changed the switch to the posted if/else statement, that was nice. I've also removed the String array! I admit, that was bad! As a last change ive removed most of the Serial prints, just hav a few left with F() syntax, and also the client.print uses the syntax F().
Even after I've made this changes it still works crappy, real crappy! My sketch seems to work fine when its less than 27000 bytes big after that it doesn't work, server stops responding, never stops loading and so on. I really don't understand why it doesn't work, I post my code ones more in hope that anyone would help me.

PS. this doesn't work -> client.print(F(some_string)), because no string length is known, is there any way to work around this??

The CODE

/*
 *  Project.: SwitchIt - A Tiny Ethernet Controller
 *  File....: SwitchIt.ino
 *  Date....: 2012-07-13
 *  Author..: Robert N
 */

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>
#include <TextFinder.h>

byte mac[] = { 
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192,168,1, 100);
IPAddress gateway(192,168,1,1);
IPAddress subnet(255,255,255,0);
IPAddress mydns(192,168,1,1);

int port = 86;

EthernetServer server(port);

char requestTypeString[6];  // what type of request: GET or POST
char fileString[25];   // for input from the browser

void setup() {  
  Serial.begin(9600);

  // disable w5100 while setting up SD
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);

  Serial.println(F("SwitchIt - A Tiny Ethernet Controller"));
  Serial.println(F("---------------------------------------------"));
  Serial.println(F("Initializing server!"));
  if(!SD.begin(4)){ Serial.println(F("Failed to mount SD!"));}
  else {
    Ethernet.begin(mac,ip,mydns,gateway,subnet);
    digitalWrite(10,HIGH);
  
    /*
     * Setup Pins
     */
    pinMode(2,OUTPUT);
    pinMode(3,OUTPUT);
    pinMode(5,OUTPUT);
    pinMode(6,OUTPUT);
    pinMode(7,OUTPUT);
    pinMode(8,OUTPUT);
    pinMode(9,OUTPUT);
  
    delay(2000);
    server.begin();
    Serial.println(F("Server Up and Running"));
  }
}


void loop() {
  int requestedFileLength = 0; // length of the filename they asked for

  // listen for incoming clients:
  EthernetClient client = server.available();
  if (client) {
    // make an instance of TextFinder to look for stuff from the client:
    TextFinder finder(client );

    while (client.connected()) {
      if (client.available()) {
        // look for whatever comes before the /. It should be GET or POST:
        if(finder.getString("","/", requestTypeString,6)){
          // gather what comes after the / into an array,
          // it's the filename the client wants:
          requestedFileLength = finder.getString("", " ",fileString, 16);

          // now you're done with the GET/POST line, process what you got:
          if(String(requestTypeString) == "POST ") {
            // skip the rest of the header,
            // which ends with newline and carriage return:
            finder.find("\n\r");

            //Check for I/O PIN POST values
            checkPINPOST(finder);
          }  

          // whether it's GET or POST, give them the string they asked for.
          // if there's nothing after the /,
          // then the client wants the index:
          if (requestedFileLength < 2) {
            sendFile(client, "index.htm");
          }
          // otherwise send whatever file they asked for:
          else {
            sendFile(client, fileString);
          }     
        }
        delay(100);
        client.stop();
      }
    }
  }
}

void sendFile(EthernetClient client, char file[]) { 

  int i = 0;
  char ch;
  String outputString = "";

  File page = SD.open(file);

  if (page) {
    sendHttpHeader(client, 200);
    while (page.available()) {
      ch = (char)page.read();
      outputString += ch;
      i++;

      //Check for $status_x tag and replace it with proper string
      if(outputString.endsWith("$status_2")){if(digitalRead(2) == 1){outputString.replace("$status_2","På");}else{outputString.replace("$status_2","Av");}}    
      if(outputString.endsWith("$status_3")){if(digitalRead(3) == 1){outputString.replace("$status_3","På");}else{outputString.replace("$status_3","Av");}}
      if(outputString.endsWith("$status_5")){if(digitalRead(5) == 1){outputString.replace("$status_5","På");}else{outputString.replace("$status_5","Av");}}
      if(outputString.endsWith("$status_6")){if(digitalRead(6) == 1){outputString.replace("$status_6","På");}else{outputString.replace("$status_6","Av");}}
      if(outputString.endsWith("$status_7")){if(digitalRead(7) == 1){outputString.replace("$status_7","På");}else{outputString.replace("$status_7","Av");}}
      if(outputString.endsWith("$status_8")){if(digitalRead(8) == 1){outputString.replace("$status_8","På");}else{outputString.replace("$status_8","Av");}}
      if(outputString.endsWith("$status_9")){if(digitalRead(9) == 1){outputString.replace("$status_9","På");}else{outputString.replace("$status_9","Av");}}      

      //Ceck for the ethernet configuration tags and replace them
      //      if(outputString.endsWith("Ip adress")){outputString.replace("$Ip adress",(String)ip);}
      //      if(outputString.endsWith("$Nätmask")){outputString.replace("$Nätmask",(String)subnet);}
      //      if(outputString.endsWith("$Gateway")){outputString.replace("$Gateway",(String)gateway);}
      if(outputString.endsWith("$Port")){
        outputString.replace("$Port",(String)port);
      }      

      if(i == 64) {
        client.print(outputString);
        // Serial.println(outputString);
        outputString = "";
        i = 0;
      }
    }

    client.print(outputString);
    outputString = "";

    page.close();
  }
  else {
    sendHttpHeader(client, 404);
  }
}

void sendHttpHeader(EthernetClient client, int error) {
  client.print(F("HTTP/1.1 "));
  switch(error) {
  case 200: // OK
    client.println(F("200 OK"));
    client.println(F("Content-Type: text/html"));
    break;
  case 404: // file not found
    client.println(F("404 Not Found"));
    break;
  }
  // response header ends with an extra linefeed:
  client.println();
}

/*
 * Function to check if any PIN POST values exists
 * if so, set POSTed PIN to HIGH or LOW
 */
void checkPINPOST(TextFinder finder) {
  int POSTvalue = 0;
  int PIN = 0;
  //If pin control POST found
  if(finder.find("p")) {
    PIN = finder.getValue('p');
    POSTvalue = finder.getValue('=');

    if(POSTvalue == 1) {
      digitalWrite(PIN,HIGH);
    } 
    else {
      digitalWrite(PIN,LOW);
    }
  }
}
  Serial.println(F("---------------------------------------------"));

Are all those dashes really needed?

#include <SD.h>

There goes 1/4 of your memory.

#include <TextFinder.h>

Another resource hog.

          if(String(requestTypeString) == "POST ") {

What's wrong with strcmp()? Creating a String object so you can use the overloaded == is a waste of resources.

  String outputString = "";

There's that dreaded String class again. Wasting resources; fragmenting memory.

PS. this doesn't work -> client.print(F(some_string)), because no string length is known, is there any way to work around this??

What is some_string? A variable? You have several client.print(F("xxx")) statements that presumably work. Or, don't they?

Well, those dashes are a obvious waste of memory, is corrected now.

For real, does you men that just including one library takes up 1/4 of the memory??
I need to use that SD library, and also the textfinder library, otherwise it would lead to too much refactoring...

Have corrected that string typecast... It did save more memory than I thaught.

The reason why I use the string for outputString is because I need to compare some strings with the string method endsWith, I have no problem using an fixed size array but I need to do a string compare, based on that comparison and some pinmodes I give the user information that a pin are high or low, if there are any other way to do this comparison you're very welcome to tell me, cause I'm out of ideas.

The client.print(F()) doesn't work just because of that, I try to print out an object of type String and therefore it has no fixed length, but if I can get this to work without the strings I get rid of this problem too.

The client.print(F()) doesn't work just because of that, I try to print out an object of type String

You can use the F() macro just for constants! A String object is not a constant. The F() macro is storing the string in the program flash and the Print object is accessing it then byte by byte. As you cannot change the flash outside the bootloader you cannot store anything there except constants.

For real, does you men that just including one library takes up 1/4 of the memory??

Yes. It needs a buffer of the size of one FAT block (512 bytes). As the standard Arduino (like the UNO) has just 2kB RAM, that's 1/4 of it.