Arduino gives up / Flooding ethershield?

Hello again!

I'm trying to figure out why my ethershield appears to just give up polling an external site. My project is to poll a site at semi-regular intervals in order to decide if the Arduino should engage a solenoid. The issue I'm seeing is that after a while, the arduino no longer polls the site and stops responding to anything but the reset switch. I suspect it may be a timing issue, however I'm not certain.

void loop(){
  int val = 0;
  int bytesread = 0;
  char code[10];
   
  delay(1000); 
  //Turn on RFID reader, set LED to orange to "Neutral"

  // If API says we are supposed to be unlocked, (returns 1), then
  // unlocked we are.
  if( ServerComms(stationID,code,1,server)) 
    {
    //LED to GREEN, Solenoid to HIGH
      digitalWrite(solenoid,HIGH);
      digitalWrite(redled,LOW);
      digitalWrite(greenled,HIGH);
      delay(5000);
    }
    //Else, we go through the card stuff.
    else
    {
      //LED to ORANGE, Solenoid to LOW
      digitalWrite(solenoid,LOW);
      digitalWrite(rfid,LOW);
      digitalWrite(redled,HIGH);
      digitalWrite(greenled,HIGH);
      if(Serial.available() > 0)    // if data available from reader 
        {
          if((val = Serial.read()) == 10) 
          {   // check for header 
            bytesread = 0;
            digitalWrite(rfid,HIGH);
            digitalWrite(redled,LOW);
            digitalWrite(greenled,LOW); 
            while(bytesread<10) 
            {              // read 10 digit code 
              if( Serial.available() > 0) 
              { 
                val = Serial.read(); 
                if((val == 10)||(val == 13)) // if header or stop bytes before the 10 digit reading 
                {
                  break;                       // stop reading 
                } 
                code[bytesread] = val;         // add the digit           
                bytesread++;                   // ready to read next digit  
              } 
            } 
            if(bytesread == 10)                // if 10 digit read is complete 
            {
              Serial.println(code);
              if( ServerComms(stationID,code,2,server))
              {
                Serial.println("Good Card - Unlocking goes here.");
                digitalWrite(greenled,HIGH);
                digitalWrite(solenoid,HIGH);
                delay(2000);
                digitalWrite(solenoid,LOW);
                char code[10]="         ";
              }
              else
              {
                Serial.println("BAD CARD - Flashing red LED.");
                for (val = 0; val < 6; val++ )
                {
                  digitalWrite(greenled,LOW);
                  digitalWrite(redled,HIGH);
                  delay(500);
                  digitalWrite(redled,LOW);
                  delay(500);
                }
              }
            } 
            bytesread = 0; 
                                // deactivate the RFID reader for a moment so it will not flood
           delay(2000);         // wait for a bit 
          }
        }
      }    
   }

In english:

The code polls an API server, if "1", it unlocks, if 0 it checks the serial buffer for anything on the RFID reader. If nothing, then it sleeps for two seconds before checking the API server again. The ServerComms function returns either a 0 or a 1 depending on what the API returns on polling. If the Arduino is responding, swiping a "good" card results in an unlock, and swiping a "bad" card results in a flashing LED, then returns to normal polling state as expected.

I have a theory that I'm "flooding" the ethershield, however I'm still learning about how to do network communications with the ethershield that's long-term reliable. This is my first experiment with the Ethershield as a client.

debug output at time of failure:

Request:GET /ardurfid/commander.php?sta=2&mode=poll HTTP/1.0
API says YES
Request:GET /ardurfid/commander.php?sta=2&mode=poll HTTP/1.0
API says YES
Request:GET /ardurfid/commander.php?sta=2&mode=poll HTTP/1.0
API says YES
Request:GET /ardurfid/commander.php?sta=2&mode=poll HTTP/1.0
Request:GET /ardurfid/commander.php?sta=2&mode=poll HTTP/1.0

Any time I see the double REQUEST line above, I know the arduino has frozen up. How long should I wait in seconds between API polling?

What I'm trying to do is that if there is no card swiped, the device should check with the API to make sure it's supposed to be checking for cards (API=0) or supposed to be unlocked (API=1). If it's supposed to be unlocked, it should be checking the API at regular intervals (currently ~5sec) to check when it should go back to checking for cards again. If it's in checking for cards mode, it should be checking the serial buffer for a card to report to the API server(done as soon as it has a 10 character card ID) and checking for whether or not it should be unlocked (at regular intervals). The goal is to make a networked RFID door lock that uses a centralized API to manage card access.

If you need full-code or have suggestions, please let me know. I appreciate your help.

Where is the code that does the client.connect() client.available() client.read() client.stop(), and "API says yes" stuff?

Hello SurferTim:

The function "ServerComms" does that. The code is below:

int ServerComms( int sid, String bid, int mode, byte srv )
{
//  sid = Device ID
//  bid = Badge ID (for reading, 0 otherwise)
//  mode 0 = Register, 1 = Poll 2 = GetAuth
//  srv = Server

  char resp;         // Server response, should always be one character.
  String getrequest; // GET request formatted prior to sending to server.

  switch (mode)
  {
    //register  "GET /ardurfid/commander.php?sta=XX&mode=reg"; XX is Station ID
    case 0:
      getrequest = "GET /ardurfid/commander.php?sta=";
      getrequest += stationID;
      getrequest += "&mode=reg HTTP/1.0";
    break;
  
    //poll "GET /ardurfid/commander.php?sta=XX&mode=poll"; XX is Station ID
    case 1:
        getrequest = "GET /ardurfid/commander.php?sta=";
        getrequest += stationID;
        getrequest += "&mode=poll HTTP/1.0";
    break;
  
    //getauth "GET /ardurfid/commander.php?sta=XX&mode=auth&bid=YYYYYYYYYYY" XX is Station ID, YYYYYYYY is read card ID.
    case 2:
      getrequest = "GET /ardurfid/commander.php?sta=";
      getrequest += stationID;
      getrequest += "&mode=auth&card=";
      getrequest += bid;
      getrequest +=" HTTP/1.0";  
  break;

  } // END of case statement
  Serial.print("Request:");
  Serial.println(getrequest);

  if (client.connect(server, 80)) 
    {
    client.println(getrequest);
    client.println(); 
    } 
    else
    {
    // Do nothing if we have no connection. Light stays red
    // indicating fault.
    for (;;) 
    {
      }
    }   
  while(client.connected()) 
  {
    while(client.available()) 
    {
      char c = client.read();
      // Read for API's response started with a #
      if ( c == '#' )
      {
        c = client.read();
        if ( c == '1')
          {
           client.stop();
           Serial.println("API says YES");        
           return 1;
          }
          else
          {
            client.stop();
            Serial.println("API says NO");
            return 0;
          }
      }
    }
  }
}

Have you tried something simpler as a debug?

  Serial.println("connected");

  while(client.connected()) 
  {
    while(client.available()) 
    {
      char c = client.read();
      Serial.print(c);
    }
  }
  client.stop();
  Serial.println();
  Serial.println("disconnected"););

Are you seeing the characters you expect the code to parse between connected and disconnected?

Hello SurferTim:

Yes, the code as presented works. The problem is that after a while of constant disconnect/reconnects, its like the client gets tired and halts.

The function and the initial code provided work as advertised, except when it comes to leaving the Arduino alone for a while. From the moment I hit RESET, the board gets an IP, blinks the "OK" blink, then goes into the correct mode (lock/scan or unlock). If it's in lock/scan mode, I can scan a card and it'll read it, report to the API and act on the API's response as expected. If it's in unlock mode, it'll poll the API as expected.

The issue is that after a minute or so of sitting "idle" (not actively working on a received card) the polling will arbitrarily stop. When it does, the card reader doesn't see a new card and I stop seeing the polls go out.

Yes, the code as presented works. The problem is that after a while of constant disconnect/reconnects, its like the client gets tired and halts.

Halts where? What is the last line you see in the serial monitor after the halt? That is why the additional serial prints in mine. Is the last line "connected" or "disconnected"?

edit: I failed to see this. My bad!

I can scan a card and it'll read it, report to the API and act on the API's response as expected. If it's in unlock mode, it'll poll the API as expected.

The issue is that after a minute or so of sitting "idle" (not actively working on a received card) the polling will arbitrarily stop. When it does, the card reader doesn't see a new card and I stop seeing the polls go out.

Please post your setup function.

It's a long shot I'll suggest, but try making the connection strings constants, and define them using PROGMEM.

So, instead of having this:

getrequest = "GET /ardurfid/commander.php?sta=";

Do this at a higher scope (right after any #include you might have and outside and procedure/function):

prog_char sGET[] PROGMEM = "GET /ardurfid/commander.php?sta=";

In your replaced code, just start with:

getrequest = sGET + stationID;

You can do that for all the string constants you have, specially the longer ones.

Hello SurferTim:

Looks like it dies on opening a new connection. I modified the client.connected while loop to print a message on failed connection as that was the only place where it goes into an endless (halted) loop without flashing an LED and sure enough, that was the location where it was dying.

After some additional testing, it appears that the ethershield IS flooding the server. During a locked/poll session, the device should be phoning home every few seconds to see if it needs to change state. I did some digging around and found that just before the Arduino halted, I had over 200 incoming HTTP connections from the Arduino's IP. It appears that the Wiznet can only support so many connections and when too many connections are open, it will cause client.connect to fail. :blush:

Every 1.0s: netstat -an | grep tcp | grep 80                                                                                                                                               Thu Aug 30 16:06:39 2012

tcp6       0      0 :::80                   :::*                    LISTEN
tcp6       0      0 172.16.222.100:80       172.16.222.160:1027     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1058     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1038     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1045     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1043     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1066     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1044     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1061     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1056     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1057     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1051     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1048     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1062     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1042     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1055     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1039     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1060     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1053     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1036     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1029     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1041     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1034     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1037     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1030     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1067     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1033     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1032     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1059     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1065     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1052     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1047     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1050     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1064     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1025     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1031     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1040     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1054     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1035     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1046     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1028     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1063     TIME_WAIT
tcp6       0      0 172.16.222.100:80       172.16.222.160:1049     TIME_WAIT

I guess the big question now is how can I tell the Arduino to only phone home once every 30sec or so and still be able to read cards in between pollings? Because it has to be able to read a card, I think that delay(5000) is out?

I had over 200 incoming HTTP connections from the Arduino's IP. It appears that the Wiznet can only support so many connections and when too many connections are open, it will cause client.connect to fail.

The Arduino can support 4 connections, max.

I guess the big question now is how can I tell the Arduino to only phone home once every 30sec or so and still be able to read cards in between pollings? Because it has to be able to read a card, I think that delay(5000) is out?

It is. Think about how YOU would do it, given a watch and a piece of paper (to write down when you last checked). Then, look at the Blink Without Delay example to see how to translate your method into Arduino-ese.

I notice that you are using the String class in the ServerComms routine. The String class can cause problems with random hang ups etc. due to an error in the Free routine. There is a patch available I believe, but in most casses coding without using the String class is a better approach.

Paul

Looks like it dies on opening a new connection. I modified the client.connected while loop to print a message on failed connection as that was the only place where it goes into an endless (halted) loop without flashing an LED and sure enough, that was the location where it was dying.

So the last thing on the serial monitor is "Request: " and you request string? Is it printing the connected and disconnected messages correctly until then?

edit: Sending a request every few seconds will not flood the server. Not reading the response completely and not closing the connection correctly (like your original code) will leave the socket open. You can do that 4 times before it fails on the fifth attempt.