Ethernet Shield

Hi all,

after some problems in the beginning I put Uno with official Ethernet Shield ( version 5 with SD without POE) into operation uploading data to Pachube. Everything worked well for more than 3 weeks with no single problem. Yesterday morning without any change to HW and SW I found the data are not sent because there is something wrong with Uno+EtheShield. After hardware reset everything went back and data were sent again. Nevermind from that time the problem occurs regurally very often (in interval 15 - 90 minutes). I did some temporary changes to SW and HW causing the UNO resets automatically when the problem appears ... but the question WHY is still in my mind.

The SW takes a part standard Ethernet library i.e. here Ethernet - Arduino Reference . As the problem seems to start with FALSE result of query "if (client.connect()) { ..." (I get message "connection failed") I would like to ask what it does in fact mean or where the bottleneck can be.

For sake of good order being in troubles I can 'ping' the Shield getting proper reply with IP and respond time = EtheShield is physically connected to LAN.

Can be the problem rather on UNO side loosing the 'Client' class or there is something wrong on EtheShield's side not responding to Uno requests ??

At the end I state that for 3 weeks it worked without single connection problem and now it switched to endless sizing serie ....

Any ideas ???

Kind regards
Vladimir

First, I do NOT know what the problem is. However, I do know that this happens a lot in my experience. The first time I used the ethernet card I thought it was broken. I eventually got it to work pretty reliably like you did, but it would drop out on occasion and not come back. After even more testing and code changing I gave up and took a solution similar to yours. However, I went even further. I modified the hardware such that I can reset the ethernet board from one of the digital pins such that I can just reset the ethernet board to see if it recovers. If that fails, I can then reset the arduino and start over from scratch. This works quite well since the arduino can reboot in a couple of seconds and pachube doesn't even notice I went away.

To see the degree I went to trying to handle this problem take a look at my blog draythomp.blogspot.com under the thermostat tab where I put pictures of the modifications I made. This problem existed with the previous version of the ethernet card as well as the newer ones.

I suspect the ethernet protocol on the chip itself has a problem and goes away from time to time. The library can't overcome something like that, so we have to do it ourselves.

Greetings to the desert :slight_smile: ,

I move a little bit further and here are my find-outs. For communication with Pachube I use significant part of the code coming with IDE0022 and it can be found under Examples/Ethernet/PachubeClient. Herebelow is the most important part:

void loop() {
  // read the analog sensor:
  int sensorReading = analogRead(A0);   

  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if there's no net connection, but there was one last time
  // through the loop, then stop the client:
  if (!client.connected() && lastConnected) {
    Serial.println();
    Serial.println("disconnecting.");                               //<------------------------standard end of session---------------
    client.stop();                                                       //<------------------------standard end of session---------------
  }

  // if you're not connected, and ten seconds have passed since
  // your last connection, then connect again and send data:
  if(!client.connected() && (millis() - lastConnectionTime > postingInterval)) {
    sendData(sensorReading);
  }
  // store the state of the connection for next time through
  // the loop:
  lastConnected = client.connected();
}

// this method makes a HTTP connection to the server:
void sendData(int thisData) {
  // if there's a successful connection:
  if (client.connect()) {
    Serial.println("connecting...");
    // send the HTTP PUT request. 
    // fill in your feed address here:
    client.print("PUT /api/YOUR_FEED_HERE.csv HTTP/1.1\n");
    client.print("Host: www.pachube.com\n");
    // fill in your Pachube API key here:
    client.print("X-PachubeApiKey: YOUR_KEY_HERE\n");
    client.print("Content-Length: ");

    // calculate the length of the sensor reading in bytes:
    int thisLength = getLength(thisData);
    client.println(thisLength, DEC);

    // last pieces of the HTTP PUT request:
    client.print("Content-Type: text/csv\n");
    client.println("Connection: close\n");

    // here's the actual content of the PUT request:
    client.println(thisData, DEC);

    // note the time that the connection was made:
    lastConnectionTime = millis();
  } 
  else {
    // if you couldn't make a connection:
    Serial.println("connection failed");  
                                                                                   //<-----insert     client.stop()     --HERE----------------------
  }
}

So I found out that standard connection's output starts with "connecting ...", further prints replies of remote server and finish with "disconnecting." and instruction "client.stop()". Problems occur when sketch don't get the proper information whether connection is lost or not, it means the query "if( !client.connected() && lastConnected)" is all the time FALSE. In this case the client is not stopped and the loop() in next step checks via the function "sendData(sensorReading)" "if( client.connect() )" ... As the result of this query is also FALSE sketch jumps to endless loop with "connection failed" statement ...

I also don't know why it happens because communication with Pachube worked without any single problem for last more than 3 weeks, all the time in 24/7. So the change from nothing to sizing every 15-90 minutes made me nervous thinking if the EtheShield went wrong or what it can be.

First "solution" I applied was the Reset procedure inserted after "Serial.print("connection failed");". This, with one extra wire, reset the UNO and EtheShield without problem, but ....
Second "solution" which seems to work from yesterday evening is adding "client.stop()" after Serial.print("connection failed");" so the new code looks like:
...
else{
Serial.print("connection failed");"
client.stop();
}

So that's from my end for now. I don't know the reason but I think that above change to code may avoid the endless loop and doesn't bring nothing wrong into the functionality of the communication to Pachube.

Your comments are highly invited ....
Cheers

Here's the latest code I used for pachube on a device I haven't posted about yet.

void sendPachubeData(){
  char dataBuf[100];
  
  if(pachube.connected()) // already trying to get data, just leave
    return;
  if(round(realPower) == 0.0)  // don't send invalid data
    return;
  tNow = now();
  strcpy_P(Dbuf2,PSTR("Data at %02d:%02d:%02d: "));  // Debugging
  sprintf(Dbuf,Dbuf2,hour(tNow),minute(tNow),second(tNow));
//  Serial.print(Dbuf);
  // construct the data buffer so we know how long it is
  int poolMotorState = 0;
  if(strcmp(poolData.motorState,"High") == 0)
     poolMotorState = 2;
  else if(strcmp(poolData.motorState,"Low") == 0)
    poolMotorState = 1;
  strcpy_P(Dbuf2, PSTR("%d,%d,0.%d,%d.%02d,%d.%02d,%d.%02d,%d,%d,%d,%d"));
  sprintf(dataBuf,Dbuf2,
    (int)round(realPower),
    (int)round(apparentPower),
    (int)(powerFactor*100),
    (int)rmsCurrent,
    (int)(((rmsCurrent+0.005) - (int)rmsCurrent) * 100),
    (int)rmsVoltage,
    (int)(((rmsVoltage+0.005) - (int)rmsVoltage) * 100),
    (int)(frequency), 
    (int)(((frequency+0.005) - (int)frequency) * 100),
    (ThermoData[0].currentTemp + ThermoData[1].currentTemp)/2,
    (int)round(outsideTemp),
    poolMotorState, poolMotorState == 0 ? 10 : poolData.poolTemp);
  Serial.println(dataBuf);
//return;
  strcpy_P(Dbuf,PSTR("Pachube Connecting...")); 
  Serial.print(Dbuf);
  if(pachube.connect()){
    strcpy_P(Dbuf,PSTR("OK"));
    Serial.println(Dbuf);
    tNow = now();
    strcpy_P(Dbuf,PSTR("PUT /api/9511.csv HTTP/1.1\n"));
    pachube.print(Dbuf);
    strcpy_P(Dbuf,PSTR("Host: www.pachube.com\n"));
    pachube.print(Dbuf);
    strcpy_P(Dbuf,PSTR("X-PachubeApiKey: stuffinhere\n"));
    pachube.print(Dbuf);
    strcpy_P(Dbuf,PSTR("Content-Length: "));
    pachube.print(Dbuf);
    pachube.println(strlen(dataBuf), DEC); // this has to be a println
    strcpy_P(Dbuf,PSTR("Content-Type: text/csv\n"));
    pachube.print(Dbuf);
    strcpy_P(Dbuf,PSTR("Connection: close\n"));
    pachube.println(Dbuf);
    pachube.println(dataBuf);   
    pachube.stop();
    pachubeUpdateTime = now();
  }
  else {
    pachube.stop();
    while(pachube.status() != 0){
      delay(5);
    }
    strcpy_P(Dbuf,PSTR("failed"));
    Serial.println(Dbuf);
  }
}

This works every time the actual board is working. I also have code around the ethernet board itself to make sure it is working that is separate from this stuff. For example, this is the code to reset the ethernet board:

void ethernetReset(){

  pinMode(rxSense, INPUT);  //for stabilizing the ethernet board
  pinMode(ethernetResetPin,INPUT);
  while(1){
    digitalWrite(ethernetResetPin, HIGH);
    pinMode(ethernetResetPin, OUTPUT);
    digitalWrite(ethernetResetPin, LOW);  // ethernet board reset
    delay(100);
    digitalWrite(ethernetResetPin, HIGH);
    delay(5000);
    // now, after the reset, check the rx pin for constant on
    if (ethernetOk())
      return;
    delay(100);
  }
}

Keep in mind that this depends on the hardware mods I did to the board so that I can reset it separately from the rest. Since I'm using an arduino 2560 that has the watchdog timer bug in the boot loader I can't do a regular watchdog reset. I use one of the timer interrupts instead and do a call to address 0 to reset the board. That took some time to figure out.

I also have a little routine that I call that watches the RX led on the ethernet board. It seems in my case that when the board hangs up the RX led is solid on. So I hooked a wire to it and watch it with an arduino pin. If the led is on too many times in a timing loop, it means the board is hung up. So I use the reset routine above to reset it.

boolean ethernetOk(){
  int cnt = 10, result;

  cnt = 10;
  result = 0;
  while (cnt-- != 0){  // simply count the number of times the light is on in the loop
    result += digitalRead(rxSense);
    delay(50);
  }
  strcpy_P(Dbuf,PSTR("Ethernet setup result "));
  Serial.print(Dbuf);
  Serial.println(result,DEC);
  if (result >=6)      // experimentation gave me this number YMMV
    return(true);
  return(false);
}

Like your experience, I found that one MUST close the connections or the darn thing hangs up. I guess I was extremely lucky to have it fail immediately so that I had to work up solutions before I started relying on it.

On the new device that updates pachube, there is still one problem I have to deal with. Sometimes pachube itself stops responding and I have trouble telling if it was pachube messing up or the ethernet board failing. I'm still thinking about that problem and suspect it will be easy to fix by just checking the pachube response for 'OK' and also looking at some of the other web interactions the board is doing to see if they are working ok. If all else fails, I'll just interogate my router to see if it's alive. That way I can just wait for pachube to come back up. I've had this problem three times now which is not bad for several months of data transmission.

One other thing I want to mention is that I use the TimeAlarms library to time the updates to pachube instead of counting milliseconds or something like that. I set an alarm to fire every so often and just go do other stuff as necessary keeping the values I want to send to pachube updated. This way I don't have anything hanging me up because Ijust let the alarm fire and it takes care of that kind of thing for me. This trick allows me to have a display that steps through the various values and little LEDs that tell me what is going on at a glance. I am using the TimeAlarm libray as a task controller and then just set up the various tasks.

Too much information???

Very much information but they are very valuable ones :slight_smile: Fortunately I have minimum of your experience with different kind of drop-outs and hangs. Partially because I'm newbie to this world, partially because (I hope) I use the latest Arduino products as UNO and EtheShield v5 which maybe take a more care to handle these hangs. I read many articles how to adapt the former Ethernet Shield model to work with Arduino including burning of the new bootloader. But till Tuesday seemed it is over :smiley: ....

Nevermind in my case EtheShield was on all the time connected to LAN, only what I couldn't see was the TX LED blinking while sending data. If I compare my contribution to Pachube I take it now as the tests only which don't comply with your projects and home automation at all.

Thanks a lot for your contribution

P.S I have also Mega2560 on my desk and I'm starting with XBees ... :slight_smile: So maybe we will be in contact again :smiley:
Vladimir