Go Down

Topic: Flowmeter Node: RFM12b + Ethernet Shield (Read 2943 times) previous topic - next topic

wisof

Jun 28, 2011, 08:47 am Last Edit: Jun 28, 2011, 02:05 pm by wisof Reason: 1
Hello all,
I need a bit of technical guidance for my project.  I am measuring the water consumption in a hotel.  For each two rooms I have 4 flowmeters connected to one JeeNodehttp://jeelabs.net/projects/hardware/wiki/JeeNode, each measuring hot and cold.  I send this data to a central node via a RFM12b.  The central node is an Arduino UNO with the 'official' Ethernet Shield and a RFM12b.  This is where I need guidance.

I have done range tests and the reception is phenomenal.  There is a blindspot when the reception attempts to go though the structural core of the hotel with the elevators and such, but this is expected, and not too bad for a very low-cost, low power RF!  Mostly everything works, until I try to use Ethernet to send data to Pachube.  The whole thing works very well for about 10 minutes (have seen up to 25minutes), then it seems the RFM12b can no longer receive, and Ethernet sends nothing out because I do not reach the code where I process the Ethernet sending.

To note, I am using the RF12 library http://jeelabs.net/projects/cafe/wiki/RF12 from JeeLabs which allows me to use Pin 9 on the Arduino UNO as the chip select pin for the RFM12b.

I have read many posts on issues regarding the Ethernet Shield and other SPI devices.  This one in particular from the old forum, seemed to shed some light: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1287657487.  

It seems that by editing the W5100.cpp functions to disable/enable interrupts 'freakylabs' was able to get things working.  I did attempt the edits, but I am still getting 10-25 minutes of sending and then I can no longer receive on the RFM12b.  

I would very much appreciate if someone could guide me in how I can troubleshoot such a situation.  Its driving me nuts, and I will not accept that the hardware cannot work together if it works for some amount of time, but then fails.  Is there something I can insert in the code to check if things are going well?   I am now trying this for two weeks without getting any further with stability.  I am attaching the receiving code. This code, without Ethernet functions, can receive for days and days at a time (never had an error after many days of stability).  Your help would be greatly appreciated.

Receiving Code:
Code: [Select]

#include <SPI.h>
#include <Ethernet.h>
#include "ERxPachube.h"

#include <Ports.h>
#include <RF12.h>

#include <Wire.h>
#include <RTClib.h>


unsigned long time=0;
RTC_Millis RTC;

/////////////NETWORK DATA////////////////////////////////
byte mac[] = {
 0x90, 0xA2, 0xDA, 0x00, 0x4E, 0xBE }; // make sure this is unique on your network

byte ip[] = {
 192, 168, 0, 177   };                  // no DHCP so we set our own IP address
//byte ip[] = { 10,0,1,199 };
#define PACHUBE_API_KEY "PACHUBE API HERE" // fill in your API key PACHUBE_API_KEY
#define PACHUBE_FEED_ID 28839 // fill in your feed id
ERxPachubeDataOut dataout(PACHUBE_API_KEY, PACHUBE_FEED_ID);
void PrintDataStream(const ERxPachube& pachube);
void PrintString(const String& message) ;
/////////////////////////////////////////////////////////



//analogous data structure for incoming data
typedef struct {
 int cnt;
 unsigned long time; //ellapsed time since last reading
 byte bat;
 int p1, p2, p3, p4; //pulses
}  
Payload;
Payload inData;

//////////define class for flowmeter calculations
typedef struct {
 int p[3]; //current pulses
 int tp[3]; //total pulses
 float fr[3]; //flow rate
 float t[3]; //total liters
}  
FlowNode;
FlowNode fn1, fn2, fn3, fn4;

void setup () {

 Serial.begin(57600);
 Serial.println("Started Setup...");
 RTC.begin(DateTime(__DATE__, __TIME__));

 rf12_config();
 Ethernet.begin(mac, ip);
 


 /////////////NETWORK DATA////////////////////////////////
 dataout.addData(0); //Node 2 pulses
 dataout.addData(1); //Node 2 total pulses
 dataout.addData(2); //Node 2 flow rate
 dataout.addData(3); //Node 2 total consumption


 delay(1000);
 Serial.println("Ended Setup...");
}

static void consumeInData () {

 DateTime now = RTC.now();

 Serial.print(now.year(), DEC);
 Serial.print('/');
 Serial.print(now.month(), DEC);
 Serial.print('/');
 Serial.print(now.day(), DEC);
 Serial.print(' ');
 Serial.print(now.hour(), DEC);
 Serial.print(':');
 Serial.print(now.minute(), DEC);
 Serial.print(':');
 Serial.print(now.second(), DEC);
 Serial.println();

 inData = *(Payload*) rf12_data;
 Serial.println("+++++++++++++++++++++++++++++++++++++++");

 Serial.print("id: ");
 Serial.print(rf12_hdr, DEC); //print node id
 Serial.print(" msg:");
 Serial.print(inData.cnt); //print the message number
 Serial.print(" time:");
 Serial.print(inData.time); //print the amount of time the remote node measured water flow
 Serial.print(" bat:");
 Serial.print(inData.bat, DEC); //alert if battery low
 Serial.println();
 
 switch (rf12_hdr) {
 case 1:

   //process current pulses
   fn1.p[0] = inData.p1;
   fn1.p[1] = inData.p2;
   fn1.p[2] = inData.p3;
   fn1.p[3] = inData.p4;
   //process total pulses
   fn1.tp[0] += inData.p1;
   fn1.tp[1] += inData.p2;
   fn1.tp[2] += inData.p3;
   fn1.tp[3] += inData.p4;
   //process flow rate
   fn1.fr[0] = processRate(inData.p1, inData.time);
   fn1.fr[1] = processRate(inData.p2, inData.time);
   fn1.fr[2] = processRate(inData.p3, inData.time);
   fn1.fr[3] = processRate(inData.p4, inData.time);
   //process total liters
   fn1.t[0] += processTotal(fn1.fr[0], inData.time);
   fn1.t[1] += processTotal(fn1.fr[1], inData.time);
   fn1.t[2] += processTotal(fn1.fr[2], inData.time);
   fn1.t[3] += processTotal(fn1.fr[3], inData.time);

   dataout.updateData(0, fn1.p[1]);
   dataout.updateData(1, fn1.tp[1]);
   dataout.updateData(2, fn1.fr[1]);
   dataout.updateData(3, fn1.t[1]);
   break;
 case 2:

   //process current pulses
   fn2.p[0] = inData.p1;
   fn2.p[1] = inData.p2;
   fn2.p[2] = inData.p3;
   fn2.p[3] = inData.p4;
   //process total pulses
   fn2.tp[0] += inData.p1;
   fn2.tp[1] += inData.p2;
   fn2.tp[2] += inData.p3;
   fn2.tp[3] += inData.p4;
   //process flow rate
   fn2.fr[0] = processRate(inData.p1, inData.time);
   fn2.fr[1] = processRate(inData.p2, inData.time);
   fn2.fr[2] = processRate(inData.p3, inData.time);
   fn2.fr[3] = processRate(inData.p4, inData.time);
   //process total liters
   fn2.t[0] += processTotal(fn2.fr[0], inData.time);
   fn2.t[1] += processTotal(fn2.fr[1], inData.time);
   fn2.t[2] += processTotal(fn2.fr[2], inData.time);
   fn2.t[3] += processTotal(fn2.fr[3], inData.time);

   dataout.updateData(0, fn2.p[1]);
   dataout.updateData(1, fn2.tp[1]);
   dataout.updateData(2, fn2.fr[1]);
   dataout.updateData(3, fn2.t[1]);
   break;
 }

 cli();//is this necessary?
 int status = dataout.updatePachube();
 Serial.print("sync status code <OK == 200> => ");
 Serial.println(status);
 sei();//is this necessary?

 Serial.println(" ");
}



void loop () {

 if (rf12_recvDone() && rf12_crc == 0 && rf12_len == sizeof inData) {
   memcpy(&inData, (byte*) rf12_data, sizeof inData);
   // optional: rf12_recvDone(); // re-enable reception right away
   consumeInData();
   time = millis();
 }
 else{

   if((millis()-time)>10000){
     Serial.println("something wrong...");

   }
 }

}

float processRate(int p, unsigned long time){
 // Pulse frequency (Hz) in Horizontal Test= 7.5Q, Q is flow rate in L/min. (Results in +/- 3% range)
 //since we are receiving from the nodes every 5000 milliseconds,
 //we divide 5000/1000 to get 5 seconds...
 return (p/(time/1000)) / 7.5;
}

float processTotal(float fr, unsigned long time){

 //first get the flow rate per second,
 //second multiply by the number of seconds which have ellapsed
 return (fr/60) *(time/1000);

}




pantonvich

check if your running out of ram

http://jeelabs.org/2011/05/22/atmega-memory-use/

I would also look at a buffer overflow on the postback data from pachude

robtillaart


Another test is to leave out the ethernetshield altogether and log the usage to serial port. That way you can verify the receiving part works 100% and doesn't block.

If you want you can post the code so the forum takes a look at it.

Which version of the ethershield do you use?
There were some problems with older ones I have a project that resets every 15 minutes to keep the ethershield working. An on board RTC 1307 keeps track of time.

Rob
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

wisof

Hello!
Sincere thanks for the replies.

@pantonvich
Thank you for the links on checking available RAM.  In the sketch posted above, I did take a lot of the print statements out to simplify the code.  That being said, after running it, I came out to about 487 bytes.  I did a bit of editing of the print statements as was suggested in the next post of JCW's blog.  This is very interesting and something I was overlooking completely.
Here is a rundown on the bytes:
1014 - Just before leaving the setup method:
621 - After receiving for the first time and printing out all of the values
667 -  When things 'go wrong' and I have not received anything for more than 10 seconds

Quote
I would also look at a buffer overflow on the postback data from pachude[sic]


How would one go about this?

It is nice to see that a lot of RAM is used when I print everything out (all of the println() won't be necessary when this is installed).  I was not aware printing strings was so memory intensive.

Is running with 600 bytes of RAM an issue?  It is curious that when I do not receive then I still have 600+ left...

@robtillaart
Before I tried the Ethernet Shield, I was setting up all of the flowmetering and transmitting.  From my tests, I could receive indefinitely.  I left the system running several nights.  The only reason it stopped working was because I needed to continue working with the nodes.

I have posted the reception code above.  I am receiving a packet identical in structure to the one described here:
Code: [Select]
//analogous data structure for incoming data
typedef struct {
 int cnt;
 unsigned long time; //ellapsed time since last reading
 byte bat;
 int p1, p2, p3, p4; //pulses
}  
Payload;
Payload inData;


I purchased the Ethernet shield a few weeks ago.  It has the newish Arduino logo and has an onboard SD card slot.  Looks identical to this one: http://www.sparkfun.com/products/9026

Interesting workaround using the resetting.  How did you go about that?

Another curiosity is that when I go into the 'something wrong' part of the code, I can still send to Pachube succesfully.  It seems the RFM12b is getting knocked out somehow.  What would be the way to troubleshoot if the RFM12b and the Ethernet are conflicting?  Again,  SPI SS for RFM12b is routed to pin 9 in the RF12 library (without Ethernet this works just fine, with Ethernet it works for 10-25 minutes).

robtillaart


Quote
Interesting workaround using the resetting.  How did you go about that?


I did it the not so nice way, but it worked well for my sketch. I declared : 

void(* resetFunc) (void) = 0; //declare reset function @ address 0

And somewhere in loop()

if (millis() > 15*60*1000) resetFunc();  // a reset every 15 minutes

The problem with this kind of reset is that it is not a 100% reset and initialization, registers may keep old values especially when using timers, so this method is depreciated . The proper way to do a reset  is by means of the watchdog reset. If you search the forum you will find it discussed




Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

pantonvich

#5
Jun 28, 2011, 11:23 pm Last Edit: Jun 28, 2011, 11:44 pm by pantonvich Reason: 1
I'm using a different Ethernet shield then you - it put's all the returned postback into a char array - I don't think that it checks the sizeof of the char array so it overruns it if the returned postback stuff is larger then the length of the array. for example if I defined it at 200 chars then it would lock up but 500 worked. anyways I sort of found mine by trial and error - so I'm not sure how you track down buffer overflow's other then finding them...

draythomp

This is just a suspicion since I haven't seen the ethernet code, but as was stated before, you send something and something comes back from Pachube.  What are you doing with the return text from Pachube?  If you're using a buffer, is it big enough to handle a possible ton of data back (especially if there is an error somewhere)?  Or do you just read it out of the ethernet buffer and discard it?  You only have four active connections on the ethernet board so you could be using them all up if you aren't closing them after each interaction.  Do you have any buffers that you read data into?  If you do, these are prime suspects for killing the device.  Look to see if they are big enough and protected from running past the end while gathering data.

One complex way to check this kind of thing is to put something at the end of the buffer and see if it changes.  If your code gets to the stuff you put at the end it may give you a hint on where the problem is. 

To save on ram, look at the usage of PSTR() in the arduino documentation.  Statements like:

  strcpy_P(buffer,PSTR("Hello World"));
  Serial.print(buffer);

will save a lot of ram at the expense of code size, but we usually have plenty of code storage.  My experience has been that when I get down to around 200 or so bytes of ram left, I start seeing strange things happen.  You seem to be beyond that though.
Trying to keep my house under control http://www.desert-home.com/

wisof

#7
Jun 30, 2011, 01:32 am Last Edit: Jun 30, 2011, 01:49 am by wisof Reason: 1
@draythomp
Information from pachube is read and spit out.  The things I am storing are the total number of pulses, and the liter total.the hotel uses about 1,000,000 liters per month, so I think I will be ok measuring those values.

I have implemented PSTR() code at your suggestion...very straightforward and impressive the amount of RAM it clears up.

I am attaching a very simplified version of the code I use to receive indefinitely.  This is without any calls to ethernet lib:
Code: [Select]
//INCLUDES
#include <Ports.h>        //JeeLabs
#include <RF12.h>         //RFM12b driver
#include <Wire.h>        
#include <RTClib.h>       //JeeLabs RTC


//Globals definition
unsigned long time=0;
RTC_Millis RTC;

//analogous data structure for incoming data
typedef struct {
 int cnt;
 unsigned long time; //ellapsed time since last reading
 byte bat;
 int p1, p2, p3, p4; //pulses
}  
Payload;
Payload inData;


void setup () {

 Serial.begin(57600);
 RTC.begin(DateTime(__DATE__, __TIME__));

 rf12_config();

}

static void consumeInData () {


 DateTime now = RTC.now();

 Serial.print(now.year(), DEC);
 Serial.print("/");
 Serial.print(now.month(), DEC);
 Serial.print("/");
 Serial.print(now.day(), DEC);
 Serial.print(" ");
 Serial.print(now.hour(), DEC);
 Serial.print(":");
 Serial.print(now.minute(), DEC);
 Serial.print(":");
 Serial.print(now.second(), DEC);
 Serial.println();

 inData = *(Payload*) rf12_data;

 Serial.print("+++++++++++++++++++++++++++++++++++++++");
 Serial.println();
 Serial.print("id: ");
 Serial.print(rf12_hdr, DEC);
 Serial.print(" msg:");
 Serial.print(inData.cnt);
 Serial.print(" time:");
 Serial.print(inData.time);
 Serial.print(" bat:");
 Serial.print(inData.bat, DEC);
 Serial.println();

 Serial.print(" p1: ");
 Serial.print(inData.p1);
 Serial.print(" p2: ");
 Serial.print(inData.p2);
 Serial.print(" p3: ");
 Serial.print(inData.p3);
 Serial.print(" p4: ");
 Serial.print(inData.p4);
 Serial.println();
}

void loop () {

 if (rf12_recvDone() && rf12_crc == 0 && rf12_len == sizeof inData) {
   memcpy(&inData, (byte*) rf12_data, sizeof inData);
   // optional: rf12_recvDone(); // re-enable reception right away
   consumeInData();
   time = millis();
 }
 else{

   if((millis()-time)>10000){
     Serial.print("something wrong...");

     delay(2000);

   }
 }

}


I am attaching the code with ethernet.

The code base the same as above, but I borrowed a lot of code from:http://www.designspark.com/content/pachube-making-reliable-connections-arduino because I liked the design of the code.  Note that ALL examples I have tried (included Pachube sketches with ethernet lib and pachube libraries) all result in more or less the same behavior which is: arduino starts, receives from RFM12b, sends data out to pachube, is successful for a while, but then (5-25) minutes later, it seems something disables the ability to receive.

What would be a straightforward test to understand if the Ethernet Shield (+lib) and the RFM12b are conflicting at some point?  For example, when I cannot receive, right now I go into an 'else' condition that prints 'something is wrong.'  Could I do something there to understand why I can no longer receive or to recover?

pantonvich

I would put serial print statements at the beginning and ending of all your functions: "1s", "1e", "2s" etc - the last item that shows up when it locks up is then the point you can put even more serial print statements...

draythomp

I looked through the code you attached and you're totally correct, the return response from pachube is not the problem.  I didn't go into depth looking at your state machine, but the cursory look I did didn't turn up anything that would cause me suspicions.   I haven't worked on the RF stuff you're using so I don't know about that.  Net, nothing.

Since you're catching it after a timeout, is there something on the RF that can tell you anything?  A receive light or something?  The idea about printing where you last were is not a bad idea, it may help to see what the last thing that happened is.   Although the state machine could complicate the output too much.  One other thing, you have a nice state machine for handling the internet connection, could something like this help you with the RFM12 interaction.  It seems this is the device that is failing to respond and it could be an interrupt interaction where the ethernet board snatches the data away from the RF; this might require you to have a failure case for the RF parts.

Recovery is possible using the watchdog timer, but that solution sucks since you're hanging up so quickly.

Maybe someone familiar with the pieces I haven't worked with will speak up.
Trying to keep my house under control http://www.desert-home.com/

Go Up