Automated Reptile Control System(webserver, Data Logging, RTC and much more)

Do you do the below with your Strings before exiting the subroutines in which the Strings were used?

String datastring = "";

zoomkat:
Do you do the below with your Strings before exiting the subroutines in which the Strings were used?

String datastring = "";

no i do not. i thought that when a function is ended, any local variables used within the function were cleared.

i will give that a try though

i am also wondering if it is something with the ethernet shield. i have been able to crash the arduino if i purposfully send lots of information over to it really fast. it just stops responding.

i am also wondering why my arduino freezes when i have the watchdog enabled. should that not reset the system?

i tried to use the "Wstring.cpp" and "Wstring.h" from Paul Stoffregen's teensey system. everything compiled, but it would no longer function and i have not determined what is causing it to not work.

no i do not. i thought that when a function is ended, any local variables used within the function were cleared.

From what others have discussed, that is the premis of Strings issue, that the last String memory location used in the function is not automatically cleared as expected, resulting in that memory location no longer being available for use. May or may not help.

zoomkat:

no i do not. i thought that when a function is ended, any local variables used within the function were cleared.

From what others have discussed, that is the premis of Strings issue, that the last String memory location used in the function is not automatically cleared as expected, resulting in that memory location no longer being available for use. May or may not help.

i have read that as well.

i have added to my code the lines to set all used strings to "" before the function ends. so far the arduino has 14 hours of up time.

i am in the process of trying to remove all uses of strings within my code. the main area strings are used is in processing of the data submitted by a web-page form in the function "sendPage"

void sendPage(EthernetClient & client, int nUriIndex, BUFFER & requestContent, char * global_pUri)
{
  if (nUriIndex < NUM_PAGES){
    wdt_reset();
    String datastring = "";
    
    //page was received, need to determine if the page returned ay form data
    //if form data was was received we need to process it on a per page process
    //page 1 does not have any form data
    //page 2 have 17 form entries for different temperature settings. these settings need to be extracted from the requestcontent 
    //variable. once the data is extracted it needs to be saved to the correct variables and then to EEPROM so the setting take effect
    //between power outages
    
    /***************************************************************
    //PAGE 2 DATA PROCESSING
    ***************************************************************/
    
    if (nUriIndex == 1){//page 2
      //convert the request content variable from an array into a string
      
      for (byte x=0; x < 129; x++){
        datastring.concat(requestContent[x]);
      }  
      if (datastring.substring(0,2)=="1="){ //tests if the page has had a form submitted with data or not
          for (byte x = 1; x<18; x++){ //page 2 contains 17 data entry text boxes, so we want to run through all of them
            String temp1 = ""; //initilize temp variables to clear any previous data
            String temp2 = "";
            //the format for the submitetd form data is:
            //1=entry1&2=entry2&3=entry3 ect... 
            //we want to use the sub string command, so we need to know the index locations for where the desired data entry index is
            //along with the data entry index of the next entry. 
            temp1.concat(x);
            temp1.concat("=");
            temp2.concat(x+1);
            temp2.concat("=");
            //we need to see if the current entry is below 10. if the entry is below 10, we only need to skip 2 entries, one for the entry number
            //and another for the equals sign. however if the entry is above 10, we need to skip 3 entries becuase of the two digits for the entry
            //and another entry for the equals sign
            if (x<10){
              char valueArray[datastring.substring(datastring.indexOf(temp1) + 2,datastring.indexOf(temp2) -1).length() + 1];
              datastring.substring(datastring.indexOf(temp1) + 2,datastring.indexOf(temp2) -1).toCharArray(valueArray, sizeof(valueArray));
              if (x == 1){      //if the currently processing entry is entry 1
                if (atoi(valueArray)>= min_temp_setting && atoi(valueArray) <= max_temp_setting){    //is the entred entry within the allowable bounds?
                  INCSGDTON = atoi(valueArray);    //converts the string value of the entred data into a interger
                  if (EEPROM.read(INCSGDTONEEPROMADDR)!=INCSGDTON){    //if the user has not changed the entry from what is already in memory, then we do not want to save to EEPROM again as the EEPROM has a shorter life time
                    EEPROM.write(INCSGDTONEEPROMADDR, INCSGDTON);    //since the user has entered a new number, we now need to save this to memory
                    INCSGDTON_incorrect = false;    //because a valid entry was entered, we do not want the web-page to display "Invalid Entry!"
                  }
                }else{
                  INCSGDTON_incorrect = true;//if the user did enter an invalid value then we want the web-page to display "invalid entry!" when the substitution function runs
                }
              }else if (x == 2){
                if (atoi(valueArray)>= min_temp_setting && atoi(valueArray) <= max_temp_setting){
                  CSGDTOFF = atoi(valueArray);
                  if (EEPROM.read(CSGDTOFFEEPROMADDR)!=CSGDTOFF){
                    EEPROM.write(CSGDTOFFEEPROMADDR, CSGDTOFF);
                    CSGDTOFF_incorrect = false;
                  }
                }else{
                  CSGDTOFF_incorrect = true;
                }
              }else if (x == 3){
                if (atoi(valueArray)>= min_temp_setting && atoi(valueArray) <= max_temp_setting){
                  MSGDTON = atoi(valueArray);
                  if (EEPROM.read(MSGDTONEEPROMADDR)!=MSGDTON){
                    EEPROM.write(MSGDTONEEPROMADDR, MSGDTON);
                    MSGDTON_incorrect = false;
                  }
                }else{
                  MSGDTON_incorrect = true;
                }
              }else if (x == 4){
                if (atoi(valueArray)>= min_temp_setting && atoi(valueArray) <= max_temp_setting){
                  MSGDTOFF = atoi(valueArray);
                  if (EEPROM.read(MSGDTOFFEEPROMADDR)!=MSGDTOFF){
                    EEPROM.write(MSGDTOFFEEPROMADDR, MSGDTOFF);
                    MSGDTOFF_incorrect = false;
                  }
                }else{
                  MSGDTOFF_incorrect = true;
                }
              }else if (x == 5){


ect....

i turn the submitted data into a string so i can use this code

char valueArray[datastring.substring(datastring.indexOf(temp1) + 2,datastring.indexOf(temp2) -1).length() + 1];
              datastring.substring(datastring.indexOf(temp1) + 2,datastring.indexOf(temp2) -1).toCharArray(valueArray, sizeof(valueArray));

to extract the different entry values

i have not tested it yet, but i am going to try using the following demonstration code instead

#include <.h>
void setup()
{
  // start serial port at 9600 bps:
  Serial.begin(9600);
}

void loop()
{
 //incomming string of data. string is actually 128 characters long, but this is the only section we care ot process. The "=%" indicates the end of the part of the strign we care about
 char str[] = "1=80&2=82&3=83&4=84&5=85&6=86&7=87&8=88&9=89&10=90&11=91&12=92&13=93&14=94&15=95&16=96&17=97&18=%";
 Serial.print("The string of submitted data is: ");
 Serial.println(str);
 for (byte x = 1;x<=17;x++){
   char ValueArray={NULL};
   PROCESSREQUESTCONTENT(x, str, &ValueArray);
   Serial.print("The value of entry # ");
   Serial.print(x);
   Serial.print(" is equal to: ");
   Serial.println(atoi((char *)ValueArray));
 }
}

void PROCESSREQUESTCONTENT(byte index, char * str, char * output){
 int pch=0;
 byte beginning=0;
 byte ending=0;
 byte counter=0;
 //locate where "%" is located so we know where to stop
 pch=atoi(strchr(str,'%'));
 
 //"pch-4" is required because pch indicates where the "%" sign is with the first character starting at 1. however the char array starts at 0. in addition we do not care about the "&17=" part of the ending.
 //the length of the "5" itself is removed because of the offset from starting point 1 to starting point 0
 //the "-4" is so the code ignores the "&18=" characters
 for (byte x=0;x<= pch-4;x++){
   if (strncmp((char *)str[x],"=",1)==0){
     beginning = x+1; //starting character position of actual data entry
   }else if (strncmp((char *)str[x],"&",1)==0){
     ending=x-1;//ending character position of actual data entry
     counter++;//increment counter so we can see if this is the data entry we care about
   }
   if (counter==index){//have we found the data antry we care about?
     counter = 0;
     for (byte i=0; i<=(ending-beginning);i++){//copy the data entry so we can use it
       output[i] = str[beginning + i];
       break;
     }
   }     
 }
  
}

wish me luck

Check in the playground for memory usage. There are a couple of routines there that can tell you how much you have left at any given time. I use them extensively to keep away from running out of memory. You can put them in temporarily to measure what you have left and isolate a problem area then fix whatever is needed.

In a couple of hard cases, I put them in the loop() and automatically rebooted the board to clean things out when free memory reached a low number. This was a stop gap measure I used while I tried to figure out what the heck was going on. These days, I try desperately to avoid using Strings and use the old reliable strcat, strcmp, etc. instead.

Something like:

#include <avr/pgmspace.h>

void showMem(){
  uint8_t * heapptr, * stackptr;
  
  strcpy_P(Dbuf,PSTR("Mem = "));
  Serial.print(Dbuf);
  stackptr = (uint8_t *)malloc(4);   // use stackptr temporarily
  heapptr = stackptr;                // save value of heap pointer
  free(stackptr);                    // free up the memory again (sets stackptr to 0)
  stackptr =  (uint8_t *)(SP);       // save value of stack pointer
  Serial.println(stackptr - heapptr);
}

// and then sprinkle these around to get a feel for what is going on

    showMem();

Or maybe you would prefer:

#include <MemoryFree.h>

void showMem(){
  strcpy_P(Dbuf,PSTR("Mem = "));
  Serial.print(Dbuf);
  Serial.println(freeMemory());
}

// and then, as above, I sprinkle these around to see what is happening

showMem();

I use both in different applications. Don't have a preference either way.

i do already have a free memory routine that displays the amount of memory on one f the web-pages. i have never seen it go below 4200 bytes. i will have to post the exact routine i am using later, in case it is displaying the wrong result

You probably need to sprinkle them around more places. I've had to go so far as to put them before the return of many routines to get a clue what was happening. Sometimes when you're recursing, the level of recursion can get so deep that you run out of memory. If it returns, the memory unwinds back to a good level. This can happen when some routine allocates and then calls something else as well.

You can get lost and not have any idea what the heck happened until you really start to dissect it.

But then, you could also just be running past the end of some buffer somewhere and clobbering yourself. These problems can be tough to find. Been there, done that...sigh.

Fantastic job! Congratulations!

One proposal, and it will be a perfect system. DS18B20 sensors can be connected on a single cable. It is more convenient.
But then you need to know their address. Here is a sample of how addresses are assigned sensors: Arduino - Networked Temperature Monitor | Page 2 | Homebrew Talk - Beer, Wine, Mead, & Cider Brewing Discussion Forum
The combination of both programs get a great system.

My English is very bad, but I hope you got the idea... :slight_smile:

alvydas001:
Fantastic job! Congratulations!

One proposal, and it will be a perfect system. DS18B20 sensors can be connected on a single cable. It is more convenient.
But then you need to know their address. Here is a sample of how addresses are assigned sensors: http://www.homebrewtalk.com/f51/arduino-networked-temperature-monitor-340294/index2.html#post4286666
The combination of both programs get a great system.

My English is very bad, but I hope you got the idea... :slight_smile:

i see nothing wrong with your English.

i did think about having them on one bus, however per the Dallas Semiconductor data sheets, a star topology is not recommended. i wanted the least possible issues with the data feedback, and so i choose to have each sensor on its own "one wire master" giving me the best possible electrical setup to reduce error.

and with so many I/O on the MEGA i had no worries about running out of pins.

I have project and use 10 DS18B20 sensors to connected on one cable (parasite power mode). Longest distance of 16-18 meters (2 pieces), 10-12 meters (5 units), the other 3-4 meters. Runs great. The project is here: Saulė Virė - Saulės kolektoriai, valdikliai, pasidaryk pats (and here- http://www.ksduino.org/?devices&device_id=3331) (works half the year, but the site still has not been made).

i am also wondering if it is something with the ethernet shield. i have been able to crash the arduino if i purposfully send lots of information over to it really fast. it just stops responding.

I think the 5100 chip is capable of only four sockets, so if a socket is not properly allowed to close, it may become unavailable for future use.

i am also wondering why my arduino freezes when i have the watchdog enabled. should that not reset the system?

I haven't looked at the watchdog code. Is it a proven design?

i tried to use the "Wstring.cpp" and "Wstring.h" from Paul Stoffregen's teensey system. everything compiled, but it would no longer function and i have not determined what is causing it to not work.

I think those librarys are included in the latest arduino core design. If you use the string functions, under some conditions it is said that used memory is not automatically released, and might eventually deplete available memory, causing a freeze. Recently various patch solutions have been posted

I have completly removed the string library from my code and replaced it with char arrays. I am testing the code now to see if it crashes, resets, or freezes.

Luckily removing the string class actually saves about 4k of flash and I appear to be using about 600 fewer bytes of SRAM.

I will post the code shortly.

I have found a bug with my "system uptime calculator" where it calculates wrong when transitioning from February to march. I wi have to look onto that issue

Go to the playground and look at the Time and TimeAlarms library. You don't have to actually set the time, but you already have an RTC, so it should be easy to do. In the Time header file are a lot of macros including days between dates and such. Also, out on the web are a ton of days between dates routines that you can use. Most of the samples also have the seconds between date that will work for you as well, although saving the unix time when you start and just subtracting it from the current time will give you seconds pretty easily.

i have a few other bugs now, because of the removal of strings, my ability to download log files, and to delete the log files no longer works. i will look into that now as well.

edit: those are fixed now 8)

still going to look into the "system up time" calculator

draythomp:
Go to the playground and look at the Time and TimeAlarms library. You don't have to actually set the time, but you already have an RTC, so it should be easy to do. In the Time header file are a lot of macros including days between dates and such. Also, out on the web are a ton of days between dates routines that you can use. Most of the samples also have the seconds between date that will work for you as well, although saving the unix time when you start and just subtracting it from the current time will give you seconds pretty easily.

thanks for the info, i will take a look at those.

edit: latest code
https://www.dropbox.com/s/4s3vzc5gopik7re/Arduino3-1-2013.zip

Hi,

could you tell me what moisture sensor have you used?

thanks!

i am using Safety and Productivity Solutions | Honeywell

Hi,

No humidity sensor attached so far, just checking if all relays can be controlled in manual mode
but for unknown reason humidifier does not turn off in manual mode.
Any ideas ?

thanks!

ps. Talking about RTC, which pin SCL and SDA is connected on the arduino board please ?

On a standard Uno it is SDA to A4 and SCL to A5. Note that it is pins 20, 21 on a Mega. Most Megas have this marked on the board.

mkcinek:
Hi,

No humidity sensor attached so far, just checking if all relays can be controlled in manual mode
but for unknown reason humidifier does not turn off in manual mode.
Any ideas ?

thanks!

sorry for not responding sooner, been really busy.

to answer your question, i had the same issue. i know i fixed it, and if i recall there is a typo in the version of code i posted. when i get home tonight i will post the latest. that WILL fix that issue. :grin:

mkcinek:
ps. Talking about RTC, which pin SCL and SDA is connected on the arduino board please ?

i have the RTC connected to pins 20 and 21 on the MEGA

i would also like to update that the random system reboots and freezes have been fixed. now that i have removed the string class the system is stable. i also removed some code that had the unit auto-update its time every 24 hours. without that, you can still get the correct time by manually initiating a time update. the system has now been going for 25 days while logging data without issues.