Arduino Uno locks up semi-inconsistently

I’ve been working with Arduino Uno for a number months, so I’m not new, but not terribly experienced, either.

I run a small water system and have been using Arduino to provide remote monitoring capabilities for system pressure, reservoir level, time, and ambient temperature.

One Arduino is programmed to output sensor data to the serial window every two seconds. A second Arduino’s sole function is to turn a small air pump on and off every ten minutes to equalize pressure in the reservoir level air line. This second unit outputs to the serial monitor the times when the air pump is cycled.

Both Arduinos are connected to the same powered USB hub, which is connected to a scrap laptop. The second Arduino has an additional 9v power supply to provide adequate power to the air pump. Everything is powered through an APC battery backup typically used for desktop computers.

The problem: About once per day, on average, the Arduino with the sensors connected will lock up. Output to the serial window stops. I can close the window without problem, but I have to use the Windows Device Manager to disable the Arduino and re-enable it before it will again output to the serial monitor. The second Arduino has never, ever had this problem.

I have replaced the original Arduino with a completely new one, but same problem. I’ve also replaced the USB cable, and I added the battery backup in the interest of trying to smooth out potential power issues.

The key, I think, is that failure coincides with our system’s pressure pump shutting off. The last entry in the serial monitor always shows the system at its peak pressure. It will run fine for hours, up to a day or so, so it doesn’t fail every time the pump shuts off. I assume this is a power problem, but I’m a little perplexed that the second unit has never had this issue.

If anyone has experienced something similar or if you have suggestions, that’s what I’m here for and your help would be appreciated!

Running code below.

/*
  Water System Pressure Sensor
  
  script assembled by me on April 10, 2015
  script last changed by me on July 29, 2015
  
  assumption is that the system pressure will never go above 100 psi
  
  0-100 PSI sensor purchased from eBay:  http://www.ebay.com/itm/261260635816
  Output: 0.5V – 4.5V linear voltage output. 0 psi outputs 0.5V, 50 psi outputs 2.5V, 100 psi outputs 4.5V
  Pin 1 (notched) Vout, Pin 2 Ground, Pin 3 Vcc
 */
 
 /*
This example demonstrates how to read time, date, and other information
from the DS3232 realtime clock chip.

This example is placed into the public domain.
*/

#include <SoftI2C.h>
#include <DS3232RTC.h>

SoftI2C i2c(A4, A5);
DS3232RTC rtc(i2c);

const char *days[] = {
    "Mon, ", "Tue, ", "Wed, ", "Thu, ", "Fri, ", "Sat, ", "Sun, "
};

const char *months[] = {
    " Jan ", " Feb ", " Mar ", " Apr ", " May ", " Jun ",
    " Jul ", " Aug ", " Sep ", " Oct ", " Nov ", " Dec "
};

int sensorPin = A0;  // select the input pin for the pressure sensor
int sensorValue;  // integer variable to store the value coming from the sensor
float voltage;  // float variable to store voltage from calculation
float psi;  // float variable to store psi from calculation
float headFt; // float variable to store level of reservoir in feet from calculation

int sensorPin0 = A1;  // select the input pin for the SYSTEM pressure sensor
int sensorValue0;  // integer variable to store the value coming from the SYSTEM sensor
float voltage0;  // float variable to store SYSTEM voltage from calculation
float psi0;  // float variable to store SYSTEM psi from calculation
int minSecsBetweenEmails = 600; // 10 min
long lastSend = -minSecsBetweenEmails * 1000l;

void setup() {
  Serial.begin(9600); // initialize serial
}

void loop() {
  long now = millis();
  
      RTCTime time;
    RTCDate date;
    rtc.readTime(&time);
    rtc.readDate(&date);

    Serial.print("Time: ");
    printDec2(time.hour);
    Serial.print(':');
    printDec2(time.minute);
    Serial.print(':');
    printDec2(time.second);
    Serial.println();

    Serial.print("Date: ");
    Serial.print(days[RTC::dayOfWeek(&date) - 1]);
    Serial.print(date.day, DEC);
    Serial.print(months[date.month - 1]);
    Serial.print(date.year);
    Serial.println();

    Serial.print("Temp: ");
    int temp = rtc.readTemperature();
    if (temp != RTC::NO_TEMPERATURE) {
        Serial.print((temp / 4.0) * 1.8 + 32);
        Serial.println(" Fahrenheit");
    } else {
        Serial.println("not available");
    }

    Serial.println();
   
// read from the SYSTEM sensor and do the calculations
  sensorValue0 = analogRead(sensorPin0);  // read the value from the sensor
  voltage0 = sensorValue0 * (.004887); // maths to determine voltage based on sensorValue: sensorValue times (5v divided by 1023 bit output range equals .004887)
  psi0 = ((voltage0 - .49) * 25); // maths to determine PSI based on voltage: (sensor voltage minus minimum voltage) times (100 PSI divided by 4v range)
// output to the serial monitor SYSTEM information
  Serial.println("********************************************"); // display header
  Serial.print("System Pressure = ");  // display label
  Serial.println(psi0, 1);  // display psi and CR
  // read from the sensor and do the calculations
  sensorValue = analogRead(sensorPin);  // read the value from the sensor
  voltage = sensorValue * (.0049); // maths to determine voltage based on sensorValue: sensorValue times (5v divided by 1023 bit output range equals .004887)
  psi = ((voltage - .16) * 1.61); // maths to determine PSI based on voltage: (sensor voltage minus minimum voltage) times (7.25 PSI divided by 4.5v range)
  headFt = psi / .43; // maths to determine head feet based on psi: one foot head equals .43 psi
// output to the serial monitor and wait a sec before starting over again
//  Serial.print("Sensor Value = ");  // display label
//  Serial.println(sensorValue);  // display sensorValue and CR
//  Serial.print("Voltage= ");  // display label
//  Serial.println(voltage, 2);  // display voltage and CR
//  Serial.print("PSI= ");  // display label
//  Serial.println(psi, 2);  // display psi and CR
  Serial.print("ResLvl= ");  // display label
  Serial.println(headFt, 1);  // display reservoir level in feet and CR
  Serial.println("-------------------------------------------"); // display footer
  Serial.println(""); // send extra CR
  Serial.println(""); // send extra CR
  
    if ((psi0>53) && (psi0<=57))
    {
      if (now > (lastSend + minSecsBetweenEmails * 1000l))
      {
        Serial.println("Info! Under pressure: ");
        Serial.println(psi0, 1);
        lastSend = now;
      }
    }
    else if ((psi0>=45) && (psi0<=53))
    {
      if (now > (lastSend + minSecsBetweenEmails * 500l))
      {
        Serial.println("Warning! Under pressure: ");
        Serial.println(psi0, 1);
        lastSend = now;
      }
    }
    else if (psi0<45)
    {
      if (now > (lastSend + minSecsBetweenEmails * 100l))
      {
        Serial.println("Critical! Under pressure: ");
        Serial.println(psi0, 1);
        lastSend = now;
      }     
    }
    else if (psi0>=85)
    {
      if (now > (lastSend + minSecsBetweenEmails * 500l))
      {
        Serial.println("Critical! Over pressure: ");
        Serial.println(psi0, 1);
        lastSend = now;
      }     
    }
//   else
//      {
//        Serial.println("Too soon");
//      }     
    delay(2000);  // delay for two seconds to make serial output easier to read
}

void printDec2(int value)
{
    Serial.print((char)('0' + (value / 10)));
    Serial.print((char)('0' + (value % 10)));
}

You're lucky it happens about once per day. You can add debugging statements to narrow down where it is locking up.

By the way, why are you using soft I2C?

jboyton: By the way, why are you using soft I2C?

On the hardware I2C pins no less.

All of your maths related to millis() should be using unsigned long. See http://www.gammon.com.au/millis

I have only a marginal understanding of how to program these components to work together. I buy a piece of hardware, find examples of how others have gotten it to work, and play around until I get an acceptable result. Then I mash that together with other things I've gotten to work via a similar process.

If I'm not doing something you'd expect, it's because I don't know any better, or haven't gotten a component to work another way. The real time clock is the most recent addition to these boards, and the libraries, code, and wiring diagrams I found for it didn't work for me until I put them together this way. The issue I described occurred prior to the addition of the clock, so how it's set up is not really my focus, but I would certainly appreciate any guidance on how to set it up correctly.

About once per day, on average, the Arduino with the sensors connected will lock up. Output to the serial window stops. I can close the window without problem, but I have to use the Windows Device Manager to disable the Arduino and re-enable it before it will again output to the serial monitor.

Is the program still running and only the Serial output has stopped, or is the program itself stopped?

It sounds like using the reset button and reopening the monitor does does not restart the serial output, but rather you need to do something with device manager and the com port. Is that correct?

marvinpandroid: I have only a marginal understanding of how to program these components to work together. I buy a piece of hardware, find examples of how others have gotten it to work, and play around until I get an acceptable result. Then I mash that together with other things I've gotten to work via a similar process.

That doesn't sound like an approach that will build an understanding of how things work. It really sounds like you need to slow down and solidly grasp some principles as you move forward. If you did that, you probably wouldn't need to be asking here.

MorganS: All of your maths related to millis() should be using unsigned long. See http://www.gammon.com.au/millis

This is helpful. Thank you for taking the time to point me to a resource that helps me learn and understand!

cattledog: Is the program still running and only the Serial output has stopped, or is the program itself stopped?

It sounds like using the reset button and reopening the monitor does does not restart the serial output, but rather you need to do something with device manager and the com port. Is that correct?

Truthfully, I don't know how to answer your first question.

Because this hardware is in a remote location, the reset button is not readily accessible. I found that I could use Windows Device Manager to reset the Arduino's COM port. I disable the port, then immediately re-enable it, then when I access the serial monitor it will display the running program (effectively this is like hitting the reset button, yes?). If I do not reset the COM the port, the serial monitor will open but no output is displayed there.

Thank you for taking the time to ask questions. I appreciate your desire to help.

I would think that the first thing to fix is the use of soft i2c as suggested in reply #1 and #2.

Look at the examples for your RTC library, and run it using Wire.h. You are already using the hardware i2c pins A4 and A5 so it will be a simple software change.

Thanks again for the suggestions and guidance. I learned more about I2C and the RTC, and borrowed example code that stripped interaction with the RTC down to something that helps me understand the process better.

I also cleaned out all of the code in my sketch that wasn’t being used, and I added more and better comments about how I put the code together. It should be much easier to read.

This revised code is currently running on the remote system. We’ll see if there is a difference in performance, or if the unit still locks up.

/*
 * Water System Monitor Project using Arduino Uno
 * 
 * Goal is to have remote monitoring, logging, and alerting
 * for critical system metrics, including:
 * 
 *    System pressure
 *    Reservoir level
 *    Well level
 *    Well pump duty cycle count and duration
 *    Pressure pump duty cycle count and duration
 *    Power status alerts
 *    Backup pressure pump alerts
 *    Exceptional pump duty cycle alerts
 *    Exceptional system pressure alerts
 *    Exceptional reservoir level alerts
 * 
 * Project started by David Greenwood on April 10, 2015
 * Last updated by David Greenwood on December 13, 2015
 * 
*/


/*
  Reservior Water Level Sensor
  
  Assumption is that the level of the reservoir will never exceed 12 feet
  12 ft * .43 psi/ft = 5.16 psi
  
  0-7.25 psi sensor purchased from Digi-Key:  http://www.digikey.com/product-detail/en/0/MPX5050GP1-ND
  Datasheet for the sensor:  http://www.freescale.com/files/sensors/doc/data_sheet/MPX5050.pdf
  Output: 0.2V – 4.7V (typ) linear voltage output. Vout = Vs * (0.018 * P + 0.04)
*/
 
int sensorPin0 = A0;  // select the input pin for the RESERVOIR pressure sensor
int sensorValue0;  // integer variable to store the value coming from the RESERVOIR sensor
float headFt0; // float variable to store level of reservoir in feet from calculation


/*
  Water System Pressure Sensor
   
  assumption is that SYSTEM pressure will never go above 100 psi
  
  0-100 psi sensor purchased from eBay:  http://www.ebay.com/itm/261260635816
  Output: 0.5V – 4.5V linear voltage output. 0 psi outputs 0.5V, 50 psi outputs 2.5V, 100 psi outputs 4.5V
  Pin 1 (notched) Vout, Pin 2 Ground, Pin 3 Vcc
*/

int sensorPin1 = A1;  // select the input pin for the SYSTEM pressure sensor
int sensorValue1;  // integer variable to store the value coming from the SYSTEM sensor
float psi1;  // float variable to store SYSTEM psi from calculation


/*  
  Real Time Clock - DS3231

  RTC module purchased via Amazon:  http://www.amazon.com/gp/product/B00HF4NUSS?psc=1&redirect=true&ref_=od_aui_detailpages02
  Datasheet for the module:  http://www.sunfounder.com/js/edit/attached/file/20150730/20150730124115_60389.zip
*/

#include "Wire.h"
#define DS3231_I2C_ADDRESS 0x68


void setup() {

  Wire.begin();
  Serial.begin(9600); // initialize serial

  // If necessary, set the initial time here:
  // DS3231 seconds, minutes, hours, day, date, month, year
  // setDS3231time(40,06,15,1,13,12,15);
  
}

void loop() {

  displayTime(); // display the real-time clock data on the Serial Monitor  
    
// read from the RESERVOIR sensor and do the calculations
  sensorValue0 = analogRead(sensorPin0);  // read the value from the sensor
  headFt0 = (((sensorValue0 * .0049) - .16) * 1.61) / .43; // maths to determine head feet based on sensorValue0: one foot head equals .43 psi (combined voltage and psi equation)

  Serial.println("********************************************"); // display header

// output to the serial monitor RESERVOIR information
  Serial.print("Reservoir Lvl= ");  // display label
  Serial.println(headFt0, 1);  // display reservoir level in feet and CR
    
// read from the SYSTEM sensor and do the calculations
  sensorValue1 = analogRead(sensorPin1);  // read the value from the sensor
  psi1 = ((sensorValue1 * .004887) - .49) * 25; // maths to determine psi1 based on voltage1: (sensor voltage minus minimum voltage) times (100 psi divided by 4v range) (combined voltage equations)
  
// output to the serial monitor SYSTEM information
  Serial.print("System Pressure = ");  // display label
  Serial.println(psi1, 1);  // display psi1 and CR
 
  Serial.println("-------------------------------------------"); // display footer
  Serial.println(""); // send extra CR
  Serial.println(""); // send extra CR
  
  delay(2000);  // delay for two seconds to make serial output easier to read
    
}

/*
 * Functions that support RTC
*/

// Convert normal decimal numbers to binary coded decimal
byte decToBcd(byte val)
{
  return( (val/10*16) + (val%10) );
}

// Convert binary coded decimal to normal decimal numbers
byte bcdToDec(byte val)
{
  return( (val/16*10) + (val%16) );
}

void setDS3231time(byte second, byte minute, byte hour, byte dayOfWeek, byte
dayOfMonth, byte month, byte year)
{
  // sets time and date data to DS3231
  Wire.beginTransmission(DS3231_I2C_ADDRESS);
  Wire.write(0); // set next input to start at the seconds register
  Wire.write(decToBcd(second)); // set seconds
  Wire.write(decToBcd(minute)); // set minutes
  Wire.write(decToBcd(hour)); // set hours
  Wire.write(decToBcd(dayOfWeek)); // set day of week (1=Sunday, 7=Saturday)
  Wire.write(decToBcd(dayOfMonth)); // set date (1 to 31)
  Wire.write(decToBcd(month)); // set month
  Wire.write(decToBcd(year)); // set year (0 to 99)
  Wire.endTransmission();
}

void readDS3231time(byte *second,
byte *minute,
byte *hour,
byte *dayOfWeek,
byte *dayOfMonth,
byte *month,
byte *year)
{
  Wire.beginTransmission(DS3231_I2C_ADDRESS);
  Wire.write(0); // set DS3231 register pointer to 00h
  Wire.endTransmission();
  Wire.requestFrom(DS3231_I2C_ADDRESS, 7);
  // request seven bytes of data from DS3231 starting from register 00h
  // convert the byte variable to a decimal number when displayed
  *second = bcdToDec(Wire.read() & 0x7f);
  *minute = bcdToDec(Wire.read());
  *hour = bcdToDec(Wire.read() & 0x3f);
  *dayOfWeek = bcdToDec(Wire.read());
  *dayOfMonth = bcdToDec(Wire.read());
  *month = bcdToDec(Wire.read());
  *year = bcdToDec(Wire.read());
}

void displayTime()
{
  byte second, minute, hour, dayOfWeek, dayOfMonth, month, year;
  // retrieve data from DS3231
  readDS3231time(&second, &minute, &hour, &dayOfWeek, &dayOfMonth, &month,
  &year);
  // send it to the serial monitor
  // display time in 24h format
  Serial.print(hour, DEC);
  Serial.print(":");
  if (minute<10)
  {
    Serial.print("0");
  }
  Serial.print(minute, DEC);
  Serial.print(":");
  if (second<10)
  {
    Serial.print("0");
  }
  Serial.print(second, DEC);
  Serial.print(" ");
  switch(dayOfWeek){
  case 1:
    Serial.print("Sunday");
    break;
  case 2:
    Serial.print("Monday");
    break;
  case 3:
    Serial.print("Tuesday");
    break;
  case 4:
    Serial.print("Wednesday");
    break;
  case 5:
    Serial.print("Thursday");
    break;
  case 6:
    Serial.print("Friday");
    break;
  case 7:
    Serial.print("Saturday");
    break;
  } 
  Serial.print(" ");
  
  // Display date in typical US format (mm/dd/yyyy)
  Serial.print(month, DEC);
  Serial.print("/");
  Serial.print(dayOfMonth, DEC);
  Serial.print("/");
  Serial.print("20");
  Serial.println(year, DEC);
}

Nice job on cleaning up and simplifying your code.

Let's see how it runs. If it stops running the structure is pretty clear and you should be able to see where it hung up. If not, you can get more precise tracking print outs.

You have a good foundation for getting the additional functionality working in future code.

Well, that didn't take long. It locked up about two and a half hours after the last restart with the revised code. Again, at the top of the pressure range as the pressure pump turned off.

Hi,

Can you please post a copy of your circuit, in CAD or a picture of a hand drawn circuit in jpg, png?

Do you have bypass capacitors on motors and DC voltage lines?

Tom.... :)

Given your new code, I believe that you are dealing with electrical noise or power supply issues.

Can you provide a sketch of how your entire system of sensors, pumps, arduinos, power supplies, usb hubs, etc is wired up? Is your set up in a mechanical room with other pumps and relays? Weird interference problems in mechanical rooms with motors and relays often show up when people are using LCD's for display rather than Serial, but it is a tough environment.

How the second Arduino is connected to the air pump? Is there a relay or a transistor? What powers the air pump? How are the sensor lines routed in relationship to the pump and its control circuit.

One way you could trouble shoot this would be to get the two Arduinos onto two different USB power supplies from two different lap tops. Get them separated so there are no common wired elements. If the sensor Arduino continues to lock up, then I would think there are radiated emi issues from the pumps going off or spikes transmitted to the sensor signal lines rather than issues of the stability of the power supply.

If you can't get another lap top on site, perhaps you could separate the Serial communication part of the USB from the power supply part. Run both Arduinos off external 9v supplies through the barrel jack, and just run communications through the USB hub and usb connections to the lap top.

It might provide a useful clue if you can determine if the underlying program is halting or is it only the Serial communication part.

If you have access to an SD card module, you could use it to log the data, or just a simple High/Low a heart beat signal, from the sensor arduino and see if it continues to run when serial communication is down.

Alternatively, if the two Arduino's are close to each other, you might be able to run a wire from a pin on the sensor Arduino to another on the pump Arduino, and produce and read a heart beat signal which could be output to the monitor by the pump Arduino. I hate to add more cross connections while trying to diagnose and interference problem, but it would certainly be a simple thing to do.

To your point, @cattledog, when I first set up the system pressure monitor, I had it output to an LCD. Just like this, it would eventually freak out and it would display garbage text on the LCD, but I can no longer remember whether it would continue to output varying garbage, or whether it would stop cycling and display the same garbage until I reset it...

It will take me some time to put together the diagrams for everything. Please stay tuned. Thank you again for your help.

I agree with the comment about noise. You need to make sure you are decoupling the power rail with adequate capacitors and doing what you can to remove the pump motor noise from your circuit. Especially when the pump stops, which is when the motor will dump the electric charge stored in the motor coil back into your electronics.

About Decoupling.

Finally got a diagram done that’s close to my standards. There’s still a lot of information missing, like the backup pressure pump and the well pumps, but those are not currently part of the monitoring system.

I purchased an isolation transformer and now have it positioned between mains and the UPS. Just connected it up today, and I’m curious if that makes a difference.

I also bought an oscilloscope in the hope of being able to see what’s happening. If anyone has any guidance around the best way to do that I would appreciate it. I’m still familiarizing myself with the scope and am not rushing out to do this, but am curious. I’m currently attending a class on electrical for water systems, but I am sure the use of an oscilloscope will not be part of the curriculum.

Thanks again for those who are willing to help.

pumphouse monitor diagram.pdf (140 KB)

It's been about 42 hours since I put the isolation transformer into place and the Arduino has not locked up. The is easily the longest it's lasted in many, many weeks, if not ever. A pricier solution than I had hoped for, and I'm sure I could have accomplished the same thing for less, but it is proving out the problem.

Thanks for the suggestions about de-coupling and isolation, and for the push to clean up the code and learn more about how the RTC works. This has been fun.

Congratulations.

Your system diagram explains a lot, and the isolation transformer should solve the problem if the turning off of the primary pressure pump was sending a big noise spike back into the system. It is interesting that the APC Smart UPS did not provide isolation. I guess it wasn't that smart.

I think now that if you continue to see problems it would likely be caused by radiated emp picked up directly by wires connected to the Arduino's.

The system documention and code clean up is very important in an industrial setting. If you unfortunately get hit by a bus, your successor will thank you. No matter what, in every job I've been in, a common explanation of all problems was that "the dead guy did it". :)