Guidance and real time clock - Remote River Sensor

Hello there!

I have a question that I've been scouring the internet for but have yet to find an answer to, so I figured joining this forum would be the best next step - Rest assured I've done a search of the forum to hopefully find my answer, but I've so far come up short.

This is my first electronics project, so please let me know if there is any information I am missing. I decided I wanted to create a remote river-level monitor for a river we like to kayak that is way out there in the backcountry. The schematic above represents what I've cobbled together, and it seems to be working just fine, apart from 2 issues.

1)Time slippage

I tried to use the built-in millis() library on the Arduino (See code below) to keep track of the hours. The idea was to take a reading every hour and send the collected readings to the satellite modem every 8 hours. Unfortunately, each hour that passes the time seems to slip a minute or two. The result is that within a few days, the readings are taken hours after they are originally meant to be taken.

2) Power consumption.

I would like to leave this out in the backcountry unattended for very long periods of time. The current setup seems to draw a lot of power from the 12v battery I have in place. This is not so much an issue when there is plenty of light, but the winter months would be challenging.

Ideal Solution

After looking into the issue an RTC seems to hold a solution here. Ideally, I could use an RTC to wake the Arduino at a set interval of an hour to take a reading. The RTC could then store the level readings taken each hour and transfer them to the Arduino at a given time each day to relay to the Satellite Modem.

Your help

As I mentioned this is the first electronics project I've undertaken so I am a little out of my depth. I was hoping that someone could guide me towards a good RTC to purchase that might be able to achieve my ideal solution above. Also, any tutorials/diagrams/videos that could get me up to speed on working with an RTC-Arduino combo would be greatly appreciated - So far I've seen people making digital clocks, but nothing about waking sleeping Arduinos or passing them data. Ideally, if someone could talk through what would need to be done to achieve the above, I could hopefully fill in the blanks myself !

My current code:

#include <SoftwareSerial.h>
#include <IridiumSBD.h> // Click here to get the library: http://librarymanager/All#IridiumSBDI2C
#include <Wire.h>       //Needed for I2C communication

#define IridiumWire Wire
#define MINUTE_VALUE 60000
#define HOUR_VALUE 60
#define REPORT_PERIOD 8

// Declare the IridiumSBD object using default I2C address
IridiumSBD modem(IridiumWire);

// Set the variables for the sensor reading
static const int sensorReadPin = 7;
int rangevalue[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
long pulse;
int modE;
int arraysize = 7;
bool measuring;

// Set the variables for timing
int minuteCount = 0;
int hourCount = 0;
unsigned long beginningOfTime;
unsigned long currentTime;

// Variables for sending the message
String lvl_message;
int levels[REPORT_PERIOD] = {0};
int txMsgLen;
int j = 0;
char satMessage[50] = {0};

void setup()
{
    // Set the initial time that the gauge was turned on
    beginningOfTime = millis();
    // Step for Serial Debugging only
    Serial.begin(9600);
    while (!Serial)
        ; // Wait to connect serial port. For native USB port only || Remove once live

    // Confirgure the pins used
    pinMode(sensorReadPin, INPUT);
    pulseIn(sensorReadPin, LOW);

    /*
     * Initiating the satellite modem
     *
     * Below we initiate the satellite modem. We first tell the arduino to
     * communicate with it via I2C, we then charge the supercapacitors.
     * Finally, we turn on the modem and set the power profile to be the default, then turn the system off.
     */

    // Start the I2C wire port connected to the satellite modem
    Wire.begin();
    Wire.setClock(400000); // Set I2C clock speed to 400kHz

    while (!modem.checkSuperCapCharger())
    {
        Serial.println(F("Waiting for supercapacitors to charge..."));
        delay(1000);
    }
    Serial.println(F("Supercapacitors charged!"));

    // Enable power for the 9603N
    Serial.println(F("Enabling 9603N power..."));
    modem.enable9603Npower(true);
    // Set the modem power profile to be the default power profile
    Serial.println(F("Starting modem..."));
    modem.setPowerProfile(IridiumSBD::DEFAULT_POWER_PROFILE); // Set the default power source for the modem
    // Disable 9603N power
    Serial.println(F("Disabling 9603N power..."));
    modem.enable9603Npower(false);
    // Turn off the modem
    Serial.println(F("Putting modem to sleep and starting the loop"));
    modem.sleep();
}

void loop()
{
    // Set current time the loop starts
    currentTime = millis();

    // Prevent Millis Rollover
    if ((currentTime - beginningOfTime) < 0)
    {
        beginningOfTime = currentTime;
    }

    // It has been 1 minute, update the minute count and check if a reading is required
    if ((currentTime - beginningOfTime) >= MINUTE_VALUE)
    {
        // Counts up to 1hr
        minuteCount++;
        beginningOfTime = millis();
        // Check if a measurement is required
        if (measuring)
        {
            // Gather 11 samples for rangeValue
            for (int i = 0; i < arraysize; i++)
            {
                pulse = pulseIn(sensorReadPin, HIGH);
                //Divide by 58 to convert to cm
                rangevalue[i] = pulse / 58;
                Serial.println((String) "Reading is " + (pulse / 58));
            }
            // We have 11 samples report the median to the levels array and shut off the sensor
            isort(rangevalue, arraysize);
            modE = mode(rangevalue, arraysize);
            levels[hourCount] = modE;
            Serial.println((String) "The mode at " + (hourCount) + " is " + levels[hourCount]);
            measuring = false;
            pulseIn(sensorReadPin, LOW);
        }
    }

    // An hour has passed reset the minute count and request the sensor to start reading again, also increment the hour reading
    if (minuteCount >= HOUR_VALUE)
    {
        minuteCount = 0;
        hourCount++;
        measuring = true;
    }

    // 8 Hours has passed, create the levels array and send it to the satelite. Reset the hours count to 0 to start the 8 hour cycle again.
    if (hourCount >= REPORT_PERIOD)
    {
        lvl_message = "[";
        for (j = 0; j < REPORT_PERIOD; j++)
        {
            lvl_message.concat(levels[j]);
            if (j < (REPORT_PERIOD - 1))
            {
                lvl_message.concat(",");
            }
            else
            {
                lvl_message.concat("]");
            }
        }

        txMsgLen = lvl_message.length() + 1;
        lvl_message.toCharArray(satMessage, txMsgLen);
        //Send the message to the satellite modem
        sendSatelliteMessage();
        hourCount = 0;
    }
}

// Sorting function
void isort(int *a, int n)
{
    //  *a is an array pointer function
    for (int i = 1; i < n; ++i)
    {
        int j = a[i];
        int k;
        for (k = i - 1; (k >= 0) && (j < a[k]); k--)
        {
            a[k + 1] = a[k];
        }
        a[k + 1] = j;
    }
}

// Mode function, returning the mode or median.
int mode(int *x, int n)
{
    int i = 0;
    int count = 0;
    int maxCount = 0;
    int mode = 0;
    int bimodal;
    int prevCount = 0;

    while (i < (n - 1))
    {
        prevCount = count;
        count = 0;
        while (x[i] == x[i + 1])
        {
            count++;
            i++;
        }
        if (count > prevCount & count > maxCount)
        {
            mode = x[i];
            maxCount = count;
            bimodal = 0;
        }

        if (count == 0)
        {
            i++;
        }

        if (count == maxCount)
        { // If the dataset has 2 or more modes.
            bimodal = 1;
        }
        if (mode == 0 || bimodal == 1)
        { // Return the median if there is no mode.
            mode = x[(n / 2)];
        }
        return mode;
    }
}

//Function to send sat message
void sendSatelliteMessage()
{
    int signalQuality = -1;
    int err;

    // Enable the supercapacitor charger
    Serial.println(F("Enabling the supercapacitor charger..."));
    modem.enableSuperCapCharger(true);

    // Wait for the supercapacitor charger PGOOD signal to go high
    while (!modem.checkSuperCapCharger())
        ;
    Serial.println(F("Supercapacitors charged!"));

    // Enable power for the 9603N
    Serial.println(F("Enabling 9603N power..."));
    modem.enable9603Npower(true);

    // Begin satellite modem operation
    Serial.println(F("Starting modem..."));
    err = modem.begin();
    if (err != ISBD_SUCCESS)
    {
        Serial.print(F("Begin failed: error "));
        Serial.println(err);
        if (err == ISBD_NO_MODEM_DETECTED)
            Serial.println(F("No modem detected: check wiring."));
        return;
    }

    // Send the message
    Serial.println(F("Trying to send the message.  This might take several minutes."));
    Serial.println((String) "The message being sent to the satellite is " + satMessage);
    err = modem.sendSBDText(satMessage);
    if (err != ISBD_SUCCESS)
    {
        Serial.print(F("sendSBDText failed: error "));
        Serial.println(err);
        if (err == ISBD_SENDRECEIVE_TIMEOUT)
            Serial.println(F("Message Sending Failed"));
    }

    else
    {
        Serial.println(F("Satellite message sent!"));
    }

    // Clear the Mobile Originated message buffer
    Serial.println(F("Clearing the MO buffer."));
    err = modem.clearBuffers(ISBD_CLEAR_MO); // Clear MO buffer
    if (err != ISBD_SUCCESS)
    {
        Serial.print(F("clearBuffers failed: error "));
        Serial.println(err);
    }

    // Power down the modem
    Serial.println(F("Putting the 9603N to sleep."));
    err = modem.sleep();
    if (err != ISBD_SUCCESS)
    {
        Serial.print(F("sleep failed: error "));
        Serial.println(err);
    }

    // Disable 9603N power
    Serial.println(F("Disabling 9603N power..."));
    modem.enable9603Npower(false);

    // Disable the supercapacitor charger
    Serial.println(F("Disabling the supercapacitor charger..."));
    modem.enableSuperCapCharger(false);

    Serial.println(F("Message Send Function Complete"));
}

I stopped reading there.

Also, please don't SHOUT

Is preventing millis rollover not a good idea?

That's not schematics but a very good picture of the system build! Not often such overview is posted.

That's fully normal. It's not a precision crystal running the controller, only a simple resonator. Use an RTC if You need higher precision.

Does such an RTC exist? I doubt that. The Arduino will have to do the sensor readings and transmit them.

Almost madness to aim that high. This business is not at all "plug and play".

Try again and use other search criteria. Plenty has been done.

Me neither.

Power saving is a never dying task. Lots of projects exist.

You seem to tackle the project piece by piece. That's the way to proceed. Know that putting the controller asleep is one thing but what about the rest of the electronics? Which parts consumes which power? Check that up! Else You will filter out the fly and swallow the elephant.

No. It rolls over once per some 42 days. That's not Your problem.

Thanks for breaking down each of those points! I really wasn't sure how else to post the schematic, but thank you.

I should have been clearer - I could find details about waking sleeping Arduinos, but not anything about persisting data between sleep cycles.

That's a great point - I'll look into what is consuming the most power and work backwards.

Thank you for your insights and guidance here - I'm always so impressed with how willing strangers are to help with hobby projects like this :pray:

Think about your expression.
You have an unsigned variable, minus another unsigned variable.
And, somehow, that could be less than zero.

How does that make sense?

The trick is to write write code that does not have a problem when the millis timer rolls over. Please read the page of Blink Without Delay: https://docs.arduino.cc/built-in-examples/digital/BlinkWithoutDelay

If you add extra code to "solve" the millis rollover, then you make it worse. Who told you to write such code ? ChatGPT ?

Some Arduino boards have a crystal, such as the Arduino Leonardo. Then you can make a clock with millis() that is just as accurate as the crystal.
A board with a resonator is not accurate. A good RTC is the DS3231. However, if you buy a module with the DS3231 on Ebay/AliExpress/Amazon then your DS3231 is a fake/reject/counterfeit.

Can you get the time from the Iridium module ?

What is that resistor of "2.182kΩ" doing ? Is it to keep the aliens away :alien: ?

You should be able to retrieve the current time from the satellite modem - see the examples sketches for Example3_GetTIme. That would allow you to sync the time on the arduino at intervals.

The DS3231 RTC is fairly accurate, and generally has a small EEPROM on the same board, which you could use to save data readings (the EEPROM in the ATMEGA328 chip could be used, but is very limited in how many times it can be written to).

If you are running the DC-DC converter with an output of 5V, you can disable the voltage regulator and power LED on the Pro Mini to save power, but the Iridium module is going to be using the bulk of the power anyway.

Get rid of SoftwareSerial, the sketch does not use it.

Avoid using String variable, they can cause memory problems over long-term usage.

As for the millis() discussion, notice the warning message from the compiler:

/tmp/arduino_modified_sketch_342084/BlinkWithoutDelay.ino:86:39: warning: comparison of unsigned expression < 0 is always false [-Wtype-limits]
   if ((currentTime - beginningOfTime) < 0)
       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~

When I was doing reading through other code examples, this came up as an issue that needed to be accounted for - So I added it in. No chat GPT recommendation on that just yet. I'll take a look at that link thank you.

I considered getting the time from Sat, but the modem draws the most amount of power. The purpose of this project is to leave it in the bush for 10 months at a time, hence the need to keep this as efficient as possible.

According to the spark fun docs, I need to add a resistance of 2.182kΩ to power my 3.3v Arduino from the breakout board.

The DS3231 RTC is fairly accurate, and generally has a small EEPROM on the same board, which you could use to save data readings (the EEPROM in the ATMEGA328 chip could be used, but is very limited in how many times it can be written to).

Thank you, I will take look there.

All of the points above are great folks, thank you. I can't tell you how much I appreciate it. When your working on this stuff alone it's nice to get some other ideas into the mix.

The 2.812kΩ resistor goes from GND to TRIM on the DC-DC converter, to set the output to 3.3V.

An 8MHz Pro Mini will run fine off of 5V, just disconnect the voltage regulator - the genuine Pro Mini had a jumper pad just for this purpose, and the power LED gets disconnected at the same time, to save a bit more power.

Why if the river 'is way out there in the backcountry', and presumably it will take a long while to get there, do you need the accuracy of an RTC in the first place ?

What possible reason, if the interest is kayaking, do you need to know the river level, of a river that is a long way away, to a few seconds ?

A remote sensor, that perhaps wakes up every hour, takes a reading, and sends it seems perfecrty adequate. You will know, from the time of receipt of the message, what the river level was a few seconds ago.

This what has not been mentioned that I see is considered required reading for low power projects

It may seem like a slog, but it covers this subject A to Z, provides many examples ready to try and is chock full of just plain good stuff.

He has also published on the art of making things meant to be left out near the river and so forth.

If you develop, test and perfect the various pieces of the whole system, as @Railroader has advised, and anyone would, you will get the best chance of success.

I think it will necessarily mean understanding at more than a superficial level any code you find that does any part of what you are up to.

Conceptually this is fairly straightforward. The more you can think ahead about the project, the better.

Even in the 21st century there is a place for block diagrams for hardware and flow charts for software.

Ahead of schematics and code.

Be aware that low power can be addictive. Ferreting out the last 10s of microamps will take some care, and may mean you need get to buy a better current meter…

a7

Your Sparkfun DC/DC converter has an On/Off pin. The DS3231 RTC has an INT/SQW pin that is open drain and which goes low when the alarm time is reached - when you want to take a reading. You could connect those two pins with a transistor in between, and the RTC could completely switch off the power to the Arduino, sensor and modem between readings. Each time the alarm goes off, the Arduino would power up and do its thing, and then decide what time to set the next alarm for, which might vary depending on a number of things. Then when it' s ready to shut down, the Arduino would clear the RTS's alarm flag, which would turn off the INT/SQW pin, and the power would shut down. The RTC would operate on its own coin cell battery, which would probably last years. If you have any interest in this, I can provide more details.

If I were going to buy the RTC today, I would probably buy it on Ebay from seller Alice1101983. No guarantees of course, but they've been around a long time and have a 99.7% favorable rating.

https://www.ebay.com/itm/401093092939

The other thing is your satellite service. Are you sure there isn't a cell tower anywhere near your remote site? Does anybody live near there? If so, and they have internet, maybe you could do a deal.

Those modules work very well, even if an instance may keep time a bit off full priced accuracy*, that can usually be noticed and tossed, at that price, or compensate for in software.

It is worth noting that they need a bit of field modification, perhaps, see

and ask yourself if you shouldn't.

* if only paying full price guaranteed getting a genuine part. :expressionless:

a7

looks an interesting project!
I implemented a river monitoring system using The Things UNO with a BMP280 temperature/pressure sensor, a SR04M ultrasonic transducer and a DS18B20 DallasTemperature sensor uploading data over LoRaWAN to the myDevices/cayenne desktop
image

I was reading the sensors and uploading data every 10 minutes

you can see the river is tidal – rise and fall of about 10cm

The Sparkfun DC/DC-converter: https://www.sparkfun.com/products/9370
It is 6A and is expensive.

Take a look at a TPL5110. Wired in front of your power supply, this will wake your board every so often when the timer expires. And it uses a lot less power than the Arduino. You'll want to make sure it's in front of the DC/DC converter as it uses current whether the arduino is running or not.

Thanks for the callout on this, I missed that in the documentation! Though I am a little confused if you wouldn't mind clearing something up for me? Please excuse my crude wiring diagram, but would that look something like this?

Meaning two wires would come out of the ground pin? One to the Arduino, and one to connect to the trim with a resistor in the middle?

That is my understanding from the product description page on SparkFun's website, although I have no actual experience with the DC-DC converter board.