MKR NB 1500 routinely disconnects after a week of transmission

[dzmn_01]

Hey everybody,

First post, so let me know if I need to work on anything for the future.

Project setup:
Board: MKR NB 1500
ublox firmware: A.02.16
Power source: Adafruit bq24074 solar/Li Ion charger to 6600mAH Li Ion with 9W 6V solar panel.
Sensor: Ratiometric 0-4.5V pressure transducer with voltage divider to scale output to 0-3V.
Backend: HTTP Post request through Google Cloud Platform.
SIM: Soracom Io
Fleet size: Over 20 boards
Objective: Transmit a sensor reading every minute from a remote location. My backend is set up to subset tables based on SimID and generate reports.

Situation:
The MKR board goes offline after 5-10 days of successful transmission despite above average signal qualities with AT+CSQ. The board, charger and sensor still outputs voltage, but the modem goes offline. The only way to restart transmission is a power cycle, which is unrealistic in remote locations.

I have tested the following:

• A permanent USB power source produces the same outcome.
• No sensor. Simple analog pin read still results in the board going offline.
• Arduino support informed me to move my sketches to Arduino IoT Cloud, where I removed my manual AT commands on resets and focused on their ArduinoCloud.update() function to solve any modem issues - same outcome.
• I have attached a hybrid sketch that involved their .update() function with a simple modem reset based on the AT+CEREG? response- this lasted 8 days of transmission before going offline. I have left the boolean flags.

#include <ArduinoHttpClient.h>
#include <ArduinoJson.h>
#include <MKRNB.h>
#include "thingProperties.h"

#define NB_TIMEOUT 60 * 1000UL

const char server[] = "funnel.soracom.io";
const int  port = 80;

NBClient client;
NBModem modem;
GPRS gprs;
NB nbAccess;
HttpClient httpClient = HttpClient(client, server, port);

// connection state
bool connected = false;
bool restart = false;

// Publish interval
long previousMillis = 0;
long interval = 60000; // transmit every minute

void setup() {
  initProperties();
  ArduinoCloud.begin(ArduinoIoTPreferredConnection);
  nbAccess.setTimeout(NB_TIMEOUT);
  gprs.setTimeout(NB_TIMEOUT);
  SerialSARA.begin(115200);
  delay(10);
  SerialSARA.println("AT+CFUN=0");
  delay(10);
  SerialSARA.println("AT+UMNOPROF=2"); //AT&T profile.
  delay(10);
  SerialSARA.println("AT+CFUN=15");
  delay(2000);
}

void connectNB() {
  Serial.println(F("Attempting to connect to cellular network"));
  bool connected= false;
  bool restart = false;
  while(!connected) {
    if ((nbAccess.begin(PINNUMBER, GPRS_APN, GPRS_LOGIN, GPRS_PASSWORD) == NB_READY) && (gprs.attachGPRS() == GPRS_READY)) {
      Serial.println(F("NB Success"));
      connected = true;
    } else {
      Serial.println(F("NB Connection Failed"));
      restart = true;
      delay(500);
    }
  }
  
}


void post_data(String postData) {
    Serial.println(F("POST Request"));

    httpClient.beginRequest();
    httpClient.post("/");
    httpClient.sendHeader("Content-Type", "application/json");
    httpClient.sendHeader("Content-Length", postData.length());
    httpClient.beginBody();
    httpClient.print(postData);
    httpClient.endRequest();

    delay(5000);   
  }
  
void verifyConnection() {
  if (nbAccess.isAccessAlive()){
    connected = true;
  }
    else{
      connected = false;
      Serial.println(F("Restart"));
      nbAccess.shutdown();
      connectNB();
    }
}


void loop() {
  ArduinoCloud.update();
  delay(5000);
  verifyConnection();

  unsigned long currentMillis = millis();
  // Enforce accurate interval
  if (currentMillis - previousMillis > interval) {
    previousMillis = currentMillis;

    // Construct the JSON data to send
    StaticJsonDocument<200> payload;
    int ptval = analogRead(A1);
    float pvolt = ptval * (3.3f / 1023.0f);
    payload["pdigi"] = ptval; 
    payload["pvolt"] = pvolt; 

    int battval = analogRead(ADC_BATTERY);
    float voltage = battval * 3.3f / 1023.0f / 1.2f * (1.2f + 0.33f); 
    payload["vbatt"] = voltage;

    String ICCID = modem.getICCID();
    payload["SimID"] = ICCID;
    int unixTime = nbAccess.getTime();
    payload["unixTime"] = unixTime;

    char jsonBuffer[512];
    serializeJson(payload, jsonBuffer);

    post_data(jsonBuffer);
  }

}

Any feedback on the sketch above to help eliminate this modem disconnect issue is greatly appreciated. The JSON construction and sending is based off recommendations on soraom github repos to send to their backend service - I was wondering if such a set up is causing a memory leak.

Arduino support informed me they are working on a fix for their modem stabilities and the reintroduction of OTA sketch uploads for the MKR NB 1500 - which is attractive for me as my fleet will be in remote areas.

[b707]

Most of GSM and cable net providers breaks the session after specified interval - usually few days. My local cable provider reset the session every day.
If this is your case - all what you need - just reconnect after session reset.

[dzmn_01]

Hi b707, thanks for the reply. If the connection is broken after a specific interval, wouldn't that be caught from verifyConnection() when checking AT+CEREG?

[lorithai]

Hi

I am also having some issues with the MKRNB 1500, though firmware is L0.0.00.00.05.06 [Feb 03 2018 13:00:41]. Maybe we have the same issue.

I am getting program freezes earlier than you, usually within a day. After reading through posts and documentation about the MKRNB 1500 issues i have found that there are some probmels with how this device functions and in particular how the MKRNB library works.

It appears that the MKRNB library was copied over from the MKRWIFI projects and just modified. There are multiple locations where the timings are wrong and the library makes a fatal mistake where it assumes the modem isn't ready to receive commands and just waits/loops forever.

One mistake is the actual hardware setup of the MKRNB 1500. There is a pin on the modem that allow you to check if the modem is on or not, but for some reason the Arduino team didn't connect this one. So the MKRNB library can only check if the modem is on or not by sending "AT" commands and check for response.

There is also a bug in the modem itself that causes it to become unresponsive to any AT command.

There are a couple of solutions to the problem.

1. update waitforresponse timers
   There are several locations where the times set for response are wrong in the MKRNB library (too short). You can increase these to what the documentation recommends. CFUN, CMGS USOCL, USOST +++.

int ModemClass::reset()
{
  send("AT+CFUN=15");

  return (waitForResponse(180000) == 1); // Was 1s in original code, but datasheet says 180s!!!
}

2. Update the firmware of the ublox Modem.
   You can update the firmware of the modem so it doesn't freeze to begin with. This does however require soldering certain pins and asking ublox for the firmware update files. Seems like a very tedious process.

3. Use the reset pin.
   In the documentation there is a reset pin that can be sent a signal for 10 seconds and the modem will do a hard reset. The documentation warns against using this too frequently as any use may leave the modem in an unrecoverable state. You can access this through MODEM.hardReset() in the MKRNB library. The issue with this function in the library is that the timer they have set is wrong and has to be changed (in modem.ccp) if this is to work reliably.

void ModemClass::hardReset()
{
  Serial.println("Inside Hard reset");
  // Hardware pin reset, only use in EMERGENCY
  digitalWrite(_resetPin, HIGH); // High on arduino is low on SARA-R410-02B-00
  delay(11000); // Original code says 1s, but datasheet says minimum 10s!!!!!,
  digitalWrite(_resetPin, LOW);
  setVIntPin(SARA_VINT_OFF); // doesn't really do anything unless you have connected the VINT pin yourself to the SARA modem
}

But this could end up not working because of how the MKRNB library handles unresponsive modems. If it at any point fails to receive a response after a timeout it loops forever in the NBClient::ready() method. Since it never leaves this loop you won't reach any other part of your code where you call hardReset(). There has to be at some point in the MKRNB library after some number of attempts and getting no response that either the shutdown(), end(), or the hardReset() functions are called.
AFAIK the end() function probably won't do anything as it is dependent on the _vintpin of the modem being connected, which it isn't by default. The shutdown() function also relies on a functioning modem (AT commands). So if the modem becomes unresponsive the best bet is hardReset().

I just haven't yet been able to figure out at what point this should be done, but i am working on it.

[gallus1234]

Thanks for elaborating @lorithai. Having exactly the same problem, I would appreciate if you share any update on that matter.

[lorithai]

Hi
@gallus1234
This code works for me for at least a week. Just replace the data part in line 122 with whatever data you wish to put there. The data is saved as a json object called jsonData, so just do jsonData["somekey"] = avalue and the code should handle the rest. Again, i need to warn that there is always a possibility that the modem might become bricked with the hard reset.

Also, the code i have is messy, and there might still be some fail states that i haven't captured, but i haven't seen that happen yet. Oh, and remember to fill in your arduino_secrets.h with whatever details you need (gprs credentials, mqtt credentials etc...)

#include <Arduino.h>
#include <ArduinoJson.h>
//#include <ArduinoRS485.h>
//#include <ArduinoModbus.h>
#include "arduino_secrets.h"

DynamicJsonDocument jsonData(1024);
char jsonChar[1024]; // char array to contain serialized jsonData
char outputBuffer[1024]; // contains entire output
bool read_error = false;
char SARAATCommand[1024];
char response[1024];
int responsePosition = 0;
int MIN_COMMAND_DELAY = 30;
char topic[] = SECRET_TOPIC;
int modemStatus = 2;

int creg_counter = 0; // count number of creg failures (if connected to network or not)
int hard_reset_counter = 0; // track number of hard resets
int fail_counter = 0; // general failure counter

const int loop_delay = 3 * 60  * 1000;
int loop_counter = 0;
int test_counter = 0;

// GSM details
const char PINNUMBER[]     = SECRET_PINNUMBER;
const char GPRS_APN[]      = SECRET_GPRS_APN;
const char GPRS_LOGIN[]    = SECRET_GPRS_LOGIN;
const char GPRS_PASSWORD[] = SECRET_GPRS_PASSWORD;


// MQTT credentials
char TOKEN[] = SECRET_TOKEN;
char DEVICEID[] = SECRET_DEVICEID;
char MQTT_CLIENT_ID[] = SECRET_MQTT_CLIENT_ID;
char MQTT_USER[] = SECRET_MQTT_USER;
char MQTT_PW[] = SECRET_MQTT_PW;
const char MQTT_BROKER[] = SECRET_MQTT_BROKER;
char MQTT_BROKER_IP[] = SECRETE_MQTT_BROKER_IP;
const char publishTopic[] = SECRET_TOPIC;

// doesn't work unless i define at top (something to do with the default values in the function)
void SARAcommunicate(const char command[1024],int timeout=10000,int pause=100,char positiveResponse[20] = "OK"){
  responsePosition = 0;
  memset(response,0,sizeof(response)); // reset response storage
  sendATCommand(command); // sends command and places response in response variable
  delay(1000);
  int start_millis = millis();
  while (!SerialSARA.available()){
    //Serial.println("SARA not available");
    delay(100);
    if ((start_millis+timeout)<millis()) {
      Serial.println("Communication timed out");
      break;
    }
  }
  delay(pause);
  Serial.println("waiting for OK");
  while (!strstr(response,positiveResponse)) {
    if (SerialSARA.available()) {
      response[responsePosition++] = SerialSARA.read();
    }
    if ((start_millis+timeout)<millis()) {
      Serial.println("AT reading timed out timed out");
      break;
    }
  }
  delay(10);
  while (SerialSARA.available()) {
      response[responsePosition++] = SerialSARA.read();
      delay(10);
  }
  Serial.println("response");
  Serial.println(response);
}


void setup() {
  Serial.begin(9600);
  SerialSARA.begin(115200);
  digitalWrite(SARA_RESETN,LOW); // a high value here hard resets the modem. set it LOW by default.
  delay(5000);
  if (modemIsResponsive(true)) {
    modemStatus = 1;
  } else {
    modemStatus = 0;    
  }
  if (modemStatus == 0) {
    powerModemOn();
  }
  modemIsResponsive(true);
  delay(5000);
  
  if (modemIsResponsive(true)) {
  //communicate("AT+CFUN=0"); // Turn off radio

  sprintf(outputBuffer,"AT+CGDCONT=1,\"IP\",\"%s\"",GPRS_APN);
  SARAcommunicate(outputBuffer,10000,100,"OK"); // SET context IP version and APN

  SARAcommunicate("AT+CMEE=2",10000,1000,"OK");
  SARANetworkConnected();
  sprintf(outputBuffer,"AT+UMQTT=0,\"%s\"",MQTT_CLIENT_ID);
  SARAcommunicate(outputBuffer); // MQTT client id 
  SARAcommunicate("AT+UMQTT=1,1883"); // MQTT server port
  sprintf(outputBuffer,"AT+UMQTT=2,\"%s\"",MQTT_BROKER);
  SARAcommunicate(outputBuffer); // MQTT server name
  //sprintf(outputBuffer,"AT+UMQTT=3,\"%s\"",MQTT_BROKER_IP);
  //SARAcommunicate(outputBuffer); // MQTT server IP
  sprintf(outputBuffer,"AT+UMQTT=4,\"%s\",\"%s\"",MQTT_USER,MQTT_PW);
  SARAcommunicate(outputBuffer); // set MQTT user and pw
  }

}

void loop() {
  loop_counter += 1;
  Serial.print("Loop: ");
  Serial.println(loop_counter);

  // get values here
  jsonData["value1"] = 1.2; // example data, replace with your own data 
  delay(1000);

  // turn on modem if it isn't responsive
  if (!modemIsResponsive(true)){
    powerModemOn();
  }

  // check if the modem is responsive and turn the modem on and off until it is. If if fails 5 times, try a hard reset.
  while (!modemIsResponsive(true)) {
    fail_counter +=1;
    Serial.println("modem not responsive, trying to power cycle");
    Serial.print("Failures: ");
    Serial.println(fail_counter);
    delay(1000);
    powerModemOff();
    delay(1000);
    powerModemOn();

    if (fail_counter>=5) {
      hardReset();
      Serial.println("hard reset");
      fail_counter = 0;
      hard_reset_counter += 1;
    }
  }

  Serial.println("Data collected");
  sendDatatoMQTT();
  delay(1000);

  if (modemStatus == 1) {
    powerModemOff();
  }
  
  Serial.println("next loop in: ");
  Serial.print(loop_delay/(1000));
  Serial.print(" seconds");
  Serial.println("");
  delay(loop_delay);
}



bool SARANetworkConnected() {
  responsePosition = 0;
  memset(response,0,sizeof(response));
  sendATCommand("AT+CREG?");
  int start_millis = millis();
  while (!strstr(response,"OK")) {
    if (SerialSARA.available()) {
      response[responsePosition++] = SerialSARA.read();
    }
    if ((start_millis+10000)<millis()) {
      Serial.println("Communication timed out");
      return false;
    }  
  }
  Serial.println(response);
  if (strstr(response,"0,1")){
    Serial.println("CEREG response positive");
    return true;
  } else {
    return false;
  }

}

void sendDatatoMQTT() {
  serializeJson(jsonData, jsonChar);
  delay(1000);

  
  
  //sprintf(outputBuffer,"AT+UMQTT=3,\"%s\"",MQTT_BROKER_IP);
  //SARAcommunicate(outputBuffer); // MQTT server IP
  sprintf(outputBuffer,"AT+UMQTT=4,\"%s\",\"%s\"",MQTT_USER,MQTT_PW);


  sprintf(outputBuffer,"AT+CGDCONT=1,\"IP\",\"%s\"",GPRS_APN);
  SARAcommunicate(outputBuffer,10000,1000,"OK"); // SET context IP version and APN
  //SARAcommunicate("AT+CEREG?");
  sprintf(outputBuffer,"AT+UMQTT=0,\"%s\"",MQTT_CLIENT_ID);
  SARAcommunicate(outputBuffer); // MQTT client id 
  SARAcommunicate("AT+UMQTT=1,1883"); // MQTT server port
  sprintf(outputBuffer,"AT+UMQTT=2,\"%s\"",MQTT_BROKER);
  SARAcommunicate(outputBuffer); // MQTT server name
  //sprintf(outputBuffer,"AT+UMQTT=3,\"%s\"",MQTT_BROKER_IP);
  //SARAcommunicate(outputBuffer); // MQTT server IP
  sprintf(outputBuffer,"AT+UMQTT=4,\"%s\",\"%s\"",MQTT_USER,MQTT_PW);
  SARAcommunicate(outputBuffer); // set MQTT user and pw
  //SARAcommunicate("AT+UMQTTNV=1");// restore saved MQTT credentials. Couldn't get the restore function to work, so contect is redefined on each send
  //SARAcommunicate("AT+UMQTT?");
  //SARAcommunicate("AT+UMQTT?");
  delay(1000);
  // Send payload 
  while (!SARANetworkConnected()) {
    SARAcommunicate("ATI");
    delay(1000);
    creg_counter +=1;
    if (creg_counter>= 30){
      powerModemOff();
      delay(1000);
      if (!modemIsResponsive(true)){
        delay(1000);
        powerModemOn();
      }
 
      delay(1000);
      sprintf(outputBuffer,"AT+CGDCONT=1,\"IP\",\"%s\"",GPRS_APN);
      SARAcommunicate(outputBuffer,10000,1000,"OK");
      delay(1000);
      creg_counter = 0;
      if (creg_counter>= 30){
        //hardReset();
        //break;
      }
    }
  }
  //SARAcommunicate("AT+CEREG?");
  SARAcommunicate("AT+UMQTTC=1",10000,1000); // MQTT login
  delay(1000);
  sprintf(outputBuffer,"AT+UMQTTC=2,0,0,\"%s\",\"%s\"",topic,jsonChar); // create AT command to send to modem
  //Serial.println(outputBuffer);
  SARAcommunicate(outputBuffer);
  delay(10000);
  SARAcommunicate("AT+UMQTTC=0"); // MQTT logout

}

void sendATCommand(const char command[1024]){
  Serial.print("Sending command: ");
  Serial.println(command);
  SerialSARA.println(command);
  delay(MIN_COMMAND_DELAY);
}

void powerModemOn(){
  Serial.println("FUNCTION: Powering modem ON");
  digitalWrite(SARA_PWR_ON, HIGH);  // Datasheet LOW = ARDUINO HIGH
  delay(200); // Datasheet says power-on pulse should be >=150ms, <=3200ms
  digitalWrite(SARA_PWR_ON, LOW); // Datasheet HIGH = ARDUINO LOW
  delay(1000);
}

void powerModemOff(){
  // Power modem off. 
  Serial.println("FUNCTION: Powering modem OFF");
  // Gracefully power off
  digitalWrite(SARA_PWR_ON, HIGH);  // Datasheet LOW = ARDUINO HIGH
  delay(1600); // Datasheet says power-off pulse should be >=1500ms
  digitalWrite(SARA_PWR_ON, LOW); // Datasheet HIGH = ARDUINO LOW
  delay(1000);
}

void hardReset(){
  Serial.println("Inside Hard reset");
  // Hardware pin reset, only use in EMERGENCY
  digitalWrite(SARA_RESETN, HIGH); // High on arduino is low on SARA-R410-02B-00
  delay(11000); // Datasheet says min 10s, so set 11s just to make sure
  digitalWrite(SARA_RESETN, LOW);
}

bool modemIsResponsive(bool display_return){
  sendATCommand("AT"); // send AT command
  delay(20); // wait for a while
  int start_millis = millis();
  while (!SerialSARA.available()){ // check if SARA responds or not
    //Serial.println("SARA not available");
    delay(100);
    if ((start_millis+1000)<millis()) {
      Serial.println("Response test timed out");
      return false;
    }
  }
  while (!strstr(response,"OK")) { // check if response contain "OK"
    if (SerialSARA.available()) {
      response[responsePosition++] = SerialSARA.read();
    }
    if ((start_millis+1000)<millis()) {
      Serial.println("AT reading timed out");
      return false;
    }
  }
  return true; // if response was read and it contained "OK"
}

[gallus1234]

thanks @lorithai ..one week is not a lot though. Like if the sensor is in a remote area, each week somebody has to go there to reset the sensor?

[lorithai]

Hi

One week is the longest i have run it for for now, it is still running without interruption.

[IcePickMick]

@dzmn_01 have you found a solution yet? I’m dealing with the same issue. My MKR 1500 remains connected 1-3 days before disconnecting. Requires complete disconnection from power before working again.

[Cabsteena]

I've been using these boards for a few years now measuring water tank levels in remote locations with poor connections for stock water monitoring. They wake up from a deepSleep every hour, turn on the watch dog, connect to the network and broker, take a measurement, send it, disconect, turn off the watchdog, then go back to sleep. They use a 2000mA liPo and solar panel with USB output to charge the battery.

I've had the same issues and have come to the conclusion that like a $1600 iphone, the modem occasionally needs resetting, and the main board also needs resetting, more so when the connection is poor. Apple, a trillion dollar company overcomes these issues by getting you to toggle the airplane mode to reset the modem or turn the phone on and off to reset the main board.
That obviously isn't possible when the boards are in remote locations so I've been using the
MODEM.hardReset();
liberally with no ill affect with the change on line 134 in Modem.cpp to 10000

digitalWrite(_resetPin, HIGH);
  delay(10000); // Datasheet does actually say 10 Seconds

and using a 4 minute watchdog timer to reset the board when they crash.

MyWatchDoggy.setup(WDT_SOFTCYCLE4M);  // initialize WDT-softcounter refesh cycle on 4Min interval
MyWatchDoggy.setup(WDT_OFF); // turn off the WDT while asleep

I also changed the timeout in NB.cpp line 61 to 30000
_timeout(30000)

I can see when the board resets as the modem reset count goes back to zero.

The code resets the modems more often than is probably needed but after years of stuffing around, they no longer go unresponsive. I save the tankheight into flash memory so it doesn't have to re-learn the tank height after a watchdog reset.
I'm a farmer with no IT training so my code is messy and basic but it works better than my iphone.
Feel free to use some or all of the below code. I tried the latest UBLOX firmware update where you have to solder but i wrecked a board with my bad soldering so I just run with the earlier version.
If you are publishing to Azure you'll need to upgrade your certificates which I've described how to do here.

MKR NB 1500 Azure IoT hub - new DigiCert Global Root G2 root certificate

I've been publishing data to Microsoft Azure Central using seven MKR NB 1500's for a few years now using a modified version of this sketch. Two days ago they all stopped connecting to the MQTT broker. Below is the Serial Output. ERROR .-2 Connecting to MQTT - FAILED AT+CPWROFF OK V : 3.96 GOING TO SLEEP I've been getting this notification on my Azure Central dashboard so I suspect this may be the cause. I'm guessing that the Baltimore Cyber Trust Root Certificate has expired and I n…

/*
  Azure IoT for Arduino MKR NB 1500, and water tank level sensor.

  Author: Matt Sinclair (@mjksinc)
  Edited by: Cameron Backus for use with WNK 8010-TT 0-5m 3.3V I2C pressure transducer.


  This sketch securely connects to either Azure IoT Hub or IoT Central using MQTT over NB-IoT/Cat-M1.
  The native NBSSL library is used to securley connect to the Hub, then Username/Password credentials
  are used to authenticate.

  BEFORE USING:
  - Ensure that SECRET_BROKER and CONN_STRING in arduino_secrets.h are completed
  - Change msgFreq as desired
  - Check ttl to change the life of the SAS Token for authentication with IoT Hub

  If using IoT Central:
  - Follow these intructions to find the connection details for a real device: https://docs.microsoft.com/en-us/azure/iot-central/tutorial-add-device#get-the-device-connection-information
  - Generate a connection string from this website: https://dpscstrgen.azurewebsites.net/

  Full Intructions available here: https://github.com/mjksinc/TIC2019

  **Hardware Setup**
  Tank level sensor using Arduino MKR NB 1500 and WNK 8010-TT 0-5m 3.3V I2C pressure transducer.
  Red wire to 3.3v, blue to ground, yellow to SCL pin 12, black to SDA pin 11
  Serial prints water level in mm, water volume in L, tank level as a % of full, battery voltage, battery percentage, signal strength
  Automatically sets the tank height from the maximum water level to calculate tank level as a percentage.
  Tank height saved to Flash storage so board can be reset using WDT in case of board crashing.
  Tank height can be reset to 0mm by grouding PIN 7.
*/

// Libraries to include in the code
#include <ArduinoLowPower.h>
#include <ArduinoMqttClient.h>
#include <MKRNB.h>
#include "./base64.h"
#include "./Sha256.h"
#include "./utils.h"
#include <WDTZero.h>
WDTZero MyWatchDoggy; // Define WDT
#include <FlashStorage.h>
FlashStorage(tankHeightStorage, int);

//Tank level sensor libraries.
#include <Wire.h>
#define SENSOR_ADDRESS 0x6D
#define SENSOR_REG 0x06

//Tank Level Sensor Integers
int waitTime = 90 * 1000;
int tankHeight;    //5000gal is 2400mm, 10000gal is 2758mm
float fadc;
float pressure;
float waterLevel;
int L;
float full_battery_voltage = 4.20; // use voltmeter
float empty_battery_voltage = 3.0; // use voltmeter
int sensorValue;
float V;
float batteryPercentage;
int modemResetCount = 0;

// Additional file secretly stores credentials
#include "arduino_secrets.h"

// Enter your sensitive data in arduino_secrets.h
const char broker[] = SECRET_BROKER;
// CONN_STRING: connection string from Hub;


// Pin used to trigger a wakeup
const int tankHeightResetPin = 7;

String iothubHost;
String deviceId;
String sharedAccessKey;

long ttl = 864000; //Time-to-live for SAS Token (seconds) i.e. 864000 = 10 day (240 hours)
NBModem modem;
NB nbAccess(true);
GPRS gprs;
NBSSLClient sslClient;
MqttClient mqttClient(sslClient);
NBScanner scannerNetworks;
NB nb;
/*
   Establish connection to cellular network, and parse/augment connection string to generate credentials for MQTT connection
   This only allocates the correct variables, connection to the IoT Hub (MQTT Broker) happens in loop()
*/

void setup() {
  int t = 20; //Initialize serial and wait for port to open, max 10 seconds
  Serial.begin(9600);
  while (!Serial) {
    delay(500);
    if ( (t--) == 0 ) break;
  }
  Serial.println("Setup Soft Watchdog at 4M interval");
  MyWatchDoggy.setup(WDT_SOFTCYCLE4M);  // initialize WDT-softcounter refesh cycle on 4Min interval

  tankHeight = tankHeightStorage.read();

  pinMode(tankHeightResetPin, INPUT_PULLUP);
  // Attach a wakeup interrupt on pin 7,
  LowPower.attachInterruptWakeup(tankHeightResetPin, tankHeightReset, CHANGE);
}

/*
   Connect to Network (if not already connected) and establish connection the IoT Hub (MQTT Broker). Messages will be sent every 60 seconds, and will poll for new messages
   on the "devices/{deviceId}/messages/devicebound/#" topic
   This also calls publishMessage() to trigger the message send
*/
void loop() {
  getBattery();
  if (V > 3.5) {
    //  Serial.begin(9600);
    Serial.println("******Splitting Connection String - STARTED*****");
    splitConnectionString();

    //Connects to network to use getTime()
    connectNB();

    if (nbAccess.status() == NB_READY && gprs.status() == GPRS_READY) {
      createSasToken();
    }
    // Set the message callback, this function is
    // called when the MQTTClient receives a message
    //  mqttClient.onMessage(onMessageReceived);

    if (nbAccess.status() == NB_READY && gprs.status() == GPRS_READY) {
      connectMQTT();  // MQTT client is disconnected, connect
    }

    if (mqttClient.connected()) {
      // MQTT client is disconnected, connect
      publishMessage();
    }

    else {
      Serial.println("******Connecting to MQTT - FAILED******");
    }


    MODEM.shutdown();
    MODEM.end();
  }

  MyWatchDoggy.setup(WDT_OFF);
  Serial.print("V : ");
  Serial.println(V);
  Serial.println("******GOING TO SLEEP******");
  LowPower.deepSleep(60 * 60 * 1000);
  MyWatchDoggy.setup(WDT_SOFTCYCLE4M);  // initialize WDT-softcounter refesh cycle on 4Min interval

}

void tankHeightReset() {
  tankHeightStorage.write(0);
}

/*
   Gets current Linux Time in seconds for enabling timing of SAS Token
*/
unsigned long getTime() {
  // get the current time from the cellular module
  return nbAccess.getTime();
}

/*
   Handles the connection to the NB-IoT Network
*/

void connectNB() {
  unsigned long nbTime = millis();
  Serial.println("\n******Connecting to Cellular Network - STARTED******");
  MODEM.begin();
  while ((nbAccess.begin() != NB_READY) ||
         (gprs.attachGPRS() != GPRS_READY)) {
//    delay(10000);
    // failed, retry
//    Serial.println("NB Hard Reset");   DON'T DO THIS WITH BAD BATTERY
//    MODEM.hardReset();                     OR MODEM WILL BRICK
//    modemResetCount ++;
//    delay(1000);
    if (millis() - nbTime > waitTime) {
      break;
    }
  }
  if (nbAccess.status() != NB_READY || gprs.status() != GPRS_READY) {
    Serial.println("******Connecting to Cellular Network - FAILED******");
  }
  else {
    Serial.println("******Connecting to Cellular Network - COMPLETED******");
  }
}

/*
   Establishses connection with the MQTT Broker (IoT Hub)
   Some errors you may receive:
   -- (-.2) Either a connectivity error or an error in the url of the broker
   -- (-.5) Check credentials - has the SAS Token expired? Do you have the right connection string copied into arduino_secrets?
*/
void connectMQTT() {
  unsigned long mqttTime = millis();
  Serial.print("Attempting to connect to MQTT broker: ");
  Serial.print(broker);
  Serial.println(" ");

  while (!mqttClient.connect(broker, 8883)) {
    // failed, retry
    Serial.print(".");
    Serial.println(mqttClient.connectError());
    //    delay(5000);
    if (millis() - mqttTime > waitTime) {
      break;
    }
  }
  if (mqttClient.connected()) {
    Serial.println();
    Serial.println("You're connected to the MQTT broker");
  }

  // subscribe to a topic
  //  mqttClient.subscribe("devices/" + deviceId + "/messages/devicebound/#"); //This is for cloud-to-device messages
  //  mqttClient.subscribe("$iothub/methods/POST/#"); //This is for direct methods + IoT Central commands

}

void createSasToken() {
  Serial.println("******Create SAS Token - STARTED*****");
  // create SAS token and user name for connecting to MQTT broker
  String url = iothubHost + urlEncode(String("/devices/" + deviceId).c_str());
  char *devKey = (char *)sharedAccessKey.c_str();
  long expire = getTime() + ttl;
  String sasToken = createIotHubSASToken(devKey, url, expire);
  String username = iothubHost + "/" + deviceId + "/api-version=2018-06-30";

  Serial.println("******Create SAS Token - COMPLETED*****");

  // Set the client id used for MQTT as the device id
  mqttClient.setId(deviceId);
  mqttClient.setUsernamePassword(username, sasToken);
}
/*
   Calls getMeasurement() to read sensor measurements (currently simulated)
   Prints message to the MQTT Client
*/
void publishMessage() {
  Serial.println("Publishing message");

  String newMessage = getMeasurement();

  // send message, the Print interface can be used to set the message contents
  mqttClient.beginMessage("devices/" + deviceId + "/messages/events/");
  mqttClient.print(newMessage);
  mqttClient.endMessage();
  delay(1000);
  Serial.println("Disconnecting from MQTT...");
  mqttClient.stop();
  delay(1000);
}

void getBattery() {
  sensorValue = analogRead(ADC_BATTERY);
  // Convert the analog reading (which goes from 0 - 1023)
  V = (sensorValue * (full_battery_voltage / 1023.0));
  batteryPercentage = ((V - empty_battery_voltage) / (full_battery_voltage - empty_battery_voltage) * 100);
}

/*
   Creates the measurements. This currently simulates and structures the data. Any sensor-reading functions would be placed here
*/
String getMeasurement() {
  Wire.begin();
  uint32_t low, med, high;
  uint32_t value;
  Wire.beginTransmission(SENSOR_ADDRESS);
  Wire.write(SENSOR_REG);
  Wire.endTransmission(false);
  Wire.requestFrom(SENSOR_ADDRESS, 3);
  if (Wire.available() == 3 )
  {
    high = Wire.read();
    med = Wire.read();
    low = Wire.read();

    value = (high << 16) | (med << 8) | low;
    //Serial.print("Value read: ");
    //Serial.println(value);
  }
  Wire.endTransmission();
  Wire.end();

  if (value & 0x800000)
  {
    fadc = value - 16777216.0;
  }
  else
  {
    fadc = value;
  }
  pressure = 50 * ((3.3 * value / 8388608.0) - 0.5) / 2.0;

  waterLevel = pressure * 101.97162129779;

  if (waterLevel > tankHeightStorage.read() && waterLevel < 3000) {
    tankHeightStorage.write(waterLevel);
    tankHeight = tankHeightStorage.read();
  }

  L = waterLevel / tankHeightStorage.read() * 100;

  //  radius = tankDiameter / 2;
  //  volume = PI * (radius * radius) * waterLevel / 1000000;


  getBattery();

  //  float batteryPercentage = ((V - empty_battery_voltage) / (full_battery_voltage - empty_battery_voltage) * 100);

  // Begin network scan to get signal strength
  scannerNetworks.begin();
  String signalStrength = scannerNetworks.getSignalStrength();

  String formattedMessage = "{\"L\": ";
  formattedMessage += L;

  formattedMessage += ", \"pressure\": ";
  formattedMessage += pressure;

  formattedMessage += ", \"waterLevel\": ";
  formattedMessage += waterLevel;

  //    formattedMessage += ", \"lastMillis\": ";
  //   formattedMessage += lastMillis;

  formattedMessage += ", \"tankHeight\": ";
  formattedMessage += tankHeight;

  //    formattedMessage += ", \"sensorValue\": ";
  //  formattedMessage += sensorValue;

  formattedMessage += ", \"V\": ";
  formattedMessage += V;

  formattedMessage += ", \"modemResetCount\": ";
  formattedMessage += modemResetCount;

  formattedMessage += ", \"signalStrength\": ";
  formattedMessage += signalStrength;

  formattedMessage += "}";

  Serial.println(formattedMessage);
  return formattedMessage;
}

/*
   Handles the messages received through the subscribed topic and prints to Serial

  void onMessageReceived(int messageSize) {

  String topic = mqttClient.messageTopic();

  // when receiving a message, print out the topic and contents
  Serial.print("Received a message with topic '");
  Serial.print(topic);
  Serial.print("', length ");
  Serial.print(messageSize);
  Serial.println(" bytes:");

  // use the Stream interface to print the contents
  while (mqttClient.available()) {
    Serial.print((char)mqttClient.read());
  }
  Serial.println();

  //Refresh measurements and publish
  if (topic.startsWith(F("$iothub/methods/POST/publishMessage()"))) {
    Serial.println("Publish Message");
    publishMessage();
  }

  // Responds with confirmation to direct methods and IoT Central commands
  if (topic.startsWith(F("$iothub/methods"))) {
    String msgId = topic.substring(topic.indexOf("$rid=") + 5);

    String responseTopic = "$iothub/methods/res/200/?$rid=" + msgId; //Returns a 200 received message

    mqttClient.beginMessage(responseTopic);
    mqttClient.print("");
    mqttClient.endMessage();
  }


  }
*/
/*
   Split the connection string into individual parts to use as part of MQTT connection setup
*/
void splitConnectionString() {
  String connStr = CONN_STRING;
  int hostIndex = connStr.indexOf("HostName=");
  int deviceIdIndex = connStr.indexOf(F(";DeviceId="));
  int sharedAccessKeyIndex = connStr.indexOf(";SharedAccessKey=");
  iothubHost = connStr.substring(hostIndex + 9, deviceIdIndex);
  deviceId = connStr.substring(deviceIdIndex + 10, sharedAccessKeyIndex);
  sharedAccessKey = connStr.substring(sharedAccessKeyIndex + 17);
  Serial.print("******Splitting Connection String - COMPLETED*****");
}

/*
   Build a SAS Token to be used as the MQTT authorisation password
*/
String createIotHubSASToken(char *key, String url, long expire) {
  url.toLowerCase();
  String stringToSign = url + "\n" + String(expire);
  int keyLength = strlen(key);

  int decodedKeyLength = base64_dec_len(key, keyLength);
  char decodedKey[decodedKeyLength];

  base64_decode(decodedKey, key, keyLength);

  Sha256 *sha256 = new Sha256();
  sha256->initHmac((const uint8_t*)decodedKey, (size_t)decodedKeyLength);
  sha256->print(stringToSign);
  char* sign = (char*) sha256->resultHmac();
  int encodedSignLen = base64_enc_len(HASH_LENGTH);
  char encodedSign[encodedSignLen];
  base64_encode(encodedSign, sign, HASH_LENGTH);
  delete(sha256);

  return "SharedAccessSignature sr=" + url + "&sig=" + urlEncode((const char*)encodedSign) + "&se=" + String(expire);
}```

[fredsco]

Please note that something strange is happening when sending the AT+CEREG? command, also known as using the isAccessAlive function. The response is only correct upon modem startup and the initial connection. After establishing an MQTT connection using the lwmqtt library, the response is consistently +USOWR: 0,27 during an active connection. I am unsure whether this is due to command overlap or the fact that the MQTT connection is not directly established from within the modem.

[tegwyn_twmffat]

My system has a low power timer set to 2 hours which turns everything off, apart from itself. This, hopefully, means there wont be any elongated connection probs with the MKR 1500 as the device is turned off evry 2 hours.

[ocran]

@Cabsteena Thank you for your detailed code.

I have been struggling with LowPower.deepSleep() and the MKR1500 NB sending every other topic to AWS IoT. Your code showed the use of mqttClient.stop(); after publishMessage which has solved that problem.

Thanks again.

[RichardNordkvelde]

I've experienced the same. I've reproduced this problem easily by just removing the antenna so that AT+CEREG? loops for ~ 15 times before the UART between the CPU and Modem dies. I have the correct waiting for the command, and everything seems to be according to documentation (thanks to @zbelding ).

[fredsco]

The MKRNB library, which is supposed to support Sara functionality, really doesn't do a good job. I've been working on a project with the Sara R410M module and hit a point where I just couldn't use the library anymore. I ended up starting from scratch, using the Modem.CPP file to send commands directly to the modem. This meant figuring out myself which commands to send for things like starting a connection, checking status, and setting up MQTT, all based on the Sara R410 AT command sheet. Once you start looking into the AT command sheet, you realize the library gets a lot wrong, which explains why the modem often crashes or doesn't work well. Even though it claims to be compatible with the Sara R410, the library just isn't cut out for it. I even talked to a distributor of these Sara modules, and he showed me some example AT commands for connecting. There's a lot the library misses, like turning data roaming on or off for certain situations, or the right way to use one AT command after another, which you find in the AT command sheet from u-blox.

My modem has been maintaining a stable connection for over three weeks now. I added some error handling to cover scenarios where the connection might drop, and the module struggles to reconnect on its own. This extra step has significantly improved the reliability of the connection.

In short: if you are working on a serious project, the libraries you find online simply won't do.

[fredsco]

Please see my new post.

[RichardNordkvelde]

Yes, I totally agree with you. Amazing that they don't do a better job with these libraries since they might brick their own devices..

Do you have a counter to show how many times you've encountered this UART issue? My devices are also running smoothly, i.e. they send data when they have to. My only issue is that I'm powering the device with a Li-Ion 18650 battery, and I really don't want to hardReset the device too often since it will eat up all the power.

[Cabsteena]

I've discovered the hard way that the modem will brick if you do a hard reset on the modem with a battery that is on the way out and has a USB solar panel attached. I'm using a 2000mA Lipo and haven't had any issues for a few years until now. Also if you're having issues connecting either to the network or to mqtt, check the battery. They must have a good battery connected for the modem to work. They don't like USB power alone. I was getting a signal strength of 8 and very poor modem operation with a bad battery and a signal strength of 16 with a good one.

[nels52]

If one is willing to provide their own timeout protections within their script the infinite loops caused by repetitive calls to NBClient::ready() from NBClient::connect(), NBClient::write(buf, size), and NBClient::available() can be avoided by creating the NBClient instance with synch=false. I have been using this implementation.

I also use NBAccess.begin(NULL, true, false) which bypasses the synchronous implementation of the NB instance begin() method (NBAccess is my NB instantiation). This did require modification of the NB_NetworkStatus_t enumeration and NBAccess::begin() method to differentiate the begin() method ASYNCHRONOUS return from an ERROR status:

In NB.h

enum NB_NetworkStatus_t { ASYNCHRONOUS, ERROR, IDLE, CONNECTING, NB_READY, GPRS_READY, TRANSPARENT_CONNECTED, NB_OFF};

In NB.cpp

NB_NetworkStatus_t NB::begin(const char* pin, const char* apn, const char* username, const char* password, bool restart, bool synchronous)
{
  if (!MODEM.begin(restart)) {
    _state = ERROR;
  } else {
    _pin = pin;
    _apn = apn;
    _username = username,
    _password = password;
    _state = IDLE;
    _readyState = READY_STATE_SET_ERROR_DISABLED;

    if (synchronous) {
      unsigned long start = millis();

      while (ready() == 0) {
        if (_timeout && !((millis() - start) < _timeout)) {
          _state = ERROR;
          break;
        }

        delay(100);
      }
    } else {
        _state = ASYNCHRONOUS;     ///RTN Return ASYNCHRONOUS state so it is not confused with ERROR state
      ///RTN return (NB_NetworkStatus_t)0;
    }
  }

  return _state;
}

[ocran]

@fredsco , have you considered sharing your library?