ArduinoCloud.update cant reconnect - MQTT client connection lost

Original posted in Networking, Protocols and Devices incorrectly, so posted here. (will try and delete that one). With an update (below) on power saving.

Hello, I've been having a difficult time with the ArduinoCloud.update() function and its reliability. I have to use both cell (sms) and cloud to get my application working. I have to reset the GSM roughly every hour at this point to make sure the device can receive the sms. This works well and the receive is dependable. However, in the reset, the MQTT client connection is lost. By re-running ArduinoCloud.update() in a loop, it re-connects after about 4 loops. This works only once. Next time around, it runs to 3 loops and no response. I've tried other methods (seeing if I can turn off powersave on the Ublox. My hardware is a MKR GSM 1400, on a grove shield. The Ublox is a SARA-U201 03B-00 and on page 532 of the ublox spec, it indicates powersave is off by default. Since I cant do an "AT" command in code for some reason, I have to assume the AdruinoCloud doesnt change that. So, cant used that method, so I reset the GSM by shutdown and reconnect - works fine).
Here is my code that sometimes works, sometimes not. I've tried some other methods found in ArduinoCloud.update interrupt workaround? - IoT Cloud - Arduino Forum but the compiler rejected the .update on ArduinoIoTPreferredCloud so substituted ArduinoCloud.update() to no effect. In this code if the "on DISCONNECT" call back is used, the onNetworkDisconnect call is made and it never gets reconnected, so eventually does the reboot. Not very useful.

In summary, this code works until the gsm is reset, it reconnects once, and next time, halts. Am I using ArduinoCloud.update() wrong? Does it need any "time" to recover? Are there any other routines I could use to re-connect? As I understand, the ArduinoCloud.update() is supposed to re-connect, but it seems to do that only once and never "comes back".

Should anything be done before the re-connect to GSM ? Or after? Quite at a loss at this point. Pls forgive code construct. Had to remove some to get withing 9000char limit in forum

Deets:
My signal level is good. Info:
***** Arduino IoT Cloud - configuration info *****
Device ID: 6e649599-6fae-4f7b-a65d-4696e8ca3e87
Thing ID: 7b9f26a3-d47d-4afb-b40e-150f75168c35
MQTT Broker: mqtts-sa.iot.arduino.cc:8883
Carrier is : BELL
SignalStrength(dBm) = 31
SIM card ok
GPRS.attachGPRS(): 4
aSending PING to outer space...
GPRS.ping(): 210
Connected to GPRS Network
Connected to Arduino IoT Cloud

CONNECTED to network

Update: found out that AT+UPSV=3 is sent by connecting to GSM. (gsmAccess(true)) This puts power saving on (vs default of off). controlled by DTR line. So, not possible to control DTR without halting communications. So, trying to find out how I can send AT command (AT+UPSV=0).

#include "thingProperties.h"

// Include the GSM library
#include <MKRGSM.h>


// initialize the library instance
GSM gsmAccess;
GSM_SMS sms;
GSMLocation location;
GPRS gprs;
GSMScanner carrier;

bool netstate = true;
int netFailState = 1;

#ifdef DEBUG
#define DEBUG_PRINTLN(x)  Serial.println(x)
#define DEBUG_PRINT(x)  Serial.print(x)
#else
#define DEBUG_PRINTLN(x)
#define DEBUG_PRINT(x)
#endif

bool gsmConnected = false; 

bool causeFailFlag = false;



void setup() {
  Serial.begin(9600);

  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

  // Defined in thingProperties.h
  initProperties();


  // Connect to Arduino IoT Cloud
  ArduinoCloud.begin(ArduinoIoTPreferredConnection);

  setDebugMessageLevel(4);
  ArduinoCloud.printDebugInfo();
  ArduinoCloud.addCallback(ArduinoIoTCloudEvent::CONNECT, onNetworkConnect);
  ArduinoCloud.addCallback(ArduinoIoTCloudEvent::DISCONNECT, onNetworkDisconnect);

  // Start GSM shield.  This starts the GSM.  Then ArduinoCloud.update() connects to the cloud. It takes a while.
  // If your SIM has PIN, pass it as a parameter of begin() in quotes
  gsmConnected = connectGSM();


  if (!CLDEBUG) {  // This usually works. Takes about 4 loops

    while (ArduinoCloud.connected() != 1) {
      ArduinoCloud.update();
      delay(500);
    }

  }



}

void loop() {

  ArduinoCloud.update();

  // Lots of code goes here


  smsThere = checkSerial() ;  // (routine removed for forum)

  if (!smsThere) {


    oks = gsmAccess.isAccessAlive();

    if (causeFailFlag) {  // flag is set via sending an 'f' on sms

      oks = 0;   // *** fake failure to test mqtt alive after gsmconnect
      causeFailFlag = false;
    }


    if (!oks) {   // Either cell is down or its reset time
      // Note: this will cause the MQTT connection to fail. 02.17
      // it seems to recover only once.
      //
      gsmAccess.shutdown();   // shut it all down to allow proper restart
      delay(3000);

      // reconnect GSM then cloud
      gsmConnected = connectGSM();
      if (gsmConnected) DEBUG_PRINT(" Success ! ");  // this works pretty much all the time.
      else DEBUG_PRINTLN(" argh. failed. ");

      delay(500);  // wait a bit. MQTT will become disconnected at some point.

      // This works the first time in about 4 loops and reconnects.
      // The second time its run via another 'f' fail invoked, it runs 3 loops then never recovers.
      while (ArduinoCloud.connected() != 1) {
        ArduinoCloud.update();
        delay(500);
      }

    }
    else {  // ok, check

      if (gprs.status() != GPRS_READY) {
        if (gprs.attachGPRS(GPRS_APN, GPRS_LOGIN, GPRS_PASSWORD) == GPRS_READY)
          DEBUG_PRINT("mainLoop: GPRS ready!");
        else
          DEBUG_PRINT("mainLoop: GPRS not ready!");
      }
    }
  }
  else {
    DEBUG_PRINTLN("received sms");
    sms.flush();
  }

}


bool connectGSM() {
  gprs.setTimeout(180000);
  gsmAccess.setTimeout(180000);
  boolean connected = false;
  while (!connected) {

    if ((gsmAccess.begin() == GSM_READY) &&
        (gprs.attachGPRS(GPRS_APN, GPRS_LOGIN, GPRS_PASSWORD) == GPRS_READY)) {
      connected = true;
    }
    else {
      delay(1000);
    }
  }
  return connected;
}


void onNetworkDisconnect() { // (void *_arg) {
  // from jbloam (2019) https://forum.arduino.cc/index.php?topic=654299.0 but modified to fit
  // This never seems to recover.  It will always just reboot eventually.
  Serial.println(">>>> DISCONNECTED from network");
  netstate = false;
  digitalWrite(LED_BUILTIN, HIGH);

  delay(30000);
  if ((netstate == false) && (netFailState == 1)) {
    Serial.println("Attempting first GPRS reconnect");
    netFailState = 2;
    // ArduinoIoTPreferredConnection.update();
    ArduinoCloud.update();
  }
  delay(30000);
  if ((netstate == false) && (netFailState == 2)) {
    Serial.println("Attempting second GPRS reconnect");
    // ArduinoIoTPreferredConnection.update();
    netFailState = 3;
    ArduinoCloud.update();
  }
  delay(120000);
  if (netstate == false) {
    reboot();
  }
}

void reboot() {
  Serial.println("Rebooting...");
  NVIC_SystemReset();
}

Update: due to thought that had leaks in memory due to Strings, I've converted to char arrays and reduced functionality to minimum to keep memory available.
This was used to determine mem available: Right now its 1147 left. ArduinoCloud.update() is working fine. When the reset (powerdown of gsm and reconnect) occurs, the memory still retains the same number but the arduinocloud.update dies after about 5 loops of the main code. Still investigating, but no clue. (setting gsmAccess(true) for debug sees "ok" on just about everything.

Another update. I've found that using the "cloud" takes up about 6900 of available memory (it looks to me like 25%)
Since my needs are mostly cell, this is a large constraint on feature set. Anyone else found this to be a large overhead? If running just a non arduinocloud mqtt - would there be a benefit?

Update: code is working after lots of testing and some constraints. . The main issue about all this is that I reset (gsm shutdown/reconnect) every hour or so to keep the sms "receiving" working. From the forum its a common issue that either the Ublox stops receiving (maybe is the UPSV=3 mode for powersave that is configured by gsm) or other. To fix this I shutdown/reconnect. The mqtt connection then is lost.
I used the code from from jbloam (2019) ArduinoCloud.update interrupt workaround? - IoT Cloud - Arduino Forum to recover, but modified it. code snippet below.
This is called by adding the callbacks when the cloud is .begin in init. (note there is an example code that has a typo for the DISCONNECT being CONNECT., so changed that).
ArduinoCloud.addCallback(ArduinoIoTCloudEvent::CONNECT, onNetworkConnect);
ArduinoCloud.addCallback(ArduinoIoTCloudEvent::DISCONNECT, onNetworkDisconnect);

Some other useful info:
-Dont let the freeMem (value from freeMemory) get below about 1200 else the ArduinoCloud.update will fail "somewhere" as mem is lost.
-Dont add cloud variables over something like a Float or Int if that can be used as more mem is taken up for cloudvars.
-If you delete/add/delete cloudvars in 'Setup' sometimes Arduinocloud.update gets confused and dies. Its better to copy all code, delete the thing, make a new thing, copy back in code. This is also necessary when using the App (on iOs) from what I've seen as it kills the app. (Ive seen it kill the app if the thing is renamed too, but thats anecdotal).

Also, if Anyone can figure out any other way how to keep the sms-receive alive, LMK.

I did try and use GSM_DTR line to turn off the power save but it killed all AT commands (in gsmAccess(true) for debug).
Page 532 is the powersave stuff in the ublox manual (MKR GSM 1400 is SARA-U201 03B-00)
https://www.u-blox.com/sites/default/files/u-blox-CEL_ATCommands_%28UBX-13002752%29.pdf

void
onNetworkConnect()  {// (void *_arg) {



  Serial.println(">>>>
CONNECTED to network");



 
ArduinoCloud.addCallback(ArduinoIoTCloudEvent::DISCONNECT,
onNetworkDisconnect); // connected, turn callback on



  netstate = true;



  netFailState=1;



  digitalWrite(LED_BUILTIN, LOW);



}



 



 



void
onNetworkDisconnect() { // (void *_arg) {



  // from jbloam (2019) [iurl=https://forum.arduino.cc/index.php?topic=654299.0]https://forum.arduino.cc/index.php?topic=654299.0[/iurl]
but modified to fit



  // This never seems to recover.  It will always just reboot eventually. 



  // 02.26 This often works now with simple
loops instead of 'delay'.  It connects on
third try?



  // however, although it connects, the cloud
can still crap out. 



  int i;



  Serial.println(">>>>
DISCONNECTED from network");



  netstate = false;



  digitalWrite(LED_BUILTIN, HIGH);



 



    if((netstate == false) &&
(netFailState==1)){



      Serial.println("Attempting first
GPRS reconnect. wait 30");



      // for (i = 0; i < 120; i++)
delay(250);  // about 30 seconds



      netFailState = 2;



     
ArduinoCloud.addCallback(ArduinoIoTCloudEvent::DISCONNECT,NULL);  // This needs to be off, else will never
reconnect



      for (i=0; i<5; i++) {



      ArduinoCloud.update();



      }



    }



    if((netstate == false) &&
(netFailState == 2)){



      Serial.println("Attempting second
GPRS reconnect. wait 30 again");



      for (i = 0; i < 120; i++)
delay(250);  // about 30 seconds



      netFailState = 3;



      ArduinoCloud.update();



      }



    if (netstate == false) {



      Serial.println("Third attempt
Attempting second GPRS reconnect. wait 120. might reboot. ");



      for (i = 0; i < 500; i++)
delay(250);  // about 120 seconds  



      if(netstate == false){



        reboot();



        }



    }  



}



 



void reboot() {



  Serial.println("Rebooting...");



  delay(3000);



  NVIC_SystemReset();



}

The freeMemory code snippet. This was fromr Measuring Memory Usage | Memories of an Arduino | Adafruit Learning System

#ifdef __arm__
// should use uinstd.h to define sbrk but Due causes a conflict
extern "C" char* sbrk(int incr);
#else  // __ARM__
extern char *__brkval;
#endif  // __arm__
 
int freeMemory() {
  char top;
#ifdef __arm__
  return &top - reinterpret_cast<char*>(sbrk(0));
#elif defined(CORE_TEENSY) || (ARDUINO > 103 && ARDUINO != 151)
  return &top - __brkval;
#else  // __arm__
  return __brkval ? &top - __brkval : &top - __malloc_heap_start;
#endif  // __arm__
}

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.