MKRNB1500 hangs with MQTT

Official Hardware MKR Family MKR NB 1500
1

I'm attempting to use a MKR1500 for a remote sensor application to send data via MQTT and having problems with hangs. What I've found is if a send times out in waitForResponse that the state is such that another send will hang in a loop checking for ready which has no timeout thus hangs forever. In NBClinet.cpp there are multiple places with code "while (ready() == 0);". When the waitForResponse times out ready() seems to always return 0. My test sketch is sending a message every 2 seconds to a broker. With the timeout values in the MKRNB library the program usually hangs within 10 minutes usually due to a AT+USORD=0,512 timing out in waitForResponse (timeout value is 10000) but does hang on other commands. The waitForResponse sets _buffer = ""; for a timeout. In all the cases I've observed the response does come in after the timeout. In Modem.cpp in the poll code (used to check for ready) with _buffer empty it doesn't handle the delayed response. I bumped the timeout values up to 120000 and the test ran for over 24 hours before hanging. A send of a mqtt message took longer than 2 minutes for the response but the response did come in after the timeout. I don't have timestamps on the debug data so don't know how much longer.

My latest hang after running for over 24 hours:
Publishing message
write uint_8, sizewrite uint_8, sizeEndMessage
endMessage payloadbufferbeginpacket type
endPacket
TX: 30 21 00 10 61 72 64 75 69 6E 6F 2F 6F 75 74 67 6F 69 6E 67
AT+USOWR=0,20,"3021001061726475696E6F2F6F7574676F696E67"
Modem waitForResponse timeout value 120000
WaitForResponse returned -1

+USOWR: 0,20

OK
Modem waitForResponse timeout value 120000
beginpacket type
endPacket
TX: E0 00

I don't believe there should be while loops without some timeout in code that's running unattended in remote areas. So thinking of adding timeouts for these but need to determine what the timeout value would be and what the action should be on such a timeout. My first thought is to reset the modem and reconnect.

Anyone have ideas as to what the timeout values should be for waitForResponse? Maybe there are problems with the SARA firmware?

Another thing I plan on looking into is whether MqttClient handles the responses correctly as well as my mqtt sketch. Fairly sure the test sketch is lacking in handling responses.

Was wondering if others have experienced similar problems and have resolved them or if anyone has any suggestions. This has to be rock solid before it can be deployed.

2

I've been trying to debug this and have found that the problem is a timeout sending a mqtt packet. I've bumped the timeout on the send to 180000 which results in less frequent failures but when it happens a "stop" is called which sends a AT+USOCL=0 command to close the socket and then I see the +USOWR:0,56 response to the wrote/ At his point things are all screwed up and nothing good happens after this. Think the modem probably needs to be reset when the timeout occurs as well as setting the "states" appropriately.. Of course on can keep bumping the timeout up and make the problem less likely to happen but that doesn't fix the problem just makes it more intermittent. The requirement is to never hang, rather recover from what ever. In our case loosing a message rarely wouldn't be a big problem.