NB 1500 Data upload limits and performance

Official Hardware MKR Family MKR NB 1500
1

Hi all,

I'm putting together a prototype using an MKRNB1500 to take pictures out in the wild and upload them to a server for image analysis.

The images need to a somewhat decent resolution which is why I'm using the Arducam 5mp camera. I've done some initial tests and the minimum image size I can get away with is about 450k. I don't have specific time limit but ideally it wouldn't take more than a few minutes to upload

After some research it appears that uploading a file this size is beyond the limits of both the board as well as the NB; however I wanted to confirmation as everything I've read out there is not 100% clear.

Out of desperation I ordered the GSM1400 thinking that would do the trick, but now after further readying (which I should have done before) it appears that board might not cut it either.

So yeah, I want to hear from the community about the limitations of the board get some understanding as to what my issue is (is it board memory or NB limits or both?) and what my options are. If you know of any board in the marker with cellular capability that could do the job please le me know.

Lastly , I wanted to ask a code question about the upload just for my learning experience. The code below is what has sporadically worked with <= 5kb files (most of it omitted for brevity)

while (myFile.available()) 
{
  client.write(myfile.read())
}

However this appears to be very slow and inefficient. A quick google shows the following patter being used by a few people, but I can't get this version working even with the small files.

const int bufSize = 2048;
byte clientBuf[bufSize];
int clientCount = 0;
      
while (myFile.available()) 
{
 clientBuf[clientCount] = myFile.read();
 clientCount++;
 if (clientCount > (bufSize - 1)) 
 {
   client.write((const uint8_t *)clientBuf, bufSize);
   clientCount = 0;
 }
}

I've tried different variations of this code but the request never appears to be sent out at all.

any help will be greatly appreciated

2

drezzy:
Hi all,

while (myFile.available())

{
 client.write(myfile.read())
}





However this appears to be very slow and inefficient. A quick google shows the following patter being used by a few people, but I can't get this version working even with the small files.

const int bufSize = 2048;
byte clientBuf[bufSize];
int clientCount = 0;
     
while (myFile.available())
{
clientBuf[clientCount] = myFile.read();
clientCount++;
if (clientCount > (bufSize - 1))
{
  client.write((const uint8_t *)clientBuf, bufSize);
  clientCount = 0;
}
}




I've tried different variations of this code but the request never appears to be sent out at all.

any help will be greatly appreciated

The code for buffering the data needs some additional code after the while loop to check for a partial block and send it.

Something such as:
while (myFile.available())
{
clientBuf[clientCount] = myFile.read();
clientCount++;
if (clientCount > (bufSize - 1))
{
client.write((const uint8_t *)clientBuf, bufSize);
clientCount = 0;
}
}
If (clientCount > 0)
{
client.write((const uint8_t *)clientBuf, clientCount);
}

Being you included just a fragment of code this code may have been in your program, but if not it will be needed.

3

Hey icole. Yeah I noticed that not long after posting the question. So the good news is that I did get it to work. It takes about 4mins to upload the image.

I was hoping that upping the buffer will decrease the upload time but nothing has changed. I've moved it form 64 all the way to 1024 but time remains the same more or less. I presume this is just a limitation of the board specs, or perhaps the time it takes to write a large chunk is inversely proportional to the time it takes to read the data from the image.

The not so good news is that it's not reliable at all. Irrespective of the image size, (I've tried anything between 12kb, up to 420kb) there are times where it just appears to stop midway through looping the data stream and writing it to the client. After a reset of the board (or re-deploy the sketch) it works (or may not. .who know!).

What makes this really annoying is that there's no way of knowing if the operation has halted. I guess I could check the time and if it's run longer than expected then I could re-start the process.

4

I'm having lot of issues with the NB 1500 hanging in loops (no timeout) using MQTT (software not modem) where my code never gets control back with an error, rather just hung in a loop that never is satisfied and no timeout. I've no experience with what you're trying so my comment is general. It's my opinion that the Arduino library for the board doesn't work well and often is in a loop checking for 'ready' which never happens. The board seemed to be a perfect fit for my project from the specs. I hope your get your application working properly.

5

NBClient write chops the data into by default 256 byte blocks. It then converts the data into what I call "ASCII HEX" which doubles the size going to the modem. Thus speed wise there is probably little to be gained by writing more that 256 bytes at a time. There is a line 'int status MODEM.waitForResponse(10000, &response);' and 10 seconds seems should be ample but in my environment I increase 10000 to 240000 which helps with hangs but doesn't eliminate them. This may make no difference with your environment. I've found that a timeout here can lead to a loop polling that never exits.

I don't know if the modem sends the ASCII HEX or if it converts it back to binary HEX and sends it. I suspect it converts it back to binary and sends 256 bytes.

6

I traced the data (MQTT) from my MKR1500 to the server and the modem does convert the 'ASCII HEX' to binary HEX thus tansmitts the actual number of bytes. Makes sense since the AT commands only alloy for ASCII characters while TCP/IP allows binary data.