I have a Nano 33 IoT associated with an IoT Cloud Thing managing home automation. When the Thing grew to approximately 75 Variables, it would no longer sync with the Cloud on powerup. It appears that ArduinoCloud.update() hangs and the Watch-Dog Timer resets the device. If the WDT is disabled, the call hangs indefinitely. I narrowed it down to that call by turning on an LED immediately before the call and turning it off immediately after. The LED is always on when it hangs. To eliminate the possibility that it was hanging in one of my Read/Write variable call back routines, I inserted a return statement as the first line of each one. The rest of the code is stable if I do not make the call.
I needed to remove four variables to get it to sync reliably. The remaining 71 variables break down as follows:
9 Read Only String
20 Read Only Float
3 Read/Write Float
9 Read Only Int
7 Read/Write Int
13 Read Only Bool
10 Read/Write Bool
I hoped to narrow it down to a definite number of variables but it is not that consistent. There is a grey area between 72 and 75 variables where it might work. Below that range it has always worked and above it, it has never worked. The variables I removed are Read/Write Int.
Because the LED is flashing for each call, I get a rough sense of where it is hanging in the sync process. Typically the second call is fairly long, measuring several seconds. It usually moves past this but then hangs around call 8, usually before the Nano has received the Cloud data. In the grey area I have seen instances where it gets further and at least some of the Cloud data makes it to the Nano. It may fully sync and appear to be running fine, then hang 15 seconds later.
To complete the program, I will require a total of about 90 variables so I am hoping this limit can be overcome. The Maker plan specifies unlimited variables per Thing.
The transfer of IoT Cloud variables is facilitated by the call to ArduinoCloud.upate(). All my code can do is ensure the call is made in a timely fashion - i.e. non-blocking code.
Hi @rodgilmour my suspect is you probably hit the board limit filling up all available heap space. You should try to optimize your heap usage or switch to a board with more RAM.
The compiler says I have 23660 bytes of ram available:
/usr/local/bin/arduino-cli compile --fqbn arduino:samd:nano_33_iot --build-cache-path /tmp --output-dir /tmp/513582978/build --build-path /tmp/arduino-build-DF842DECD139F09F6945A425A6E8CB49 --library /mnt/create-efs/webide/03/e1/03e1a6aeac7bb5900020bf52a73c0d25:rodgilmour/libraries_v2/Adafruit SSD1306 /tmp/513582978/Test_South_Hall_may09a
[info] Sketch uses 178620 bytes (68%) of program storage space. Maximum is 262144 bytes.
[info] Global variables use 9108 bytes (27%) of dynamic memory, leaving 23660 bytes for local variables. Maximum is 32768 bytes.
I do not dynamically allocate memory in the program and I use string.reserve() for all string variables used with IoT Cloud to prevent memory fragmentation. Those reservations total 610 bytes.
Do you think that 23050 bytes is insufficient for ArduinoCloud.update() ?
Is there any chance I am on the bleeding edge of variable usage or have you seen larger Things?
I don't know if this is useful information, but I track the time spent in ArduinoCloud.update() once the Thing has synced. It averages 200ms.
Thank you for this. I called the function from loop() and it returned a value of 4039.
I then called it from a Cloud Variable callback routine (to include the memory used by ArduinoCloud.update() ) and it returned a value of 3047. About 1000 bytes for ArduinoCloud.update() seems reasonable. These numbers are stable with time.
I am confused why the compiler thinks there are 23050 bytes of available memory while this call only only sees 4039, but regardless, there seems to be ample memory available.
My bad for asking two question in my last message, I know better. Please allow me to ask the second question again:
Is there any chance I am on the bleeding edge of variable usage or have you seen larger Things?
Considerable testing with the freeMemory function has added much insight.
First, I think it answers my question - I AM on the bleeding edge of variables, at least on the Nano 33 IoT board. With about 55 variables, freeMemory shows that the initial memory available is 20695. That's pretty close to what the compiler says. After calling ArduinoCloud.begin, freeMemory drops to 9355. That was a bit shocking so I started deleting variables to see what the impacts were. A bool varies between 128 and 152 bytes and a float is about 172 bytes. So the variables account for about 80% of memory ArduinoCloud.begin allocation.
As you mentioned, freeMemory isn't precise so I will round off the remaining numbers:
Initial 20k
after ArduinoCloud.begin 10k
after sync callBack 5k
So, on top of a large allocation per variable, the sync process consumes another huge block. Here I am suspicious that some memory may be being lost. From one reboot to the next, that block can vary in size by 600 bytes. That doesn't sound right and could explain why a bump in the internet connection can cause ArduinoCloud.update() to hang and trigger a WDT reboot. I previously reported that on the Contact Form and will create a new post for that once this is resolved.
I am pretty invested in Nano 33 IoT boards so suggesting I replace them all is not going to go down well. What are the chances that some of this memory can be reduced?
the compiler takes in account only statically allocated variables, all things properties when added are calling a wrapper function that is using heap to allocate all the needed context to handle properties update, that memory usage can only be measured at runtime.
I've seen things with a larger amount of variables, the problem is not the number of variables itself but the RAM usage.
This is interesting,with release 2.0.0 we have changed how the sync process is handled by the library, i will doublecheck it to make sure we are freeing all the needed memory
If there is a bug related to the sync mechanism it needs to be fixed, but optimizing memory usage on MRK boards can be a very hard task.
Thanks for the updates. Hopefully you can find some lost memory in that 5k chunk.
I discovered that {variable.value} can be used as the subject line for email triggers. I'm not sure if that is new or I just missed it in the past. Coupled with the messenger widget, the pair make an excellent alarm notification / log. I was able to remove about 10 variables that were used for alarming so I have a bit of breathing room. I'm not there yet, but a small win on your end will probably get me to where I need to be.
For sure the memory used in my sketch is important, but it is swamped by the memory used to communicate with IoT Cloud. I am very stingy with memory thanks to my first Arduino project done on an Uno. I am aware of things like (F("text")) and other memory saving techniques. I appreciate the thought, thank you.
It may be swamped by other uses of memory, but you were getting excited by being able to remove 10 variables so aa review of the types of variables used seems like a worthwhile excercise
Any luck looking for memory leaks in the sync process?
I now call freeMemory in the doThisOnSync() callback. It seems to be the point I need to monitor - the place available to me where the most memory has been consumed. Does that sound right to you?
The available memory varies from one restart to the next but it is always at least 6k less than the memory available after calling ArduinoCloud.begin().