I'm using Arduino IoT Cloud to record a few variables from an Arduino Nano 33 IoT in my application. It appears that the variables are polled/reported to the cloud at different frequencies. For example, variable 1 may report 1 time per second and variable 2 may report 6 times per second. I need them to all report at the same frequency so that when I download each cloud variable and paste them into a spreadsheet, the columns are the same length and the timestamp for each row is the same (and thus, the timestep between rows is the same) for post-processing. How can I achieve this?
In the past, I've tried using 'periodically' updating the values rather than 'on change' but while this is better it is limited to 1 second periods and the timestep still varies in this mode.
If there's a similar thread on this already please do share. It seems like I could resolve this issue by pushing the variables from cloud to Google Sheets like in this video but I want to be sure that when I add a line of variables to the sheet, that those values were actually collected at the same time.
I've attached the data as I've described and here is the section of code that pushes data to the Arduino IoT Cloud.
void loop() {
if (ArduinoCloud.connected() == 1 or !connectedYet) {
load_cloud = load;
engineRPM_cloud = engineRPM;
throttleIn_cloud = throttleIn;
vehicleSpeed_cloud = vehicleSpeed;
}
if (ArduinoCloud.connected() != 1) delay(45);
}
}
Hi @bryons. I think the best way to handle this is to add a Variable which will contain all the data you want to import to the spreadsheet. In this way, you have a single timestamp for each data series instead of a separate timestamp for each data point. The data will be formatted as CSV text in this Variable.
I'll provide instructions you can follow to do that:
Open your Thing in Arduino Cloud.
Select the "Setup" tab of the Thing page.
Click the "ADD" button under the "Cloud Variables" section of the Thing setup page.
The "Add variable" dialog will open.
Type data_cloud in the "Name" field of the dialog. ⓘ I chose an arbitrary variable name. You are welcome to use an other one if you like. If so, make sure to adjust the later steps in these instructions accordingly.
Click on the "Select variable type" menu.
The menu will expand.
Select "Character String" from the menu.
Select the "Read Only" radio button under the "Variable Permission section of the dialog.
Click the "ADD VARIABLE" button.
The "Add variable" dialog will close.
Select the "Sketch" tab at the top of the Thing page.
The Thing sketch will open in Arduino Cloud Editor.
Add the following code to the loop function of the sketch:
Upload the updated Thing sketch to your Arduino board.
Open the Arduino Cloud dashboard for your Thing.
Click the pencil item on the dashboard's toolbar to put the dashboard into edit mode.
Click the "ADD" button near the top left corner of the page.
A menu will expand.
Select "Value" from the menu.
A "Widget Settings" dialog will open.
Click the "Link Variable" button.
The "Link Variable to Switch" dialog will open.
Select this project's Thing from the list of Things under the left column of the dialog.
Select "data_cloud" from the list of Variables under the middle column of the dialog
Click the "LINK VARIABLE" button.
You will be returned to the "Widget Settings" dialog.
Click the "DONE" button.
The "Widget Settings" dialog will close.
Click the icon that looks like an eye near the top left corner of the page.
This will put the dashboard into display mode, where the layout and widgets are no longer editable.
Wait until some data has been collected from the data_cloud Variable.
Click the downward pointing arrow button ("Download historic data") at the right side of the dashboard toolbar.
The "Download historic data" dialog will open.
Check the checkbox next to the data_cloud Variable.
Click the "SELECT DATA SOURCE" button.
Click the "GET DATA" button.
Wait until you receive the data download email.
Download the data from the link in the email.
Open the .csv file from the download in Google Sheets.
You will see that it has an unusable format at this point, something like this:
time
value
2024-06-18T21:02:29.015180053Z
0,0,6,6
2024-06-18T21:02:29.527905195Z
123,0,6,6
2024-06-18T21:02:30.059872753Z
234,0,7,7
2024-06-18T21:02:30.57385969Z
456,0,8,8
2024-06-18T21:02:31.092015753Z
567,0,8,8
Click the "B" column heading to select the entire "value" column of the spreadsheet.
Select Data > Split text to columns from the Google Sheets menus.
You will now see the data has a usable format, something like this:
time
value
2024-06-18T21:02:29.015180053Z
0
0
6
6
2024-06-18T21:02:29.527905195Z
123
0
6
6
2024-06-18T21:02:30.059872753Z
234
0
7
7
2024-06-18T21:02:30.57385969Z
456
0
8
8
2024-06-18T21:02:31.092015753Z
567
0
8
8
You can fill in the first row of the sheet with meaningful labels for each of the data columns.
This is exactly what I need. Thank you so much for the thorough reply. I'll try this out tomorrow and hopefully remember to reply to the thread with how it went.
I was wondering if you could do something like what you described here but I'm not savvy enough yet with the cloud so your explanation is great.
@ptillisch I attempted the solution. Except I just changed an existing variable, throttleIn_cloud, from int to a character. So the steps after step 11 were already completed. That has worked for in the past to avoid having to set up an entire new dashboard because we've already maxed out the number of variables in our plan. The variable was already linked and sending data to cloud so it should continue doing so even if the type declaration is changed?
When I download historic data and check the variables that are used to create the throttleIn_cloud variable, they are populating and appear to be working as before. When I check the throttleIn_cloud variable, it is blank and only shows me the header 'time,value' in the downloaded csv file.
I will troubleshoot some more later and see if I can find anything.
Remove the dashboard Widget you created for the throttleIn_cloud Variable before the type change (or if it is in a multi-Variable Widget such as "Advanced Chart", you can just remove the Variable from the Widget) and then add a new "Value" Widget to your dashboard, linked to the throttleIn_cloud Variable.
I did some experimentation and found that existing Widgets are broken when you change the type of a variable. So I think this is probably the reason why you didn't get any data for the throttleIn_cloud Variable after you changed its type.
@ptillisch it's working now. Thank you for your help!
And for future reference, is there a max length of a character string that can be uploaded to the cloud? I see in this thread that the Uno can handle from 32768-65536 bytes depending on the available memory at that instant. But I'm wondering if there's a further limitation on the cloud or if cloud variables have that same limit.
Unfortunately I don't know the answer to that. Maybe one of the other Arduino team members involved in the Arduino Cloud project who monitor this forum category, or one of the forum helper will be able to provide an answer.
