Difficulty with Python and ArduinoCloudClient

[jddent9822]

I have an ESP32 project Thing that connects to the Arduino Cloud and with one variable. makes swimming pool water temperature available to my Widget and subsequently to Alexa. I would also like to use Python on my PC and the ArduinoCloudClient to retrieve the value, write it to a ,reg file, and use it to update the Windows registry for a 3D Text screen saver value in my pool cabana PC.

I received some help on this project using a different method in a previous Topic from @J-M-L . He/She suggested using Python and IOT Cloud , so I decided to create a new, more succinct and detailed Topic.

So far, Thing created and works very well. Widgets work great. Alexa integration is great.
Pretty sure I can complete the .reg / .bat file process to update the registry and change the 3D Text.

I tried the "Cloud Python Doc" example in its entirety shown here and it worked perfectly https://docs.arduino.cc/arduino-cloud/guides/python/#install-packages

I then tried adapting my own situation to the above example. The following is my Thing Detail, Python Script and Shell output.

Problem is with the Python script using basic connection to my Device. It appears to run once and then I get an "exception: EOF ... violation of protocol" error. Not sure if it's a Python or Client issue.

Thing ID: dcac0c9f-5a79-436b-ae27-affd53d742a8
Variable Declaration: CloudTemperatureSensor pool_Temp Read Only Update: Periodically
Device ID: ee6dbf5b-cd98-4bf1-a9e2-59cf3cdc579a

My Python Script

import time
import logging
import sys
sys.path.append("lib")
from arduino_iot_cloud import ArduinoCloudClient

DEVICE_ID = b"ee6dxxxxxxxxxxxxxxxxxxxxxxdc579a"
SECRET_KEY = b"Eoxxxxxxxxxxxxxxxxxxxxx3n"

def logging_func():
    logging.basicConfig(
        datefmt="%H:%M:%S",
        format="%(asctime)s.%(msecs)03d %(message)s",
        level=logging.INFO,
    )   

# This function is executed each time the "pool_Temp" variable changes 
def on_sensor_changed(client, value):
    print("Pool Temperature is: ", value)

if __name__ == "__main__":

    logging_func()
    client = ArduinoCloudClient(device_id=DEVICE_ID, username=DEVICE_ID, password=SECRET_KEY)

    client.register("pool_Temp", value=None, on_write=on_sensor_changed)
    
    client.start()

Shell output:

12:01:03.475 task: pool_Temp created.
12:01:03.490 task: conn_task created.
12:01:03.490 Connecting to Arduino IoT cloud...
12:01:03.953 task: discovery created.
12:01:03.991 task: mqtt_task created.
12:01:03.991 Subscribe: b'/a/d/ee6dbf5b-cd98-4bf1-a9e2-59cf3cdc579a/e/i'.
12:01:04.226 task: conn_task complete.
12:01:05.220 Unkown record found: OTA_URL value: 
12:01:05.220 Unkown record found: OTA_REQ value: False
12:01:05.360 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/e/i'.
12:01:05.438 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/shadow/i'.
12:01:05.735 Device configured via discovery protocol.
12:01:05.844 task: discovery complete.
12:01:05.906 Unkown record found: OTA_URL value: 
12:01:05.906 Unkown record found: OTA_REQ value: False
Pool Temperature is:  78.48619079589844
12:01:58.032 task: mqtt_task raised exception: EOF occurred in violation of protocol (_ssl.c:2423).
12:01:58.093 task: conn_task created.
12:01:58.093 Connecting to Arduino IoT cloud...
12:01:58.514 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/e/i'.
12:01:58.593 task: mqtt_task created.
12:01:58.692 task: conn_task complete.
12:02:53.108 task: mqtt_task raised exception: EOF occurred in violation of protocol (_ssl.c:2423).
12:02:53.108 task: conn_task created.
12:02:53.108 Connecting to Arduino IoT cloud...
12:02:53.780 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/e/i'.
12:02:53.890 task: mqtt_task created.
12:02:53.983 task: conn_task complete.

Looks like it might be something simple, but ....
Thanks for any help.

[J-M-L]

if you try this

import time
import logging
import sys
sys.path.append("lib")
from arduino_iot_cloud import ArduinoCloudClient

DEVICE_ID = b"ee6dxxxxxxxxxxxxxxxxxxxxxxdc579a"
SECRET_KEY = b"Eoxxxxxxxxxxxxxxxxxxxxx3n"

def logging_func():
    logging.basicConfig(
        datefmt="%H:%M:%S",
        format="%(asctime)s.%(msecs)03d %(message)s",
        level=logging.INFO,
    )   

# This function is executed each time the "pool_Temp" variable changes 
def on_sensor_changed(client, value):
    print("Pool Temperature is: ", value)

if __name__ == "__main__":
    logging_func()
    
    while True:
        try:
            client = ArduinoCloudClient(device_id=DEVICE_ID, username=DEVICE_ID, password=SECRET_KEY)
            client.register("pool_Temp", value=None, on_write=on_sensor_changed)
            client.start()
        except Exception as e:
            logging.error(f"An error occurred: {e}")
            logging.info("Retrying connection in 10 seconds...")
            time.sleep(10)

what do you see?

[jddent9822]

Hello @J-M-L . Thanks once again for your reply. I tried your script as suggested. Here's the output I got. Again, it appears to run once and then ...

========== RESTART: C:\Users\Jon Dent\AppData\Local\Programs\Python\Python311\JMLPoolTempAPI.py ==========
08:58:11.666 task: pool_Temp created.
08:58:11.729 task: conn_task created.
08:58:11.729 Connecting to Arduino IoT cloud...
08:58:12.156 task: discovery created.
08:58:12.172 task: mqtt_task created.
08:58:12.172 Subscribe: b'/a/d/ee6dbf5b-cd98-4bf1-a9e2-59cf3cdc579a/e/i'.
08:58:12.391 task: conn_task complete.
08:58:12.506 Unkown record found: OTA_URL value: 
08:58:12.538 Unkown record found: OTA_REQ value: False
08:58:12.678 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/e/i'.
08:58:12.756 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/shadow/i'.
08:58:13.043 Device configured via discovery protocol.
08:58:13.090 Unkown record found: OTA_URL value: 
08:58:13.090 Unkown record found: OTA_REQ value: False
Pool Temperature is:  78.60690307617188
08:58:13.168 task: discovery complete.
08:58:55.003 task: mqtt_task raised exception: EOF occurred in violation of protocol (_ssl.c:2423).
08:58:55.003 task: conn_task created.
08:58:55.003 Connecting to Arduino IoT cloud...
08:58:55.396 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/e/i'.
08:58:55.711 task: mqtt_task created.
08:58:55.836 task: conn_task complete.
09:00:01.008 task: mqtt_task raised exception: EOF occurred in violation of protocol (_ssl.c:2423).
09:00:01.008 task: conn_task created.
09:00:01.008 Connecting to Arduino IoT cloud...
09:00:01.660 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/e/i'.
09:00:01.754 task: mqtt_task created.
09:00:01.847 task: conn_task complete.
09:00:56.103 task: mqtt_task raised exception: EOF occurred in violation of protocol (_ssl.c:2423).
09:00:56.103 task: conn_task created.
09:00:56.103 Connecting to Arduino IoT cloud...
09:00:56.592 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/e/i'.
09:00:56.941 task: mqtt_task created.
09:00:57.035 task: conn_task complete.
09:01:52.028 task: mqtt_task raised exception: EOF occurred in violation of protocol (_ssl.c:2423).
09:01:52.028 task: conn_task created.
09:01:52.028 Connecting to Arduino IoT cloud...
09:01:52.423 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/e/i'.
09:01:52.502 task: mqtt_task created.
09:01:52.611 task: conn_task complete.
09:02:47.067 task: mqtt_task raised exception: EOF occurred in violation of protocol (_ssl.c:2423).
09:02:47.114 task: conn_task created.
09:02:47.114 Connecting to Arduino IoT cloud...
09:02:47.588 Subscribe: b'/a/t/dcac0c9f-5a79-436b-ae27-affd53d742a8/e/i'.
09:02:47.698 task: mqtt_task created.
09:02:47.807 task: conn_task complete.

Okay, five reps is enough. I'm going to press Ctrl+C

Traceback (most recent call last):
  File "C:\Users\Jon Dent\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
  File "C:\Users\Jon Dent\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
  File "C:\Users\Jon Dent\AppData\Local\Programs\Python\Python311\Lib\site-packages\arduino_iot_cloud\ucloud.py", line 380, in run
    await asyncio.gather(*self.tasks.values(), return_exceptions=False)
  File "C:\Users\Jon Dent\AppData\Local\Programs\Python\Python311\Lib\site-packages\arduino_iot_cloud\ucloud.py", line 160, in run
    await asyncio.sleep(self.interval)
  File "C:\Users\Jon Dent\AppData\Local\Programs\Python\Python311\Lib\asyncio\tasks.py", line 639, in sleep
    return await future
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Jon Dent\AppData\Local\Programs\Python\Python311\JMLPoolTempAPI.py", line 28, in <module>
    client.start()
  File "C:\Users\Jon Dent\AppData\Local\Programs\Python\Python311\Lib\site-packages\arduino_iot_cloud\ucloud.py", line 403, in start
    asyncio.run(self.run())
  File "C:\Users\Jon Dent\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
  File "C:\Users\Jon Dent\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 123, in run
    raise KeyboardInterrupt()
KeyboardInterrupt

Thanks again for your interest!

[J-M-L]

OK seems an issue with the access to the API - the error you see usually flags an issue with the SSL/TLS connection used in an MQTT taks

I don't use the API so can't help much...

@pert any idea ?

[c_sarnataro_arduino]

Hi @jddent9822
I built something similar to your project a few days ago, you can have a look at this repo:

csarnataro/build_your_own_iot_app_barcelona_may_2024/blob/main/sketches/soil_humidity_micropython/main.py#L104C5-L104C10

If I understand your sentence right:

it appears to run once and then ...

you would be expecting that your temperature sensor is read continuously like in a standard loop() function (as it is done in C++)

I'm doing my first experiments with micropython, so there might be better solutions, but what I did is to register a Task to mimic the loop function.

csarnataro/build_your_own_iot_app_barcelona_may_2024/blob/main/sketches/soil_humidity_micropython/main.py#L93C5-L93C86

And inside that function I was reading my sensor (in my case, humidity) and send it to the Cloud using client["humidity"] = humidity

Please note that after client.start() is called, the app will stay there forever, so the code you possibly write after this instruction will never be executed.

[jddent9822]

Thanks for the reply, @J-M-L I appreciate your interest. We'll see if @pert has any ideas.

[jddent9822]

Thanks for your response @c_sarnataro_arduino Your Humidity project looks pretty neat! Running micropython on the Nano looks interesting. I'll explore it further. However, I need to run Python (or some language) on a PC to be able to get the data into my Windows 10 registry.

1. ESP32 gathers temperature data and relays to Cloud. Strictly OverTheAir. No wired connection to network or PC. Works great!

2. Python runs on a PC connects to Cloud IOT and receives data for Windows. Errors!

Are you suggesting that I use something from your code to run on my PC to connect to the IOT Cloud?

[c_sarnataro_arduino]

@jddent9822 , my bad, I wrongly assumed you're project was running micropython directly on the ESP32 board.

Now, I assume you followed this tutorial, https://docs.arduino.cc/arduino-cloud/guides/python/ , right? I see the script is quite similar.

I have one question: is the temperature updated in the cloud? I mean, can you see the "Last Value" updating every X seconds in this section? (picture from the tutorial)

[jddent9822]

Hello @c_sarnataro_arduino, Thanks again for your interest.
Yes. My construction of the Tutorial Thing is EXACTLY as the Tutorial instructs. My Python script is EXACTLY as the Tutorial instructs. When I toggle the Switch state in the Dashboard, my Python IDLE Console responds accordingly. (shown below)

08:34:41.358 task: test_switch created.
08:34:41.405 task: conn_task created.
08:34:41.405 Connecting to Arduino IoT cloud...
08:34:41.832 task: discovery created.
08:34:41.895 task: mqtt_task created.
08:34:41.895 Subscribe: b'/a/d/05df0f35-592b-42b3-aabe-a8674a23484f/e/i'.
08:34:42.135 task: conn_task complete.
08:34:42.588 Subscribe: b'/a/t/0ad442d1-d692-4d64-b57d-6c49e97d403c/e/i'.
08:34:42.703 Subscribe: b'/a/t/0ad442d1-d692-4d64-b57d-6c49e97d403c/shadow/i'.
08:34:43.015 Device configured via discovery protocol.
08:34:43.062 Ignoring cloud initialization for record: test_value
08:34:43.193 task: discovery complete.
Switch Pressed! Status is:  True
Switch Pressed! Status is:  False
Switch Pressed! Status is:  True
Switch Pressed! Status is:  False

When I check in the Thing Setup page (shown below), I do have to refresh the page to get the updated Switch state. I assume that is a browser thing because there's nothing to cause it to update automatically.

Now, when I look at my Temperature Thing Setup page (shown below) the Last Value update occurs in the appropriate interval, about every twenty seconds based on Arduino Sketch delays and Cloud Variable Interval setting. I don't have to refresh my browser page.

My Dashboard (show below) also updates in the appropriate interval.

When I run my Python script I get the same output on my console as I have shown in previous posts along with the same errors.
My Thing Status page stops updating even when refreshed. My Dashboard stops updating even when refreshed. It's as if my Cloud Thing has stopped running entirely. I have no way to check on my ESP32 to see if it is still functioning.
When I execute a Ctrl+C on my console to stop my script, I then have to close my Thing Status page as well as my Dashboard and they then begin updating again.
My Python script shown in a previous post is a minor adaptation of the script from the Tutorial but my syntax or something must be wrong. Why does it appear to run once and then error?
Perhaps @pert can weigh in here? Or maybe @J-M-L Or maybe @mirkocurtolo who helped me with an Alexa issue on this project.
Thanks everyone for whatever assistance you can provide.

Update for all. @c_sarnataro_arduino @J-M-L @pert

After reading and re-reading everything I could find about the Arduino Cloud Python Client, I happened to zero in on one paragraph that perhaps should have been written in Bold, All-Caps.

The following is from Python Client | Arduino Documentation by Karl Soderby

"Connection via this client is achieved by registering a manual device, i.e. a virtual device that is not associated with an Arduino hardware board."

I created another virtual Thing with a CloudTemperatureSensor as a variable linked to my actual project Thing variable. After adjusting my Python code to run on that Virtual Device, Voila! No errors! Works like Magic!

I guess I find it a little odd that the Arduino Cloud Python Client can only be used in conjunction with a virtual device and not an actual Arduino board. Maybe this is not true and I'm still missing something. In any case it works.

My next post will hopefully be in the Gallery of IoT Projects.
Again, thanks everyone for all of the help and encouragement!