Portenta X8 RPC tutorial is not working on the latest firmware

michelgokan · July 30, 2024, 5:23pm

I followed this tutorial to establish a "Data Exchange Between Python on Linux & Arduino Sketch": https://docs.arduino.cc/tutorials/portenta-x8/python-arduino-data-exchange/

I'm using the latest firmware image (861 (Release arduino-91.2)) on my Portenta X8 without any breakout board attached.

I have py-serialrpc running as follows:

bash-5.1$ sudo docker compose up
[+] Building 0.0s (0/0)
[+] Running 2/2
 ✔ Network py-serialrpc_default           Created0.2s
 ✔ Container py-serialrpc-py-serialrpc-1  Created0.1s
Attaching to py-serialrpc-py-serialrpc-1
py-serialrpc-py-serialrpc-1  | server: m4-proxy, port: 5000

this is the output of my sudo journalctl -u m4-proxy -f:

bash-5.1$ sudo journalctl -u m4-proxy -f
Password:
Sorry, try again.
Password:
Jul 30 15:30:23 portenta-x8-200fc209dab6fad9 systemd[1]: Started M4 builtin led forwarder.
Jul 30 15:30:29 portenta-x8-200fc209dab6fad9 m4_proxy[750]: Registering service on port  5005  with functions  map[count:5005 len:5005 tty:5005]
Jul 30 15:31:05 portenta-x8-200fc209dab6fad9 m4_proxy[750]: Registering service on port  5005  with functions  map[count:5005 len:5005 tty:5005]

Here is the Arduino Sketch code according to the python-sensor-rpc mentioned in the tutorial without any sensors:

#include <RPC.h>
#include <SerialRPC.h>
#include <Wire.h>

void setup()
{
    Serial.begin(115200);
    Serial.println("BME680 test on M4");
    Wire.begin();

    Serial.println("Trying to find sensor...");
    
    //RPC.bind("status", []{ return bme.sensorID() == 0x60; });
    RPC.bind("temperature", []{ return 100; });
    RPC.bind("humidity", []{ return 200; });
    RPC.bind("pressure", []{ return 300; });
    RPC.bind("gas", []{ return 400; });
    RPC.bind("altitude", []{ return 500; });

    Serial.println("Starting");
}

void loop()
{
  Serial.print("Temperature = ");
  Serial.print(100);
  Serial.println(" *C");

  Serial.print("Pressure = ");
  Serial.print(200);
  Serial.println(" hPa");

  Serial.print("Humidity = ");
  Serial.print(300);
  Serial.println(" %");

  Serial.print("Gas = ");
  Serial.print(400);
  Serial.println(" KOhms");

  Serial.print("Approx. Altitude = ");
  Serial.print(500);
  Serial.println(" m");

  Serial.println();

  delay(1000);
}

...and here is the python code:

import time
from msgpackrpc import Address as RpcAddress, Client as RpcClient, error as RpcError


# Fixed configuration parameters
port = 8884
publish_interval = 5

# The M4 Proxy address needs to be mapped via Docker's extra hosts
m4_proxy_address = 'm4-proxy'
m4_proxy_port = 5005

def get_data_from_m4():
    """Get data from the M4 via RPC (MessagePack-RPC)

    The Arduino sketch on the M4 must implement the following methods
    returning the suitable values from the attached sensor:

    * `temperature`
    * `humidity`
    * `pressure`
    * `gas`
    * `altitude`

    """

    rpc_address = RpcAddress(m4_proxy_address, m4_proxy_port)

    data = ()

    try:
        rpc_client = RpcClient(rpc_address)
        temperature = rpc_client.call('temperature')

        rpc_client = RpcClient(rpc_address)
        humidity = rpc_client.call('humidity')

        rpc_client = RpcClient(rpc_address)
        pressure = rpc_client.call('pressure')

        rpc_client = RpcClient(rpc_address)
        gas = rpc_client.call('gas')

        rpc_client = RpcClient(rpc_address)
        altitude = rpc_client.call('altitude')

        data = temperature, humidity, pressure, gas, altitude

    except RpcError.TimeoutError:
        print("Unable to retrive data from the M4.")

    return data


if __name__ == '__main__':

    print()
    print("============================================")
    print("==       Portenta X8 Sensor reading       ==")
    print("============================================")
    print()

    try:
        while True:
            data = get_data_from_m4()
            if len(data) > 0:
                print("Temperature: ", data[0])
                print("Humidity: ", data[1])
                print("Pressure: ", data[2])
                print("Gas: ", data[3])
                print("Altitude: ", data[4])
            time.sleep(publish_interval)
    except KeyboardInterrupt:
        print('Stopped.')

Here is the error after bringing up the container:

bash-5.1$ sudo docker compose up
[+] Building 0.0s (0/0)
[+] Running 2/2
 ✔ Network python-sensor-rpc_default                Created0.2s
 ✔ Container python-sensor-rpc-python-sensor-rpc-1  Created0.1s
Attaching to python-sensor-rpc-python-sensor-rpc-1
python-sensor-rpc-python-sensor-rpc-1  |
python-sensor-rpc-python-sensor-rpc-1  | ============================================
python-sensor-rpc-python-sensor-rpc-1  | ==       Portenta X8 Sensor reading       ==
python-sensor-rpc-python-sensor-rpc-1  | ============================================
python-sensor-rpc-python-sensor-rpc-1  |
python-sensor-rpc-python-sensor-rpc-1  | Traceback (most recent call last):
python-sensor-rpc-python-sensor-rpc-1  |   File "/app/m4_to_python.py", line 65, in <module>
python-sensor-rpc-python-sensor-rpc-1  |     data = get_data_from_m4()
python-sensor-rpc-python-sensor-rpc-1  |   File "/app/m4_to_python.py", line 33, in get_data_from_m4
python-sensor-rpc-python-sensor-rpc-1  |     temperature = rpc_client.call('temperature')
python-sensor-rpc-python-sensor-rpc-1  |   File "/usr/local/lib/python3.9/site-packages/msgpackrpc/session.py", line 41, in call
python-sensor-rpc-python-sensor-rpc-1  |     return self.send_request(method, args).get()
python-sensor-rpc-python-sensor-rpc-1  |   File "/usr/local/lib/python3.9/site-packages/msgpackrpc/future.py", line 45, in get
python-sensor-rpc-python-sensor-rpc-1  |     raise error.RPCError(self._error)
python-sensor-rpc-python-sensor-rpc-1  | msgpackrpc.error.RPCError: b"'temperature' method not found"
python-sensor-rpc-python-sensor-rpc-1 exited with code 0
bash-5.1$

Any ideas how to solve this?

UPDATE: I downgraded the firmware all the way down to version 746. I followed the tutorial once again, without changing anything, even I left the ports to 5001. I'm now getting this in m4-proxy output:

Jul 30 21:53:04 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: Serving 172.23.0.2:37126
Jul 30 21:53:04 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: [0 0 [116 101 109 112 101 114 97 116 117 114 101] []]
Jul 30 21:53:04 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: {[148 0 0 171 116 101 109 112 101 114 97 116 117 114 101 144] 0 0}
Jul 30 21:53:04 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: wait for response
Jul 30 21:53:05 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: done
Jul 30 21:53:20 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: Serving 172.23.0.2:37128
Jul 30 21:53:20 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: [0 0 [116 101 109 112 101 114 97 116 117 114 101] []]
Jul 30 21:53:20 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: {[148 0 0 171 116 101 109 112 101 114 97 116 117 114 101 144] 0 0}
Jul 30 21:53:20 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: wait for response
Jul 30 21:53:21 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: done
Jul 30 21:53:36 portenta-x8-200fc209dab6fad9 m4_proxy[20457]: Serving 172.23.0.2:37130
...

...and this in python-sensor-rpc output:

sudo docker compose up
[+] Running 2/2
 ⠿ Network python-sensor-rpc_default                Created 0.2s
 ⠿ Container python-sensor-rpc-python-sensor-rpc-1  Created 0.1ss
Attaching to python-sensor-rpc-python-sensor-rpc-1
python-sensor-rpc-python-sensor-rpc-1  |
python-sensor-rpc-python-sensor-rpc-1  | ============================================
python-sensor-rpc-python-sensor-rpc-1  | ==       Portenta X8 Sensor reading       ==
python-sensor-rpc-python-sensor-rpc-1  | ============================================
python-sensor-rpc-python-sensor-rpc-1  |
python-sensor-rpc-python-sensor-rpc-1  | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1  | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1  | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1  | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1  | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1  | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1  | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1  | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1  | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1  | Unable to retrive data from the M4.
...

I dont' see any output in py-serialrpc output (I had to pip install backports.ssl_match_hostname and backports.weakref to make dockerfile compile though):

sudo docker compose up
[+] Running 2/2
 ⠿ Network py-serialrpc_default           Created 0.1s
 ⠿ Container py-serialrpc-py-serialrpc-1  Created 0.1ss
Attaching to py-serialrpc-py-serialrpc-1
py-serialrpc-py-serialrpc-1  | server: m4-proxy, port: 5000

Any ideas what's wrong? I can't even see any of the Serial outputs, this makes the whole board useless..

brandonbeattie23 · September 20, 2024, 8:01pm

I'm having the same problem with the latest version now, 881.

I have an older version of the OS, I think somewhere in the 300's, that the example still works on. I'm not sure at what version it stopped working entirely.

I need to upgrade to the newer OS because of issues I'm having with devicetree & u-boot where these issues seem to no longer be a thing, but I absolutely need the linux<->arduino connection to be working, so I am hopeful this can be fixed soon. With any luck, they may have just made a change that requires slightly different code for m4-proxy or the kernel modules regarding the x8h7.

If you are not doing anything funky with editing the devicetree, I can send you an older copy of the OS that for sure works with this example if you'd like. It works perfectly fine for me except for the devicetree thing and the annoying terminal wrapping quirkiness.

internmoxy · September 21, 2024, 2:56am

Hey Brandon,

I am also having a similar issue getting m4-proxy to work and in the same situation as michelgokan. Mind if I ask what version OS youre using when you got it working?

craftysupply · September 25, 2024, 1:56am

I've spent about 16 hours trying to get this to work before I found this post. I guess I'm happy to know it's not something I did wrong but I am definitely extremely annoyed that I can't even get to use this on my project. Has anyone figured any way around this yet?

thayit · January 2, 2025, 11:13pm

Hi all,
Is there any solution for that? I need to communicate between Arduino and Linux.

tuggybear · January 5, 2025, 10:19pm

I am in the same situation.... it would be good, if Arduino Team can provide any information/update on the progress of a solution here...

thayit · January 7, 2025, 5:38pm

They have sent a message to me about the issue. I dropped below:

"I appreciate you taking the time to report this. This is a known issue and our team is actively working on a solution. The latest image version they are working on will fix this problem. Currently, there is no estimated time frame for the new release to be published.

I am sharing with you the official Arduino Repository, releases section where you can check the roadmap of updates. The current image version is 881-arduino-x8. Once the new release is ready you will find them named the same, except for the first three numbers, which will change according to the revision.
Once ready, this is a guide that explains some different methods to update your Portenta's X8 image: 09. How to Update Your Portenta X8.

From Arduino, we apologize this situation represents an inconvenience for your ongoing project."

thayit · February 4, 2025, 6:53pm

I received a suggestion email from the Arduino team, tried it and it worked!

First we need to push the binary (attached) into Linux, you can use either adb or ssh to do so:

adb push STM32H747AII6_CM7.bin /home/fio

Now from inside the Linux enviorment on the X8 we need to run the following commands:

sudo mount -o remount,rw /usr

sudo cp STM32H747AII6_CM7.bin /usr/lib/firmware/arduino/stm32h7-fw/STM32H747AII6_CM7.bin

sudo /usr/arduino/extra/program.sh

Afterwards you can test with the RPC functionality again.

Hopefully, it works for you as well..
STM32H747AII6_CM7.bin.zip (44.2 KB)

tuggybear · February 8, 2025, 4:17am

Hello thayit,

Tranks for sharing a solution. May I ask from where you downloaded the firmware for the STM32H7? Is it a new version from Arduino GitHub? Maybe you can provide more details on the binary file you have uploaded.

Trank you

Kind regards,
tuggybear

tuggybear · February 16, 2025, 6:20pm

Hello thayit,

I am well aware of, that the "firmware" I have mentioned in my previous reply is not the Linux firmware of the Portenta X8. I just would like to know, if Arduino Team sent you the binary file (firmware for the STM32H7!!) via E-Mail or something else. Maybe you can provide more information on this.

Thanks you

Kind regards,
tuggybear

thayit · February 17, 2025, 6:08am

Hi tuggybear,
Arduino team sent me the .bin file via e-mail.

4545867485 · February 28, 2025, 10:52am

For those concerned about loading up a .bin file that is just posted here, you can open up the .bin using HxD, a binary viewer. If you scroll down to 00040090 it says "Portenta X8 - STM32H7 companion fw". Which a least is an indicator that this is really portenta X8 firmware. I can't state 100% it's source is secure but at least it appears to be specifically arduino / Portenta X8 firmware.

Topic		Replies	Views
SerialRPC and Serial.print on PortentaX8 Portenta X8	3	625	July 29, 2024
Portenta X8 RPC Communication Issue: Python<->Arduino Data Exchange Not Working Even By Following the Official Tutorial Portenta X8	0	92	July 30, 2024
Using RPC communication between iMX8 and M4 cores on a Portenta X8 Portenta X8	2	128	November 23, 2024
Problem with RPC on Linux side Portenta X8	1	472	August 18, 2023
Issues with M4 Proxy on Portenta X8 in Docker-Container Portenta X8	1	129	November 14, 2024

Portenta X8 RPC tutorial is not working on the latest firmware

Related topics