SerialRPC and Serial.print on PortentaX8

I am currently working with the Arduino Portenta X8 and am trying to get the serial communication to work.
I tried using the py-serialrpc to forward the Serial.println output so I can debug the Arduino script. That didnt work at all and was telling me that certain files are missing.
This person had the same issue:

I used the suggested solution provided in that thread to at least have the docker up and running with your provided python code.

While it is running now, it does not forward any serial output.

Then I tried to get the serialrpc connection running following the tutorial:
https://docs.arduino.cc/tutorials/portenta-x8/python-arduino-data-exchange/
I used the provided python script here aswell, and followed the tutorial thoroughly to push it over to the x8, build the docker and start it. The only change I made to the python script was limit the number of data fields it fetches to 1, the temperature.

On the M4 I run a small sketch, with
RPC.bind("temperature", []{ return getTemperature(); });
in the setup and a small function:
float getTemperature(){
return 5;
}

Now when I load up everything and start it, the python script only tells me:

python-sensor-rpc-python-sensor-rpc-1 |
python-sensor-rpc-python-sensor-rpc-1 | ============================================
python-sensor-rpc-python-sensor-rpc-1 | == Portenta X8 Sensor reading ==
python-sensor-rpc-python-sensor-rpc-1 | ============================================
python-sensor-rpc-python-sensor-rpc-1 |
python-sensor-rpc-python-sensor-rpc-1 | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1 | Unable to retrive data from the M4.
python-sensor-rpc-python-sensor-rpc-1 | Unable to retrive data from the M4.

I tried flashing the device completely with the newest provided image and set it up again from scratch, but nothing helped.

Does anyone have an idea what i'm doing wrong?

The scripts I'm using:

On the M4:

#include <RPC.h>
#include <SerialRPC.h>

void setup(){
  pinMode(LED_BUILTIN ,OUTPUT);
  Serial.begin(115200);
  RPC.bind("temperature", []{ return getTemperature(); });
}

void loop(){
  digitalWrite(LED_BUILTIN , HIGH);
  Serial.println("led on");
  delay(1000);
  digitalWrite(LED_BUILTIN , LOW);
  Serial.println("led off");
  delay(1000);
}

float getTemperature(){
  return 5;
}

Python on the Linux system:

import time
from msgpackrpc import Address as RpcAddress, Client as RpcClient, error as RpcError


Fixed configuration parameters
port = 8884
publishinterval = 5

The M4 Proxy address needs to be mapped via Docker's extra hosts
m4_proxy_address = 'm4-proxy'
m4_proxy_port = 5001

def get_data_from_m4():
    """Get data from the M4 via RPC (MessagePack-RPC)

    The Arduino sketch on the M4 must implement the following methods
    returning the suitable values from the attached sensor:

    
temperature
humidity
pressure
gas
altitude

    """

    rpc_address = RpcAddress(m4_proxy_address, m4_proxy_port)

    data = ()

    try:
        print("Trying to fetch data")
        rpc_client = RpcClient(rpc_address)
        temperature = rpc_client.call('temperature')

        print("temp: ")
        print(temperature)
        data = temperature

    except RpcError.TimeoutError:
        print("Unable to retrive data from the M4.")

    return data


if __name == '__main':

    print()
    print("============================================")
    print("==       Portenta X8 Sensor reading       ==")
    print("============================================")
    print()

    try:
        while True:
            data = get_data_from_m4()
            if len(data) > 0:
                print("Temperature: ", data[0])
            time.sleep(publish_interval)
    except KeyboardInterrupt:
        print('Stopped.')

This error is typically seen (in my brief experience) when there is specifically a failure in the communication between the cores. Since you already have the sketch that binds the call "temperature" to a function that returns a float, we can assume that the call is calling a valid function. My next guess would be that something is going wrong with the system service that handles the communication.

When you run this docker container, I would recommend you monitor m4-proxy.service while this happens. m4-proxy is the systemd service that handles the communication between the Linux & Arduino cores. I've had it be the case that the service will panic due to some unexpected data type somewhere and you can see that in the logs when it happens. On older versions of the Portenta X8 image, m4-proxy.service may fail to restart on error (I don't think the Restart param was in the .service file).
You can view the logs in real time of m4-proxy.service with the command sudo journalctl -u m4-proxy -f.

When your docker container makes the request, you will see it show up in this logs, I forget the exact pattern, but there is an array of numbers that are partially ascii. To quickly view them, I have been just dropping them into a python3 jupyter notebook with bytes([arrayFromJournalctlCommand]), which prints it then in a more human-readable format. In this, you should be able to find your "temperature". The reply from the H7 core should be returned soon after your request, also as an array of numbers. If the call was successful, the 4 bytes that represent the float should be in a small subarray, if not, the array of numbers will contain an ascii-encoded error message, that may or may not cause the service to panic (as it's written in Go).

My apologies for how vague this last paragraph is, but if anyone sees this and it seems to be in the right direction, I will help debug further once I am reunited with my laptop.

This example worked for me, if you haven't seen it you could compare to your files.

It's not working for me :frowning: I'm using latest firmware version (861).

I tried everything, sudo journalctl -fu m4-proxy prints this:

Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 m4_proxy[46282]: panic: reflect: call of reflect.Value.Type on zero Value
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 m4_proxy[46282]: goroutine 34 [running]:
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 m4_proxy[46282]: reflect.Value.typeSlow({0x0?, 0x0?, 0x40000ba001?})
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 m4_proxy[46282]:         reflect/value.go:2610 +0x12c
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 m4_proxy[46282]: reflect.Value.Type(...)
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 m4_proxy[46282]:         reflect/value.go:2605
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 m4_proxy[46282]: github.com/msgpack-rpc/msgpack-rpc-go/rpc.(*Server).Run.func1.1()
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 m4_proxy[46282]:         github.com/msgpack-rpc/msgpack-rpc-go/rpc/server.go:78 +0x1c0
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 m4_proxy[46282]: created by github.com/msgpack-rpc/msgpack-rpc-go/rpc.(*Server).Run.func1
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 m4_proxy[46282]:         github.com/msgpack-rpc/msgpack-rpc-go/rpc/server.go:57 +0xec
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 systemd[1]: m4-proxy.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jul 29 20:59:19 portenta-x8-200fc209dab6fad9 systemd[1]: m4-proxy.service: Failed with result 'exit-code'.

and here is the output of python code:

bash-5.1$ sudo docker compose up
[+] Building 0.0s (0/0)
[+] Running 2/2
 βœ” Network python-blink-rpc_default               Created0.3s
 βœ” Container python-blink-rpc-python-blink-rpc-1  Created0.1s
Attaching to python-blink-rpc-python-blink-rpc-1
python-blink-rpc-python-blink-rpc-1  |
python-blink-rpc-python-blink-rpc-1  | ============================================
python-blink-rpc-python-blink-rpc-1  | ==       Portenta X8 M4 output            ==
python-blink-rpc-python-blink-rpc-1  | ============================================
python-blink-rpc-python-blink-rpc-1  |
python-blink-rpc-python-blink-rpc-1  | Starting RPC loop...
python-blink-rpc-python-blink-rpc-1  | RPC calls: ['count', 'led']
python-blink-rpc-python-blink-rpc-1  | Retrieving data from the M4...
python-blink-rpc-python-blink-rpc-1  | Starting RPC client...
python-blink-rpc-python-blink-rpc-1  | RPC client started.
python-blink-rpc-python-blink-rpc-1  | Unable to retrieve count from the M4.
python-blink-rpc-python-blink-rpc-1  | count: no data!
python-blink-rpc-python-blink-rpc-1  | Starting RPC client...
python-blink-rpc-python-blink-rpc-1  | RPC client started.
python-blink-rpc-python-blink-rpc-1  | Unable to retrieve led from the M4.
python-blink-rpc-python-blink-rpc-1  | led: no data!
python-blink-rpc-python-blink-rpc-1  | Waiting for the next cycle...
python-blink-rpc-python-blink-rpc-1  | ---------------------------
python-blink-rpc-python-blink-rpc-1  |
python-blink-rpc-python-blink-rpc-1  | Retrieving data from the M4...
python-blink-rpc-python-blink-rpc-1  | Starting RPC client...
python-blink-rpc-python-blink-rpc-1  | RPC client started.

here is the modified code (running on port 5000 instead of 5001):

import os
import time
from typing import Any

from msgpackrpc import Address as RpcAddress, Client as RpcClient, error as RpcError
import sys

# how long to wait in seconds between loop cycles
LOOP_INTERVAL = int(os.getenv("LOOP_INTERVAL", 1))

# The M4 Proxy address needs to be mapped via Docker's extra hosts
M4_PROXY_ADDRESS = sys.argv[1] or 'm4-proxy'
M4_PROXY_PORT = int(sys.argv[2]) or 5000
RPC_ADDRESS = RpcAddress(M4_PROXY_ADDRESS, M4_PROXY_PORT)

# list of RPC calls to make
RPC_CALLS = ['count', 'led']


def main() -> None:
    print_banner()

    try:
        rpc_loop(RPC_CALLS, RPC_ADDRESS, LOOP_INTERVAL)

    except KeyboardInterrupt:
        print('Stopped.')
        exit(0)


def print_banner() -> None:
    print()
    print("============================================")
    print("==       Portenta X8 M4 output            ==")
    print("============================================")
    print()


def rpc_loop(calls: list, rpc_address: RpcAddress, interval: int = 1) -> None:
    print("Starting RPC loop...")

    """Print data retrieved from m4 in a constant loop"""
    if not calls:
        print("No RPC calls defined")
    else:
        print(f"RPC calls: {calls}")

    while True:
        print("Retrieving data from the M4...")
        for call in calls:
            if (value := get_data_from_m4(rpc_address, call)) is not None:
                print(f"{call}: ", value)
            else:
                print(f"{call}: no data!")

        print("Waiting for the next cycle...")
        time.sleep(interval)
        print("---------------------------\n")


def get_data_from_m4(address: RpcAddress, key: str) -> Any | None:
    """Send `key` value to RPC client and return received response"""
    try:
        print("Starting RPC client...")
        client = RpcClient(address)
        print("RPC client started.")
        return client.call(key)

    except RpcError.TimeoutError:
        print(f"Unable to retrieve {key} from the M4.")



if __name__ == '__main__':
    main()