communication between the 2 processors: what API to use

[NewLine]

I want to communicate between the Linino and microntroller. On the Linino side I want to use Python.
I am trying to find out what would be the best way for me to communicate between both. I have been looking around, but am getting a bit confused how the different methods differ, and when to use the one or the other.
Up to now the most influencial has been this thread: http://forum.arduino.cc/index.php?topic=188998.msg1459890#msg1459890
For my application the communication speed is not critical at all. The code space taken is important.
Also in my application the Linino side is in control and will 'push' data (settings) to the arduino side, and 'pull' data (measurements) from the arduino side. It will always be the Linino side that initiates the communication, and the Arduino side will have to respond.

As far as I can see following methods exist:
1- Bridge.put and Bridge.get
2- YunClient and YunServer
3- Mailbox
4- MQTT
5- using 'bare' serial port and linux TTY

Now the questions (they are quite general - and some might not make sense):
- how do 1,2,3 differ?
- is 2 made for pure Arduino - Linino Python communication? To me method 2 seems to be for when you want some 'internet' type of control. Correct?
- it seems to me as if 1 just offers a shared data between both sides, so in my case the Arduino side should pull the data store to see if the Linino has put something in it. Is this a correct way of viewing things?
- when would you use which method?
- In case of YunClient and YunServer. Does the server actually run on the Arduino or on the Linino side?
- Does 4 build on the bridge? What does it offer extra?

[federicofissore]

Now the questions (they are quite general - and some might not make sense):
- how do 1,2,3 differ?

1 is asynchronous: you're "parking" data in the shared storage, but the fact that calling put will immediately return saying "ok" does not mean that the other side has got that data and it's making something with it
2 is synchronous: you're sending commands. A call made with 2 won't end until the other side has processed that command
3 is a mix of them: it's an asynchronous way to send commands

- is 2 made for pure Arduino - Linino Python communication? To me method 2 seems to be for when you want some 'internet' type of control. Correct?

No, you can make a YunServer listen on some port on all network interfaces and have some external (your smartphone for instance) open a bare socket to that tcp port and talk to your sketch

- it seems to me as if 1 just offers a shared data between both sides, so in my case the Arduino side should pull the data store to see if the Linino has put something in it. Is this a correct way of viewing things?

Exact

- when would you use which method?

I would ask myself if I need my sketch to be synchronous or asynchronous when receiving a command

- In case of YunClient and YunServer. Does the server actually run on the Arduino or on the Linino side?

As few things as possible run on the sketch side

- Does 4 build on the bridge? What does it offer extra?

There is no out-of-the-box bridge support to mqtt and I'm not expert either

[scrot]

With regards to the "bare serial port" option , the Firmata sketch combined with PyFirmata make a very easy to use and robust programming model.  You need to disable the console for this to work, but once you do it works like a champ.

[NewLine]

Now the questions (they are quite general - and some might not make sense):
- how do 1,2,3 differ?

2 is synchronous: you're sending commands. A call made with 2 won't end until the other side has processed that command

- is 2 made for pure Arduino - Linino Python communication? To me method 2 seems to be for when you want some 'internet' type of control. Correct?

No, you can make a YunServer listen on some port on all network interfaces and have some external (your smartphone for instance) open a bare socket to that tcp port and talk to your sketch

Thanks for all the clarifications. But I am still wondering about the YunServer and yunClient option (2). What I meant is that all examples I have seen that use YunServer and YunClient have some '3th party' involved with sockets/tcp/network interfaces (I am not so at home in this area). the 3th party could be a webpage, a smartphone, a PC etc...
I dont have any network interface directly (the Linino gets its commands via mail, and forwards some to the sketch side).
Is there any point/way of using YunClient and YunServer for just Linino - Sketch (synchrounous) communication? E.g. suppose the Linino itself (from a cron python script) issues a command to the sketch (e.g. turn an a LED) and the Linino will continue when the sketch has done so. Is that an intended use of YunServer/Yunclient?
The other way (and what I am inclined to at this moment) would be to use the Bridge.put() and Bridge.get() methods to setup a 2-way handshaking so that I can make blocking versions of put() and get().

[federicofissore]

In your project, I would use Bridge.get and put, because IMHO it fits well  and because I like the "set and forget" approach.

But suppose your python script in cron has to report each time it has run with either a SUCCESS or FAILURE status. With a synchronous approach this is a no-brainer. But with an asynchronous approach your script has to wait the "right" amount of time and look for some success confirmation.

[NewLine]

Yes, I was thinking in the same direction. I also think the put and get are the simplest to get started.

One last question: is the data store secured for simultaneous access by the 2 sides? I.e. if both sides are synchronous, they might be trying to access the common data store at the same time. Is this handled correctly by the Bridge library?

[federicofissore]

Bridge queues actions and each is executed sequentially.
However you should be aware of a typical concurrency issue. Say your python script and arduino sketch use a shared "counter" variable. Arduino and python want to sequentially update it. Its initial value is 1, after arduino updates it, it will be 2 and after python updates it, it will be 3. Now suppose the updates are triggered at the same time and this is the real sequence of actions

• Arduino GETs the variable (value 1)


• Python GETs the variable (value 1)


• Arduino PUTs the updated value (2)


• python PUTs the updated value (2)

You would expect 3, but the final value is 2.
In essence, you cannot "lock" access to a shared variable

I don't pretend this to be a clear explanation: the subject is tough and worth tens of books. If you want to know more, you could start from wikipedia

[bilica]

I have an additional question: although option 1 is asynchronous, when bridge.put() returns, the data is guaranteed to be on the datastore, regardless if the other side has used or not.

Rephrasing: bridge.put() blocks the execution until the data has been saved to the datastore ?

Thanks!

Now the questions (they are quite general - and some might not make sense):
- how do 1,2,3 differ?

1 is asynchronous: you're "parking" data in the shared storage, but the fact that calling put will immediately return saying "ok" does not mean that the other side has got that data and it's making something with it
2 is synchronous: you're sending commands. A call made with 2 won't end until the other side has processed that command
3 is a mix of them: it's an asynchronous way to send commands

[chriddyplz]

I asked myself this a couple of weeks ago and wrote a library to make this communication a bit easier.

It's called the YunMessenger, the code and docs are here: https://github.com/plotly/YunMessenger

It uses the Console library and takes care of the handling sockets on the Python side for you. It's synchronous: your message handler on the Linino will fire with the message contents right when it receives the message.

I've been using it for a real-time graphing API that I'm writing at Plotly:
A demo: https://vine.co/v/Mq2LQexrbl7
The development: https://github.com/plotly/arduino-api/tree/master/plotly_yun

I hope you find it useful!

[Sea2605]

SORRY FOR INTERRUPTING YOU GUYS. I'M ALSO INTERESTED IN THE COMMUNICATION BETWEEN THE 2 CPUs. BUT THIS IS THE PERFECT MOMENT TO ASK ANOTHER QUESTION

@chriddyplz:
I'm trying to achieve the same "live"-plotting!

I monitor temperatures with Plotly, but use the python api of Plotly. The Arduino 32u4 measures temperatures with sensors, writes them into a txt-file every 3-4 sec (and shows them on a lcd-display) and a infinitly looping python script (in the linino-side) uploads the latest temperature every 30 sec to plotly. ***

My problem:
Why isn't my graph refreshed/updated on the Plot.ly website on its own (currently I have to press "reload" manually)?
Is this "live"-streaming only possible if I use plotly's arduino sketch code? If I reload the plot.ly-page the latest data is included in the chart...but I'd like to see it "live"/every 30-40s refreshed by itself (a browser autorefresh-plugin is not what I'm looking for ).

This is the code regarding plotly in my python script:

Code: [Select]


py = plotly.plotly( "MYUSERNAME", "MYKEY")

xaxesstyle = {
     "title" : "Time",
   # "titlefont" : {
   #   "color" : "lightgrey",
   #   "family" : "Arial, sans-serif",
   #   "size" : 18,
   # }
     "showticklabels" : True,
     "showgrid" : True,
     "gridcolor" : "#bdbdbd",
     "gridwidth" : 1,

     "zeroline" : True,
     "zerolinecolor" : "#969696",
     "zerolinewidth" : 3
}
yaxesstyle = {
     "title" : "Temperatures in C",
     "showticklabels" : True,
     "showgrid" : True,
     "gridcolor" : "#bdbdbd",
     "gridwidth" : 1,
     "zeroline" : True,
     "zerolinecolor" : "#969696",
     "zerolinewidth" : 3


}

layout = { "title": "PLOT TITLE",
                  "xaxis" : xaxesstyle,
                  "yaxis" : yaxesstyle,
                }

#((THE FOLLOWING LINES  are included in a neverending WHILE-loop (= 1.open txt, 2.put temperatures into variables, 3.upload the variables to Plotly, 4. close the txt, 5. wait until the 32u4 writes new temperatures into the txt, then reopen txt again... ))

        data = [{'x': x,
                       'y': temp1,             # variable 1
                       'name':'Room 1',
                       'type':'scatter',
                       'marker': {'symbol':'square','color':'red'}
                       },
                       {'x': x,
                        'y': temp2,           # variable 2
                        'name':'Room 2',
                        'type':'scatter',
                        'marker':{'color':'orange'}
                        },
                        {'x': x,
                         'y': temp3,          # variable 3
                         'name':'Loggersensor(Temp+Humidity)',
                         'marker':{'symbol':'circle', 'size': humidity3, 'color':'green', 'opacity': 0.50}         # humidity3 is variable 4 (for bubbles)
                         }
                        ]
          response = py.plot(data, filename='TempLog' , fileopt='extend', layout=layout)
          url = response['url']

Where is my mistake?! Why doesn't my chart refresh itself "live"? Why do I have to press manually on "reload" to see the latest temp-data?

*** = if you are interested why I use this method (32u4->AR9331->Plot.ly) instead of the direct way (32u4->Plot.ly):
This way....

1. ....I save a lot of sketch-space and can do almost everything within one file (the python script),
2. ...can be (more) sure that timestamps are correct after a internet or power loss (I get timestamps from the linino which tries to sync to a ntp-server every minute and e.g. halts temperature-uploading when the internet-connection halts or something else happens, so the timeline of my plot won't ever get "out of order, then the script restarts etc....),
3. ....exceptions of all kinds can be handled quite nicely,
4. ....I can change the design/layout etc. of my plot within the python-script quite easily (see my code).
5. ....there is far more information on the internet about python-scripts/-code than there is about arduino-sketches

[Sea2605]

I found an interesting different way to communicate from 32u4 to Ar9331: a command for python/shell, which allows to put variables into python-scripts via shell:
http://docs.python.org/2/library/argparse.html

we basically could put variables from the sketch as arguments into a shell command (which we then execute with runshellcommand() from within the sketch) ....so the linino would run the python script with the variable from the arduino side

was this already mentioned?

[chriddyplz]

Hey Sea2605 --

Sorry I missed this. Your code looks good, but it is using the non-real-time API. There is a slightly different API that is used for the real-time plotting. There is a nice tutorial of its usage here: github.com/plotly/raspberrypi and in here: nbviewer.ipython.org/github/plotly/Streaming-Demos/tree/master/IPython examples/. You can also reach out to me directly at chris[at]plot[dot]ly or at @plotlygraphs on twitter!