Program in the sketch side vs program in the linino side...

[Roque Campos]

Dear Friends,

First I request apologizes about my English, but it is not my native language.

Those days I'm playing around with an arduino yun, trying to develop a remote heating control. This involves a sketch in the arduino side, a php script to interact with an sqlite database on the linino side, and an external Java application that can access  the arduino yun webservices and the webservices implemented with the php scripts.

All is working very well, but sometimes I have thoughts and reflections about the sketch vs php. Actually my sketch is finished, but it takes 90% of the 32kb arduino memory and 50% of the 2Kb ram memory. It runs very well now but I had to optimize it as much as possible to fit  in the arduino. I'm not saying that I'm a C master, I'm sure that the sketch can be more optimized to take less space...

But I was thinking that in the linino side I have more ram available, and more space for programs (I've an sd card with 2Gb). Then the question is, why not develop all the functionalities only in php or phyton? With the rest webservices from the arduino yun, you can interact with the different sensors or relays that may be connected to the board. If real time access to the sensors or realys is not an requirement, I think that I can replace all the sketch functionalities with php and the arduino yun webservices.

And in a php script you can have more power and flexibility to program as in a sketch...

What do you thing about these questions?

Best regards.

R. Campos

[GreyCon]

Hi,
  I am working on something very similar to this for Hot Water control. Here's how I designed it:

1. On the Arduino Side.
The Arduino uses the bridge, and an MQTT Client (http://jpmens.net/2013/10/15/mqtt-on-the-arduino-yun/) to listen on an MQTT Topic. (See http://knolleary.net/arduino-client-for-mqtt/) This is very small, about 15K Flash memory.

Requests come from MQTT Broker, (See next section) in the form of JSON strings.  I then decode these strings to perform commands. (Set Timers, Set real time clock, etc.) I use the topic /Palms/time and /Palms/Timers

2. Linino Side
Here, I have installed the Mosquitto MQTT Broker. (See http://mosquitto.org/2011/08/mosquitto-on-openwrt/)
Now I can use mosquitto_pub on the Yun Linino side to test messages to my /palms/timers topic. Works like a charm.

3. Next, I've written an iPhone app, using the MQTT Client for iOS, which has a nice GUI to set Timers, and then publishes these timers as MQTT messages to the Linino broker. (I've allowed access to this through my router.)

It all works very well indeed... I've been working on various incarnations of this for 2 years, including older Arduinos with WiFi shields, and Web Services on a BeagleBone talking to these, but the Yun and MQTT are the most elegant solution I've found so far.

Hope this give you some food for thought. Let me know if you have any questions or need any code.

Cheers,
Con

[francescofurfari]

Hi Con,

I agree with you that MQTT is a good and quick solution for publishing sensor data. You can also use it for getting data like you do with web services, but in my opinion it is manageble only for small solutions. As a standard solution I would prefer using Rest services for accessing data. I mean that instead of parsing JSON strings to reply to service requests I would publish metadata (by using MQTT)  about the a REST service you can access to get sensor data on demand.

My approach is to use  each Yun asa sensor node or data context source.
The Arduino side maintains the sensor's state and publishes events on state changes meanwhile the Linino side answers to restful requests about the sensor status. At the beginning the arduino side  publishes metadata about the sensor(s) by using a predefined topic: i.e. /ContextBus/sensors/light  with a JSON message reporting Endpoints for the associated service and other useful information.

In this schema the MQTT server (e.g. Mosquitto)  should run on the home gateway; for instance based on RaspberryPi, BeagleBone  or Arduino Yun.

Cheers,
Francesco

[NewLine]

I have also been considering using the linino for the main tasks and intelligence, and let the 'other side' be pretty dumb.
I think this actually means that it is more logical to the linino side being the 'master'.

But this seems to conflict with the way the way an arduino sketch looks like. Up to now I have always been working with the standard IDE and use process() to launch a command on the linino side. Is there a better way of doing this?

In the posts above how do you start the processes on the Linino side?

[GreyCon]

Hi Newline,
  I'm not sure if  I understand you, but to start something like the Mosquitto MQTT Broker at Linux boot time, you create an Init script, which the Linux boot process will run for you. Have a look at http://wiki.openwrt.org/doc/techref/initscripts

Francesco, I have to say, my work to date has always been to use a Linux Web Service , (On BeagleBone) communicating with a MQTT Broker, which in turn communicates MQTT the the ardiono sensors. I talk JSON to the Web Service, and a private, more compressed protocol to the Arduino.

I just recently talked myself into going MQTT end-to-end. I think I just want to try this, and get something going - I've been playing around with various refinements of this project (iPhone to control and monitor domestic water tank) for two years, and I still haven't actually put somethign live :-)

My problem right now is that I am just not comfortable with the bridge right now, and I am having a hard time getting the sequence and timing of all the various components (Arduino Code, Python Bridge and Mosquitto Broker) right. Still, great fun!

Cheers,
Con

[NewLine]

Hi GreyCon,

I see,

  I'm not sure if  I understand you, but to start something like the Mosquitto MQTT Broker at Linux boot time, you create an Init script, which the Linux boot process will run for you. Have a look at http://wiki.openwrt.org/doc/techref/initscripts

That's a good way to avoid having to launch it via the sketch. So if I understand it correctly, the Linino side starts automatically as well as the sketch side (so they run 'asynhcronously') and use MQTT to communicate?

[GreyCon]

Hi NewLine,

   the Yun has 2 separate processors. An Atheros AR9331 chip, which runs Linux, and at ATMega32U4, which runs your sketch code. The 2 processors are connected by a serial link. The AR3991 has a UART normally used by the Linux console, which is hard-wired to the ATMega32U4 Serial port.

So, they're 2 completely autonomous systems. There's some code called the Bridge code which can help you do co-operative processing between the 2 systems. It consists of a C++ class (A Library) which you use in your sketches, so which will run on the Ardiono side, and a Python script which runs on the Linux side.

Hope this helps.
Con

[roadfun]

I have a home monitoring project I started on Uno and recently moved to Yun. I wanted to project to take advantage of Dropbox, Temboo etc and I quickly discovered that although programming everything in a sketch was convenient I rapidly hit the wall on variable storage. Several things I wanted to do required non-trivial strings and those were the main culprit. Since the strings were mainly being used to connect outside the Yun I decided to change the architecture and make fuller use of the Linux side (compared to driving everything via Bridge).

So I enabled cron to start on boot up on the Linux side and created several cron entries to run PHP scripts periodically. For things I want to trigger immediately the sketch side will use the Bridge to poke the Linux side (e.g. via a curl command or Temboo call). But for other items I run a script periodically on the Linux side which looks for data from the sketch side and processes/communicates/posts in response as needed. (BTW I hacked the Temboo SDK so I can do Temboo calls from the Linux side as well as the Arduino side).

As far as which is master, I'm not sure that's a valid or important distinction. The Arduino is still the interface to the "real world". In my architecture, for the most part, the Linux side is the interface to the "connected world".

[NewLine]

I think it is a good idea to start the Linux jobs with e.g. cron jobs. For some reason I always thought of starting them via the sketch. Thanks for the ideas.

[Roque Campos]

Thank you all for your answers. @Newline, roadfun answered your question, I think the best way to execute periodicall tasks is with a cron entry that executes php scripts with the command line "php-cli".

I suppose that the same can be done with phyton scripts but in my case, I have no knowledge in phyton development. I was thinking about the possibility of install a java virtual machine in the linino side but I did not have time enough to test it. And I think that it will not be easy in 64Mb of storage.

At this moment, my arduino sketch is using the 95% of the storage, and I need to implement some functionalities more. I'm not sure it the final program will fit in the storage memory. If not, I will migrate some or all functions to a php script and query the inputs and outputs with webservices... Will see what happens.

Best regards.

[OttoF]

Hello @ all

sounds great, i build my next projects on this way.
I have one more questions,
there are problems to start a php sketch during boot and run them continuously ?
and also run more then one php sketches ?

[Roque Campos]

I'm not a Linux shell master and I have not tried yet but I suppose that in linino you can run a Linux command in the background appending and & to the end of the command.

Then with a cron you can start a script or multiple script which execute commands in the bacground . For example multiple php scripts

[federicofissore]

Just to clarify (quite pretentious, I must admit): if you start your php scripts with cron, you don't need to put them in the background with &

[mediamad]

Hi NewLine,

   the Yun has 2 separate processors. An Atheros AR9331 chip, which runs Linux, and at ATMega32U4, which runs your sketch code. The 2 processors are connected by a serial link. The AR3991 has a UART normally used by the Linux console, which is hard-wired to the ATMega32U4 Serial port.

So, they're 2 completely autonomous systems. There's some code called the Bridge code which can help you do co-operative processing between the 2 systems. It consists of a C++ class (A Library) which you use in your sketches, so which will run on the Ardiono side, and a Python script which runs on the Linux side.

Hope this helps.
Con

I was just struggling with how to communicate between the 2 today.... I've got Python app handling network connections on the linino side and simply want to know from the 32UA side when a DI has closed/opened and then be able to close/open a DO.  The easiest way I can see to communicate between Python and the sketch would be if I could simply monitor the serial connection from within Python and likewise on the 32UA.   Is this possible?  Is there a library on the Python side for this?

[federicofissore]

What about making the sketch "publish" the state of the digital pins on Bridge shared memory storage (Bridge.put and Bridge.get: take a look at Bridge example)
Your python script on the linux side can then poll the storage say every half a second and react to a change