Serial communication between Linux-part and Arduino-part

My brains get tired while trying to realize serial communication between the yun linux-part and the yun arduino part. It still does not work.

This communication should take place between a program written in C which runs on the Linux-part and it should send simple messages through ttyATH0 which should be processed by a script which runs on the arduino part. This arduino-script should also be able to write back some responses to the c-program.

I bypassed Bridge by changing /etc/inittab (#ttyATH0::askfirst:/bin/ash --login) and dis- and re-connected the yun.

Strings which I sent from the C-program to the Arduino-script are received by the script, no problem there. But … the other way around; strings written from the Arduino-script back to the C_program do not “arrive”.

Please have a look at the attached .ino and the .c files.
I hope someone can tell me what I do wrong or … maybe it is just impossible ?

arduinoSerial.ino (575 Bytes)

main.c (2.09 KB)

It certainly is possible; I did it, using Java and it was similar to what you did:

  • don't start a terminal on the /dev/ttyATH0 line
  • set the baud rate
  • set the line in raw mode, no echo
  • read an write all you want.

You are using tcget/setattr() for steps 2 and 3; Java can't do that, so i used stty (a separate package) for the purpose ...

kind regards,

Jos

Hi JosAH,
your remark that is possible .... made me try harder and finally the serial comunication works as it should.

Kind regards

Cool; what was the reason you couldn't write back from the AtMega to the Linux side?

kind regards,

Jos

Hey,

I'm interested too. Can you please share your working sketch and c-program ?
Other things to do than change /etc/inittab ?

Thanks for sharing
Leo

Hi,
I will attach the required source-files, then you can see for yourself.

The Arduino-part is just one script-file but the C/Linux part consists of a makefile, a src, inc and obj directory.
I installed yun-gcc and binutils on the Yun.

I did not yet went into comparing the failing version with the version which works now.
The version which works now I wrote a few of years ago and used it for serial communication
between a Linux-PC and an Arduino Uno (through ttyUSBx).

For this Yun-version I replaced the Serial.x calls with Serial1.x calls (and do not use Serial.x calls at all).

I hope this stuff is usefull for you.

Regards

arduinoSerial.zip (3.69 KB)

I only changed /etc/inittab

Btw, I still have to do more testing (but for now it seems to work).

OK thanks a lot !!
I'll give it a try one of these days :slight_smile:

Don't forget to set the printk level to 0 (it's in /etc/rc.local as a comment line if I remember well), because you don't want the kernel babbling to the AtMega ...

kind regards,

Jos

As I said; still some testing had to be done.
Somehow the Arduino-script receives \n characters only (not followed by the relevant text) while the C-program did not explicitly sent them.

Don’t know yet why. But one can simply prevent the script from sending back those empty strings which only contain a \n-character by changing the loop-function as follows (added a test on strlen):

void  loop()
{
    isAv = Serial1.available();
    if (isAv > 0)
    {
        // read the incoming byte:
        incomingByte = Serial1.read();

        if((incomingByte == 10) || (incomingByte == 13) || (i >= (MAX_BUFLEN - 2))) 
        { 
            // Ascii 10 == end of line which means, in this test,
            // that sending of 1 unit of data has completed

            if(strlen(inBuf) > 2)
            {
                sprintf(outBuf,"Arduino to PC: %s\n", inBuf);
                Serial1.print(outBuf);

                // After sending the char-string to the PC, fill inBuf with zero's
                // so there is always a null-terminated char-string
            }
            for(j = 0; j < i; j++)
                inBuf[j] = (char) 0;

            i = 0;
        }
        else
        {
             // as long as the last character has not been read,
             // the incoming bytes are put into a char-string
             inBuf[i] = (char) incomingByte;
             i++;
         }
    }
    else
        delay(5); // there is nothing to read, so wait a few msec's
}

@gtme,
where is this code running?
On the Atmel side or the Linux side?
And what is it trying to do?
Jesse

@jessemonroy650
That little piece of code above (loop()) runs on the Atmel side. It communicates (serial) with a program which runs on the Linux side which is written in C. See the attached zip-file for all required program-code.

-First the atmel code is uploaded through the Arduino-IDE, next the program on the Linux side is started (from within ssh).

-The program on the Linux side, from within a loop, writes a string to the program on the Atmel through /dev/ATH0
and it waits for incoming data (a string written from the Atmel side)
-On the Atmel side the string is read through calls to Serial1.read().
-After a string is read at the Atmel side, it is written back through a call to Serial1.print().
-On the Linux side the string which was written back from the Atmel side is read through a call to read()
-Next, on the Linux side that string is displayed (through printf()).
-and on the Linux side starts another pass through the loop

I wanted to test how to bypass the Bridge and how to directly communicate between AR9391 (Linux) and ATmega32u4 (Atmel) because I assume that the Bridge has a lot of overhead and is much slower than this "direct" communication.

gtme:
I wanted to test how to bypass the Bridge and how to directly communicate between AR9391 (Linux) and ATmega32u4 (Atmel) because I assume that the Bridge has a lot of overhead and is much slower than this "direct" communication.

Yes, the Bridge has some overhead to it, but it also has a lot of power and flexibility, and is pretty easy to use. If you are trying to get a lot of bandwidth between the two processor, and move large amounts of data, they using a direct serial link is a practical solution.

On the other hand, while there is more overhead, the Bridge gives you a variety of communications methods and capabilities, and the they can even be used at the same time. For example, I have one personal project where I have three different Python scripts running independently. They are all started by the sketch (no need to SSH in to Linux to start the scripts) and they all communicate independently with the sketch without having to write a central communications process that sorts out the source/destination tasks - it's like having three (or more) independent serial ports between the two processors.

I wrote that I wanted to bypass the Bridge and use direct serial communication between Linux and Atmel for the reason that I assumed that the Bridge has a lot of overhead.

I do agree that many applications can benefit from the Bridge but I think there is another reason to not always depend on the Bridge:

By using the Bridge as starting point for communication between Linux and Atmel, one accepts a program design where the centre of the system which you want to program is the Atmel (Linux as Atmel slave).

I can think of applications where a Linux program is the centre of the system from where one can make calls (through serial communication) to the Atmel part. So the Atmel just executes commands issued from the Linux part (Atmel as Linux slave).

For example I wrote a C-program to manage an irrigation system. It had to be a multi-threaded program. This C-program obtains its data for the irrigation settings from an external (far away) web-server through a socket. Those data contain info on when pumps and valves have to be activated and for how long and also data on how to deal with changes in temperature (when it is hot plants need more water).
When the time is there, the program just has to send a message to the Atmel to start a pump and open a valve. When the time is over, the program just sends the message to the Atmel to close a specific valve and if necessary to stop the pump.
While irrigation takes place, the C-program, in another thread, also checks on the remote web-server whether a user has changed irrigation settings and if necessary send a message to the Atmel to stop irrigation.

Here the only function of the Atmel was to open or close certain relays as well as to provide the Linux-part with sensor readings on temperature.
This way the program-code for the Atmel can be kept very simple apart from the fact that one has to write a simple command-interpreter in order to make the Atmel understand the meaning of the commands sent from the C-program on the Linux-part.

Since it had to be a multi-threaded program I choose for the Linux-part as the central part of the application (Atmel does not really supports multi-threading).
Before I found out that Linux and Atmel can directly communicate through /dev/ATH0, the communication went through files. So I’m glad that this direct serial communication is possible because communication through files is not so fast.

@gtme,

From my bookmarks, the following threads relate to your question and quest:

I want to note someone said "delete" this or that. DO NOT DO THAT, comment out any unwanted lines and add comments.

Best of Luck
Jesse

gtme:
I wrote that I wanted to bypass the Bridge and use direct serial communication between Linux and Atmel for the reason that I assumed that the Bridge has a lot of overhead.

I read that, and I was keying in on your comment that you "assumed" there was overhead. Of course there's overhead, there's overhead in everything you do, even if you write your own communications protocol. The real question is whether that overhead is too much? Have you tried to quantify it in your application and found the overhead to be intolerable? By your "assume" comment it would appear that you haven't.

Sure, there are high bandwidth or very low latency applications where every microsecond counts. For such an application, you need to take extreme measures, and writing your own direct protocol may be part of it. But for most applications, it's probably not necessary. Of course, if you want to write your protocol, just for the sake of writing it, go for it. But don't be too quick to dismiss the Bridge, there is a lot there that can make your life easier.

By using the Bridge as starting point for communication between Linux and Atmel, one accepts a program design where the centre of the system which you want to program is the Atmel (Linux as Atmel slave).

I wholeheartedly disagree with this statement - the Bridge forces no such architecture. My design goals when developing Yun projects is to have the Linux side be the master - it's does all of the heavy processing and communications, and makes all of the decisions - the sketch is the slave and acts as a dumb I/O processor. And this is all done using the Bridge library, so clearly it does not force the sketch to be the master.

I can think of applications where a Linux program is the centre of the system from where one can make calls (through serial communication) to the Atmel part. So the Atmel just executes commands issued from the Linux part (Atmel as Linux slave).

And I have written several systems using just that paradigm, except that the Linux processes (one of my projects has three independent processes) sends those commands through Process objects, not the raw serial port.

When the time is there, the program just has to send a message to the Atmel to start a pump and open a valve. When the time is over, the program just sends the message to the Atmel to close a specific valve and if necessary to stop the pump.

So this clearly is NOT one of the high bandwidth or low latency exceptions that I mentioned earlier. You are not sending a lot of data, so transmission speed is not an issue. It will likely take a few seconds for the pump to get up to speed and build pressure, the valves to open, and the lines to pressurize and water to start flowing, so a few tens or even hundreds of milliseconds of latency will make no difference.

This sounds just like my lighting controller project: the Linux processes are in charge, the sketch only receives on/off commands and sends raw analog current readings, and the Linux side is also acting as a web server and web application, plus sends out alert emails and text messages, and also processes incoming email commands and status requests. All using the Bridge.

This way the program-code for the Atmel can be kept very simple apart from the fact that one has to write a simple command-interpreter in order to make the Atmel understand the meaning of the commands sent from the C-program on the Linux-part.

I fully agree that limiting the code and the intelligence in the sketch is the way to go. But you don't have to write your own custom communications handler to do so.

Before I found out that Linux and Atmel can directly communicate through /dev/ATH0, the communication went through files. So I'm glad that this direct serial communication is possible because communication through files is not so fast.

Ouch! Using files is most certainly not the way to do it. If you think that's what you have to do when using the Bridge library, it's no wonder it left a bad taste in your mouth.

You really owe it to yourself to do a little experimenting with the Process class. Once you get the process started, the sketch talks to the Process in exactly the same way as the Serial and Serial1 interfaces - they all derive from Stream, so once the object is created, you interface to all of them the same way. The advantage over rolling your own communications really comes into play on the Linux side: instead of having to set up and configure the serial port, and explicitly sending data through it, the Linux process simply reads from STDIN and writes to STDOUT. This makes it real easy to test by simply running the process on the SSH command line: you can see everything that is sent to the sketch, and can manually type in responses from the sketch to test various situations. The only caveat us that you probably have to enable unbuffered mode, which is usually pretty easy: for example, in Python, it's simply a matter of adding the "-u" option to the shebang on the first line of the script.

So, using a Process object over rolling your own really doesn't change anything in the script. And it simplifies the Linux process I/O. But where it really shines is when you want more than one process to talk to the sketch: to do it your way, you need to write an overall process that manages the communications, and then it must creat additional threads or sub-processes and pass the communications on to them. With the Bridge library, just create another Process instance, and it keeps everything straight - it's as if each had their own dedicated serial port.

Now, while it is true that the sketch must initially launch the Process object, rather than having Linux launch it, that does not mean that the sketch is the master. I launch the Process object to run asynchronously, and the Linux process is set up to run indefinitely. You may think that having the sketch launch the process is a disadvantage, but I see it as an advantage: you don't have to mess with cron, rc.local, or have to manually SSH in to start the process, and the sketch can act as a watchdog where it monitors whether the process is still running and automatically restarts it if it crashed - that's easy to do in the sketch, and much more difficult to do locally in Linux.

I'm not saying that the Bridge library is perfect and is always the way to go - it depends on the application. But I haven't heard anything yet in your descriptions that precludes using the bridge - it can do everything you've stated in this thread, and can do it well. Don't be so quick to dismiss it.

@ShapeShifter
Hi, thanks for your reaction.

It is clear that you have good experiences with the Bridge.
I will follow you recommendation and do more experimenting with the Bridge.

I'm especially interested in the communication between atmel and linux when the linux program spawned by the atmel is running in an (endless) loop and communication takes place through STDIN en STDOUT.

Have a nice day.

Here is a very simple example. The sketch spawns a Python process (and keeps restarting it if it dies for whatever reason.) Of course, it doesn’t have to be Python, it can be any of the interpreted or compiled languages that the Yun supports.

This is all too long for a single post, so I’m breaking it up. First, the sketch:

#include <Bridge.h>
#include <Process.h>

// A simple process demo.
//
// The sketch uses a Bridge library Process object to start a Linux process,
// in this case it's a simple Python script.
// That script sends simple commands to the Yun:
//    '0' tells this sketch to turn the D13 LED off
//    '1' tells this sketch to turn the D13 LED off
//
// In addition, this script debounces and monitors digital input D2,
// which is assumed to have a switch to ground (uses internal pullup):
//    If the switch input becomes grounded, sends the line "Low" to the Linux process
//    If the switch input becomes open, send the line "High" to the Linux process
//
// The sketch starts the process, and monitors that it's still running. If the
// process has terminated for whatever reason, the process is restarted.
//
// That's all the sketch does. What happens with a button press, and how the LED
// is controlled, is completely up to the Linux process.


// Pin definitions
#define  SW    2
#define  LED   13

// A Process object to manage control of a Python process.
Process Proc;



void setup()
{
   pinMode(SW, INPUT_PULLUP);
   pinMode(LED, OUTPUT);
   digitalWrite(LED, HIGH);
   
   delay(5000);      // Give a little startup delay to not interrupt the Linux boot process.
   
   Bridge.begin(); // Initialize the Bridge
   digitalWrite(LED, LOW);
}



void loop()
{
   static int lastSW = HIGH;
   
   // If the process is not currently running start (or restart) it.
   if (!Proc.running())                         // Is the process running?
   {
      Proc.close();                             // Close out the old instance.
      Proc.begin("/mnt/sda1/processDemo.py");   // Initialize the process
      Proc.runAsynchronously();                 // Run the process asynchronously
   }

   // Check for and handle incoming data from the process
   while (Proc.available())                     // As long as there is data available...
   {
      switch (Proc.read())                      // Read and decode the next incoming character.
      {
         case '0':   digitalWrite(LED, LOW);    // Turn off the LED
                     break;

         case '1':   digitalWrite(LED, HIGH);   // Turn on the LED
                     break;

         // Anything else is ignored.
      }
   }

   // Process the input. Really should be debounced, not going to worry about that now...
   int sw = digitalRead(SW);                    // Read the input state.
   if (sw != lastSW)                            // Has the input changed?
   {
      if (sw)                                   // Is the switch input currently high?
         Proc.println("High");                  // Tell the process the input is high
      else
         Proc.println("Low");                   // Tell the process the input is low.

      lastSW = sw;                              // Remember the new state for next time.
   }
}

It starts out with the normal stuff to set up the I/O ports, and start the Bridge. While a global instance of the Process object is defined, I don’t start the process in setup() as would be the normal situation. I do that because I always want it running, so I check if it’s running in loop() and start it if it’s not. If I were to start in setup(), I would still have to restart it in loop(). I don’t want to paste the same string in two places, or make the same call in two places if not necessary. Sometimes, I write a function to start the process, and call it in setup(), and also in loop() if the process is not running, but I’m just trying to keep it simple here - on the first pass of loop(), the process is not running, so it is started. Now, if there is some code in setup() that requires the process to already be running (you want to get some initial data form the process?) then you would want to start it in setup(). Most of the time, waiting to start/restart it in loop should work. It certainly works here.

Inside loop, it looks to see if the process is running, and if not, it restarts it.

Next, it looks to see if there are any characters available from the process. If so, it loops through all available input, and checks for the special values ‘0’ or ‘1’ and turns the LED on or off. Of course, your message list and how you decode it is up to you, it can be as complex or simple as you want. I kept it very simple here so as not to clutter things up. It’s the communications I’m trying to show, not the message parsing methods. The thing to remember is that the Process object inherits from Stream, just as the Serial and Serial1 object does. So if you have a favorite way to read data from the Serial object, that method should work just fine with the Process object.

After handling any input, the sketch does some very simple checking of the switch input, looking for any changes, Any changes are sent down to the process. Again, what you send can be as simple or complex as you like.

And that’s it. The sketch keeps the process running, it reacts to input from the process, and sends it status updates. The sketch is making no decisions on what to do about the data, it really is just a dumb I/O processor.

Part 2:

Now for the Python script that gets run on the Linux side. Again, I'm trying to keep it as simple as possible:

#!/usr/bin/python -u

import sys
import select

# A loop that repeats forever
while (True):

   # Check if there is a line of input ready. Wait up to 0.1 seconds
   # This way the script doesn't block on the input, and can continue
   # to do other things even if there is no data ready.
   ready = select.select([sys.stdin], [], [], 0.1)[0]
   if ready:

      # There is a line of input ready from the sketch
      line = sys.stdin.readline()
      line = line.rstrip()   # Strip whitespace from the end of the line

      if (line == "High"):   # Is the switch input high?
         print('0')          # Turn off the LED

      elif (line == "Low"):  # Is the switch input low?
         print('1')          # Turn on the LED

The shebang on the first line says that this should be run with the Python interpreter, which should use the '-u command line argument to use unbuffered I/O mode. Other languages may handle things differently, but the default for Python is to use buffered mode. This script sends a single character at a time, and buffering would mean that nothing would be sent out by the Python script until the output buffer filled up (which would take a long time) or until the script exited (which by design should never happen.)

The Python code enters an infinite loop. (Any one time setup code would be before the loop.)

At the top of the loop, the select() system function is used to check for any input on STDIN. I only do this so the read can time out, and other things can happen even if there is no input (although this particular script doesn't do anything else.) If the script made a simple blocking read on STDIN, nothing else would happen until something was received. Using the non-blocking read is a lot simpler than starting a separate thread to handle background tasks. To each their own, there are a lot of different ways to do this.

If there is a line of input ready, it's read in, and the trailing whitespace is removed (this will be at least the trailing newline character.) If the line is not blank, it's compared to the list of known commands:

  • if the input is High, the LED is turned off.
  • if the input is low, the LED is turned on.

In this way, it works like a simple pass-through: whatever happens on the input pin is echoed on the output. This is an overly simple script, but it shows that it can react to inputs and control outputs. Sure, you could do that a lot simpler inside the sketch, but the point is to demonstrate the communications without getting bogged down in details. Of course, the Linux process would be doing something far more interesting than merely translating and echoing the inputs. This is not a demo on how to do some interesting processing in Linux, it's demonstrating getting the data up and down between the two processors.

What I feel is the real benefit of this method is the ability to monitor the Linux process and restart it. While this is running, SSH into the Yun, and run the ps command. You will be able to see that the processDemo.py script is running. Note it's task number, and use the kill command to stop that process. Now, run the ps command again. The processDemo.py script is still shown as running, but it has a different process number now. That shows that you really did kill it, and that the sketch did properly restart the process. That's much more complicated to do in Linux without the sketch's help. If you are manipulating the switch on the digital input, you will see that it hardly skips a beat while the process is restarted.

This is slightly simpler that disabling the serial port in Linux, and manually opening up the serial port yourself. But that difference is not really enough to sway anyone into doing it one way or another. The real benefit of using the Bridge is that you can do several different things at once. For example, you can talk independently to another Linux process by just creating and accessing a second (or third, or fourth, or...) Process object. (Note that one Process object can only run one Linux process at a time: you can run multiple Linux processes, but each one must have it's own Process object.)

Or, you could do things like read/set Bridge variables in addition to talking to the Process object. I didn't do it in this simple demo sketch, because I don't want to clutter things up, but normally, once the Bridge is started, I will do a few Bridge.put() calls:

  Bridge.put("SketchName", "SimpleProcessDemo");
   Bridge.put("SketchVersion", "1.0");
   Bridge.put("SketchDate", __DATE__ " " __TIME__);

Now, even though I don't do anything with that data in the sketch or the Linux process, it lets me use a web browser to access http://yun1.local/data/get and I end up with a response of:

{"value":{"SketchVersion":"1.0","SketchName":"SimpleProcessDemo","SketchDate":"Apr  4 2016 17:13:27"},"response":"get"}

This gives me an easy way of verifying what version of what sketch is loaded in the Yun. Besides the Bridge variables, any of the other Bridge library functions can still be used, like sending Mailbox messages to the Yun, or using the Console class to give you remote networked debugging output from the sketch.

This looks very interesting.
The script is clear and since C/C++ is my first language, it also doesn't seem very complicated to translate Python into C.

Thanks for all information you share.
I'm going to play and experiment with the Bridge and after that I will will again evaluate it against "direct" serial communication through /dev/ATH0.

For now, I have the impression that this direct serial communication could be enough for simple communication but if one needs more complicated communication, then one could save a lot of programming time by using the Bridge.

Anyway, our conversation was very stimulating for me (sorry for my english; it's not my first language), thanks again.