Allow YunServer to manage multiple YunClient connections

This thread is in response to a question raised in this other thread: Send data from Arduino yun to particular IP address

I'm making a different thread to prevent clutter in the original thread, because this discussion is basically the opposite of what was discussed in that other thread: it talked about making an outgoing connection to a server, while this one is talking about being a server and accepting multiple incoming connection. This discussion may be triggered by that thread, but it's really a different topic.

This is the original question:

zerox12:
Great post! Your example is exactly what I was looking for. I just want simple communication of data between Yuns.

However, how can the Yun Server keep multiple TCP connections open at once? My project could have up to 12 Yun clients that all need to send data to a "master" Yun. Any idea how to do this?

This was my original response:

ShapeShifter:
I do have an example sketch that supports two client connections. But since writing that sketch I have learned that one must clean up connections after the client closes down, or else YunServer stops accepting clients after 256 connections are made.

Give me a chance to clean up the code a little bit and do better housekeeping, and I will post the example.

Just be aware that RAM on the '32U4 is limited, and each connection will need a chunk of memory. It might not scale up well. If you're getting into a bunch of connections (like the dozen you need) you might be better off managing the network communications on the Linux side. This is especially true if the Yun is acting as data concentration point where it's storing the data from the 12 clients in an SD card file or database. Someone more experienced on the Linux side might have good suggestions.

And this is the example sketch I came up with:

// Yun Multi Client Example

#include <Bridge.h>
#include <YunServer.h>
#include <YunClient.h>

// Definition of the port that the server uses to listen
// for incoming connections. Set to port 23 for now so
// a standard Telnet client can make a connection. For
// actual deployment, a different application specific
// port might be a better choice since this is really
// just a generic TCP server, not a Telnet server (which
// implies an additional terminal protocol riding above
// the TCP stream.)
#define PORT 23


// The one global server object listening for incoming connections.
YunServer Server(PORT);

// Define an array of simultaneous clients
#define  NUM_CLIENTS    12
YunClient clients[NUM_CLIENTS];


void setup()
{
   Bridge.begin();

   Serial.begin(19200);
   while (!Serial)
      ;  // Wait until a Serial Monitor is connected.

   Serial.print("Multi client test sketch listening on port ");
   Serial.println(PORT);
   Serial.print("Supports ");
   Serial.print(NUM_CLIENTS);
   Serial.println(" simulataneous client connections");

   Server.noListenOnLocalhost(); // Listen for external connections
   Server.begin();
}


void loop()
{
   // There are two interesting flags in the client:
   //    opened    - read by converting the client object to a bool.
   //    connected - read by calling the connected() method
   // This gives four combinations:
   //    closed and no connection   - Try to accept a new connection.
   //    closed and connected       - Not possible.
   //    open and no connection     - Connection just closed, stop the client to clean up.
   //    open and connected         - Ready to go, process the connection

   // Process every client
   for (byte i = 0; i<NUM_CLIENTS; i++)
   {
      if (clients[i].connected())            // Is it connected?
         process(i);                         // It's connected, process the client
      else
      {
         if ((bool)clients[i])               // Not connected. Is the client open?
         {
            // The client is not connected, but is currently open.
            Serial.print("Lost connection to client ");
            Serial.println(i);
            clients[i].stop();              // Close and clean up the old connection
         }

         // At this point, the client is not connected, and is closed.
         clients[i] = Server.accept();       // Try to establish a connection.
         if (clients[i].connected())         // Was a connection made?
         {
            Serial.print("New connection on client ");
            Serial.println(i);
         }
      }
   }
}



void process(byte index)
{
   // Read and echo all available data from the client.
   String str = "";

   // Read all available data, append to string
   while (clients[index].available())
      str = str + (char)clients[index].read();

   // Trim off non-printable characters.
   str.trim();

   // Got something? If so, print out what was recevied, send client ID back to other side
   if (str.length() > 0)
   {
      Serial.print("Received from client ");
      Serial.print(index);
      Serial.println(": received \"" + str + "\"");

      clients[index].print("Client ");
      clients[index].println(index);
   }
}

This example handles up to 12 simultaneous connections. The number of connections can be easily changed by redefining NUM_CLIENTS. Each client takes 83 bytes of RAM, so it adds up fast and you shouldn't define more than you need. This doesn't scale well, since 12 clients already takes up 40% of the available RAM. Like I said in my original reply, handling as much of this as possible on the Linux side is probably a better solution.

There is a single global YunServer instance. Only one is needed: it listens for incoming connections, and if one is established it creates a YunClient instance to handle the connection, and goes back to listening for more connections. It is currently listening on port 23, which makes it easy to test using a Telnet client. But in a real application you would want to use a unique port number to prevent confusion with well known services: like how I arbitrarily chose port 255 in the example in the other thread.

There is an array of YunClient instances, one for each supported simultaneous connection. The code in loop() manages the array of clients, and prints out messages to the serial port when new connections are made, or established connections are closed. If your application needs to do anything special when connections are made or broken, your code would go in there. However, if you make changes, it's critical that the call to the client's stop() method not be removed, as it will eventually stop accepting connections without it.

The process() function is called for each client that has an active connection. It simple looks to see if there is any incoming data from the client, and if so, it prints that data to the serial port and sends a message back to the client. Of course, a real application would do something much more interesting here.

The sketch starts out by waiting for a serial port connection, so that you can see the initial messages. A real application won't want to do this, or else it will never run if you don't have the micro-USB port connected to a computer with the serial monitor open.

When making connections, which client number receives the connection is random: it is constantly looping through clients and trying to accept a connection for each one that doesn't already have a connection. It's just a matter of where it was in the loop when when the connection came in.

If all of the clients are busy with connections, and you try to make another connection, the Linux side will accept the connection, but nothing will happen until a client is freed up (its connection is closed.) At that point, the client will accept the connection and establish communications. So if you get the situation where it looks like the connection is established, but there is no communications, you've likely run out of client objects to handle the connection.

Amazing post. Thanks a lot!

To give you an idea of what I'm trying to do I'll explain the application.

I have a machine with a lot of temperature sensors. Each Yun will have 4 to 8 temperature probes wired to it and there will be many of these on the machine.

All these temperatures need to come to a central spot to be processed by a python program (analytics intensive, will be done on a PC). I really need to update all the temperatures values every 1 second.

So far I have these possible solutions to try:

  1. Each Yun uses REST put commands via the Bridge library to "host" the temperature values; the central python program reads via a RESTful api. The problem is I doubt this will approach real time data transfer, REST seems slow on the Yun.

  2. Use straight TCP, like in this post. Either all data is centralized in a single Yun, or every Yun is a server and the python program becomes a client to every Yun (any advice on this solution?). I suppose all the Yun's could be clients to the python PC, but I'm not that good at python so I don't know how to do this.

  3. Make every Yun an OPC UA server. I will disable the Bridge on linux, read my temperatures with pyserial, and "host" the values via python OPC UA server [GitHub - FreeOpcUa/python-opcua: LGPL Pure Python OPC-UA Client and Server](a good option, but definitely the most difficult to achieve).

  4. Final option is to forget about the Yun and just make a RS485 network of Uno's using Modbus RTU.

Thanks again for the great advice.

If you want to stay with the Yun, I like the second idea in option 2: each Yun is a server which supports a single connection (modeled after the simpler server version in the original thread.) Then, the PC that is the central data collector makes the dozen outgoing connections, one to each Yun. Not only does this put the burden of managing multiple connections on the computer with more memory and processing power, it makes it easier for the PC to figure out which Yun it's talking to on each channel (since it made the connections in the first place.) Hopefully the PC has Bonjour or another zerconf client running on it so it can address the Yuns by name (eg: yun1.local)

Option 4 also has merit, and might be cheaper in the long run, as long as the cabling between them is manageable. A quick search didn't turn up an RS-485 shield in an Arduino Pro Mini shield format, but if I were doing it, that's probably the direction I would head. Assuming there is room, design a small shield with the RS-485 chip and the temperature sensor interface on it, and basically make a bunch of smart addressable temperature sensor nodes.

I know I probably sound like a stuck gramophone (I was told earlier that nobody young knows what a HiFi is so I thought I would regress).

Wouldn't it be much easier to do all the multiple clients in Python on the Yun. I think if you use one of the Python web frameworks like Bottle it just happens automatically. AFAIK there would be no "hit" on the Arduino SRAM.

It seems to me that Bottle would also make it easy for @zerox12 to implement a Python server on his PC that all the Yuns could call as clients. If the Yuns are using Python for the web client they may be able to handle cookies. But if they can't each message could include an identifier for its sender.

...R

Robin2:
Wouldn't it be much easier to do all the multiple clients in Python on the Yun.

Gee, now why didn't I think of that... wait a minute, I did:

ShapeShifter:
Just be aware that RAM on the '32U4 is limited, and each connection will need a chunk of memory. It might not scale up well. If you're getting into a bunch of connections (like the dozen you need) you might be better off managing the network communications on the Linux side. This is especially true if the Yun is acting as data concentration point where it's storing the data from the 12 clients in an SD card file or database.

And I ended my original post with:

Someone more experienced on the Linux side might have good suggestions.

And what do you know, here you are! :wink: 8)

To be clear. I don't need anything to be "web based" I'm not showing the data in web format. I only need the raw temperature data.

What I'm making is more like an IO network than web access to the data.

My problem with doing it directly in the Yun's python environment is that:

  1. It seems inherently difficult just to get the arduino data to the Linux side. As far as I can tell there is no standard format. I would have to make my own serial protocol between Arduino and Linux just to get the data. I'm trying to avoid that, especially since I kind of struggle with serial communication as is. Unless there is way to let the linux side directly access the arduino IO (via some serial interface)? Perhaps someone has already made this interface?

  2. It increases complexity. To set this up on 12+ Yuns could be difficult as I need to load an arduino program, and the necessary python modules and scripts to make it work.

If I was a much better programmer I'm sure I could do it, but I'm really new to this stuff. I come from an automation background as a PLC programmer where communicating data is ridiculously easy.

Thanks for the inputs!

zerox12:
Unless there is way to let the linux side directly access the arduino IO (via some serial interface)? Perhaps someone has already made this interface?

Do some searching on Firmata - it's a sketch you load into the Arduino processor, and it allows low level control over the I/O pins using standardized serial commands. You'd still have to implement the commands in Python (unless someone has a Python library that implements that side of it, which is quite likely.) I have no personal experience with Firmata, but it appears to be used by many.

There is something to be said about keeping it simple: while the YunServer/YunClient has some overhead and is not the most efficient way of handling a network connection, it is simple. If you don't need the efficiency that comes with doing it on the Linux side, why go to the effort?

A single server handling a single connection is easy to do on the Arduino side. A dozen connections is more stressful and is worth moving management to the Linux side, or even better yet to the host PC.

If you want to use the Yun and the network, then I would have each Yun talk directly to the host PC using an Arduino sketch. The sketch manages a single connection: which side initiates the connection (Yun or PC) is your decision.

But given your PLC experience, especially with their communications, you might be be best off using simple Arduinos and an RS-485 adapter, and write a simple sketch using a MODBUS library.

ShapeShifter:
Gee, now why didn't I think of that... wait a minute, I did:

Sorry - I missed that bit.

...R

zerox12:

  1. It seems inherently difficult just to get the arduino data to the Linux side.

This is no more difficult than getting data from an Arduino to a PC using the USB cable.

  1. It increases complexity. To set this up on 12+ Yuns could be difficult as I need to load ...

It is a shame that the Arduino folks did not create a simple way to upload Python code to the Linux side.

In fact if you create (and test) a Python program including all its dependencies in a directory on your PC you should be able to copy the whole thing to a Yun with a single SCP command. This Thread illustrates the process.

...R

Robin2:
Sorry - I missed that bit.

No problem, just teasing you... :wink:

Robin2:
It is a shame that the Arduino folks did not create a simple way to upload Python code to the Linux side.

Well, they kind of did. If you create a www folder inside your sketch's source code folder on your computer, and upload your sketch using a network connection (not the USB serial) it will automatically copy everything in that www folder to /mnt/sd/arduino/www. While this is originally intended for web pages associated with the sketch, it should work with any files. It's not as flexible as SCP, and assumes you have an SD card with an Arduino folder att he root level, but may be an option for some.

Robin2:
This is no more difficult than getting data from an Arduino to a PC using the USB cable.It is a shame that the Arduino folks did not create a simple way to upload Python code to the Linux side.

In fact if you create (and test) a Python program including all its dependencies in a directory on your PC you should be able to copy the whole thing to a Yun with a single SCP command. This Thread illustrates the process.

...R

I will try this. In fact I have to try it because it turns out the Yun crashes when byte-compiling large files. I just found this out after installing pip and doing:

pip install -v freeopcua

It installs all the dependencies perfectly, then trys to byte compile a 1.3MB .py into .pyc and locks up, then drops the SSH connection. When I reconnect and do

pip freeze

only the dependencies got installed. :frowning:

Thanks for the info.

zerox12:
I will try this. In fact I have to try it because it turns out the Yun crashes when byte-compiling large files.

I have no idea whether my suggestion will deal with that.

...R

@zerox12,
as a software programmer, I'm missing some critical information on this implementation

  1. what types of thermometers are you using?
  2. what is the range they are measuring?
  3. what is the I/O path to the i/o pins? I2C? Bluetooth? direct connection?
  4. analog or digital interface?

To me it appears that a series of i2C thermometers could all be wired to a single Arduino processor. For that matter the Yun-mini might be better or the new Tian

Jesse

So I have my Arduino Yun set up as TCP server. I'm connecting to it via the sockets library in python.

It turns out talking with the Yun via straight TCP is a real pain if you want to leave the socket open to increase speed.

After some work I manged to create a simple handshake type protocol.

Anyways, I was able to read one INT on my protocol every ~40ms (not on wifi). This is way faster than using REST get commands, which takes between 500 to 700ms.

When I finish the code I will post it in case others want to try it.

PS. The modbus option using SimpleModbus maxed out around 14ms per single INT read. Unfortunately SimpleModbus doesn't support reading more than 1 register per transaction.

jessemonroy650:
To me it appears that a series of i2C thermometers could all be wired to a single Arduino processor.

Now that's a scary proposition! I2C is Ok for it's intended purpose: communicating between ICs on a circuit board. But it is not a robust communications protocol, and doesn't deal well with noise or long lead lengths. I would never use it to scatter sensors around a piece of machinery. The issues are many: it is sensitive to bus capacitance because the passive nature of the pull-ups and open collector drives mean that rise times can quickly become a problem. Yes, there are various bus driver chips that try to overcome this by giving the lines brief moments of active drive, but you are still limited to very short bus lengths (I've routinely run into problems when bus lengths get more than a foot or two.) Furthermore, it's very susceptible to noise, and many times it results in a bus lock up where slaves don't respond, requiring a series of start/stop sequences and extra clock pulses to clear the lockup (I usually have to switch the I2C pins to GPIO and bit-bang the bus to clear it.) in the worst cases, the slave can get stuck in a clock-stretching state where it's impossible to clock the bus because the slave is holding the clock line low: the only way to recover is to pulse the slave's reset line or cycle power to it. In my mind, if you want a reliable system, I2C is a last resort, and never for a situation where the environment is noisy or the wires are more than a foot or two long.

It seems like people on this forum are particularly enamored with I2C - I don't understand it. My only thought is that they are typically hobbyists that don't have to design a robust system and don't mind having to periodically reset/cycle power to keep it running. Personally, I much prefer SPI: it takes an extra wire, but it's pretty much bullet proof. But I wouldn't use it to distribute sensors around an electrically noisy machine. Something like RS-485 is much better for that.

zerox12:
It turns out talking with the Yun via straight TCP is a real pain if you want to leave the socket open to increase speed.

Can you expand on this? I've had no trouble opening a TCP stream port between a PC and a Yun and leaving it open for hours or days while talking in both directions with no special handshaking.

ShapeShifter:
Something like RS-485 is much better for that.

Reminds me of the humourous song "There's a hole in the bucket"

...R

ShapeShifter:
Can you expand on this? I've had no trouble opening a TCP stream port between a PC and a Yun and leaving it open for hours or days while talking in both directions with no special handshaking.

Well, basically in my case I only want to READ data. If my Arduino code is continually doing client.print("myDataString") and python is continually doing arduino.socket.recv(bufflength), python will return portions of what the arduino is sending, because it might read the buffer while the arduino is still sending.

In other words, python will get data like this:
myDa
taStrin
g
myDataS
tring
my
Dat
aString

etc. and I was having an extremely difficult time parsing my information out of a stream like this. So what I did is have python send a keyword to the arduino. In my case, I sent "update". When the arduino server got a string that = "update", it only execute client.print("myDataString") once. Then python send back "done" after the data is read. So it works like this after TCP connect:

arduino sends "ready"
python sees "ready", python sends "update"
arduino sees "update", arduino sends the data string once
python sees the data and handles it, python sends "done"
arduino sees "done", goes back to top

I have no idea if this is needed, but it was the only way I could get a complete string in python. Like I said i'm new to PC languages.

zerox12:
Well, basically in my case I only want to READ data. If my Arduino code is continually doing client.print("myDataString") and python is continually doing arduino.socket.recv(bufflength), python will return portions of what the arduino is sending, because it might read the buffer while the arduino is still sending.

In other words, python will get data like this:
myDa
taStrin
g
myDataS
tring
my
Dat
aString

etc. and I was having an extremely difficult time parsing my information out of a stream like this. So what I did is have python send a keyword to the arduino. In my case, I sent "update". When the arduino server got a string that = "update", it only execute client.print("myDataString") once. Then python send back "done" after the data is read. So it works like this after TCP connect:

arduino sends "ready"
python sees "ready", pythong sends "update"
arduino sees "update", arduino sends the data string once
pythong sees the data and handles it, python sends "done"
arduino sees "done", goes back to top

I have no idea if this is needed, but it was the only way I could get a complete string in python. Like I said i'm new to PC languages.

At PC (with a lot of memory), you could write python multiple threads app to take care it, one thread per Yun client. Once it works then port code into Yun.

sonnyyu:
At PC (with a lot of memory), you could write python multiple threads app to take care it, one thread per Yun client. Once it works then port code into Yun.

That is how I'm going to test it. Python will make a thread with a client for every Yun. I tried to use other non blocking methods like asyncio, but these modules are rather confusing to implement.

zerox12:
In other words, python will get data like this:
myDa
taStrin
g
myDataS
tring
my
Dat
aString

Yes, this is expected. YunClient implements a TCP stream port, which while it does have guaranteed deliver (meaning all bytes will show up in the correct order with no missing or lost bytes) it does not preserve record blocking or timing. That means that even if you send the data in distinct "print" blocks, there is no expectation that reading the data on the opposite end will return the same blocks.

As you discovered, you need to add some delimiters to your data (simple new line characters may be enough in many cases) and then read the data a character at a time, buffering the data until the terminator is read. Then, when you get a complete buffer, you can try to parse it.

Another thing I've done is to use a state machine to process the incoming data. The data has record and field delimiters (for example, newline to delimit records, commas to delimit fields.) Then, read the characters one by one, and feed them to the state machine. The state machine sits in one state collecting data until a comma, then parses and stores the field, and moves on the next state.