RFC: SerPro v3 (RPC library for arduino)

Hi.

I am right now closing development on SerPro (GitHub - alvieboy/arduino-serpro: Serial Protocol (packet/frame based) library for Arduino intercommunication) and so I decided to start new version (v3).

Those who never heard of it, SerPro is some sort of RPC for interfacing devices, like arduino and a computer. It's used mainly on my oscope.

For v3, I'd like to use an easier approach. This mean minimizing what developer must to to use the library. I came up with a mockup of next version, for which I'd like to have your feedback. This demo code below exports 4 functions from arduino (very well known ones), and PC (glib-based) code to flip arduino led each second.

Here's arduino code:

#include <SerProArduino.h>

SERPRO_ARDUINO_BEGIN();

EXPORT_FUNCTION(1, pinMode);
EXPORT_FUNCTION(2, digitalWrite);
EXPORT_FUNCTION(3, digitalRead);
EXPORT_FUNCTION(4, analogWrite);

SERPRO_ARDUINO_END();

void setup()
{
      Serial.begin(115200);
}

void loop()
{
      if (Serial.available()>0) {
            SerPro::processData(Serial.read());
      }
}

And PC code to interact with it:

#include <SerPro-glib.h>

#define OUTPUT 1

SERPRO_GLIB_BEGIN();

IMPORT_FUNCTION(1, pinMode, void (uint8_t,uint8_t) );
IMPORT_FUNCTION(2, digitalWrite, void (uint8_t,uint8_t) );
IMPORT_FUNCTION(3, digitalRead, int16_t (uint8_t) );
IMPORT_FUNCTION(4, analogWrite, void (uint8_t,int16_t) );

SERPRO_GLIB_END();

const uint8_t ledPin = 13;

gboolean myfunction(gpointer data)
{
      uint8_t val = digitalRead(ledPin);
      /* Invert signal */
      digitalWrite(ledPin, !val);
      return TRUE;
}

void connect() // This is called when protocol connects
{  
      pinMode(ledPin, OUTPUT);
      // Start asynchronous function, called each 1/2 second (500 milisseconds)
      g_timeout_add(500, &myfunction, NULL);
}


int main(int argc, char **argv)
{
      if (argc<2)
            return -1;

      if (SerProGLIB::init(argv[1])<0)
            return -1;

      SerProGLIB::onConnect( &connect );
      SerProGLIB::start();
      SerProGLIB::run();
}

I'd like to head a lot of comments and rants and whatever you think might help to make library easy to use so I can publish it when it's finished (this code works, but still needs improvements).

Álvaro

I have been looking for something exactly like this. I found the previous version of arduino-serpro on github, but it seemed overly complicated. This proposal looks like a step in the right direction.

I have one question: are you planning Windows support for your library? I've been using the boost asio library.

Also, it is not clear to me if or how the PC receives return codes from it's remote function calls.

-Ryan

Hi

are you planning Windows support for your library? I've been using the boost asio library.

Well, serpro code is C++, and does not implement low-level functions, so should work with any OS capable of C++. There's already a wrapper for glib, I think we can write one for boost, I'll look at it.

Also, it is not clear to me if or how the PC receives return codes from it's remote function calls.

Basically it's a reverse-direction function call, which argument is the return value from the called function:

Let's say we export a function SUM in arduino with index 1, like in:

int SUM(int a, int b) { return a+b; }
EXPORT_FUNCTION(1,SUM);

When we call this function on PC, a RPC call with id 1 goes from PC to arduino. The SerPro logic knows in advance that it has to deserialize two integers, and has one integer return value. So after calling SUM, arduino serpro will call function id 1 on PC with an integer argument, which value is the sum result.

Like:

PC->ARDUINO   CALL(1, <integer1>,<integer2>)
ARDUINO->PC   CALL(1, <integerresult>)

The PC will block waiting for this "callback" to happen, except if function return type is "void" (if so, the second call will never occur).

Álvaro

Hi again, and thanks for your reply.

Well, serpro code is C++, and does not implement low-level functions, so should work with any OS capable of C++. There's already a wrapper for glib, I think we can write one for boost, I'll look at it.

The reason asked about Windows compatibility is that I came across a few posts like this one:

suggesting to me that it might not be portable. I did find a great article with simple wrapper classes for the boost library:

http://www.webalice.it/fede.tft/serial_port/serial_port.html

Basically it's a reverse-direction function call, which argument is the return value from the called function:

I can see now that the digitalRead function in your example demonstrates this by returning a value.

Regarding your mockup, I think it might be helpful for less experienced programmers if the documentation were more explicit and if the program were as simple as possible (i.e. maybe avoid the asynchronous function?). I'll take a stab at it, but please correct me where I'm wrong ;). I've interspersed a few comments/questions within the code.

On the Arduino:

#include <SerProArduino.h>

////////////////////////////////////////////////////////////////////////////////
//
// Functions that you would like to make available from the Arduino
// are defined between SERPRO_ARDUINO_BEGIN(); and
// SERPRO_ARDUINO_END(); with the form:
//
//   EXPORT_FUNCTION(function_id,function_name);
//
////////////////////////////////////////////////////////////////////////////////
SERPRO_ARDUINO_BEGIN();

EXPORT_FUNCTION(1, pinMode);
EXPORT_FUNCTION(2, digitalWrite);
EXPORT_FUNCTION(3, digitalRead);

SERPRO_ARDUINO_END();

void setup()
{
      Serial.begin(115200);
}

void loop()
{
      if (Serial.available()>0) {
            SerPro::processData(Serial.read());
      }
}

On the PC:

#include <SerPro-glib.h>

#define OUTPUT 1

////////////////////////////////////////////////////////////////////////////////
//
// Functions that were exported from the Arduino are defined between
// SERPRO_GLIB_BEGIN(); and SERPRO_GLIB_END(); with the form:
//
//   IMPORT_FUNCTION(function_id,
//               function_name,
//               return type (param1 type, param2 type) );
//
// In the examples below, functions 1 and 2 return void, so SerPro
// will not send a reply.  When the functions are called on the PC, they
// they will return immediately.
//
// After calling function 3, the PC will block and wait for the return
// value from the Arduino.  Once the Arduino has executed the
// digitalRead function, it will send a response message back to the
// PC with an int16_t argument equal to the function's return value.
//
// Comment: Is there a timeout for the blocking?  If the packet
// doesn't show up within the timeout, will SerPro retry?
//
////////////////////////////////////////////////////////////////////////////////
SERPRO_GLIB_BEGIN();
IMPORT_FUNCTION(1, pinMode, void (uint8_t,uint8_t) );
IMPORT_FUNCTION(2, digitalWrite, void (uint8_t,uint8_t) );
IMPORT_FUNCTION(3, digitalRead, int16_t (uint8_t) );

SERPRO_GLIB_END();

const uint8_t ledPin = 13; // LED connected to digital pin 13
uint8_t ledPinState; // state of the LED pin (1 or 0)

// This function is called when protocol connects. It is analogous to the
// setup() function on the Arduino.
void connect() 
{  
      pinMode(ledPin, OUTPUT); // Initialize the digital pin as an output
}

int main(int argc, char **argv)
{
      // Comment: I'm assuming that in your initial post, argv[1]
      // specified the serial port to connect to? Is the baud rate hard
      // coded?  I thought it might be easier to see what's going on if
      // you showed the initialization explicitly.

      // Initialize SerProGLIB with the serial port and baud rate (Note
      // that on Windows, serial ports have names like "COM1").
      if (SerProGLIB::init("/dev/ttyS0",115200)<0)
            return -1;

      // Register connect function so that it will be called when
      // SerProGLIB connects.
      SerProGLIB::onConnect( &connect );

      // Comment: I'm not sure what these functions do...
      SerProGLIB::start();
      SerProGLIB::run();

      // This for loop will flip the state of the LED 100 times, waiting 1
      // second between each call
        for(int i=0; i<100; i++)
      {      
            ledPinState = digitalRead(ledPin); // Read the state of the LED
            digitalWrite(ledPin, !val); // Invert the signal
            usleep(1000); // Wait one second
      }
}

One last suggestion, why not just call your PC library SerProPC? Not all Arduino users will know what GLIB is, and I would imagine there's a way to release it as a statically library so that users would not have to install GLIB?

I'd be happy to help you with this if there's anything that I can do. My C++ is a bit rusty (all of the fancy preprocessor macros in your code make my head hurt), but I could probably help with testing and/or Windows support. Also, I think that Python wrappers might be a nice addition that I could probably hack together.

Thanks,
-Ryan

Well, your PC approach requires use of a thread because of the blocking sleep. It can be done though, let me think about it.

One last suggestion, why not just call your PC library SerProPC? Not all Arduino users will know what GLIB is, and I would imagine there's a way to release it as a statically library so that users would not have to install GLIB?

Actually library is the same, then you have small wrappers to avoid specifying all required definitions for SerPro instantiation.

Here's what SerProArduino.h does:

#ifndef __SERPRO_ARDUINO_H__
#define __SERPRO_ARDUINO_H__

#include <inttypes.h>
#include <avr/io.h>

#define SERPROLIBRARY

#include <serpro/SerPro.h>
#include <serpro/SerProHDLC.h>

#include <WProgram.h>


struct SerialWrapper
{
public:
      static inline void write(uint8_t v) {
            Serial.write(v);
      }
      static inline void write(const uint8_t *buf,int size) {
            Serial.write(buf,size);
      }
      static void flush() {
      }
};

struct HDLCConfig: public HDLC_DefaultConfig
{
      static unsigned int const stationId = 3;
};

#ifndef  SERPRO_ARDUINO_MAXFUNCTIONS
# define SERPRO_ARDUINO_MAXFUNCTIONS 16
#endif

#ifndef  SERPRO_ARDUINO_BUFFERSIZE
# define SERPRO_ARDUINO_BUFFERSIZE 16
#endif


#define SERPRO_ARDUINO_BEGIN(DUMMY)   \
struct SerProConfig {                                                     \
      static unsigned int const maxFunctions = SERPRO_ARDUINO_MAXFUNCTIONS; \
      static unsigned int const maxPacketSize = SERPRO_ARDUINO_BUFFERSIZE;  \
      static SerProImplementationType const implementationType = Slave;     \
      typedef HDLCConfig HDLC; /* HDLC configuration */                     \
}; \
DECLARE_SERPRO(SerProConfig,SerialWrapper,SerProHDLC,SerPro)



#define SERPRO_ARDUINO_END(DUMMY) \
      IMPLEMENT_SERPRO(SERPRO_ARDUINO_MAXFUNCTIONS,SerPro,SerProHDLC)

#endif

The glib "frontend" or "wrapper" is useful for Gtk+ applications. Other wrappers can exist, but they will always depend on several factors, like your libraries.

I'd be happy to help you with this if there's anything that I can do. My C++ is a bit rusty (all of the fancy preprocessor macros in your code make my head hurt), but I could probably help with testing and/or Windows support. Also, I think that Python wrappers might be a nice addition that I could probably hack together.

Macros are not as fancier as the template specializations :slight_smile:

Thanks, feel free to send in sugestions. Regarding python, I am not sure it can be easy to do, and whether it would be typesafe.

I'll try using ASIO as soon as possible, but I've to reformulate a few things due to the multithreaded approach (this might require changes for synchronization on the lowlevel protocol too).

Álvaro