Go Down

Topic: Remote procedure calls (Read 4978 times) previous topic - next topic

liuzengqiang

I have thought about devising protocols and writing code for remote procedure calls for a whole until a job really pushed me to do it. I'm trying to make this protocol simple and web friendly. I would like some critic from the forum.

Getters and setters are easy to make. I used HTTP GET command syntax. Say if you want values of x,y,z from arduino, just send string to serial port with "GET GET?names=x,y,z  HTTP/1.0\r\n". The reply will be:
x=value
y=value
...

You can treat the second GET as URI while the first is HTTP method. Same as SET below.

"GET SET?name1=value1&name2=value2  HTTP/1.0\r\n". Arduino will set these values and then arduino replies "HTTP/1.0 200 OK\r\n\r\n"

Now the difficult part is to call a remote procedure with parameters. What I envision is the URI would reflect the remote procedure. So an RPC would look like this:

"GET pump_start_autorun  HTTP/1.0\r\n".  This calls pump.start_autorun().
"GET pump_turn_on?period_ms=10000  HTTP/1.0\r\n". This calls pump.turn_on(10000) to turn on the pump for 10 seconds.

Reply could be "HTTP/1.0 200 OK\r\n\r\n" or "HTTP/1.0 404 Not Found\r\n\r\n"

Or should I make use of slash to do "GET pump/turn_on?period_ms=10000  HTTP/1.0\r\n"? This makes it a bit more organized but Arduino need more time to process the strings.
Any other suggestions?

Serial LCD keypad panel,phi_prompt user interface library,SDI-12 USB Adapter

pYro_65

#1
Jun 12, 2014, 12:45 pm Last Edit: Jun 12, 2014, 12:49 pm by pYro_65 Reason: 1
I would try and minimize the data to send.
When sending data to the Arduino ( or a web server ) you do not need to use key-value pairs, you can just have keys.

A generic format like this might be good for a flexible system:
GET feature.function?param1&param2&param3

So if the 'pump on' function takes a single parameter you can send using get:
GET pump.on?10000

You could also send the time in seconds if it is always longer than 999 milliseconds.

In your code, an array can map the string 'pump' to an array of pump functions, and can all be stored in PROGMEM maybe. Then your whole API can be run using a single piece of code (once the array indices have been found ).

A quick check shows that you can store your function addresses in PROGMEM too:
Code: [Select]
void (*fptr)() PROGMEM = printIt;
void setup() { Serial.begin( 9600 ); }
void printIt() { Serial.println( "running" ); }
void loop(){ ((void(*)()) pgm_read_word( &fptr ))(); }
Forum Mod anyone?
https://arduino.land/Moduino/

robtillaart


...
A quick check shows that you can store your function addresses in PROGMEM too:
Code: [Select]
void (*fptr)() PROGMEM = printIt;
void setup() { Serial.begin( 9600 ); }
void printIt() { Serial.println( "running" ); }
void loop(){ ((void(*)()) pgm_read_word( &fptr ))(); }


interesting!!
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

liuzengqiang

Pyro,

Thanks for the suggestions. I'm going to experiment on these. My main concern is not yet the speed of the protocol, but how it can be easily used in various projects with the least learning curve. So I picked the HTTP protocol that everybody has some experience with. I think in the spirit of reducing footprint and keeping readability, I can try to use number instead of abbreviation for a function and define say #define pump_turn_on 101. This way it will be quicker. Same for variable names.
Serial LCD keypad panel,phi_prompt user interface library,SDI-12 USB Adapter

robtillaart

@liudr,

Are you familiar with - http://tools.ietf.org/html/rfc5531 - esp. chapter 9

although it is probably (too?) heavy you could create a light implementation
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

liuzengqiang

Thanks Rob. That does seem a bit heavy. I'm trying to make the protocol easily understandable and can be used with a serial monitor and a developer just typing in texts and see what happens. This protocol needs some serious programming experience to even understand, might be out of my own league too.

The development of a decent Arduino RPC has been in my mind for a while, since I have seen none here. With raspberry pi, arduino tri and other ways of using arduino with a more powerful processor, it is absolutely critical to have a robust yet easy to use protocol instead of sending and receiving one byte over serial (stone age of RPC). If I have enough time, I should be able to get out a V 1.0 before the summer (northern hemisphere of course) ends.
Serial LCD keypad panel,phi_prompt user interface library,SDI-12 USB Adapter

kowalski

@liudr

Decent RPC is much more than a serialization protocol. Most middleware tools include an IDL (Interface Description Language), a compiler and a run-time library. The high end of these are systems such as OMG CORBA and DDS.

Typical RPC support systems look like this;

1. Run-time library with serialization functions, parser and dispatcher.
2. Interface Description Language with compiler to generate Proxy and Adapter functions.
3. The Proxy functions (or class) give the full illusion of local function but will perform the serialization of the call and the deserialization of return value (if any). The Proxy is the client/caller side support.
4. The Adapter functions (or class) perform the deserialization of the call request, call the implementation, and the serialization of the return value (if any). The Adapter is the server size support.

IDL has been around since the late 80'ies when introduced with XDR/RPC by Sun for implementing for instance NFS (Network File System). XDR (eXternal Data Representation) is the name of the serialization method. Microsoft has used IDL for a number of things (DCOM, etc). In Java there is RMI (Remote Method Invocation) which has its own serialization method but can also use IIOP (as defined by CORBA). Should not forget ASN.1 and SNMP which also addresses the issue of interface/data description.

Most of the above serialization methods are binary and have to handle that data may be represented differently in memory on the client and server (e.g. endian). With a textual representation this is avoided at the price of more processing and larger messages (on average).

Other popular textual serialization methods are XML, YAML and Json. On the binary side the Google ProtocolBuffers is maybe the more popular to handle large data sets.

The bottom line: Having a textual format and HTTP is a great way to start and provides a command line style access. Scaling up becomes very difficult and the footprint and processing needed grows fast. From a programmers perspective the RPC should simply be a set of functions and the necessary serialization hidden. Having to implement RPC function calls as HTTP request generation is work that can be automated to avoid errors in parameter printing and parsing (e.g. escape characters in strings).

Cheers!

Ref.
1. Middleware, http://en.wikipedia.org/wiki/Middleware
2. IDL, http://en.wikipedia.org/wiki/Interface_description_language
3. CORBA, http://en.wikipedia.org/wiki/CORBA
4. DDS, http://en.wikipedia.org/wiki/Data_Distribution_Service
5. RMI, http://en.wikipedia.org/wiki/Java_remote_method_invocation
6. ASN.1, http://en.wikipedia.org/wiki/ASN.1
7. REST, http://en.wikipedia.org/wiki/REST

liuzengqiang

Kowalski,

That's quite complex for arduino. I'm shooting for projects that will not use significant amount of resources on Arduino. Also I think to make the protocol popular, beginners must be able to test their programs without black box and magic. So a simple text protocol that can by typed in a serial port and return value seen on the same port is preferred. If I want to build something a lot more complex, I'd not be implementing it on Arduino, but some processors with real amount of memory.

So to use the correct term, I'm writing a protocol for RPC, a series of sample adapter and proxy code to help spread this protocol. I'm only proficient in C++ and Java/Processing. I might consider Python if I can master it before the summer ends. This way PC/Linux/Mac can use C++/Processing/Python to talk with Arduino.

Right now, there is this primitive "send one byte, do something" type of communication, and also constant questions regarding "how do I send arduino 3 numbers?" or the like. Most Arduino users are very novice and can't appreciate anything complex. The situation is helpless unless I do something about it.
Serial LCD keypad panel,phi_prompt user interface library,SDI-12 USB Adapter

jfjlaros

@liuzengqiang

You might find this simpleRPC library interesting, it has the properties @kowalski mentioned, except that it operates without a run-time library on the device, the interface is generated compile time instead. It has its own IDL, data is transmitted in binary and things like types and endianness are taken care of. Vectors, tuples and complex objects are supported.

The client reference implementation for Python builds a proxy which in turn can be used to set up a WSGI server, this approach is discussed in this post.

I hope you find this useful,
Jeroen.

Go Up