Modular API

[obiwanjacobi]

Hi all,

There is a problem I see with the current (default Arduino) API.

Say I have an LCD, so I use the LCD library. But it allows only a direct connection of the LCD to the Arduino board pins. Because I have a lot of other IO happening on my project I have rigged in some shift register chips and need to address the LCD in a different way.

So because the LCD (and almost all other libs) both implement the protocol and the physical layer in one library, I am stuck.

What I seek is a modular system where I can plug in different objects on top of one-another that each only servers exactly one goal. So for example, I would have a Hardware Pin dirver object that can be used by a bit shift object, that can be used by a SPI protocol object that can be used by a LCD functional object that can be used by my program.

In my head (at this time) I think of them as templated C++ classes that allow some sort of chaining. You simply typedef the chains you need for your app and work with those types in the program/sketch.

So, before I embark on a long journey to write all this good stuff from scratch, does anybody know if something like this already exists?

I hope I made my point clear, if not, just ask.

Thanx,
Marc Jacobi
Netherlands

[obiwanjacobi]

Ah I see I'm not alone:

http://arduino.cc/forum/index.php/topic,37940.msg280717.html#msg280717

[graynomad]

Looks like Matteo did a lot of work on that but got no response.

Personally I think some further abstraction like this isn't a bad idea and there are I think a couple of libraries around that do similar but I can't remember where. I think there's a PWM one for shift regs and another that allows digitalRead/Write from shift regs.

Essentially it's like the ISO layer model where you plug in another PHY level to handle different physical requirements.

You can't however dick around with things like SPI, UARTs etc as they are mapped to certain pins.

______
Rob

[obiwanjacobi]

Yeah, too bad the forum he posted on is now read only, or I would have resurrected his thread.  XD

Yeah, it is a lot like the ISO layer, because we try to solve similar problems.

About SPI: I think of it as, you can use the built-in SPI or a software implementation for it - that should be plugable into the chain. But the hardware concepts that don't make sense to have a soft-counter part for are just not available for override.

I have to get into the details of what MCU features are there and how they're typically used etc. At this time its more a thought experiment 

Thanx, hope you find the libs you talked about...

[obiwanjacobi]

I've been reading about all sorts of (attempts on) Real-Time OS's (RTOS) and a port of Free RTOS for the Arduino called Duinos seems to be one of the options.

But then I started thinking, you have all these interrupts on the ATmega chip why not utilize them (more)? That way you have 'hardware accelerated' 'threading' and no overhead of some OS that tries to give you threads. Also I think that a good (finite) State Machine can provide you with goo performance and hopefully -depending on its architecture- still have coherence in the code that logically belongs together.

So what is your take on this? Do you use some kind of OS or task scheduler or any of this type of libraries?
Like to know how the work in real-life.

Thanx.

[fat16lib]

Your idea of using interrupts for 'hardware accelerated' 'threading' is forty years old.  That's how we wrote embedded systems in the 1970s. We wrote a custom threading framework for each system and often a custom disk driver and other subsystems.

It became apparent that a more general approach would be better. This lead to the modern RTOS.  My friends founded Wind River Systems and produced VxWorks in the early 1980s.  VxWorks is still very popular and has been used in many major projects like the Mars Rovers.

The main reason people experience a lot of overhead using a RTOS is lack of knowledge/experience in good embedded system design and misuse of RTOS features.

There are a number of choices for Arduino.  If you want a state machine try QP http://arduino.cc/playground/Code/QP from Quantum Leaps http://www.state-machine.com/.

For true preemptive multitasking on Arduino I like ChibiOS/RT.  I have ported it as a library here http://code.google.com/p/beta-lib/downloads/list.  The current version is ChibiOS20120221.zip .

I have also tried FreeRTOS but it is not as efficient as ChibiOS/RT on Arduinos.  I also ported FreeRTOS as a library and a trial version is FreeRTOS20111031.zip  at http://code.google.com/p/beta-lib/downloads/list.

[obiwanjacobi]

I understand that my idea is not unique 
I also understand that abstraction can increase ease of use even when it comes with a (considerable) overhead.
I was just considering the choice: how much overhead are you willing to take to gain certain benefits? On a small platform like the Arduino those choice can become important - for performance or size.

What you said about lack of knowledge is key, I think. It takes time to really get into any library or API and know what to do, why and in what way.

My own thoughts keep coming back to a device driver / HAL kind of layer that will allow reuse of any device related library (like a LCD) on any project, even one that has a different way of hooking it up (think shift registers for instance). That way you can really optimize that code (and benefit from its maturity over time) in any type of project. For this essential base layer that communicates with physical devices, using (hardware) interrupts and the MCU's built-in IO facilities seems a logical choice and should be abstracted (but not necessarily hidden ;-) by the device driver...

How you organize your main program (threads or other os's) is (or at least could be) a different matter.

Thanx for the links, I will check them out.

[fat16lib]

A modern RTOS does not have "(considerable) overhead" and a system like ChibiOS is very small.  My example sketch to that runs two tasks to measure context switch time on Arduino 1.0 takes 3014 bytes of flash.

ChibiOS provides a HAL layer for device drivers.  It provide an efficient way for device drivers to activate tasks from an interrupt routine when required.  Interrupt routines can be very efficient on ChibiOS.

The notion that using a RTOS on small processors requires some big sacrifice of performance and memory is an archaic idea and most people who propagate this idea have no real experience with a modern RTOS.

Clearly you believe you need to return to the good old days and reinvent the RTOS.  Thousands of people have gone before you so you might want to understand what they have done.  Here is a partial list of RTOSs to study http://en.wikipedia.org/wiki/List_of_real-time_operating_systems.

I don't think you can do much better than Giovanni Di Sirio, who wrote ChibiOS.  He has been been working on versions this small fast system for 15 years.  He used the very best algorithms to implement ChibiOS.

[obiwanjacobi]

Thanx for your response.

You are right on with your analysis

I do think at the moment that most RTOSs (I have seen) are overhead for stuff I don't need for my project and I think the way the libraries are programmed for the Arduino are rubbish in that you cannot use any of them out of the box if you have some minor differences. I think a lot of the code is not as optimal as it could be (if-statements inside a loop, which can easily be inverted - or will the compiler do that for you?) and the code design is horrible at times (mixing multiple responsibilities in one class) and "doubtful" linker optimization support (based on object files).

So yes, from the code I have seen I'd say I (know I) can do better. But I haven't looked at the your code links yet and I know enough that I cannot easily outperform 15 years of code maturity by an expert

And yes I also do not have any experience with RTOSs, I'm just browsing to find something that looks good and is appropriate for the project I am about to begin. Seemed like a good idea at the time

Thank you (again) for your insightful remarks.