next topic »

[Reply #15 on:]

fat16lib,

I've gotten some good feedback in the form of "thanks for writing this up", but no specific additional questions on the QF hack. I've been pleased with the traffic to the specific blog post on state machines, it seems to be gaining momentum over the last month, perhaps in light of recent talk about schedulers like this.

Thanks for this post, I believe it builds awareness of some of the tools that are available for solving various control problems.

[Reply #16 on:]

to answer the question
   yes it dosn't

depends what your doing with the thing.

would be nice if a simple rtos was available,

as for the comment back a while about the good old 8008,
   the work on the apollo computer used a rtos / interupt / schedular system,
     its what saved the 11 mission when buzz left the  return to orbit  radar on as well as the landing radar.

http://ed-thelen.org/comp-hist/vs-mit-apollo-guidance.html

have fun

[Reply #17 on:]

Would an extremely small preemptive RTOS appeal to Arduino users?

I have been playing with an experimental RTOS written by Giovanni Di Sirio, the author of ChibiOS/RT. 

His goal is to build the smallest possible kernel for tiny chips.  Giovanni calls the system Nil RTOS since the goal is a zero size kernel.

Nil RTOS has only the most fundamental functionality.

A preemptive fixed priority scheduler.

Counting semaphores that can signal from a thread or ISR.

Sleep until a specified time and sleep for a specified period.

Here is an example sketch with a total size under 2KB  on an Uno:

Code:

// Connect a scope to pin 13.
// Measure difference in time between first pulse with no context switch
// and second pulse started in thread 2 and ended in thread 1.
// Difference should be about 10 usec on a 16 MHz 328 Arduino.
#include <NilRTOS.h>

const uint8_t LED_PIN = 13;

// Semaphore used to trigger a context switch.
Semaphore sem = {0};
//------------------------------------------------------------------------------
/*
 * Thread 1 - high priority thread to set pin low.
 */
NIL_WORKING_AREA(waThread1, 128);
NIL_THREAD(Thread1, arg) {

  while (TRUE) {
    // wait for semaphore signal
    nilSemWait(&sem);
    // set pin low
    digitalWrite(LED_PIN, LOW);
  }
}
//------------------------------------------------------------------------------
/*
 * Thread 2 - lower priority thread to toggle LED and trigger thread 1.
 */
NIL_WORKING_AREA(waThread2, 128);
NIL_THREAD(Thread2, arg) {

  pinMode(LED_PIN, OUTPUT);
  while (TRUE) {
    // first pulse to get time with no context switch
    digitalWrite(LED_PIN, HIGH);
    digitalWrite(LED_PIN, LOW);
    // start second pulse
    digitalWrite(LED_PIN, HIGH);
    // trigger context switch for task that ends pulse
    nilSemSignal(&sem);
    // sleep until next tick (1024 microseconds tick on Arduino)
    nilThdSleep(1);
  }
}
//------------------------------------------------------------------------------
/*
 * Threads static table, one entry per thread. Thread priority is determined
 * by position in table.
 */
NIL_THREADS_TABLE_BEGIN()
NIL_THREADS_TABLE_ENTRY("thread1", Thread1, NULL, waThread1, sizeof(waThread1))
NIL_THREADS_TABLE_ENTRY("thread2", Thread2, NULL, waThread2, sizeof(waThread2))
NIL_THREADS_TABLE_END()
//------------------------------------------------------------------------------
void setup() {
  // Start nil.
  nilBegin();
}
//------------------------------------------------------------------------------
void loop() {
  // Not used.
}

I wrote this sketch to determine the performance of Nil RTOS.  I was amazed to find how fast it is.  The time to signal a semaphore, do a contex switch and take the semaphore is only about 12 microseconds on an Uno.
 

[Reply #18 on:]

I have read all of this thread and found it very interesting.  I worked on powertrain controllers as a consultant for brands x and y.  They both had a mechanical engineering mind set in the early days and put any old EE on the coding for the controllers.  Glad to say that is no longer the case.  Some went kicking and screaming from absolute assembly to relocatable assembly.  Then the same with going to "C".  And, again for a RTOS.  At this time they are doing model based control algorithms and auto code generation with an RTOS.  There were safety concerns about allowing higher level interrupts interrupt lower level interrupts.  It was believed that circumstances might arise that could not be reliably predicted.  I disagreed, but I was probably wrong.  A friend working at Wind River was involved in the code for the Mars rovers.  You may recall the first rover froze up.  The software locked up.  The problem was task scheduling in the very complex multitasking RTOS.  Luckily, or cleverly, they had some code in the system that recognized how much trouble it was in and went into a mode to accept new code by telemetry.  So, by the time the second rover landed, they had the fix.  I am an old engineer 65+ and age is not the problem, mind set is.  You are either open to new ideas, or not.  I have a hobby farm and I am retired, so my current project is to use a Raspberry Pi(RPI)... (google) and an attached I/O board with an ATmega328p on it.  The ATmega is much more powerful that the first chips we ran the engine with which was (1K RAM, 16K ROM, and 2MHz).  We used a 10msec interrupt to read the tone wheels for RPM and schedule background tasks.  It was a rudimentary O/S for scheduling and accurate sensor reading.  The building security/monitoring/controlling I will do for three out buildings on the property will use a similar rudimentary O/S.  The ATmega is a slave to the RPI on a 115K serial UART channel.  The RPI is a powerful processor with 512K RAM and SD flash ROM (8Gig) with netork connector and two USB connectors.  I will run WiFi on one of the USB ports. The RPI runs Debian Linux and can do an Apache server, if you wish.  The cost is very low.  The RPI is $35 and the Gert I/O board is $48.  I was working on computerized test equipment at Bell Labs when the Intel 4004 and 8008 came out.  We were doing 148 pin circuit board testers and wanted to go to a processor per pin, but the 8008 and 4004 were not fast enough or powerful enough.  We stayed with DEC and Data General minicomputers.  Anyway, this thread was very interesting and I think that the power of the Arduino boards does warrant the use of a periodic interrupt and simple task scheduler for many applications.  I have used the 'MicroC/OS-II' real-time kernel by Jean J. Labrosse, which has been ported to many uPs.  It does a fine job and allows the user to pick features and leave out features to arrive at the proper size and power.  Worked well for me on my greenhouse controller, which reports over the internet.  I have seen another similar featured RTOS called 'freeRTOS'...(google) that costs nothing.  I think it would be useful for some to look at these.  Have fun .

[Reply #19 on:]

Much was made of the Mars rover bug but it was just that, a bug.  Like all bugs, it shouldn't have happened since the proper design for avoiding "priority inversion" was well known since the early 1970s.  The Mars rover problem happened in 1997.

When created, a VxWorks mutex object accepts a boolean parameter that indicates whether priority inheritance should be performed by the mutex. The mutex in question had been initialized with the parameter off; had it been on, the low-priority meteorological thread would have inherited the priority of the high-priority data bus thread blocked on it while it held the mutex, causing it be scheduled with higher priority than the medium-priority communications task, thus preventing the priority inversion. Once diagnosed, it was clear to the JPL engineers that using priority inheritance would prevent the resets they were seeing.

I did become part of the fear factor of using a RTOS.  Like this kind of misinformation - "an RTOS shouldn't be used on an Uno since the overhead is too high".

Here is a simple case study for an example I am developing.  The problem is to read data from analog pins at regular intervals and write it to an SD card.

The simple solution is a loop like this:

1. Wait till start of period.

2. Read data.

3. Write data to SD.

4. Repeat.

A problem occurs when the period between points is less than about 100 milliseconds.  SD cards can have occasional latencies of over 100 milliseconds so data overruns occur.

A possible solution is to use an RTOS with two threads.  Can the Uno support the extra overhead?

The answer is that the RTOS solution is far more efficient than the above loop.  Here's why.

The RTOS solution has two threads.

The analog read thread runs at high priority and is a loop like this:

1. Wait till start of period.

2. Read data.

3. Write data to a FIFO buffer.

4. Repeat.

The SD write thread is a loop that runs at lower priority.

1. Wait for data in the FIFO.

2. Write data to SD

3. Repeat.

The two thread solution is more efficient than the first single loop solution.  CPU time is recovered when the SD is busy and the higher priority thread is scheduled.

Now comes the real payoff.  The Arduino analogRead() take about 115 microseconds.  Almost all of this is in a busy loop waiting for the ADC conversion.

I wrote an RTOS based replacement for analogRead() that is transparent to users but sleeps during the ADC conversion.  This saves over 90 microseconds of CPU time per read after factoring in a context switch.

The result is that the RTOS version can log more than twice as fast and doesn't suffer data overruns.  The simple loop version has the high overhead of busy loops that the RTOS avoids.

I have ported three RTOSs to Arduino http://code.google.com/p/rtoslibs/.

My favorite for Uno is NilRTOS.  Its author, Giovanni Di Sirio, says it's "Smaller than ChibiOS/RT, so small it's almost nil."

[Reply #20 on:]

Interesting thread. I have been working on commercial hard- and soft- real-time control systems for a number of years. Since those terms are commonly misused and misunderstood here is an example of a hard-time system we did a few years ago: a mud-pump controller driving two pistons with 2-meter stroke, controlled with about 500 HP of hydraulic pumps (at 10,000 psi, the manifold pipe is something like 200 mm in dia), through a servovalve for each piston. Each piston moves by a polynomial equation and their outputs are summed through a check valve so that the flow is constant, and adjustable over a wide range. LVDTs with 2-meter stroke monitor each piston. Smooth startup and shutdown and several considerations of fail-safe behavior were used. We achieved this on an 8051 with external ADCs and DACs, and interrupt handlers (two levels, pre-emptive) with re-entrant libraries. We almost ran out of code space. In our case the time interval was not tiny, but we had hard completion deadlines. The piston profile came out of a lookup table (actually 1/4 of the entire profile, mirrored and phase-shifted as needed), the actual piston position was read, and that was all fed into a firmware PID routine which then calculated the next value for the servovalves. That had to be completed before the timer tick to update the valves. That timer got faster as the flow rate was increased. Then in the background was interrupt-driven serial I/O for the machine interface to control the mud pump and to monitor critical performance values (such as the current error value vs target position). This I/O had to be safely pre-empted by the piston routines with due consideration of atomicity of variables which might be in the midst of being updated by the control interface. In the end it all worked well, we delivered complete documentation and source code, and I have not heard of any problems. I went to Japan to help install and tune the system and then our part of the project was complete.

Where execution deadlines got tight I would set and clear some spare I/O bits and watch them on an oscilloscope or logic analyzer, something like High when active in a critical routine and Low when in safe extra time margin. As the flow rate ramped up you could see the bit transitions coming closer together. If they ever collided that would be potentially catastrophic since the control loop could not correctly function. We set the scope to trigger on the smallest safe interval and left it running overnight (without driving the actual pumps - simulated input). There was also simple instrumentation in the code to log deadline violations.

The project engineer was a delight to work with: a very experienced, practical guy. One example: we had to monitor the end-of travel limit clearance of the pistons (which weighed over 1000 kg as I recall) since we didn't ever want to drive them into their stops. No one knew what would happen if that occurred. Safe clearance was something like 5-10 mm, and it could not be observed easily by eye while running. His brilliant idea was to use an empty aluminum soft drink can in the gap and measure the crushed thickness. Worked great and no risk of harm to the pistons.

In this project a simple RTOS might have been a huge timesaver. But we also had to have timer interrupts and serial I/O interrupts and I am not aware of how easily RTOSes can fit with those. If it is possible to weave RTOS features into your own needed I/O hardware support, that could be helpful but also could get a bit complex.

I am using Teensy++2 on a project now and just got the ARM Teensy 3, and there is enough code and data space on these new Arduino devices. My concern would be: is the RTOS granular so we only need to use what fits our case, and can it co-exist with I/O device interrupt handlers?

Thanks

[Reply #21 on:]

In this project a simple RTOS might have been a huge timesaver. But we also had to have timer interrupts and serial I/O interrupts and I am not aware of how easily RTOSes can fit with those. If it is possible to weave RTOS features into your own needed I/O hardware support, that could be helpful but also could get a bit complex.

My concern would be: is the RTOS granular so we only need to use what fits our case, and can it co-exist with I/O device interrupt handlers?

A good RTOS allows independent interrupt handlers.  The three systems I ported to AVR/ARM Arduinos can be used as simple kernels with all I/O from the Arduino core or custom libraries.

I am starting to write optional I/O libraries that are integrated with the RTOS.  Many existing Arduino I/O functions just do busy loops so putting a thread to sleep and waking it with an I/O done interrupt is more efficient.

If you need a highly optimized ISRs, systems like ChibiOS are designed to allow your ISR to run with no interference from the RTOS.

On ARM this works very well with priority interrupts.  ChibiOS is designed to allow "Fast interrupts".

On some architectures ChibiOS supports a special class of “Fast Interrupts”, such interrupt sources have a higher hardware priority than the kernel so it is not possible to invoke system APIs from there.

The invocation of any API is forbidden here because fast interrupt handlers can preempt the kernel even within its critical zones in order to minimize latency.

The only hardware resource I uses is access to a timer interrupt for the system tick. 

I piggyback on sysTick on ARM since both Teensy 3.0 and Due will have a hook.  So no new hardware is used on ARM.

On AVR I am using the timer 0 compare A interrupt.  I did this so no Arduino core code needs to be changed.  I will release an optional patch to piggyback on the timer 0 overflow interrupt.  This is already used for delay(), millis(), and micros().  Then not new hardware will be used on AVR.