Pages: 1 2 [3] 4   Go Down
Author Topic: NilRTOS - A Fast Tiny Preemptive RTOS  (Read 15334 times)
0 Members and 1 Guest are viewing this topic.
0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I modified the program to log three ADC channels every 1024 microseconds.  

I added another performance measure, the minimum number of free data records.  For the the three ADC test the result is:
Quote
Done
Maximum SD write latency: 48364 usec
Minimum free record count: 89
Unused Stack: 57 152
I ran for about 15 minutes with a high quality industrial SD. I allocated 150 FIFO records and the minimum number of free records was 89 so I was never close to an overrun.

The log file size was about 10 MB.

pito,

Thanks for your effort to find the compiler bit field bug.  Compiler bugs can be nasty.
« Last Edit: January 26, 2013, 10:59:49 am by fat16lib » Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I've tried with up to 5 channels, but getting overrun errors (with much smaller fifo_sizes, and cheap sdcard). It seems to me we have to find a way how to work with bitfields, though. Otherwise we waste the fifo space we desperately need..  smiley
« Last Edit: January 26, 2013, 01:05:07 pm by pito » Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I tried a Mega and five is the limit.  The problem is that each ADC read takes 110 microseconds so there isn't enough time left in the SD thread to format the data.

You don't lose much accuracy by speeding up the ADC clock so that could help.

Finally you could trigger the ADC with a timer compare and read it in an ISR.  You could write binary...

But all of this is way beyond an example for NilRTOS.  Right now I want to improve all the RTOS ports.

I plan to post improved versions of ChibiOS/RT, FreeRTOS, and NilRTOS on http://code.google.com/p/rtoslibs/.

I was thinking about porting NilRTOS to ARM boards but I decided ChibiOS/RT is a better fit.

Here is a sample of 16 ADC channels logged every 4096 microseconds on a Mega:
Quote
Period,ADC0,ADC1,ADC2,ADC3,ADC4,ADC5,ADC6,ADC7,ADC8,ADC9,ADC10,ADC11,ADC12,ADC13,ADC14,ADC15
NA,507,448,413,388,375,360,347,337,362,346,352,342,333,345,334,335
4096,399,406,402,391,382,370,357,344,347,340,343,337,329,334,329,329
4096,350,364,372,372,370,364,355,341,334,330,331,329,322,324,321,323
4096,331,340,349,353,356,356,351,339,329,327,327,326,321,322,320,322
4096,325,329,335,340,344,346,345,335,326,325,325,324,320,321,319,320
4096,318,318,320,323,327,330,331,323,316,315,315,314,311,312,309,310
4096,289,290,294,297,302,305,308,304,299,296,294,293,290,291,291,293
4096,305,301,299,298,298,300,301,297,295,296,298,298,296,297,295,293
4096,296,297,296,295,295,296,297,293,291,292,293,294,293,294,293,290

Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

For example it works here with:
Code:
struct FifoItem_t {
  uint16_t usec : 16;        // Low 16-bit of time in usec.
  uint16_t value0 : 10;  // ADC value.
  uint16_t value1 : 10;  // ADC value.
  uint16_t value2 : 10;  // ADC value.
  uint16_t value3 : 10;  // ADC value.
  uint16_t value4 : 10;  // ADC value.
  uint8_t error : 6;   // Overrun count since last point.
};
fifo_size 100, intervalTicks=2, no analogreference, no overrun for 30secs

PS: not sure we can do this (the original struct causing troubles):
Code:
struct FifoItem_t {
  uint16_t usec;        // Low 16-bit of time in usec.
  unsigned value : 10;  // ADC value.
  unsigned error : 6;   // Overrun count since last point.
};
within a struct - mixing int and bitfields - maybe within an union? ..



* 5ch.jpg (32.58 KB, 487x230 - viewed 63 times.)
« Last Edit: January 26, 2013, 02:10:24 pm by pito » Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I started using bit fields in this example because I have used them before on AVR.  

The original structure worked with some assignment statements and failed with others.  I tested it with a counter and that worked OK.

I don't trust bit fields now.  I wonder if the avr-gcc people have a description of the bug.

You could store the low 8-bits of each ADC in a byte and the upper 2-bits packed by using shifts and masks with four to a byte.  The inverse could be used in the SD print thread.

I was just at the gym exercising and I started to think about the ADC read problem.  I think I will do an ISR/semaphore replacement for analogRead().  I will setup the ADC so it interrupts when done.  I will sleep on a semaphore that will be signaled by the ADC ISR like the nilInterrupt example.

Edit: That was easy.  I now have nilAnalogRead(pin).  It takes 126 microseconds 110 is sleep while the ADC is doing the conversion and 16 is ISR, semaphore, and context switch overhead.  I get back 94 microseconds of CPU time per read but a read take 16 microseconds longer.

I can now log all six ADC channels on a Uno using a period of 1024 microseconds with nilAnalogRead().
« Last Edit: January 26, 2013, 05:23:14 pm by fat16lib » Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Few pictures for you tutorial  smiley
fifo_b - ..basic situation..
fifo5  - ..adc maybe faster than sdcard..
fifo6 -  ..till overrun error..

fifo_a - ..counting semaphores handshaking..


* fifo_a.jpg (48.87 KB, 497x509 - viewed 64 times.)

* fifo5.jpg (44.39 KB, 496x507 - viewed 45 times.)

* fifo6.jpg (48.82 KB, 495x507 - viewed 51 times.)

* fifo_b.jpg (39.71 KB, 496x505 - viewed 49 times.)
« Last Edit: January 27, 2013, 02:53:49 pm by pito » Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

pito,

Wow these are great.  I was about to post an update but I will put your pictures in the Examples explanation.

I also found that I must restructure Nil into several libraries.  The ISR for nilAnalogRead() is always linked.  I guess I will need separate libraries for some parts of Nil.  I can't think of a workaround.
Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I have posted an updated version of Nil RTOS as NilRTOS20130127.zip http://code.google.com/p/rtoslibs/downloads/list.

It has new features including an ADC library that sleeps while conversions are in progress.  This saves CPU time and allows other threads to execute.

A new version of SdFat is included with a function that formats uint16_t fields three times faster than the standard Arduino Print.

A data logging example is included that can log five analog pins on an Uno to a csv file every tick (1024 usec) which is 976 Hz.  All 16 analog pins on a Mega can be logged to a csv file every 3 ticks which is 325 Hz.  A quality SD card is required to achieve these results.

Documentation has been improved and two bugs have been fixed.
Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Works nice!  smiley
(ie. Uno, 5 adc channels, 2048usec period, cheap card, 7 minutes logging time, 5.5MB data.csv):
Code:
type any character to begin
type any character to end
Done
FIFO record count: 75
Minimum free record count: 54
Maximum SD write latency: 41064 usec
Unused Stack: 51 125
Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Fyi - here is the 1024usec period, 4 ADC channels, Latency and FreeFifo stored to the sdcard. You may see Latency and FreeFifo chart over the time (32k measurements shown).
Time is in msec, Latency in usec,  FreeFifo in free records..
Code:
type any character to begin
type any character to end
Done
FIFO record count: 90
Minimum free record count: 0
Maximum SD write latency: 35072 usec
Unused Stack: 51 144

** overrun errors **
Maximum overrun count: 32


* latency.jpg (69.27 KB, 1163x546 - viewed 54 times.)

* fifo.jpg (61.88 KB, 1165x545 - viewed 54 times.)
« Last Edit: January 28, 2013, 08:46:02 am by pito » Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

And the same with 2048usec period (Time is *2 msecs). You may see how the freefifo dropped to 69 during the highest latency hit (40msec).
Code:
type any character to begin
type any character to end
Done
FIFO record count: 90
Minimum free record count: 69
Maximum SD write latency: 40292 usec
Unused Stack: 51 151


* lat2.jpg (71.38 KB, 1191x564 - viewed 48 times.)

* freefifo2.jpg (55.01 KB, 1183x565 - viewed 50 times.)
« Last Edit: January 28, 2013, 08:37:25 am by pito » Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

pito,

I wrapped the FIFO code in a C++ template so it is easy to use. Here is an example:
Code:
#include <NilRTOS.h>
#include <NilFIFO.h>

// FIFO with ten ints
NilFIFO<int, 10> fifo;

// Use tiny unbuffered NilRTOS NilSerial library.
#include <NilSerial.h>
#define Serial NilSerial

//------------------------------------------------------------------------------
NIL_WORKING_AREA(waThread1, 64);

NIL_THREAD(Thread1, arg) {
  int n = 0;
  while (TRUE) {

    nilThdSleep(400);
    int* p = fifo.waitFree(TIME_IMMEDIATE);
    
    // continue if no free space
    if (p == 0) continue;
    *p = n++;
    fifo.signalData();
  }
}
//------------------------------------------------------------------------------
NIL_THREADS_TABLE_BEGIN()
NIL_THREADS_TABLE_ENTRY(NULL, Thread1, NULL, waThread1, sizeof(waThread1))
NIL_THREADS_TABLE_END()
//------------------------------------------------------------------------------
void setup() {

  Serial.begin(9600);
  // start kernel
  nilSysBegin();
}
//------------------------------------------------------------------------------
void loop() {
  int* p = fifo.waitData(TIME_IMMEDIATE);
  
  // return if no data
  if (p == 0) return;
  
  int n = *p;
  Serial.println(n);

  fifo.signalFree();

  if (n == 20) {
    fifo.printStats(&Serial);
    while(1);
  }
}
Here are the last lines of output:
Quote
19
20
FIFO record count: 10
Minimum free count: 9
I attached the template as NilFIFO.h. It's not documented yet

I downloaded your new diagrams for a future update of the documentation.

The SD performance is interesting.  SD controllers are impossible to understand since they are a trade top secret.
Users could benefit from your plots, I should write an article to include with SdFat.  My to-do list is getting so long.

* NilFIFO.h (1.3 KB - downloaded 28 times.)
« Last Edit: January 28, 2013, 08:52:35 am by fat16lib » Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

..as a very first step I would strongly recommend to ask kindly the maintainers to open a new item on this forum ie. in Topics or Development - "RTOS", as it is clear the topic will cover a very important concept all arduino users may easily benefit from..
PS: and maybe to move all the rtos-related topics there..

« Last Edit: January 28, 2013, 01:35:39 pm by pito » Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I've tried with 1284p (16MHz) and 1024us period. Fifosize=10kBytes, 2minutes logging time.
5ADC channels:
Code:
FIFO record count: 833
Minimum free record count: 0
Maximum SD write latency: 48860 usec
Unused Stack: 51 5341

** overrun errors **
Maximum overrun count: 44
4ADC channels:
Code:
FIFO record count: 1000
Minimum free record count: 0
Maximum SD write latency: 55124 usec
Unused Stack: 51 5354

** overrun errors **
Maximum overrun count: 11
3ADC channels:
Code:
FIFO record count: 1250
Minimum free record count: 1115
Maximum SD write latency: 49640 usec
Unused Stack: 51 5354
We need a faster ADCs and SDcards smiley

8ADC channels and 2048us period:
Code:
FIFO record count: 555
Minimum free record count: 499
Maximum SD write latency: 49764 usec
Unused Stack: 51 5364
« Last Edit: February 01, 2013, 12:58:53 pm by pito » Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

First, I just posted a new NilRTOS http://code.google.com/p/rtoslibs/downloads/list.  Look at the new nilSdLogger.ino example, it uses the FIFO template and NilTimer1.

The problem is CPU power to format numbers for these examples.

My new SdFat printField() is about three times faster than Arduino Print but is still the bottleneck when you have lots of RAM.

Quote
FIFO record count: 833
Minimum free record count: 0
Maximum SD write latency: 48860 usec
Unused Stack: 51 5341
The 833 FIFO records at 1024 usec provides about 850 milliseconds of buffering.  The max latency was under 50 milliseconds so text formatting is the problem.
« Last Edit: February 01, 2013, 01:57:27 pm by fat16lib » Logged

Pages: 1 2 [3] 4   Go Up
Jump to: