Go Down

Topic: runningMedian class on playground (Read 28026 times) previous topic - next topic

Wareemba

#75
Jan 29, 2016, 11:40 am Last Edit: Jan 29, 2016, 11:52 am by Wareemba
You can also analyse the code of the class.


ah, so where it states:
Code: [Select]
double getAverage(uint8_t nMedian);

whatever is the in () in the sketch is the 'n' :) this was not obvious, but now is! so if my N is larger, then i can widen my n to see how it looks? but i really want to be just using median() for my readings i think? or DO i want to be using a "middle average" of the sensors readings?

any thoughts on DHT22 sensor sampling frequency? closer together - or further apart?

and why is N set to a maximum of 19? is looking at more than 19 not worth it? or is it processor power related?

oh - and what happens with a NaN?? is it discarded? or left in and ignored?

robtillaart

#76
Jan 29, 2016, 08:25 pm Last Edit: Jan 29, 2016, 08:57 pm by robtillaart
>> ah, so where it states: double getAverage(uint8_t nMedian);

That function but then in  the .cpp file

>> so if my N is larger, then i can widen my n to see how it looks?
Yes

>> but i really want to be just using median() for my readings i think?
I assume you know what you think, but yes

>> or DO i want to be using a "middle average" of the sensors readings?
It is your project so you define your requirements. Normally median() should work for you. The average(5); can be useful if you want one decimal place but the sensor does only return integers e.g. the DHT11.

>> any thoughts on DHT22 sensor sampling frequency? closer together - or further apart?
You can make high frequency measurements (20x / minute) and see that the temp/hum changes are changing only once per 5 minutes. That would be overkill. Make a test run and check the fastest change you see and compare that to the smallest change you want to see.
E.g. if you want to see every 0.1 delta in temp you must probably sample fast. If you are interested in only 0.5 or 1.0 changes you can sample a lot less. Especially when run on batteries you want to minimize power usage.

>> and why is N set to a maximum of 19?
Because it can be done? It is just a demo program, I can do a theoretical course about median, but read wikipedia about it and think for yourself when you should use a bigger buffer and when a smaller?
when 5, when 11, when 19 or even bigger. What happens to the data stream if you do so?
Why did you just copy it?

>>  is looking at more than 19 not worth it?
You determine the requirements

>> or is it processor power related?
No on board memory and common sense

>>oh - and what happens with a NaN?? is it discarded? or left in and ignored?
Not handled, good point.
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

flrs

runningMedian folks,

I just published the Hampel filter library which identifies outliers not only based on the median, but also based on the distribution of the data. Find the library on GitHub and the related forum post here.

flrs

SkobyMobil

Hello,
This all works wonderfully, but-
2 things I do not get in the sketch ...

How to delete counters and variables?
How do I find the value that is most common?
With "mode ()"? If yes how?

Many thanks for your help
Greeting and fun
Andreas
die zweite Maus bekommt den Speck...

xdeschain

I found a bug in the insertion sort of FastRunningMedian.h and fixed it. The left inseration sort must be:

int16_t j = i - 1;
   while (j > -1 && new_value < _sortbuffer[j])


Furthermore I have inserted  binary search, getHighest, getLowest, getAverage to the code and changed uint8_t to int16_t.

Code: [Select]

//
// FILE: FastRunningMedian.h
// AUTHOR: rkaul
// PURPOSE: RunningMedian library for Arduino
// VERSION: 0.1.01
// HISTORY:
// 0.1.00 rkaul initial version
// 0.1.01 rob.tillaart -> insertion sort
// 0.1.02 xdeschain -> insertion sort fixed bug, binary search, getHighest, getLowest, getAverage
// Released to the public domain
//
// Remarks:
// This is a lean but fast version.
// Initially, the buffer is filled with a "default_value". To get real median values
// you have to fill the object with N values, where N is the size of the sliding window.
// For example: for(int16_t i=0; i < 32; i++) myMedian.addValue(readSensor());
//
// Constructor:
// FastRunningMedian<datatype_of_content, size_of_sliding_window, default_value>
// maximim size_of_sliding_window is 255
// Methods:
// addValue(val) adds a new value to the buffers (and kicks the oldest)
// getMedian() returns the current median value
//
//
// Usage:
// #include "FastRunningMedian.h"
// FastRunningMedian<unsigned int,32, 0> myMedian;
// ....
// myMedian.addValue(value); // adds a value
// m = myMedian.getMedian(); // retieves the median
//

#include "Arduino.h"

#ifndef FastRunningMedian_h
#define FastRunningMedian_h

#include <inttypes.h>

template <typename T, int16_t N, T default_value> class FastRunningMedian {

public:
 FastRunningMedian() {
 _buffer_ptr = N;
 _window_size = N;
 _median_ptr = N / 2;

 // Init buffers
 int16_t i = _window_size;
 while (i > 0) {
 i--;
 _inbuffer[i] = default_value;
 _sortbuffer[i] = default_value;
 }
 };

 T getMedian() {
 // buffers are always sorted.
 return _sortbuffer[_median_ptr];
 }

 T getHighest() {
 // buffers are always sorted.
 return _sortbuffer[_window_size - 1];
 }

 T getLowest() {
 // buffers are always sorted.
 return _sortbuffer[0];
 }

 float getAverage() {
 float sum = 0;

 for (int16_t i = 0; i < _window_size; i++) {
 sum += _sortbuffer[i];
 }
 float value = sum / _window_size;
 return value;
 };


 float getAverage(int16_t nMedians) {
 if (nMedians > 0)
 {
 int16_t start = ((_window_size - nMedians) / 2);
 int16_t stop = start + nMedians;
 float sum = 0;
 for (int16_t i = start; i < stop; i++) {
 sum += _sortbuffer[i];
 }
 float value = sum / nMedians;
 return value;
 }
 return 0.0f;
 }

 void addValue(T new_value) {
 // comparision with 0 is fast, so we decrement _buffer_ptr
 if (_buffer_ptr == 0)
 _buffer_ptr = _window_size;

 _buffer_ptr--;

 T old_value = _inbuffer[_buffer_ptr]; // retrieve the old value to be replaced
 if (new_value == old_value)      // if the value is unchanged, do nothing
 return;

 _inbuffer[_buffer_ptr] = new_value;  // fill the new value in the cyclic buffer

 // search the old_value in the sorted buffer with binary search
 int16_t _start = 0, _end = _window_size - 1;
 int16_t _mid = 0; //middle
 while (_start <= _end)
 {
 _mid = _end + (_start - _end) / 2;    // average value...
 if (old_value == _sortbuffer[_mid]) { // If the middle element is the value, mid is the index of the found value
 break;
 }
 else if (old_value < _sortbuffer[_mid]) { // If the value is lesser than the  middle element, then the element must be in the left most region
 _end = _mid - 1;                      // pick the left half
 }
 else {                  // If the value is greater than the  middle element, then the element must be in the right most region
 _start = _mid + 1;  // pick the right half
 }
 }

 //Serial.print("found "); Serial.print(old_value); Serial.print("at location"); Serial.println(_mid);

 if (_start > _end) {
 Serial.print("ERROR old_value not found: "); Serial.println(old_value);
 }

 int16_t i = _mid;

 // insert in _sortbuffer
 if (new_value > old_value)
 {
 int16_t j = i + 1;
 while (j < _window_size && new_value > _sortbuffer[j])
 {
 _sortbuffer[j - 1] = _sortbuffer[j];
 j++;
 }
 _sortbuffer[j - 1] = new_value;
 }
 else
 {
 int16_t j = i - 1;
 while (j > -1 && new_value < _sortbuffer[j])
 {
 _sortbuffer[j + 1] = _sortbuffer[j];
 j--;
 }
 _sortbuffer[j + 1] = new_value;
 }
 }


private:
 // Pointer to the last added element in _inbuffer
 int16_t _buffer_ptr;
 // sliding window size
 int16_t _window_size;
 // position of the median value in _sortbuffer
 int16_t _median_ptr;

 // cyclic buffer for incoming values
 T _inbuffer[N];
 // sorted buffer
 T _sortbuffer[N];
};

Go Up