Go Down

Topic: Filtering outliers with a Hampel filter (Read 137 times) previous topic - next topic

flrs

Feb 19, 2016, 07:47 am Last Edit: Feb 19, 2016, 07:52 am by flrs
Hi all,

I am writing this post to introduce you to the Hampel filter library I just finished writing and posted here: http://github.com/flrs/HampelFilter.

The Hampel filter library uses a Hampel filter to identify outliers in data. The library is optimized for speed and comes with an example sketch.

Hampel filters are known to perform better for outlier detection than median filters. This is because, other than median filters, Hampel filters also take the distribution of data into account. While median filters often work with a fixed threshold for outlier detection, Hampel filters use a scaling factor which takes the distribution of the data into account. In other words: If all recorded values are really close to the median, the Hampel filter adjusts to find the values that are not so close to the median. When later in the process recorded values are generally farther away from the median, the Hampel filter automatically adjusts to this and identifies outliers as points that are even farther away from the median. With usual Median filters with a fixed threshold this behavior is difficult to achieve.

In the past there have been a couple of posts related to filtering outliers, see this question by strange quark and robtillaart's post introducing the RunningMedian library.

My Hampel filter library code has been inspired by rkail's post about a fast running median filter. However, I changed this code quite a bit. Major changes are:
  • Changed original bubble sort algorithm to comb sort algorithm as it performs faster.
  • Allow only floating point numbers in the range from -255.00 to 255.00 as inputs to the buffer to be able to perform integer-only calculations internally. This improves performance.
  • Added functions to determine the median absolute deviation.

I am looking forward to your feedback and comments!

flrs

Go Up
 


Please enter a valid email to subscribe

Confirm your email address

We need to confirm your email address.
To complete the subscription, please click the link in the email we just sent you.

Thank you for subscribing!

Arduino
via Egeo 16
Torino, 10131
Italy