Go Down

Topic: filter library (Read 11656 times) previous topic - next topic

scjurgen


I did some filter design scripts  :smiley-sweat: which could come in handy for (y)our measurements.
It covers for now some IIR filters with Butterworth, Chebyshev, Bessel characteristics (1st order and up to an exaggerate 5th order). Furthermore you will find Resonance and Proportional Integral filters. All pass types are present (low, high, band, stop, all). It will generate classes for floating point or integer calculation.

if you are interested check it out here: http://www.schwietering.com/jayduino/filtuino/

Comments and bug reports are welcome. If you find them useful, drop me a line, so I just know that I did something useful for others  ;)
Juergen

robtillaart

Nice work, bit for those not familiar with this theory (like myself) could you provide links to some background info, or provide a playground article?
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

scjurgen

Quote
Nice work, bit for those not familiar with this theory (like myself) could you provide links to some background info, or provide a playground article?


Natuurlijk. Best would be to provide some info where I do actually need them, or where typical applications are in microcontroller projects.

There is a lot of info about filters on wiki, but that is pretty toughhttp://en.wikipedia.org/wiki/Digital_filter. A nice tutorial but in the field of Audio you find here: https://ccrma.stanford.edu/~jos/filters/filters.html.

The more practical approach:
For cleaning sensor data I used Bandpass and Lowpass filters, so the higher frequency fluctuations are reduced.
For such a purpose you can use: 1st order, Bessel or Chebyshev, Lowpass or Bandpass filters.
A Bandpass filter would also get rid of the DC that you might have in a sensorvalue without need of calibrating (i.e. in calm state, you have +2V, with a signal more or less, the bandpass will give you positive and negative values around).

For musical applications (bit tough on the Arduino because of the max Samplerate), you would use Butterworth filters as they will distort less the useful frequencies.

For myself I use them to convert Vibrations measured with an optical sensor where I am interested in the mean frequency of the vibration (which is 1 stable frequency) but I want to avoid upper harmonics of the vibration. i.e. the vibrations are 100Hz, my samplingrate is 2Khz, the first harmonic would be 200Hz which I am not interested in. In the filter design I will use the samplingrates I stated (2000Hz), use the same frequencies for low and high (100Hz) and using a Bandpass filter. This pretty much cleans the signal and just by doing an average over the sampleperiod (10msecs) I get a perfect reading without adding weird math.

Another typical application would be to get rid of 50/60Hz in sensor readings caused by power supply which you could not avoid by using electrical filters (i.e. optical sensors in an ambient with artificial light). In this case you use a Bandstop using the 50Hz (Europe) or 60Hz (US) as frequencies and the sampling rate of your readings (which must be > 100Hz actually).

Furthermore, the Proportional Integral Filters are widely used in process control where something does not respond quickly to your input (steering a ship for example), so if you have a gyro you would use such a filter to avoid 'over-reacting', see also here http://en.wikipedia.org/wiki/PID_controller.


Juergen

RuggedCircuits

That is very nice work. Congratulations.

I don't suppose you could be coaxed into adding a graph showing the frequency response?  ;)

--
The Aussie Shield: breakout all 28 pins to quick-connect terminals

scjurgen

Quote
coaxed into adding a graph showing the frequency response?


I had some thoughts about it. What I am actually interested in is not a graph of the theoretical response (you can find  in the net) but a simulation of the calculation, so I can see if the integer calculation optimizations are ok for the kind of signals I got. But looks like pretty much work!  :smiley-sweat:
Juergen

RuggedCircuits

Quote
What I am actually interested in is not a graph of the theoretical response (you can find  in the net) but a simulation of the calculation, so I can see if the integer calculation optimizations are ok for the kind of signals I got.


That's what I was going to ask for once you got the theoretical graph part done ;)

--
The Rugged Motor Driver: two H-bridges, more power than an L298, fully protected

rocketgeek

That's immensely cool. I've had a couple of discussions with people recently about doing DFTs on signals acquired with an Arduino for various purposes and the potential use of filter banks instead. This library should be most useful.

MarkT

Yes, here here, great work.

Would be lovely to have a phase-linear FIR design tool as well  ;)
[ I won't respond to messages, use the forum please ]

doggenj

I made little library based on some of your filter code.
The goal of this library is to make is very easy for anyone to use (basic) signal filters when reading analog sensors.

I plan on adding more filters later on: running average, Kalman,...

http://code.google.com/p/arduino-signal-filtering-library/

Magician

To doggenj:  nice and promising work, definitely deserve separate topic. Have been myself tweaking with filtering algorithm, I wander what is the base or thoughts for your math, why you decide to store "history" _V  as 8-bit short value? IMHO, it considerably reduce resolution, especially for second order filters, where coefficient small.
Code: [Select]
long tmp = ((((data * 662828L) >>  4) //= (    7.901529699e-2 * x)
                      btw, should it be 662828 / 2^23 = 0.079015255?
+ ((_v[0] * -540791L) >> 1) //+( -0.5157387562*v[0])
+ (_v[1] * 628977L) //+(  1.1996775682*v[1])
)+262144) >> 19; // round and downshift fixed point /524288

_v[2]= (short)tmp;

Multiplication by 0.08 means, that anything less than 1/0.08 = 12.5 not accounted

If it for speed/performance, than why do "long" math, if "integer" would be sufficient?

doggenj

Thanks!

I actually did not really pay to much attention to the math and just used that version based on performance.
(the other version of the code written by 'scjurgen' all use floating point math)

The 16 bits in an Arduino short seemed reasonable, considering the 10 bit ADC.
(16 vs 8 bits for the short type: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1261049420/1#1)




Magician

#11
Jan 19, 2012, 03:09 pm Last Edit: Jan 19, 2012, 03:22 pm by Magician Reason: 1
Opsss, so "short" is 16-bit value, my mistake. The way it's declared and how you "scale" gives me wrong impression.
Code: [Select]
int _order;
short _v[2];

Probably,  
Code: [Select]
int _order;
int _v[2]; or int16_t _V[2];
would be less confusion. But I more concerned with "scaling", why you are shifting 19 -right, if you reserved a store room for 16-bit variable?
There is a formula:
tmp = ((((data * 662828L)>>4)+((_v[0] * -540791L)>>1)+(_v[1] * 628977L))+262144) >> 19;
Or
tmp = ((((data * 662828L)>>23)+((_v[0] * -540791L)>>20)+(_v[1] * 628977L>>19))+ 0.5);
Data  multiplied by 0.08, with 10-bit ADC (max 1023) it's always less than 82 (7-bit),  there is no chance tmp would ever accumulate anything bigger than  9-bits, because worst case when both history data in right phase and add up biggest 7-bit they hold. Why not increase tmp in size up to 15-bit (plus sign it fits in int16_t), simply shifting to right 6 steps less, or >> 13 instead of >> 19?

tmp = ((((data * 662828L)>>4)+((_v[0] * -540791L)>>7)+(_v[1] * 628977L>>6)) +0.5) >> 13;
Am I missing something?

doggenj

#12
Jan 20, 2012, 04:17 pm Last Edit: Jan 20, 2012, 04:23 pm by doggenj Reason: 1
Maybe this was not really clear: the filtering code was written by scjurgen, I just reused it.

I wrote a small c program to do some tests and to understand the calculations better.
Data  multiplied by 0.08 is always less than 82, but the "long tmp" value in binary representation is much bigger (18 up to 30 bits in my tests).

I have split up the calculations in four parts and I print the intermediate results in decimal and binary.
The program prints the tmp value for three possible incoming samples: 1, 512 and 1023 (using 0,0,0 or 1023,1023,1023 as the 'history')

tmp1 = (data * 662828L);
  --> print tmp1:
       tmp1: 662828
       tmp1: 00000000000010100001110100101100

tmp2 = ((data * 662828L) >>  4);
  --> print tmp2...
tmp3 = ((((data * 662828L) >>  4) + ((_v[0] * -540791L) >> 1) + (_v[1] * 628977L))+262144);
  --> print tmp3...
tmp4 = ((((data * 662828L) >>  4) + ((_v[0] * -540791L) >> 1) + (_v[1] * 628977L))+262144) >> 19;
  --> print tmp4...

Total output:
http://arduino-signal-filtering-library.googlecode.com/hg-history/2770a5e0dae02e0ef56ace52d35fc6c350648b25/TestCode/FilterShiftingTest/debug.txt

The code: (+compiled linux and win32 binary)
http://code.google.com/p/arduino-signal-filtering-library/source/browse/TestCode#TestCode/FilterShiftingTest

Magician

You test just confirmed what I'm trying to explain. Tmp4 is a value that goes to history, and because tmp3 shifted right 19 bits, tmp4 is always less than 10-bit. So, the question is why someone would shift 19-bit right, if there is a space for 16-bit variable? Why not shift 13-bit right,
and get million times more accurate value? You can downscale it whenever you gonna use it again, if it's too big.  This about integer math, not the filter code.
In integer, IMHO, you always trying to keep value as big as possible, barely avoiding overflow .
Simply, because if it casted/shifted, data / precision lost forever. In this particular case,
the best results you would get, if all variables in the formula about size of 29-bit, so sum of 3 not overflow long-32. 
You doing it about right, 10-bit Data multiply by 20-bit coefficients:

tmp = (A(20-bit) x Data(10-bit) + B(20-bit) x Hist1(10-bit) + C(20-bit) x Hist2(10-bit))>>19.

A - 662828 ( 1010 0001 1101 0010 1100 = 20 bit)
B -  -540791 ( 1000 0100 0000 0111 0111 = 20 bit)
and C - 628977 ( 1001 1001 1000 1111 0001 = 20 bit)

, only it'd be more accurate, if :

tmp = (A(20-bit) x Data(10-bit) + B(14-bit) x Hist1(16-bit) + C(14-bit) x Hist2(16-bit))>>13.
Wouldn't it?

boganazhapa

[google Translate]

Hello community

Interesting program filters, I'm performing a signal processing project and I would like to know how to get the cutoff frequency for each filter, in my case I want to enter cutoff frequencies of 0.1 Hz and 45Hz, would be helpful ...

This library that I'm using, or if you have other cross me ...
http://jeroendoggen.github.io/Arduino-signal-filtering-library/

Go Up