Eye blinking rate and FFT Realtime

Hello guys! I need your help about a project of mine...

I have an analog input that is read from a myoware sensor and I want to count the number of blinkings of my eyes.

  1. First of all, do you have any ideas about restrictions I could impose on the read values in order to classify a blinking or not? Only a threshold? I did it that way, but many times the threshold isn't something strict and for example simply by moving my head or my eyes (not blinking) is satisfied and thus it is wrongly recognized as blinking...

  2. I thought about implementing some FFT in realtime, in order to impose some stricter restrictions, depending on the frequency of the blinking waveform. But since I'm new to Arduino and this is the first time I implement (besides in theory) FFT, I don't know how to do that.
    The logic of my program is that in every loop an analog value is read and when some flags and the threshold are satisfied then I have an eye blinking. For the FFT I use the library "arduinoFFT.h" and my complete code is the following:

#include "arduinoFFT.h"

#define SAMPLES 128            //Must be a power of 2
#define SAMPLING_FREQUENCY 1000 //Hz, must be less than 10000 due to ADC
 
arduinoFFT FFT = arduinoFFT();
 
unsigned int sampling_period_us;
unsigned long microseconds;
 
double vReal[SAMPLES];
double vImag[SAMPLES];


void setup() {
  // initialize serial communication at 9600 bits per second:
  Serial.begin(9600);
  sampling_period_us = round(1000000*(1.0/SAMPLING_FREQUENCY));
}
int blinkings = 0;
int up=0, down=0;

// the loop routine runs over and over again forever:
void loop() {

  /*SAMPLING*/
  for(int i=0; i<SAMPLES; i++)
  {
    // read the input on analog pin 0:
    int sensorValue = analogRead(A0);
    vReal[i] = sensorValue;
    //unsigned long time_up = 0, time_down, time_diff;
    vImag[i] = 0;
    while(micros() < (microseconds + sampling_period_us)){
    }
    // Convert the analog reading (which goes from 0 - 1023) to a voltage (0 - 5V):
    float voltage = sensorValue * (5.0 / 1023.0);
    // print out the value you read:
    //Serial.println(voltage);
    
    if (voltage > 1.35) {
      up = 1;
    }
    if (up) {
      if (voltage < 1.35) {
        down = 1;
      }
    }
    
  }
  //Serial.println(blinkings);

      FFT.Windowing(vReal, SAMPLES, FFT_WIN_TYP_HAMMING, FFT_FORWARD);
      FFT.Compute(vReal, vImag, SAMPLES, FFT_FORWARD);
      FFT.ComplexToMagnitude(vReal, vImag, SAMPLES);
      double peak = FFT.MajorPeak(vReal, SAMPLES, SAMPLING_FREQUENCY);
    
      

      if (up && down && (peak >= 20 || peak <=150)) {
        blinkings++;
        up = 0;
        down = 0;
        //Serial.println('a');
        Serial.println(blinkings);
        //Serial.println(peak);
    }
    delay(20);
}

I found the frequency values of 20 and 150 by trial and error but in vain, since they are not very correct as restrictions...

I have an Arduino Uno and the MyoWare Sensor with the 3 extended electrodes shield.

Thank you all for advance, I am struggling over this for days!

This bit of code:

BabisI:

    while(micros() < (microseconds + sampling_period_us)){

}

...is doing nothing, you don't even set microseconds to anything other than zero.

You want:

while( (micros() - microseconds) < sampling_period_us ){
   //wait
}
microseconds= micros();

Since you are doing an FFT, try printing the results or use the serial plotter to get an idea of what the data means.

Yours,
TonyWilk

Threads merged.

Hello guys, I need your help for this project I want to do...

I need to count the number of eye blinkings that I make, using the MyoWare sensor, which reads an analog input. I also have an Arduino Uno. I have placed the sensor around my eye, using three external electrodes, with a MyoWare shield and for lower interference levels, I also use conductive gel.

My problem, to which I need your help/suggestions/advice/insight, is how to properly classify the various signals I read as a blinking or not. For this purpose, I have already implemented a first version code that is working well enough, but yet makes some mistakes; some false positives or misses some blinkings completely.

I need to further optimize it.

Generally, I impose three restrictions on the signal I read. The first one is obviously the voltage level. I have normalized the sensor values input
to 0-5 V for my convenience and seeing the serial plot, I found out an empirical threshold, below which is definitely not a blinking, but noise.

The second is the time passed since the threshold is overcome and until the voltage becomes again of lower value of the threshold. This corresponds to a full active waveform period of a blinking, since studying the analog voltage plot, I observe a pattern for the blinkings waveforms, they are like tiny waves with a peak and narrow band in time. So, I thought that I could make something out of this, but for sure this needs improvement.

And the third, I thought that it would be a good idea to use FFT (DFT), real time for the input values, in order to distinguish the different frequencies of the waves. I observed that the blinkings have a uniform waveform generally, so maybe this could be a good restriction.

I have to note that this is my first time I implement FFT on arduino and the major problem I faced was the limited memory of Uno, since I can only use 128 sampling points for the FFT, which is not sufficient I think for a good classification.

Finally, the problem I deal with is that, in practice, when I blink gently, it misses some of these blinkings or for example when I move my head not slowly it can sometimes recognize a false positive, due to the fact that the threshold was surpassed I guess, or something like that... In addition to that, when I move my eye to the side or deviating strongly from the center, then it wrongly classifies it as a blinking too.

Also the FFT is little effective and is generally between 40-60 Hz and when I blink, then it sometimes yields 90-110 Hz, which is good, but most of the times it does not, it is not recognized by FFT at all and on the contrary, it still stays between 40-60... I guess this happens because 128 points (for 1 KHz sampling frequency) is not sufficient, I don't know.. I don't have much experience in signal processing, let alone in Arduino.

What can I do? What are your suggestions/ideas/advice?
I would appreciate any kind of help on this, thank you all in advance!

My code:

#include "arduinoFFT.h"

#define SAMPLES 128            //Must be a power of 2
#define SAMPLING_FREQUENCY 1000 //Hz, must be less than 10000 due to ADC
 
arduinoFFT FFT = arduinoFFT();
 
unsigned int sampling_period_us;
unsigned long microseconds;
 
double vReal[SAMPLES];
double vImag[SAMPLES];

unsigned long start, finished, elapsed;

void setup() {
  // initialize serial communication at 9600 bits per second:
  Serial.begin(9600);
  sampling_period_us = round(1000000*(1.0/SAMPLING_FREQUENCY));
}
int blinkings = 0;
int up=0, down=0;

// the loop routine runs over and over again forever:
void loop() {

  int sensorValue;
  int FFT_flag = 1;
  /*SAMPLING*/
  for(int i=0; i<SAMPLES; i++)
  {
    // read the input on analog pin 0:
    sensorValue = analogRead(A0);
    vReal[i] = sensorValue;
    vImag[i] = 0;
    while((micros() - microseconds) < sampling_period_us){
    }
    microseconds = micros();
            
  }
  
  // Convert the analog reading (which goes from 0 - 1023) to a voltage (0 - 5V):
  float voltage = sensorValue * (5.0 / 1023.0);
  // print out the value you read:
  Serial.println(voltage);
  
  if (voltage >= 0.89) {
    up = 1;
    start = micros();
  }
  if (up) {
    if (voltage <= 0.89) {
      down = 1;
      finished = micros();
      elapsed = finished - start;
      //Serial.println(elapsed);
    }
  }
  
  //Serial.println(elapsed);
  
  if (up && down && (elapsed>=213000 && elapsed<=215000)) {
    blinkings++;
    up = 0;
    down = 0;
    FFT_flag = 0;
    //Serial.println('a');
    Serial.println(blinkings);
  }

  // If the blinking is not already recognized, only then we implement the FFT spectrum characteristic restriction to further classify the signal as blinking or not; in this way, we don't slow down the program
  // by implementing unnecessary FFTs.

  if (FFT_flag) {
    FFT.Windowing(vReal, SAMPLES, FFT_WIN_TYP_HAMMING, FFT_FORWARD);
    FFT.Compute(vReal, vImag, SAMPLES, FFT_FORWARD);
    FFT.ComplexToMagnitude(vReal, vImag, SAMPLES);
    double peak = FFT.MajorPeak(vReal, SAMPLES, SAMPLING_FREQUENCY);
    //Serial.println(peak);
    
    if (peak>=60 && peak<=110) {
      blinkings++;
      up = 0;
      down = 0;
      //Serial.println('a');
      Serial.println(blinkings);
    }
  }
  
  //Serial.println(blinkings);
  delay(20);
}

Thank you for your help!
Now it is implemented somehow better, but misses many blinkings. I wrote a better thread, more in detail and with my current code in "Project Guidance" section, so (I am asking the moderators) I would like this one to close (or deleted) and any relevant replies to be continued on there.

Sounds very interesting!

The memory problem could be solved by using a Mega or better a Teensy3.5. THis latter has hardware support for floating point and will have a faster FFT processing.

Not a FFT guru but there also exists the FHT (Fast Hartley Transform) which might use less resources.
Check - ArduinoFHT - Open Music Labs Wiki

robtillaart:
The memory problem could be solved by using a Mega or better a Teensy3.5. THis latter has hardware support for floating point and will have a faster FFT processing.

Not a FFT guru but there also exists the FHT (Fast Hartley Transform) which might use less resources.
Check - ArduinoFHT - Open Music Labs Wiki

Thank you for your answer, it is very helpful!
I tried FHT and is indeed faster, less memory-demanding and more efficient! I think it is more suitable for my project and achieves my goal very well...
Thank you again very much!

@BabisI, stop cross-posting. Threads merged.

Besides FFT/FHT, can anyone think of any other restrictions and ideas to improve the blinkings classification?
Or any other suggestion/idea for my project to further optimize/improve it?

BabisI:
Besides FFT/FHT, can anyone think of any other restrictions and ideas to improve the blinkings classification?
Or any other suggestion/idea for my project to further optimize/improve it?

Yes there are many I can think of. I use my own terms as I do not know the scientific onces

A blink of an eye has:

  1. frequency, how often per second
  2. attack, how fast does it close
  3. sustain, how long stays it closed
  4. decay, how fast does it open
  5. shift, does the lid closed as one or does left part closes before right or vice versa
  6. completeness, does the eye closes completely or is there a small opening
  7. sync, do both eyes blink simultaneously or short after each other
  1. single/both, is there a blink of one eye only
  1. movement, what does the eye do during the blink (down, up, left, right, rotate, other...)
  2. pupil, what does the pupil do during the blink (static or change, how and how much)
  3. facemuscles, are these relaxed or not (esp. nose-wings, upper lip)

Then there are the parameters of the environment of the eye

  • humidity,
  • temperature,
  • wind (force and direction)
  • light (intensity + color spectrum)

There are a lot of things you could measure around / during the eye blink. So which parameters would help to answer your questions.

Hope this helps,

robtillaart:
Yes there are many I can think of. I use my own terms as I do not know the scientific onces

A blink of an eye has:

  1. frequency, how often per second
  2. attack, how fast does it close
  3. sustain, how long stays it closed
  4. decay, how fast does it open
  5. shift, does the lid closed as one or does left part closes before right or vice versa
  6. completeness, does the eye closes completely or is there a small opening
  7. sync, do both eyes blink simultaneously or short after each other
  1. single/both, is there a blink of one eye only
  1. movement, what does the eye do during the blink (down, up, left, right, rotate, other...)
  2. pupil, what does the pupil do during the blink (static or change, how and how much)
  3. facemuscles, are these relaxed or not (esp. nose-wings, upper lip)

Then there are the parameters of the environment of the eye

  • humidity,
  • temperature,
  • wind (force and direction)
  • light (intensity + color spectrum)

There are a lot of things you could measure around / during the eye blink. So which parameters would help to answer your questions.

Hope this helps,

Wow, these are a lot of parameters thank you!
But with the myoware sensor I have, how any of these can be specifically applied?
I mean that:
1-6 : These can absolutely differ perhaps even substantially from blinking to blinking
7-8 : I study only one eye and suppose that the blinking occurs to both eyes at the same time
9- : With only the myoware sensor I cannot think of how I can measure such detailed and advanced quantities...

You need to analyze the raw data per sensor.

Check if you see certain signal form, a peak or a dip around the main signal