Signal Filtering and Waveform Capturing

I have the following photodiode (this one) connected to an Arduino Uno, and I would like to be able to record at least the waveform of an incoming light signal whether it is off a simple blinking LED or off a PC monitor directly. For now I am running my tests on an LED.

so far I am able to record the following signal off a blinking LED (on/off) after few filtering like exponential filtering followed by moving average. However, I noticed that the waveform looks reasonable only for very low frequency like 5 Hz where the waveform looks like square (first figure) but not when the frequency starts to get higher like 30 Hz or higher where the waveform starts to be less regular and more messy (second figure).

Any advice or tips for how to filter the signal better and be able to capture the waveform more clearly without being limited by the frequency (It would be great if I can measure the waveform at a high frequency like up to 300 Hz. However for now I would like at least to be able to measure it at 60, 120 Hz if possible).

enter image description here

enter image description here

enter image description here


Arduino Code:


    //#include <TimerOne.h>
    #include <Arduino.h>
    
    const unsigned long ONESEC = 1000000; // microseconds
    const int size = 600; // number of samples in 1 second
    int sensorValue[size];
    int dly = ONESEC / size; // delay so to fill N samples during 1 complete second (1,000,000 microseconds)
    
    
    
    int maxValue = 0;
    int minValue = 1000000;
    // the following parameters are used for exponential filtering
    double Alpha = 0.3; // [0.0 - 1.0] weight parameter
    double filteredValue = 0.0;
    double prevValue = 0.0;
    double avgsum = 0.0;
    
    // the following parameters are used for moving average filtering
    int total = 0;
    int avgtotal = 0;
    const int s = 10; 
    int rounds = size / s;
    int tmparr[s];
    
    const int btnPin = 7;
    const int sensorPin = A0;  // sensor pin
    
    
    void setup() { 
      Serial.begin(57600);
      
      // init tmparr
      for(int i=0;i<s;i++){
        tmparr[i] = 0;
      }
    }
    
    
    void waitForButton(int btnPin){
      Serial.println("Waiting...");
      while(digitalRead(btnPin) != 1){
        continue;
        }
      Serial.println("Button has been pressed!");
    }
    
    void loop() {  // code that loops forever
    
      // waitForButton(btnPin);
      
      // for(int i=0;i< size;i++){
      //   // Read sensor value
      //   sensorValue[i] = analogRead(sensorPin);
      //   // wait between readings for N microseconds
      //   delayMicroseconds(dly);
      // }
      int i = 0;
      unsigned long curTime = micros();
      unsigned long prevTime, startTime;
      prevTime = curTime;
      startTime = curTime;
      while(curTime - startTime <= ONESEC){ // sofar the time between the current and the start time is less than 1 second, keep recording data
        if(curTime - prevTime < dly){ // sofar the current and the previous time stamp has a differene of less than N seconds, do not record anything
          curTime = micros();
          continue;
        }
        sensorValue[i++] = analogRead(sensorPin);
        
        if(i > size){
          break;
        }
        prevTime = curTime;
        curTime = micros();
      }
      
      ///////////////////////
      ///////////////////////
      // Data processing ////
      ///////////////////////
      ///////////////////////
    
      // filter the signal and find maximum/minimum and avg
      
      for(int i=0;i<size;i++){
        int v = sensorValue[i];
        filteredValue = (v * Alpha) + (prevValue * (1-Alpha)); // exponential filtering 
        sensorValue[i] = filteredValue; // re-writing the sensor data !! RAW DATA ARE BEING OVERWRITTEN !!
        avgsum = avgsum + filteredValue;
        prevValue = filteredValue;
    
        if(maxValue < filteredValue){
          maxValue = filteredValue;
        }
        if(minValue > filteredValue){
          minValue = filteredValue;
        }
      }
      avgsum = avgsum / size;
    
      // plotting and segregating the signal 
      int peaks = 0;
      int prevState = -1;
    
      // moving average filter
      for(int i=0;i<rounds;i++){
        for(int j=0;j<s;j++){
          int v = sensorValue[(i*s)+j];
          total = total - tmparr[j];
          total = total + v;
          avgtotal = total / s;
          tmparr[j] = v;
          Serial.print(avgtotal); // new filtered value
          Serial.print(",");
          Serial.print(avgsum);
          Serial.print(",");
          Serial.println(v); // filtered value
    
          if(avgtotal > avgsum + (avgsum*0.01)){
            // Serial.println(1);
            if(prevState != 1){
              peaks++;
              prevState = 1;
            }
          }
          else{
            // Serial.println(0);
            prevState = 0;
          }
        }
      }
      Serial.print("peaks:");
      Serial.println(peaks);
    
    
      ///////// process the cycles in the signal
      // we have N samples (size) every (dly) second
      
    
    }

To measure the frequency you need to measure signal at least twice on the period ( see Nyquist theorem). To capture the waveform, I think, you need a ten points on the period or more. Therefore, to capture the 300 Hz signal you need 3000 Hz measure procedure.
As I see in your code, you capture a 600 points in a second - so you can expect to obtain the waveform for signals up to 60Hz freq

yeah true, I understand Nyquist theorem as a theory, however I never worked with electronics before and I am just not sure if my current signal makes any sense or not and how to improve its readability.

also I am not sure if the components I have like the Arduino itself and the photodiode are enough to aim for such high measurements or what is the limit I can measure with these equipment.

if you or anyone may give their insights and ideas, it would be immensely helpful

As a theoretical matter, the Nyquist sampling frequency will allow capture of the waveform, not just the frequency.

The practical limit to the theory is it needs a signal free of any frequency content above the limit.

You can filter the signal in advance of sampling, and you can increase above the Nyquist frequency until your sampling is a satisfactory replica.

Obvsly, satisfactory is up to the smaller, and some things are going to be inherently harder to filter and sample.

a7

I don't think so.
Theoretically speaking, the Nyquist frequency is applicable only to signal with known waveform - namely a regular sine wave.
For a signal of unknown shape, the Nyquist frequency is not enough even to measure the frequency, not to mention the shape.

Read

and see

Simply stated, the Nyquist criterion requires that the sampling frequency be at least twice the highest frequency contained in the signal, or information about the signal will be lost . If the sampling frequency is less than twice the maximum analog signal frequency, a phenomenon known as aliasing will occur.

Of course there is nothing simple about it.

You mean like some Beatles song? How do you think CDs work? :wink:

a7

And what conclusion do you draw from this? That it is possible to determine the frequency of a signal even if it is higher than the theorem indicates? :slight_smile: - yes, sometimes this true, but you will not be able to distinguish the frequency N + x from the frequency N - x, where N is the frequency determined by the theorem.
I am a spectroscopist and dealing with all this for forty years :slight_smile:

I never said, nor does the paper I linked say, any such thing.

a7

And how is your diode connected to the arduino?

You can do accurate sampling up to 8KHz with an Uno
As for your electronics, the maximum frequency, will depend on your photodiode/transimpedance amplifier bandwidth.

just connecting the photodiode through a resistor and then directly into one of the analog pins (like A0) on the board, no other elements in between.

for the photodiode specs I have included in my text the link to that, here >> https://www.thorlabs.com/thorproduct.cfm?partnumber=FDS010

and I think it is good enough to run accurate measurements but I do not seem to be able or to know how to filter the incoming signal correctly so to get a more proper waveform than the one I am getting as it shows in the above figure for 30Hz for example, which looks very frantic

This one looked much better.
What is your transimpedance amp circuit?

I do not have any, my circuit is as simple as just a transistor and a photodiode, no other elements. Hence why I am asking, because I have 0 knowledge in circuit I suspect that I am missing some components that should be on the circuit board even before doing the measurement.

I thought things could be just plug-n-play!
please forgive my ignorance but I am trying to learn and figure out what I may need.

here is how the circuit looks like, very primitive and simple.

The red button is only to trigger measurements

I don't understand.
That is a very expensive photodiode with a very specific responsivity. You must have chosen it for a particular reason, otherwise, a $0.50 phototransistor would have worked

1 Like

well, the back story is that we want to register the display pixel on/off waveform at a certain frequency, and for that we have chosen this photodiode that has a reseponse time of 1ns believing that it is fast enough to capture and change in signal within that timeframe.

second, I thought it would be easy to buy this sensor and an Arduino kit so to have a little measuring device/sensor that is easy to use by just plugging the components and doing some programming, so that at the end the sensor can be simply fixed on a display or in front any blinking light source and then it will plot the signal and the frequency out.

PS: I am a computer scientist, and have very little knowledge in electronics but good enough to do signal processing once I have my signal correct.

It would be helpful if you started by simplifying your plot to show the data as captured by the sensor (raw data) and from some subsequent signal processing (e.g. the moving average). Then we could determine whether your issue is on the data collection side (as Jim-P is poking at) or on the subsequent signal processing side of things.

Please post a schematic diagram, with component values and types.

Those long wires are very good antennas, and will pick up all sorts of interesting signals from the overhead lights, nearby AC wiring, etc.

For this experiment you need a transimpedance amplifier, as mentioned above. They are not complicated, but for fast signals you need a fast op amp, and the circuitry must be shielded from electrical noise.

That waveform could contain frequencies in the kilohertz range depending on the frequency.
So you should sample at least 5 times that rate but you are only sampling a 600Hz
At one point you said you just wanted to know the on/off frequency, that would be much simpler