smoothing digital input

Hardware
XeThru X2M200 Respiration Sensor
Arduino Zero
Servo Motor

I am currently working on a project that allows me to translate my respiration rate to control a servo. The servo will eventually be integrated into a larger project that controls air flow within an inflatable object, allowing anyone to make the object breath with them. I have run into a problem and am wondering if anyone could assist me in thinking through how to smooth the data I am receiving so that the servo responds as accurately as possible(real time is the goal) to my breathing. I have been able to get the servo to rotate according to whether I am inhaling or exhaling, now I want it to basically track this so that the + and - rotation is incremental based on the data the Arduino is receiving from the board. Long slow breaths would produce different results than short breaths and I am trying to have the servo respond in real time to these variations. I have mapped the range but I need to now smooth the data so that the response is much less erratic. I need to control both speed and degree of rotation to correspond to the breathing rate. I have messed around with the smoothing example that is in the 03 analog sketch and tried to substitute variables:

//analog sensor location
int inputPin =A0;

becomes

// equals data.movement reading from the XeThru 
float val;




and then change the call later in the sketch:

 // read from the sensor:
 readings[readIndex] = analogRead(inputPin);

to


 // read from the data.movement variable:
 readings[readIndex] = val;

When I do this it compiles and loads but the servo stops responding to movement and no longer detects inhalation

Here is the smoothing sketch for reference:

void setup() {
  // initialize serial communication with computer:
  Serial.begin(9600);
  // initialize all the readings to 0:
  for (int thisReading = 0; thisReading < numReadings; thisReading++) {
    readings[thisReading] = 0;
  }
}

void loop() {
  // subtract the last reading:
  total = total - readings[readIndex];
  // read from the sensor:
  readings[readIndex] = analogRead(inputPin);
  // add the reading to the total:
  total = total + readings[readIndex];
  // advance to the next position in the array:
  readIndex = readIndex + 1;

  // if we're at the end of the array...
  if (readIndex >= numReadings) {
    // ...wrap around to the beginning:
    readIndex = 0;
  }

  // calculate the average:
  average = total / numReadings;
  // send it to the computer as ASCII digits
  Serial.println(average);
  delay(1);        // delay in between reads for stability
}

This is the most current servo sketch without an adapted smoothing sketch. Additional coding is to provide the ability to visually understand what state the sensor and servo are in.

// Servo - Version: Latest 
// Written by Øyvind Nydal Dahl
// www.build-electronic-circuits.com
// May 2017
//modified by Michael Casselli
//July 2017
//

#include <XeThruRadar.h>

#include <Servo.h>

// create servo object to control a servo
Servo myservo; 

XeThruRadar radar;
            
// variable to read the value data.movement
float val;

// Serial port for debugging (Change to match the serial port on your Arduino)
#define SerialDebug SerialUSB


//LED pins
const int red_pin = 8;
const int green_pin = 10;
const int blue_pin = 12;



void setup() {
    
  pinMode(red_pin, OUTPUT);
  pinMode(green_pin, OUTPUT);
  pinMode(blue_pin, OUTPUT);
  
  // attaches the servo on pin 6 to the servo object
  myservo.attach(6);

  //Set LED to pink(?) while initializing radar
  setColor(255, 0, 255, 1.0);

  // Setup debug port in library (for developers)
  // Note: Do not use without making sure the same port is NOT used in this sketch
  //radar.enableDebug();

  // Setup debug port for this sketch
  SerialDebug.begin(115200);

  // I use a delay so that I have 5 seconds to connect the radar after programming
  delay(5000);

  // Setup radar
  radar.init();

  // Tell the radar to load the respiration app
  radar.load_respiration_app();

  // Set detection zone (0.5 - 1.2 gives radar frame 0.3 - 2.3)
  radar.setDetectionZone(0.5, 1.2);

  // Set low sensitivity.
  radar.setSensitivity(3);

  // Start the app (the radar will start sending a constant stream of measurements)
  radar.execute_app();
}


void loop() {

  // Get respiration data
  RespirationData data = radar.get_respiration_data();

  if (data.valid_data == true)
  {
    // Set brightness of LED if in breathing state
    if (data.state_code == radar._xts_val_resp_state_breathing) {
      //Movement is usually between -1 and 1, so move it to 0 to 1 instead:
      float brightness = data.movement + 1.0;
      brightness += 1.0;
      brightness = brightness / 5.0;

      // Set brightness of the blue LED
      setColor(0, 0, 255, brightness);
      
      // reads the value of data.movement (value between 0 and 1023)
      val = data.movement;
      
      // scale it to use it with the servo (value between 0 and 180)
      val = map(val, -5.5, 5.5 , 0, 180);
      
      // sets the servo position according to the scaled value
      myservo.write(val);
      
      // waits for the servo to get there
      SerialDebug.println(val);
      //delay(15);

      SerialDebug.println(data.movement);
    }
    else if (data.state_code == radar._xts_val_resp_state_initializing) {
      setColor(255, 0, 255, 1.0); // Set color to pink
      SerialDebug.println("State: Initializing");
    }
    else if (data.state_code == radar._xts_val_resp_state_movement) {
      setColor(255, 255, 0, 1.0); // Set color to yellow
      SerialDebug.println("Detects motion, but can not identify breath");
    }
    else if (data.state_code == radar._xts_val_resp_state_movement_tracking) {
      setColor(0, 255, 0, 1.0); // Set color to green
      SerialDebug.println("Detects motion, possible breathing");
    }
    else if (data.state_code == radar._xts_val_resp_state_no_movement) {
      setColor(255, 0, 0, 1.0); // Set color to red
      SerialDebug.println("No movement detected");
    }
    
    else {
      setColor(255, 255, 255, 1.0); // Set color to white
      //SerialDebug.println("Unknown state");
    }
  }
  else {
    setColor(0, 255, 255, 1.0); // Set color to cyan
    SerialDebug.println("Valid respiration data NOT received!");
  }
}


void setColor(int red, int green, int blue, float brightness)
{
  //Make sure brightness is between 0 and 1:
  if (brightness > 1.0)
    brightness = 1.0;
  else if (brightness < 0.0)
    brightness = 0.0;

  //Set the brightness of each color
  analogWrite(red_pin, 255 - red * brightness);
  analogWrite(green_pin, 255 - green * brightness);
  analogWrite(blue_pin, 255 - blue * brightness);
}

void blink_red() {
  while (1) {
    setColor(255, 0, 0, 1.0);
    delay(500);
    setColor(0, 0, 0, 0.0);
    delay(500);
  }
}

void blink_green() {
  while (1) {
    setColor(0, 255, 0, 1.0);
    delay(500);
    setColor(0, 0, 0, 0.0);
    delay(500);
  }
}

void blink_blue() {
  while (1) {
    setColor(0, 0, 255, 1.0);
    delay(500);
    setColor(0, 0, 0, 0.0);
    delay(500);
  }
}

Many Thanks to Oyvind Nydal Dahl for his work on this with me. I am pretty much a noob with this level at this level of programming but am trying to learn and willing to experiment. Any suggestions, tips, recommendations about how I can achieve a real time response that controls the servo will be much appreciated.
Thank you.

XeThruRadar.cpp (11.7 KB)

XeThruRadar.h (4.04 KB)

float_no_smoothing.ino (4.23 KB)

Digital filters - I don't have much experience in time series analysis but maybe I can help. One way I have used to smooth data in arduino is with an exponentially weighted moving average. This works by counting the most recent sensor reading as a percent of the new reading. The best way to read about it would be just to google exponentially weighted moving average - wikipedia has a good explanation. This method should filter out a good amount high frequency noise depending on the memory variable you choose (and is easy to implement), but the trade off is the responsiveness of your system.

Another more complicated way is to use a Kalman filter - I have never used it but I have read about this filter being used to smooth accelerometer data. It uses statistical confidence intervals to check the signal to noise ratio of the data - and makes changes if the signal is greater than the noise ratio (I think). Anyway I think there is a arduino library for this filter. This will probably be a harder implementation, but would probably give you better responsiveness than the EWMA.

Hope this helped!

Probably an application for a Bessel low pass filter - minimum phase distortion, limited time lag and memory.

What is the raw sample rate?

Please edit your post to add code tags ("</>") button, so it looks like

this.

How are you measuring your "breathing"? Can you post an example of the data you are trying to smooth? There is no useful information on the manufacturer's product page for that sensor.

Yes, its one of those websites with lots of style and no content, be prepared for disappointment is my advice.

Thanks for everyone's responses, I corrected my code so that it is contained within tags now. I am attaching some more information about the 2XM200 sensor from the manufacturer, let me know if it answers your questions. Enclosed are the serial protocols for the sensor. I will include the data sheet on a separate post immediately following this one as it is too large to be included here. I am currently using the sketch that I included and reading the data.movement to capture my respiration cycles. I also included additional information(library, etc.) in my earlier post.
Thanks again and I will look forward to your thoughts.
Michael

XeThruExplorer_protocol_doc.pdf (131 KB)

PDF is too large to attach here, follow this link to access

Also, this link to their webpage may be more helpful, I am able to sign in and out and this gives me access to documentation.

It looks like it has a high level mode reporting breathing rate, and a low-level mode
returning baseband quadrature samples from the radar direct.

How are you going to use that?

I don't understand how something they say is time-of-flight is even providing a quadrature
baseband channel at all...

You need to talk to one of their application engineers in the first instance. Be warned this
could be very complex stuff.

Thinking about it the stuff about time-of-flight is probably marketing bullshit and its a
chirped doppler radar with digital baseband. Continuous-wave radar - Wikipedia

I am using serial data via the tx and rx pins of the sensor to the tx and rx pins of the zero-I have data and I have seen it via both the serial monitor and the serial plotter by using the debug function in the sketch. I want to smooth the data I am receiving so that I have control of the servo that is as close to real time in reaction and speed as possible. It is the real time control that is most important to me so that the response of the serv o tracks the inhale and exhale cycles of the person who is in front of the sensors-can you help with ideas that pertain to smoothing of this data?

I want to smooth the data I am receiving so that I have control of the servo that is as close to real time in reaction and speed as possible.

Put up some examples of the data, and tell us how you want it to look. Be sure to explain the data rate and timing constraints.