Automatic threshold setting for peak hold indicator

Lagom · November 23, 2023, 10:50am

I'm using an Adafruit HUZZAH ESP8266 with an ADXL345 accelerometer, powered from a 2,500mAh battery. I want a visual indication when the aforementioned set-up is kicked or thrown. An LED lights up for 1.5s after a peak from a kick or throw is detected. It's a bit like the "peak hold VU-meter" effect I had on my tape deck in the 70s ; )

I observed the serial plotter for a long time with many kinds of kicks or throws, and decided on 13 as a suitable threshold value. My question is if there is a way to automatically set the threshold instead of me using a "magic number" purely from observation, and then storing it on an EPROM IC?

#include <Wire.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_ADXL345_U.h>
#include <RunningMedian.h>

Adafruit_ADXL345_Unified accel = Adafruit_ADXL345_Unified(12345);

RunningMedian amSamples = RunningMedian(21);

float ax = 0;
float ay = 0;
float az = 0;

float am = 0;

float amEMA = 9.81;
float alphaEMA = 0.1;

float amPeak = 0;
float amThreshold = 13; // Magic number, selected after many kick and throw observations in the serial plotter

int timeLEDOn = 1500; // Peak hold duration

void setup()
{
  Serial.begin(115200);

  pinMode(0, OUTPUT); // With Adafruit HUZZAH ESP8266 LOW = HIGH and HIGH = LOW
  digitalWrite(0, HIGH); // On-board LED off at start

  accel.begin();

  accel.setRange(ADXL345_RANGE_2_G);
}

void loop()
{
  static unsigned long timeNow = 0;

  // Sensor data acquisition and filtering code
  sensors_event_t event;
  accel.getEvent(&event);

  ax = event.acceleration.x;
  ay = event.acceleration.y;
  az = event.acceleration.z;

  am = sqrt(ax * ax + ay * ay + az * az);

  amEMA = (alphaEMA * am) + ((1 - alphaEMA) * amEMA);

  amSamples.add(am);
  float amRM = amSamples.getMedian();

  Serial.print(am); // Blue curve
  Serial.print(" ");
  Serial.print(amEMA); // Red curve
  Serial.print(" ");
  Serial.println(amRM); // Green curve

  // Peak detection and hold code
  if (amEMA > amPeak)
  {
    amPeak = amEMA;
  }

  if (amEMA < amThreshold)
  {

    if (amPeak > amThreshold)
    {
      digitalWrite(0, LOW); // LED on

      timeNow = millis();

      amPeak = 0;
    }

    if (millis() - timeNow >= timeLEDOn)
      digitalWrite(0, HIGH); // LED off
  }
}

UKHeliBob · November 23, 2023, 4:30pm

What was the basis of using this value ?
Does it have any significance in relation to the maximum and minimum values, for instance ?

Lagom · November 23, 2023, 5:52pm

I selected it as suitable threshold value after observing many kicks and throws (long lightweight USB cable for now, set-up in a foam ball Molten Rugby softball SOFT-AF, yellow 170g • Pris ») in the serial plotter. People with less energetic kicks and throws may not trigger, the threshold would have to be lower, meaning I would have to open the ball to connect a USB cable and upload the code with a new threshold. I'd like to avoid that. Maybe I have to have people interact with the thing for two or three minutes during setup() to find out the lows and highs, then set a threshold based on that?

UKHeliBob · November 23, 2023, 6:27pm

So it is not so much a case of automatically setting the threshold, more a case of doing it remotely

How about running a web server on the ESP8266 an controlling it via a browser ?

Lagom · November 23, 2023, 6:31pm

Yeah, that's better than connecting a cable; I did not think of that one. So, do you think automating the threshold-setting prior to loop() is too difficult, or even impossible?

UKHeliBob · November 23, 2023, 6:34pm

The difficult part of automating the threshold is defining what the the method will be

Maybe take a series of initial readings and use the average as the threshold for the actual tests

Lagom · November 23, 2023, 6:44pm

Thanks, I will try that.

Lagom · November 23, 2023, 8:54pm

Or, maybe I calculate a second exponential moving average with alphaEMA/2, and that becomes my constantly adapting threshold value?

gcjr · November 23, 2023, 9:14pm

we had to automatically set thresholds in speakerphones to recognize when speech is present.

we use 2 leaky integrators (avg += (samp - avg) / N) with different attack and decay values.

the speech-avg has a short attack value (N = 8) and decay (N = 32) and the noise avg has a long attack value (N = 1024) and short decay (N = 8).

the speech-avg should capture the max values and the noise-avg the minimum. speech is recognized when the speech-avg / noise-avg > 6dB.

the noise avg should be initialized to a high value and the speech-avg to a small value.

Lagom · November 23, 2023, 9:40pm

Thanks, but that description is well over my head, I must admit.

Found out that my idea was silly : )

gcjr · November 23, 2023, 9:57pm

can you post your data values?

Lagom · November 23, 2023, 10:08pm

Sure, there's a serial plotter screenshot in the 1st post. I'd need to find a way how to record the values on my MacBook. Do you mean attaching an Apple Numbers spreadsheet with the values here?

gcjr · November 23, 2023, 10:13pm

what happens if instead of displaying the output as a plot you just display the values as text to the serial monitor and then copy&paste?

Lagom · November 23, 2023, 10:18pm

All right, it'll really bloat the length of this post though - am/amEMA/amRM values, like in the screenshot. The threshold is hard-coded with 15. I am not using amRM for peak detection though, still amEMA.

9.16 9.18 9.20
9.24 9.19 9.20
9.13 9.18 9.20
9.16 9.18 9.20
9.20 9.18 9.20
9.16 9.18 9.20
9.16 9.18 9.20
9.11 9.17 9.20
9.20 9.17 9.20
9.16 9.17 9.16
9.16 9.17 9.16
9.09 9.16 9.16
9.04 9.15 9.16
9.16 9.15 9.16
9.20 9.16 9.16
9.20 9.16 9.16
9.24 9.17 9.16
9.21 9.17 9.16
9.20 9.17 9.16
9.16 9.17 9.16
9.24 9.18 9.16
9.20 9.18 9.16
9.20 9.18 9.16
9.06 9.17 9.16
9.16 9.17 9.16
9.20 9.17 9.16
9.16 9.17 9.16
9.12 9.17 9.16
9.08 9.16 9.16
9.20 9.16 9.16
9.16 9.16 9.16
9.24 9.17 9.20
9.28 9.18 9.20
9.24 9.19 9.20
9.16 9.18 9.20
9.24 9.19 9.20
9.24 9.19 9.20
9.20 9.19 9.20
9.12 9.19 9.20
9.09 9.18 9.20
9.16 9.18 9.20
9.16 9.17 9.16
9.16 9.17 9.16
9.12 9.17 9.16
9.28 9.18 9.16
9.20 9.18 9.16
9.12 9.17 9.16
9.20 9.18 9.16
9.16 9.18 9.16
9.04 9.16 9.16
9.24 9.17 9.16
9.20 9.17 9.20
9.20 9.17 9.20
9.12 9.17 9.16
9.12 9.16 9.16
9.20 9.17 9.16
9.20 9.17 9.16
9.24 9.18 9.16
9.24 9.18 9.16
9.12 9.18 9.16
9.20 9.18 9.20
9.20 9.18 9.20
9.20 9.18 9.20
9.20 9.19 9.20
9.27 9.19 9.20
9.13 9.19 9.20
9.27 9.20 9.20
9.16 9.19 9.20
9.16 9.19 9.20
9.12 9.18 9.20
9.24 9.19 9.20
9.20 9.19 9.20
9.20 9.19 9.20
9.13 9.18 9.20
9.16 9.18 9.20
9.08 9.17 9.20
9.20 9.17 9.20
9.20 9.18 9.20
9.16 9.18 9.20
9.12 9.17 9.20
9.16 9.17 9.20
9.20 9.17 9.20
9.20 9.18 9.20
9.20 9.18 9.20
9.16 9.18 9.16
9.16 9.17 9.16
9.16 9.17 9.16
9.23 9.18 9.16
9.19 9.18 9.16
9.28 9.19 9.19
9.08 9.18 9.19
9.42 9.20 9.19
9.42 9.22 9.19
10.40 9.34 9.19
9.94 9.40 9.20
9.48 9.41 9.20
14.94 9.96 9.20
21.62 11.13 9.20
27.08 12.72 9.20
26.30 14.08 9.23
23.00 14.97 9.28
21.09 15.59 9.42
22.65 16.29 9.42
15.13 16.18 9.48
8.57 15.42 9.48
7.66 14.64 9.48
10.80 14.26 9.94
18.43 14.67 10.40
20.33 15.24 10.80
17.56 15.47 14.94
14.52 15.38 14.94
13.40 15.18 14.94
7.65 14.43 14.94
3.81 13.36 14.94
4.84 12.51 14.94
8.75 12.14 14.94
10.74 12.00 14.94
10.56 11.85 14.52
12.60 11.93 13.40
13.81 12.12 13.40
14.12 12.32 13.40
14.22 12.51 13.40
14.16 12.67 13.40
12.86 12.69 12.86
11.90 12.61 12.60
11.49 12.50 12.60
11.30 12.38 12.60
10.09 12.15 12.60
9.00 11.83 11.90
8.21 11.47 11.49
8.08 11.13 11.30
7.57 10.78 10.74
7.29 10.43 10.56
7.91 10.18 10.56
8.02 9.96 10.56
7.48 9.71 10.56
7.16 9.46 10.56
7.17 9.23 10.09
6.97 9.00 9.00
6.37 8.74 8.21
6.11 8.48 8.08
7.69 8.40 8.02
9.36 8.49 8.02
10.61 8.71 8.02
11.59 8.99 8.02
12.19 9.31 8.02
11.94 9.58 8.02
11.48 9.77 8.02
10.12 9.80 8.02
9.90 9.81 8.02
9.83 9.81 8.02
9.54 9.79 8.02
9.57 9.77 9.36
9.90 9.78 9.54
11.80 9.98 9.57
11.44 10.13 9.83
10.76 10.19 9.90
10.03 10.17 9.90
9.73 10.13 9.90
9.50 10.07 9.90
9.47 10.01 9.90
9.54 9.96 9.90
9.36 9.90 9.90
9.37 9.85 9.90
9.63 9.83 9.90
9.66 9.81 9.83
9.80 9.81 9.80
9.71 9.80 9.73
9.38 9.76 9.71
8.98 9.68 9.66
8.65 9.58 9.63
8.38 9.46 9.57
8.48 9.36 9.57
8.65 9.29 9.54
8.95 9.25 9.50
9.24 9.25 9.47
9.42 9.27 9.42
9.83 9.33 9.42
9.58 9.35 9.42
9.36 9.35 9.38
9.36 9.35 9.37
9.24 9.34 9.36
9.07 9.32 9.36
8.82 9.27 9.36
8.67 9.21 9.24
9.33 9.22 9.24
9.46 9.24 9.24
9.46 9.26 9.24
9.60 9.30 9.24
9.51 9.32 9.24
9.62 9.35 9.33
9.66 9.38 9.36
9.90 9.43 9.36
9.65 9.45 9.42
9.34 9.44 9.42
9.22 9.42 9.42
9.11 9.39 9.42
9.01 9.35 9.36
9.11 9.33 9.36
8.99 9.29 9.34
8.81 9.25 9.33
8.80 9.20 9.24
9.11 9.19 9.22
9.30 9.20 9.30
9.23 9.21 9.30
9.24 9.21 9.30
9.10 9.20 9.24
9.21 9.20 9.23
9.45 9.22 9.23
9.46 9.25 9.23
9.25 9.25 9.23
9.36 9.26 9.23
9.44 9.28 9.23
9.45 9.29 9.23
9.23 9.29 9.23
9.07 9.27 9.22
9.07 9.25 9.21
9.47 9.27 9.23
9.36 9.28 9.23
9.37 9.29 9.24
9.41 9.30 9.25
9.22 9.29 9.25
9.16 9.28 9.25
9.25 9.27 9.25
9.19 9.27 9.25
9.18 9.26 9.25
9.30 9.26 9.25
9.23 9.26 9.25
9.11 9.24 9.25
8.95 9.21 9.25
8.94 9.19 9.23
9.05 9.17 9.23
9.38 9.19 9.23
9.36 9.21 9.23
9.32 9.22 9.23
9.46 9.25 9.23
9.58 9.28 9.25
9.26 9.28 9.26
9.16 9.27 9.25
9.03 9.24 9.23
8.99 9.22 9.22
9.02 9.20 9.19
9.16 9.19 9.18
9.69 9.24 9.19
9.32 9.25 9.19
9.10 9.24 9.18
9.13 9.22 9.16
9.21 9.22 9.16
9.06 9.21 9.16
9.09 9.19 9.16
9.14 9.19 9.16
9.16 9.19 9.16
9.26 9.19 9.16
9.34 9.21 9.16
9.39 9.23 9.16
9.24 9.23 9.16
9.22 9.23 9.16
9.25 9.23 9.16
9.10 9.22 9.16
9.44 9.24 9.16
9.68 9.28 9.21
9.50 9.30 9.22
9.23 9.30 9.23
9.09 9.28 9.23
9.51 9.30 9.23
9.29 9.30 9.23
9.15 9.28 9.23
9.07 9.26 9.23
9.00 9.24 9.23
8.94 9.21 9.23
8.21 9.11 9.23
6.40 8.84 9.23
3.93 8.34 9.23
2.93 7.80 9.22
9.82 8.01 9.22
15.31 8.74 9.22
18.36 9.70 9.22
16.37 10.37 9.23
18.52 11.18 9.23
22.02 12.27 9.29
26.11 13.65 9.29
24.01 14.69 9.29
20.89 15.31 9.29
20.06 15.78 9.51
12.44 15.45 9.82
6.45 14.55 9.82
6.96 13.79 9.82
6.70 13.08 9.82
7.28 12.50 9.82
9.22 12.17 9.82
10.62 12.02 10.62
11.85 12.00 11.85
10.81 11.88 11.85
8.34 11.53 11.85
6.40 11.01 11.85
7.59 10.67 11.85
8.08 10.41 10.81
8.55 10.23 10.62
8.70 10.07 9.22
7.68 9.83 8.70
8.11 9.66 8.55
8.84 9.58 8.55
9.28 9.55 8.55
9.26 9.52 8.55
8.96 9.46 8.55
9.10 9.43 8.55
9.96 9.48 8.70
10.31 9.56 8.84
10.17 9.63 8.96
9.80 9.64 9.10
9.55 9.63 9.10
9.69 9.64 9.10
9.54 9.63 9.10
9.47 9.61 9.10
9.30 9.58 9.26
9.27 9.55 9.27
9.76 9.57 9.28
9.68 9.58 9.30
9.43 9.57 9.43
8.90 9.50 9.43
8.44 9.40 9.43
8.29 9.28 9.43
8.58 9.21 9.43
8.94 9.19 9.43
9.58 9.23 9.47
9.85 9.29 9.54
9.65 9.32 9.55
9.05 9.30 9.54
8.83 9.25 9.47
9.03 9.23 9.43
9.16 9.22 9.30
9.13 9.21 9.27
8.72 9.16 9.16
9.24 9.17 9.16
9.12 9.17 9.13
9.08 9.16 9.12
9.08 9.15 9.08
9.14 9.15 9.08
9.24 9.16 9.08
9.24 9.17 9.08
9.29 9.18 9.12
9.51 9.21 9.13
9.50 9.24 9.14
9.03 9.22 9.14
9.14 9.21 9.14
9.10 9.20 9.14
9.23 9.20 9.14
9.08 9.19 9.13
8.92 9.16 9.13
9.19 9.17 9.14
9.06 9.16 9.14
9.07 9.15 9.13
9.04 9.14 9.12
9.19 9.14 9.14
9.38 9.17 9.14
9.22 9.17 9.14
9.14 9.17 9.14
9.33 9.18 9.19
9.10 9.18 9.19
9.22 9.18 9.19
9.27 9.19 9.19
9.33 9.20 9.19
9.38 9.22 9.19
9.25 9.22 9.19
9.18 9.22 9.19
9.14 9.21 9.19
9.10 9.20 9.19
9.18 9.20 9.18
9.15 9.19 9.18
9.15 9.19 9.18
9.23 9.19 9.18
9.30 9.20 9.19
9.35 9.22 9.22
9.18 9.21 9.22
9.19 9.21 9.22
9.14 9.20 9.19
9.21 9.21 9.19
9.38 9.22 9.21
9.43 9.24 9.21
9.16 9.23 9.21
9.11 9.22 9.19
9.22 9.22 9.19
9.34 9.23 9.19
9.26 9.24 9.19
9.30 9.24 9.19
8.98 9.22 9.19
8.94 9.19 9.19
9.09 9.18 9.19
9.40 9.20 9.21
9.56 9.24 9.22
8.97 9.21 9.22
8.78 9.17 9.21
8.78 9.13 9.19
8.94 9.11 9.18
9.06 9.10 9.16
9.16 9.11 9.16
9.34 9.13 9.16
9.38 9.16 9.16
9.38 9.18 9.16
9.30 9.19 9.16
9.29 9.20 9.22
9.23 9.20 9.23
8.87 9.17 9.23
8.91 9.14 9.16
9.06 9.14 9.09
9.37 9.16 9.09
9.61 9.20 9.16
9.37 9.22 9.23
9.07 9.20 9.23
8.90 9.17 9.16
9.02 9.16 9.07
8.98 9.14 9.07
9.01 9.13 9.07
9.05 9.12 9.07
9.05 9.11 9.07
9.17 9.12 9.16
9.10 9.12 9.10
9.21 9.13 9.10
9.18 9.13 9.10
9.29 9.15 9.10
9.30 9.16 9.10
9.41 9.19 9.10
9.65 9.23 9.10
9.46 9.26 9.17
9.38 9.27 9.18
9.22 9.26 9.21
9.38 9.28 9.21
9.44 9.29 9.21
9.87 9.35 9.21
10.53 9.47 9.22
10.84 9.60 9.29
11.39 9.78 9.30
12.64 10.07 9.38
13.54 10.42 9.38
14.93 10.87 9.41
17.56 11.54 9.44
21.04 12.49 9.46
23.41 13.58 9.65
25.38 14.76 9.87
26.43 15.93 10.53
20.02 16.34 10.84
11.14 15.82 11.14
9.27 15.16 11.14
9.75 14.62 11.14
9.16 14.07 11.14
7.74 13.44 11.14
6.09 12.71 11.14
6.54 12.09 11.14
7.28 11.61 11.14
6.60 11.11 11.14
6.33 10.63 11.14
6.68 10.23 11.14
8.43 10.05 9.75
9.77 10.03 9.75
10.33 10.06 9.75
10.48 10.10 9.75
10.81 10.17 9.75
11.54 10.31 9.75
11.75 10.45 9.75
11.72 10.58 9.75
11.46 10.67 9.75
11.26 10.73 9.75
11.95 10.85 9.75
11.11 10.87 9.77
10.61 10.85 10.33
10.25 10.79 10.33
9.90 10.70 10.33
9.74 10.60 10.33
9.60 10.50 10.33
9.76 10.43 10.33
9.88 10.37 10.33
9.81 10.32 10.33
9.47 10.23 10.33
8.89 10.10 10.33
8.84 9.97 10.33
8.92 9.87 10.25
9.04 9.78 9.90
9.27 9.73 9.88
9.65 9.72 9.81
9.68 9.72 9.76
10.10 9.76 9.76
10.44 9.83 9.76
10.30 9.87 9.76
9.53 9.84 9.74
9.44 9.80 9.68
9.52 9.77 9.65
9.65 9.76 9.65
9.66 9.75 9.65
10.05 9.78 9.65
10.11 9.81 9.65
10.00 9.83 9.65
9.83 9.83 9.65
9.67 9.82 9.65
9.65 9.80 9.65
9.75 9.79 9.66
9.96 9.81 9.67
9.90 9.82 9.68
9.76 9.81 9.75
9.78 9.81 9.76
9.64 9.79 9.76
9.72 9.79 9.76
9.66 9.77 9.75
9.65 9.76 9.72
9.58 9.74 9.67
9.64 9.73 9.67
9.64 9.72 9.67
9.59 9.71 9.67
9.57 9.70 9.67
9.62 9.69 9.67
9.54 9.67 9.66
9.71 9.68 9.66
9.84 9.69 9.66
9.84 9.71 9.66
9.40 9.68 9.65
9.24 9.63 9.65
9.50 9.62 9.64
9.77 9.64 9.64
9.61 9.63 9.64
9.54 9.62 9.64
9.57 9.62 9.62
9.67 9.62 9.62
9.92 9.65 9.62
9.82 9.67 9.62
9.62 9.66 9.62
9.53 9.65 9.62
9.61 9.65 9.61
9.58 9.64 9.61
9.47 9.62 9.61
9.49 9.61 9.61
9.52 9.60 9.58
9.46 9.59 9.58
9.36 9.56 9.57
9.34 9.54 9.54
9.53 9.54 9.53
9.56 9.54 9.54
9.72 9.56 9.56
9.73 9.58 9.57
9.75 9.59 9.57
9.84 9.62 9.57
9.74 9.63 9.58
9.67 9.63 9.61
9.43 9.61 9.58
9.43 9.60 9.56
9.52 9.59 9.53
9.55 9.58 9.53
9.45 9.57 9.53
9.55 9.57 9.53
9.58 9.57 9.53
9.75 9.59 9.55
9.67 9.60 9.55
9.70 9.61 9.56
9.78 9.62 9.58
9.49 9.61 9.58
9.54 9.60 9.58
9.63 9.61 9.63
9.67 9.61 9.67
9.58 9.61 9.63
9.47 9.59 9.58
9.32 9.57 9.58
9.32 9.54 9.55
9.38 9.53 9.55
9.62 9.54 9.55
9.65 9.55 9.55
9.43 9.54 9.55
9.47 9.53 9.55
9.45 9.52 9.55
9.57 9.53 9.57
9.60 9.53 9.58
9.67 9.55 9.58
9.61 9.55 9.58
9.54 9.55 9.57
9.55 9.55 9.55
9.57 9.55 9.55
9.57 9.56 9.57
9.58 9.56 9.57
9.63 9.57 9.57
9.51 9.56 9.57
9.63 9.57 9.57
9.63 9.57 9.57
9.78 9.59 9.57
9.67 9.60 9.58
9.82 9.62 9.60
9.71 9.63 9.60
9.60 9.63 9.60
9.51 9.62 9.60
9.56 9.61 9.60
9.52 9.60 9.60
9.34 9.58 9.60
9.12 9.53 9.58
8.91 9.47 9.57
8.56 9.38 9.57
8.49 9.29 9.57
8.36 9.20 9.57
8.06 9.08 9.56
7.21 8.89 9.52
6.40 8.65 9.51
5.87 8.37 9.51
6.20 8.15 9.34
7.07 8.04 9.12
8.78 8.12 8.91
9.36 8.24 8.91
8.31 8.25 8.78
7.89 8.21 8.56
8.18 8.21 8.49
8.81 8.27 8.49
9.47 8.39 8.49
9.72 8.52 8.49
10.58 8.73 8.49
11.49 9.01 8.49
11.75 9.28 8.49
11.48 9.50 8.49
10.72 9.62 8.49
10.30 9.69 8.78
10.05 9.73 8.81
9.98 9.75 9.36
10.10 9.79 9.47
10.98 9.91 9.72
11.12 10.03 9.98
10.89 10.11 10.05
10.63 10.17 10.10
10.49 10.20 10.30
10.29 10.21 10.30
10.22 10.21 10.30
10.45 10.23 10.45
10.64 10.27 10.49
10.69 10.32 10.58
10.15 10.30 10.58
10.01 10.27 10.58
9.51 10.19 10.49
9.22 10.10 10.45
9.09 10.00 10.30
9.02 9.90 10.29
9.09 9.82 10.22
9.06 9.74 10.15
9.05 9.67 10.15
9.21 9.63 10.15
9.15 9.58 10.15
9.09 9.53 10.01
9.03 9.48 9.51
9.03 9.43 9.22
9.01 9.39 9.21
9.35 9.39 9.21
9.33 9.38 9.21
9.14 9.36 9.15
8.98 9.32 9.14
8.87 9.27 9.09
8.94 9.24 9.09
8.94 9.21 9.09
9.29 9.22 9.09
9.43 9.24 9.09
9.29 9.25 9.09
8.91 9.21 9.06
8.76 9.17 9.06
9.11 9.16 9.06
9.17 9.16 9.09
9.20 9.16 9.11
9.35 9.18 9.11
9.31 9.20 9.11
9.44 9.22 9.14
9.42 9.24 9.17
9.18 9.23 9.18
9.06 9.22 9.18
8.96 9.19 9.17
8.95 9.17 9.14
9.09 9.16 9.11
9.24 9.17 9.17
9.17 9.17 9.17
8.91 9.14 9.17
8.88 9.12 9.17
8.89 9.09 9.17
8.81 9.07 9.11
8.90 9.05 9.09
9.00 9.04 9.09
9.03 9.04 9.09
9.20 9.06 9.09
9.18 9.07 9.09
9.07 9.07 9.07
9.05 9.07 9.06
8.93 9.05 9.05
9.23 9.07 9.05
9.31 9.10 9.05
9.19 9.11 9.05
9.11 9.11 9.05
9.33 9.13 9.07
9.36 9.15 9.09
9.36 9.17 9.11
9.28 9.18 9.11
9.17 9.18 9.11
9.01 9.16 9.11
9.30 9.18 9.17
9.23 9.18 9.18
8.98 9.16 9.18
8.87 9.13 9.18
8.78 9.10 9.18
9.19 9.11 9.19
9.38 9.13 9.19
9.38 9.16 9.19
9.36 9.18 9.23
9.52 9.21 9.23
11.94 9.49 9.28
13.43 9.88 9.30
14.69 10.36 9.30
15.72 10.90 9.33
15.86 11.39 9.36
12.90 11.54 9.36
10.21 11.41 9.36
8.53 11.12 9.36
7.71 10.78 9.36
7.42 10.45 9.36
7.84 10.19 9.36
8.40 10.01 9.36
8.70 9.88 9.36
8.57 9.75 9.36
8.04 9.58 9.36
7.22 9.34 9.36
6.95 9.10 9.36
7.06 8.90 8.70
7.38 8.74 8.57
7.48 8.62 8.53
7.86 8.54 8.40
8.51 8.54 8.40
8.21 8.51 8.21
8.23 8.48 8.21
8.74 8.50 8.21
9.23 8.58 8.21
9.29 8.65 8.21
8.93 8.68 8.21
8.69 8.68 8.21
8.62 8.67 8.23
8.64 8.67 8.40
9.24 8.73 8.51
9.53 8.81 8.57
9.20 8.85 8.57
8.90 8.85 8.62
8.94 8.86 8.64
9.13 8.89 8.69
9.18 8.92 8.74
9.11 8.94 8.90
9.16 8.96 8.93
9.38 9.00 8.94
9.47 9.05 9.11
9.67 9.11 9.13
9.74 9.17 9.16
9.64 9.22 9.18
9.58 9.26 9.20
9.62 9.29 9.20
9.36 9.30 9.20
9.51 9.32 9.24
9.67 9.36 9.36
9.78 9.40 9.38
9.70 9.43 9.47
9.07 9.39 9.47
9.10 9.36 9.38
9.08 9.33 9.38
9.08 9.31 9.38
9.11 9.29 9.38
9.24 9.28 9.38
9.09 9.26 9.38
9.43 9.28 9.43
9.56 9.31 9.47
9.53 9.33 9.51
9.55 9.35 9.53
9.50 9.37 9.51
9.31 9.36 9.50
9.74 9.40 9.50
10.11 9.47 9.50
10.18 9.54 9.50
10.00 9.59 9.51
10.35 9.66 9.53
10.94 9.79 9.53
12.68 10.08 9.53
14.14 10.49 9.53
13.77 10.81 9.55
11.48 10.88 9.56
10.01 10.79 9.74
9.34 10.65 9.74
6.56 10.24 9.74
3.24 9.54 9.74
20.30 10.62 10.00
22.13 11.77 10.01
20.18 12.61 10.11
11.56 12.50 10.18
16.72 12.93 10.35
26.68 14.30 10.94
26.72 15.54 11.48
23.08 16.30 11.56
20.11 16.68 12.68
15.78 16.59 13.77
6.02 15.53 13.77
7.26 14.70 13.77
8.74 14.11 13.77
8.42 13.54 13.77
7.66 12.95 11.56
8.97 12.55 11.48
9.82 12.28 10.01
8.64 11.92 9.82
8.80 11.60 9.82
8.64 11.31 9.82
6.92 10.87 9.82
8.46 10.63 8.97
9.42 10.51 8.97
10.39 10.50 8.97
10.33 10.48 8.97
10.07 10.44 8.97
10.08 10.40 8.97
10.05 10.37 8.97
10.53 10.38 8.97
10.63 10.41 8.97
10.75 10.44 8.97
10.15 10.41 9.42
10.26 10.40 9.82
10.21 10.38 10.05
9.95 10.34 10.05
9.62 10.26 10.05
8.94 10.13 10.05
8.06 9.92 10.05
7.19 9.65 10.05
6.83 9.37 10.05
7.17 9.15 10.05
7.62 9.00 10.05
8.22 8.92 10.05
8.63 8.89 10.05
8.99 8.90 9.95
9.62 8.97 9.62
10.19 9.09 9.62
10.51 9.24 9.62
9.99 9.31 9.62
9.20 9.30 9.62
8.44 9.21 9.20
8.02 9.09 8.99
7.75 8.96 8.94
8.32 8.90 8.63
8.99 8.90 8.63
9.43 8.96 8.63
20.76 10.14 8.63
10.32 10.15 8.63
8.89 10.03 8.89
9.03 9.93 8.99
9.14 9.85 8.99
9.13 9.78 9.03
9.14 9.71 9.13
9.09 9.65 9.13
9.43 9.63 9.14
9.22 9.59 9.14
9.00 9.53 9.14
9.07 9.48 9.13
8.92 9.43 9.09
12.95 9.78 9.09
9.79 9.78 9.09
9.27 9.73 9.13
9.17 9.67 9.14
9.09 9.61 9.14
15.91 10.24 9.14
8.96 10.12 9.14
8.75 9.98 9.14
8.87 9.87 9.13
9.31 9.81 9.13
8.76 9.71 9.13
9.16 9.65 9.14
9.12 9.60 9.13
9.16 9.55 9.14
9.16 9.52 9.16
9.13 9.48 9.16
9.25 9.45 9.16
9.12 9.42 9.13
9.20 9.40 9.16
9.16 9.38 9.16
9.16 9.35 9.16
9.20 9.34 9.16
9.24 9.33 9.16
9.16 9.31 9.16
9.16 9.30 9.16
9.16 9.28 9.16
9.28 9.28 9.16
9.16 9.27 9.16
9.13 9.26 9.16
9.12 9.24 9.16
9.24 9.24 9.16
9.01 9.22 9.16
9.24 9.22 9.16
9.12 9.21 9.16
9.16 9.21 9.16
9.13 9.20 9.16
9.17 9.20 9.16
9.20 9.20 9.16
9.12 9.19 9.16
9.16 9.19 9.16
9.20 9.19 9.16
9.20 9.19 9.16
9.17 9.19 9.16
9.20 9.19 9.16
9.24 9.19 9.16
9.16 9.19 9.16
9.13 9.18 9.16
9.01 9.17 9.16
9.16 9.17 9.16
9.24 9.17 9.16
9.16 9.17 9.16
9.20 9.18 9.16
9.13 9.17 9.16
9.16 9.17 9.16
9.16 9.17 9.16
9.20 9.17 9.16
9.08 9.16 9.16
9.16 9.16 9.16
9.08 9.15 9.16
9.20 9.16 9.16
9.20 9.16 9.16
9.05 9.15 9.16
9.16 9.15 9.16
9.20 9.16 9.16
9.16 9.16 9.16
9.16 9.16 9.16
9.16 9.16 9.16
9.12 9.15 9.16
9.37 9.18 9.16
9.20 9.18 9.16
9.12 9.17 9.16
9.16 9.17 9.16
9.13 9.17 9.16
9.13 9.16 9.16
9.16 9.16 9.16
9.20 9.17 9.16
9.13 9.16 9.16
9.20 9.17 9.16
9.01 9.15 9.16
9.17 9.15 9.16
9.20 9.16 9.16
9.16 9.16 9.16
9.24 9.17 9.16
9.21 9.17 9.16
9.08 9.16 9.16
9.20 9.17 9.16
9.16 9.17 9.16
9.16 9.16 9.16
9.28 9.18 9.16
9.13 9.17 9.16
9.16 9.17 9.16
9.16 9.17 9.16
9.20 9.17 9.16
9.24 9.18 9.16
9.36 9.20 9.17
9.16 9.19 9.17
9.20 9.19 9.17
9.16 9.19 9.17
9.13 9.19 9.16
8.98 9.16 9.16
9.20 9.17 9.16
9.12 9.16 9.16
9.20 9.17 9.16
9.05 9.16 9.16
9.17 9.16 9.16
9.28 9.17 9.16
9.20 9.17 9.16
9.20 9.17 9.17
9.12 9.17 9.17
9.04 9.16 9.16
9.16 9.16 9.16
9.20 9.16 9.17
9.20 9.16 9.20
9.20 9.17 9.20
9.00 9.15 9.17
9.20 9.16 9.17
9.16 9.16 9.17
9.12 9.15 9.16
9.20 9.16 9.17
9.20 9.16 9.20
9.20 9.17 9.20
9.20 9.17 9.20
9.12 9.16 9.20
9.12 9.16 9.20
9.08 9.15 9.20
9.21 9.16 9.20
9.20 9.16 9.20
9.20 9.17 9.20
9.12 9.16 9.20
9.24 9.17 9.20
9.16 9.17 9.20
9.20 9.17 9.20
9.24 9.18 9.20
9.20 9.18 9.20
9.16 9.18 9.20
9.24 9.18 9.20
9.28 9.19 9.20
9.24 9.20 9.20
9.20 9.20 9.20
9.20 9.20 9.20
9.12 9.19 9.20
9.20 9.19 9.20
9.24 9.20 9.20
9.20 9.20 9.20
9.16 9.19 9.20
9.09 9.18 9.20
9.20 9.18 9.20
9.20 9.19 9.20
9.20 9.19 9.20
9.16 9.18 9.20
9.20 9.19 9.20
9.01 9.17 9.20
9.24 9.18 9.20
9.16 9.17 9.20
9.16 9.17 9.20
9.09 9.16 9.20
9.09 9.16 9.20
9.09 9.15 9.20
9.16 9.15 9.16
9.20 9.16 9.16
9.20 9.16 9.16
9.12 9.16 9.16
9.20 9.16 9.16
9.24 9.17 9.16
9.16 9.17 9.16
9.16 9.17 9.16
9.12 9.16 9.16
9.09 9.15 9.16
9.20 9.16 9.16
9.16 9.16 9.16
9.24 9.17 9.16
9.12 9.16 9.16
9.08 9.15 9.16
9.24 9.16 9.16
9.16 9.16 9.16
9.20 9.17 9.16
9.16 9.17 9.16
9.04 9.15 9.16
9.12 9.15 9.16
9.20 9.15 9.16
9.20 9.16 9.16
9.16 9.16 9.16
9.16 9.16 9.16
9.24 9.17 9.16
9.24 9.17 9.16
9.20 9.18 9.16
9.24 9.18 9.16
9.16 9.18 9.16
9.19 9.18 9.19
9.20 9.18 9.19
9.20 9.19 9.20
9.20 9.19 9.20
9.08 9.18 9.20
9.13 9.17 9.20
9.13 9.17 9.19
9.16 9.17 9.19
9.16 9.17 9.16
9.12 9.16 9.16
9.08 9.15 9.16

gcjr · November 23, 2023, 10:43pm

yellow is short term transients attack/decay 4/16
red is background (noise) attack/decay 512/4
green is short-term / noise > 3

awk script

awk '
function disp (vec, N, col) {
    printf "# disp:\n"
    printf "color=%s\nnext\n", col
    for (n = 0; n < N; n++)
        printf " %6d %6.3f\n", n, vec [n]
}

function det (out, sp, noise, N, thresh) {
    printf "# det: %d\n", N
    for (n = 0; n < N; n++)
        out [n] = 5 * ((sp [n] / noise [n]) > thresh)
}

function filter (out, inp, N, attack, decay) {
    for (n = 0; n < N; n++) {
        if (out [n-1] < inp [n])
            out [n] = out [n-1] + (inp [n] - out [n-1]) / attack
        else
            out [n] = out [n-1] + (inp [n] - out [n-1]) / decay
    }
}

{
    x [NR] = $1
}

END {
    Pi     = atan2(0, -1)
    N      = NR
    Thresh = 3

    printf "thickness = 1.5\n"
    printf "title_x transients (yellow), noise (red), threshold (green)\n"

    x [0] = x [1]
    disp(x, N, "cyan")

    filter(sp, x, N, 4, 16)
    disp(sp, N, "yellow")

    noise [-1] = 10
    filter(noise, x, N, 512, 4)
    disp(noise, N, "red")

    det(out, sp, noise, N, Thresh)
    disp(out, N, "green")
}'  $* | tee det.xgr

gcjr · November 23, 2023, 10:44pm

negative peaks affect noise values. could process the abs() of the data.

Lagom · November 24, 2023, 11:39pm

Thanks for your suggestion, but I must admit I understand none of the above, and how to adapt it in my code for threshold-setting.

gcjr · November 24, 2023, 11:53pm

wow
"none of the above"
i wouldn't know where to start explaining

Lagom · November 25, 2023, 2:19am

Well...

What does "attack/decay 4/16" mean?
What is "awk ' "?
What does "$* | tee det.xgr" do?
Where does "thickness" enter the picture?
What does this "{ x [NR] = $1}" do?

How do I even approach this to integrate your code suggestion into my program? I read this book "Beginning C for Arduino, Second Edition" and that book "Beginning Arduino Programming: Writing Code for the Most Popular Microcontroller Board in the World". So, how would you suggest I go about understanding "awk script" to calculate the threshold?

gcjr · November 25, 2023, 11:09am

i didn't know you background or willingness to even try to use these techniques.

start with the plot

there's your data in cyan, a short term average in yellow, longer term background noise level in red and peaks in green

these averages use leaky integration

avg += (samp - avg) / N

but different values of N can be used depending on whether same > avg or < avg. an "attack" value is used when samp > avg and a decay value in the other case. the smaller the N value, the greater the change.

the short term average curve (yellow) in the plot uses attack/decay values of 4/16 which causes it to rise quickly to a peak and fall a bit more slowly. the noise average (red) uses attack/decay values of 512/4 which mean it rises very slowly but falls quickly to to measure the background signal level.

(the averages operate on the raw data and it would be better if these operated on the absolute value of the data).

do hopefully you see the purpose of the the 2 averages. the green curve simply marks when the yellow > green by a factor of 3

awk is a simple scripting language (perhaps the oldest). i used it to read you data file into an array and post process it as a set of arrays.

after reading your data, the END block of the script outputs commands for xgraph to plot the data. it uses disp() to output the various arrays of the data.

it uses filter() to compute the averages and det() to compare the averages to generate the out array plotted as green.

the output of the script it captured in an output files used by xgraph to generate the plot

i assume you would want to understand filter() to update the averages at runtime and make a comparison to recognize a peak, using your criteria for the attack/decay rates and peak threshold. run-time code wouldn't need to use (allocate) arrays

Topic		Replies	Views
How to detect a peak and low in noisy data.. General Guidance	26	3911	May 24, 2021
What filter / algorithm to use ? Programming	87	2611	October 19, 2023
Peak detector General Guidance	37	18516	May 5, 2021
Gettin low pass filter into arduino sketch Programming	28	8795	September 20, 2023
AlkoholSensor / Peak Deutsch	23	2824	May 6, 2021

Automatic threshold setting for peak hold indicator

Related topics