Sorting data in real time

Hi everyone, I'm trying to sort data coming in real time from my sensor with the library <ArduinoSort.h> but I'm having issues.

In my sketch for example, I want to sort every n values (600 in this case) and just display the last value (n-1), the biggest value as they are sorted in ascending order. The data are coming in real time and I just display the biggest value of every n values.

#define pinn 34
#include <ArduinoSort.h>
int* myArray;
int piezo = 0;
int n = 600;

void setup() 
{
  Serial.begin(115200);
}

void loop() 
{
  for(int i = 0; i < n; i++)
  {
    piezo = analogRead(pinn);
    myArray[i] = piezo;
    sortArray(myArray, n);
    printArray("sortArray:", myArray);
    int biggest = myArray[n-1];
    Serial.print("Biggest: ");
    Serial.println(biggest);
  }
}

void printArray(String msg, int* myArray) 
{
  Serial.println(msg);
  Serial.println(myArray[n-1]);
}

This is what I've on the Serial Monitor

Guru Meditation Error: Core  1 panic'ed (StoreProhibited). Exception was unhandled.
Core 1 register dump:
PC      : 0x400d0deb  PS      : 0x00060530  A0      : 0x800d21d8  A1      : 0x3ffb1f80  
A2      : 0x3ffc014c  A3      : 0x00000000  A4      : 0x3ffbfe5c  A5      : 0x3ffbdbb4  
A6      : 0x00000000  A7      : 0x00000000  A8      : 0x00000000  A9      : 0x3ffb1f50  
A10     : 0x00000000  A11     : 0x00000006  A12     : 0x00000006  A13     : 0x00000003  
A14     : 0x00000003  A15     : 0x00000000  SAR     : 0x0000001a  EXCCAUSE: 0x0000001d  
EXCVADDR: 0x00000000  LBEG    : 0x00000000  LEND    : 0x00000000  LCOUNT  : 0x00000000  

ELF file SHA256: 0000000000000000

Backtrace: 0x400d0deb:0x3ffb1f80 0x400d21d5:0x3ffb1fb0 0x400860ed:0x3ffb1fd0

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1216
ho 0 tail 12 room 4
load:0x40078000,len:10944
load:0x40080400,len:6388
entry 0x400806b4

Can someone please help me to handle it either with or without a library?

Thanks

Why are you re-sorting for every iteration?

Google "arduino sort array" for several other examples and tutorials.

Where is memory for your array allocated? This declares a pointer:
int* myArray;

And where will you find the -1 th element?
Or any of the elements, for that matter.

Declaring the array as a pointer then stuffing 600 int values in it which take at least 1200 bytes, maybe more depending on the board being used, does not sound like a very smart thing to do

Which board are you using ?

Why bother with the array when you can determine the highest of the 600 values as you read them ?

1 Like

why not just search for the max?

#define N   600
int myArray [N];

void loop()
{
    int max = myArray [0];

    for(int i = 0; i < N; i++)
    {
        if (max < myArray [i])
            max = myArray [i];
    }
    Serial.print("Biggest: ");
    Serial.println(max);
}

It seems you are running your code on a ESP32 MCU so maybe you could have advantage using a std::set or std:map or one of the other available STL "containers".

myArray is a pointer to some random place in memory, that is never initiaized, and NEVER points to ANY allocated memory! You are scribbling all over memory! You need to create the actual array, or use malloc() to allocate memory for the array.

1 Like

Did you miss @UKHeliBob #5?

You don't need to store or sort anything to do that.

As they come in, keep track of the highest one, and what number that one was if the number in the group of 600 is important.

int largestSeen = -32767; // initial smallest possible number
int position = -1;  // at no position

// later as they come in being counted

    if (newValue > largestSeen) }

        largestSeen = newValue;
        position = countOfItems;

    }


// when countOfItems is N, you have largestSeen, and where.
// reset and repeat 

If position stays at -1 it must mean all N values were -32767.

a7

You don't need to sort. But, it sounds like OP wants to know the maximum value in a sliding window of the most recent 600 samples. Thus, you do have to keep all 600. Every time a new sample enters the the window, one leaves the window. So, you need to check for a new maximum on every iteration.

1 Like

And "check" does not necessarily mean you have to scan all 600.

If the new value is higher than the previous max, you don't need to scan, but that's probably a rare occurrence.

Otherwise, if the value being discarded is lower than the previous max, you don't need to scan, and that's going to be case most of the time.

So you only need to scan when the value leaving is equal to the previous max, and the new value is lower than the previous max.

3 Likes

If that is true then the use of a for loop to read the values is seriously flawed

Indeed.

A bit of googling sez to me that this is a popular interview question.

No solution I have found has been described in the straightforward manner of @PaulRB in #11.

I go in and out of understanding and accepting the offered algorithm as valid and correct, which makes me very glad I am way past school's out forever.

I will however enjoy convincing myself somehow, at my own pace and with my own methods. :wink:

a7

I too had to read it a couple times. I'm currently on the "correct" side of the fence. Before that changes, an implementation:

#include "Arduino.h"
#include <array>
#include <algorithm>

const uint8_t pinn = 34;
const size_t arraySize = 600;

std::array<uint16_t, arraySize> sampleArray;

void setup() {
	sampleArray.fill(0);
	Serial.begin(115200);
	delay(2000);
}

void loop() {
	static auto position = sampleArray.begin();
	static uint16_t currentMax = 0;
	uint16_t newValue = analogRead(pinn);

	uint16_t leavingValue = *position;
	*position++ = newValue;
	if (position == sampleArray.end()) {
		position = sampleArray.begin();
	}

	if (newValue >= currentMax) {
		currentMax = newValue;
	} else if (leavingValue == currentMax) {
		auto maxPtr = std::max_element(sampleArray.begin(), sampleArray.end());
		currentMax = *maxPtr;
	}

	Serial.printf("Biggest: %d\n", currentMax);
}

Random comments:

  1. An array is a poor data structure to use for this (if indeed you want them all sorted.)
  2. Even with an array, adding a new value to a sorted list is much easier than re-sorting the entire array.
  3. Finding the maximum value in an array is much easier than sorting the entire array.
  4. Some sorting algorithms (notable quicksort, IIRC) do especially poorly if the input is already "mostly sorted."
  5. If you're wanting the max of the most recent N values (as per #10, #11), you have to keep the array in "as received" order (so that you know which elements to discard as new data comes in), so the original proposal to sort the array each time wouldn't work at all.

Finding the largest elements of a much larger set of data points is a somewhat interesting problem. I wrote some code to score gymnastics meets once (in Fortran!), and it kept track of the top 6 scores (the "medalists.") I ended up popping each score into a bubble-sort of the top 6 values, reducing (I think?) the complexity from NN to ~N6, which was "good enough."

Do these work in Arduino 2.0?

Never used 2.0, but I don't see why it wouldn't. It's tool chain dependent, not IDE. Of course, they're not available for an AVR processor.

1 Like

I believe OP is thinking on batches, each of 600 samples.

void loop() {
  // prepare to read a new batch of samples
  ...
  read next batch of n samples, keep track of largest sample
  print largest sample found
}

Without any information about the project or its objective, it is all left to speculation. Or voting

Hi Mancera1979, my English is a bit poor, of course I'm trying to sort and print the highest value on batches, each of 600 samples