Correlation matrix to calculate similarity?

Projects Programming
1

Hi,

I'm working on some code in which I need to compare the similarity of 2 arrays using this:

uint8_t compare(const char Ref[128], char Act[128]) {
	uint16_t sumOfRefOrAct = 0;
	uint16_t sumOfDiff = 0;
		
	for (uint16_t i = 0; i < 64; i++) {
		uint8_t diff;
		if (Ref[i] > Act[i]){
			diff = (Ref[i] - Act[i]);
			sumOfRefOrAct += Ref[i];
		}
		else if (Act[i] > Ref[i]){
			diff = (Act[i] - Ref[i]);
			sumOfRefOrAct += Act[i];  

		}
		else {
			diff = 0;
			sumOfRefOrAct += Ref[i];

		}
		sumOfDiff += diff;
	}
	if (sumOfRefOrAct != 0) {
		return(((sumOfRefOrAct - sumOfDiff) * 100.0) / sumOfRefOrAct);
	}
	else {
		return(0);
	}
}

The purpose is to compare 3 reference arrays (already declared in the cpp file) to 3 arrays filled with the incoming data.
I used the returned values as an interval like this:

if (([b]maxPrc [/b]>= 75 && maxPrc <= 95) && ([b]minPrc [/b]>= 18 && minPrc <= 50) && ([b]moyPrc [/b]>= 45 && moyPrc <= 70))
    Serial.println("ALERT");

It works just fine (in a dumb way) for some tests but it's faaaaar from accurate.
My teacher asked me to replace it with something like "correlation matrix" and I have no idea what it is or how to use it or is it really the right method to use!

Any help?

2

If you look up Sums of Squares or Sum of Differences, you'll likely find references to Analysis of Variance (ANOVA) as well as the Correlation Coefficient (and R^2 analysis). Depending on the data, will an int be large enough to hold the result? Also, take the definition of uint8_t diff out of the loop. Finally, with the cursor in the source code window of the Arduino IDE, press Ctrl-T to reformat your code in a little easier-to-read format.

3

First you need to define what you mean by similarity. From your code, I would guess that if the elements at a given array index are all numerically close to each other, then the arrays are similar. In that case a simple correlation coefficient between the two is all you need.

However, what if the only differences between the arrays are that some elements are simply swapped?

4

Pinguu:
It works just fine (in a dumb way) for some tests but it's faaaaar from accurate.
My teacher asked me to replace it with something like "correlation matrix" and I have no idea what it is or how to use it or is it really the right method to use!

You teacher obviously expects you to recognise these terms, or at least have the initiative to go look them up and make sense of articles explaining them. Have you tried doing that?

5

The problem applied to strings seems quite difficult (and "interesting.") However, spell checkers do it...

6

econjack:
If you look up Sums of Squares or Sum of Differences, you'll likely find references to Analysis of Variance (ANOVA) as well as the Correlation Coefficient (and R^2 analysis). Depending on the data, will an int be large enough to hold the result? Also, take the definition of uint8_t diff out of the loop. Finally, with the cursor in the source code window of the Arduino IDE, press Ctrl-T to reformat your code in a little easier-to-read format.

Thanks for the advice, will do.

jremington:
First you need to define what you mean by similarity. From your code, I would guess that if the elements at a given array index are all numerically close to each other, then the arrays are similar. In that case a simple correlation coefficient between the two is all you need.

However, what if the only differences between the arrays are that some elements are simply swapped?

I already tried the correlation coefficient on Matlab but I wasn't sure if it's what I was looking for, thanks.
I only need to check if Ref ? Act , it doesn't matter if the elements are swapped.
> PeterH:
> You teacher obviously expects you to recognise these terms, or at least have the initiative to go look them up and make sense of articles explaining them. Have you tried doing that?
He didn't seem sure about it so I needed to know if it's the right way to do things before starting to dig in my researches.

7

"String metric" perhaps?

8

Pinguu:
He didn't seem sure about it so I needed to know if it's the right way to do things before starting to dig in my researches.

Really, your teacher is telling you to do something he is unsure about? That sounds very strange, and perhaps he is just trying to encourage you to think about the problem for yourself rather than tell you exactly how to solve the problem. In any case I suggest you go and 'dig into your researches' if you want to understand the approaches he has suggested. It seems obvious that is what he expects you to do - people here doing your homework for you are not doing you any favours.

9

your teacher is telling you to do something he is unsure about? That sounds very strange

Nah. That's a good teacher, one who doesn't limit their students' learning to what they already know. It shouldn't happen if the class actually convers similarity, but it's easy for me to imagine a student project that could use such things, that went beyond the bounds of the class material.

I have fond memories of when I went off an did "file" stuff in my HS basic class, and my teacher essentially said "I've never done what you're doing, but don't let that stop you."

10

westfw:
"String metric" perhaps?
String metric - Wikipedia

It's not quite what I need but thanks anyway !

PeterH:
Really, your teacher is telling you to do something he is unsure about? That sounds very strange, and perhaps he is just trying to encourage you to think about the problem for yourself rather than tell you exactly how to solve the problem. In any case I suggest you go and 'dig into your researches' if you want to understand the approaches he has suggested. It seems obvious that is what he expects you to do - people here doing your homework for you are not doing you any favours.

My teacher is supervising me for a project, he doesn't know everything about the subject so he just gives some ideas. I'm already doing all the work by myself and I don't expect anyone to do it for me that's not why I was posting here. I don't know everything about programming, I didn't know from where to start, I needed help from people with better knowledge. It was unnecessary of you to be unpleasant about it.

11

What is your data?

12

The data array is filled with values read from a sound sensor. The reference array is a stored spectra.
The purpose is to find a similar incoming signal to the reference. To do so I need to compare the 2 arrays and return a value which I can use as a condition in the rest of my code.