Is there a way to extract non-zero values from an array?

Hi all,

I'm working on a project to monitor the frequency of AC Supply in the UK (50Hz). I'm using the Uno's in-built ADC and getting an adequate sample rate of 8333 Hz using the code below.

To calculate the frequency I'm using the zero cross over method. The code below detects the point at which the voltage crosses the "zero" at 512 then storing the times in the t() and x() arrays. To improve the accuracy of the calculation I'm planning on using the first and third crossovers (as opposed to consecutive ones) at the 512, 400 and 600 positions and taking an average.

The problem is I'm struggling to extract the first and third crossovers from my t() array without severely reducing the sampling rate (I've tried introducing nested for and while loops to "fill up" the x() array one value at a time). Can anyone suggest a different way to either store the cross overs times or extract them from the t() array?

TL;DR... How can I extract three non-zero values from an array without affecting sampling rate?

int i;                // Counter
int T;                // Time Period
float f;              // Frequency
int z[250];           // Array for storing voltage values (0-1023)
unsigned long t[250]; // Array for storing time (microseconds)
unsigned long x[2];   // Smaller array for storing zero cross-over time (microseconds)

void setup() {

  Serial.begin(9600);

  for (i = 0; i < 250; i++) {   
    z[i] = analogRead(A5);

    if (z[i - 1] > 512 && z[i] < 512)         // "Zero" cross-over point (with falling voltage)
      x[0] = micros(),                        // Store cross-over time in the x-array
             t[i] = micros();                 // Also store time in the t-array for Serial.print later
    else if (z[i - 1] < 512 && z[i] > 512)    // "Zero" cross-over point (rising voltage)
      x[1] = micros(),
             t[i] = micros();

    delayMicroseconds(8);                     // This makes total sampling period 0.03 seconds long, 
  }                                           // equivalent to 1.5 wavelengths at 50Hz 

  T = abs(x[0] - x[1]);                       // Calculate time period between consecutive cross-overs
  cross-overs
  f = 1000000 / (2 * T);                      // Calculate frequency

  for (i = 0; i < 250; i++) {
    Serial.print(z[i]);
    Serial.print("\t");
    Serial.println(t[i]);
  }
  delay(1000);

  Serial.println();
  Serial.print(x[0]);
  Serial.print("\t");
  Serial.print(x[1]);
  Serial.print("\t");
  Serial.print(T);
  Serial.print("\t");
  Serial.println(f);

}

void loop() {

}

If you are continuously taking samples every 8 microseconds, there won't be much time left to do any other processing on an AVR. There's no way out of that. Maybe there is a window when you can stop sampling?

analogRead typically takes around 100 microseconds.
HTH

Why not detect the zero crossings themselves?

Atmel suggested this slightly scary looking method:
http://www.atmel.com/Images/Atmel-2508-Zero-Cross-Detector_ApplicationNote_AVR182.pdf

There are also plenty of other ways to detect zero crossings (it's a very common thing to need to do)

However you get it (either atmel's way, or some external hardware), pipe the signal into the input capture pin for timer 1 and configure it for input capture, and you'll get the time between the zero crossings and can easily calculate the frequency from that.

There are also plenty of other ways to detect zero crossings (it's a very common thing to need to do)

The h11aa1 is very commonly used for zero crossing detection.

https://www.vishay.com/docs/83608/h11aa1.pdf

BillHelmo:
I'm working on a project to monitor the frequency of AC Supply in the UK (50Hz).

Sorry, but I can't resist asking why?

Are you worried that it might not really be 50Hz?

...R

+1
Why are you trying to measure the very accurate (over time) mains frequency.
Mains frequency is likely more accurate than a resonator clocked Arduino.
Leo..

Over a short period the mains is not as accurate as a crystal or resonator. Over 24 hours it will be more accurate.

It will be an interesting thing to observe.

Thank you for all your responses!

aarg:
Maybe there is a window when you can stop sampling?

Essentially I'm doing that already, I'm sampling for 0.03 seconds then stopping and processing the samples. Is there a way to split this up further, possibly by calling separate voids/functions?

DrAzzy:
Why not detect the zero crossings themselves?

cattledog:
The h11aa1 is very commonly used for zero crossing detection.

I don't see the need to invest in additional hardware when I can record the zero crossings quite easily with this line of code:

     if ((z[i - 1] > 512 && z[i] <= 512) || (z[i - 1] < 512 && z[i] >= 512))
      t[i] = micros();

It also means I can position the crossover point wherever I want, by changing the 512 value.

Robin2:
Are you worried that it might not really be 50Hz?

Wawa:
Why are you trying to measure the very accurate (over time) mains frequency.

The national grid frequency is in fact constantly changing. Although it averages 50 Hz over long periods time, as Leo and MorganS suggest, in the short term it varies dramatically depending on the supply and demand. Have a look at this link for proof!

So back to my problem. I've put a shortened table of the data I'm getting and where it's stored. The way my code is written means I'm only capturing the last consecutive crossovers in the x array, while the t array captures all three crossovers. How can I extract the three values from t?

Value Time(us) Time(us)
z[250] t[250] x[2]
...
493 0
504 0
513 6700
528 0

516 0
507 16692 16692
499 0

504 0
513 26692 26692
522 0
...

There is no need to look at zero crossings unless you need the exact phase. Just condition it for input safety with a limiting RC circuit and feed it to a digital pin. Software will see a square wave that will make it extremely easy to determine the exact frequency.

With analog readings you are making the whole thing 100 times more complicated than it needs to be.

Also, in your response, you have not replied to the points about Arduino clock accuracy.

With analog readings you are making the whole thing 100 times more complicated than it needs to be.

not to mention the quantizing noise it introduces .
I estimate with 10 bit adc @ 5v and a 50Hz signal [ asynchronous ] the jitter introduced is in the order of +/- 4uS
how does that compare to the error you are trying to measure ?

Stop using Analog read it's a blocking function and therefore your spending all your time waiting for the A to D conversion to complete! Set the ADC to free running mode and grab the value when to want it (You may find you need to slow your program down!)

You should also reduce the sample size from 10 bits to 8 this gives you 86K (See data sheet) samples per second.

Mark

BillHelmo:
The national grid frequency is in fact constantly changing. Although it averages 50 Hz over long periods time, as Leo and MorganS suggest, in the short term it varies dramatically depending on the supply and demand. Have a look at this link for proof!

That does not really explain "why". Even if you know it is wrong at any instant there is nothing you can do about it. And Arduinos do not run at a precise 16 MHz so your measuring will not be accurate either.

...R

Thanks for the replies, keep them coming!

aarg:
Just condition it for input safety with a limiting RC circuit and feed it to a digital pin. Software will see a square wave that will make it extremely easy to determine the exact frequency.

This is really interesting, I was originally trying to form a square wave from the analog values, but I hadn't considered inputting the signal into a digital pin. I've already conditioned the signal down to 5v pk-pk, with a DC offset of 2.5v and an RC low pass filter to remove higher frequencies. In your opinion, would it be a case of removing the DC offset and using digitalRead() to get a square waveform? Would this still mean relying on the not-so-reliable Arduino clock accuracy which you mentioned?

racpi:
not to mention the quantizing noise it introduces .
I estimate with 10 bit adc @ 5v and a 50Hz signal [ asynchronous ] the jitter introduced is in the order of +/- 4uS
how does that compare to the error you are trying to measure ?

You're spot on, I'm getting a discrepancy of between 4 and 6us. After measuring the time period at three positions and taking an average I'm getting about 0.4 % error between the frequency approximation and the frequency set on the signal generator. Not bad, but I'd like to bring this down further if possible. Any suggestions?

While considering % error, could there also be a discrepancy in the generated signal? I'm using a Drok JDS6600 from Amazon, the data sheet claims frequency resolution of 0.01uHz but I'd like to verify that, any ideas how to go about doing that without spending a fortune on an oscilloscope?

holmes4:
Stop using Analog read it's a blocking function and therefore your spending all your time waiting for the A to D conversion to complete! Set the ADC to free running mode and grab the value when to want it (You may find you need to slow your program down!)

I've read about this before but never tried it, until now! I had a go at adapting this example but I didn't make much progress, have you got any other examples that you could point me to?

holmes4:
You should also reduce the sample size from 10 bits to 8 this gives you 86K (See data sheet) samples per second.

I haven't tried this before either. Should I focus on getting the ADC in free running mode first, or will this technique also work with analogRead()?

Robin2:
That does not really explain "why". Even if you know it is wrong at any instant there is nothing you can do about it.

If you do a bit of digging around that link you'll see that the National Grid have their work cut out trying to balance the grid and maintain frequency. Imagine 20 million kettles switching on at half time during the football, the frequency drops and the Grid has to respond to restore it to 50Hz. At the moment, I am interested in recording these movements to see how predictable they are, from day to day, through the night, and on a yearly basis. And yes, as individuals, there's not a lot we can do about it, but maybe if those 20 million tea and coffee drinkers worked together, we could.

Robin2:
And Arduinos do not run at a precise 16 MHz so your measuring will not be accurate either.

I take your point about the accuracy of the measurement, could this fact be a factor of the 0.4% error I have mentioned above? I was under the impression that by sampling a 50Hz signal at 8333Hz using analogRead() I would get a reasonably accurate result, and I guess +/- 0.25 Hz isn't bad, but do give me some suggestions on how this can be improved.

BillHelmo:
If you do a bit of digging around that link you'll see that the National Grid have their work cut out trying to balance the grid and maintain frequency. Imagine 20 million kettles switching on at half time during the football, the frequency drops and the Grid has to respond to restore it to 50Hz. At the moment, I am interested in recording these movements to see how predictable they are, from day to day, through the night, and on a yearly basis. And yes, as individuals, there's not a lot we can do about it, but maybe if those 20 million tea and coffee drinkers worked together, we could.

It is what it is; the National Grid admit it varies.

So what are you suggesting ?

That National Grid should be persuaded to make the frequency variation less and if so why ?

This is really interesting, I was originally trying to form a square wave from the analog values, but I hadn't considered inputting the signal into a digital pin. I've already conditioned the signal down to 5v pk-pk, with a DC offset of 2.5v and an RC low pass filter to remove higher frequencies. In your opinion, would it be a case of removing the DC offset and using digitalRead() to get a square waveform? Would this still mean relying on the not-so-reliable Arduino clock accuracy which you mentioned?

Did you read the reference cited in reply #3. With an input resistor to limit the current to less than 1 ma you can use the internal clamping diodes to create a signal to be read with an interrupt.

All timing will rely on the not so accurate internal clock unless you have an external clock source somewhere.

BillHelmo:
Imagine 20 million kettles switching on at half time during the football, the frequency drops and the Grid has to respond to restore it to 50Hz. At the moment, I am interested in recording these movements to see how predictable they are, from day to day, through the night, and on a yearly basis.

Having worked in the Dept of Energy in Ireland I am 99.999% certain than National Grid in the UK has an enormous amount of data about those movements - though I have no idea whether they make it available to the public. I can't think of any reason why it would need to be secret so it would be worthwhile asking them.

AFAIK they monitor TV schedules and sporting events and all the things are likely to result in the 20 million kettles being switched on during half-time so that they are as ready as possible to meet the surge. Turning all the kettles off at the same time is just as big a problem.

There was something on the news recently about NG preparing for the effect of the next solar eclipse (in a few years time) on the amount of solar power being produced.

And rather than build an Arduino monitoring system it may actually be more useful to write a screen-scraper program to extract data from the live National Grid link that you posted :slight_smile:

...R

srnet:
So what are you suggesting ?

That National Grid should be persuaded to make the frequency variation less and if so why ?

No I think they do a fantastic job in minimising variation, I'm thinking along the lines of managing the demands on the grid so that less energy is wasted in ramping power up and down in response to the deviations. This kind of thinking might be needed when it comes to charging 20 million electric cars overnight.

cattledog:
Did you read the reference cited in reply #3. With an input resistor to limit the current to less than 1 ma you can use the internal clamping diodes to create a signal to be read with an interrupt.

I missed that bit, thanks for pointing it out.

Robin2:
Having worked in the Dept of Energy in Ireland I am 99.999% certain than National Grid in the UK has an enormous amount of data about those movements - though I have no idea whether they make it available to the public. I can't think of any reason why it would need to be secret so it would be worthwhile asking them.

You're right, they do have stacks of data and in general they're pretty transparent on their operations. I've tracked down some good papers and sources of information, but I thought it would be interesting to capture the data first hand.

Robin2:
Turning all the kettles off at the same time is just as big a problem.

Yes, exactly, how do you suddenly drop 1MW from the grid? Switch on a load of fat resistors and dissipate it as heat.

Robin2:
And rather than build an Arduino monitoring system it may actually be more useful to write a screen-scraper program to extract data from the live National Grid link that you posted

I have considered it! But I'm also interested to see the variations geographically, for instance does it vary from city to city, north to south, coast to mainland etc.

I think it's a very interesting project. It's like measuring gravity: small variations are interesting even if you don't need to control them.

I think the Arduino crystal will be accurate enough to show some interesting things. The next step in accuracy is to use a GPS with a 1 pulse-per-second output. The GPS clock is more accurate than the rotation of the Earth itself.

cattledog:
Did you read the reference cited in reply #3. With an input resistor to limit the current to less than 1 ma you can use the internal clamping diodes to create a signal to be read with an interrupt.

All timing will rely on the not so accurate internal clock unless you have an external clock source somewhere.

Neat trick but what about the voltage? No, I really don't know how that would work!

I think that my Nano has a crystal and caps.