Measure nightsky with TSL237

GoForSmoke:

Corpze:
If i want to make ,say three measurements, and then take the median (freq1 + freq2 + freq3 / 3) ?

Operation order will screw that up. Only the last will be divided.

((freq1 + freq2 + freq3) / 3) is what you want, if that is really what you want to do **.

Extra parenthesis can't hurt unless they're in the wrong place or not balanced (as many opens as closes). If in doubt, use them. The compiler doesn't add code for extras.

**

Pulses: 10; Freq: 10869 Hz

If you averaged the 10 pulses as above, you would lose the last digit, have no rounding....

You could do something like (untested pseudo code alert!) this for rounding

samples = 3 // could be 10 for instance

freq = ((freq1 + freq2 + freq3 + samples / 2) / samples)

I will try that, but what about the frequency, why can´t it go any lower than 10869 Hz? I haven´t calibrated the ZP yet, can it be the problem?

Corpze:
I will try that, but what about the frequency, why can´t it go any lower than 10869 Hz? I haven´t calibrated the ZP yet, can it be the problem?

For single samples you get 1086. Why? I don't know!

freq = (pulses1000)/(periodarea);

Is it any surprise that 10 pulses gets 10x the freq?

Also you should mix your mixed-types math.
As above you have by types: long = (long * int) / (short * float) <<=== not a good idea

That whole business needs a total going-over.
You can throw in casts (tell the compiler to treat a variable as a different type) but if you're not careful you won't get the equation you think. Without casts, as it is now, you're letting the compiler decide.

Example of casting:
long L;
int I = 1000;
float F = 0.1;

L = (long) ((float I) * F);

And thanks to the floating point math, the result won't likely be 100L. Yes 100L because it's cast as a long to go into a long variable. 100 is a 16-bit signed integer. 100L is a 32-bit signed integer. If the letter U is used, 100U is a 16-bit unsigned integer and 100UL is a 32-bit unsigned integer.
If you use a constant and want it seen as a floating point then put in a decimal point, 100.0 is a 32-bit floating point value.

Give the compiler less room to wiggle and you may get more predictable results.

GoForSmoke:

Corpze:
I will try that, but what about the frequency, why can´t it go any lower than 10869 Hz? I haven´t calibrated the ZP yet, can it be the problem?

For single samples you get 1086. Why? I don't know!

freq = (pulses1000)/(periodarea);

Is it any surprise that 10 pulses gets 10x the freq?

Also you should mix your mixed-types math.
As above you have by types: long = (long * int) / (short * float) <<=== not a good idea

That whole business needs a total going-over.
You can throw in casts (tell the compiler to treat a variable as a different type) but if you're not careful you won't get the equation you think. Without casts, as it is now, you're letting the compiler decide.

Example of casting:
long L;
int I = 1000;
float F = 0.1;

L = (long) ((float I) * F);

And thanks to the floating point math, the result won't likely be 100L. Yes 100L because it's cast as a long to go into a long variable. 100 is a 16-bit signed integer. 100L is a 32-bit signed integer. If the letter U is used, 100U is a 16-bit unsigned integer and 100UL is a 32-bit unsigned integer.
If you use a constant and want it seen as a floating point then put in a decimal point, 100.0 is a 32-bit floating point value.

Give the compiler less room to wiggle and you may get more predictable results.

To be honest, i don´t understand anything of what you just wrote :confused:

Maybe i should just start over with a basic frequency counter and go from there?

Basicly, all i wanna do is translate frequence to mag/arcsecond2, and because the sensor is linear, wouldn´t it be quite easy to do that?

I am fairly new to arduinos and coding, i have just got a steppermotor to work with two buttons and a lcd-display :confused:

Is this the way to go maybe?:

I have one sensor, capable of sending between 0.00x Hz to 1000kHz
I also have a luxmeter, so i can calibrate the sensorvalue roughly.

if i has a light that shines with 100 lux, and the sensor is sending 20 kHz, can i from there convert to mag/arcsecond2?

I dont need any pulsecounter any else of the data, i am just interested of Mag/AS2

Is this possible?

BR/ Daniel

How about get the setup working without converting to frequency and -then- take that step?

Just show the pulses, your raw data, as you point at darker or lighter things right in the house. That will let you test and debug everything but the convert and I and or others will walk you through the conversion to the best of our abilities?

That sounds good :slight_smile:

But you are talking about pulses, the sensor outputs a frequency, that isn´t the same?

is Pulse the time and freq the hight of the wave? And the dutycycle of the sensor is 50%, is that because it ouputs a square wave or something?

I did all the maths in the sketch last night, just to see how things work together like pulses, frequency an so forth. One thing i i stumbled upon is the "Gate time", what exactly is that, because that one is set to 100ms in the sketch, but somwhere on the internets, i read that the Gatetime should be set to 1000ms to get "the true freq of the sensor"?

//

But you are talking about pulses, the sensor outputs a frequency, that isn´t the same?

I'm just going off your variable names.
How about leaving this part out until you have reliable 'pulses' then?

  freq = (pulses*1000)/(period*area);
  irradiance = (freq/1000);
  magnitude = ZP - 2.5*log10(freq);

that's where your mixed-types math is a totally different issue, let's cut that in the attempt to troubleshoot and get back to it when the rest is ready.

Starting from scratch;

This codes fetches just the freq, nothing else...

#include <FreqCounter.h>


void setup() {
  Serial.begin(57600);                    // connect to the serial port
  Serial.println("Frequency Counter");
  
}

long int frq;
void loop() {

 FreqCounter::f_comp= 8;             // Set compensation to 12
 FreqCounter::start(1000);            // Start counting with gatetime of 100ms // CHANGED TO 1000/
 while (FreqCounter::f_ready == 0)         // wait until counter ready
 
 frq=FreqCounter::f_freq;            // read result
 Serial.println(frq);                // print result
 delay(20);
}

and this is the serial monitor:

;1Frequency Counter
0
39313
39342
39573
39757
39835
39816
38690
31962
8259
7940
7956
7564
27334
5070
966
273
39
31
31
30
30
31
1087
32437
34741

where 30-ish is almost completely black.

The Irradiance responsivity for my sensor is 2.3 kHz/(uW/cm2)
and the integrated photodiode active area is 0.92 mm2 in size

How should the formula look for converting my hz to mW/cm2?

And what gatetime should i use? Martin Nawrath sais that "The Gate Time for the counting period can be chosen in the start() function where values of 10, 100 or 1000 ms are practicable for a resolution of 100, 10 and 1 Hz but any value can be used. The internal resolution of the gatetime is 2 ms so that the time can be varied in the increment of 2."

Does that means that if i use a gatetime of 1000ms, i will be viewing my sensor in Hz, and if i use the 100ms that is used in the freqcountersketch, i will be viewing the sensor in Hz/10?

Is this really the right formula? ZP - 2.5*log10(frequency) ?

Does that means that if i use a gatetime of 1000ms, i will be viewing my sensor in Hz, and if i use the 100ms that is used in the freqcountersketch, i will be viewing the sensor in Hz/10?

It means that the larger the sample taken, the less effect any roundoff or error will make, and then you use a convenient unit in pulses / time for storage and calculation and display to desired format for display.

If you choose your units right, the math should fit within the range of type long or type long long.
If neither can cover the necessary range then it's BCD, the Big Number library, roll your own way or floats, those piles of sand....

I gotta go, RL, AFK soon... mates, bar, BBL and deal with the formula, range needed and code.

Ok, i start to understand now, for the long, it can hold 32 bit of numbers, witch is 2 billions something, i think that will do, so for now, i will count Hz.

i also found this formula that seems more logical for me :slight_smile:

From candela/m2 to magnitudes/arcsec2:
B=-2.5Log(C/108000) or 12.58-2.5Log(C)

as the Hz you count is positive you better use an unsigned long which goes up to 4+ billion

B = 12.583559 - 2.5*log(C);

is preferred as it does not divide (time consuming + loss of precission if C is a int type)

How about uW/cm² to Candela then? Or is there any other way of converting uW/cm² to mag/AS2?

Edit: i found a convertion site for w/cm2 to cd/m2 and according to that, 1w/cm2 = 6 830 000 candela/m2

ref: http://www.calcul.com/light-luminance

Ok, got a little help from the guys at www.astronet.se :slight_smile:

This is the code as it looks now:

#include <FreqCounter.h>
#include <Math.h>

long Msqm;
const float A = 22;

void setup() {
  Serial.begin(57600);                    // connect to the serial port
  Serial.println("Frequency Counter");
  
}

long int frq;
void loop() {

 FreqCounter::f_comp= 100;             // Set compensation to 12
 FreqCounter::start(1000);            // Start counting with gatetime of 100ms // CHANGED TO 1000/
 while (FreqCounter::f_ready == 0)         // wait until counter ready
 
 frq=FreqCounter::f_freq;            // read result
 Serial.print(frq);   // print result
 Serial.print(" Hz ");
 Serial.print(Msqm);
 Serial.println(" Mag/Arcsecond2 ");
 delay(20);
 Msqm = A - 2.5*log(frq);
 return;
}

And that gives this on the monitor:

0 Hz 22 Mag/Arcsecond2 
1 Hz -2147483648 Mag/Arcsecond2 
3 Hz 22 Mag/Arcsecond2 
3 Hz 19 Mag/Arcsecond2 
3 Hz 19 Mag/Arcsecond2 
3 Hz 19 Mag/Arcsecond2 
3 Hz 19 Mag/Arcsecond2 
6 Hz 19 Mag/Arcsecond2 
55 Hz 17 Mag/Arcsecond2 
911 Hz 11 Mag/Arcsecond2 
1333 Hz 4 Mag/Arcsecond2 
1319 Hz 4 Mag/Arcsecond2

The formula is A - 2.5*log(frq) where A is the magnitude constant, right now, the "guess" for A is 1Hz at a level 22 mag sky. //not calibrated at all though.

Corpze:
Ok, i start to understand now, for the long, it can hold 32 bit of numbers, witch is 2 billions something, i think that will do, so for now, i will count Hz.

i also found this formula that seems more logical for me :slight_smile:

From candela/m2 to magnitudes/arcsec2:
B=-2.5Log(C/108000) or 12.58-2.5Log(C)

Yes. But the variable has to be able to handle the math you will use when scaling and squaring, be able to count much bigger than the numbers used, and smaller.

If I am dealing with kilometers, my work unit might be in millimeters, mm fractions are lost.
To help keep accuracy, instead of 1 float factor like 333.333333 I keep two integers 1000 and 3 and use those as 'x 1000 / 3' in the bigger equation.
When the bigger equation is written out don't be afraid to rearrange the operations (without breaking the algebra!) to push the intermediate value high before dividing.
X = A * (1000 / 3) * (22 / 7) would re-arrange to X = A * 22000 / 21. Note that with FP using factors you get the first.

Once you have your equation, then is the time to see how many digits you need. If A above is big then I need enough for 7000 x the biggest 'A' can be with enough to the right for display decimals.

Using mm I can show 1.234 km or 1.2 km and know I had 2 places round-safe in the math.

You deal with some squares and factors with lots of places? Some of those will be simple as a ratio of integers, some won't. Pi to 6 places is 3141592 / 1000000, to use that in an integer equation adds 7 digits to the digits needed to hold the number being multiplied by Pi to 6 places. There's a price for precision and scale, in digits and bytes.

a small bug in your code => you printed msqm before calculating it.

  • added a log(0) prevention
#include <FreqCounter.h>
#include <Math.h>

long Msqm;
const float A = 22;

void setup()
{
  Serial.begin(57600);
  Serial.println("Frequency Counter");
}

long frequency;

void loop()
{
  // MAKE MEASUREMENT 
  FreqCounter::f_comp = 100; 
  int gatetime = 1000;

  FreqCounter::start(gatetime);
  while (FreqCounter::f_ready == 0);        // wait until counter ready
  frequency = FreqCounter::f_freq;

  // CONVERT
  if (frequency > 0 )    // prevent log(0)
    Msqm = A - 2.5 * log(frequency);
  else
    Msqn = A;

  // DISPLAY RESULTS
  Serial.print(frequency);
  Serial.print(" Hz ");
  Serial.print(Msqm);
  Serial.println(" Mag/Arcsecond2 ");
  delay(20);
}

Thanks, i will try log10 instead, and add a button, one thing i want to add is higher resulotion on the Hz-side, can i use a divider or something like that? Will mHz result in higher res.?
And measure several samples and average them?

What about this:

frq = 0;
nSamples = 0;
buttonSQM = DigitalRead(HIGH);
while (buttonSQM==HIGH){
frq += FreqCounter::f_freq;
nSamples+=1;
buttonSQM = DigitalRead(HIGH);
}
frq = frq/nSamples;

And measure several samples and average them?

Watch out for rounding errors due to integer division as it truncates.

Think for your application you better sample for some time and determine the minimum and maximum.

frequency = 0;
unsigned long minimum = 1000000;
unsigned maximum = 0;

buttonSQM = digitalRead(SomePin);

while (buttonSQM == HIGH)
{
  frequency = FreqCounter::f_freq;  
  if (frequency > maximum) maximum = frequency ;
  if (frequency < minimum) minimum = frequency;
  buttonSQM = digitalRead(SomePin);
}

now do math with min & max as this gives the range (accuracy) of your measurements.
you can also do average as (min + max)/2;

I am just now building something similar in function, although I'm not interested in measuring dark sky. However, I have a few suggestions for you:

First, think in terms of the area of the sky the sensor is seeing. The shorter the focal length of the lens, the wider the area that impinges upon the sensor. Of course, the sensor is tiny, so that may not be much of a consideration. But it's something to keep in mind.

Second, use an "optical amplifier" to strengthen the signal: a lens. Surplus shops have tons of huge, short-focus lenses that were used in the now-obsolete projection monitors, and you can get a 3" objective for under $10. I found a plastic jar that fit it perfectly, cut the jar up and stuck it on, then mounted the sensor inside it. This thing was extremely sensitive: I was using the TSL235R sensor, which is not as sensitive as the one you describe, I was still getting something like 3 Hz output pointed at a midnight sky. When I realized that I didn't need that much sensitivity, I tore it apart.

GeezerFrog:
I am just now building something similar in function, although I'm not interested in measuring dark sky. However, I have a few suggestions for you:

First, think in terms of the area of the sky the sensor is seeing. The shorter the focal length of the lens, the wider the area that impinges upon the sensor. Of course, the sensor is tiny, so that may not be much of a consideration. But it's something to keep in mind.

Second, use an "optical amplifier" to strengthen the signal: a lens. Surplus shops have tons of huge, short-focus lenses that were used in the now-obsolete projection monitors, and you can get a 3" objective for under $10. I found a plastic jar that fit it perfectly, cut the jar up and stuck it on, then mounted the sensor inside it. This thing was extremely sensitive: I was using the TSL235R sensor, which is not as sensitive as the one you describe, I was still getting something like 3 Hz output pointed at a midnight sky. When I realized that I didn't need that much sensitivity, I tore it apart.

I will use a 20mm 20degree lens, so i will amplify the light :slight_smile:

About the rounding errors, i will look in to that, but something i don´t understand is why i should use log10( instead of log( ?

Then, a guy i spoke to said that using that low of freq. can result in really bad accuracy, and pointed out that i might measure the time between two uprising flanks??? and then invert, then i can get accuracy down to the CPU-speed? I have no idea of what he is meaning?

FreqCounter for very low frequencies (below 100 Hz or something) isn't the best thing you can do to count cycles.

As someone told you, you would need to measure the length of the pulse instead (FreqMeasure, FreqPeriod, InputCapture, pulseIn, millis()/micros() ).
Since at dark time it's rare to get more than 10,000Hz with those sensors, I think FreqCounter isn't useful at all for this project.