Measure nightsky with TSL237

How about get the setup working without converting to frequency and -then- take that step?

Just show the pulses, your raw data, as you point at darker or lighter things right in the house. That will let you test and debug everything but the convert and I and or others will walk you through the conversion to the best of our abilities?

That sounds good :slight_smile:

But you are talking about pulses, the sensor outputs a frequency, that isn´t the same?

is Pulse the time and freq the hight of the wave? And the dutycycle of the sensor is 50%, is that because it ouputs a square wave or something?

I did all the maths in the sketch last night, just to see how things work together like pulses, frequency an so forth. One thing i i stumbled upon is the "Gate time", what exactly is that, because that one is set to 100ms in the sketch, but somwhere on the internets, i read that the Gatetime should be set to 1000ms to get "the true freq of the sensor"?

//

But you are talking about pulses, the sensor outputs a frequency, that isn´t the same?

I'm just going off your variable names.
How about leaving this part out until you have reliable 'pulses' then?

  freq = (pulses*1000)/(period*area);
  irradiance = (freq/1000);
  magnitude = ZP - 2.5*log10(freq);

that's where your mixed-types math is a totally different issue, let's cut that in the attempt to troubleshoot and get back to it when the rest is ready.

Starting from scratch;

This codes fetches just the freq, nothing else...

#include <FreqCounter.h>


void setup() {
  Serial.begin(57600);                    // connect to the serial port
  Serial.println("Frequency Counter");
  
}

long int frq;
void loop() {

 FreqCounter::f_comp= 8;             // Set compensation to 12
 FreqCounter::start(1000);            // Start counting with gatetime of 100ms // CHANGED TO 1000/
 while (FreqCounter::f_ready == 0)         // wait until counter ready
 
 frq=FreqCounter::f_freq;            // read result
 Serial.println(frq);                // print result
 delay(20);
}

and this is the serial monitor:

;1Frequency Counter
0
39313
39342
39573
39757
39835
39816
38690
31962
8259
7940
7956
7564
27334
5070
966
273
39
31
31
30
30
31
1087
32437
34741

where 30-ish is almost completely black.

The Irradiance responsivity for my sensor is 2.3 kHz/(uW/cm2)
and the integrated photodiode active area is 0.92 mm2 in size

How should the formula look for converting my hz to mW/cm2?

And what gatetime should i use? Martin Nawrath sais that "The Gate Time for the counting period can be chosen in the start() function where values of 10, 100 or 1000 ms are practicable for a resolution of 100, 10 and 1 Hz but any value can be used. The internal resolution of the gatetime is 2 ms so that the time can be varied in the increment of 2."

Does that means that if i use a gatetime of 1000ms, i will be viewing my sensor in Hz, and if i use the 100ms that is used in the freqcountersketch, i will be viewing the sensor in Hz/10?

Is this really the right formula? ZP - 2.5*log10(frequency) ?

Does that means that if i use a gatetime of 1000ms, i will be viewing my sensor in Hz, and if i use the 100ms that is used in the freqcountersketch, i will be viewing the sensor in Hz/10?

It means that the larger the sample taken, the less effect any roundoff or error will make, and then you use a convenient unit in pulses / time for storage and calculation and display to desired format for display.

If you choose your units right, the math should fit within the range of type long or type long long.
If neither can cover the necessary range then it's BCD, the Big Number library, roll your own way or floats, those piles of sand....

I gotta go, RL, AFK soon... mates, bar, BBL and deal with the formula, range needed and code.

Ok, i start to understand now, for the long, it can hold 32 bit of numbers, witch is 2 billions something, i think that will do, so for now, i will count Hz.

i also found this formula that seems more logical for me :slight_smile:

From candela/m2 to magnitudes/arcsec2:
B=-2.5Log(C/108000) or 12.58-2.5Log(C)

as the Hz you count is positive you better use an unsigned long which goes up to 4+ billion

B = 12.583559 - 2.5*log(C);

is preferred as it does not divide (time consuming + loss of precission if C is a int type)

How about uW/cm² to Candela then? Or is there any other way of converting uW/cm² to mag/AS2?

Edit: i found a convertion site for w/cm2 to cd/m2 and according to that, 1w/cm2 = 6 830 000 candela/m2

ref: http://www.calcul.com/light-luminance

Ok, got a little help from the guys at www.astronet.se :slight_smile:

This is the code as it looks now:

#include <FreqCounter.h>
#include <Math.h>

long Msqm;
const float A = 22;

void setup() {
  Serial.begin(57600);                    // connect to the serial port
  Serial.println("Frequency Counter");
  
}

long int frq;
void loop() {

 FreqCounter::f_comp= 100;             // Set compensation to 12
 FreqCounter::start(1000);            // Start counting with gatetime of 100ms // CHANGED TO 1000/
 while (FreqCounter::f_ready == 0)         // wait until counter ready
 
 frq=FreqCounter::f_freq;            // read result
 Serial.print(frq);   // print result
 Serial.print(" Hz ");
 Serial.print(Msqm);
 Serial.println(" Mag/Arcsecond2 ");
 delay(20);
 Msqm = A - 2.5*log(frq);
 return;
}

And that gives this on the monitor:

0 Hz 22 Mag/Arcsecond2 
1 Hz -2147483648 Mag/Arcsecond2 
3 Hz 22 Mag/Arcsecond2 
3 Hz 19 Mag/Arcsecond2 
3 Hz 19 Mag/Arcsecond2 
3 Hz 19 Mag/Arcsecond2 
3 Hz 19 Mag/Arcsecond2 
6 Hz 19 Mag/Arcsecond2 
55 Hz 17 Mag/Arcsecond2 
911 Hz 11 Mag/Arcsecond2 
1333 Hz 4 Mag/Arcsecond2 
1319 Hz 4 Mag/Arcsecond2

The formula is A - 2.5*log(frq) where A is the magnitude constant, right now, the "guess" for A is 1Hz at a level 22 mag sky. //not calibrated at all though.

Corpze:
Ok, i start to understand now, for the long, it can hold 32 bit of numbers, witch is 2 billions something, i think that will do, so for now, i will count Hz.

i also found this formula that seems more logical for me :slight_smile:

From candela/m2 to magnitudes/arcsec2:
B=-2.5Log(C/108000) or 12.58-2.5Log(C)

Yes. But the variable has to be able to handle the math you will use when scaling and squaring, be able to count much bigger than the numbers used, and smaller.

If I am dealing with kilometers, my work unit might be in millimeters, mm fractions are lost.
To help keep accuracy, instead of 1 float factor like 333.333333 I keep two integers 1000 and 3 and use those as 'x 1000 / 3' in the bigger equation.
When the bigger equation is written out don't be afraid to rearrange the operations (without breaking the algebra!) to push the intermediate value high before dividing.
X = A * (1000 / 3) * (22 / 7) would re-arrange to X = A * 22000 / 21. Note that with FP using factors you get the first.

Once you have your equation, then is the time to see how many digits you need. If A above is big then I need enough for 7000 x the biggest 'A' can be with enough to the right for display decimals.

Using mm I can show 1.234 km or 1.2 km and know I had 2 places round-safe in the math.

You deal with some squares and factors with lots of places? Some of those will be simple as a ratio of integers, some won't. Pi to 6 places is 3141592 / 1000000, to use that in an integer equation adds 7 digits to the digits needed to hold the number being multiplied by Pi to 6 places. There's a price for precision and scale, in digits and bytes.

a small bug in your code => you printed msqm before calculating it.

  • added a log(0) prevention
#include <FreqCounter.h>
#include <Math.h>

long Msqm;
const float A = 22;

void setup()
{
  Serial.begin(57600);
  Serial.println("Frequency Counter");
}

long frequency;

void loop()
{
  // MAKE MEASUREMENT 
  FreqCounter::f_comp = 100; 
  int gatetime = 1000;

  FreqCounter::start(gatetime);
  while (FreqCounter::f_ready == 0);        // wait until counter ready
  frequency = FreqCounter::f_freq;

  // CONVERT
  if (frequency > 0 )    // prevent log(0)
    Msqm = A - 2.5 * log(frequency);
  else
    Msqn = A;

  // DISPLAY RESULTS
  Serial.print(frequency);
  Serial.print(" Hz ");
  Serial.print(Msqm);
  Serial.println(" Mag/Arcsecond2 ");
  delay(20);
}

Thanks, i will try log10 instead, and add a button, one thing i want to add is higher resulotion on the Hz-side, can i use a divider or something like that? Will mHz result in higher res.?
And measure several samples and average them?

What about this:

frq = 0;
nSamples = 0;
buttonSQM = DigitalRead(HIGH);
while (buttonSQM==HIGH){
frq += FreqCounter::f_freq;
nSamples+=1;
buttonSQM = DigitalRead(HIGH);
}
frq = frq/nSamples;

And measure several samples and average them?

Watch out for rounding errors due to integer division as it truncates.

Think for your application you better sample for some time and determine the minimum and maximum.

frequency = 0;
unsigned long minimum = 1000000;
unsigned maximum = 0;

buttonSQM = digitalRead(SomePin);

while (buttonSQM == HIGH)
{
  frequency = FreqCounter::f_freq;  
  if (frequency > maximum) maximum = frequency ;
  if (frequency < minimum) minimum = frequency;
  buttonSQM = digitalRead(SomePin);
}

now do math with min & max as this gives the range (accuracy) of your measurements.
you can also do average as (min + max)/2;

I am just now building something similar in function, although I'm not interested in measuring dark sky. However, I have a few suggestions for you:

First, think in terms of the area of the sky the sensor is seeing. The shorter the focal length of the lens, the wider the area that impinges upon the sensor. Of course, the sensor is tiny, so that may not be much of a consideration. But it's something to keep in mind.

Second, use an "optical amplifier" to strengthen the signal: a lens. Surplus shops have tons of huge, short-focus lenses that were used in the now-obsolete projection monitors, and you can get a 3" objective for under $10. I found a plastic jar that fit it perfectly, cut the jar up and stuck it on, then mounted the sensor inside it. This thing was extremely sensitive: I was using the TSL235R sensor, which is not as sensitive as the one you describe, I was still getting something like 3 Hz output pointed at a midnight sky. When I realized that I didn't need that much sensitivity, I tore it apart.

GeezerFrog:
I am just now building something similar in function, although I'm not interested in measuring dark sky. However, I have a few suggestions for you:

First, think in terms of the area of the sky the sensor is seeing. The shorter the focal length of the lens, the wider the area that impinges upon the sensor. Of course, the sensor is tiny, so that may not be much of a consideration. But it's something to keep in mind.

Second, use an "optical amplifier" to strengthen the signal: a lens. Surplus shops have tons of huge, short-focus lenses that were used in the now-obsolete projection monitors, and you can get a 3" objective for under $10. I found a plastic jar that fit it perfectly, cut the jar up and stuck it on, then mounted the sensor inside it. This thing was extremely sensitive: I was using the TSL235R sensor, which is not as sensitive as the one you describe, I was still getting something like 3 Hz output pointed at a midnight sky. When I realized that I didn't need that much sensitivity, I tore it apart.

I will use a 20mm 20degree lens, so i will amplify the light :slight_smile:

About the rounding errors, i will look in to that, but something i don´t understand is why i should use log10( instead of log( ?

Then, a guy i spoke to said that using that low of freq. can result in really bad accuracy, and pointed out that i might measure the time between two uprising flanks??? and then invert, then i can get accuracy down to the CPU-speed? I have no idea of what he is meaning?

FreqCounter for very low frequencies (below 100 Hz or something) isn't the best thing you can do to count cycles.

As someone told you, you would need to measure the length of the pulse instead (FreqMeasure, FreqPeriod, InputCapture, pulseIn, millis()/micros() ).
Since at dark time it's rare to get more than 10,000Hz with those sensors, I think FreqCounter isn't useful at all for this project.

Oh... can you recomend any other method, library or code?

At the first page, i linked to this sketch; How to: Light to Frequency Converter - TSL235R? - #17 by madepablo - Interfacing - Arduino Forum

i Think he is counting pulses and from that, fetches the frequency, may that do the trick?

I am not sure, but i found this site: http://www.avdweb.nl/arduino/hardware-interfacing/frequency-period-counter.html

And i am trying this code:

#include <FreqPeriodCounter.h>
 
const byte counterPin = 5; 
const byte counterInterrupt = 1; // = pin 3
FreqPeriodCounter counter(counterPin, micros, 0);
unsigned long centiHz = counter.hertz(100);
float hz = 1000/counter.period;
 
void setup(void) 
{ Serial.begin(9600); 
  attachInterrupt(counterInterrupt, counterISR, CHANGE);
}
 
void loop(void) 
{ if(counter.ready()) Serial.println(counter.hertz());
}
 
void counterISR()
{ counter.poll();
}

And im getting this now;

3068
5004
1903
5016
5008
5053
1669
3660
5034
3665
3070
5018
5005
5057
2103
5033
2497
1668
2514
5043
1907
1667
5032
2497
1666
5046
1906
1666
3651
5056
3652

Am i getting the Hz in "centiHz" now? and why is the values fluctuating that much? i left the sensor in a enclosure with the same amout of light hitting it all the time :confused:

You seem to be futzing around changing too many things at once.

If your process depends on counting pulses, you need to get that process right before
worrying and fiddling with what you do subsequently.