Go Down

Topic: Help understanding counts with Adafruit TSL2591 (Read 264 times) previous topic - next topic

Nazaar--

Hi, 

I'm using an Adafruit TSL2591 light-to-digital sensor to try and find a reasonably-priced way to measure and record moonlight, with the aim of putting it underwater. I've attached adafruit 32u4 adalogger and a featherwing to log it. 

My understanding is limited, but that photons hit the photodiodes, and ADCs convert this to a signal, resulting in 'blips' the arduino can count. My main question is how this works in practice when you're talking about integration times. Is it a matter of when enough photons hit the photodiodes, it produces enough current for the ADC to say there's a blip, regardless of how long this takes? Or enough energy within an unknown amount of time to reach the threshold, because it can dissipate?

The issue is that the maximum integration time is 600ms, and above 428x gain the noise created says there's light (counts) even when it's completely dark. So at really low levels this count is 1 or 0. I've never had it consistently be 0 for more than a few seconds at a time when it's been outside, so far. 

I've confirmed that at the integration time and gain most useful at low light (600ms/428x), when it's completely dark that there's no signal coming in by putting a logger in a box, in another box, in a drawer. During the 4 days it was logging there was no counts at all. 

So any counts coming in mean something, it's a matter of whether it's a useful something or not. So the question is whether it would be reasonable to take a mean of counts over a period like an hour and assume that it's somewhat accurate? 

Any help in understanding this is much appreciated. 
Thanks

gilshultz

I would suggest starting with this link: https://learn.adafruit.com/adafruit-tsl2591/overview and following it throuogh all the different pages.  As you proceed be sure you understand what you read as it builds as it goes.  I believe they also offer sample code to get you started.
This response is to help you get started in solving your problem, not solve it for you.
Good Luck & Have Fun!
Gil

Nazaar--

Thanks for the response. I've read through those datasheets a few times and haven't been able to find the answer to my questions. 

Looking at the actual sensor manufacturer datasheet it says this:

 "Upon completion of the conversion cycle, the conversion result is transferred to the Channel 0 and Channel 1 data registers, respectively. The transfers are double-buffered to ensure that the integrity of the data is maintained. After the transfer, the device automatically begins the next integration cycle."

So the integration time is integral to the sensor, and either the required number of photons hits the sensor to register during the set integration time, or if it doesn't get the required number it registers zero, and then the cycle starts again? 

This is where my knowledge of physics is even worse. So if it is registering alternatively 0 and 1, does that mean that the number of photons hitting the sensor is random enough that sometimes it's enough and sometimes it's not? 


davescodemusings

I've not used this particular light sensor, but I have used the one built into the Nano 33 BLE Sense. I think I can answer your questions based on experience with analog to digital conversion in general.

A lot of analog to digital conversion involves counting something over a time interval. In the case of a light sensor, it's counting photons over a fixed time period. The only way it will be accurate is if it's allowed to count for the entire time period. Think of what would happen if you asked it how many photons there were while it was still in the middle of counting. Your result would be skewed.

The double buffering (channel 0, channel 1) prevents this. In essence, you're not allowed to read directly from the counter (because it might be in the middle of counting.) What you're allowed to read is the result of the last complete count that's stored in another register.

This would be akin to me looking out a window and counting cars on the road to understand the volume of traffic. I count them in my head. At five minute intervals I write the number I have in my head on a piece of paper. You don't know what number is in my head, only what's on the paper. You also don't care what number is in my head, because the five minutes isn't up yet, so the result is wrong. You only care about the number on the paper.

Hopefully I understood your question correctly. If not, let me know.

I also have a tutorial on reading the Nano's ambient light sensor and making the value available via Bluetooth if you're interested.

Nazaar--

#4
Oct 15, 2020, 10:53 pm Last Edit: Oct 15, 2020, 10:57 pm by Nazaar--
Thanks for the response, it does help me understand.

So the question I'm left with is why is there variation in the counts, both in normal conditions and low light levels? Is the number of photons that hit the sensor variable (within a range determined by the light level), causing a variation in the count?

What I'm hoping is that the variation in counts is because the number of photons hitting is variable, and I can reduce/quantify this variation by taking a mean over a long time period, eg. if I've got 50 counts of zero, and 50 of 1, I can call it 0.5 over the 100 measurements. What I'm fearing is that the variation is something else and the counts at low light levels are unreliable due to factors beyond my control.

davescodemusings

I'm not sure about your photosensor, but the one built into the Nano gives a range of max and min with a typical value in the middle. It's on page 4 of the data sheet. Maybe the variation you're seeing is due to fluctuations within a similar min/max range? Though my experiments with the Nano's photosensor show predictable behavior when I shine a flashlight on it or move it closer to a window.

I also know that the sensor in the Nano has a function to determine if there's data ready. Like this little chunk of code:


Code: [Select]
if (APDS.colorAvailable()) {
  int red, green, blue, ambient;
  APDS.readColor(red, green, blue, ambient);
  irradiance = (float) ambient / 2360;
}


I've never tried to read a value without calling colorAvailable() first. If there's a similar function for your photosensor and you don't have it in your code, maybe it's a case of reading the data before it's ready and valid.

Nazaar--

I'm not sure about your photosensor, but the one built into the Nano gives a range of max and min with a typical value in the middle. It's on page 4 of the data sheet. Maybe the variation you're seeing is due to fluctuations within a similar min/max range? Though my experiments with the Nano's photosensor show predictable behavior when I shine a flashlight on it or move it closer to a window.
Thanks, I didn't see/understand that part in the sensor sheet before. It's got Low (0) and Max (20), but the typical field is blank. I'd hoped that with the test I did, leaving it in proper dark for 4 days, would show that the sensor doesn't show counts when it's properly dark, and that any positive count means something. 
I'll have a look into the libraries for the sensor re: the other part, it's definitely not in the main part of the code. 

davescodemusings

I took a look at the Adafruit info for your sensor. The part about gain and timing was interesting. If you look down the page just a bit, you'll see a list like this: TSL2591_INTEGRATIONTIME_100MS with the timings for the various gain settings. The way it's achieving better low-light sensitivity is by counting for a longer time period. (This is like leaving the shutter open longer on a camera to get a low-light photo.)

The longest was 600mS TSL2591_INTEGRATIONTIME_600MS. If your loop is trying to read values faster than 600mS in with this setting, you're going to run into those erroneous values, like in my car counting example, because you're reading before it's done counting. Maybe throw in a delay(1000); (or longer) to make sure you're giving it enough time to finish counting and come up with a valid count?


jremington

#8
Oct 17, 2020, 06:56 pm Last Edit: Oct 18, 2020, 12:25 am by jremington
Quote
why is there variation in the counts, both in normal conditions and low light levels
Counting inevitably shows statistical variations. For typical sorts of experiments, the expected error in a count value of N is +/- sqrt(N). Google "counting statistics" for more info.

In addition, all sensors add their own internal noise. For a light sensor, this is characterized in the data sheet as "dark current" or "dark counts", and should be experimentally determined, and subtracted from actual measurements.

It is statistically valid to report resultant negative counts, with the understanding that they are due to statistical fluctuations, and have no physical meaning.

Nazaar--

#9
Oct 18, 2020, 11:26 am Last Edit: Oct 18, 2020, 11:27 am by Nazaar--
Counting inevitably shows statistical variations. For typical sorts of experiments, the expected error in a count value of N is +/- sqrt(N). Google "counting statistics" for more info.

In addition, all sensors add their own internal noise. For a light sensor, this is characterized in the data sheet as "dark current" or "dark counts", and should be experimentally determined, and subtracted from actual measurements.

It is statistically valid to report resultant negative counts, with the understanding that they are due to statistical fluctuations, and have no physical meaning.
Thanks, this is really interesting.
RE: internal sensor noise, what do you think would qualify as a reasonable experiment to quantify it? The sensor sheet has dark values for 100ms integration time, and max gain, counts of 0 low, and 20 max, with the 'typical' field left blank, but I'm not using max gain because I found that it was too noisy; it was showing positive counts when I know it was dark. Instead I put the logger in a box, in another box, in a drawer, and left it logging on 600ms integration and 'high' gain, for 4 days. So I've ended up with about 345k readings. All of them are zero.
I'll look into the counting stats, thanks.
Quote
The longest was 600mS TSL2591_INTEGRATIONTIME_600MS. If your loop is trying to read values faster than 600mS in with this setting, you're going to run into those erroneous values, like in my car counting example, because you're reading before it's done counting. Maybe throw in a delay(1000); (or longer) to make sure you're giving it enough time to finish counting and come up with a valid count?
Thanks, I've played around with gain and integration time a fair bit, I've put together some bits that have it changed time and gain depending on how much light there is, and have it running on 600/high when it's low light, but haven't thought of whether it's having trouble like that. The code I'm using is using the adafruit sample code as a framework, and add bits into that, so I'd hope that they knew what they're doing. That said, their library has errors in it, and they know and haven't corrected them, so maybe not! I might try running two loggers in parallel, one with a 1000 delay in it, and one without, put them next to each other, and compare the results. Do you think that would be good enough see a difference?

jremington

#10
Oct 18, 2020, 04:02 pm Last Edit: Oct 18, 2020, 07:16 pm by jremington
Quote
a reasonable experiment to quantify it?
Take readings in absolute darkness, as you have done.

Quote
found that it was too noisy; it was showing positive counts when I know it was dark
It is a mistake to assume that it is "too noisy". If the gain is set so low that you cannot see the dark counts, you will not be able to measure very dim illumination. When using high gain settings, subtract the separately measured dark counts from your actual measurements. 

Keep in mind that:

1. There are several sources of noise: photodiode noise and noise in the amplifier/ADC. Changes in the gain will affect some sources more than others.

2. Electronic noise is strongly temperature dependent.

Nazaar--

Take readings in absolute darkness, as you have done.
It is a mistake to assume that it is "too noisy". If the gain is set so low that you cannot see the dark counts, you will not be able to measure very dim illumination. When using high gain settings, subtract the separately measured dark counts from your actual measurements.  

Keep in mind that:

1. There are several sources of noise: photodiode noise and noise in the amplifier/ADC. Changes in the gain will affect some sources more than others.

2. Electronic noise is strongly temperature dependent.
Ok, so the next step should be to run some experiments with max gain to quantify the dark noise. Once I've got a good amount of those, do some runs in low light to see the results, then subtract the dark counts from them to see what it gets. 
RE: temperature dependence, it sounds like it's worth running some dark noise experiments in different temperatures to try and quantify the difference?
Thanks very much, this has been incredibly helpful, I really appreciate it. 

jremington

#12
Oct 19, 2020, 04:00 am Last Edit: Oct 19, 2020, 04:01 am by jremington
Serious experimentalists cool photodiodes and CCD detectors well below 0 Celsius.

Nazaar--

The water they'll be put in shouldn't get colder than 8 degrees C, but terrestrial people might use it. I'll put it in the freezer, and heat the room up a bit as well. I guess it'll chew through batteries pretty fast in the cold. 

I've started a 600ms/max gain logger, with temperature recorded as well. Very curious to see what happens.

Thanks

Go Up