Linear CCD Detector with Reproducible, Linear Response

It turns out that some commercial spectrometers are not very linear in their response. And that means intensities, even relative intensities, reported with the expensive commercial instruments might not be very reproducible.

But, we can do better.

The commercial instrument:

The following are fluorescent lamp spectra collected with one of the most popular commercial CCD spectrometers at different exposure times. We divide intensity by exposure time. So, all of those curves should lay on top of each other. To make things more clear, the third graph shows the ratios of the heights of three of the peaks from the spectra.

What we see with this $4,000 instrument, is that the peaks heights are not proportional to the exposure time, and even the height of one peak compared to another changes when you change the exposure time.

That is really not very good. How do you rely on something like that to publish your research?


The new instrument:

And now, here is the data from the instrument that I just recently designed. BOM is about $150, optics around $200 from Thorlabs or ebay. The processor is a Teensy 4.0

As you can see, the curves do lay over each other, except where it saturates at longer exposure times. And the relative peak heights are pretty flat, again until one of the peaks saturates. That is what you want if you want to be report spectra when you publish your research.

The how and why of this is very interesting, and I plan to return to the topic and add some explanation in due time.

The design files and firmware should be on github soon.

Meanwhile, here is a picture of the first board set (sensor and controller), and assembly of the spectrometer. The resolution you see in the above is with a 200um slit. That was part of the spectrometer design, but that is a different subject. There are some minor updates since the first boards. Those will be in the github repo.

The $4K price tag sounds like an Ocean Optics spec. If so, which model?

It is tempting, but we live in a litigious society.

Message me if you like, with the proviso that it might be sunday or monday before i can answer.

Never mind.

If a commercial instrument does not meet the advertised specifications, it should be repaired or replaced under warranty. I can't imagine being sued if the alleged defective behavior is experimentally verifiable.

It's systematic, it is built into their design.

They advertise that the instrument is linear for absorbance measurements.

(Note that log $x^c = c log x$.)

So, it seems a good guess that they know, and probably other companies know too and have not been able to solve it either.

You have given Arduino forum members no reason to believe that your singular experience with a mystery instrument is representative of faulty product design or defective implementation.

If it does not meet advertised specifications, send it back.

Okay, as we scientists say, try it yourself.

Start with sufficient intensity to saturate a few peaks, and then back off with a ND filter wheel or by dialing down the exposure time until you are down to noise. Then graph it the way I did.

So far I have confirmed the effect in two of two commercial spectrometers that I own though both are by the same company, and one very expensive imaging CCD spectrometer in a collaborator's lab from a different company.

Knowing why it happens, I am guessing it will be pretty common.

Apologies for not going into detail just yet, I want to write it up first.

I'll say though that it took a few months to work it out, including lengthy discussions with device physicists at the company that makes the sensor chip and collaboration with another instrumentation physicist.

WITH WHAT? Surely you are joking.

I tire of this, so have fun.

P/S re "if it does not meet advertised specifications, send it back."

You might wind up sending back a lot of spectrometers. My guess is that almost all of them have this same issue.

It is not a singular experience. The one I showed is actually one of the better ones, others are worse.

In other words, you do not actually own a spectrometer or have access to one?

Have you ever used a spectrometer?

It seems like the answer is no. You have no idea about them and you have no basis to say whether my experience is singular or not. You are just polluting somebody else's post.

What was reported at the top of this post is absolutely remarkable. It is quite possibly the first ever lccd instrument with proper linear response and reproducibility.

Everybody who uses these knows about this issue and will recognize the importance of this result. And it is being made available as open science. It will be posted soon with the design files and codes.

And this was not done by amateurs. This result was the product of hard work by professionals each with doctorates in the subject and 50 years of experience.