Bias frames and CMOS cameras (scaled and unscaled darks)

Affiliation
British Astronomical Association, Variable Star Section (BAA-VSS)
Sun, 10/29/2023 - 21:33

I’m clear about flats and darks thanks to posts on here, but confused about bias.
I have searched online extensively for info on this and found rather mixed messages and opinions.

I have read AAVSO CCD/CMOS Photometry manual which states use zero exposure or as short as you can.

Some contributors say there are problems with very short exposures on CMOS cameras and bias frames are inconsistent.

Zero exposure problem aside, I know how to do them, but do I need them?

Manual states "You must make and use bias frames if you are using the scaled dark calibration technique; if you are using unscaled darks, bias frames are optional."

Given that I am trying to start simply should I use scaled or unscaled as this trumps the bias question.

You may sense that I am getting bogged down in the terms.

Help?
Kevin

Affiliation
American Association of Variable Star Observers (AAVSO)
Confused here too!

Not sure what scaling the CCD manual is doing. Maybe temperature?

My notion is that the zero time exposure measures the noise from the Analog Amplifier(s)  and the Analog to Digital Converter and any other associated electronics.

Some of the noise is thermal noise and gets to be more noisy at higher temperatures. That is why you take bias ( and dark ) frames at the temperature at which you expect to run the camera. Thermal noise is random, so you average about a dozen images and get the master bias. the master bias gets subtracted from the image taken at or near (scaling?) the same temperature.

Some of the noises are digitizer noise, switching power supply noise, amplifier noise, passive components noise (like thermal in the resistors). The camera has to clock the pixels to the amplifier. If a pixel has to be clocked more times to arrive at the amplifier, the thermal noise increases just because it takes more time.  The faster the clock, the noisier the rising and falling edges may be. This is some of the read noise that you see a specification for.

This kind of noise is not so random but gets subtracted off when the bias is subtracted during standard image calibration.

All this noise gets to be a mind-numbing cacophony.  Experimental physicists spend a whole lot of time understanding every component of it for their measurements.

The easy, mind-soothing, and every-day practical way to minimize it is to take bias frames and use them to calibrate every image.

A noise source that I contemplate for a future telescope is the 4GHz computer that I want to put in the observatory. As the camera swings around the pedestal, it's distance to the computer will change and the noise level at the camera could easily double or halve depending on position. How can a single master bias fix that?

The hope is that the camera electronics  makes so much noise that it doesn't even see the computer.

RayTRE

Affiliation
Variable Stars South (VSS)
Try unscaled darks first

Kevin,

Since you are trying to start simply, try unscaled darks first: same exposure and gain setting as your lights.

Roy

Affiliation
American Association of Variable Star Observers (AAVSO)
Bias frames and CMOS cameras (scaled and unscaled darks)

Kevin:

As the person who put that sentence into the manual, let me first apologize for making it confusing. Then, let me make a recommendation:

As you get started, use unscaled dark frames. That is, use dark frames made at the same exposure time (and same sensor temperature) as your "science" images. This way you don't have to worry about scaling (by exposure time), and you don't have to worry about bias frames.

What you'll be doing is creating a dark frame that includes both dark current and the bias signal. When you subtract that from your science image (which contains sky/star signal + dark current + bias signal), you'll just be left with the sky/star signal, which is what you want.

Just make sure that you combine multiple dark frames together into your master dark, instead of using just a single dark frame. You want to average multiple darks together to reduce the amount of residual noise in your final, calibrated image. The downside to this approach is that you need to make multiple master dark frames if you use multiple exposure times for your science images, and making multiple master dark frames (each of which is the average of multiple raw dark frames) takes time. Despite the additional time needed, I've been doing photometry for 20+ years, and this is exactly what I do every night. I do use a limited "palette" of exposure times (60 seconds, 30 seconds, 10 seconds, 5 seconds), and that limits the number of master darks that I need to create each night. If you can start off with a smaller palette (maybe just 30 second exposures??), that makes making master dark frames that much easier.

- Mark M

Affiliation
American Association of Variable Star Observers (AAVSO)
Dark Library?

Hello! With my CCD, I've created a library of darks, which I update once a year. With my cooled ST-402, I use temps from 5 degrees to minus 15 degrees in 5 degree steps and exposures ranging from 1 second to 10 second, 15, 20, 30, 345, 1 minute, 2min, 3min, 4min, and 5 min.

    I can create the library during the day or on cloudy evenings. I make the program with SGPro and let it go. I take 50 exposures in each and then median combine them.

    It takes about an hour to make all the dark frames, but then I don't have to worry about scaling a dark each time I need one.

    I'll probably do the same when I switch to CMOS. As other's have noted, there has been concern raised about scaling darks with CMOS sensors. And, since a dark already includes the bias, it seems like a way to avoid introducing a new error into my work flow. Best regards.

Mike

Affiliation
British Astronomical Association, Variable Star Section (BAA-VSS)
Thanks

Thanks Mark,

Apologies for slow response. I have a lot of posts on here and some get pushed to the back of admin.

I have just completed the excellent VPhot course which of course starts with calibrated images and teaches photometry.

On it we were taught briefly about some aspects of calibration. How images are calibrated using darks, flats and bias frames.

So this is the route I have more or less followed.

I have a CMOS camera which is (I'm advised) not suited to zero or very short exposures used for bias frames so these are

substituted with flat darks.

The software (I am using ASTAP at present) works its magic with the lights, darks, flats and flat darks to output an apparently calibrated image although I'm not too sure what goes on under the bonnet.

Are the darks, flats and flat darks literally subtracted pixel count by pixel count from the lights?

I take your point about actually getting to do some photometry. Had my system for a month and not actually submitted an observation yet.

The jargon and technicalities can be energy sapping. But enjoying the learning and getting there. Great support on here.

Regards

Kevin

Affiliation
American Association of Variable Star Observers (AAVSO)
Bias and Dark frames

Kevin,

I agree with the recommendations that for the sake of simplicity you use libraries of dark frames which match the exposure times and temperatures of your science images.  As Mark explained, if your science images have the same exposure times, and are taken at the same temperature as the dark frames, there is no need for separate bias frames.  The bias information is contained in the dark frames.  This simplifies image calibration.

For my CCD camera I routinely keep master darks for 120s, 60s, and 30s all taken at the same temperature.   My camera can cool to -20 C year round, so I use that temperature for all my calibration and science images.  I renew the master darks about every 3 months with the changing of the seasons.

Phil     

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Scaling Darks

Phil, Mark, Mike, Roy, Ray:

I find it interesting that you all choose to select darks of only a few fixed exposures rather than scale the darks. I don't think this has ever been regarded as necessary for CCD images and I consider it necessary only for CMOS cameras with amplifier glow which does not scale well.

For Kevin's sake and to support a broader understanding of this proposed practice, can you provide some concrete data to support why you feel this is necessary?

IMHO, collecting CMOS bias images with a very short bias exposure (if not zero) is not really that big a deal and allows one to avoid the need to use only a few science image exposures. Maxim, for example, supports maintaining calibration image libraries and applying them very easy?

I'd really like to see this issue resolved once and for all?

Ken

Affiliation
American Association of Variable Star Observers (AAVSO)
Re: Scaling Darks

Hi, Ken:
No, I'm not saying that I think it's necessary. It's one alternative, and may be attractive to newcomers to photometry because it is simpler than using scaled darks with bias frames (by reducing the number of different calibration frames from 3 to 2). I'm an advocate of promoting the simplest possible initial experiences in photometry.

Obviously, there's nothing wrong with using scaled darks with bias frames, and you're right that there are benefits to doing so. And the Photometry Guide acknowledges both techniques. And most software packages support both, although the terminology used is not always exactly the same as you go from vendor to vendor, introducing a potential source of confusion for someone who hasn't yet built a mental model of what's going on inside the software package that they're using.

I'm just trying to gently suggest that beginners can take advantage of a simplification.

- Mark

Affiliation
Variable Stars South (VSS)
Scaled or unscaled darks?

Ken,

I use unscaled darks just because it is simple, easily understandable, and easy to do. I don't stick to just a few exposures. I take darks, when necessary, of the same duration as my lights. I do not have a specific, scientific reason for working this way.

Roy

Scaled or unscaled darks?

Hi All,

I’d be cautious about using scaled darks with CMOS cameras, a little experimentation is called for. CMOS technology is used in DSLR cameras and when I revised the DSLR Observing Manual back in 2016 it was clear that dark current did not scale linearly with exposure time (at least for the Canon DSLR cameras I used back then).

Take a look at the Dark Current section in “Appendix A: DSLR Camera Testing” on page 97 of the DSLR manual for a description. I do not know if the same applies to cooled astronomical CMOS cameras, but it should be checked before relying on scaled darks.

I suggest carrying out all the tests in Appendix A to characterise your CMOS camera. Some of the terminology will differ, i.e., “ISO” is used with DSLR where “gain” is used for astronomical CMOS cameras.

I will test a ZWO ASI2600mm-pro cooled monochrome CMOS camera and report back on this forum thread what I find out about the dark current behaviour. Cheers,

Mark

Affiliation
American Association of Variable Star Observers (AAVSO)
Scaling Darks

Ken,

I don't think anyone here was proposing that simple dark subtraction, rather than dark scaling, is necessary or even preferable either for CCD or CMOS.  I'd say we were just responding to Kevin's request for simplicity.   If we can help Kevin around this (temporary) roadblock he may be able to start doing photometry.  Understanding dark scaling can wait until he has more experience.  Then he can make is own choice on which method to use.

I do not disagree with your comments on collecting bias images with CMOS cameras, but it seems to me the current perceived wisdom is that CMOS should use simple dark subtraction.  I would like to see more discussion on dark/bias calibration of CMOS images, maybe in a new photometry forum topic.

Q:  What is the dark/bias calibration method used with the AAVSOnet telescopes?

Phil

ASI2600MM-Pro CMOS camera dark current

Hi All,

I performed tests on a ASI2600MM-Pro CMOS camera to check its dark current behaviour and other parameters. The procedure is described in Richard Berry and James Burnell’s excellent “The Handbook of Astronomical Image Processing” (second edition). This is the basis of the procedure in Appendix A of the DSLR manual.

These tests require two bias frames, several dark frames with exposure times ranging from a few seconds to several minutes, and several flat frames (stable light source) with exposure times ranging from a few second up to near saturation.

I used gain setting = 0 (arbitrary units), offset = 10, temperature = -10 deg C, which are what I use for my photometry.

The measured true gain is 0.802 e/ADU and readout noise is 3.4 e rms. Both are very close to the camera’s specifications.

Dark current was measured at several exposure times:

10 sec = 0.161 e/pixel/sec

20 sec = 0.081 e/pixel/sec

40 sec = 0.041 e/pixel/sec

80 sec = 0.020 e/pixel/sec

160 sec = 0.010 e/pixel/sec

For scaled darks to be used it is necessary for the dark current measured in e/pixel/sec to be constant for all exposure times, as seen in CCD cameras. However, as you can see from the figures above, this is definitely not the case for the ASI2600MM-Pro CMOS camera so it is NOT valid to use scaled dark frames – you must use darks of the same exposure time as the light frames.

Other CMOS camera models may behave differently, you should check your particular camera. Cheers,

Mark

Affiliation
American Association of Variable Star Observers (AAVSO)
Thanks but suspicious?

Mark:

Thanks very much for conducting this calculation. I also use Berry&Burnell for these calculations. Great resource.

However, something doesn't quite look right considering the results of similar analyses for the AAVSONet CMOS cameras including a similar QHY268?

It is interesting that for each of your very linear data points that if you multiply by the exposure the result is 1.6 e/pix. Looks like a read noise value?

I'll try to reproduce the process with some cal images from a CMOS and my own CCD camera. For the moment, still scratching my head! ;-)

Ken

Affiliation
Variable Stars South (VSS)
Dark current test ZWO ASI294MM

Mark, Ken and all,

I have just done dark current testing on my 12 bit CMOS camera. Binning was 1x1 (it is described as 'unlocked bin mode' for this camera, and in this mode the camera is 12 bit), Gain was 120 units (1e-/ADU) and offset 50. I used Region of Interest (ROI) of 1/4, simply because I regularly use ROI if the field can fit within it. For lights, I normally use ROI 1/2.

I followed the description published some years ago by Craig Stark which follows. The results are after that.

"On many cameras, dark current can be measured very easily. If you've got a cooled camera,
all that is needed is to measure the mean of a bias frame and subtract this from the mean of a
long dark frame. In the Atik 314L+ I have on the bench here, the mean of a bias frame is
232.5 and the mean of a 10-minute dark frame is 234.2. That means that in a 10 minutes of
exposure, my average intensity went up by 1.7 ADU or 0.46 electrons. Typically, this is
specified as electrons per second, so we divide this by the number of seconds in this interval
(600 seconds) and get 0.00076 e-/second. This is a very low number (and is why I've often
said that regulated cooling and the use of dark frames is really unnecessary on these Sony
sensors - a cooled dark frame is almost exactly the same as a bias frame).

If your camera isn't cooled or if you think there might be something odd going on (or if you just
want a bit cleaner estimate of the dark current), you can do the same thing you did in coming
up with the system gain. In a spreadsheet, make one column for the exposure time and
another column for the mean value of the dark frame at that time. Plot time on the x-axis and
the dark current value on the y-axis and again do a linear fit. The data should fall on a line. If
they don't something is odd as doubling the exposure duration should double the number of
photons from dark current being recorded."

Frame    Exp (Sec)    Mean ADU    Dark-Bias (Means)    Dark Current (e-/sec)
Bias    0.001    3127.442        
Dark    60    3127.529    0.087    0.0014
Dark    120    3128.834    1.392    0.0116
Dark    240    3131.923    4.481    0.0187
Dark    480    3136.354    8.912    0.0186
 

There are only a few exposures, but dark current plateaus at 240 sec, and even up to that is not linear.

Roy

Affiliation
American Association of Variable Star Observers (AAVSO)
A few questions

Roy:

I had a few questions:

You mentioned a gain of 1 e/adu but then equated 1.7 ADU to 0.46 e? Edit - Ahh, you had two cameras tested.

Do you notice any amplifier glow in the corner of your images, without smaller ROI?

What happens if you use Median rather than Mean? Were the means for the entire image as opposed to ROI?

Certainly not much thermal dark noise (9 adu in 480 sec)!

Ken

PS: I'll try to run my CCD numbers tomorrow.

Affiliation
Variable Stars South (VSS)
Ken,

The 1.7 ADU etc was in…

Ken,

The 1.7 ADU etc was in Craig Stark's article.

I think I have a small amount of amp glow. It would be well outside the ROI. Have stopped thinking about it in routine work. Will need to check on my PC when I get a few minutes.

I'll re-run with medians and post when I can find time.

The means were on the ROI 1/4 images.

Roy

Affiliation
Variable Stars South (VSS)
Further dark current test, ZWO ASI294MM camera

I have repeated the dark current test for the ZWO ASI294MM camera, but this time using a gain of 81 units (to give a system gain of about 1.5e-/ADU). The results are very different from those I posted about 9 and a half hours ago. Note that, at a gain of 120 units (system gain 1e-/ADU) the readout mode of the camera changes. Since most of my images are taken at a gain of less than 120 (a system gain of >1e-/ADU) I thought that the repeated test should reflect this. I also changed the ROI to 1/2 (from 1/4 in the previous test), which is what I use for most of my images.

The results below effectively show what was done. Take one bias frame, and several dark frames at different exposures. Take the mean of the bias frame and the mean of each dark frame. Subtract the mean of the bias frame from the mean of each dark frame. Divide each result by the exposure in seconds, to get the dark current in electrons per second for each dark frame.

In a spreadsheet draw up four columns with the following headings: Exposure time in seconds; Mean ADU of dark; Mean of dark frame minus mean of bias frame; Dark current (e-/sec).

It is instructive to plot each of the second, third and fourth columns (y axis) against the exposure time (x axis).

If you plot the results below, it is quite clear that, for the ASI294MM camera at the settings used, dark current does not increase linearly with exposure (plots of mean dark ADU, and mean of dark frame minus mean of bias frame against exposure in seconds). It is equally clear that the dark current in e-/sec is not constant.

Conclusion: I will continue to use only darks and flats for callibration, with the darks always taken at the same exposure and gain as the lights.

Ken:

I wasn't able to repeat the test using medians instead of means. 

I've used ROI with the ASI294 camera now for so long I can't find any 'full frame' images to answer your previous question about amp glow.


Dark Current Test #2 ZWO ASI 294        
        
Gain    81    (system gain 1.5e-/ADU)
Offset    50    
Temp    -10    C
Binning    1   

Bias    
Exp Sec    Mean ADU
0.001    3154.126

Dark            
Exp Sec    Mean ADU    Dark-Bias (Means)    Dark Current (e-/sec)
60    3156.785    2.659    0.0443
120    3156.337    2.211    0.0184
180    3157.652    3.526    0.0196
240    3158.781    4.655    0.0194
300    3159.514    5.388    0.0180
360    3159.481    5.355    0.0149
420    3159.093    4.967    0.0118
480    3159.251    5.125    0.0107
 

Roy

Thanks but suspicious?

Hi Ken,

Yes, I'd noticed the (nearly) constant value of e/pix for the different exposure times. It looks like there has been some processing of images to produce that value. The standard deviation of dark frames does increase linearly with exposure time but "dark current" does not.

I am keen to see results of dark current tests on AAVSOnet CMOS cameras as well as your CCD camera. When I get a chance, I'll test my CCD cameras as well.

It is possible that different CMOS models have different dark current behaviour. 

Cheers,

Mark

Affiliation
Variable Stars South (VSS)
Dark current in the ASI294MM camera - a more careful look

My previous ASI294MM dark current tests should be ignored. This one is, I think, more informative.

Since my flats are exposed for less than 1 second, shorter exposures were included. The procedure is also modified from the previous post. Eighty (80) bias frames were average stacked to create a master bias, the mean of the master bias was subtracted from the mean of each dark, and the latter value was divided by the exposure in seconds to calculate the dark current in e-/sec.

Exposures of the darks ranged from 0.1 sec to 480 seconds. The first four exposures were all 0.1 sec, because I have seen different mean ADUs in the first frame of a series of darks when the exposures were short.

It is necessary to import the following table into a spreadsheet and plot each of the 2nd, 3rd and 4th columns against exposure time in seconds to see what is happening. (Plots of the 2nd and 3rd columns are of course similar, but just with markedly different y axis values). It is also necessary to copy each plot, and zoom in to the data for exposures up to about 15-20 seconds.

I was surprised to find that subtracting the master bias from the darks yielded positive values for exposures of 0.1 and 0.2 seconds, negative values for exposures from 0.4 to 20 seconds, and positive values again for exposures of >20 seconds.

The dark current is nearly (but not quite) constant for exposures of 20 sec and above. Thus, dark current increases in a linear manner (approximately) with exposure time for exposures of 20 seconds and more. Results may differ with other gain settings and for other cameras.

Roy

Gain    81    1.5e-/ADU    
Offset    50        
Temp    -10    C    
Binning    1        
            
Master Bias (Average Stack of 80 Frames)            
Exp Sec    Mean ADU        
0.001    3154.632        
            
Dark            
Exp Sec    Mean ADU Dark    Dark-Master Bias    Dark Current (e-/sec)
0.1    3154.816    0.184    1.840
0.1    3154.669    0.037    0.370
0.1    3154.807    0.175    1.750
0.1    3154.775    0.143    1.430
0.2    3154.888    0.256    1.280
0.4    3153.484    -1.148    -2.870
0.6    3153.480    -1.152    -1.920
0.8    3153.491    -1.141    -1.426
1    3153.457    -1.175    -1.175
2    3153.469    -1.163    -0.582
3    3153.595    -1.037    -0.346
4    3153.620    -1.012    -0.253
5    3153.704    -0.928    -0.186
6    3153.737    -0.895    -0.149
7    3153.890    -0.742    -0.106
8    3153.817    -0.815    -0.102
9    3153.803    -0.829    -0.092
10    3153.929    -0.703    -0.070
15    3154.208    -0.424    -0.028
20    3154.628    -0.004    0.000
30    3155.268    0.636    0.021
40    3155.965    1.333    0.033
50    3156.674    2.042    0.041
60    3157.376    2.744    0.046
120    3161.877    7.245    0.060
180    3166.633    12.001    0.067
240    3171.546    16.914    0.070
300    3176.327    21.695    0.072
360    3181.309    26.677    0.074
420    3186.297    31.665    0.075
480    3191.138    36.506    0.076
 

Affiliation
American Association of Variable Star Observers (AAVSO)
More Uncertain?

Roy, Mark et al;

I promised to run dark current calculations for my CCD (STL6303). Just adds confusion to the process? I used the Camera Characterization Tool in AIP4Win. It uses a raw bias image and a raw 'Long' Dark Image to calculate dark current. I highlighted 'Long' because I think that makes a big difference since a raw dark image includes read noise (bias) which would only be subtracted from a Master Dark? If the table formatting doesn't get messed up, it shows 3 separate dark current calculations with 3 individual images at several dark exposures. I typically collect tens of dark images at each exposure to get a good average master, but I just selected three separate raw images at each exposure for this analysis.

Data follows:

Dark Exposure (sec)    Dark Current 1    Dark Current 2    Dark Current 3

300                               0.048761              0.043782            0.050226

180                               0.05187                0.040482            0.058658

120                               0.081592              0.04727              0.083239

60                                -0.067668             -0.07848              0.051354

30                                 0.395422               0.07148             0.802434

10                                -0.289736              -0.210633           0.287411

It appears that at longer dark exposures, dark current (e/pix/sec) is reasonably small and uniform (lots of random error?). At shorter exposures, dark current is 'bonkers' (negative and/or large)?? Does bias noise cause all of this in short exposures? How can you have negative dark current?

We need help? I need to re-read Berry&Burnell Chapter 8 more carefully. Doing something wrong?

Ken

Where is Richard?

Hi Ken and all,

I'm following all of these experiments and while interesting they have not been definitive.  I'm sure we could pull in Richard Berry at this point? He is a current board member, and could probably put this whole question to bed.

Just an idea.

Andrew

 

Affiliation
Variable Stars South (VSS)
Flat darks not necessary for the ASI294MM camera?

I'm sure Andrew is right, but it is I think informative to keep looking and thinking about the results.

In both Ken's results and mine, the negative dark currents occur at shorter exposures, Ken's at 60 sec and mine up to 20 sec (except for those exposures <1sec). Different cameras entirely, of course.

Ken limited his exposures to 'longer' darks, and Craig Stark specifies a long exposure dark to determine dark current.

But at short exposures, are we worrying too much about anomalies? Looking just at the first 20 results in the first column (average ADUs of the darks) in the table of my most recent results shows:

Mean 3154.816, SD 0.539, Minimum 3153.457, Maximum 3154.888, Range 1.431. These are ADUs per pixel.

If we are talking about flat darks, and subtracting a master flat dark from a short exposure flat (a few seconds or even less than one second), from the above there will be less than 2 ADUs per pixel difference in flat darks across the entire range of exposures from 0.1 sec to 20 sec in my camera at the tested gain setting. That is negligible, given that a typical flat will have 30,000 or more ADUs per pixel.

Conclusion: Flat darks are unnecessary for flats in my camera.

Roy

 

Affiliation
American Association of Variable Star Observers (AAVSO)
I want to prove this with data and know why?

Roy;

I agree with your conclusion that the correction is negligible but that is not what I want to know. Current CMOS chips have very low dark current so some cal images lead to negligible corrections and can be ignored? I want to know why some CMOS cameras reportedly do not scale well. Is it due to amplifier glow or something else? Is it true any more? Why can't we confirm by measurement that dark current is constant and thus darks can be scaled? Or vice versa?

I find it interesting that both Stark & Berry note that dark current should be measured with a 'long' dark. Why? To avoid the problems we are seeing at short dark exposures? Are other phenomena like read noise and other (non-thermal) noise not accounted for correctly with short exposure darks? How in the heck can you get a negative dark current? Isn't this physically impossible? Is it an offset problem?

I did not find an explanation of any of this in Berry&Burnell, unless I missed a fine detail. As I said originally, I want to know the answer once and for all. If that is possible?

My software (Maxim) and others like AIJ, does calibration fully, whether it is small or not! If they can easily do this with no extra steps by me, I'm going to do the full 'best' correction until proven otherwise! I don't have a CMOS camera yet and won't until it dies. Who/what goes first is the question. ;-)

Ken

Affiliation
Variable Stars South (VSS)
Ken,

I don't know the…

Ken,

I don't know the answers to most of your questions. And if I think I might know an answer maybe I'm not correct.

Internet searches on these sorts of things usually bring up all sorts of discussions by amateur astronomers on various web sites. It is difficult to know what is authoritative. That is why I like testing things.

Arne's and Richard's knowledge would be valuable here.

From a practical point of view, this testing has confirmed my view, for my own workflow, that I don't currently need flat darks, that darks should be taken at the same exposure and gain as the lights, and that bias frames are not necessary since scaling of darks is not used.

This may not be the best workflow for other observers or other setups.

Roy

Affiliation
Variable Stars South (VSS)
Scaling of darks with the ASI294MM camera and AstroimageJ?

For those who are not exasperated by this topic, and with a little spreadsheeting knowledge and the inclination and time, there is I think an interesting exercise that can be done on dark current from the tabulated data in my post of Mon 11/13/2023 - 15:47.

You will need to use a dark current value of 0.076 e-/pixel/sec (the value for my camera at an exposure of 480 sec, gain of about 1.5 e-/ADU and offset of 50) and a mean value for the master bias of 3154.632 (again, from my camera).
 
Create a blank table in a spreadsheet and copy the first two columns of data from the post noted above. These are exposures of the darks in seconds, and the mean ADU for each dark. Add a third column, Dark-master bias reverse calculated. For each cell in this column, multiply the dark current at an exposure of 480 sec (i.e., 0.076) by the mean ADU of each dark. The formula in row 2 would be =A2*0.076.

The last calculation comes about because dark current = (dark-bias)/exposure in seconds. Rearranging this equation yields dark-bias = dark current * exposure.

Add a fourth column, headed Means of darks calculated. In each cell enter the formula that adds the dark-master bias reverse calculated to the mean ADU of the bias. For example, in (say) cell D2 the formula should be =C2+3154.632. Copy that formula into all the cells in column D of the table. I think this is the theoretical mean of the dark, resulting from scaling of the master dark.

Finally, add a fifth column headed something like Delta means of darks. In each cell of column E subtract the corresponding mean of the dark measured from the mean of the dark calculated. Thus, in cell E2 the formula should be =D2-B2. Copy the formula into the other cells in the column.

The maximum value in column E should be just under 2. If my logic and math are correct, the values in column E reflect the difference between the actual (measured) mean ADUs of the darks and the values that might be calculated by scaling the master dark (defined here as the 480 sec dark from my camera).

Since the values in column E are quite small, it seems to me that scaling of long exposure darks may indeed by done with the ASI294MM camera. I'll try it with AstroimageJ when I can. See table below, reflecting the description above.

Ken, none of this answers your questions.

Roy

Exp Sec, Means of Darks Measured, Dark-Master Bias Reverse Calculated, Means of Darks Calculated, Delta Means of Darks
0.1    3154.816    0.008    3154.640    -0.176
0.1    3154.669    0.008    3154.640    -0.029
0.1    3154.807    0.008    3154.640    -0.167
0.1    3154.775    0.008    3154.640    -0.135
0.2    3154.888    0.015    3154.647    -0.241
0.4    3153.484    0.030    3154.662    1.178
0.6    3153.480    0.046    3154.678    1.198
0.8    3153.491    0.061    3154.693    1.202
1    3153.457    0.076    3154.708    1.251
2    3153.469    0.152    3154.784    1.315
3    3153.595    0.228    3154.860    1.265
4    3153.620    0.304    3154.936    1.316
5    3153.704    0.380    3155.012    1.308
6    3153.737    0.456    3155.088    1.351
7    3153.890    0.532    3155.164    1.274
8    3153.817    0.608    3155.240    1.423
9    3153.803    0.684    3155.316    1.513
10    3153.929    0.760    3155.392    1.463
15    3154.208    1.140    3155.772    1.564
20    3154.628    1.520    3156.152    1.524
30    3155.268    2.280    3156.912    1.644
40    3155.965    3.040    3157.672    1.707
50    3156.674    3.800    3158.432    1.758
60    3157.376    4.560    3159.192    1.816
120    3161.877    9.120    3163.752    1.875
180    3166.633    13.680    3168.312    1.679
240    3171.546    18.240    3172.872    1.326
300    3176.327    22.800    3177.432    1.105
360    3181.309    27.360    3181.992    0.683
420    3186.297    31.920    3186.552    0.255
480    3191.138    36.480    3191.112    -0.026
 

Affiliation
Variable Stars South (VSS)
ZWO ASI294MM camera: successful callibration with scaled darks

Target: DX Tuc. An EW eclipsing binary, mag range 9.36 - 9.68 V, B-V ~ 0.54, period 0.37711010 d.
Check: HD 224952, V=9.235, B-V=1.081
Comp: HD 224685, V=9.224, B-V=0.698

352 images were taken over 6hr 23min, with an ASI294MM camera through a V filter and a 200mm f/2.8 Canon lens at full aperture. Exposures using ROI (region of interest) of 1/2 were 60 sec with a cadence of 64 sec, gain (random units) 41, system gain 2.5e-/ADU, offset 50.

Ten flats comprising images taken of a dimmed white screen (a small tablet) immediately in front of the lens were exposed for 0.58 sec. Ten darks for standard callibration were taken at the same exposure as the lights. Ten darks (for scaling) were taken at an exposure of 480 sec. The gain and offset for the flats and darks were the same as for the lights. 

(Edit) Eighty bias frames were taken at an exposure of 0.001 sec, and the same gain as offset as the lights.

Callibration and photometry was performed using AstroimageJ. In the CCD Data Processor module under dark subtraction, "scale" was selected but not "deBias".

A representative callibrated image and the light curve plotting non-transformed V magnitudes can be seen by (edit) copying and pasting the following links in your browser (I thought they would display as clickable links, but apparently not). (Edit) select each link, then right click it, then choose the option to go to the link.
https://www.flickr.com/photos/199603914@N08/53335762710/in/dateposted-public/
https://www.flickr.com/photos/199603914@N08/53332317537/in/dateposted-public/

The following statistics were determined:

Callibration; Ave Mag of Var; SD of Mag of Var;    Ave Mag of Check; SD of Mag of Check
No callibration    9.549    0.092    9.288    0.009
Standard call    9.548    0.092    9.279    0.008
Scaled Darks    9.548    0.092    9.279    0.008

Conclusion: it seems possible to employ either standard callibration (darks and flats only) or callibration with bias frames, flats and scaled darks with the ASI294MM camera.

Roy

Affiliation
American Association of Variable Star Observers (AAVSO)
Great Investigation!

Roy:

Thanks for putting the effort into investigating this issue. Your conclusion as to the equality of the two calibration procedures on your CMOS camera is very good to hear.

I would however quibble mainly with your use of the term 'standard' when referring to your normal calibration procedure using darks and lights of equal length, rather than the other normal calibration procedure using bias subtraction and scaled darks. Not sure that 'standard' is perfect name for either procedure.  ;-)

Other observations:

1. Since you did use bias subtraction in one procedure, I assume that you did take some bias images? Or, did you use older bias images? Not mentioned above?

2. I propose that you might edit your post above to more completely describe exactly what you did for the two different calibration procedures, without calling either 'standard'?

3. I noted that you took flat darks at an exposure that equaled the flat exposure. It matches your normal procedure. In my case, I use sky flats (25000 ADU) so taking flat darks of equal exposure is more problematic since the exposures vary in length in different filters. Scaled darks is easier for my process.

4. Any thoughts on why the std of the variable and the check were so different even though of similar magnitude? Actually, I suspect it is purely a result of the real variation in the target over the time series. Just an artifact of how std is calculated!

5. Interesting that even the 'no calibration' procedure gave almost the same result! Unfortunate in terms of convincing observers not to do that!  ;-(

5. Based on the results, I would conclude that there is no significant amplifier glow in your CMOS camera, nor any other potential non-scalable read error source found on the images. Reassuring! It would be nice to visually check that out once on an image without the smaller ROI. 

Again, thanks much for doing this for our readers/observers! They have a choice and know what to look for to see if a problem exists.

Ken

Affiliation
Variable Stars South (VSS)
Responses to comments

Ken,

I used the term 'Standard' to describe callibration using only darks and flats because that is the term use in Berry and Burnell's book "The Handbook of Astronomical Image Processing". The only difference there is that the process described in the book uses flat darks. The description is also used in the callibration screen in AIP4Win (the three options are Basic, Standard and Advanced). Also, if I don't use the term 'Standard', what other short word would I have used in the table I posted above?

1. I did take bias frames. I omitted mention of them in the first draft posted, then edited it later to include them. 80 frames at 0.001 sec, same gain and offset as lights.

2. Description of the callibration procedures. I thought it was quite complete. In fact, I was worried that the post was too long. The only further description I can add is that I used AstroimageJ for the callibration, creating the master bias, master flat and master darks in one run first, then setting up the callibration as one run using the CCD Data Processor dialogue. It requires setting up the directories containing each of the types of frames (light, bias, dark, flat), and selecting 'Enable' for all of them. The other part of the setup is in the DP Coordinate Converter Dialogue. I don't use astrometry, and therefore don't use WCS in my images. I simply enter the target name in the SIMBAD Object ID box, and let AIJ pull the coordinates of the target from SIMBAD (very fast). And of course I also have my observatory geographical position in the setup. All this means that I must select the target, check and comp(s) manually in the index image just prior to the photometry run. The output after photometry includes airmass of the target.

3. You wrote "I noted that you took flat darks at an exposure that equaled the flat exposure." I didn't take flat darks. That is the thing. AIJ (if I am correct) scales the darks not only for the lights but also for the flats.

4. Yes, SD of the DX Tuc mags is just a result of the varying mag of the star.

5. I have actually made use of the fact that the 'No callibration' procedure gives almost the same results, when callibration frames failed on one occasion. I simply processed the images without them! Since my aim in studying these stars is to determine the times of mid eclipse (it is part of a Variable Stars South ongoing project), any slight changes in the mag. is not really critical.

Amp glow. Yes, I really must get back and take or find a 'full frame' long exposure dark. I think there is some glow from memory. I'll try to do that and post an image and line profiles to try to demonstrate anything that is there.

Roy

Affiliation
Variable Stars South (VSS)
Plots of dark current versus exposure time, ASI294MM camera

Now that I can post links to images, here is the plot of the dark current I posted previously as a table of data:

https://www.flickr.com/photos/199603914@N08/53336755119/in/dateposted-public/

Roy

Affiliation
Variable Stars South (VSS)
ASI294MM camera 'amp glow'

Ken asked me about amp glow. There is some. The first image is a full frame 600 sec exposure at 0 gain (4.0e-/ADU), offset 10 (correction, 50), binning 1x1 with a line profile plot at the top right, and the yellow line the location of the profile. The second image is a 60 sec exposure, same gain and offset. The glow is still visible, but faint.

Links to the images are:

https://live.staticflickr.com/65535/53337481275_5f7aae0e8b_z.jpg
https://live.staticflickr.com/65535/53337248068_dca2227f0f_z.jpg

The full frame of the ASI294MM at binning 1x1 is 8288 x 5611 pixels, and half of those numbers with the Region of Interest set at 1/2.

The field of view at ROI 1/2 is well clear of the glow. I use ROI 1/2 routinely, and used it also during the recently posted callibration tests.

Roy