Hello! I am wondering if there is an advantage of using sky flats (not twilight flats) instead of lightbox flats when calibrating images.
Currently, I us a lightbox using white LEDs and a single incandescent bulb which gives me half full images with 2,5, and 10 second integrations in I,V, and B filtes respectively.
At the end of eac night's run, I typially have about 150 images in each image. After going through my LPVs, I follow a couple of short period variables for several hours.
When I performed a median combine of 150 I images from last night's run, the min pixel was 1150 and the mean was 1192 and median 1186. When I examined a 25 pixel radius circle, the mean was 1188 with a varfiance of 74 and s.d. = 8.6. The peak on the histogram is much narrower than that from the lightbox flat.
Would this be a better flat to use than the lightbox flat since it uses the night sky spectrum through the filters even though the average is much less than half full? Best regards.
Mike
Hi Mike,
In principle such flat should be the best one, when trying to correct gradients and other issues of non-uniform illumination of FoV. And as you said, correctly take into account the colour of the night sky. Still, I suspect that signal to noise ratio of sky flat is typically not good enough for millimagnitude work. Professionals sometimes combine screen or twilight flats (for high SNR) and sky flats for correct spatial flatness.
When your target and comp star(s) are located next to eachother in the center of the frame, IMHO you hardly benefit from sky flats. The story is different when compstar and target are in very different positions on the sensor.
Best wishes,
Tõnis
Hi Michael,
I have not used night sky flats for photometric purpose but to produce deep sky images. I have used the night sky-flats to resolve fine tidal streams and structures around early type galaxies. One of our image was used in the works of Pierre-Alain Duc et al. (page 7 in The ATLAS-3D project – XXIX, 2014)
http://arxiv.org/abs/1410.0981
The image you can also see here: http://www.irida-observatory.org/CCD/NGC474/NGC474.html
What I have learned is that to use sky-flats is not so easy task:
The main conclusion for me was that to use sky-flats is a HARD job and incredibly TIME consuming.
I hope this initial information will help you for your future investigations. I like sky-flats but sometimes is not necessary to spend so much time for such a reduction.
The other observers may have different experience and I hope they will share it with us.
Velimir
I have gone through an exercise of trying different flat fielding methods recently. Velmir's response is very much in line with my experience trying nighttime flats. I returned to twilight flats having decided the time and effort to get good night time flats was significantly outweighing the benefits - and for someone with a day job, not so good on that front either.
Additionally I investigated light sources and paints for light box/dome flats.
Ideally you need a black body light source of the correct temperature. LEDs can be/are too narrow in terms of the light they produce.
The light used for flats also ideally needs to be parallel to the optical axis. My belief is that otherwise dust and dirt on optical surfaces will not be properly recorded in the flat if a lot of off axis light is present. Methods involving light boxes and tee shirts will produce unwanted off axis light - but the counter argument is of course the convenience.
Regarding dome flats, any paint needs to ideally have known reflective properties. Otherwise you might take a perfectly acceptable tungsten light source of 5800 K and unwittingly turn it into something quite different once reflected off an apparently white paint that has unfortunate absorption properties. I have priced up paint with 98% reflectivity from UV - IR (available from http://www.aviantechnologies.com/products.php) but it is prohibitively expensive (from my point of view).
I may of course be totally over thinking the problem - I will accept that, its my pefectionistic streak.
All of this has led me to continue with the twilight flats. This can be the most stressful part of the observing to be honest. I have to get my skates on to get them done between the time the Sun is low enough below the horizon and the appearance of too many stars.
I guess ultimately the flats need to be "good enough" a compromise of some sort regardless of method.
All of that said, the experimentation was worth while. I'm pleased I attempted night time flats at least. I'm now more comfortable with my decision to continue with twilight flats for now. I may reconsider if I ever achieve retirement and can take all the time I want.
- Carl.
Hello Carl
You wrote
"The light used for flats also ideally needs to be parallel to the optical axis. My belief is that otherwise dust and dirt on optical surfaces will not be properly recorded in the flat if a lot of off axis light is present. Methods involving light boxes and tee shirts will produce unwanted off axis light - but the counter argument is of course the convenience."
I completely agree with the need for parallel light to the optical axis. Having a diffuser or T shirt over the end of the scope is not the same light path as the twilight.
I use no diffuser whatsoever. This is on a scope/chip with 14 arc minute field.
For a wide field setup like BSM, there are other problems with twilight flats. The field is so large, that the gradients can cause an error that is large, depending upon what you are doing. It that case, the errors introduced by the T shirt/diffuser may be much smaller than the gradients that they would tend to reduce or eliminate. So as Arne says, "it Depends".
Gary
Hi Carl, Garry,
I fully agree with you, the diffuse light from boxes, panels, diffusers, sent into the instrument a lot of flux of diffuse light that is nothing like the collimated light from the star. It can be easily 90% or more of the total flux depending the FOV of the instrument. That diffuse light is for most eliminated by baffles, coatings... but even if only a few percent remains and reaches the sensor this a large percentage of the useful light.
For imaging, using the twilight flats is excellent as the illumination law is the same as the sky background to be subtracted. But in photometry it's not perfect even if much better than other techniques. This is very depending from the instrument.
In my experiments I did compare flats made with diffusers and flats from collimated light (like the stars). In number of cases I found center to edge errors of 3~5%, but from one refractor I got several 10% ! A small poorly baffled surface is enough to create such problem. For us, photometrists, this is a serious issue.
The geometric distorsion of the image is also an issue, it affects the flats but not the total flux of each individual star. Normally long focus telescopes do not have such problem (0.1% or so). But a zoom telelens could easily show several percents of barrel or pincushion distorsion, resulting in something like 10% error !
I think all this is important for us and not well considered at time being, anything we could do ?
Clear Skies !
Roger
Hi Gary,
Your point is well made and not one I had considered. My FOV is around 15 arcmin. Not a lot of room for a large gradient there.
"It depends" is probably why the material I've found is full of discussion of advantages and disadvantages and less concrete commitment.
Would it be fair to say that if in the filters you use, having derived transformation/extinction coefficients, flat fielded, dark and bias subtracted and done your data reduction of some standard stars, you find the result within 1% or better of the catalog values - that whatever method you used for flat fielding was good enough?
I don't know the answer but that measurement seems reasonable to me. If you don't get that result, then investigate possible causes further. If using a dome flat, maybe you are not getting all the colours needed and switching to a twilight flat (or vice versa if your FOV is large and gradients dominate) might fix that problem. Or maybe there is some other combination of causes?
The goal after all is accurate photometry. This is aided by good flats. But the flats are not the end target.
- Carl.
Carl wrote: "I guess
Hello
Carl wrote: "I guess ultimately the flats need to be "good enough" a compromise of some sort regardless of method."
I used to thing that it was possible to make the perfect flat. After lots of work, I concluded that the 20 methods I tried all had flaws, some bigger than others. I still think about it, but have given up the active pursuit. I still focus on minimizing Photometric Errors to get better understanding the science (the real goal).
Roger, wrote: "The goal after all is accurate photometry. This is aided by good flats. But the flats are not the end target."
Ah Men--the only flaw here is that if you have a flawwed method of flat fielding and a flawed something else, and they compensate for each other, you may make a change and the errors increase. Its really good discipline to observer standard stars often. I don't do that enough.
Gary
I too have spend countless hours doing flat fielding. I'll bet I have tried two dozen different ways. None is perfect--some better than others. I have settled on Twilight Flats for my photometry. They seem to be the least of the evils.
I am able to get 10B,10V,10R, &10I flats in a single night. Its hard to get more. If you cannot, perhaps your proceedure needs some tweaking. Let me know if that is the case, I will describe how to do this.
Gary
WGR
[quote=WGR]
I too have spend countless hours doing flat fielding. I'll bet I have tried two dozen different ways. None is perfect--some better than others. I have settled on Twilight Flats for my photometry. They seem to be the least of the evils.
I am able to get 10B,10V,10R, &10I flats in a single night.
[/quote]
This sounds exactly like the method BSM-Hamren uses for the twilight flats. I use a layer of white cloth (T-shirt) over the objective to diffuse the light. Not sure if this is the best way, especially if clouds pass over during the shots.
Mike
I am currently using lightbox flats (smallish scopes so I can use a nice long box with lots of diffusion and it seems to work very well) but to test them I've done some twilight flats. I wrote a script to shoot all the flats which I can start before the sky is dark enough. It shoots test exposures until the sky is the appropriate darkness then shoots the actual flats. Each exposure is measured and the duration of the next exposure is adjusted to maintain similar signal levels. The filters are done in an order which allows the darkening sky to continue providing adequate light as the filters get 'darker' relative to the colour of the twilight sky. I'm only shooting in B and V so I have lots of time to get 20 flats in each filter with ~32k ADUs each.
Richard Wagner WCR
Thank you all for your insights! I appreciate them. Best regards.
Mike
There has been some good advice given on this thread. There are 3 basic types of flatfields that you might use:
- twilight sky flats, taken in the ~20 minute period when the sky brightness is "just right".
- night sky flats, using science images taken during the night.
- dome/lightbox flats, using an external illumination system.
Each has its good and bad features. The lightbox flats, for example, can be done on cloudy nights and can achieve extremely high signal/noise from stacking. The night sky flats match the color of the night sky the best. Twilight flats are often the only way you can obtain very blue (say, u') flats. I think they all can be made to work, and any such method will help in removing the systematic effects in your system, such as vignetting and pixel-pixel sensitivity variations. Lightbox flats have difficulty getting uniform spectral coverage and illumination pattern. Night sky flats have trouble with signal/noise (not many photons in the night sky!) and removal of gradients, such as from the moon. Twilight flats have gradients (from the moon and the setting sun), stars to median-filter and potential sodium emission contamination. Of the three, I like twilight flats personally as they are easy to do and work most of the time.
I strongly recommend either doing a raster pattern (moving a well-sampled star across the CCD sensor and comparing its magnitude at each position to that of the center) or a single exposure of a well-calibrated, relatively crowded, region (even using APASS photometry might work for this) and comparing the differential photometry with respect to CCD (x,y) position. Do the extractions and comparisons before, and after, flatfielding to see what the original system produced and whether things improved with flatfielding.
There are many variables in getting scientific images calibrated properly. The end result should be photometry that is accurate, not just precise, at the 10-20mmag level on non-pathological objects. It can be done! At the same time, don't throw away data that is worse than this, just recognize that photometry, like most other endeavours, is a learning process and you will keep improving throughout your observing life.
Arne
Bruce Gary is a friend of mine that does exoplanet work. He has been using the "Dual T-Shirt" method for years and acheieving MMag results, enough so that he has discovered several exo-planets! Bruce built a wooden frame, attached a double layer of white T-shirt material to it and uses that to get high quality flats! He shoots his flats at twilight but the T-shirt extends the time you can image a bit. For more info see his free ebook.
http://brucegary.net/book_EOA/x.htm
Just so you know Bruce is a retired JPL Scientist! So I have his book on my computer, kindle, nook and phone and refer to it frequently!
Paul Temple