Hello! I just uploaded my AAVSONet readings for RR CET.
I had wanted to look at what I called micro-oscillations around the baseline. I moved out of NM, so I asked AAVSONet NM to give me some readings. The macroclimate is similar, though I am sure that the microclimate is different from my site.
Anyway, the descending limb of RR CET's curve has always been the noisiest part of the curve, and these observations are no different.
I'm curious - observations close in time seem to vary more than the 0.003 mag error of individual observations. Could this variability be due to air cell lensing?
As far as "micro-oscillations" - might it be helpful to average adjacent 2 or 3 readings in order to smooth out the noise? Thank you and best regards.
Mike
Looking just at your recent data for 2023 Nov 22 UT = JD 2460270, it seems to me that the rms scatter in the data-points is about 0.015 to 0.020 mag (full range in the data around 0.04 to 0.05 mag). This despite the nominal uncertainty per-point of 0.003 mag. If your exposures are short, like under 5 seconds, then scintillation noise ("air cell lensing") could contribute to that scatter. Data from the camera may simply be noisier than you think it is (from imperfect flats, clouds, etc). It also seems that the reported errors could be strictly from a S/N (signal-to-noise ratio) calculation rather than the actual scatter in the data. An easy test is to measure some random fairly bright star in the field using the same data and comp stars to look at the scatter you get from a (likely) constant star over the same interval. That will give you an estimate of the "real" external errors.
For a star like this it certainly makes sense to average batches of three to five such data-points (spanning a couple clock-minutes at most, say) since the star itself it not varying so quickly even during the rapid rise to maximum. If there are additional reasonably bright comp stars in the field that are not too faint and not too red (the usual limitation for RR Lyr-type variables), it also makes sense to use more than one comp star (four or five is enough) to help beat down errors from various causes.
\Brian
Brian wrote:
"It also makes sense to use more than one comp star (four or five is enough) to help beat down errors from various causes."
This was one of the parameters addressed by Ed Wiley and Ken Menzies in their study of accuracy and precision in CCD and CMOS photometry (JAAVSO 50(1), 2022, p71). Their conclusion was:
"2. There are no statistical differences between single-comp
and ensemble comp methods shown in this study. However, we
prefer ensemble methods because they can result in statistically
meaningful measurement uncertainties given three or more
comps using the standard deviation of all the comparison stars.
We did not evaluate whether more stars in an ensemble than
used in this study would result in greater accuracy than sole use
of a single comparison star."
If I read the methods section correctly, Ed and Ken studied ensembles of 2, 3 and more than 3 comp stars.
Roy
Roy is correct in quoting the Wiley & Menzies report. This may be OK if one is reckoning for "tonight's" data. But I tend to think long-term, like decades, and I worry (perhaps over-much) about the stability of comps stars (including instrumental effects) over the long haul, including what's in the literature back a century. As a simple example, what happens to one's data series when you get a new telescope/camera/filters or move from site A to site B, or from one's backyard to starting to get AAVSOnet data? Will one comp star control for those changes? I kinda doubt it.
In a perfect world one would like to have at least two stable comp stars matching in color with the target. The reason for avoiding a single comp (even if demonstrably stable) is that you have no way of estimating realistic uncertainties in the data --- simple S/N is insufficient. The 'real sky' case means you want to be able to have more comps to look at effects due to star color, ambient temperature, optics etc.
Some additional perspectives in re long-term comparison star stability can be found here:
https://ui.adsabs.harvard.edu/abs/2007ApJS..171..260L/abstract
...which deals with some of the most boring 'variable' stars imaginable. See the discussion especially in section 2 of the text. In the era of single-channel photometry using two or three comp stars was all we could spend the time on even with a robotic telescope. The right sort of small-aperture, wide-field system might do as well or better on the same stars now.
\Brian
Thanks Brian,
They may be "some of the most boring 'variable' stars imaginable", but the data isn't. Very impressive. Possibly unique?
I nearly didn't post my comment on Ed and Ken's paper, because I knew that it would be saying something you already knew.
Very glad I did.
Roy
Hello! I just stack averaged every 4 images and repeated the analysis
The range of the measurements from what I would call the best fit baseline through the data points is about 0.03mag - 10 times higher than the stated SNR of the measurements.
I'm not sure where these come from. Obviously, not from SNR. It would be curious if there was some sort of actual/physical process going on. Absent that, it would be curious to identify what causes such fluctuations that are higher than the stated SNR. Best regards.
Mike
The "AAVSO Guide to Photometric Uncertainty" contains material that is relevant to this discussion. It can be found at:
https://www.aavso.org/sites/default/files/publications/Uncertainty-V9.pdf
I suspect that the tight "error" bars around individual observations may represent 1/SNR. SNR represents statistical uncertainty (perhaps of just one measurement of one star), and nothing more.
A measure such as standard deviation, on the other hand, also catches systematic uncertainty, and must be calculated from several measurements. Its value must therefore presumably be greater than simple statistical uncertainty.
Roy
Roy,
Thank you for recommending this document. It was once the "text book" for an AAVSO CHOICE course Uncertainty about Uncertainty, and it was assigned reading in later CCD2 courses. (Ed, do you use this in your combined CCD1-2 course?)
I think most new AAVSO observers would find it enlightening. This forum topic would be a good place for the reader to ask questions that may come up..
Phil
Hi Phil and Roy et al:
Yea, I took that course way back when!
I read the manual again yesterday. That was an eye opener! Pardon, but IMHO, they really made some of the terms (error, uncertainty, noise, random error (precision), systematic error (bias)) more confusing than they needed to be. In a couple of cases I felt they used different terms for the same statistical tool. I find that splitting standard deviation into two separate terms is unnecessarily confusing. It is unfortunate that these terms pose such ambiguity in common usage. I find it symptomatic of the whole unfortunate question of why is precision best (more realistically) measured by taking multiple images and measuring the mean and standard deviation rather than expecting 1/SNR of a single bright star to reasonably represent the actual uncertainty of a measuring a target magnitude. Fortunately, it does support the usage of a comp ensemble.
Ken
So by taking means of groups of four data-points you are getting roughly a root-2 improvement in the data, which is in line with the earlier scatter. You are correct in surmising that simple S/N ratio does not capture the complete error budget, i.e. all the sources of noise or uncertainty in the data. Things like small changes in sky conditions, imperfections in the flat-field calibration, and general electronic noise in one's system will add to the total.
I just had a quick look at similar lightcurves of mine for asteroids, RR Lyraes, and Cepheids that aren't too faint, and where data were taken over multiple nights. About the best I can do on a high-order Fourier fit to a lightcurve is around 0.007 mag rms. The nominal S/N per point is much higher, however, something like 3 to 5 millimags. To some extent this is limited by the model fits, and there's always this interplay between a series of data versus weighting and fitting of lines/curves. For the purposes of the AAVSO archive, I would tend to worry more about getting high consistency amongst the multifarious datasets, so that things can be matched downstream.
To give another example of cobbling together diverse datasets, see the preprint of a paper regarding the asteroid Didymos after the impact of the DART spacecraft into its satellite Dimorphos:
https://arxiv.org/pdf/2311.01971.pdf
This is now through the referee mill and is 'in press'. The paper summarizes data from more than two dozen telescope facilities from large to small, including some semi-pro set-ups, and includes commercial cameras and software. See section 3 of the text for a run-down, and sec 3.11 in re the Lowell 42-inch (1.1-m) telescope that I use. The lightcurve snippet shown in Figure 3 analyses a favorable case. Notice that the noise of the individual data-points here is larger than the errors on the complicated fit (about 5 millimags), which is derived from the well-determined shape-model for the primary asteroid. Figures 4 and 5 are also illustrative of 'real world' data with big telescopes at good sites. By the usual standards of asteroid lightcurve photometry this is quite good stuff, and testimony to the first three authors (mainly) in putting all this together. Can you do better from your back garden? B-)
\Brian