I'm trying to figure out how CLEANest calculates it's Model values so I can produce light curves that result from subsets of the dominant component frequencies found by CLEANest.
Based on a brief review of the CLEANest code in the AAVSO Website Software Directory, I assumed the following equation for a one term CLEANest Model :
CLEANest Model (t) = Avg Mag of Obs + a * Cos (2 *Pi * Freq * (t-t0) ) + b * Sin (2 *Pi * Freq * (t-t0) )
Where a = Amp * Cos(Phase); b= Amp * Sin(Phase) ; t = JD of an Observation; t0 = Avg Time of All Observations (JD)
Values Calculated by CLEANest:
Freq = 0.002203 (=Period = 453.967 d) Amp = 0.041543 Phase = 0.1703 rads
Values Calculated by Me:
Avg Mag of All Observations = 11.823
Avg Time of All Observations = 2457595.87
Results:
(SORRY I AM UNABLE TO EMBED OR ATTACH THE PLOT COMPARING THE CLEANEST EXPORTED MODEL TO MY CALCULATED LIGHT CURVE. THEY MATCH EXCEPT FOR A SMALL MAG DIFFERENCE AND A LARGE TIME/PHASE DIFFERENCE)
Differences between CLEANest Model and Calculated Model
Mag Error = -0.004 mag
t0 Error = -91 days == Phase Error = -1.25949 rads = 2*Pi* (-91/453.96)
Results with Mag & t0/Phase Corrections:
(SORRY I AM UNABLE TO EMBED OR ATTACH THE PLOT COMPARING THE CLEANEST EXPORTED MODEL TO MY CALCULATED LIGHT CURVE AFTER ADD CORRECTION FOR MAGNITUDE AND TIME/PHASE. THE TWO CURVES MATCH)
My gut feel is that I incorrectly assumed that t0 is the average time of all observations. There are a lot of time index changes going on in the CLEANest code that I don't follow.
Appreciate any suggestions you might have.
Thanks
Dave Hollinberger
Hi Dave
I'm looking at the details of your question.
In the meantime, pages 71 to 81 of the user manual may help. In addition, the CLEANest unit tests may provide some insight into the behaviour of the code:
https://github.com/AAVSO/VStar/blob/master/test/org/aavso/tools/vstar/util/period/dcdft/CleanestTest.java
David
Hi Dave
Perhaps it would be helpful to reason about a particular dataset, DCDFT search parameters, top hits selected for CLEANest, Fourier model creation parameters.
David
Hi Dave,
Thank you very much for your suggestions back in early September. I'm sorry for my delay in replying.
I took your advice and inputted a set of test signals consisting of a single sinusoids with varying phase into CLEANest implemented by Peranso. The CLEANest phase value outputted to the CLEANest Workbench Detailed Info Output Table is incorrect. The problem is probably caused by the "arctan(-PeakMatrix(i).sIni,PeakMatrix(i).cosi)" function call in Public Sub CLEANest_CalculateDetailedPeakInfo(...).
I think I also sorted out why Ao and To are close but not exactly equal to the average magnitude and average time of occurrence of all the datapoints in the light curve.
I wrote up my results in 5 page whitepaper. Please let me know if you want a copy.
Thanks
Dave Hollinberger
Hi Dave
I would be interested in your paper, yes. Please send when you get the chance via my AAVSO contact page.
Also, I note above that you found a bug in Peranso's CLEANest. Is that right? It would be good to report that to the developer if you haven't already.
Did you try the same operation in VStar? Perhaps your paper covers this.
Thanks.
David
Hi Dave,
Sorry for the delay in replying - again.
1. I'm sorry but I can't figure out how to attach a document via your AAVSO contact page. I can only send a blog message like this one. Appreciate suggestions.
2. Yes, the bug is in Peranso's CLEANest - that's what my tests were run on. And yes I have already sent my paper to the Peranso developer, Tonny Vanmunster.
3. I have not run similar tests on VStar's CLEANest . I will try to do that in the future to make sure. The reason I believe the same problem may exist in VStar CLEANest is that when I reviewed the CLEANest code dated 3/10/2005 that is posted on the AAVSO website Software Directory, I found code that appears to be causing the problems t I saw while testing Peranso's CLEANest. The reason I was reviewing the AAVSO CLEANest code was that Tonny was unable provide me with the Peranso CLEANest code because he has a policy of not releasing active software code. He did provide me with a copy of TS12.bas.
Thanks
Dave Hollinberger
Here is the…
Hi David,
Here is the CLEANest paper in plain text. Hope it more readable than the earlier version
Thanks
Dave Hollinberger
------------
Independently Calculating Signal Models Using CLEANest Signal Parameters
“Analyzing Light Curves” by Grant Foster (2010) (ALC Foster) gives two equivalent inverse Fourier series equations for calculating a model signal using the signal parameters produced by CLEANest:
ALC Foster Eqn 9.5
Model1(t) = Amp(0) + Sum(n=1:NoP)[ (c(n)*cos{2*Pi*F(n)*(t-t(0))} ) + (s(n)*sin{2*Pi*F(n)*(t-t(0))} ) ]
ALC Foster Eqn 9.8
Model2(t) = Amp(0) + Sum(n=1:NoP)[Amp(n)*cos{(2*Pi*F(n)*(t-t(0)))+ Phase(n)}]
Where:
Amp(0) = ~Average of the magnitude of all datapoints in the time series
t(0) = ~Average of the time of occurrence of all datapoints in the time series
t = time
F(n) = Frequency of nth term
c(n) = Fourier series coefficient for nth cosine term
s(n) = Fourier series coefficient for nth sine term
Amp(n) = Amplitude of nth term = Sqrt[c(n)^2 + s(n)^2] ALC Foster Eqn 9.6
Phase(n) = Phase of nth term = arctan[-s(n)/c(n)] ALC Foster Eqn 9.7
Problem #1 - CLEANest Phase(n) is Incorrect
Private Subroutine CLEANestSmooth uses ALC Foster Eqn 9.5 to compute the CLEANest model time series values. It appears to do so correctly since the CLEANest model signal agrees well with the original signal, provided that the CLEANest Number of Periods is made large enough.
Table 1 is the Peranso CLEANest Workbench Detailed Info Output Table for Star ASAS-SN-V J002355.01-715729.7 with CLEANest Auto Search Number of Periods set to 3.
n Freq FrqErr Period PerErr Theta Amp AmpErr Phase
(c/d) (days) (mag) (cycles)
1 0.002203 0.000035 453.97 7.203 79.67 0.0483 0.0049 0.1995
2 0.025250 0.000060 39.60 0.093 56.92 0.0280 0.0049 0.0887
3 0.003074 0.000051 325.31 5.390 49.14 0.0331 0.0049 0.8957
Table 1 – Example Peranso CLEANest Workbench Detailed Info Output Table
The CLEANest Output Table does not provide the Fourier series coefficients c(n) and s(n) used in ALC Foster Eqn 9.5. Instead, it provides a value for Phase(n). As result, you have to use ALC Foster Eqn 9.8 if you want to independently calculate the CLEANest model values.
I tested Peranso CLEANest using a set of synthetic signals that were single frequency sinusoids, all with the same amplitude and frequency. Only the phase of the test signals varied,from 0 to 1 cycle (=0 to 2*Pi rads). Table 2 shows the results.
TEST SIGNAL PHASE || CLEANEST PHASE
Sig Deg Cycles Quadrant || Deg Cycles Quadrant
A 0 0.000 Q1/Q4 0 0.000 Q1/Q4
B 1 0.003 Q1 1 0.003 Q1
C 45 0.125 Q1 45 0.125 Q1
D 89 0.247 Q1 89 0.247 Q1
E 90 0.250 Q1/Q2 90 0.250 Q1/Q2
F 91 0.253 Q2 271 0.753 Q4
G 135 0.375 Q2 315 0.875 Q4
H 179 0.497 Q2 359 0.997 Q4
I 180 0.500 Q2/Q3 360 1.000 Q4/Q1
J 181 0.503 Q3 1 0.003 Q1
K 225 0.625 Q3 45 0.125 Q1
L 269 0.747 Q3 89 0.247 Q1
M 270 0.750 Q3/Q4 90 0.250 Q1/Q2
N 271 0.753 Q4 271 0.753 Q4
O 315 0.875 Q4 315 0.875 Q4
P 359 0.997 Q4 359 0.997 Q4
Table 2 – Results of Testing CLEANest Phase Calculation
When the phase of the test signal was in Quadrant 1 or 4, the CLEANest phase was correct. However, when the phase of the test signal was in Quadrant 2, the CLEANest phase was off by +0.5 cycles (= -0.5 cycles), putting it in Quadrant 4. And when the phase of the test signal was in Quadrant 3, the CLEANest phase was off by -0.5 cycles ( = +0.5 cycles),putting it in Quadrant 1.
The CLEANest phases were calculated using ALC Foster Eqn 9.7 Phase(n) = arctan[-s(n)/c(n)]. The Visual Basic “arctan” function only returns a value of +/- 0.25 cycle = +/- Pi/2 rads. I believe that this limitation of the Visual Basic “arctan” software function causes the problems with the CLEANest phase calculation.
Problem #2 CLEANest does not provide values for Amp(0) and t(0)
The Peranso CLEANest Workbench Detailed Info Output Table does not include the values for Amp(0) (= CLEANest variable DCOEF(0)) and t(0) (( = CLEANest variable DTZERO). Amp(0) is close (within several millimags) to the average of the magnitude of all datapoints in the time series. I can’t tell from the calculations in CLEANest what is causing the difference. t(0) is close (within a 1 day) to the average of the time of occurrence of all datapoints in the time series. The source of the difference is maybe due to rounding of DTZERO, though I can’t find where in the CLEANest code DTZERO is determined.
Possible Solutions:
1. Do Nothing - CLEANest has been around for a long time. It could be that I am the first person who wanted to independently calculate my own signal model using subsets of the CLEANest frequency components. I have found a workaround that is less flexible, being limited to breaking the CLEANest frequency components into frequency sub-bands rather than combining them as I choose in order to investigate effect of possible harmonic components etc. This workaround appears to be less accurate but it is much easier to use.
2. Revise/Correct the CLEANest Workbench Detailed Info Output Table – In Public Subroutine CLEANest_CalculateDetailedPeakInfo , delete Phase (= PeakMatrix(i).Phase) and add c(n) (=PeakMatrix(i).cosi), s(n) (=PeakMatrix(i)sIni), Amp(0) (=DCOEF(0)) and t(0) (= DTZERO) to the Output Table. These changes would allow independently calculating the signal model using Foster ALC Eqn 9.5.
3. Correctly calculate PeakMatrix(i).Phase - Revise the code for calculating PeakMatrix(i).Phase based on Visual Basic code that calculating the arctangent that preserves the quadrant information (http: //www.vb-helper.com/howto_atan2.html). Also, in Public Subroutine CLEANest_CalculateDetailedPeakInfo, add Amp(0)(=DCOEF(0)) and t(0) (= DTZERO) to the Output Table. These changes would allow independently calculating the signal model using Foster ALC Eqn 9.8.
Note: CLEANest uses the variable “dphase” in a number of different places with different values and meanings. Changes to the variable “dphase” should be avoided as much as possible.
Hi David,
Here is the CLEANest paper in plain text. Hope it more readable than the earlier version
Thanks
Dave Hollinberger
------------
Independently Calculating Signal Models Using CLEANest Signal Parameters
“Analyzing Light Curves” by Grant Foster (2010) (ALC Foster) gives two equivalent inverse Fourier series equations for calculating a model signal using the signal parameters produced by CLEANest:
ALC Foster Eqn 9.5
Model1(t) = Amp(0) + Sum(n=1:NoP)[ (c(n)*cos{2*Pi*F(n)*(t-t(0))} ) + (s(n)*sin{2*Pi*F(n)*(t-t(0))} ) ]
ALC Foster Eqn 9.8
Model2(t) = Amp(0) + Sum(n=1:NoP)[Amp(n)*cos{(2*Pi*F(n)*(t-t(0)))+ Phase(n)}]
Where:
Amp(0) = ~Average of the magnitude of all datapoints in the time series
t(0) = ~Average of the time of occurrence of all datapoints in the time series
t = time
F(n) = Frequency of nth term
c(n) = Fourier series coefficient for nth cosine term
s(n) = Fourier series coefficient for nth sine term
Amp(n) = Amplitude of nth term = Sqrt[c(n)^2 + s(n)^2] ALC Foster Eqn 9.6
Phase(n) = Phase of nth term = arctan[-s(n)/c(n)] ALC Foster Eqn 9.7
Problem #1 - CLEANest Phase(n) is Incorrect
Private Subroutine CLEANestSmooth uses ALC Foster Eqn 9.5 to compute the CLEANest model time series values. It appears to do so correctly since the CLEANest model signal agrees well with the original signal, provided that the CLEANest Number of Periods is made large enough.
Table 1 is the Peranso CLEANest Workbench Detailed Info Output Table for Star ASAS-SN-V J002355.01-715729.7 with CLEANest Auto Search Number of Periods set to 3.
n Freq FrqErr Period PerErr Theta Amp AmpErr Phase
(c/d) (days) (mag) (cycles)
1 0.002203 0.000035 453.97 7.203 79.67 0.0483 0.0049 0.1995
2 0.025250 0.000060 39.60 0.093 56.92 0.0280 0.0049 0.0887
3 0.003074 0.000051 325.31 5.390 49.14 0.0331 0.0049 0.8957
Table 1 – Example Peranso CLEANest Workbench Detailed Info Output Table
The CLEANest Output Table does not provide the Fourier series coefficients c(n) and s(n) used in ALC Foster Eqn 9.5. Instead, it provides a value for Phase(n). As result, you have to use ALC Foster Eqn 9.8 if you want to independently calculate the CLEANest model values.
I tested Peranso CLEANest using a set of synthetic signals that were single frequency sinusoids, all with the same amplitude and frequency. Only the phase of the test signals varied,from 0 to 1 cycle (=0 to 2*Pi rads). Table 2 shows the results.
TEST SIGNAL PHASE || CLEANEST PHASE
Sig Deg Cycles Quadrant || Deg Cycles Quadrant
A 0 0.000 Q1/Q4 0 0.000 Q1/Q4
B 1 0.003 Q1 1 0.003 Q1
C 45 0.125 Q1 45 0.125 Q1
D 89 0.247 Q1 89 0.247 Q1
E 90 0.250 Q1/Q2 90 0.250 Q1/Q2
F 91 0.253 Q2 271 0.753 Q4
G 135 0.375 Q2 315 0.875 Q4
H 179 0.497 Q2 359 0.997 Q4
I 180 0.500 Q2/Q3 360 1.000 Q4/Q1
J 181 0.503 Q3 1 0.003 Q1
K 225 0.625 Q3 45 0.125 Q1
L 269 0.747 Q3 89 0.247 Q1
M 270 0.750 Q3/Q4 90 0.250 Q1/Q2
N 271 0.753 Q4 271 0.753 Q4
O 315 0.875 Q4 315 0.875 Q4
P 359 0.997 Q4 359 0.997 Q4
Table 2 – Results of Testing CLEANest Phase Calculation
When the phase of the test signal was in Quadrant 1 or 4, the CLEANest phase was correct. However, when the phase of the test signal was in Quadrant 2, the CLEANest phase was off by +0.5 cycles (= -0.5 cycles), putting it in Quadrant 4. And when the phase of the test signal was in Quadrant 3, the CLEANest phase was off by -0.5 cycles ( = +0.5 cycles),putting it in Quadrant 1.
The CLEANest phases were calculated using ALC Foster Eqn 9.7 Phase(n) = arctan[-s(n)/c(n)]. The Visual Basic “arctan” function only returns a value of +/- 0.25 cycle = +/- Pi/2 rads. I believe that this limitation of the Visual Basic “arctan” software function causes the problems with the CLEANest phase calculation.
Problem #2 CLEANest does not provide values for Amp(0) and t(0)
The Peranso CLEANest Workbench Detailed Info Output Table does not include the values for Amp(0) (= CLEANest variable DCOEF(0)) and t(0) (( = CLEANest variable DTZERO). Amp(0) is close (within several millimags) to the average of the magnitude of all datapoints in the time series. I can’t tell from the calculations in CLEANest what is causing the difference. t(0) is close (within a 1 day) to the average of the time of occurrence of all datapoints in the time series. The source of the difference is maybe due to rounding of DTZERO, though I can’t find where in the CLEANest code DTZERO is determined.
Possible Solutions:
1. Do Nothing - CLEANest has been around for a long time. It could be that I am the first person who wanted to independently calculate my own signal model using subsets of the CLEANest frequency components. I have found a workaround that is less flexible, being limited to breaking the CLEANest frequency components into frequency sub-bands rather than combining them as I choose in order to investigate effect of possible harmonic components etc. This workaround appears to be less accurate but it is much easier to use.
2. Revise/Correct the CLEANest Workbench Detailed Info Output Table – In Public Subroutine CLEANest_CalculateDetailedPeakInfo , delete Phase (= PeakMatrix(i).Phase) and add c(n) (=PeakMatrix(i).cosi), s(n) (=PeakMatrix(i)sIni), Amp(0) (=DCOEF(0)) and t(0) (= DTZERO) to the Output Table. These changes would allow independently calculating the signal model using Foster ALC Eqn 9.5.
3. Correctly calculate PeakMatrix(i).Phase - Revise the code for calculating PeakMatrix(i).Phase based on Visual Basic code that calculating the arctangent that preserves the quadrant information (http: //www.vb-helper.com/howto_atan2.html). Also, in Public Subroutine CLEANest_CalculateDetailedPeakInfo, add Amp(0)(=DCOEF(0)) and t(0) (= DTZERO) to the Output Table. These changes would allow independently calculating the signal model using Foster ALC Eqn 9.8.
Note: CLEANest uses the variable “dphase” in a number of different places with different values and meanings. Changes to the variable “dphase” should be avoided as much as possible.
Hi David,
I looked at the VStar User's Manual briefly.
1. Model
On page 67 & 68 of the VStar User's Manual there is an example model equation. It is in the Foster ALC Equation 9.5 form, which does not use phase values. In place of phase values, cosine coefficients c(n) = -0.250292723 etc and sine coefficients s(n)= 0.154254105 etc values are provided. The example model also includes the ~average magnitude = Amp(0) = c(0) = 3.910242905 value. And it separately provides the ~average time = t(0) = zeroPoint = 2455973 JD value.
So it looks to me that VStar already meets the Option 2 solution that I suggested in my paper. Great.
2. Period Analysis (DC DFT) Table
The example tables on VStar User's Manual Page 63, 73 & 76 include Frequency, Period, Power and Semi-Amplitude values. But unlike Peranso Period Analysis Tables, the VStar tables do not include Phase values. So it appears that VStar dodged the bad phase bullet here also. Also great.
3. Relative Phases
On page 67 of the VStar User's Manual there is an example "Relative Amplitudes & Phases by fundamental frequency" table given. I cant really follow the format of the table - If I had to guess, the last 3 values (2.9029, 0.0276 and 6.1310) might be the relative phases for Freq 2, 3 & 4 given in rads. I have read the Foster ALC section on Relative Amps & Phase several times but am still mystified by the concept of Relative Phase. So I really cant tell if VStar has a bad phase issue here or not. There is that potential.
Hope this helps
Dave Hollinberger