Affiliation
American Association of Variable Star Observers (AAVSO)
Thu, 09/18/2014 - 15:08
The new version of the Photometry Transformation Generation Program - PTGP V. 5.0 is now available. The program makes the calculation of Photometry transforms much easier. Spreadsheets are no longer required! There is also a new interface to VPHOT enabling automatic selection of the standard stars in M67 and NGC7790. Details can be found at www.aavso.org/tg Note the Users Guide provides answers to specific questions. The program runs on Windows, Mac, and Ubuntu (Linux).
Major changes since Version 4.0 include –
- Calculation of transform one sigma error estimates
- Calculation of transform “r-squared” measure
- Use of AUID or “Boulder” star ids
- VSP standard star magnitudes retrieved from the AAVSO web site (requiring an active Internet connection)
- VPHOT download file interface
- MaxIm instrument file interface
- Change of transform names to use AAVSO standard (previously saved 4.0 transform sets are lost)
- Added V-I transforms (Tvi, Tv_vi, Ti_vi, Tr_vi)
- Instrumental magnitudes are averaged if multiple values provided
- Changed format of final transform export file to match .ini format used by AAVSO tool TransformApplier
RE:"MaxIm instrument magnitude"
I don't see this option under photometry in my MaximDL 5.23. Is this a feature in the new MaximDL 6.XX?
James
Barbara Harris (barbharris1@hughes.net) sent me the instrument magnitude file I used to develop the MaxIm format.. Check with her.- she knows more about MaxIm than I do...
Hello James
To get instrumental Mags in Maxim, you have to install the "artficial star plug in" and use it as the reference star. I usually give it a value of 0.0.
How did you get your magnitudes before?
Gary
RE:"How did you get your magnitudes before?"
For all of my AASVO submissions, I've used either MaximDL 4 or 5. I used the Photometry function, tagged all my comp stars from VSP as reference stars, with BVRI magnitudes inputted from the variable's VSP photometry chart. The last star I would tag would be the variable or as Maxim calls it "New Object." I then go back to each Maxim reference star and re-type it from its star number on the VSP chart; i.e. 115, 135, 142, 156. Please note that Maxim will not let you process an extended AASVO output until all objects are labeled including the variable. I look at the Maxim photometry plot to see if there are any outliers to eliminate (clouds, bad flats, etc) then process a data file with the AASVO format. I do this after collecting 4-6 months of data split-up between the different filters and cameras I use; i.e. ST-10, ST-8, STL-11K. (I'm using the ST-10XME for most of my work).
It usually takes me about 5-10 minutes per variable/filter to process 120 images of a variable star per filter; any more than this crashes MD, since each unbinned ST10 image is a 6Mb plus sized file. Using MaximDL 5's stacking tool, I now do a pre-sort of that eliminates images that have elongated stars or empty fileds (clouds) and automatically aligns to the best image in my particular grouping.
James
To: Gordon,
Here are my formated NGC 7790 Transform results PTGP4 results compared to the unformatted NGC 7790 Transforms PTGP5 results; using AIP4WIN:
PTGP4.0
PTGP5.0
PTGP4.0
PTGP5.0
PTGP4.0
PTGP5.0
PTGP4.0
PTGP5.0
PTGP4.0
PTGP5.0
PTGP4.0
PTGP5.0
PTGP4.0
PTGP5.0
PTGP4.0
PTGP5.0
30-Aug-14
30-Aug-14
6-Sep-14
6-Sep-14
7-Sep-14
7-Sep-14
10-Sep-14
10-Sep-14
11-Sep-14
11-Sep-14
12-Sep-14
12-Sep-14
AVERAGES
AVERAGES
STD_DEV
STD_DEV
Tbv
1.385
1.391
1.369
1.375
1.372
1.379
1.373
1.380
1.380
1.387
1.375
1.382
1.376
1.382
5.854E-03
5.785E-03
Tb(BV)
0.316
0.324
0.299
0.307
0.302
0.310
0.300
0.307
0.311
0.319
0.313
0.321
0.307
0.315
7.360E-03
7.554E-03
Tv((BV)
0.039
0.043
0.030
0.034
0.031
0.035
0.028
0.032
0.036
0.040
0.041
0.044
0.034
0.038
5.269E-03
5.020E-03
Tvr
1.085
1.086
1.100
1.100
1.099
1.100
1.103
1.104
1.083
1.083
1.085
1.085
1.093
1.093
9.072E-03
9.295E-03
Tv(VR)
0.071
0.081
0.055
0.064
0.057
0.066
0.052
0.061
0.066
0.075
0.074
0.083
0.063
0.072
9.094E-03
9.288E-03
Tr(VR)
-0.007
0.002
-0.035
-0.026
-0.033
-0.025
-0.042
-0.034
-0.011
-0.002
-0.004
0.005
-0.022
-0.013
1.649E-02
1.687E-02
Tri
0.996
0.971
0.991
0.968
0.990
0.966
0.982
0.957
1.001
0.977
1.007
0.982
0.995
0.970
8.826E-03
8.750E-03
Tr(RI)
-0.007
0.002
-0.040
-0.030
-0.038
-0.029
-0.048
-0.040
-0.011
-0.002
-0.003
0.006
-0.025
-0.016
1.962E-02
1.972E-02
Ti(RI)
-0.003
0.032
-0.031
0.003
-0.028
0.006
-0.030
0.005
-0.013
0.021
-0.010
0.024
-0.019
0.015
1.199E-02
1.209E-02
Tvi
1.042
1.030
1.047
1.035
1.046
1.034
1.044
1.032
1.043
1.032
1.047
1.035
1.045
1.033
2.137E-03
2.000E-03
Tv(VI)
0.039
0.044
0.030
0.035
0.031
0.036
0.028
0.033
0.036
0.041
0.040
0.045
0.034
0.039
5.020E-03
5.020E-03
Ti(VI)
-0.002
0.015
-0.015
0.001
-0.013
0.003
-0.014
0.002
-0.006
0.010
-0.005
0.011
-0.009
0.007
5.492E-03
5.762E-03
Averages:
8.853E-03
8.929E-03
As you can see there not much different from each other. The exception to this is the calculations for the Ti(RI) data for PTGP5 which was always larger than PTGP4's results. One other thing I noticed, I got the message, in a red warning box, "Reference Star ids not found in VSP - Stars excluded from calculation." I'm not sure why I got this, since the AIP4WIN format for instrumental magnitudes contained the AUID comp star designations. I enclose a link to the 30-Aug-14 AIP4WIN .txt file I processed with PTGP5.
http://www.astroimage.info/files/log%28Redo-30Aug14%29corrDESIG3%2822Stars%29.txt
James
p.s. I know that Dr. Barbara was a beta tester for MaximDL 6.0 a few months back so I'm sure this is a new feature, that I won't have using V5.23. Ah well, I'll have to use AIP4WIN for all my transformed photometry. At least testing out all these different transform software got me familair with it!
James,
The warning box is telling you that some of the stars in the original Bouler list are no longer considered standard reference stars i.e. they are not downloaded in the current VSP standards list. So when you run PTGP 4.0 you're using the old values for those stars. When running PTGP 5.0 those stars are excluded from the calculattion.
The differences between the two sets of results are small, but are caused by the combination of the new standard star values Arne recently updated combined with the elimination of the stars shown in the error message.
Thanks for the analysis!
To: Gordon,
RE:"When running PTGP 5.0 those stars are excluded from the calculattion."
This is interesting. it would be nice to know what is being excluded. As far as I see, the slope chart that comes out of PTGP4 (say for Tbv) looks like it contains the exact same number of stars in PTGP5 slope chart. Please note that the format I used in PTGP4 has only one set of instrument magnitudes (averaged), per filter compared to the one I hyperlinked you for PTGP5. Btw, this avoidance of pre-formatting the AIP4WIN's instrument magnitude report is a great improvement!
James
James,
The warning message on the screen shows the star id's found in the instrument magnirude measurements file that were either not matched to an AUID or found in the VSP standard star data downloaded from the AAVSO web site. In a future release I'll also show the AUID's.in addition to Boulder ids - if available - in the message. The mapping I have for NGC7790 and M67 are shown below.
(When the AUID below is 000-000-000 no match could be found.)
NGC7790
Boulder ID AUID
1 000-BLJ-963
2 000-BLJ-993
3 000-BLJ-978
4 000-BLJ-991
5 000-BLK-002
6 000-BLK-000
7 000-BLJ-972
8 000-000-000
9 000-BLJ-981
10 000-BLJ-976
11 000-000-000
12 000-000-000
13 000-BLJ-985
14 000-BLJ-992
15 000-BLJ-989
16 000-BLJ-964
17 000-BLJ-980
18 000-000-000
19 000-BLJ-982
20 000-BLJ-970
21 000-BLJ-987
22 000-BLK-004
23 000-BLJ-990
24 000-BLJ-973
25 000-BLK-003
26 000-BLK-022
27 000-000-000
28 000-BLJ-971
29 000-BLJ-965
30 000-000-000
31 000-BLJ-966
M67
1 000-000-000
2 000-BLG-886
3 000-BLG-887
4 000-BLG-888
5 000-BLG-889
6 000-BLG-890
7 000-BLG-891
8 000-000-000
9 000-000-000
10 000-BLG-892
11 000-BLG-893
12 000-BLG-894
13 000-BLG-895
14 000-BLG-896
15 000-BLG-897
16 000-BLG-898
17 000-BLG-899
18 000-000-000
19 000-BLG-900
20 000-BLG-901
21 000-BLG-902
22 000-000-000
23 000-BLG-903
24 000-BLG-904
25 000-BLG-905
26 000-000-000
27 000-BLG-906
28 000-BLG-907
29 000-BLG-908
30 000-BLG-909
31 000-BLG-910
32 000-000-000
33 000-BLG-911
34 000-BLG-912
35 000-000-000
36 000-BLG-913
37 000-BLG-914
38 000-BLG-915
39 000-BLG-916
40 000-BLG-917
42 000-BLG-918
41 000-BLG-919
43 000-BLG-920
44 000-BLG-921
45 000-BLG-922
46 000-000-000
47 000-BLG-923
48 000-BLG-924
49 000-000-000
50 000-BLG-925
51 000-BLG-926
52 000-000-000
53 000-BLG-927
54 000-BLG-928
55 000-000-000
56 000-BLG-929
57 000-BLG-930
58 000-BLG-931
59 000-BLG-932
60 000-BLG-934
61 000-000-000
62 000-000-000
63 000-BLG-935
64 000-BLG-936
To: Gordon,
Thanks for your reply. The only stars on PTGP5's NGC 7790 list that are not on my set of 22 stars are:
7 000-BLJ-972
8 000-000-000
11 000-000-000
12 000-000-000
18 000-000-000
22 000-BLK-004
26 000-BLK-022
27 000-000-000
30 000-000-000
If you do not include the repeative 000-000-000 stars, I'll be then missing only 3 AUID stars that are on your list imported into PTGP5! My list came from Arne's Standard Star set under the coordinates 23:58:23.2 +61:12:25 in the VSP as of 10-Sep-14. This 22-star Iist is very truncated as I have omitted most comps below 14.5 V-magnitude and many duplicates of stars brighter than v-mag 14 that have similar B-V values so I would have the largest variety of bright B-V stars to get transform coefficients on.
Here are the Tvb slope graphs of the NGC 7790 data I took, processed by PTGP4 and PTGP5 for the 30-Aug-14:
http://www.astroimage.info/files/slopes-30-Aug-14.jpg
Please note that both charts seem to capture the same stars in the slope-graph shown in both PTGP4 and PTGP5. It doesn't look like PTGP5 omitted much.
James
p.s. PTGP5's ability to save 6 data sets of TCs and average/saving them seems to work flawlessly with AIP4WIN instrument magnitude data:
http://www.astroimage.info/files/PTGP5-Jrf%28NGC7790-30Aug,6,7,10,11,&%2012sep14%29.jpg
Since were still a couple of months from getting M67 high up for transforms, I decided to process two of my M67 shots from January this year. One, ST-8XE, I shot from a dark sky, high elevation site, on 02Jan14. the other was shot from home (highly light polluted sea-level sky) with the ST-10XME. The Tbv looks worse on the M67 TCs because I was using another set of optics, namely the SBIG AO-L guide accessory; the ST-8XE looks worse on blue as well. Here is a small .txt file that has those transforms next to my average 6-transformed set of NGC7790 calculated TCs with AIP4WIN and Gordon's PTGP5.0:
FROM PTGP V5.0 Program using AIP4WIN
Instrumental magnitudes:
NGC 7790
M67-ST10XME
M67-ST-8XE
6-Sets Avg.
26-Jan-14
2-Jan-14
Tbv
1.382
1.468
1.5
Tb_bv
0.315
0.334
0.381
Tv_bv
0.038
0.015
0.048
Tvr
1.093
1.182
1.203
Tv_vr
0.072
0.034
0.097
Tr_vr
-0.013
-0.12
-0.072
Tri
0.97
0.96
0.932
Tr_ri
-0.016
-0.138
-0.081
Ti_ri
0.015
-0.097
-0.007
Tvi
1.033
1.067
1.062
Tv_vi
0.039
0.02
0.055
Ti_vi
0.007
-0.043
-0.003
Tr-vi
-0.007
-0.064
-0.038
James
Gordon,
Thank you for PTGP. I know it was a lot of work. I have been slow to adopt it because I wanted to work through a new set of transforms the "old fashioned" way with spreadsheets. Attached is just one spreadsheet for one set of observations at one aperture setting on one night. Multiply that by three sets of data a night, then by probably 3 different apertures selected by "visual inspection from perhaps measurements in 10 apertures, and then multiplied by 5 night of data. That is almost more spreadsheet fun than a person can stand. Now I am ready to compare the new, fast, easy and highly automated PTGP with the old, slow, mind-numbing and error-prone spreadsheet calculations and am anxious to do so.
Thank you for liberating us from indentured spreadsheet drudgery that yields only one tiny table harboring a pittance of numbers.
Asides:
A sharp eyed person might notice in the attachment that the average of residuals of the data vs. the trend line for Mvi isn't zero as the mathematics dictates it must be for a least squares fit. However, I have triple checked and find no error. The only cause that seems plausible is that LINEST in Excel has a convergence problem or reaches an iteration limit when the slope is so small.
I probably didn't have to make the 2nd-pass charts but I wanted to visually check that I had eliminated the outliers. before I applied LINEST directly to the data to get the slope error estimate as well as the slope..
Brad Walter, WBY
Ok this is only a comparison of one set of data on one good night but WoW! The results are very close. In all cases within a single standard deviation. PTGP and VPhot required a few percent of the time and less than 1% of the effor of grinding through the manual calculations.
The only improvement that I can envision for PTGP (besides a more memorable name) would be to be able to show point rejection criteria lines on the graph based on (perhaps a selectable) multiple of the standard deviation of the residuals of the points from the original trend line based on all points (they stay fixed even though the trend line moves). Selecting outliers by eye is pretty good but having an analytical criteria to help you identify the outliers to click on, would be nice. I don't think you want to replace the point-click outlier elimination with an automated elimination based on residuals. Looking at the data is important to determine that the regression line makes sense. You don't want something peculiar going on, such as the data failing to be homoscedastic. That would indicate extraneous systematic factors were giving you bad transformation coefficients and might completely invalidate the results depending on the severity and nature of the non random error. The human brain is often still the best tool for spotting bad data.
Honest, "homoscedastic" isn't a word I made up. I'm not that creative and it's too hard to pronounce after a glass of wine or two. Check out Anscombe's quartet for a classic example of non-homoscedastic data that give the same sample statistics as "normal" data with random error.
Brad Walter, WBY
WOW is right!
I'd call that NO difference.
Go VTransform ;-))
Ken
Kudo's again Gordon. Now I am doing the other 4 nights and as long as I can upload data to VPhot I will have them all done today. What an unbelievable difference in time and effort. For those that are compelled to look under the hood. I can make a copy of the spreadsheet I used to do the calculations. I guarantee that you won't do it manually more than once now that this program is out there combined with the wonderful ability of VPhot to automatically place all of the annuli That VPhot feature alone can save an hour or two.
Brad Walter, WBY
Brad,
Glad the program is helpful. I, too, spent many long nights with spreadsheets and just figured there had to be a better way.
I will look at adding the sigma error lines to the plot.
New name idea - maybe VTransformGen and then also change TransformApplier to be VTransformApply?? I'm open to anyone's ideas... George, Ken, Arne and Sara - what do you think?
Gordon
For those interested, I have a beta version of an updated PTGP which adds one sigma error lines to the plot. I removed the line on the plot showing the previous line fit (pre the most recent add/delete of a point) because it was interfering with what you see. I'm not convinced these one sigma error lines really help the analysis. If you want this file to test, please email me and let me know what you think.
gordonmyers@hotmail.com
Grrrrr! one of these days I will learn to compose all of my forum posts in Word before posting into the forum. For the third time in as many days I have closed Chrome when I only meant to close a tab and lost an essentially complete post
Gordon,
In the attached spreadsheet I compare results of the same data set comprised of means from 3 sets of observations very near meridian on a single night. The results are for no outlier rejection, version 5.1 with visual outlier rejection only, version 5.2 with 2.5 sigma outlier guidelines and version and version 5.3 with 3 sigma outlier guidelines. I like version 5.3 best.
Version 5.1 works very well EXCEPT when a difference of one or a couple of points changes the slope a lot and it isn't easy to tell which trend line tits better. That was the case in plot Ti_vi. In version 5.1 I eliminated only very obvious outliers. Including or excluding that single point way out to the right which lies outside the 3 sigma guidelines makes a big difference.
The version 5.3 initial and final plots can be downloaded from http://we.tl/nB8KYB7FVO .
I am keeping version 5.1 as well as 5.3, but I will use 5.3. I find the 3 sigma outlier guidelines extremely helpful even though the distribution of residuals clearly is not Gaussian.
Brad Walter, WBY
PS: Gary I have also included an xls version of the spreadsheet for those who can only open older versions of Microsoft Office files.
Can't change names after the product presentation!
We'll live we these children of ours as we've named them at the start.
George
I think Gorden's proposed names are brilliant!
Lot's of companies and products have changed names.
Photometrica became VPhot!!
If you like the name as is, that is a good reason to keep it.
Ken
Hello Gordon and George et al.,
I was not asked specifically, but I respond nevertheless
:
How about VTGen (for PTGP) and VTUse (for TA) (pronounced as VT-Gen and VT-Use)?
I am thinking descriptive, short and memorable (in the sense of memorizable, if that is an English word(?)...).
Cheers,
Helmar (AHM)
P.S.: And again: Thanks for two great programs!!
Thanks for mentioning Anscombe's Quartet -- I hadn't seen that before. It is brilliant.
And yes, I can vouch for your claim that homoscedastic is a word. I learned it in a Econometric course, that taught us that we should always apply the "interoccular trauma test", i.e. look at your data (graph it) and see if anything hits you between the eyes... so nicely demonstrated by Anscombe's Quartet!
Cheers, Gary Billings
I think changing the names to something starting with "V" is appropriate. Gordon's names are good starting points for discussion! My suggestion is to pick something shorter. It is VPHOT, not VPHOTOMETRY; VSP, not Variable_Star_Plotter; etc. VTCALC? VTAPLY? Think product marketing here!
Arne
At the Annual Meeting Gary Walker suggested we change PTGP to VTX.
I am actually of two minds about that. On the one hand it is short and simple, on the other it is awfully similar to VSP, VSD, and VSX. Will people be confused?
-Sara
VTX to me is V-transmit. Goes back to my amateur radio days! If you just want short acronyms, a couple of possibilities would be:
VTRC (V-TRansform-Calc)
VTRA (V-TRansform-Apply)
or even VTC, VTA.
Arne
Well, I like Arne's original suggestion because I can remember which of the two does what just by the name, But with practice, I could eventually remember which is what with the shorter acronyms as well.
In an earlier post I suggested to Gordon that it would be nice to have +/- ,say, 2.5 sigma (based on residuals) to use as guides for tagging outliers. Gordon did that quicker than I could type the suggestion and posted a notice in this thread about a beta version 5.2. I tried it on a set of data last night and compared it data for which I had previously pick outliers by eye. I really like having the outlier guide to help prevent bias from creeping in because we know what the desirable outcome is. I was also surprised at how much trouble I had comparing the distance from the trend of points above the trend line to those below. I often saw as equal distances that were different by 15% -20% and my ability to estimate the equality of distances was greatly affected by the positions of surrounding points. I think my suggestion of 2.5-sigma is too narrow after using it I concluded that 3-sigma would be a better choice when you have somewhere around 100 points rather than slightly less than 50 that I had previously used.
If you would like to see a comparison you can download plots and results comparison from http://we.tl/yeBGGKIOIs
I saved all of the png plots from the first pass selecting outliers by eye using version5.1. Using the 5.2 beta version I made PDFs of the initial plots showing the 2.5 sigma outlier limit guide lines before any points were tagged as outliers. Then I saved the png files of the plots with outliers beyond the outlier guide lines tagged. My method for tagging an outlier was that the point had to be completely outside and not touching the original 2.5 sigma lines. The lines shift as you tag points and you should select outliers based on the original position of the outlier guide lines. Otherwise, you are iteratively selecting outliers which except in rare cases is not statistically sound practice. If you compare the plots it becomes obvious why my initial suggestion of 2.5 sigma is a bit too narrow for this many points.
Gordon, Thanks for the incredibly fast response to a late suggestion.
Brad Walter, WBY
I like VTran for the name of the program.
Barbara
I like VTran too, but since there are two programs, what about VTranCalc and VTranApply?
Mike Simonsen
I like VTC for the PTGP and VTA for the applier. I can remember them easily.
Gary
It appears that most agree that the use of V is good to yield a consistent set of names for AAVSO software? The disagreement occurs with respect to the number of letters. How many are "good"?
I think if one shrinks it too far to an acronym that needs to be remembered in a brute force fashion, but has no internal hint to what it is about, is not a good choice, e.g, VTC? We have PTGP now.
Having some abbreviation in the name that helps prompt a recollection and some hint of what it does is a good thing? Is VTransform too long? It also leaves Transformation Applier without a similar V name, although VTApplier would work.
Mike and Arne proopsed something similar. VTRCalc or VTCalc, VTAply or VTApply. I think VTCalc and VTApply work in terms of length and use of abbreviation (spelling?). I'm not sure the R adds much but that is subjective call.
Ken
I represent the SPVD, the Society for Prevention of Vowel Discrimination. We are shocked, and dismayed that an organization such as AAVSO that has more vowels than consonants in the acronym for the organization's name and one that has at least adequately supported equality for vowels with such names as VPhot and Vstar should contemplate new acronyms lacking vowels.
We demand that new acronyms contain at least one vowel. If not, vowels will boycott all AAVSO communications. Jst tr wrtng wtht vwls fr whl nd s hw lk t.
NO LESS THAN ONE, NO LESS THAN ONE, NO LESS THAN ONE NO LESS THAN ONE, NO LESS THAN ...
for the SPVD
Brad Walter, WBY
"Terrance, this is stupid stuff: ..."
I think it will take longer to name the program than it took me to write it :)
How about VOEAIOAAA for V (mandatory) + only extremely accurate ideal observations allowed at AAVSO? I think that covers the vowels... :)
I'm enjoying watching the ideas. I lean toward VTCalc and VTApply but will take direction from whomever...
Gordon
Well,
Mine had a very little humor and I will not discuss names further. I don't care what you call it as long as you don't call me late for dinner. I can only repeat, at least for my own involvement,
"Terrance, this is stupid stuff:
You eat your victuals fast enough
But ponder nonsense far too much ..."
Apologies to Mr Housman, or responding only in vowels, A. E., I O U.
Brad Walter, WBY