Transforming Ensembles

Affiliation
American Association of Variable Star Observers (AAVSO)
Sun, 03/15/2015 - 14:27

The Campaign thread is getting a bit long, so I'm going to break out a few subtopics (and actually get involved with answering questions!).  The first of these is dealing with ensembles.

Certainly using a single comp and the target star is a simple operation, for both determining the differential photometry and applying the transformation coefficients.  The CCD Manual describes the process.

For ensembles, there are many aspects to consider.  Let's assume for this post that you are doing the brute-force ensemble: find the standardized magnitude for each target-comp pair, and average those standardized magnitudes together to form the final result.  If this ensemble calculation is then written to the AAVSO Extended Format, you will have: the average standardized magnitude for the target; the word "ensemble" for the "comparison" star; the average standardized magnitude for a check star.  Some observers take an extra step and actually report in the notes field the AUIDs or labels of the stars used for the ensemble.

TA cannot handle this record because there is no single comparison star.  However, transformation IS important even for ensembles, as has been mentioned before - often the color of the comparison stars is quite different than the color of the target star (red LPVs are a classic example).  So how do you transform such observations?

Option#1:  find the mean magnitude and color of the comparison-star ensemble.  Use this as the CREF information, and you will get close to the right value.  Note that if the ensemble changes from night to night (or even during the night for a time series), you will have to recalculate these mean values.

Option#2: transform each target-comp pair individually, then average the results in a second step (say, with Boxster).  This should give you a very similar value to Option#1.  Note that VPHOT only does this manually (use one comp, calculate the magnitude, write to a file, and then manually select another comp, etc.).

Option#3: doing an all-sky solution, obtain the transformed magnitudes for all of the stars involved (the target and the comps).  Then apply ensemble techniques as a second step.  This option does not use TA.  This option is what I personally use.

None of these options are handled automatically in the current setup.  My recommendation is that you either consider using the classic target-comp-check processing, avoiding ensemble for the time being, or else you use option#2 with Boxster.

One of the reasons why ensemble is not handled as cleanly as you might expect is that there are many different ways of doing ensemble.  The best that I've found is by Kent Honeycutt, called inhomogenous ensemble photometry:

1992PASP..104..435H

which uses flux weighting for each comparison star, and explicitly handles differing numbers of comparison stars on multiple images.  My personal program takes a different approach, weighting comparison stars based on their closeness in magnitude, color and spatial distance to the target.  However, I don't know of any software package that does anything other than the brute-force technique.  Ensemble photometry is very useful, as it not only averages out possible problems in your image, but also gives you a lot of diagnostic information.  However, if I had to choose between transforming my data with a single comparison star, or not transforming but using an ensemble, I'd definitely choose to transform.

Arne

Affiliation
American Association of Variable Star Observers (AAVSO)
Transforming Ensembles

Hi Arne,

Thank you for the usefull directions. All seems more clear about ensemble transforms.  The only doubts and hesitations that I have are related with time series photometry that I do most of the time (6-8 hours) in different air mass and sky conditions. I am not sure whether can I implement the described methods and how difficult it can be. Let us now to leave aside the question that I use mainly Sloan type filters.

Regards,

Velimir

Affiliation
American Association of Variable Star Observers (AAVSO)
Arne,
Thanks for your help -

Arne,

Thanks for your help - Mike

<Option#2: transform each target-comp pair individually, then average the results in a second step (say, with Boxster).  This should give you a very similar value to Option#1.  Note that VPHOT only does this manually (use one comp, calculate the magnitude, write to a file, and then manually select another comp, etc.).>

I am currently taking images of M67 for developing my coefficients. Below is a report generated from a not–ready–for–prime–time set.  The report was a result of selecting a subset of Standard stars as targets (3) and one Check star so that I get see how well the coefficients get to the Standard magnitudes. Using VPHOT, I checked the Transform box and provided a stacked image of V and of I filters.  The sequence used includes the remaining Standard stars (>50) in what I thought would be an ensemble, calculated as you mention in Option#2.  I assumed that this was the case for the Targets since at the bottom of the report it shows the Check star calculated value as an average of the list of Comp stars magniudes (I did the arithmetic  to confirm this). An excerpt from the report is pasted below.  Would you confirm one more time that the Targets are not an average of the Comp stars.  I am also unsure about how the errors are calculated.  I realize that my questions are a mix of ensemble and How-does-VPHOT-work questions.

Observation date V:

2015-03-10 03:55:45

Observation date I:

2015-03-10 04:09:21

JD V:

2457091.66372

JD I:

2457091.67316

FWHM V: 2.311

FWHM I: 3.553

Average FWHM: 2.932

Aperture: 4.397 pix

 

Target Star Details

Results

Details

Name

V*

I*

(V - I)*

V-img

I-img

 

Err

Std

SNR

123 MID TO RED

12.387

11.415

0.972

http://184.73.185.143/app/img/cim40ecjosmk4f0hfypyvkqx/855496_123%20MID%20TO%20RED.jpghttp://184.73.185.143/app/img/cim40ecjosmk4f0hfypyvkqx/855494_123%20MID%20TO%20RED.jpghttp://184.73.185.143/app/img/gf.pngV:

http://184.73.185.143/app/img/if.pngI:

 

0.042

0.050

 

0.042

0.050

 

226

326

 

112_1 MID

11.272

10.212

1.060

http://184.73.185.143/app/img/cim40ecjosmk4f0hfypyvkqx/855496_112_1%20MID.jpghttp://184.73.185.143/app/img/cim40ecjosmk4f0hfypyvkqx/855494_112_1%20MID.jpghttp://184.73.185.143/app/img/gf.pngV:

http://184.73.185.143/app/img/if.pngI:

 

0.042

0.050

 

0.042

0.050

 

475

585

 

114 - Red

11.507

10.473

1.034

http://184.73.185.143/app/img/cim40ecjosmk4f0hfypyvkqx/855496_114%20-%20Red.jpghttp://184.73.185.143/app/img/cim40ecjosmk4f0hfypyvkqx/855494_114%20-%20Red.jpghttp://184.73.185.143/app/img/gf.pngV:

http://184.73.185.143/app/img/if.pngI:

 

0.042

0.050

 

0.042

0.050

 

413

519

 

 

Check Star Details

Results

Details

Name

V*

I*

(V - I)*

V-img

I-img

 

Err

Std

SNR

113_1 BLUE

11.335 
(0.000)

11.001 
(0.000)

0.334 
(0.000)

http://184.73.185.143/app/img/cim40ecjosmk4f0hfypyvkqx/855496_113_1%20BLUE.jpghttp://184.73.185.143/app/img/cim40ecjosmk4f0hfypyvkqx/855494_113_1%20BLUE.jpghttp://184.73.185.143/app/img/gf.pngV:

http://184.73.185.143/app/img/if.pngI:

 

0.042

0.050

 

0.042

0.050

 

457

400

 

 

Catalog Information

Details V

Details I

Check est.1

Incl

Name

V

I

V - I

I.M.

FWHM

Max

SNR

I.M.

FWHM

Max

SNR

V*

I*

 

132_1

13.220

12.569

0.651

-6.390

1.864

5537

137

-6.245

3.131

2126

167

11.327

10.984

 

105

10.524

9.471

1.053

-9.096

2.508

34566

713

-9.366

3.773

21157

846

11.335

11.023

 

132_7

13.204

12.535

0.669

-6.375

2.182

4840

125

-6.279

3.542

1974

164

11.297

10.987

 

135

13.509

12.847

0.662

-6.159

1.657

4193

115

-6.036

3.619

1646

148

11.386

11.055

 

132_5

13.285

12.618

0.667

-6.330

1.948

4439

133

-6.209

3.832

1740

158

11.332

10.999

 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 

 

Average

 
 
 
 
 
 
 
 
 
 
 

11.335

11.001

 

Std

 
 
 
 
 
 
 
 
 
 
 

0.042

0.050

 

1 Check star estimates applies to star 113_1 BLUE

Affiliation
American Association of Variable Star Observers (AAVSO)
time series

Hi Velimir,

There are at least three ways of transforming time series.

1) use the single-filter transformation that I mention in a separate thread.

2) Take a second filter exposure at the start, midway through the time series, and at the end.  Assume some form for the change in color during the series, by averaging the 3 color determinations, using a linear fit, etc.  Apply the proper color for each individual filter exposure during the series.

3) take the series by alternating filters, such as BVBVBV...  This is the preferred way if you have sufficient cadence to resolve any light variations of interest.  Then it is a simple application of TA.  This is the approach that I almost always use.  In many cases, there is as much astrophysical interest in the color change of a variable as a function of time as there is in the overall light variation.  For example, an eclipsing binary may change color during an eclipse, telling you the temperature of the two stars, necessary for modeling the system.

My approach is always to get the most astrophysical information as possible out of my observing time.  Then the observations can be used for many different future research projects and I've maximized my return.  Obtaining the highest precision (and accuracy!) is part of that picture, so I always shoot for high signal/noise and transformation, even if that means monitoring fewer targets.

Arne

Affiliation
American Association of Variable Star Observers (AAVSO)
Transforming Ensembles

Hi Arne,

Thank you for the information. The 3-th example is what I always have done. If I use the single Comp-Check stars it is easy to do transformation. That’s easy.

The differences are that I use SG-SR-SI (more often SG-SI) time series photometry, and to do transformation is not so easy due to the lack of AAVSO sequences for standard stars, and TG, and TA do not work with Sloan type data. They can work but as George Silvis have shown to me with a few tricks.

Anyhow I have to do my homework first with the ensemble to see how the thing are going on.

Velimir

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Question about ensemble extended format records

Why does the AAVSO extended format have CMAG set to "na" when it is an ensemble record?

VMAG is set to "average standardized magnitude for the target" as Arne stated above which is for n stars in the ensemble:

sum( (vs - vcn) + Vcn  )/n  =

vs - sum(vcn)/n + sum(Vcn)/n  ie, the (obs instrumental mag) - (avg comp star instrumental mag) + (avg comp star reference mag).

If CMAG were defined as the average comp star instrumental mag then Option #1 is workable: All you need is to use TA's CREFMAG mechanism to provide the (avg comp star reference mag).

Without this CMAG information you can't get the target back to the instrumental magnitude.

I'm suggesting a change to the extended format rules. Include information that is available when the ensemble is assembled so that we have some transform options down stream. 

George

 

 

 

Affiliation
American Association of Variable Star Observers (AAVSO)
VPhot Ensemble Photometry

Arne:

I believe your comment about VPhot not conducting Option 2 without repetitively calculating the transformed magnitude with one comp at a time is not correct.

Based on my email to Geir to confirm, transformed ensemble photometry is carried out all at one time. When one selects two filter images (e,g, VR) and clicks Transform, all the comps in the ensemble are initially used to calculate a transformed target magnitude from each comp. Subsequently, the final reported transformed target magnitude is calculated by the average of each comp's derived transformed magnitude.

I have attached a print screen of this color photometry page.  It does not prove my conclusion directly but Geir's response to my email states that this is the case.

Ken 

Affiliation
American Association of Variable Star Observers (AAVSO)
2-filter transform with VPHOT

Hi Ken,

Thanks for setting me straight.  My comment was in regards to using VPHOT to generate a file for TA to use, not the internal transformation method that VPHOT uses for the 2-filter transform.  Sorry to have not made that clear!  So if you can work within the constraints that VPHOT currently has for transformation, it handles ensembles correctly.

Arne

Affiliation
American Association of Variable Star Observers (AAVSO)
2 filter transform with VPhot

Thanks Arne:

Oops, I should have read your preceding sentence more carefully: "TA cannot handle this record because there is no single comparison star."

I guess we are both right about the different circumstances. I suspect it would be a lengthy upgrade to get VPhot to handle transformed ensemble time series!  ;-(

Ken

Affiliation
American Association of Variable Star Observers (AAVSO)
Lesve Photometry

Pierre,

I realize that this thread is a bit dated, but if you are following it perhaps you can describe how your program approaches this.

Thanks

Cliff

Affiliation
American Association of Variable Star Observers (AAVSO)
Boxster

I'm not familiar with Boxster and the search engines seem to be overloaded with an automobile of the same name.

Can anyone give me a link to look into this?

thanks

Cliff