FITS images with negative pixels, plate solving, and interpreting negative values

Affiliation
American Association of Variable Star Observers (AAVSO)
Sat, 09/22/2012 - 13:47

See below screen shots.  First one is from AIP4WIN...examining a calibrated image that I uploaded to VPHOT yesterday as a test.  (It's the first/earliest V filter image in a small image set I uploaded to my account.)

Note that in AIP4WIN pixel (X,Y) 331, 237 has a negative ADU value....   -3.

Second screen shot shows the display of ADU values for that region (of the very same image) from VPHOT.  It does not show a negative number...it shows it as zero.  (I have confirmed it's the same image...I have looked at pixel/ADU values for non-negative values...and the x,y pixel coords and ADU values match up perfectly.)

AIP4WIN says the FITS header:

BZERO   = +3.141300000000E+004                                                 
BSCALE  = +1.000000000000E+000

VPHOT says FITS header:

BZERO   = +3.141300000000E+004
BSCALE  = +1.000000000000E+000
...they appear to be the same values.

When I calibrate images...a handful pixels end up with negative values.  Two issues:

1.  VPHOT does not appear to handle/display them properly.  That can have an impact on photometry math.

2.  It spoils plate solve attempts on shorter exposures (on dark sky, with 'dark' filters, such as B)...when there may be hundreds of negative pixel values in the sky in the calibrated image.  And based on my experience before with PinPoint...I think PinPoint treats a negative pixel (e.g. ADU = -3) as a pixel of ADU = 65535 - 3 = 65532....which is a very bright false star that confuses PinPoint...especially when there are dozens or hundreds of them in the image.  (If I take a 'problem' image that won't solve in PinPoint because of negative pixels...and create a new image, with a constant ADU value added to all pixels so that there are no more negative pixels....the image plate solves beautifully.  You can see this change in the PinPoint log...instead of detecting 500 stars (because there are 300 negative pixels falsely recognized as stars)...it now detects 200 stars.)

Third screen shot shows histogram of image that solved in VPHOT...and there are a surprising number of 'slightly negative' pixels.

What can we do to improve the way VPHOT handles images with a handful of negative pixels?  We can improve the photometry results, and improve the probability of successful PinPoint plate solving.

Thanks in advance.

Affiliation
Variable Stars South (VSS)
Negative pixels

I have very dark skies and found that my background is very close to the offset value for the images. When this is subtracted with darks etc I have quite a few negative pixels.

I get around this problem by adding an offset of 200 to my images after preprocessing. This gets rid of all the negative pixels. I then save the image as a 16bit unsigned fit file. This makes all the images have a range from 0 to~65000 so it doesn't affect pinpoint etc.

Cheers

Terry

Affiliation
American Association of Variable Star Observers (AAVSO)
Not following the FITS standard

Your negative pixels from dark skies...the same problem I face.

But because a big-name software application does not follow the FITS standard (which allows signed integers...i.e. negative as well as positive)....you are forced to deviate from proper calibration protocol.

Not good.  But that's the world of 'common practice today is....'

If you and I, and all others that properly calibrate (in dark skies, using shorter exposures)...stick with proper calibration...and then try and use various software packages...we are 'punished' because we don't follow 'common practice.'

How much does this deviation from the FITS standard, and deviation from correct calibration...skew/alter CCD photometry results?

Affiliation
Variable Stars South (VSS)
Negative pixels

[quote=KTC]

Your negative pixels from dark skies...the same problem I face.

But because a big-name software application does not follow the FITS standard (which allows signed integers...i.e. negative as well as positive)....you are forced to deviate from proper calibration protocol.

Not good.  But that's the world of 'common practice today is....'

If you and I, and all others that properly calibrate (in dark skies, using shorter exposures)...stick with proper calibration...and then try and use various software packages...we are 'punished' because we don't follow 'common practice.'

How much does this deviation from the FITS standard, and deviation from correct calibration...skew/alter CCD photometry results?

[/quote]

If I don't add the offset and then save as an unsigned fit file then it does have an effect on the photometry. If you look at the histogram for the image there is a large number of pixels at 0 that looks like a cut off gaussian. All the info below this level has been lost. If I add the 200 count offset then the histogram is then a gaussian so it shouldn't have any effect on the photometry.. The background is subtracted in the photometry algorithm so any offset has no effect on the final results.

Not sure what software you are talking about but I just use the standard software for my SBIG camera for capture. The camera has an offset level of 100 and this is subtracted out at preprocessing. Preprocessing lots of images with different filters all in the same imaging run is not handled well by all of the software out there. I use MIRA for preprocessing because it seems to handle the different filters well. It is based on the IRAF but is a bit expensive for what you get.

 

Cheers

Terry

Affiliation
American Association of Variable Star Observers (AAVSO)
FITS standard - it's good. Use it.

If I don't add the offset and then save as an unsigned fit file then it does have an effect on the photometry

The FITS standard allows pixels to be negative.  Why deviate from the standard?  Probably because some big-name software doesn't handle *proper/correct* FITS files...so you follow 'common practice.'

(Remember mom saying:  "And if everyone were jumping off a bridge...you would too?!"  ;-) )

If I add the 200 count offset then the histogram is then a gaussian so it *shouldn't* have any effect on the photometry.

That key word - should.  Is this your opinion, or is it fact?

Not sure what software you are talking about....

For starters, Maxim, PinPoint, and CCDSoft.  Too bad they do not properly handle correct FITS images that have negative pixels.  (OK, with Maxim you can alter a setting to 'always add a constant to eliminate negative pixels'...which should not have to be done because the FITS standard says that negative pixels are acceptable.)

The camera has an offset level of 100...

This is a typical bias value for many SBIG CCD's.

...is subtracted out at preprocessing.

And if your exposures are rather short, the sky dark, the f/ratio not-fast, and the filter is low throughput...you will end up with a good number of negative pixels...which is ok.  The math/concepts/electronics/physics allow this...and so does the FITS standard.  But big-name software out there will choke on files with many negative pixels...which is not good.

OK, you cave in to the popular software...and add a constant to all pixels to make the software happy...but does that step alter the photometry, or otherwise alter the science value of the processed image?

Opinions not permitted here, only hard math and proofs.

Am I too serious about this?  How serious would you be with ten scopes sitting on your hill? ;-)