Affiliation
American Association of Variable Star Observers (AAVSO)
Tue, 04/23/2013 - 01:08
I just uploaded some fits files and they are displayiing all white. The fits header seems to be OK. I can open and view these fits files in AIP4WIN and MaxIm DL. The fits files pass fitsverify (http://fits.gsfc.nasa.gov/).
I tried using Tools > Image Display in Vphot, but hat does not have any effect.
An example fits file is attached.
File Upload
V380-Oph.fits18.01 MB
If you select 'Pixel/ADU mode' and click anywhere in the image you will see that all pixels have ADU = 0. So the image is in fact blank.
I can't say why, though. Perhaps something happened during calibration?
Geir
The CALIBRATED images display fine in AIP4WIN, MaxIm DL, Aladin, . Also, the fits header shows:
CBLACK 1957
CWHITE 2532
If I download the image from VPhot, I can display the images with no problem.
VPHOT uses the CFITSIO library for reading FITS files (http://heasarc.gsfc.nasa.gov/fitsio/), and it gives the following error message on your image:
BAD_ELEM_NUM 308 illegal starting element number in vector
I have no idea what that means, so I'm afraid I won't get any further. Could be a bug in the library.
Geir
This is a 1536x1024 QSI 516ws (KAF-1603 sensor). The FITS image itself is a floating-point file, with three tiles. These all look like V images of the field; I don't know why you have made it a 3-plane image. You might try uploading each V image separately and see if that takes care of the problem; I don't know if VPHOT understands the 3rd dimension.
BTW - each plane has reasonable pixel values, ranging from a hard minimum of 0 (this is unphysical, BTW, after processing, so your software is truncating your file, something that would be important for short exposures where there is little sky background) to about 34K. I also notice that your flat exposures indicate subsecond values; I do not recommend this, even with something like the QSI's photometric shutter.
Arne
I'm an absolute novice at variable star observing and VPhot so the problem most likely is my ignorance. I am seeing the same blank (white) image in VPhot as the original poster.
The file passes both the VPhot Wizard test and fitsverify (the status, after upload, shows partial calibration. I'm not sure why since the CALSTAT keyword is "BDF"). The original image is visible in both PixInsight and AperturePhotometryTool as is the VPhot imaged downloaded after processing. The pixel/adu mode also shows all zeros.
File attached if anyone has time to review it. Thanks,
Bob
The problem turns out to be in the size of the image. The image was originally saved as a 32 bit floating point, I changed the format to 16 bit unsigned integer and the image is now displayed normally. Onward and upward!
Bob
I thought I should summarize my findings for anyone who might follow this thread.
The normalization to [0.0,1.0] that Arne mentioned is definitely happening in PixInsight at the point where I did a "Save As" to convert the .xisf PI format to .fit 32 bit floating point. I tried the BatchProcessing|BatchFormatConversion (BFC) script, with the Output extension set to ".fit" and the Output format hint set to "unsigned", which did not work (in VPhot) either.
No method (that I discovered) in PixInsight would convert the .xisf files to a 32 bit floating point fit format that VPhot could use.
What I finally did was batch convert to .fit format using the BFC script Sample format output option set to 16 bit unsigned integer. All files successfully uploaded to VPhot, processed, and data reduced.
Thanks to all who helped me with this problem.
Bob
Bob:
FYI, 32 bit floating point images (bitpix= -32) can normally be opened, viewed and reduced (photometry) in VPhot. I just ran such an image to convince myself that nothing had changed.
It appears to be some other issue that caused the white background? Could you share the image in question with me at MZK? Tell me your observer code and I can share my test image with you.
Sometimes the cblack and cwhite levels can get mesed up?
BTW, look at the earlier posts in this forum thread above about an identical/similar problem. No obvious solution but apparently problems with image?
At least you were able to use the workaround with a 16 bit image!
Ken
Thanks Ken,
I appreciate the help. I've uploading one of the 32 bit images and have shared it with you. It is 1 of 16 that I originally uploaded, none of which were visible. Unfortunately, the new upload overwrote the 16 bit version with the same timestamp. No big deal, I can reload it later.
The 16 bit files that are there now worked great (after a few false starts) and generated a nice data set for KV Gem. I certainly hope I can get the 32 bit images working, it is a real pain to reformat the to 16 bit.
Yes, please share one of your 32 bit images and I'll try it here. It is probably some keyword in the header that is wrong on my end. My observer code is TBOA.
Bob
I looked at the FITS 32-bit floating point image that Bob uploaded. This image is normalized, running from a minimum of 0 to a maximum of 0.987. As it was processed by PixInsight, I'm going to guess that the normalization took place there, and when you stored it a second time as a 16-bit image, the normalization was removed and you got more normal pixel values.
When VPHOT tries to display an image whose maximum value is near 1, it will likely look like a white image. I'd consider this a minor flaw of VPHOT. Certainly DS9 and IRAF can display images no matter how they are scaled, including those with negative numbers (which many commercial programs have trouble with).
When you have trouble displaying images, it is always good to find some program that will give you basic image statistics. Look at min and max values, the mean or median of an image, and often the standard deviation of an image. For typical CCD images of the night sky, min and max will likely be from about 0 to maybe 50,000 and the mean or median will usually be ~100 counts (at least, something low in the dynamic range). Standard deviation should be non-zero, probably in the 100 count range as s.d. takes into account both the deviation in the sky background as well as the much higher values in star images themselves. You can often inspect an image that you know works, and then compare with a deviant image to see the differences.
I also tend to look at the fits header to see what keywords are present. Most image processing systems give you the option of displaying this header. If not, a kludge is to use a 24x80 terminal window and page down through the image. Another useful package to look at is fv from the NASA/HEASARC people, which is available on many platforms. There are a certain subset of keywords that are absolutely required, and some optional ones (like FILTER) that VPHOT requires as well. Make sure all are present.
Good luck on variable star observing! It is always nice to see someone coming from the deep sky imaging community and discovering the variable universe.
Arne
Arne,
Thanks! That is good information to have. I need to check the PixInsight docs to see how to avoid the issue in the future. VPhot is really amazing once you begin to get the hang of it, my first practice time series has gotten me hooked.
Bob
I uploaded some images into VHOT this weekend and I seem to be getting the totally white image problem as well. This is the first set of data processed a new installation of IRAF. I used the same scripts as before, but I'm at a loss as to how to resolve the issue, as the images display fine in DS9. As a check, I uploaded an old image (SS Cyg) processed previously and it displays normally in VPHOT.
I examined the images. The cells are floating point, ranged from 0 to 1.0
Somewhere in your new processing pipeline you need to find where this normalization happens and turn it off. VPhot is expecting ADU's.
Ok, many thanks. The hunt begins....
Update: the probelm turned out to be the flats weren't being normalized properly. So the flat calibration
using imarith operator was giving the wrong values.
.
PixInsight scales everything to 0.0 to 1.0 (floating point), by design. They consider it a feature, not a bug.
I saw this same image white-out a couple of years ago, and I wrote a script to reverse their scaling, then I just gave up on PixInsight. Their math is beautiful, but for photometry I just found it too hard to clean up after.