Thu, 08/06/2015 - 13:15
Hello everyone,
If you receive images from OC61 please be aware we have to reprocess several nights from mid-late July and into August, and you may received duplicates of images you've already seen and processed. The cause was a problem with file transfers from the site itself to the intermediate server at SWRI. We're reprocessing to pick up missed images and redo calibrations. This will happen between now and Saturday evening, following completion of fixes to VSP API.
Apologies for the inconvenience,
Matthew
Hello everyone,
This was delayed over the weekend, but we are now reprocessing OC61 data.
This will likely take the better part of a day, ending some time before
start of business Tuesday morning.
Matthew
Matthew,
It seems like this need for reprocessing AAVSOnet images happens at least once or twice a year for the last three years. I could see having an occasional problem with an isolated night's data, but having to reprocess a month's worth is unusual. Is there a problem with the pipeline?
As an observer, I appreciate all that HQ does to keep AAVSOnet up and running, but having to remeasure a month's worth of images is a huge inconvenience and shouldn't be necessary. Please make whatever changes to the pipeline are needed so this never happens again.
Thanks,
Bob
Hi Bob,
The reprocessing of OC61 data is not Matt's fault; it primarily rests on my shoulders. The process is more complex than it should be, partly due to the three parties that use the telescope, partly due to instrument swaps with associated flatfield changes, and partly due to image transfer to an intermediate site and then on to AAVSO headquarters. Since flats are taken over multiple nights, any missing flatfield data means that current processing is not using timely flats. While we could just overlook the use of older flats, my desire is to use the best quality calibrations, so when we discover a discrepancy, we reprocess. The trick is to improve the pipeline so that these discrepancies occur with less frequency. I've taken some recent steps that should eliminate much of the issue. I apologize for any extra analysis on your part that is involved, and really appreciate the fact that you are inspecting your images quickly and telling us when you see potential problems.
Arne
Arne,
Thanks for the quick reply. I wasn't trying to assign blame for any problems. I just wanted to understand the nature of the problem since it was so unusual.
Thanks,
Bob