Upcoming changes in our infrastructure

Affiliation
American Association of Variable Star Observers (AAVSO)
Fri, 08/03/2018 - 18:17

Friends,

 

With this post, I want to give you a heads up for some positive changes that are related to our web and software infrastructure. Behind the scenes at the AAVSO HQ, we are constantly working diligently on ensuring that our computer infrastructure is up to date, and that all software necessary for your hard work is functioning properly. A major project that is currently underway is the upgrade of our web page content management software from Drupal 7 to Drupal 8. As Drupal 7 will not be supported any more, this necessary upgrade ensures that our web page and all its links (including Django apps) will be working smoothly in the near future. Our fundraising efforts this year help provide the financial resources to subcontract necessary experts for this project, and I am grateful for all your support!

 

While planning this migration, we found out that Drupal 8 is very different in its structure and in what software it supports, and that we need to adjust some of our Django apps accordingly. Essential applications (such as webobs and the data download tool) need major re-writing to ensure that they work. Forums are not supported by Drupal 8; since they are very important for communication within our community, we are working to identify a solution for this problem. Databases need to be connected to various tools in a reliable way, to ensure continuous access. At the same time, some pieces of old software will have to be discontinued. For example, the old light curve generator (lcg1) will no longer be supported; the new tool (lcg2) that was announced in 2017 and has been in use for more than a year now, will be replacing lcg1 for a quick lookup of data. Furthermore, the look-and-feel of our web page will have to change as the current one is a custom Drupal theme. We will try to replicate as much of it as possible. I am sure you will notice differences – all of which will ensure easy access to valuable tools, while highlighting your hard work and provide information on courses, meetings, the JAAVSO and AAVSO resources.

 

We aim to introduce the new web page/tools in ~2 month’s time. We need to make sure that our software and databases are stable and work well before launch, so we are enabling time for extensive beta testing. I will keep you posted on our progress as necessary. Thank you for your understanding and support, while we work on ensuring that essential tools are functioning seamlessly for our international community. The AAVSO is marching confidently forward in the 21st century, despite technological challenges.

 

Best wishes – clear skies,

Stella.

Affiliation
American Association of Variable Star Observers (AAVSO)
Hi Stella,

Hi Stella,

That certainly explains a few problems seen at this end - including a PHP error message last night, which ideally users should not be able to see as it can expose things like file and folder information on the server (as indeed it did in this case), database structures and so on. I guess that is built into many CMSs like Drupal and you have to turn off error messages in the CMS's dashboard (if possible). Weird that forums aren't supported now, since I believe that was one of the main reasons for developing CMS's as blog hubs in the first place!

Affiliation
American Association of Variable Star Observers (AAVSO)
errors turned off

Hello,

Thanks for the heads up. I didn't realize this setting was enabled, and you are correct it should not have been. It has been turned off and errors like that will no longer be displayed.

Thanks,

Bert Pablo

Staff Astronomer, AAVSO

Affiliation
Vereniging Voor Sterrenkunde, Werkgroep Veranderlijke Sterren (Belgium) (VVS)
A pity that LCG1 disappears

Hi Stella,

I find it a pity that LCG1 will disappear. I still like it very much. One can copy it into documents, it is much faster than LCG2 if large datasets are displayed (it is a pain with LCG2) and is easily been modified in terms of displayed time intervall. All this is much more cumbersome with LCG2. I guess you will see a lot of people complaining if LCG1 will cease to exist.

Josch

 

Affiliation
American Association of Variable Star Observers (AAVSO)
A pity that LCG1 disappears

Hi Josch,

Thank you very much for explaining why you prefer LCGv1. Your comments are very valuable.

Just out of curiosity, have you tried using VStar for creating plots for publication purposes? It is very flexible in terms of appearance and scale. You can change colors, dot sizes and shapes, axis labels and scales, include or exclude error bars, mean curves, highlight your own data, create phase plots, and much more. I prefer VStar to any other tool for making nice-looking easily customizable plots.

Best regards,
Sara

Affiliation
Vereniging Voor Sterrenkunde, Werkgroep Veranderlijke Sterren (Belgium) (VVS)
Never tried VSTAR

Hi Sara,

I have never tried VSTAr as it was not the preferred app to be chosen rom the menues.

I did not even know it can handle those data you mention. Will try it next time.

Josch

Affiliation
American Association of Variable Star Observers (AAVSO)
LCG1 on Apple Devices

If I understand things correctly, I will no longer have access to a light curve generator for my Ipad and Iphone.  These are the devices that I use for AAVSO activities including looking up and printing light curlves.   Only LCG1 works for me.  If someone knows how to use my devices using LCG2, please let me know.   Otherwise, I would not consider the deletion of LCG1 to be a positive change.  

Thanks,

Rich GRIB

Affiliation
American Association of Variable Star Observers (AAVSO)
LCG1 on Apple Devices

Hi Rich,

I am able to plot light curves on my iPad using LCGv2. Admittedly, it only gives the light curve and not the table of observers or any of the other interactive features (but that was intentional for the small screen). If this is not working for you, please contact me offline and we can compare notes and discuss your specific needs.

Many thanks,
Sara

Affiliation
American Association of Variable Star Observers (AAVSO)
Keep LCG V1 at any cost

I cannot accept the abandonment of LCG V1. It is at the center of my daily workflows.

Affiliation
American Association of Variable Star Observers (AAVSO)
7 to 8 Downgrade not upgrade?

7 to 8 Downgrade not upgrade? Lose LCG1, lose forums, major rewrite of critical core apps - WebObs, data downloader = asking for major trouble!?

Affiliation
Vereniging Voor Sterrenkunde, Werkgroep Veranderlijke Sterren (Belgium) (VVS)
I agree with Mike

I agree with Mike, if that much change and loss is apparent then that is not an upgrade. Maybe there is another solution possible to this issue.

I tried VPHOT on a star for which I prepare a poster and there are still a couple of issues which are much simpler in LCG1 and maybe not existing in VPHOT (e.g. to select my own observations).

Josch

Affiliation
American Association of Variable Star Observers (AAVSO)
Drupal 8's an upgrade, all

Drupal 8's an upgrade, all right. And there seems to exist a forum module in Drupal 8, so I don't understand that problem as stated. Either way I'm sure AAVSO will have a forum so that's not really germane.

But if LCG v1 is thrown away, then to continue observing for AAVSO I would probably have to write a hook into WebObs and write my own plotting software. Besides making me stop and rethink the time and expense of my observing in toto, certainly that WebObs approach would increase my demand, every day, on AAVSO's servers compared to just keeping LCG V1 (please). Unfortunately LCG V2 is too hopelessly slow for my work (80-120 plots per day).

 

Affiliation
American Association of Variable Star Observers (AAVSO)
More Technical Information

Hello All,

 

Let me provide a little more technical information to supplement Stella's post. Our infrastructure has not changed noticeably in several years. While this might seem stable, the reality is that computer technology changes on a 1-2 year time scale, meaning that virtually everything on our web server is out of date, so much so that many of the versions of software that we are currently using have been deprecated. This is extremely problematic from a security aspect as new vulnerabilities will inevitably arise and we will have no way to address them.  To provide a safe, stable infrastructure, updating drupal and all of the other, more minor, software packages is an absolute necessity. This change will necessarily come with deprecation of some of our pieces of software where a replacement exists. Many of these programs are written in very old versions of programming languages (php for example) and are extremely costly to port, rewrite and maintain. Moreover, we simply don't have the resources currently on staff to support multiple versions of tools such as the light curve generator. Each time we make security updates we have to make changes to two entirely separate tools (not simply two versions of the same lcg tool), which is diverting resources from other tasks.  We do however, value your opinion and want our resources to meet your needs as much as is possible and feasible. Feel free to contact us to let us know what functionality is most important to you. Our goal is to provide a consistent, sustainable architecture that will be able to meet your needs not just now, but for many years to come.

 

 

Thanks,

Bert Pablo

Staff Astronomer, AAVSO

Affiliation
American Association of Variable Star Observers (AAVSO)
Re: Upcoming changes in our infrastructure

I use an older laptop for my AAVSO stuff, and find that the LCG V2 uses too much of my PCs resources. So, I use LCG V1 most of the time. Any loss of LCG V1 would put a crimp in my VSOing as others in this thread have mentioned.

Why is it that many so-called upgrades lately just break existing software? There was a similar problem earlier this year with the computer that handles the processing for AAVSOnet. An "upgrade" of Linux to the newest version broke some of the installed software and required a reinstall/upgrade/modification of IRAF just to get things working again.

Is this desire to upgrade the AAVSO website software about adding functionality or is it more about security? If this is just about security, then maybe that issue needs to be dealt with separately so individual apps don't need to be rewritten so often.

Bob

Affiliation
American Association of Variable Star Observers (AAVSO)
LCG v2 is EXTREMELY FAR from

LCG v2 is EXTREMELY FAR from "the same lcg tool" as LCG v1. People have advertised it that way a million times, so a million times they've been wrong. V2 is a different tool altogether--for me unusable.

In the end, my own work would suffer more from throwing away LCG v1 than it would from throwing away VSX.

 

Affiliation
None
I'll try to adapt but...

I am about to begin observing again after several years.  I had made myself an Excel spreadsheet with hyperlinks that would bring up LCG v1 for a given set of parameters and also a link with an extra click would generate a chart for me.  I have never been able to make it work with LCG v2.  I'm also afraid that the chart generation links won't work either after the upgrade.  If someone could show me how to do this with LCG v2 and if the chart generation would work the same, I'll be happy.

I understand the need to upgrade for security reasons and for lack of vendor support but a liitle help for members who have developed their own tools would be appreciated.

Tim

Affiliation
American Association of Variable Star Observers (AAVSO)
LCGv2 API access

Hi Tim,

I don't think we are (intentionally) going to break anything having to do with VSP - the chart plotter.

As for getting lightcurves from LCGv2, it does have the ability to create light curves using an API.

Here is an example:

https://www.aavso.org/LCGv2/static.htm?DateFormat=Julian&RequestedBands=&Grid=true&view=api.delim&ident=X%20CrB&fromjd=2457608.1668854053&tojd=2458338.1668854053&delimiter=@@@

The "?" next to the URL (which can be seen when you click "Preferences") says this:

A Static light curve displays only the curve and symbols. The viewer has no selections.
This URL is the address to this current light curve. You may use this address to call this curve directly. It can also be placed as link address in other applications, such as journals or reports.
You may also use this format to call your own static light curve. You can change the following values:

https://www.aavso.org/LCGv2/static.htm?
DateFormat=Julian (or Calendar)
&RequestedBands=Vis,V,R (or, blank for all bands)
&Grid=true (or false)
&view=api.delim
&ident=V482 CYG (Star Name)
&fromjd=2457201 (Julian Start Day)
&tojd=2457931 (Julian End Day)
&delimiter=@@@

The URL and values are Case Sensitive.

There is also an API to get a "dynamic" version of the plot:

https://www.aavso.org/LCGv2/index.htm?DateFormat=Julian&RequestedBands=&Grid=true&view=api.delim&ident=X CrB&fromjd=2457608.1668854053&tojd=2458338.1668854053&delimiter=@@@

If this does not work for you, if you find a bug with it, or if there is some way in which it could be tweaked to improve the results, I would really like to hear from you so we can try to fix it.

Many thanks,
Sara

Affiliation
None
Hi Sara,

Hi Sara,

Thanks very much for that example.  I'll give it a try when I'm back at my home computer.  One quick question: Can I call up a light curve for the last X number of days?  That was easy to do with the API  to LCG v1.

Tim

Affiliation
American Association of Variable Star Observers (AAVSO)
Plot last X days

Hi Tim,

Yes that is easy. Just ignore the dates given in the "From Date:" and "To Date:" boxes and instead, put in the "Plot Previous" box the number of days you wish to plot (counting backward from today). Then click the checkbox outlined in green marked "Days" as shown in the attached screenshot. The From and To dates will then turn green and reflect the JD of the new date range.

Alternatively, if you wish to select "All" data for a star, just click the checkbox outlined in brown that is labelled "All:"

I hope that helps,
Sara

Affiliation
American Association of Variable Star Observers (AAVSO)
Plot last X Days continued

Hi again Tim,

Re-reading your message I see you were talking about the API grabbing the last X days not directly using the LCGv2.

I don't think that such capability exists yet, but I can certainly request it if you think that would be useful.

Best regards,
Sara

Affiliation
American Association of Variable Star Observers (AAVSO)
Never change a wining tool !

Houla, LCG1 could be discontinued!!! that’s so sad!
It’s the worst news that I ever received from the AAVSO.
I understand the security issues but…
For my personal usage AAVSO do have 3 superb working tools, VSP, WebObs and LCGv1.
I tried to use LCGv2 it’s much more complex and so slow on my computer, that this is nearly impracticable to use.   Just as an example SS Cyg cost me 21 seconds on LCGv2 and only 3 seconds on LCGv1.  I tried to use VStar as well; I had so many difficulties with technical errors and database issues that I had to disregard this tool as well, maybe I'm wrong.  As you know probably I write articles in scientific French magazines as the last one on Mira, without LCGv1 my workload will be too heavy to be efficient.  I make also a lot of conferences and demonstrations to push people to observe; sometime I use just my Ipad to show a real-time curve on a specific star… Imagine to do that with LCGv2 or Vstar !  I just need a simple, fast, working, and issue free tool, and today you have one.  If you have to make a choice, delete the version 2 of LCG.
I hope sincerely that you will reconsider deleting this so fantastic tool.  Please!
Never change a wining tool, sometime this type of decision could be a disaster!

PS: How to adress continuity as in the attached file ?

Michel

Affiliation
American Association of Variable Star Observers (AAVSO)
LCG V1 - I give up

Thanks to Sara and Bert for their helpful responses offline--sincerely appreciated. But I've now decided to write my own local LCG. I start this afternoon. Even if AAVSO should now reverse its LCG V1 decision, that reversal could be reversed, perhaps without notice. I do not accept that risk.

The unexpected and unwelcome effort to cover for V1's loss now or later will, unfortunately, have consequences. It's best I make them clear at the outset. (1) The upload format review and WebObs archive db coding I started in December must be suspended. Sorry, George. By tomorrow I will complete my last repo push, and someone else can take it from here. Or conceivably I could but just when I couldn't say. (2) If AAVSO trashes LCG V1 before I can finish my own LCG, then of course my Mira/LPV monitoring stops until it again has the tool it needs. This could put the first major gap in data whose value emerges from its very absence of gaps. But again, any such gap in AAVSO's database would result directly from AAVSO's own decisions.

Finally, thanks to the original, unknown author of LCG V1, and its unsung maintainers since! It's been a good show, but these rolling credits tell me the V1 show is over, and it's time to move on somehow.

Affiliation
Vereniging Voor Sterrenkunde, Werkgroep Veranderlijke Sterren (Belgium) (VVS)
Try to plot ASASSN-18ey with LCG2

Try to plot ASASSN-18ey with LCG2. It tells you "Data not found for this star". LCG1 can do the job, though (less than 30 sec). I tried VSTAR which takes its time (more than 45 min, then I gave up). Be aware that there are thousands of observations for this star, Hence except LCG1 all the other tools are unusable for stars with lots of data.

So far I always used LCG1 to make plots for presentations and posters. Cannot imagine to do similar plots with LCG2 (sorry to say). I agree with others that LCG2 is not at all at the level of easiness and versatility as LCG1.

Josch

Affiliation
American Association of Variable Star Observers (AAVSO)
Try to plot ASASSN-18ey with LCG2

I've had the same experience with both LCG2 and VStar Josch.  I've been able to get LCG2 to plot  the last couple fo days of ASASSN-18ey.....sometimes

Jim

Affiliation
American Association of Variable Star Observers (AAVSO)
Addressing Concerns

Hello All,

Thanks for the feedback so far. We want to keep an open dialogue as there are many changes coming and we want you to be a part of them. I want to reiterate one point to everyone: It isnt' that we want to update our infrastructure, we need to. There is a mountain of work that goes into this, but many things on our site have slowly been breaking.  Unfortunately, there isn't much we can do about them without redesigning how our website functions. I do want to assuage any concerns about functionality though. For the most part, we will have the same functionality as before, it just may take a slightly different form. 

Since one of the major concerns of lack of functionality has to do with LCGv1, I have started a new forum thread  here:

https://www.aavso.org/lcg-v1-issues-clarification-understanding-and-sol…

In order to for us to address your concerns we need to understand your workflow better. Therefore, we encourage to give us more feedback on LCGv1 at the link above. We realize that change is difficult, but it is also inevitable. However, we want to work with you, our community,  to make this transition as painless as possible.

Thanks,

Bert Pablo

Staff Astronomer, AAVSO