Having read through the recent publication "Analysis of Seven Years of Globe at Night Data", JJ Birriel, et. al. (JAAVSO vol. 42, 2014), I am pleased to see the results of this significant worldwide study on trends in night sky brightness. However, I think one of their main conclusions, that "over the period from 2006 to 2012 global averages of NELMs have remained essentially constant" is not properly supported by their data analysis.
Specifically, their conclusions based upon the statistical analysis of the NELM trend from 2006 to 2012 in Figure 3, appear to be faulty. The problem is they use a simple linear regression on data that is clearly not "straight-line" to get a correlation coefficient of R^2 = 0.6 yielding a p-value = .041, which they claim is not significant enough (at the 3 sigma level) to justify the conclusion that NELM values have been decreasing.
However, visual inspection of the plot, shows a marked drop in NELM values (approx. - 0.4) from 2006-10 to 20011-12 intervals. Also, a residual vs. fits plot of their linear regression shows a clear systematic over-under bias of the straight line fit, rather than a nice random spread of the data, as would be seen if a good model was used to fit the data.
I used Excel to do a basic second order regression on the data (plot attached), and get an R^2 = 0.85 which corresponds to a p-value = 0.003, which is significant at the 3 sigma level. A better model such as this second order fit matches the observations more cleanly, and would provide statistical support of the more reasonably obvious conclusion, that the global averages of NELM have in fact declined significantly over the period 2006 to 2102.
I hope the authors will take note of this issue, and correct the analysis of the data, particularly as shown in the regression on Figure 3.
Mike LMK
Author J. Birriel has asked me to post the response below on her behalf.
M. Saladyga
====
Our choice of a linear fit is appropriate for a couple of reasons.
1. A study of North America light pollution from 1947-2000 indicates a linear increase over this time period. Although if one selects smaller time intervals in the data it does exhibit a slight non-linear trend just as our data.
"The Growth of the Artificial Night Sky Brightness over North America in the Period 1947-2000: A Preliminary Picture" by Cinzano, P.,
in Light Pollution: The Global View. Edited by H.E. Schwarz, Cerro Tololo Inter-American Observatory, NOAO, La Serena, Chile. Astrophysics and Space Science Library Volume 284 Kluwer Academic Publishers, Dordrecht, 2003., p.39-46.
2. Physically the second order polynomial doesn’t make sense – it implies a slight decrease in brightness from 2006-2008 followed by a more rapid increase in a short time. In particular, it is unlikely that the brightness of the night sky increased by 0.4 magnitudes in a single year.
The large standard deviation in Mean NELM from year to year is more likely due to the fact that the different observers participate from year to year.
====
I thank the author for her quick reply, however I remain unconvinced that the linear fit of their data is appropriate, nor that their conclusions that NELM have remained essentially unchanged is correct. Specific replies below:
[quote=M.Saladyga]
Our choice of a linear fit is appropriate for a couple of reasons.
1. A study of North America light pollution from 1947-2000 indicates a linear increase over this time period. Although if one selects smaller time intervals in the data it does exhibit a slight non-linear trend just as our data.
[/quote]
The fact that a linear increase in light pollution was found in the period 1947-2000 has limited value for this study. First, this study covers a period well outside this time span (2006-2012) and thus requires an extrapolation of a previous trend into "unknown territory". In fact, isn't a prime purpose of this new study to determine if the light pollution trends have remained the same, worsened or improved? Assuming a linear trend from a previous, disconnected time period introduces inappropriate bias into the study.
[quote]
2. Physically the second order polynomial doesn’t make sense – it implies a slight decrease in brightness from 2006-2008 followed by a more rapid increase in a short time.
[/quote]
My particular second order fit is not based on a physical model, but simply a better fit to a non-linear set of data points than a linear fit, using the basic polynomial trendlines available in Excel. Physically, of course, a linear fit over the long term does not make sense, since NELM was always constant at 6.5-7 for all human history prior to the industrial revolution, yet in the future it could continue to worsen. Thus, the true slope of a physical model must start out at zero and tend to increase, which is more in line with a second order fit, or exponential fit, rather than a linear fit.
[quote]
In particular, it is unlikely that the brightness of the night sky increased by 0.4 magnitudes in a single year.
The large standard deviation in Mean NELM from year to year is more likely due to the fact that the different observers participate from year to year.
[/quote]
If the authors believe that the data is affected by a large 0.4 magnitude systematic error 2010-2012, then certainly the regression analysis showing a poor correlation with a linear decline in NELM, and the conclusion then drawn that there is then "no significant change" due to statistical confidence, is clearly no longer valid. Calculated correlation coefficients and 2- or 3-sigma statistical confidence limits require the underlying data contain only random error! If systematic error is their assumption, then the better course of action would be to attempt to determine why there is a large discontinuity in the NELM over one year, and present some evidence to support it. Certainly, the statistical analysis of Figure 3 becomes inappropriate and misleading, and no proper statistical conclusions can be drawn from it.
I hope the authors can appreciate the veracity of my arguments and make the proper corrections to their study and its conclusions.
Thank you,
Mike LMK
The author's response to your posting is given below.
====
Thank you for your interest and for raising some interesting points to consider. As the paper has already been accepted via the referee process it would not be appropriate to make further changes to the analysis in the paper. These issues would best be addressed by a followup paper sent through the referee process. I am currently working on a comparison of GWWSC and GaN data from 2007-2013 and will address these issues in this subsequent paper. -JJB
====