Model Evaluation Guidelines for Geomagnetic Index Predictions

0499196 - ÚFA 2019 RIV US eng J - Journal Article
Liemohn, M. W. - McCollough, J.P. - Jordanova, V. K. - Ngwira, Ch.M. - Morley, S. K. - Cid, C. - Tobiska, W.K. - Wintoft, P. - Ganushkina, N.Yu. - Welling, D.T. - Bingham, S. - Balikhin, M. A. - Opgenoorth, H.J. - Engel, M.A. - Weigel, R.S. - Singer, H. J. - Burešová, Dalia - Bruinsma, S. - Zhelavskaya, I.S. - Shprits, Y. Y. - Vasile, R.
Model Evaluation Guidelines for Geomagnetic Index Predictions.
Space Weather-the International Journal of Research and Applications. Roč. 16, č. 12 (2018), s. 2079-2102. E-ISSN 1542-7390
EU Projects: European Commission(XE) 776011 - TechTIDE
Institutional support: RVO:68378289
Keywords : forecasting * geomagnetic indices * metrics * ROC curve * space weather * statistical analysis
OECD category: Meteorology and atmospheric sciences
Impact factor: 3.691, year: 2018
https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2018SW002067

Geomagnetic indices are convenient quantities that distill the complicated physics of some region or aspect of near‐Earth space into a single parameter. Most of the best‐known indices are calculated from ground‐based magnetometer data sets, such as Dst, SYM‐H, Kp, AE, AL, and PC. Many models have been created that predict the values of these indices, often using solar wind measurements upstream from Earth as the input variables to the calculation. This document reviews the current state of models that predict geomagnetic indices and the methods used to assess their ability to reproduce the target index time series. These existing methods are synthesized into a baseline collection of metrics for benchmarking a new or updated geomagnetic index prediction model. These methods fall into two categories: (1) fit performance metrics such as root‐mean‐square error and mean absolute error that are applied to a time series comparison of model output and observations and (2) event detection performance metrics such as Heidke Skill Score and probability of detection that are derived from a contingency table that compares model and observation values exceeding (or not) a threshold value. A few examples of codes being used with this set of metrics are presented, and other aspects of metrics assessment best practices, limitations, and uncertainties are discussed, including several caveats to consider when using geomagnetic indices.
Permanent Link: http://hdl.handle.net/11104/0291442