Model averaging with ridge regularization

0576123 - NHU-C 2024 CZ eng V - Research Report
Skolkova, Alena
Model averaging with ridge regularization.
Prague: CERGE-EI, 2023. 31 s. CERGE-EI Working Paper Series, 758. ISSN 2788-0443
Institutional support: Cooperatio-COOP
Keywords : linear regression * shrinkage * model averaging
OECD category: Applied Economics, Econometrics
https://www.cerge-ei.cz/pdf/wp/Wp758.pdf

Model averaging is an increasingly popular alternative to model selection. Ridge regression serves a similar purpose as model averaging, i.e. the minimization of mean squared error through shrinkage, though in different ways. In this paper, we propose the ridge-regularized modifications of Mallows model averaging (Hansen, 2007, Econometrica, 75) and heteroskedasticity-robust Mallows model averaging (Liu & Okui, 2013, The Econometrics Journal, 16) to leverage the capabilities of averaging and ridge regularization simultaneously. Via a simulation study, we examine the finite-sample improvements obtained by replacing least-squares with a ridge regression. Ridge-based model averaging is especially useful when one deals with sets of moderately to highly correlated predictors because the underlying ridge regression accommodates correlated predictors without blowing up estimation variance. A toy theoretical example shows that the relative reduction of mean squared error is increasing with the strength of the correlation. We also demonstrate the superiority of the ridge-regularized modifications via empirical examples focused on wages and economic growth.
Permanent Link: https://hdl.handle.net/11104/0345726