Code and Data Repository for An Efficient Jackknife Model Averaging Method

Abstract

Model averaging, an effective method for reducing uncertainty in data modeling and learning, has been widely applied across various fields, including economics, finance, and operations research. The leave-one-out cross-validation (LOOCV) criterion is a widely used technique to choose weights in model averaging. It has achieved great success, but its application is limited by the high computational complexity especially when the sample size is large. To address this issue, focusing on generalized linear models (GLMs), we propose a computationally efficient and approximate leave-one-out cross-validation (ALOOCV) weight choice criterion, which no longer suffers from the need to repeatedly solving optimization problems many times. Our theoretical analysis presents the discrepancy between the weight estimators produced by LOOCV and ALOOCV which is proved to vanish asymptotically under certain conditions. Moreover, the efficient jackknife model averaging (EJMA) procedure with ALOOCV for GLMs is proposed. The asymptotic optimality under the model misspecification and the convergence rate of the selected weights by ALOOCV are derived. And the over-consistency of the weight estimators and the consistency of averaging coefficient estimators are established if there exists at least one correct model. Additionally, to reduce the computational burden caused by an excessive number of candidate models, we provide a model screening procedure. Numerical experiments show the remarkable computational efficiency exhibited by EJMA relative to the model averaging procedure with LOOCV and the excellent prediction performance of our method over some common model averaging and selection procedures.