Improving sub-seasonal hydrological forecasts utilizing the randomness in Deep Learning models

What the paper says

Recent advances in Deep Learning have demonstrated significant potential for improving hydrological forecasts, yet the inherent stochasticity of these models–particularly the impact of random seeds–remains underexplored. Whereas reproducibility often dictates fixing random seeds, the resulting variability of multiple random seeds in model performance across stations and lead times presents an opportunity to enhance probabilistic forecasting through combination techniques. We use the Temporal Fusion Transformer (TFT) model to forecast water temperature and streamflow across 22 gauging stations in Switzerland. The TFT is trained globally, optimizing quantile losses for all stations and lead times jointly. A key aspect of our study is that here we use multiple random seeds when training the TFT model, resulting in an ensemble of seed-specific models to generate predictions. To optimally combine the forecasts of these different models we use two methods: Nonhomogeneous Gaussian Regression (NGR) and Beta-transformed Linear Pool (BLP). These combination methods improve the predictive skill compared to forecasts from the individual seed-specific models. The average Continuous Ranked Probability Score across stations and lead times for water temperature (streamflow) predictions improved from 0.85 °C (0.95 mm/d) for the average of the seed-specific models to 0.73 °C (0.81 mm/d) for the NGR and 0.73 °C (0.79 mm/d) for the BLP. Whereas both methods perform similarly well for predicting water temperature (near-Gaussian), the BLP performs better for streamflow as it is less dependent than the NGR on the underlying distribution of the data. Nevertheless, the skill of the NGR streamflow predictions can match those of the BLP by assuming a heavier-tailed distribution such as the lognormal. Overall, we demonstrate that Deep Learning model ensembles built from random seeds, coupled with principled combination methods, can improve forecast skill across hydrological variables.