COVID-19 virus has spread to most countries around the world, negatively affecting people livelihood. Providing accurate forecasts of COVID-19 cases can help governments to find the optimal combination of measures. In this study, a sequential ensemble forecasting approach that combine the gated recurrent unit (GRU) model with particle swarm optimisation (PSO) algorithm is proposed for forecasting of COVID-19 cases. The PSO method was used to select the best hyperparameters of the base predictor of the proposed model. The t-test was used to statistically compare the suggested model against a single optimised GRU, in addition to other benchmark models. Results revealed the superiority of the proposed method. Further, adding models sequentially improved the forecasting quality, compared to a single PSO-GRU model, the mean error was reduced by 15.52%, 16.05%, 16.53%, 16.39%, and 12.83% in terms of RMSE, MAP, MAPE, RMSPE, and RMSLE, respectively.