This paper assesses the robustness of the popular Goyal and Welch graphical procedure which has been extensively used in the literature to evaluate the performance of predictive models for stock returns among other contexts. To this end, we simulate the graphical diagnostic and construct a sign-based test allowing us to examine its behaviour under various sample sizes, data generating processes and levels of correlation. Our simulations reveal that correlation does have an effect on the graph in smaller samples but the technique is quite robust when sufficiently large data sets are employed. Moreover, we demonstrate that the graphical diagnostic is generally well-sized and yields a satisfactory power performance in most cases. This result holds also under the assumption of heteroskedasticity. Overall, our analysis suggests that the graphical diagnostic can be an important complement to the more conventional methods seeking to assess out-of-sample predictive ability.