Detecting Test Speededness Using Responses and/or Response Times: Change Point Analysis Approaches Based on Schwarz Information Criterion

What the paper says

Change point analysis (CPA) detects structural shifts in a response sequence by partitioning it into segments with different statistical properties. This paper proposes three CPA approaches based on the Schwarz information criterion (SIC; hereafter SIC-CPA): response data only, response time (RT) data only, and the combination of response and RT data, to detect the prevalent test speededness in time-limit tests. To comprehensively investigate the efficiency and accuracy of the proposed approaches, six simulation studies were conducted under diverse conditions. Simulation results demonstrate that SIC-CPA can effectively enhance the power of change point detection and reduce Type I errors, while improving computational efficiency compared to the likelihood ratio and Wald tests. Moreover, the SIC-CPA combining response and RT data outperforms the SIC-CPA based solely on RTs, and the latter is substantially superior to the SIC-CPA based solely on responses. In addition, SIC-CPA accurately identifies two change points in RT patterns, corresponding to early warm-up and later test speededness. Using an iterative detect-clean-recalibrate procedure, SIC-CPA achieves more reliable Type I error control than likelihood ratio and Wald tests when item parameters are estimated from contaminated data. A real data analysis was conducted to show the application of the proposed approaches.