Understanding the decision-making process of choice modellers

Abstract

Choice Modelling is a widely used framework for understanding human choice behaviour across disciplines. Building a choice model is a complex, semi-structured process that involves a combination of prior assumptions, behavioural theories, and statistical methods. This complex set of decisions, coupled with diverse workflows, can lead to substantial variability in model outcomes. To investigate these modelling processes, we introduce the Discrete Choice Modelling Serious Game (DCM-SG), a novel tool that mimics the workflow of choice modellers and tracks the modelling decisions participants make. In our application, participants developed models to estimate willingness-to-pay values for reducing noise pollution. Their actions were tracked, enabling analysis of workflow patterns and modelling strategies. Forty participants, most with over five years of experience, completed the game. Our contributions are twofold. Methodologically, the DCM-SG captures sequential data on modellers’ workflows, which we analyse using telemetry and sequential pattern mining techniques to uncover dynamic patterns of in-game tool usage, phase transitions, and model specification approaches. Substantively, there was a strong preference for data visualisation and frequent specification of simpler models (Multinomial Logit), alongside attempts to specify more complex models. These findings suggest that in time-constrained or resource-limited settings, modellers may underexplore important factors such as covariates, non-linearities, and complex specifications. Moreover, participants who engaged more thoroughly in data exploration and iterative model comparison consistently achieved superior model fit and parsimony. These results demonstrate how sequential data from the DCM-SG can uncover variations in modelling practices and provide a foundation for understanding the art of choice modelling. • Introduces the first serious game designed to capture the choice modelling process. • Provides a methodological contribution to analyse modellers’ decision-making process. • Reveals heterogeneity in workflows and modelling strategies among choice modellers. • Shows that data exploration and iterative refinement improve model fit and parsimony.