Incentive alignment in conjoint analysis: a meta-analysis on predictive validity

Abstract

Conjoint analysis is a widely used method in market research for predicting consumer purchases, making predictive validity a central tenet. Conjoint analyses, however, are typically conducted in hypothetical settings, making them susceptible to hypothetical bias. One solution is incentive-aligning conjoint studies to trigger truthful answering behavior, thereby increasing the accuracy of predictions. However, despite incentive alignment’s conceptual appeal, practitioners rarely use it. One reason for this is the uncertainty of its effectiveness. This research systematically investigates the gains in predictive validity employing a meta-analysis of 134 effect sizes from 34 articles ( N = 12,980). Incentive alignment increases the predictive validity (i.e., hit rate) by 12%, providing a significant increase in accuracy. In addition, its effectiveness is amplified when researching durable and service goods (vs. non-durable goods) and when the payout probability rises. In contrast to conventional wisdom, indirect (vs. direct) incentive procedures do not mitigate the positive effects on predictive validity. We hope to stimulate a rethink in practice to make more use of incentive alignment and help decide whether incentive alignment is worth the additional effort.