Trade-offs between internal and external validity in economic experiments to inform agricultural policy
Thomas Rellensmann & Fabian Thomas
What the paper says
Abstract Economic experiments are increasingly used to evaluate agri-environmental policies ex-ante. They typically rely on one of three subject pools: standard subject pool students, agricultural students, or farmers. While farmers are often considered most informative for policymaking, experiments outside controlled lab environments face challenges in identifying causal effects. Thus, trade-offs between internal and external validity arise but remain underexplored empirically. Using exploratory analyses, we compare conservation behaviour under a novel agri-environmental policy, four behavioural factors, and methodological measures across farmers, agricultural students, and standard subject pool students in lab, classroom, and online settings in Germany. Conservation behaviour and environmental preferences differ significantly across pools, indicating that student samples – even with agricultural backgrounds – cannot fully substitute for farmers. Yet agricultural samples also show threats to internal validity, including more extreme response times, miscomprehension, and attrition. Trade-offs are thus evident, demonstrating that experiments with standard subject pools and agricultural samples are complements rather than substitutes. We formulate design recommendations for researchers to address threats to internal validity in experiments involving agricultural samples.
Evidence weight
Balanced mode · F 0.40 / M 0.15 / V 0.05 / R 0.40
| F · citation impact | 0.50 × 0.4 = 0.20 |
| M · momentum | 0.50 × 0.15 = 0.07 |
| V · venue signal | 0.50 × 0.05 = 0.03 |
| R · text relevance † | 0.50 × 0.4 = 0.20 |
† Text relevance is estimated at 0.50 on the detail page — for your query’s actual relevance score, open this paper from a search result.