Evaluating “Effectiveness” under the Policy on Results
Ardyn Nordstrom
What the paper says
Evaluation units provide essential information for administrators of public programs. Canada's 2016 Policy on Results emphasizes evaluating “effectiveness,” with a focus on outcomes. However, it is unclear if evaluation units use this or more implementation‐oriented definitions in practice, which has implications for the information evaluations will provide. This study examines 370 evaluation reports published by the largest federal departments to determine how evaluation functions are approaching effectiveness, using a combination of text mining and manual thematic analysis. Since 2014, evaluation reports have increasingly adopted implementation‐oriented approaches, with variations across departments. Explanations for these patterns are discussed, along with policy implications.
Evidence weight
Balanced mode · F 0.40 / M 0.15 / V 0.05 / R 0.40
| F · citation impact | 0.50 × 0.4 = 0.20 |
| M · momentum | 0.50 × 0.15 = 0.07 |
| V · venue signal | 0.50 × 0.05 = 0.03 |
| R · text relevance † | 0.50 × 0.4 = 0.20 |
† Text relevance is estimated at 0.50 on the detail page — for your query’s actual relevance score, open this paper from a search result.