Looking at smilesheets through an HRD practitioner lens: a mixed methods approach to exploring their contents and explaining their usage in practice
Martin Lauzier et al.
What the paper says
Purpose Despite huge investments made in training every year, few organizations evaluate their training. When they do, their evaluation focuses mainly on trainees’ reactions collected through smilesheets. Regardless of smilesheets’ popularity in practice, little is known about the principles guiding organizations in this area. The same goes for the aspects that organizations deem important enough to include in these assessment tools. In view of the criticism to which reaction-based evaluations are subjected, it is essential to gain a clearer understanding of the factors that explain their continued use in practice. Hence, the purpose of this study is to propose to lift the veil on these aspects by adopting a practitioner’s perspective. Design/methodology/approach A mixed methods approach was adopted to address two objectives: provide an overview of over 100 tools used to measure trainees’ reactions (retrospective phase); and explore practitioners’ views on the possible reasons for the use of these tools in practice (prospective phase). Findings The results from the retrospective phase indicate that organizational practices for collecting trainees’ reactions remain highly heterogeneous and muddled. Results from thematic analysis in the prospective phase suggest that evaluation of trainees’ reactions remains partly based on intuition or a preference for simplicity and habitual approaches rather than on evidence-based practices. These results suggest that organizations are not taking full advantage of the many relevant suggestions and recommendations in the literature. Practical implications This research provides valuable information on the current state of organizational practices regarding the evaluation of trainees’ reactions and the reasons why practitioners continue to rely on such assessment tools. Originality/value To the best of the authors’ knowledge, this study is the first to present a “behind the glass” view of smilesheet content and usage in practice.
Evidence weight
Balanced mode · F 0.40 / M 0.15 / V 0.05 / R 0.40
| F · citation impact | 0.50 × 0.4 = 0.20 |
| M · momentum | 0.50 × 0.15 = 0.07 |
| V · venue signal | 0.50 × 0.05 = 0.03 |
| R · text relevance † | 0.50 × 0.4 = 0.20 |
† Text relevance is estimated at 0.50 on the detail page — for your query’s actual relevance score, open this paper from a search result.