Using AI-Generated Avatar Interactions To Study Socio-Affective Functions of Feedback Signals in Language: an Exploratory Experimental Study on Head Nods

What the paper says

Feedback signals are a central aspect of human interaction that often fulfil not only an interaction-structuring function (e.g., prompting a speaker to continue his story) but also serve socio-affective purposes. While both head nods and verbal continuers are used to signal affirmation, approval and continuation in a given interaction, head nods are also claimed to index social affiliation and stance-taking – in contrast to verbal continuers where this function is less clear. To investigate this potential functional difference further, I present the results of a pilot study that used AI-generated videos of dyadic interactions in which one avatar narrates a positive or negative life event while another avatar (the recipient) simply listens and occasionally provides feedback in the form of head nods, verbal continuers, a combination of both, or no feedback. While watching the videos, participants continuously rated how affiliative they perceived the behaviour of recipients in response to narrators and afterwards rated how socially adequate and empathic the recipient was. While recipients displaying head nods only or both feedback signals were rated as behaving more affiliative, more socially adequate and more empathic, avatars using verbal continuers only received lower ratings, with these ratings varying also depending on the story’s valence. The results of this study support the idea that head nods and verbal continuers differ in their socio-affective function and illustrate that using AI-generated avatars is a useful addition to interactional research, particularly to shed light on specific functions underlying different feedback signals.