Speaker
Description
Artificially generated stimulus material and digital entities controlled by artificial intelligence (AI) offer immense potential for linguistic and cognitive research. Imagine, for instance, stimulus material that dynamically adapts to a study participant’s behaviour or textual input. Or consider an interaction between a human participant and an AI-backed agent whose linguistic and social-cognitive behaviour is both configurable and convincingly natural. These technologies enable the creation of highly dynamic and naturalistic, yet more controllable, study designs—combining experimental rigor with increased ecological validity, particularly in a world where digital interaction is becoming more and more normal.
In this contribution, I present a selection of my recent work using virtual avatars (fully generated by AI) and artificial agents (especially chatbots) to investigate a range of linguistic and social-cognitive research questions. First, I showcase studies exploring the integration and processing of social and linguistic cues (e.g., facial expressions aligned or misaligned with utterances) and the perception of non-verbal feedback signals (e.g., head nods) in conjunction with verbal responses. In both cases, the textual material, speech output, and video stimuli (representing avatars) were generated using state-of-the-art AI text-to-speech and text-to-video tools.
Second, I present pilot work demonstrating how text-to-speech models can be directly embedded into experimental scripts to enable prompting-based designs. These allow for dynamic, yet trackable, subject-specific adaptation that responds in real time to participants’ behaviours. Especially these behaviour-adaptive designs offer considerable promise for cognitive research. Third, I illustrate how chatbots (using an adapted version of the LeoLM model) can be employed and, crucially, modified to investigate linguistic alignment phenomena and the emergence of language-based affect in digital interactions.
Importantly, I do not focus on the specific findings of individual studies in the present contribution but rather, I aim to provide a broad overview of the potential of these tools for addressing diverse research questions in linguistics and interaction research and focus particularly on methodological aspects.