Empirical investigation of the temporal relations between speech and facial expressions of emotion
Article dans une revue avec comité de lecture
Date
2010Journal
Journal on Multimodal User InterfacesAbstract
Behavior models implemented within Embodied Conversational Agents (ECAs) require nonverbal communication to be tightly coordinated with speech. In this paper we present an empirical study seeking to explore the influence of the temporal coordination between speech and facial expressions of emotions on the perception of these emotions by users (measuring their performance in this task, the perceived realism of behavior, and user preferences).We generated five different conditions of temporal coordination between facial expression and speech: facial expression displayed before a speech utterance, at the beginning of the utterance, throughout, at the end of, or following the utterance. 23 subjects participated in the experiment and saw these 5 conditions applied to the display of 6 emotions (fear, joy, anger, disgust, surprise and sadness). Subjects recognized emotions most efficiently when facial expressions were displayed at the end of the spoken sentence. However, the combination users viewed as most realistic, preferred over others, was the display of the facial expression throughout speech utterance. We review existing literature to position our work and discuss the relationship between realism and communication performance. We also provide animation guidelines and draw some avenues for future work.
Files in this item
Related items
Showing items related by title, author, creator and subject.
-
Article dans une revue avec comité de lectureNELSON, Julien;
BUISINE, Stéphanie; GAZO, Claude;
AOUSSAT, Améziane (Presses Universitaires de France, 2014)
Prospective ergonomics is concerned with the anticipation of future needs and activities to assist the early stages of user-centered innovation design projects. Few studies have proposed methodologies to assist prospective ... -
The effects of speech–gesture cooperation in animated agents’ behavior in multimedia presentations Article dans une revue avec comité de lectureUntil now, research on arrangement of verbal and non-verbal information in multimedia presentations has not considered multimodal behavior of animated agents. In this paper, we will present an experiment exploring the ...
-
Article dans une revue avec comité de lectureTo increase the believability and life-likeness of Embodied Conversational Agents, we introduce a behavior synthesis model for the generation of expressive gesturing. A small set of dimensions of expressivity is used to ...
-
A methodological proposal to assist scenario-based design in the early stages of innovation projects Article dans une revue avec comité de lectureThis paper presents a methodological contribution to ergonomics in the early stages of innovative design projects, focused on the anticipation of future needs and activities. We examined the effect, on designer activity ...
-
Article dans une revue avec comité de lectureNELSON, Julien;
BUISINE, Stéphanie;
AOUSSAT, Améziane (King Mongkut's University of Technology North Bangkok, 2009)
In this paper, we present some theories sover past decades describing interactions between designers and users, and a state of the art of methods and tools to support these interactions in user-centred design. We discuss ...