Empirical investigation of the temporal relations between speech and facial expressions of emotion
Article dans une revue avec comité de lecture
Date
2010Journal
Journal on Multimodal User InterfacesRésumé
Behavior models implemented within Embodied Conversational Agents (ECAs) require nonverbal communication to be tightly coordinated with speech. In this paper we present an empirical study seeking to explore the influence of the temporal coordination between speech and facial expressions of emotions on the perception of these emotions by users (measuring their performance in this task, the perceived realism of behavior, and user preferences).We generated five different conditions of temporal coordination between facial expression and speech: facial expression displayed before a speech utterance, at the beginning of the utterance, throughout, at the end of, or following the utterance. 23 subjects participated in the experiment and saw these 5 conditions applied to the display of 6 emotions (fear, joy, anger, disgust, surprise and sadness). Subjects recognized emotions most efficiently when facial expressions were displayed at the end of the spoken sentence. However, the combination users viewed as most realistic, preferred over others, was the display of the facial expression throughout speech utterance. We review existing literature to position our work and discuss the relationship between realism and communication performance. We also provide animation guidelines and draw some avenues for future work.
Fichier(s) constituant cette publication
Cette publication figure dans le(s) laboratoire(s) suivant(s)
Documents liés
Visualiser des documents liés par titre, auteur, créateur et sujet.
-
Article dans une revue avec comité de lectureDUCHOSSOY, M.; ROOSEN, Christine; TRIVALLE, C.; BUISINE, Stéphanie; MARANZANA, Nicolas (Elsevier, 2015)Les troubles de la déambulation et de la manipulation incessante d’objets sont un phénomène fréquent chez les malades à un stade avancé de la maladie d’Alzheimer. Leur prise en charge est très complexe pour les institutions ...
-
Article dans une revue avec comité de lectureBUISINE, Stéphanie; BESACIER, Guillaume; AOUSSAT, Améziane; VERNIER, Frédéric (Elsevier, 2012)This paper examines some aspects of the usefulness of interactive tabletop systems, if and how these impact collaboration. We chose creative problem solving such as brainstorming as an application framework to test several ...
-
Article dans une revue avec comité de lectureNELSON, Julien; BUISINE, Stéphanie; AOUSSAT, Améziane; GAZO, Claude (Presses Universitaires de France, 2014)Prospective ergonomics is concerned with the anticipation of future needs and activities to assist the early stages of user-centered innovation design projects. Few studies have proposed methodologies to assist prospective ...
-
Article dans une revue avec comité de lectureAGNES, Aurélien; FLEURY, Sylvain; AUZERAIS, Aristide; BISSON, Isaline; DULAU, Eva; BUISINE, Stéphanie; RICHIR, Simon (2020-02-29)This study presents a user test in order to ascertain the advantages and disadvantages of three different text input methods in immersive virtual environment: individual Speech-to-Text, collective Speech-to-Text and a ...
-
Article dans une revue avec comité de lectureMARTIN, Jean-Claude; BUISINE, Stéphanie; PITEL, Guillaume; BERNSEN, Niels Ole (Elsevier, 2006)Most existing multi-modal prototypes enabling users to combine 2D gestures and speech input are task-oriented. They help adult users solve particular information tasks often in 2D standard Graphical User Interfaces. This ...