The user experience measurement : a challenge of the twenty-first century

Abstract

Technologies are at the heart of the 21st century revolution. They are more and more changing the way we work, learn and the way we define human. Recently, researchers have been trying to understand and master technologies through the concept of User eXperience (UX). ISO DIS 9241-210 tentatively defines UX as "all aspects of the user’s experience when interacting with the product, service, environment or facility. This is why, UX has been in the last years an approach used in numerous disciplines (e.g., HCI, ergonomics, information architecture, branding, ...) to improve user-centric products or services. No doubt is made that UX is still an important dimension in product quality. A specific product that highlights the importance of considering UX is virtual reality (VR), a technology greatly in vogue that provides applications to the general public. Studies show that UX consideration can alleviate the cybersickness (e.g., nausea, vertigo, disorientation, ...) affecting users in VR [1]. Today’s technologies are more complex, they are fashionable, they stimulate the user's senses, they are immaterial. Therefore, early UX research strategy, rather than focusing on instrumental measures, focuses on more holistic subjective perceptions such as hedonic qualities, affects or emotions. The report is clear: most UX evaluation methods are subjective, self-developed or not statistically validated [2]. There is an urge for clear measurement standards for UX evaluation. The current situation is alarming since a product’s intrinsic quality does not have to be subjective but has to be objectively measured in order to be easily compared and controlled. According to the International Vocabulary of Metrology (i.e., VIM 3), a measurement process should generate ‘‘one or more quantity values that can reasonably be attributed to a quantity”. Moreover, “the results of the process, by which quantity values have been obtained, are characterized by the metrological compatibility, metrological comparability and, if necessary, metrological traceability” [3]. It is currently not the case for UX. Consequently, the main challenge UX is facing in this 21st century context, is to ensure that new products and services provide good UX to users. To do so, UX research needs to define actual “objective measurement” metrics whether the UX criterion is hedonic (e.g., pleasure, motivation, satisfaction, ...) or instrumental (e.g., performances, ...). This requires UX to be defined and determined as a measurement result. In this paper, we cover the importance of designing UX measurement methods in order to turn UX as a mesurande. We review several studies in which objective UX methods are proposed for UX assessment. We focus on the few measurement techniques existing today such as electroencephalographic and electromyographic responses that, in some cases, are used to assess objective UX (i.e., user mental state, facial expressions). Finally, we point out the difficult technical challenge UX measurement represent today, as, technologies to properly evaluate the user’s perceptions do not exist or are still immature.