Immersive Hand Instructions in AR – Insights for Asynchronous Remote Collaboration on Spatio-Temporal Manual Tasks

Abstract

Collaborative Virtual Environments (CVEs) have recently become more prominent in academic research and industrial applications. Physical tasks often rely on sharing spatial information between the collaborators, which are difficult to communicate verbally or by 2D media. Immersive technologies, like Mixed Reality (MR), can sup-port more natural and intuitive interaction between the collaborators, share important non-verbal cues, like gaze and deictic gestures, and improve the performance time of remote guidance tasks. In this work we present HoloHands - a concept for immersive hand instructions which can be used for asynchronous remote collaboration on spatial tasks. A Hololens device is utilized to capture hand motions and interactions with virtual representations of physical objects to create a guidance by simply demonstrating the instruction steps. The recordings can be consumed by other collaborators at a later time to perform the physical task which we validated in a preliminary user experiment. Since it allows the creation and retention of digital information for a later consumption, it provides the possibility to revisit previous work sessions at any time and thus opens new possi-bilities to collaborate asynchronously on spatio-temporal tasks. In this work we share valuable insights from our experience with the HoloHands concept and the utilization of the Hololens headset.