Enhanced cognitive workload evaluation in 3D immersive environments with TOPSIS model

Abstract

Research puts forward perception-based cognitive workload evaluation methods to help VR developers and users measuring their workload when playing with a VR application. Approaches to measure workload based on biosensors have progressed significantly, while evaluation based on subjective methods still rely on standard questionnaires such as the NASA-TLX table, the Subjective Workload Assessment Technique and the Modified Cooper Harper scale. The pre-defined questions enable operators to carry out experiments and analyse the data more easily than with biofeedback. However, the subjective evaluation process can bias the results because of unperceived internal changes and unknown factors among users. It is therefore necessary to have a method to handle and analyse this uncertainty. We propose to use the Technique for Order Performance by Similarity to Ideal Solution (TOPSIS) model to analyse the NASA-TLX table for measuring the overall user workload instead of using the classical weighted sum method. To show the advantage of the TOPSIS approach, we performed a user experiment to validate the approach and its application to VR, considering factors including the VR platform and the scenario density. Three different weighting methods, including the fuzzy Analytic Hierarchy Process (AHP) from fuzzy logic, the classical weighting based on pairwise comparison and the uniform weighting method, were tested to see the applicability of the TOPSIS model. The results from TOPSIS were consistent with those from other evaluation methods; a significant reduction in the coefficient of variation (CV) was observed when using the TOPSIS model to analyse the NASA-TLX scores, indicating an enhanced precision of the workload evaluation by the TOPSIS method. Our work has a potential application for VR designers and experimenters to compare cognitive workload among conditions and to optimize the settings.