Cognitive-Adaptive Multimodal Interaction in Extended Reality: A Survey of Attention-Aware, Privacy-Preserving, and Intelligent Human–Computer Interaction Frameworks

Extended Reality (XR) has enabled immersive Human Computer Interaction (HCI) and consists of Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). The recent advancement of eye tracking, multimodal sensing, and AI has created a greater potential for interactive systems. Though, the available XR systems still rely on rigid interaction models and do not adapt to the user's dynamic cognitive states. Here, we provide a comprehensive review of cognitive adaptive multimodal interaction systems in XR, focusing on attention-aware, privacy-preserving, and intelligent HCI. The systems rely on and fusion multiple data streams and measurements, such as gaze, head pose, gestures, and other contextual data, to assess the users cognitive load and interaction. The survey focuses on the assessment of cognitive load and the design of adaptive interfaces and multimodal machine learning for personalization and context in XR. The review includes emerging privacy-preserving methods for XR as well as the security and privacy concerns for the persistent acquisition of behavioral and biometric data. Based on this literature, we identify the future directions for cognitive digital twins, child-locked, explainable adaptive systems, and privacy-by-design XR frameworks. The survey encourages the innovation of cognitive adaptive XR systems that are intelligent, scalable, secure and privacy-preserving.