VR

The paper proposes a condition-based multi-modal human-AI cooperation framework for enhancing user intent identification and information presentation in VR. It utilizes intent tuples and a 2-Large-Language-Models (2-LLMs) architecture, improving prompt length and response generation. A VR furniture purchasing system based on this framework outperforms in user study phases, promising personalized VR experiences.

This paper introduces a novel approach to AR dialogue systems, utilizing the SIMMC2-Point dataset to incorporate pointing modality. It employs BART and CLIP models to design multi-modal dialogues capturing spatial and attribute data. Ablation experiments underscore the pointing modality’s importance, advancing AR dialogue systems for immersive interactions.