Human-AI Cooperation

The paper proposes a condition-based multi-modal human-AI cooperation framework for enhancing user intent identification and information presentation in VR. It utilizes intent tuples and a 2-Large-Language-Models (2-LLMs) architecture, improving prompt length and response generation. A VR furniture purchasing system based on this framework outperforms in user study phases, promising personalized VR experiences.

This paper introduces a dialogue-driven framework for intelligent simulation of power systems, aiming to alleviate operator workload. It combines human and AI interaction to regulate power grids effectively. The prototype system enhances operator efficiency and usability, suggesting potential for broader applications in human-AI collaboration in engineering.

This paper introduces a novel approach to AR dialogue systems, utilizing the SIMMC2-Point dataset to incorporate pointing modality. It employs BART and CLIP models to design multi-modal dialogues capturing spatial and attribute data. Ablation experiments underscore the pointing modality’s importance, advancing AR dialogue systems for immersive interactions.

Our paper proposes a novel approach to task assignment in human-AI cooperation, utilizing the capability-aware shared mental model with the unified form of tuples to represent task-specific capabilities of both human and AI. Results from our user study show that this approach improves accuracy and time efficiency while facilitating better understanding of each team member’s capabilities.