This paper presents SurgicalRoomAgent, a voice-interactive multi-agent system for smart operating rooms based on large language models (LLMs), which achieves natural language understanding, device control, intraoperative recording, and surgical report generation through a layered architecture comprising a voice interaction pipeline and an agent core.
Abstract
This paper presents SurgicalRoomAgent, a voice-interactive multi-agent system for smart operating rooms based on large language models (LLMs). The system achieves natural language understanding, device control, intraoperative recording, and surgical report generation through a layered architecture comprising a voice interaction pipeline (wake, ASR, turn detection, agent reasoning, TTS) and an agent core (skill registry, task planner, device manager). Three key technologies are investigated: (1) KV Cache prefix warming for low-latency inference, reducing recomputation overhead from approximately 500 ms to tens of milliseconds via byte-level Longest Common Prefix reuse; (2) streaming partial JSON parsing with early parallel task execution, reducing end-to-end latency by approximately 30%; and (3) progressive skill prompt disclosure, which dynamically filters system prompts based on user role, connected devices, and surgical phase to maximize information density within limited context windows. The system is implemented using the Qwen3-27B model with llama.cpp/sglang inference engines. Experimental analysis demonstrates effective operation within a 16,384-token context limit and multi-device parallel control response times meeting OR real-time requirements.
A distributed conversational framework for human-robot collaborative manipulation that integrates local language and vision-language models (VLMs) with a Robot Operating System 2 (ROS 2)-based execution stack, enabling flexible deployment across distributed hardware while maintaining a responsive control loop.
Arash Ghasemzadeh Kakroudi, Roel Pieters· IEEE International Symposium...· 0 citations
A LiveKit-based AI Assistant designed to enable seamless, low-latency human-computer interaction by integrating real-time audio streaming, large language model (LLM) reasoning, and automated system control within a unified architecture is introduced.
Somil Shekhar, Saniyah Ahmad, P. Pawar et al.· International Journal of Dru...· 0 citations
We present Qwen-Audio-Agent, a harness that combines full-duplex voice interaction with asynchronous task execution through a foreground-background architecture. A Frontend Agent manages dialogue and selects between direct tool use and delegation, while a Backend Agent carries out delegated tasks in a separate context....
Chong Deng, Yunjie Ji, Yu-Xiang Kong et al.· 1 citation· ⚡1
This paper presents a seat-driven automation layer for the Laboratory of Hybrid Communication (LHC), a room-centric hybrid meeting platform that maps virtual seats to physical display nodes. Unlike conventional videoconferencing layouts, the system synchronizes digital participant presence with room infrastructure by t...
Stanislav Šidla, Javlon Tursunov, Jaroslav Venjarski et al.· International Symposium ELMA...· 0 citations
UI-Venus-2 is presented, a general-purpose foundation GUI agent designed to operate across mobile, web, and desktop environments through a unified closed-loop reasoning-action framework that integrates safety-aware mechanisms to ensure controlled execution of consequential actions.
Venus Team, Zhuo-Hang Cai, Hao-Xin Chen et al.· 4 citations
Qwen-Audio-3.1-Realtime brings these requirements together through Think, Act, and Speak and Coordinate, evaluating audio reasoning, multilingual understanding, tool use, conversational behavior, full-duplex interaction, and safety.
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
A new method, called CW-Net, translates the reasoning process of an autonomous vehicle’s AI system into understandable concepts that explain its behavior.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.