Tag
This paper presents SurgicalRoomAgent, a voice-interactive multi-agent system for smart operating rooms that uses large language models to enable natural language device control, focusing on architecture design and key technologies like KV Cache optimization and progressive prompt disclosure to reduce latency in sterile environments.
This project implements Sliding Window Attention as a reusable inference layer for pretrained Hugging Face LLMs, using bounded KV caches with attention sinks to dramatically reduce memory usage and improve decoding speed. Benchmarks on Qwen2.5-7B show memory dropping from ~923 MB to ~3.5 MB at 16K context, though tasks requiring distant context may degrade.