Tag
提出一种协作学习框架，用多个异构小模型替换单个大模型进行推理（model slicing），并通过金字塔监督蒸馏改进跨模型协作，在多跳问答、数学推理、代码生成等任务上达到与大模型相当或更优的性能，且推理效率更高。
AgentRouter is a lightweight classifier for routing steps in agentic workflows to different model tiers, achieving 72% cost reduction with minimal quality degradation compared to frontier-only models.
The paper proposes DualMLC, a dual-branch framework that combines autoregressive and bidirectional language models for large-scale multi-label text classification, achieving state-of-the-art results on benchmarks.
The paper introduces XKV, a method for efficient latent space communication between heterogeneous language models in multi-agent systems, improving accuracy and speed over existing text and cache-based protocols.
This paper introduces Mixture-of-Translators (MoT), a framework for translating KV caches across heterogeneous LLMs, enabling cache reuse between different architectures. Experiments show preserved QA performance and long-context quality across Qwen2.5, GPT-2, and OPT models.
Introduces Collate, a training framework for collaborative neural network learning that handles heterogeneous edge devices with latency constraints, achieving accuracy improvements with minimal overhead.