llm-offloading

COPE: Composable Performance Emulation for Mixture-of-Experts Offloading Policies——上海交大IPADS提出的一个框架，通过Policy Matrix和Virtual Heterogeneous Machine在运行时集成前预测并选出最优的MoE专家卸载策略，将最快策略选择准确率提升24%–100%，性能预测误差降低1.7×–6.8×，并减少最多74%的策略实现工作量。

This paper introduces CARGO, a training-free routing framework that uses the local LLM's own inference-time agreement across sampled responses to decide when to offload to a cloud model, enabling controllable collaboration ratios without additional training.