Tag
作者更新了将稠密模型转换为混合专家（MoE）模型的项目，发布了 Qwen2.5-1.5B-MoE-A0.9B 及 0.5B/1.5B 的 GGUF 版本。结果显示 GPU 和 CPU 推理速度明显提升（约 32-41%），但困惑度上升约 40%，作者称其为小模型上的概念验证。
This paper introduces an unsupervised metric based on attention dispersion to detect hallucinations in large language models, showing significant AUC improvements on mathematical reasoning benchmarks using the Qwen2.5 model family.
ShardFlow, a distributed LLM inference framework, achieves 28 TPS on Qwen2.5-7B across cloud regions by using speculative decoding and CUDA Graphs to mitigate WAN latency.
Comrade-7B is a fine-tuned Qwen2.5 model designed for multilingual efficiency and versatility, supporting Russian, English, and Chinese.