Tag
A preview of the upcoming VocabX paper showing how specializing a small language model's vocabulary around token cost rather than frequency cut measured inference cost by 77% on a production Portuguese essay grader built on Qwen3-4B, without changing the base model.
对500个AI模型价格与评分的追踪显示，某个分数首次被顶级模型达到后约6个月，达到同等分数的成本通常降至约20分之一，一年后降至105分之一，而顶级模型自身的价格基本持平。
用户讨论 DeepSeek V4.1 Flash 的 token 效率表现，初步测试显示在高思考强度下 CoT 更短且质量更高，解决任务所需轮次更少，值得尝试。
提出「在前沿LLM上迭代开发行为，再蒸馏到小模型（SLM）上线」的两阶段方法：用LoRA微调的Qwen3-8B在家庭护理排班智能体的生产决策上达到0.913一致率，优于教师模型GPT-5.4-nano自身重跑时72.7%的自我一致性，且延迟降低约2.5倍、成本大幅下降。论文已被NeurIPS 2026第二届企业任务智能体基准与应用研讨会（AABA4ET）接收。
LEASH 是一个免训练框架，利用隐层不确定性在弱模型与强模型之间按需切换，并在交接后通过影子观察者持续监督强代理，在 OSWorld 上以更低推理成本达到与 Claude Opus 5 相当的成功率。
A developer argues that running Qwen3.8FN Q4 locally on a budget eBay-built server is cheaper than nearly all cloud models (except GPT-6 Luna and Claude Haiku 5.5 with caching), enabling longer-running agents that produce better results than cost-prohibitive frontier cloud models.
ScaleDown 发布了三款小型 Constrained Decoding 模型（Classify、Extract、Summarize），支持 1M token 输入窗口和图像输入，通过单次 prefill 直接从输入中选取答案而非逐 token 生成，据称比前沿 LLM API 快 200 倍、便宜 467 倍。
据称 OpenAI 使用其能解出此前未解数学问题的内部模型来改进自身算法，将推理成本降低了约 50%，显示出 AI 开始反哺提升自身效率。
EdgeWise 是一个边缘-云协作框架，在本地部署小语言模型（SLM）作为主要求解器，并通过强化学习在关键步骤选择性地将任务委派给云端大模型，从而在 SWE-Rebench 上降低高达 62%、在 Terminal-Bench 2.1 上降低 34% 的推理成本，同时保持与纯 LLM 执行相当或更好的性能。
Cloudflare 发布 clef-omni，支持音频和视频输入，被用于 PII 检测、恶意域名识别等场景；同时 clef-flash 价格降至每百万输入 token 0.038 美元，具备较高性价比。
论文提出用零训练的 System-One LLM（Jev）直接对学习者状态输出答对概率，在无任何目标平台数据时 AUC 达 .706，超过 28 个深度 KT 模型在 8 名学习者上训练的结果以及 System-Two 推理式 KT，且成本仅约 1/100；加入 64-shot 示例和近邻统计后（JevKT）提升至 .722，在约 64 名学习者以内保持领先。
该论文提出「先澄清、再聚焦」的声明归一化方法，将对话转化为带来源引用和语义标签的简短陈述，使仅由小模型构成的推理管线能以约100倍更低的成本处理数百万条企业客服对话的分析任务。
A deep retrospective on three years of LLM technical evolution: long-context handling shifted from RAG to prefix caching and million-token context windows; the Transformer still wins over linear-complexity architectures thanks to the hardware ecosystem of GPU tensor cores; inference costs were democratized through quantization, distillation, and task-based routing; agents moved toward pragmatic engineering paths like progressive disclosure, command-line sandboxes, and test feedback loops; and hallucination remains unsolved.
A technical comparison shows that running a cheaper, lighter model like Gemini 3 Flash can cost far more than a pricier reasoning model like Gemini 3 Pro — Flash burned ~1,000 steps and $14 in tokens before failing, while Pro succeeded in 85 steps for just $1, highlighting how token usage varies across models and even runs.
The paper analyzes six commercial dynamic LLM routers across 14 settings and finds none outperforms a router that randomly selects between two well-chosen models at matched cost, attributing failures to difficulty blindness, length reversal, semantic matching, and roster suboptimality — patterns the standard evaluation objective actually rewards.
作者赞同判断：智能体中超过一半的所谓「推理」其实是路由决策，应把工具选择、路径分支等交给小模型以毫秒级速度完成，只在真正需要推理的步骤上花费昂贵的大模型 token。
The paper proposes SaveRouter, a sparse-supervision LLM routing framework that selectively acquires query-model feedback and shares capability information across related queries, cutting supervision costs while maintaining competitive routing quality and reducing break-even deployment volume by 1.9-9.5x.
This paper proposes softmax reparameterization, a post-training method that selects a functionally equivalent output head before quantization to reduce inference cost in small language models. The method demonstrates significant improvements across various quantization techniques and datasets.
Hunyuan 3.5, Tencent's latest image-generation model, is praised for its ability to follow references and produce polished images at 2K resolution, made affordable through OnSolo's credit system.
The dedicated evaluation model JEV demonstrated high efficiency and low-cost potential in processing interview transcripts, emphasizing the advantages of using large language models as logical judgment layers rather than content generators.