Tag
Manning Books is offering a free ebook, 'Local AI and Small Language Models', featuring chapters on small language models, CPU inference with GGUF and llama.cpp, quantization, LoRA/DoRA specialization, and building private local AI apps with Ollama's Python library.
The paper introduces Counterfactual Pairwise Augmentation (CPA), a fine-tuning method that synthesizes opposite-labeled counterfactual responses to make decision boundaries explicit for small pointwise LLM judges, improving macro-F1 by 2.5 points over pointwise-only training and beating Gemini 3.6 Flash by 11.8 points on six binary criteria.
LanG 2.0 is an agentic AI system for security operations centers that uses small local models for threat detection, alert correlation, and rule generation, reporting higher correlation precision and rule acceptance than its predecessor while matching Claude Opus 5.5 performance on some tasks at up to 650x less energy.
EDU-Jev introduces a family of single-pass 'System One' decision models (0.8B/2B/4B) that return probability distributions over fixed options for educational assessment tasks, achieving 67.6% average accuracy across twelve benchmarks and outperforming leading commercial decision APIs by 9.4 points.
I recommend Manning’s new book, *Domain-Specific Small Language Models: Efficient AI for local deployment* (by Guglielmo Iozzia, published in May 2026), which explains how to fine-tune small models on domain-specific data, quantize them, and deploy them on local hardware to achieve high-quality performance on specialized tasks. It also covers RAG and agent systems that rely solely on small models.
The paper introduces ELBP, a four-layer evidence-grounded architecture that uses small language models (≤3B params) for on-device prediction of future assistance-seeking behavior, with Qwen2.5-3B achieving F1 0.80 and improved calibration over raw prompting and XGBoost baselines.
This paper studies how small an encoder can be for closed-set decisions like tool selection, finding that per-option scoring beats joint scoring and that 68M/150M parameter joint encoders outperform a 435M DeBERTa-v3 zero-shot classifier while answering requests in milliseconds.
A preview of the upcoming VocabX paper showing how specializing a small language model's vocabulary around token cost rather than frequency cut measured inference cost by 77% on a production Portuguese essay grader built on Qwen3-4B, without changing the base model.
The paper presents MedPAO-Assist, an edge-deployable, modality-aware agentic AI framework that structures radiology reports from dictation using a ~0.6B-parameter small language model, running with ~3.3GB RAM and ~31s latency on a CPU-only laptop.
The paper introduces FederatedBytes, a tokenizer-free byte-level small language model with linear attention for efficient multimodal federated learning, outperforming ByT5 and LSTM on sentiment analysis and ScienceQA while reducing memory and communication overheads.
This preprint argues that learned embedding tables dominate small language models' size and shows a frozen 16 MB vocabulary of 512-bit codes can match or beat trained tables on benchmarks, enabling a 46 MB BitNet ternary model with 5-6x faster CPU inference at a seventh of the memory.
Salesforce and Stony Brook researchers train an 8B reasoning model with a set-level reinforcement learning objective to generate diverse, high-quality scientific ideas, achieving higher creative utility than Claude Opus 4.8 and GPT-6 Astra on a contamination-free benchmark.
作者以语音 AI 编排平台产品负责人的视角，分析 Jev 及同类分类模型在电话语音机器人中的实际用途，并梳理了 Laya、Clef、CMF、Nimble、Kev 等竞品生态。文章指出单一 LLM 无法独立支撑生产级对话，分类模型在延迟、成本和结构化输出上仍不可或缺。
NeurIPS 2026 论文提出 Progress-Aware Distillation (ProD)，通过检测并惩罚学生智能体轨迹中的停滞、利用师生进度差异调整训练目标，帮助 0.6B–8B 小语言模型智能体摆脱无效循环，提升任务成功率与完成效率。
论文提出 HybridResearcher，一个 4B 小型深度研究智能体，通过合成跨源轨迹与强化学习在结构化与非结构化数据之间进行推理，在 HybridDeepResearch 基准上超越更大的 Qwen3.5-397B-A17B，同时不遗忘单源能力。
UC San Diego 与 Amazon Annapurna Labs 提出 'How to Train Your Little Dragon' 训练方法，通过监督微调与强化学习使一个 8B 小模型学会构建和改进 GPU 优化脚本，在正确率和内核性能上超越多个万亿参数级前沿 LLM。
Qwenfant 是慕尼黑工业大学团队为 2026 BabyLM Challenge 多语言赛道提交的论文，通过 LLM 摘要与翻译生成高质量合成平行数据，结合更大模型与字符级词表（配合 LoRA 微调），较基线提升 11 分，提交时排名第一。
提出「在前沿LLM上迭代开发行为，再蒸馏到小模型（SLM）上线」的两阶段方法：用LoRA微调的Qwen3-8B在家庭护理排班智能体的生产决策上达到0.913一致率，优于教师模型GPT-5.4-nano自身重跑时72.7%的自我一致性，且延迟降低约2.5倍、成本大幅下降。论文已被NeurIPS 2026第二届企业任务智能体基准与应用研讨会（AABA4ET）接收。
论文研究了一种混合架构：云端大模型预先编写程序，将涉及用户私有数据的运行时判断委托给本地小模型（SLM）通过普通函数调用来解释。在 AppWorld 基准上，该方法在所有六种模型与数据分区组合中均提升了纯代码 agent 的成功率（最高 14.9 个百分点），在保持数据本地化的同时缩小了与云端 ReAct 循环的差距。
实验展示了两个现成小模型（Gemma 4 E2B 和 Qwen3.5-0.8B）在 Galaxy S21 本地运行，通过 E2LLM/SiFR 浏览器感知完成同一 Amazon 购物任务，两次运行均生成不同但有效的购物车方案并以 READY_FOR_HUMAN_REVIEW 结束。所有日志、脚本与哈希均公开在 GitHub。