Tag
提出「在前沿LLM上迭代开发行为，再蒸馏到小模型（SLM）上线」的两阶段方法：用LoRA微调的Qwen3-8B在家庭护理排班智能体的生产决策上达到0.913一致率，优于教师模型GPT-5.4-nano自身重跑时72.7%的自我一致性，且延迟降低约2.5倍、成本大幅下降。论文已被NeurIPS 2026第二届企业任务智能体基准与应用研讨会（AABA4ET）接收。
A practitioner shares how their ASR pipeline failed on code-switching calls because language detection ran once at the start, mangling the Hindi portion of English-Hindi utterances; they now run continuous detection and route unstable chunks to a review queue, exposing how benchmark numbers diverged from real-world caller behavior.
Victor Velazquez announces his new book 'The Agentic Engineer,' which outlines principles for building production systems with LLMs and includes real-world incident stories.
An aspiring AI engineer seeks advice from senior professionals on which portfolio project best demonstrates production skills, proposing ideas like LLM routing services, agent systems with guardrails, and local inference microservices, while asking for feedback on common gaps in portfolios.
This paper introduces the Lightweight Ranking Heads framework to accelerate multi-task experimentation in production recommender systems by enabling dynamic task injection without retraining backbone models, reducing iteration cycles from weeks to days at YouTube scale.
LangChain's guide explores how organizations like Madrigal Pharma, Abridge, and Vizient are scaling AI agents in healthcare and life sciences to accelerate research and reduce documentation burden, focusing on infrastructure for reliability and compliance.
The article questions whether a dedicated reversibility layer is necessary for AI agents in production to handle and undo mistakes, highlighting the manual effort currently required for recovery.
The paper evaluates when zero-shot LLMs can replace fine-tuned NLU classifiers for intent detection in production, highlighting LLM advantages in out-of-scope detection, ASR robustness, and dynamic schemas, and offers a decision framework for practitioners.
QCon London is programmed by practitioners who work on production systems, providing relevant insights for software development.
A user is seeking practical experiences from others running AI agents with write access to production systems, discussing operational challenges like action verification, retry handling, audit trails, and internal ownership.
The article discusses the feasibility of building custom AI agents for non-developers using visual tools like n8n, but highlights four key breaking points where production systems fail due to lack of engineering controls.
The discussion explores the current stance on allowing AI agents to make unsupervised writes to production systems such as ERP and CRM, questioning the barriers and comfort levels in the industry.
SAGE is a learned SLO-aware adaptive retrieval policy for production RAG systems that dynamically selects the number of retrieved passages per query, improving SLO compliance and reducing latency/cost with minimal quality loss.
A practical note on AI agent reliability, arguing that production agents need explicit gates for evidence thresholds, retry budgets, and impact assessment rather than relying on memory alone to determine task completion.
Argues that the key to production-ready AI agents is accountability infrastructure (e.g., scoped memory, HITL, trace logging) rather than model intelligence.
A tweet lists key projects to build in inference engineering for understanding production LLM systems, including inference servers, paged KV cache, speculative decoding, quantization libraries, and guardrails.
A tweet thread promoting the book 'Context Engineering for Multi-Agent Systems' by Denis Rothman, which teaches how to build transparent, modular multi-agent systems with context engineering, dual RAG, and other techniques.
A discussion on the emerging security challenges when AI agents can take real actions (e.g., update records, call APIs) rather than just generate text, exploring needed infrastructure like scoped identities, policy layers, and action logging.
This paper presents a method for compiling repeated standard operating procedure steps into validated, versioned tools before deployment, replacing inference-time code generation. In a fulfillment center alarm-triage system, this approach reduces p50 latency by 42% and end-to-end error rate by up to 53%.
The author draws parallels between the early microservices hype and current multi-agent system hype, arguing that engineering practices—not better models—may be the key to reliable multi-agent systems.