Tag
Neuralwatt 分析一个月云端流量发现，agent harness 如何构建 prompt 决定了大量 GPU 计算量：缓存命中率差异导致未缓存输入在 1.6% 到 11.1% 之间，一次缓存失效大致会让请求能耗翻倍，且这些问题多数可通过避免编辑已发送内容来修复。
微软的研究论文提出在用户轮次边界进行上下文感知的模型路由，发现将会话历史提供给路由器可在相同卸载率下提升多轮电子表格智能体的任务通过率，并分析了提示缓存切换的不对称成本与廉价模型的成本优势衰减。
一位自动化机构从业者逐项拆解4分钟语音外呼的真实成本（约39美分），指出平台费占一半、LLM仅占10%，并建议开启prompt caching、裁剪工具返回结果而非一味换便宜模型。
一位推主根据 Anthropic 工程师 Lydia Hallie 的官方解释，澄清了 Claude Code 的 auto-compact 机制：默认在接近 967K 上下文时才触发压缩，导致长任务持续携带巨量上下文而快速消耗 5 小时配额。文中建议通过 /autocompact 400k、autoCompactWindow、promptCacheTtl 等配置提前压缩并优化缓存。
Asana says it reduced a browser-agent workflow's model cost by 76x (from $36.21 to $0.47 per run) and cut runtime from 22.5 to about 4 minutes by stabilizing page history to enable prompt caching and pruning screenshots in batches, rather than changing the underlying model.
Raft Computer 1.0.40 introduces constructed initial context, which restarts long-idle agent sessions from transcripts instead of re-caching expired context, cutting cold-wake cost by 60% and cost per request by 29% without breaking continuity.
Obie Fernandes notes that Claude Code's prompt cache expires one hour after your last message, causing expensive full-price re-reads after stepping away, and shares a plugin that writes a handoff doc at the 50-minute mark while the cache is still warm.
The author shares notes from OpenAI DevDay in a multi-part thread, starting with key levers for optimizing cost and performance in agentic applications: prompt caching, reasoning effort, programmatic tool calling, and batch requests, recommending users measure these with evals.
OpenAI published a practical guide for building with the GPT-6 model family, covering model selection, reasoning effort, prompt caching, compaction, and managing long-running agentic workflows in production.
作者在为财务智能体添加 Agent Skills 支持时，总结了三个需要分离的架构决策：服务器端加载方式、模型上下文可见性、以及工具可用时机，并指出技能加载只教工作流而非安装能力。
Devin's latest update makes the AI software engineering agent 30-70% cheaper depending on mode, achieved by integrating the latest models like SWE-2 and optimizing its harness through batched tool calls and improved prompt caching, while Devin Fusion now leads FrontierCode 1.1 Extended at $0.60 per task.
Henry Broomfield shares recent improvements in prompt caching at OpenAI, including tips to enhance cache hit rates for applications.
PSA advising to increase the -cram parameter in llama.cpp for better performance in agentic workflows with long contexts, based on personal experience with Qwen 27B 3.8.
OpenAI announces improved prompt caching for GPT-6, offering higher cache hit rates, discounts, and new tools for monitoring and optimization.
The article discusses the trade-off between context compaction and prompt caching in production AI agents, proposing subagents as a strategy to maintain cache efficiency and reduce context pollution.
claudebill is a CLI tool that reads Claude Code session logs to show cost breakdowns per session, project, git branch, model, and day, including prompt caching savings, with offline and zero-dependency features.
A developer describes a technique for AI agents in dynamic simulations where action schemas change each turn, placing them in the last user message to preserve prompt caching efficiency, and compares various models and output modes.
OpenAI's prompt cache reduces request costs by 90% but has a cache key limit of 15 requests per second. Unify GTM built a custom routing solution to bypass this limit, achieving a 95% cache hit rate.
The article argues that with prompt caching, longer, stable prompts can be cheaper than frequently changing short ones, sharing insights from running AI agents with high cache hit rates.
The article describes a cost-efficient setup for AI coding agents by routing Claude Code through DeepSeek's API, leveraging prompt caching to significantly reduce token costs while maintaining high reasoning capabilities.