Tag
The article tests whether DeepSeek Harness maintains high prompt caching rates when using alternative AI models, finding that GLM and Kimi achieve 97-99% cache reuse, while Opus shows no cache activity and GPT test failed.
Unify GTM reduced AI model costs by 90-95% through architectural optimization and prompt caching, emphasizing the importance of adversarial judging models for scalable sales AI agents.
This article provides a detailed introduction to Pi's coding agent's context compression mechanism, analyzing the reasons for context expansion, compression trigger conditions, and the cost of cache invalidation, aiming to optimize AI performance in long conversations.
Anthropic published a practical guide for Claude Code, emphasizing context management through session commands and discussing token optimization and prompt caching to improve efficiency in AI-assisted coding.
The article reports an experiment comparing Grok 4.6 and GPT-5.6 Sol on agentic loop efficiency for coding tasks, showing Grok 4.6 is more cost-effective with fewer model calls and effective prompt caching.
Anthropic has published instructions on how to run Claude Code efficiently by using the /compact command before the prompt cache expires to save tokens.
This article explains how to maximize Claude Code sessions by detailing input/output token costs, prompt caching mechanisms, and optimization tips for efficiency.
A PSA warning DeepSeek-V4-Flash-0731 users that system role messages are hoisted to the top, ruining prompt caching and proximity, and advising the use of latest_reminder instead.
DigitalOcean introduces prompt caching for LLM inference, automatically caching repeated context like system prompts to reduce input token costs by up to 80% without code changes, with a detailed tutorial on break-even math.
GitHub Copilot now uses prompt caching, tool search, and automatic model selection (HyDRA) to reduce cost and improve efficiency, achieving 3.3x savings while matching OpenRouter Auto's resolution rate.
The article explains how prompt caching works in large language model agents, covering KV cache mechanics, prefill and decode phases, and the impact on latency, cost, and agent design.
A practical discussion questioning whether prompt caching delivers meaningful cost savings for AI agents in production, examining real-world factors like cache hit rates, routing strategies, and scale.
A tool for lossless compression of prompt caches designed specifically for AI agents.
CacheBlend, a EuroSys 2025 Best Paper, solves the problem that 90% of KV cache is never reused due to rigid prefix-matching in prompt caching. By selectively recomputing only boundary tokens between documents, it achieves 2-4x faster multi-document processing without quality loss, implemented in the open-source LMCache layer.
Explains why model routing in agent tasks may not save costs due to cache warmup, and describes a production solution with model affinity and the open-source proxy Plano to achieve actual savings.
Researchers introduce Programmable KV Cache, a method for editing and composing KV caches to avoid re-prefilling long contexts during LLM agent inference, achieving 53–398× reduction in p90 time-to-first-token while maintaining decision identity.
A developer built an AI agent using Claude Sonnet 4.6 to handle Instagram DM orders for a 7-location sushi chain, leveraging prompt caching to keep costs low.
An article discussing how prompt caching can significantly reduce LLM API costs, pointing out that providers under-explain it and offering a simple rule to structure prompts for maximum cache hits.
Claude is now officially available on Microsoft Foundry, allowing Azure accounts to use it directly with existing authentication, billing, and compliance. The initial rollout includes Claude Opus 4.8 and Haiku 4.5, supporting prompt caching and extended thinking.
Alex, a new LangChain team member, published an article explaining how Deep Agents uses prompt caching to reduce API costs.