@akshay_pachaar: Redis built a cache that cuts LLM costs by 70%! Production LLM apps often receive different versions of the same questi…

X AI KOLs Timeline Tools
redis llm semantic-cache cost-reduction ai-tools caching

Summary

Redis LangCache is a semantic caching tool that reduces LLM costs by up to 70% by storing and reusing similar question-response pairs, making AI applications faster and more cost-effective.

Redis built a cache that cuts LLM costs by 70%! Production LLM apps often receive different versions of the same question. For instance, an internal developer assistant might receive: - "How do I rotate an API key?" - "Where can I replace my API key?" The wording is different, but both questions have the same answer. Prefix caching cannot handle this repeated generation because it only reuses computation when the cache matches bit-by-bit. And if the cache hits, an LLM response is still generated again. A semantic cache stores the complete question-response pair outside the model. When a query arrives, it searches for previously answered questions with similar meaning. A valid match returns the stored response with no LLM call. If you want to use this in practice, @Redisinc already implements it as a managed service called Redis LangCache. Under the hood, LangCache embeds the incoming question, searches stored responses, and applies the configured similarity threshold and filters. A cache hit returns the earlier response. A miss falls back to the LLM, after which the new response can be stored for future requests. Redis also handles access scopes, custom filters, embedding selection, TTL, eviction, and monitoring through a REST API, without another database to deploy or manage. While the actual savings depend on how much safe repetition exists in the workload, cache-hit responses are up to 15x faster and 70% cheaper. Try Redis LangCache: https://fandf.co/4d3ixqL I built a small interface comparing LangCache with direct LLM inference. The video below shows this in action, and I worked with Redis on this post to put it together. To dive deeper, my co-founder published a detailed article on KV, prefix, prompt, and semantic caching. Read it below.
Original Article
View Cached Full Text

Cached at: 09/23/26, 10:16 PM

Redis built a cache that cuts LLM costs by 70%!

Production LLM apps often receive different versions of the same question.

For instance, an internal developer assistant might receive:

  • “How do I rotate an API key?”
  • “Where can I replace my API key?”

The wording is different, but both questions have the same answer.

Prefix caching cannot handle this repeated generation because it only reuses computation when the cache matches bit-by-bit.

And if the cache hits, an LLM response is still generated again.

A semantic cache stores the complete question-response pair outside the model.

When a query arrives, it searches for previously answered questions with similar meaning. A valid match returns the stored response with no LLM call.

If you want to use this in practice, @Redisinc already implements it as a managed service called Redis LangCache.

Under the hood, LangCache embeds the incoming question, searches stored responses, and applies the configured similarity threshold and filters.

A cache hit returns the earlier response. A miss falls back to the LLM, after which the new response can be stored for future requests.

Redis also handles access scopes, custom filters, embedding selection, TTL, eviction, and monitoring through a REST API, without another database to deploy or manage.

While the actual savings depend on how much safe repetition exists in the workload, cache-hit responses are up to 15x faster and 70% cheaper.

Try Redis LangCache: https://fandf.co/4d3ixqL

I built a small interface comparing LangCache with direct LLM inference. The video below shows this in action, and I worked with Redis on this post to put it together.

To dive deeper, my co-founder published a detailed article on KV, prefix, prompt, and semantic caching.

Read it below.

Redis LangCache

Source: https://redis.io/langcache/?utm_source=influencer&utm_medium=paid-post&utm_campaign=2026-09-ai_in_production-influencer&utm_content=a-pachaar-x PlatformDeployTools

SolutionsAI & ML Apps

Core Workloads

DevsLearning

ResourcesUlta logo

saw a 40% increase in revenue generation through their new Redis powered online experience.See how

Superlinked logo

uses Redis for 100+ vector search queries per second.See how

Sony LIV logo

achieved 37% faster API response times while having a 15% lower memory footprint with Redis.See how

plivo logo

saw over a billon API request per month with Redis.See how

Latest

Image

Office hours: Why your agents keep losing the plot

Sep. 09, 2026

Similar Articles