@tomaarsen: At 1400x cheaper, I know I'm sticking to embeddings (dense, sparse, multi-vector), plus hybrid (incl. bm25) and reranke…
Summary
A discussion on the cost-effectiveness of embeddings versus LLMs, referencing a study called 'embedder's dilemma' that finds LLMs outperform embedding models at significantly higher cost.
View Cached Full Text
Cached at: 08/23/26, 09:45 PM
At 1400x cheaper, I know I’m sticking to embeddings (dense, sparse, multi-vector), plus hybrid (incl. bm25) and rerankers.
Perhaps I’d even use listwise cross-encoders, they seem interesting.
Niklas Muennighoff (@Muennighoff): can LLMs replace embedding models? in a new work “embedder’s dilemma” we find LLMs now beat embedding models – but at much higher cost. when to choose which?…
📜
Similar Articles
The Embedder's Dilemma: LLMs Are Better, but at What Cost?
This paper presents a cost-aware comparison of LLMs versus dedicated embedding models across 37 tasks, finding that the best LLM and embedding model are nearly tied on aggregate performance but LLMs are up to 1,431x more expensive and slower, leading to a recommended division of labor.
The Embedder's Dilemma: LLMs Are Better, but at What Cost?
The paper compares large language models and embedding models across 37 tasks, finding that while aggregate performance is similar, embedding models are far cheaper and faster, supporting a division of labor for cost-efficiency.
If You Already Pay for an LLM Service, Running Local Embeddings and Rerankers Feels More Useful Than Running Local LLMs
The author argues that for users who already subscribe to LLM services like ChatGPT Pro, running local embedding and reranker models for a memory system is more practical than running local LLMs, and details their GBrain-based setup.
@v0xium: LLM Inference Engineering: Embedding Models Explained 1. An Embedding Model (EM) converts a chunk of text, or any other…
This article explains embedding models and their role in LLM inference, covering architectures, traffic profiles, and optimization techniques like quantization.
@mixedbreadai: By now, everyone knows that single-vector embedding models are hugely limiting for modern workflows. But they contain t…
Single-vector embedding models can be used to extract sparse latent terms, and BM25 can turn this vocabulary into a strong retriever.