@lu__jasper: Some early results from playing around with search on a subsampled version of OBLIQ-bench. Mixedbread's reranker is a b…
Summary
Early results from testing search on a subsampled OBLIQ-bench show that Mixedbread's reranker achieves strong MRR, sometimes outperforming GPT 5.5 on certain metrics with faster speed, though the benchmark remains challenging.
View Cached Full Text
Cached at: 06/03/26, 01:40 AM
Some early results from playing around with search on a subsampled version of OBLIQ-bench.
Mixedbread’s reranker is a beast of a model! It gets a pretty strong MRR on the task, and even edges out GPT 5.5 on some metrics while being significantly faster.
I was mostly interested more traditional retrieve-and-rerank setups here, since agentic approaches felt a little too obvious / boring. But I came away convinced that this benchmark is hard enough that:
- Even today’s best embedding + reranker models will only get so far
- Higher cost / latency is tolerable.
Going to see how far I can push the cost / latency vs quality tradeoff here.
Similar Articles
@RuiTheBaker: GPT 5.5-level ranking but 27x faster?! @mixedbreadai
Mixedbread's reranker achieves GPT 5.5-level performance on OBLIQ-bench while being 27x faster, according to early results.
@jerryjliu0: We've provided some updated results on Mistral OCR that make use of the annotation feature for charts. The overall scor…
Jerry Liu reports updated results for Mistral OCR on ParseBench, showing it outperforms GPT-5.5 and trails only Gemini 3.1 Pro, with strong performance on content faithfulness and semantic formatting.
@dianetc_: We set out to build a better retriever, so we looked for the hardest IR benchmarks. For each, we asked how much headroo…
The authors introduce OBLIQ-Bench, a new benchmark designed to evaluate information retrieval systems on significantly harder search queries where previous benchmarks showed little remaining headroom.
How much of MLE-Bench's gains are the algorithm vs. better models + more search? [R]
A new benchmark, FML-Bench, reveals that recent improvements in MLE-Bench scores are largely due to better base models and increased search budget rather than algorithmic advances.
Introducing Toast 1
Mixedbread introduces Toast 1, a specialized search agent that matches frontier model quality while being up to 10x cheaper and 12x faster. It automates agentic search loops and achieves state-of-the-art results on benchmarks like OfficeQA Pro V2 and legal knowledge tasks.