@llama_index: Most document extraction APIs can't tell you where a value came from. For ExtractBench, we scored grounding strictly: a…
Summary
ExtractBench is a new benchmark for document extraction that strictly scores grounding accuracy, showing VLMs and coding agents lack evidence while LlamaExtract Agentic Plus leads with high page-level and word-level performance.
View Cached Full Text
Cached at: 08/17/26, 04:14 PM
Most document extraction APIs can’t tell you where a value came from.
For ExtractBench, we scored grounding strictly: a field only counts if the value AND its citation are correct, word-level box at IoU 0.5. A perfect box around a wrong value earns nothing.
Results: VLMs and coding agents return no evidence at all — zero at both levels. Among systems that do return boxes, the best word-level F1 is still under 50%. And grounding collapses with length: one specialized API goes from 61.7% page-level on short docs to 0.0% on long ones.
LlamaExtract Agentic Plus leads at both levels — 84.9% page-level, 46.4% word-level — and holds at 87.1% on long documents where others hit zero.
Every extracted value should come with receipts. ExtractBench now gives the field a baseline to track it
Blog: https://lnkd.in/gNm97fXp Paper: https://lnkd.in/euAfScWx
Source: https://lnkd.in/gNm97fXp LinkedIn## This link will take you to a page that’s not on LinkedIn
Because this is an external link, we’re unable to verify it for safety.
https://www.llamaindex.ai/blog/introducing-extractbenchThis experience is optimized for Chrome, Edge, and Safari
Similar Articles
ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction
ExtractBench is a new benchmark for schema-guided enterprise document extraction, evaluating value accuracy, record completeness, grounding, and cost across 4,869 pages of enterprise documents. The authors find that commercial VLMs struggle with long documents while coding agents are more accurate but costly, and LlamaExtract AgenticPlus leads on all metrics.
@jerryjliu0: Our "agentic plus" extractor in LlamaParse is great for extracting out massive volumes of fields (e.g. 10k-100k+ fields…
Jerry Liu introduces ExtractBench and highlights the 'agentic plus' extractor in LlamaParse for handling massive volumes of fields in long documents, with benchmark results available on ExtractBench.
@llama_index: How do you know your document parser is ready for production? Existing benchmarks miss what AI agents actually need. Th…
LlamaIndex announces ParseBench, a new benchmark for evaluating document parsing for AI agents, and invites AI engineers to a live webinar on May 27th to discuss its methodology and how it addresses gaps in existing benchmarks like OlmOCR.
@jerryjliu0: There are a lot of coding and reasoning benchmarks for AI agents, but not a lot for document understanding - which is a…
LlamaIndex released ParseBench, a comprehensive benchmark for evaluating document understanding in AI agents, covering complex enterprise documents with tables, charts, and layouts. A live webinar will discuss the benchmark methodology and results.
@llama_index: "OCR is just a feature now. Frontier models will eat it." We hear this constantly. The data says otherwise. Across thre…
LlamaIndex argues that document OCR is not being commoditized by frontier models, using benchmark data showing specialized parsers remain more accurate and cheaper.