Tag
This tweet benchmarks Gemini 3.6 Flash and Gemini 3.5 Flash Lite on document understanding, finding that while the Flash series initially excelled at visual understanding, recent versions have plateaued or regressed due to posttraining for coding and reasoning.
DeepSeek OCR 2 is a 3B parameter model that outperforms Gemini 3 Pro in OCR and document understanding, featuring a human-like reading order and support for local fine-tuning.
SynthDocBench is a fully synthetic benchmark for long-context visual document understanding that systematically controls document length, layout, modality, and question type, revealing failure modes in current VLMs such as length degradation and positional sensitivity.
LlamaIndex benchmarked GPT-5.6 on document understanding and found no improvement over GPT-5.5; the model performs well on text and tables but struggles with charts and layout.
Introduces BaFCo, a benchmark dataset for Bangla form comprehension focusing on Document Layout Analysis (DLA) and Key Information Extraction (KIE). It includes 200 multi-page complex Bangladeshi government forms with fine-grained annotations across 26 entity types and evaluates multiple MLLMs, revealing limitations in understanding complex Bangla forms.
HunyuanOCR-1.5 is a lightweight end-to-end OCR vision-language model that improves efficiency via DFlash (6.37x inference speedup) and capability via Agentic Data Flow, achieving top-tier performance on document parsing, OCR, and multilingual tasks.
LEDGER is a new benchmark for evaluating long-context capabilities of LLMs on corporate annual reports, providing 4,999 digitized reports with 31 financial KPIs and three evaluation tasks spanning retrieval and extraction.
Jerry Liu's team is presenting ParseBench, a comprehensive document understanding benchmark for VLMs, at CVPR 2026. The benchmark includes 2,000 pages of real-world enterprise documents with evaluation metrics for tables, charts, and visual grounding.
MM-BizRAG is a multimodal retrieval-augmented generation system for enterprise Q&A that uses document structure-aware splitting and layout-aware parsing to outperform vision-centric baselines by up to 32% on heterogeneous enterprise documents. The paper also introduces FastRAGEval, a cost-efficient LLM-based evaluation metric with stronger human alignment than RAGChecker.
LFRAG proposes a layout-oriented fine-grained retrieval-augmented generation framework that moves from page-level to block-level retrieval in multimodal documents, achieving state-of-the-art performance and 73% token reduction on the new LFDocQA benchmark.
Nasdaq features an interview with Llama Index CEO Jerry Liu, discussing the company's document understanding and OCR technologies powering enterprise AI agents, in partnership with Wing VC's Enterprise Tech 30 list.
LlamaIndex released ParseBench, a comprehensive benchmark for evaluating document understanding in AI agents, covering complex enterprise documents with tables, charts, and layouts. A live webinar will discuss the benchmark methodology and results.
Infinity releases two open-weight models, Infinity-Parser2-Pro (35B) and Infinity-Parser2-Flash (2B), which top the ParseBench leaderboard for document understanding, leveraging a synthetic data engine and a novel joint RL algorithm.
CiteVQA is a benchmark for document vision-language models that evaluates both answer correctness and citation of supporting evidence, revealing widespread attribution hallucinations where models provide correct answers but cite wrong regions.
DocScope is a new benchmark for evaluating the verifiable reasoning and trustworthiness of Multimodal Large Language Models on long documents, introducing a four-stage evaluation protocol for page localization, region grounding, fact extraction, and answer verification.
DocAtlas is a framework that creates high-fidelity OCR datasets and benchmarks across 82 languages, using differential rendering and synthetic generation. It demonstrates that Direct Preference Optimization improves multilingual model adaptation without degrading base-language performance.
NuExtract3 is a 4B vision-language reasoning model for document understanding, enabling structured extraction and image-to-Markdown conversion.
ParseBench introduces the first benchmark evaluating vision-language models on chart comprehension within full enterprise documents, addressing gaps in prior chart-only benchmarks.
dots.ocr is a new lightweight 1.7B parameter multilingual vision-language model that achieves state-of-the-art performance on OmniDocBench, outperforming much larger models (72B+) at document parsing and OCR tasks.
IBM releases Granite 4.0 3B Vision, a compact vision-language model designed for enterprise document understanding, featuring specialized capabilities for table extraction, chart interpretation via ChartNet, and key-value pair grounding.