Tag
A DocInsights 2026 shared-task system paper describing EviCalc, a PDF document QA pipeline that pairs evidence passage selection with LLM-generated arithmetic expressions evaluated in local code, reporting 8.61% joint accuracy on DocSem's test set along with descriptive failure analysis, including OCR and block-grouping errors that merged relevant passages.
This paper systematically evaluates input representations (images, text, or both) for multimodal document QA, finding that images improve accuracy but increase latency, and proposes a TF-IDF router to optimize performance.
The author describes consolidating four small AI models from separate services into a single server using Superlinked's inference engine to reduce operational overhead, while discussing trade-offs like GPU sharing and blast radius concerns.
DocHop introduces a benchmark for evaluating multi-hop reasoning in information-dense documents, focusing on chart-context integration in multimodal large language models, with experiments showing a significant gap between model and human performance.
Recommends 5 open-source RAG tools (RAGFlow, AnythingLLM, Onyx, Khoj, kotaemon) that turn documents into a Q&A knowledge base with zero code, each with unique features.
A user seeks recommendations for PDF preprocessing tools to improve input quality for local LLM-based document QA, comparing pymupdf, pdfplumber, docling, and llamaparse for handling messy layouts like tables and multi-column text.
A benchmark comparing vision-capable LLMs (native PDF reading) against OCR-based pipelines on 30 long, image-heavy PDFs finds that OCR with layout extraction still outperforms vision models on chart/table-heavy pages and has a 0% failure rate vs. 7% for native PDF, though the sample size is small and many gaps are within noise.