Tag
The user shares their 6-month experience with NotebookLM and provides 10 prompts, claiming to convert 200 pages of documents into clear answers in 1 hour.
Microsoft has open-sourced MarkItDown, a tool that can convert PDF, Word, Excel, PPT and other files into well-structured Markdown format with a single click, making it easy to feed directly into LLMs. It has garnered over 138k stars on GitHub.
Attended the Applied AI Conference in Berlin and gave a talk on building document agents, including a detailed walkthrough of LobsterX, a document-processing agent built with LlamaIndex that uses structured outputs and event-driven workflows.
This blog post describes the architecture for a scalable ingestion pipeline using Temporal to handle crawling, extracting, chunking, and embedding customer documentation from various sources, emphasizing durability, statefulness, and concurrency control.
The author shares their three-year experience of feeding PDFs to AI, pointing out that Markdown is a better input format for AI than PDF, because PDF is essentially a mix of coordinates and characters. AI needs to parse the structure first, which is error-prone and consumes more tokens. The article provides specific cases and recommended tools (markitdown, pandoc, LlamaParse), and teases a new series called 'The Art of Feeding AI'.
LightRAG v1.5 is released with six major improvements including multimodal document processing, enhanced parsing, and role-specific LLM configuration, making RAG simpler, faster, and more powerful.
LlamaParse now offers latency metrics for Parse, Extract, and Classify jobs, providing queue time, processing time, and total latency breakdowns. This helps users monitor and scale their document processing.
Parsewise is an API for agentic multi-document processing, enabling efficient handling of multiple documents.
pdf-inspector is an open-source Rust library for intelligently classifying PDF types (text or scanned), extracting text, and converting to Markdown, avoiding unnecessary OCR to improve speed and save costs.
MADP is a multi-agent architecture for enterprise document processing that combines deep learning and LLMs with human-in-the-loop validation, achieving 97% automation and significant reductions in resource usage.
This reference implementation demonstrates how to run an LLM agent securely within a local sandbox to process and analyze various document types using Rust, LiteParse, and microsandbox. The open-source CLI leverages OpenAI's GPT models and native bash commands to perform file retrieval and analysis in an isolated environment.
Paper2Any is an open-source AI tool that converts research papers into editable diagrams, technical roadmaps, and slide decks with support for universal file formats and custom styling.
LiteParse is an open-source, heuristic-based PDF parser that quickly converts complex layouts, text, and tables into a clean spatial grid without relying on ML models.
Researchers from Banting Health AI present an AI system using generative LLMs with Retrieval-Augmented Generation (RAG) for automated clinical trial protocol information extraction, achieving 89% accuracy compared to 62.6% for standalone LLMs, with AI-assisted workflows completing tasks 40% faster and reducing cognitive demand.
Jerry Liu discusses challenges with using Vision Language Models for PDF parsing, particularly around ensuring text correctness and maintaining proper reading order while avoiding hallucinations.
BankStatementLab is an AI-powered tool that converts bank statement PDFs into Excel, CSV, or JSON formats.
OpenAI shares how it built an internal contract data agent that automates the extraction and structuring of contract data from various document formats while keeping finance experts in control through a human-in-the-loop review process. The system has reduced contract review time by half and enabled the team to process thousands of contracts monthly without proportional headcount expansion.
SmolDocling is a compact 256M parameter vision-language model designed for end-to-end multi-modal document conversion. It introduces a new universal markup format called DocTags to capture page elements with location, competing with models 27 times larger.
Docling is an open-source document processing tool that parses diverse formats, including advanced PDF understanding, and provides integrations with generative AI ecosystems like LangChain and LlamaIndex.
book-to-skill is a tool that converts technical books, document folders, or collections of sources into structured agent skill files for use with GitHub Copilot CLI, Amp, and Claude Code, reducing token usage by 24x-51x compared to dumping full book text into context.