Tag
An introduction to the open-source Agent Skill "learn-from-materials," which converts hundreds of pages of material — PDF, EPUB, Word, PPT, web pages, and more — into an interactive learning website and knowledge base, with citation tracing precise down to page numbers and chapter sections. Compatible with environments like Claude Code, Codex, and Copilot CLI.
A practitioner asks the community which open-source OCR stack is best for fully local batch processing of 20-25 page mixed-content PDFs, shortlisting tools like Docling, GLM-OCR, PaddleOCR-VL, OlmOCR 2, and MinerU while weighing VLMs against text-layer parsers for tables and scanned pages.
The Claude API has increased the per-request size limit for Messages and Token Counting from 32MB to 128MB, roughly a fourfold increase, to support tasks such as long conversations containing many images and large PDFs.
A Rust-based text chunking library claiming ~20x speedups over LangChain, LlamaIndex, and similar Python solutions, with multiple chunking strategies, a native PDF loader, and parallel batch processing. Benchmark tables show Recursive chunking at ~2,264 MB/s and end-to-end PDF pipelines ~15x faster than pypdf-based baselines.
Datalab's Marker v2 is an open-source document parsing tool that handles PDFs, images, DOCX, and PPTX, outputting clean Markdown and supporting over 90 languages for batch organization, knowledge bases, and AI document processing.
The post highlights the use of Antigravity with Gemini 3.8 Flash for finding and organizing PDFs into a corpus, and mentions a Claude Code skill for bulk-converting PDFs to markdown to reduce costs.
DocJev is a free, open-source Python tool for classifying and splitting complex document packets, supporting PDF, DOCX, and PPTX with optional OCR.
PixelRAG is an open-source project from UC Berkeley SkyLab and other teams. By capturing web pages and PDFs as screenshots and using visual indexing for retrieval, it improves RAG accuracy and significantly reduces token costs for AI Agents.
Google Developers showcased using Gemini 3.7 Flash in Google AI Studio to transform a PDF annual report into an interactive web app, highlighting the model's data extraction and UI generation capabilities in a single prompt.
Google AI Devs demonstrated Gemini 3.7 Flash by using it to extract and classify plants from historical botanical PDFs with an interactive visualization.
Stirling-PDF is an open-source PDF processing platform offering 50+ tools (merge, split, OCR, compression, etc.), supporting browser use, Docker self-hosting, and a REST API, as well as a no-code automation pipeline. The project has 89k+ stars on GitHub and can be fully deployed locally to protect privacy.
Introduces a tool called book-to-skill that converts books or document packages into AI Agent callable Skills. It supports PDF and other formats, generates SKILL.md and chapter indexes, avoiding loading the full context at once.
A benchmark comparing vision-capable LLMs (native PDF reading) against OCR-based pipelines on 30 long, image-heavy PDFs finds that OCR with layout extraction still outperforms vision models on chart/table-heavy pages and has a 0% failure rate vs. 7% for native PDF, though the sample size is small and many gaps are within noise.
Synthadoc is an open-source tool that compiles PDFs, documents, and other project materials into a structured local Markdown wiki, automatically establishing cross-references and detecting contradictions. It is suitable for personal or small teams for offline knowledge management.
book-to-skill converts technical books into structured skills for Claude Code, allowing on-demand reference and eliminating hallucination.
olmOCR is an open-source toolkit using a fine-tuned vision language model to extract clean text from PDFs while preserving structure, optimized for large-scale batch processing.
MinerU is an open-source tool by OpenDataLab for extracting data from PDFs and documents.