MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

Hugging Face Daily Papers Papers
document-ai ocr visual-text multimodal pretraining foundation-model

Summary

MonkeyOCRv2 is a visual-text foundation model for document AI, pretrained on a large corpus of 113 million images across 17 languages using joint image-to-text generation and pixel-level document reconstruction. It achieves state-of-the-art results on document parsing and understanding tasks, outperforming previous models with a smaller vision encoder.

Mainstream visual encoders are pretrained on natural images and cannot be effectively applied to document images without document-oriented adaptation, as dense text and fine-grained character strokes demand character-level visual perception. We present MonkeyOCRv2, a visual-text pretrained model for document AI. First, we construct MonkeyDoc v2, to our knowledge the largest document-image pretraining corpus, comprising 113 million images spanning 17 languages. Second, we propose a pretraining strategy that jointly learns image-to-text generation and pixel-level document reconstruction: the former aligns visual representations with textual content, while the latter preserves character strokes and layout details. Extensive experiments are conducted on five representative document analysis tasks, including text recognition, formula recognition, text detection, document tampering detection, and overlapping text segmentation. Replacing the original encoders with MonkeyOCRv2 consistently improves performance across all five tasks. Finally, we validate its effectiveness as the vision encoder of multimodal large language models on the more challenging tasks of document parsing and document understanding. Kept frozen and paired with a lightweight language model, it yields a 0.7B document parsing model that sets a new open-source state-of-the-art on MDPBench, a recent benchmark spanning digital-born and photographed documents across 17 languages, surpassing the previous best 3B dots.mocr by 2.8% absolute with a vision encoder roughly 11times smaller. The frozen encoder also powers a document understanding model that outperforms counterparts built on CLIP, DINO, and SAM across eight benchmarks under identical training settings. These results suggest that document-oriented visual pretraining can serve as a foundation for document intelligence in its own right.
Original Article
View Cached Full Text

Cached at: 07/15/26, 04:22 PM

Paper page - MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

Source: https://huggingface.co/papers/2607.11562 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Mainstreamvisualencodersarepretrainedonnaturalimagesandcannotbeeffectivelyappliedtodocumentimageswithoutdocument-orientedadaptation,asdensetextandfine-grainedcharacterstrokesdemandcharacter-levelvisualperception.WepresentMonkeyOCRv2,avisual-textpretrainedmodelfordocumentAI.First,weconstructMonkeyDocv2,toourknowledgethelargestdocument-imagepretrainingcorpus,comprising113millionimagesspanning17languages.Second,weproposeapretrainingstrategythatjointlylearnsimage-to-textgenerationandpixel-leveldocumentreconstruction:theformeralignsvisualrepresentationswithtextualcontent,whilethelatterpreservescharacterstrokesandlayoutdetails.Extensiveexperimentsareconductedonfiverepresentativedocumentanalysistasks,includingtextrecognition,formularecognition,textdetection,documenttamperingdetection,andoverlappingtextsegmentation.ReplacingtheoriginalencoderswithMonkeyOCRv2consistentlyimprovesperformanceacrossallfivetasks.Finally,wevalidateitseffectivenessasthevisionencoderofmultimodallargelanguagemodelsonthemorechallengingtasksofdocumentparsinganddocumentunderstanding.Keptfrozenandpairedwithalightweightlanguagemodel,ityieldsa0.7Bdocumentparsingmodelthatsetsanewopen-sourcestate-of-the-artonMDPBench,arecentbenchmarkspanningdigital-bornandphotographeddocumentsacross17languages,surpassingthepreviousbest3Bdots.mocrby2.8%absolutewithavisionencoderroughly11timessmaller.ThefrozenencoderalsopowersadocumentunderstandingmodelthatoutperformscounterpartsbuiltonCLIP,DINO,andSAMacrosseightbenchmarksunderidenticaltrainingsettings.Theseresultssuggestthatdocument-orientedvisualpretrainingcanserveasafoundationfordocumentintelligenceinitsownright.

View arXiv pageView PDFGitHub144Add to collection

Models citing this paper7

#### zenosai/MonkeyOCRv2-B-Parsing Image-Text-to-Text• 0.9B• Updated1 day ago • 16 • 2 #### zenosai/MonkeyOCRv2-S Image-Text-to-Text• 28.5M• Updated1 day ago • 6 • 2 #### zenosai/MonkeyOCRv2-B Image-Text-to-Text• 0.1B• Updated1 day ago • 16 • 2 #### zenosai/MonkeyOCRv2-S-Parsing Image-Text-to-Text• 0.8B• Updated1 day ago • 15 • 1 Browse 7 models citing this paper## Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.11562 in a dataset README.md to link it from this page.

Spaces citing this paper1

Collections including this paper1

Similar Articles

OvisOCR2 Technical Report

Hugging Face Daily Papers

OvisOCR2 is a 0.8B parameter end-to-end document parsing model that converts document page images to Markdown, achieving state-of-the-art scores on public benchmarks through a combination of supervised fine-tuning, reinforcement learning, and model fusion.

OvisOCR2: a promising 0.8B local document parser

Reddit r/LocalLLaMA

OvisOCR2 is a new 0.8B end-to-end OCR model based on Qwen3.5-0.8B that converts full document pages directly into structured Markdown, including text, tables, formulas, and reading order. It achieves strong benchmark scores and is released under Apache 2.0 with vLLM support.

ATH-MaaS/OvisOCR2

Hugging Face Models Trending

OvisOCR2 is a compact 0.8B end-to-end model for page-level document parsing, achieving state-of-the-art performance on OmniDocBench and PureDocBench benchmarks.

@geekbb: Baidu's open-source visual language model OCR project, upgraded from DeepSeek-OCR, focuses on one-shot parsing of extremely long documents. The model has two inference modes: 'gundam' mode for dense text in a single image, and 'base' mode for multi-page or PDF processing. https://github…

X AI KOLs Timeline

Baidu has open-sourced the visual language model Unlimited-OCR, upgraded from DeepSeek-OCR, supporting one-shot parsing of extremely long documents, offering two inference modes: gundam (dense text in a single image) and base (multi-page/PDF).