nvidia/NVIDIA-Nemotron-Parse-2.0 · Hugging Face

Source: https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#model-overviewModel Overview

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#descriptionDescription:

NVIDIA Nemotron Parse 2.0 transforms document images into structured, machine-readable representations with text, layout classes, bounding boxes, and reading-order information. Given a Red, Green, Blue (RGB) document image and a task prompt, the model produces formatted text and spatial annotations for document elements such as titles, paragraphs, captions, tables, charts, page headers, page footers, footnotes, pictures, and bibliography entries. Compared with NVIDIA Nemotron Parse v1.2, NVIDIA Nemotron Parse 2.0 adds an approximately 20k-token vocabulary expansion for more efficient multilingual support, chart-aware document parsing with the<class\_Chart\>class token, and updated training coverage for chart/table-heavy documents. NVIDIA Nemotron Parse 2.0 is intended for document understanding, information retrieval, data extraction, and multimodal data-curation workflows.

This model is ready for commercial or non-commercial use.

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#licenseterms-of-useLicense/Terms of Use:

Governing Terms: Your use of this model is governed by theNVIDIA Open Model License Agreement. Use of the tokenizer included in this model is governed by theCC-BY-4.0 license.

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#deployment-geographyDeployment Geography:

Global

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#use-case-Use Case:

NVIDIA Nemotron Parse 2.0 is designed for developers and teams building document intelligence, retrieval-augmented generation (RAG), curator, extractor, and agentic AI applications. It can be used to convert scanned or rendered PDFs, presentation slides, forms, reports, tables, and mixed-content document pages into structured outputs for downstream indexing, retrieval, analytics, model training-data creation, and human-in-the-loop review.

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#capability-highlights-Capability Highlights:

• Expanded multilingual OCR support, with substantial gains on CJK and Indic-script document text.
• Improved handwritten-text extraction for document pages containing informal, handwritten, or note-like content.
• Chart-to-table parsing that can identify chart regions and convert visible chart information into structured text for downstream use.
• Improved table handling, including stronger table detection, structure recovery, and text extraction on table-heavy documents.

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#release-date–Release Date:

Hugging Face August 3, 2026 viaURL

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#referencessReferences(s):

• Hugging Face Transformers mBART documentation
• NVIDIA C-RADIO

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#model-architectureModel Architecture:

**Architecture Type:**Transformer-based vision-encoder-decoder model

Network Architecture:

• Vision Encoder: ViT-H model based on NVIDIA C-RADIO
• Adapter Layer: 1D convolutions and normalization layers that compress the vision latent sequence before decoding
• Decoder: mBART decoder with 10 blocks
• Auxiliary Prediction Head: One training-time decoder prediction head is preserved separately inauxiliary\_prediction\_heads\.safetensors\.extrafor future multi-token prediction research. Standard generation uses the tied decoder input/output embeddings; the defaultmodel\.safetensors, Transformers examples, and vLLM examples do not load this auxiliary head.
• Tokenizer: The tokenizer contains 72,256 entries, including an approximately 20k-token expansion over NVIDIA Nemotron Parse v1.2 to improve multilingual token efficiency. The model also includes task/control tokens such as<predict\_bbox\>,<predict\_classes\>,<predict\_text\_in\_pic\>, and<predict\_no\_text\_in\_pic\>, plus the chart class token<class\_Chart\>. Use of the tokenizer included in this model is governed by theCC-BY-4.0 license.
• Number of Parameters: < 1B

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#inputs-Input(s):

**Input Type(s):**Image, Text

Input Format(s):

• Image: Red, Green, Blue (RGB)
• Text: Prompt string

Input Parameters:

• Image: Two-Dimensional (2D)
• Text: One-Dimensional (1D)

Other Properties Related to Input:

• Recommended maximum input resolution (Width, Height): 1664, 2048
• Recommended minimum input resolution (Width, Height): 1024, 1280
• Channel Count: 3
• Prompt format: one task prompt composed from the supported control tokens. The default prompt extracts bounding boxes, semantic classes, and text in markdown format:</s\><s\><predict\_bbox\><predict\_classes\><output\_markdown\><predict\_no\_text\_in\_pic\>. The model can emit chart regions using<class\_Chart\>when chart content is detected.

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#outputs-Output(s):

**Output Type(s):**Text

**Output Format(s):**String

**Output Parameters:**One-Dimensional (1D)

**Other Properties Related to Output:**Nemotron Parse 2.0 returns a string encoding document text, semantic element classes, and bounding boxes. Postprocessing utilities in this repository can transform generated bounding boxes back to original image coordinates and convert table or chart-associated text into LaTeX, HTML, markdown, JSON, or CSV where supported.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA hardware and software frameworks, the model achieves faster training and inference times compared to CPU-only solutions.

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#quick-startQuick Start

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#install-dependencies-in-your-environmentInstall dependencies in your environment

You can use a public imagenvcr.io/nvidia/pytorch:25.03-py3with the following library versions installed on top:

pip install accelerate==1.12.0
pip install albumentations==2.0.8
pip install transformers==5.6.1
pip install timm==1.0.22
pip install open_clip_torch==3.2.0
pip install einops==0.8.1

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#usage-exampleUsage example

import torch
from PIL import Image, ImageDraw
from transformers import AutoModel, AutoProcessor, AutoTokenizer, GenerationConfig
from postprocessing import extract_classes_bboxes, transform_bbox_to_original, postprocess_text

# Load model and processor
model_path = "nvidia/NVIDIA-Nemotron-Parse-2.0"  # Or use a local path
device = "cuda:0"

model = AutoModel.from_pretrained(
    model_path,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16
).to(device).eval()
tokenizer = AutoTokenizer.from_pretrained(model_path)
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)

# Load image
image = Image.open("path/to/your/image.jpg")
task_prompt = "</s><s><predict_bbox><predict_classes><output_markdown><predict_no_text_in_pic>"
# task_prompt = "</s><s><predict_bbox><predict_classes><output_markdown><predict_text_in_pic>"

# Process image
inputs = processor(images=[image], text=task_prompt, return_tensors="pt", add_special_tokens=False).to(device)

generation_config = GenerationConfig.from_pretrained(model_path, trust_remote_code=True)

# Generate text
outputs = model.generate(**inputs, generation_config=generation_config)

# Decode the generated text
generated_text = processor.batch_decode(outputs, skip_special_tokens=True)[0]

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#postprocessingPostprocessing

from PIL import ImageDraw
from postprocessing import extract_classes_bboxes, transform_bbox_to_original, postprocess_text

classes, bboxes, texts = extract_classes_bboxes(generated_text)
bboxes = [transform_bbox_to_original(bbox, image.width, image.height) for bbox in bboxes]

# Specify output formats for postprocessing
table_format = "latex"  # latex | HTML | markdown | json | json_hierarchical | csv
text_format = "markdown"  # markdown | plain
blank_text_in_figures = False  # set True to remove text inside 'Picture' class
texts = [
    postprocess_text(
        text,
        cls=cls,
        table_format=table_format,
        text_format=text_format,
        blank_text_in_figures=blank_text_in_figures,
    )
    for text, cls in zip(texts, classes)
]

for cl, bb, txt in zip(classes, bboxes, texts):
    print(cl, ": ", txt)

draw = ImageDraw.Draw(image)
for bbox in bboxes:
    draw.rectangle((bbox[0], bbox[1], bbox[2], bbox[3]), outline="red")

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#table-output-formatsTable output formats

Supported values fortable\_format:latex|HTML|markdown|json|json\_hierarchical|csv

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#inference-with-vllmInference with vLLM

**Tested vLLM versions:**v0.20-v0.26.

Nemotron Parse 2.0 can be served with vLLM when using a vLLM build that includes Nemotron Parse remote-code support. On A100 and A10 systems, we recommend runningvllm servewith\-\-attention\-backend=TRITON\_ATTN.

Install the following dependencies on top of the serving image:

pip install albumentations timm open_clip_torch einops

This export keepslm\_head\.weighttied todecoder\.embed\_tokens\.weightand does not materialize a duplicate output-head tensor. Current vLLM 0.20 Nemotron Parse builds create a separate output head unless patched. If your vLLM build does not already support tied Nemotron Parse output embeddings, fetch the included runtime patch and add it toPYTHONPATHbefore starting vLLM:

PATCH_ROOT=$(python - <<'PY'
from huggingface_hub import snapshot_download
print(snapshot_download(
    "nvidia/NVIDIA-Nemotron-Parse-2.0",
    allow_patterns="vllm_tied_patch/sitecustomize.py",
))
PY
)
export PYTHONPATH="${PATCH_ROOT}/vllm_tied_patch:${PYTHONPATH}"

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#vllm-inference-examplevLLM Inference example

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#option-1-end-to-end-python-inferenceOption 1: end-to-end Python inference

from vllm import LLM, SamplingParams
from PIL import Image

def main():
    sampling_params = SamplingParams(
        temperature=0,
        top_k=1,
        repetition_penalty=1.1,
        max_tokens=9000,
        skip_special_tokens=False,
    )

    llm = LLM(
        model="nvidia/NVIDIA-Nemotron-Parse-2.0",
        max_num_seqs=64,
        limit_mm_per_prompt={"image": 1},
        dtype="bfloat16",
        trust_remote_code=True,
    )

    image = Image.open("<YOUR-IMAGE-PATH>")

    prompts = [
        {
            "prompt": "</s><s><predict_bbox><predict_classes><output_markdown><predict_no_text_in_pic>",
            "multi_modal_data": {
                "image": image,
            },
        },
        {
            "encoder_prompt": {
                "prompt": "",
                "multi_modal_data": {
                    "image": image,
                },
            },
            "decoder_prompt": "</s><s><predict_bbox><predict_classes><output_markdown><predict_no_text_in_pic>",
        },
    ]

    outputs = llm.generate(prompts, sampling_params)

    for output in outputs:
        prompt = output.prompt
        generated_text = output.outputs[0].text
        print(f"Decoder prompt: {prompt!r}, Generated text: {generated_text!r}")

if __name__ == "__main__":
    main()

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#option-2-vllm-serveOption 2: vLLM serve

vllm serve nvidia/NVIDIA-Nemotron-Parse-2.0 \
    --dtype bfloat16 \
    --max-num-seqs 8 \
    --limit-mm-per-prompt '{"image": 1}' \
    --trust-remote-code \
    --port 8000 \
    --chat-template chat_template.jinja

Then run inference through the OpenAI-compatible API:

import base64
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
)

with open("<YOUR-IMAGE-PATH>", "rb") as f:
    img_b64 = base64.b64encode(f.read()).decode("utf-8")

prompt_text = "</s><s><predict_bbox><predict_classes><output_markdown><predict_no_text_in_pic>"

resp = client.chat.completions.create(
    model="nvidia/NVIDIA-Nemotron-Parse-2.0",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": prompt_text,
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/png;base64,{img_b64}",
                    },
                },
            ],
        }
    ],
    max_tokens=9000,
    temperature=0.0,
    extra_body={
        "repetition_penalty": 1.1,
        "top_k": 1,
        "skip_special_tokens": False,
    },
)
print(resp.choices[0].message.content)

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#prompt-and-logits-processor-optionsPrompt and logits-processor options

The recommended default prompt extracts bounding boxes, semantic classes, and text in markdown format:

</s\><s\><predict\_bbox\><predict\_classes\><output\_markdown\><predict\_no\_text\_in\_pic\>

To extract text that appears inside embedded images or figures, use:

</s\><s\><predict\_bbox\><predict\_classes\><output\_markdown\><predict\_text\_in\_pic\>

If only bounding boxes and classes are needed, use:

</s\><s\><predict\_bbox\><predict\_classes\><output\_no\_text\><predict\_no\_text\_in\_pic\>

This repository includes two optional logits processors:

• NemotronParseRepetitionStopProcessor: detects repeating n-grams during generation and forces the model to close the coordinate block when repeated structured output suggests a potential hallucination.
• NemotronParseTableInsertionLogitsProcessor: forces every block to follow a table structure, which can be useful when running the model on table image crops.

Please refer toexample\_with\_processor\.pyfor Python-model usage. With vLLM, exportlogitsprocs/toPYTHONPATHand pass the desired processor tovllm serve, for example:

vllm serve nvidia/NVIDIA-Nemotron-Parse-2.0 \
  --dtype bfloat16 \
  --max-num-seqs 4 \
  --limit-mm-per-prompt '{"image": 1}' \
  --attention-backend=TRITON_ATTN \
  --trust-remote-code \
  --logits-processors nemotron_parse_vllm_logitprocs:NemotronParseTableInsertionLogitsProcessor \
  --port 8000

An example of inference with the vLLM OpenAI-compatible server is available invllm\_example\.py.

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#software-integrationSoftware Integration:

Runtime Engine(s):

• Transformers
• vLLM

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Ampere
• NVIDIA Blackwell
• NVIDIA Hopper
• NVIDIA Turing

Supported Operating System(s):

• Linux

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels is essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#model-versionsModel Version(s):

Nemotron Parse 2.0

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#training-testing-and-evaluation-datasetsTraining, Testing, and Evaluation Datasets:

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#training-datasetTraining Dataset

Data Modality:

• Image
• Text

Image Training Data Size:

• 1 Million to 1 Billion Images

Text Training Data Size:

• 1 Billion to 10 Trillion Tokens

Data Collection Method by dataset:

• Hybrid: Automated, Human, Synthetic

Labeling Method by dataset:

• Hybrid: Automated, Human, Synthetic

**Properties:**The training set contains millions of image-text items aggregated across large document, table, and layout datasets. The data consists of document-page and table images paired with OCR text, bounding boxes, and layout labels. Sources include rendered digital documents, scientific papers, PDFs, Wikipedia-style pages, and synthetic document, table, word, and character renderings. Annotations come from OCR and layout models, third-party OCR services, synthetic-generation pipelines, and human labeling.

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#testing-and-evaluation-datasetTesting and Evaluation Dataset

Testing and evaluation use internal and public document-understanding benchmarks that cover OCR quality, layout structure, table parsing, reading order, and visual grounding. Dataset collection and labeling methods are hybrid and include automated, model-derived, manually labeled, and synthetic annotations.

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#evaluation-resultsEvaluation Results

Benchmark coverage:

• ParseBench evaluates document parsing across text fidelity, semantic formatting, tables, charts, and visual grounding with layout boxes/classes.
• IndicVisionBench evaluates OCR quality for Indic-language pages, including Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, and Telugu.
• MOSCAR evaluates multilingual synthetic OCR across broad script coverage, including Latin, Arabic, Cyrillic, Chinese, Hangul, Japanese, Indic scripts, Hebrew, Thai, Greek, and others.
• OmniDocBench Notes (Handwriting) evaluates text-block edit distance on note-style document pages from thedata\_source: noteslice.

The following results compare NVIDIA Nemotron Parse 2.0 with NVIDIA Nemotron Parse v1.2 on internal and public evaluation benchmarks. NVIDIA Nemotron Parse 2.0 results use the equal-weight checkpoint soup from training checkpoints 58k, 60k, 62k, 64k, and 66k. Unless marked otherwise, metrics are higher-is-better; arrows indicate the direction of the 2.0 change relative to v1.2.

BenchmarkMetricNVIDIA Nemotron Parse v1.2NVIDIA Nemotron Parse 2.0ChangeParseBenchOverall score0.57820.6391↑ +0.0609OmniDocBench Notes (Handwriting)Text edit distance (lower is better)0.97390.3395↓ -0.6343IndicVisionBenchOverall ANLS character0.06120.7203↑ +0.6592MOSCAR (Multilingual)Overall BoC F10.44100.9102↑ +0.4692

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#inferenceInference:

**Acceleration Engine:**Transformers, vLLM Test Hardware:

• H100
• A100

https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-2.0#ethical-considerationsEthical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please make sure you have proper rights and permissions for all input image and video content. If an input image contains people, personal health information, confidential business data, or intellectual property, the model may extract or reproduce visible text from that content.

For more detailed information on ethical considerations for this model, please see the Model Card++ subcards:Bias,Explainability,Safety & Security, andPrivacy. Please report model quality, risk, security vulnerabilities, or NVIDIA AI Concernshere.