StarDoc-AI/TeleOCR · Hugging Face 来源：https://huggingface.co/StarDoc-AI/TeleOCR

https://huggingface.co/StarDoc-AI/TeleOCR#%F0%9F%94%A5-news🔥 最新动态

• 2026/09/10
  • 我们已将 NaviDC-OCR 更名为 TeleOCR，后续所有模型迭代将在 TeleOCR 版本下开发和发布。
• 2026/09/01
  • 我们注意到 EMNLP 2026 正在举办 Dr.DocBench 挑战赛 (https://eval.ai/web/challenges/challenge-page/2717/overview)，这是一个文档解析竞赛。我们使用 NaviDC-OCR 的原生权重进行了评估，取得了优于 MinerU 2.5 Pro 和 PaddleOCR-VL 1.6 的结果。详细结果如下表 dr.docbench-challenge 所示。我们欢迎使用 NaviDC-OCR 参加竞赛。未来，我们将继续为社区提供有竞争力的解析模型。
• 2026/08/29
  • 感谢 Nandraj 提供的 GGUF 转换和 llama.cpp 支持 🔗 NaviDC-OCR-GGUF (https://huggingface.co/nandraj/NaviDC-OCR-GGUF)，以及社区分享的在 Ascend 910B 上部署 NaviDC-OCR 的经验！(https://zhuanlan.zhihu.com/p/2078516899093155893)
• 2026/08/17
  • NaviDC-OCR 模型权重 (https://huggingface.co/StarDoc-AI/NaviDC-OCR) 和技术报告 (https://arxiv.org/pdf/2608.12898) 已发布。

https://huggingface.co/StarDoc-AI/TeleOCR#%F0%9F%93%96-introduction📖 简介

TeleOCR 是一个轻量级（约 1.2B 参数）的开源视觉语言模型，专为文档解析设计。与现有主要针对数字文档或相机捕获文档的方法不同，TeleOCR 将这两种场景统一在一个框架内。与之前的文档解析模型相比，TeleOCR 引入了：

• 用于自动生成伪标签的多节点共识投票 (Multi-node Consensus Voting, MCV)
• 针对相机捕获文档的几何感知文档建模
• 曲率引导的道格拉斯-普克采样 (Curvature-Guided Douglas-Peucker Sampling, CGDP)
• 用于自动数据精炼的图像到图像自验证
• 渐进式四阶段训练流程
• 用于表格和公式的“内容-结构解耦学习” 这些技术使得 TeleOCR 在数字文档和相机捕获文档基准测试中均达到了最先进的性能，同时保持了足够的轻量级以供实际部署。

https://huggingface.co/StarDoc-AI/TeleOCR#%F0%9F%93%8A-experimental-results📊 实验结果

TeleOCR 在多个公开的文档解析基准测试中取得了最先进的性能。

https://huggingface.co/StarDoc-AI/TeleOCR#layout-visualization-of-distorted-documents扭曲文档的布局可视化

为了评估模型理解复杂文档形变的能力，我们在公开的去畸变数据集 DocUNet 和 DIR300 上进行了视觉评估，代表性结果如图所示。TeleOCR 直接对扭曲文档进行布局和内容解析，无需去畸变预处理或专门的矫正模型，展示了在复杂几何形变下的稳健解析能力。

• DIR300 数据集上的解析评估。
• DocUNET 数据集上的解析评估。

https://huggingface.co/StarDoc-AI/TeleOCR#drdocbench-challengeDr.DocBench 挑战赛

https://huggingface.co/StarDoc-AI/TeleOCR#omnidocbench-v16OmniDocBench v1.6

https://huggingface.co/StarDoc-AI/TeleOCR#wild_omnidocbenchWild_OmniDocBench

https://huggingface.co/StarDoc-AI/TeleOCR#puredocbenchPureDocBench

https://huggingface.co/StarDoc-AI/TeleOCR#icdar2026-sci-imageminerICDAR2026 Sci-ImageMiner

https://huggingface.co/StarDoc-AI/TeleOCR#%F0%9F%9A%80-installation🚀 安装

pip install transformers torch pillow

https://huggingface.co/StarDoc-AI/TeleOCR#quick-start快速开始

import html
import itertools
import json
import re
from dataclasses import dataclass
from PIL import Image
import torch
from transformers import AutoProcessor, AutoModel

@dataclass
class ContentBlock:
    type: str
    bbox: list[float]
    angle: int | None = None
    content: str | None = None

@dataclass
class TableCell:
    text: str
    start_row_offset_idx: int
    end_row_offset_idx: int
    start_col_offset_idx: int
    end_col_offset_idx: int
    row_span: int = 1
    col_span: int = 1

OTSL_NL = ""
OTSL_FCEL = ""
OTSL_ECEL = ""
OTSL_LCEL = ""
OTSL_UCEL = ""
OTSL_XCEL = ""
OTSL_TOKENS = [OTSL_NL, OTSL_FCEL, OTSL_ECEL, OTSL_LCEL, OTSL_UCEL, OTSL_XCEL]

def _otsl_extract_tokens_and_text(text: str):
    pattern = "(" + "|".join(map(re.escape, OTSL_TOKENS)) + ")"
    tokens = re.findall(pattern, text)
    parts = [part for part in re.split(pattern, text) if part.strip()]
    return tokens, parts

def _count_right(rows, row_idx, col_idx, tokens):
    span = 0
    while col_idx < len(rows[row_idx]) and rows[row_idx][col_idx] in tokens:
        span += 1
        col_idx += 1
    return span

def _count_down(rows, row_idx, col_idx, tokens):
    span = 0
    while row_idx < len(rows) and col_idx < len(rows[row_idx]) and rows[row_idx][col_idx] in tokens:
        span += 1
        row_idx += 1
    return span

def _otsl_parse_texts(parts, tokens):
    rows = [list(row) for is_nl, row in itertools.groupby(tokens, lambda token: token == OTSL_NL) if not is_nl]
    if not rows:
        return [], []
    max_cols = max(len(row) for row in rows)
    for row in rows:
        row.extend([OTSL_ECEL] * (max_cols - len(row)))
    cells = []
    row_idx = 0
    col_idx = 0
    for idx, part in enumerate(parts):
        if part in (OTSL_FCEL, OTSL_ECEL):
            cell_text = ""
            right_offset = 1
            if part != OTSL_ECEL and idx + 1 < len(parts) and parts[idx + 1] not in OTSL_TOKENS:
                cell_text = parts[idx + 1].strip()
                right_offset = 2
            next_right = parts[idx + right_offset] if idx + right_offset < len(parts) else ""
            next_bottom = rows[row_idx + 1][col_idx] if row_idx + 1 < len(rows) and col_idx < len(rows[row_idx + 1]) else ""
            col_span = 1 + (_count_right(rows, row_idx, col_idx + 1, {OTSL_LCEL, OTSL_XCEL}) if next_right in {OTSL_LCEL, OTSL_XCEL} else 0)
            row_span = 1 + (_count_down(rows, row_idx + 1, col_idx, {OTSL_UCEL, OTSL_XCEL}) if next_bottom in {OTSL_UCEL, OTSL_XCEL} else 0)
            cells.append(TableCell(
                text=cell_text,
                row_span=row_span,
                col_span=col_span,
                start_row_offset_idx=row_idx,
                end_row_offset_idx=row_idx + row_span,
                start_col_offset_idx=col_idx,
                end_col_offset_idx=col_idx + col_span,
            ))
        if part in (OTSL_FCEL, OTSL_ECEL, OTSL_LCEL, OTSL_UCEL, OTSL_XCEL):
            col_idx += 1
        elif part == OTSL_NL:
            row_idx += 1
            col_idx = 0
    return cells, rows

def convert_otsl_to_html(otsl_content: str) -> str:
    if otsl_content.startswith("<"):
        return otsl_content
    tokens, parts = _otsl_extract_tokens_and_text(otsl_content)
    cells, rows = _otsl_parse_texts(parts, tokens)
    if not cells or not rows:
        return ""
    grid = [[None for _ in range(len(rows[0]))] for _ in range(len(rows))]
    for cell in cells:
        for row_idx in range(cell.start_row_offset_idx, min(cell.end_row_offset_idx, len(rows))):
            for col_idx in range(cell.start_col_offset_idx, min(cell.end_col_offset_idx, len(rows[0]))):
                grid[row_idx][col_idx] = cell
    html_rows = []
    for row_idx, row in enumerate(grid):
        html_rows.append("<tr>")
        for col_idx, cell in enumerate(row):
            if cell is None or cell.start_row_offset_idx != row_idx or cell.start_col_offset_idx != col_idx:
                continue
            attrs = ""
            if cell.row_span > 1:
                attrs += f' rowspan="{cell.row_span}"'
            if cell.col_span > 1:
                attrs += f' colspan="{cell.col_span}"'
            html_rows.append(f"<td{attrs}>{html.escape(cell.text.strip())}</td>")
        html_rows.append("</tr>")
    return "<table>" + "".join(html_rows) + "</table>"

def post_process(blocks: list[ContentBlock]) -> list[ContentBlock]:
    for block in blocks:
        if block.type == "table" and block.content:
            block.content = convert_otsl_to_html(block.content)
        elif block.type == "equation" and block.content:
            content = block.content.strip()
            content = content.removeprefix("\\[").removesuffix("\\]").strip()
            if not (content.startswith("$") and content.endswith("$")):
                content = f"$${content}$$"
            block.content = content
    return [block for block in blocks if block.type != "equation_block"]

def infer(image: Image.Image, prompt: str) -> str:
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": [
            {"type": "image"},
            {"type": "text", "text": prompt},
        ]},
    ]
    chat_prompt = processor.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )
    inputs = processor(
        text=[chat_prompt],
        images=[image.convert("RGB")],
        padding=True,
        return_tensors="pt",
    ).to(device=model.device, dtype=model.dtype)
    output_ids = model.generate(
        **inputs,
        use_cache=True,
        max_new_tokens=4096,
        do_sample=False,
    )
    output_ids = output_ids.cpu().tolist()[0][len(inputs.input_ids[0]):]
    return processor.batch_decode(
        [output_ids],
        skip_special_tokens=True,
        clean_up_tokenization_spaces=False,
    )[0].strip()

processor = AutoProcessor.from_pretrained("StarDoc-AI/TeleOCR", trust_remote_code=True, use_fast=True)
model = AutoModel.from_pretrained(
    "StarDoc-AI/TeleOCR",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).cuda().eval()

# 文本
image = Image.open("./assets/text.png").convert("RGB")
raw_text = infer(image, "Please output the text content from the image.")
print(raw_text.strip())

# 表格
image = Image.open("./assets/table.png").convert("RGB")
raw_otsl = infer(image, "This is the image of a table. Please output the table in OTSL format.")
print(convert_otsl_to_html(raw_otsl))

# 公式
image = Image.open("./assets/formula.png").convert("RGB")
raw_formula = infer(image, "Please write out the expression of the formula in the image using LaTeX format.")
formula_block = ContentBlock("equation", [0.0, 0.0, 1.0, 1.0], content=raw_formula)
formula = post_process([formula_block])[0].content
print(formula)

# 代码
image = Image.open("./assets/code.png").convert("RGB")
raw_code = infer(image, "The image contains a code snippet, please output the parsing result.")
print(raw_code.strip())

# 布局
image = Image.open("./assets/layout.jpg").convert("RGB")
image = image.resize((1036, 1036), Image.Resampling.BICUBIC)
raw_layout = infer(image, "Analyze the image layout.")
print(raw_layout.strip())

# 扭曲文档布局
layout_image = Image.open("./assets/layout_distorted.jpg").convert("RGB")
layout_image = layout_image.resize((1036, 1036), Image.Resampling.BICUBIC)
raw_layout = infer(layout_image, "\nMulti-point Layout Segmentation Analysis.")
print(raw_layout.strip())

# 科学图表
image = Image.open("./assets/scientific_figure.png").convert("RGB")
raw_scientific_figure = infer(image, "This is a scientific figure. Please extract the table implied by this figure.")
print(convert_otsl_to_html(raw_scientific_figure))

如需执行完整的文档解析，请参考我们的 GitHub 仓库：https://github.com/caipeng328/NaviDC-OCR。

https://huggingface.co/StarDoc-AI/TeleOCR#citation引用

@article{teleocr,
    title={TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents},
    author={Cai, Peng and Zou, Zhaofan and Liu, Shifa and Wang, Yikun and Tang, Jiawei and Yang, Kaicheng and Tong, Meng and He, Zhongjiang and Sun, Hao},
    journal={arXiv preprint arXiv:2608.12898},
    year={2026}
}

https://huggingface.co/StarDoc-AI/TeleOCR#community-contributions社区贡献

感谢 Nandraj 提供的 GGUF 转换和 llama.cpp 支持 🔗 NaviDC-OCR-GGUF

https://huggingface.co/StarDoc-AI/TeleOCR#acknowledgements致谢

TeleOCR 构建于以下项目：

• MinerU
• Qwen2.5-VL
• Qwen3
• Transformers
• PyTorch
• FlashAttention 我们衷心感谢这些优秀的开源项目。

https://huggingface.co/StarDoc-AI/TeleOCR#contact联系我们

如果您有任何问题，请随时提交 issue 或联系我们。