On the Design Fundamentals of Pixel Text Representation Learning

Hugging Face Daily Papers Papers
visual-text-encoding multilingual-learning representation-learning pixel-linguist-ii contrastive-grounding compression-robustness

Summary

Pixel Linguist II improves visual text representation learning via variable resolution training, natural image-text grounding, layout-aware rendering, and multilingual curricula, achieving state-of-the-art results and robust compression performance.

Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders struggle with fixed resolution pretraining, visual shortcut learning, weak visual grounding, and multilingual visual text understanding. In this work, we investigate the fundamental design principles required for robust visual text representation learning. Through systematic controlled ablations, we identify four critical components: variable image resolutions and rendered font sizes provide spatial proxies for high-resolution document generalization; natural image-text pairs are indispensable for grounding and prevent text-only collapse; layout-aware rendering helps prevent pixel-level shortcuts; and a two-stage multilingual curriculum enables effective cross-lingual alignment. By integrating these principles into a scalable training recipe, we train Pixel Linguist II, a native-resolution vision encoder trained with on-the-fly rendering, unified contrastive grounding, and a multilingual curriculum over 280M training examples. Pixel Linguist II sets new state-of-the-art results on English, cross-lingual, and multilingual Visual STS and ViDoRe, while also enabling better MLLM downstream evaluation. Notably, Pixel Linguist II remains robust under 80\% visual token compression, showing great promise for optical context compression. Our code and resources are available at https://github.com/Pixel-Linguist/Pixel-Linguist-II.
Original Article
View Cached Full Text

Cached at: 09/03/26, 03:50 AM

Paper page - On the Design Fundamentals of Pixel Text Representation Learning

Source: https://huggingface.co/papers/2609.01147

Abstract

Pixel Linguist II improves visual text encoding through variable resolution training, natural image-text grounding, layout-aware rendering, and multilingual curricula, achieving state-of-the-art results and strong compression robustness.

Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders struggle with fixed resolution pretraining, visual shortcut learning, weak visual grounding, and multilingual visual text understanding. In this work, we investigate the fundamental design principles required for robustvisual text representation learning. Through systematic controlled ablations, we identify four critical components:variable image resolutionsand rendered font sizes provide spatial proxies for high-resolution document generalization; natural image-text pairs are indispensable for grounding and prevent text-only collapse;layout-aware renderinghelps prevent pixel-level shortcuts; and atwo-stage multilingual curriculumenables effective cross-lingual alignment. By integrating these principles into a scalable training recipe, we trainPixel Linguist II, a native-resolution vision encoder trained with on-the-fly rendering, unifiedcontrastive grounding, and a multilingual curriculum over 280M training examples.Pixel Linguist IIsets new state-of-the-art results on English, cross-lingual, and multilingual Visual STS and ViDoRe, while also enabling better MLLM downstream evaluation. Notably,Pixel Linguist IIremains robust under 80\%visual token compression, showing great promise for optical context compression. Our code and resources are available at https://github.com/Pixel-Linguist/Pixel-Linguist-II.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2609\.01147

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.01147 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.01147 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.01147 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

L2P: Unlocking Latent Potential for Pixel Generation

Hugging Face Daily Papers

The L2P paper introduces a Latent-to-Pixel transfer paradigm that leverages pre-trained latent diffusion models to create efficient pixel-space models capable of 4K generation with minimal training overhead.

TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment

Hugging Face Daily Papers

TIPSv2 introduces enhanced vision-language pretraining techniques including patch-level distillation, an upgraded masked image objective (iBOT++), and improved caption sampling strategies to achieve superior dense patch-text alignment. The resulting family of image-text encoder models demonstrates strong performance across 9 tasks and 20 datasets.

ViQ: Text-Aligned Visual Quantized Representations at Any Resolution

Hugging Face Daily Papers

ViQ presents a visual quantization framework that balances semantic richness and detail preservation in discrete representations, enabling efficient multimodal training with native-resolution inputs by using text-aligned pre-training and proximal representation learning.