TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training
Summary
TorchUMM is a unified codebase for evaluating and analyzing multimodal models across understanding, generation, and editing tasks, with standardized protocols and diverse datasets, enabling fair comparisons and deeper insights into model strengths and limitations.
View Cached Full Text
Cached at: 07/21/26, 06:47 AM
Paper page - TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training
Source: https://huggingface.co/papers/2604.10784
Abstract
TorchUMM presents a unified codebase for evaluating and analyzing multimodal models across understanding, generation, and editing tasks with standardized protocols and diverse datasets.
Recent advances inunified multimodal models(UMMs) have led to a proliferation of architectures capable of understanding, generating, and editing across visual and textual modalities. However, developing a unified framework for UMMs remains challenging due to the diversity of model architectures and the heterogeneity of training paradigms and implementation details. In this paper, we present TorchUMM, the first unified codebase for comprehensive evaluation, analysis, and post-training across diverse UMM backbones, tasks, and datasets. TorchUMM supports a broad spectrum of models covering a wide range of scales and design paradigms. Ourbenchmarkencompasses three core task dimensions:multimodal understanding, generation, and editing, and integrates both established and novel datasets to evaluate perception, reasoning, compositionality, and instruction-following abilities. By providing a unified interface andstandardized evaluation protocols, TorchUMM enables fair and reproducible comparisons across heterogeneous models and fosters deeper insights into their strengths and limitations, facilitating the development of more capable unified multimodal systems. Code is available at: https://github.com/AIFrontierLab/TorchUMM.
View arXiv pageView PDFProject pageGitHub279Add to collection
Get this paper in your agent:
hf papers read 2604\.10784
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2604.10784 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2604.10784 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2604.10784 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation
PixelUMM is an encoder-free unified multimodal model that operates directly in pixel space, representing images as spatial patches and videos as spatiotemporal tubelets fed to a shared backbone via single-layer linear projections. Its Mixture-of-Transformers architecture supports both autoregressive text prediction and pixel-space flow matching for image and video understanding and generation, achieving competitive performance while providing design insights for future pixel-space UMMs.
UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervision
UniCorn is a framework that enables unified multimodal models to self-improve by using a multi-agent system for prompt generation, image creation, and quality evaluation, achieving state-of-the-art results on text-to-image benchmarks like TIIF, WISE, and OneIG-EN.
FlexUMM: Flexible Representation Learning for Unified Multimodal Models
FlexUMM提出了一种灵活的统一多模态模型框架，动态决定何时保留共享计算、何时路由专门化容量、何时提前停止处理，从而在提升理解与生成任务准确性的同时提高计算效率，可作为即插即用模块增强现有UMM。
LatentUMM: Dual Latent Alignment for Unified Multimodal Models
LatentUMM introduces dual latent alignment to improve cross-modal consistency in unified multimodal models by aligning transformations and stabilizing latent dynamics.
Steering Visual Generation in Unified Multimodal Models with Understanding Supervision
This paper introduces UNO, an Understanding-Oriented Post-Training framework that uses comprehension tasks as supervisory signals to enhance image generation and editing in unified multimodal models.