MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities
Summary
Modus is a decoder-only model that predicts any modality from any combination of others, achieving strong performance across diverse benchmarks without modality-specific heads or losses.
View Cached Full Text
Cached at: 07/29/26, 11:52 AM
Paper page - MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities
Source: https://huggingface.co/papers/2607.25948 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Any-to-anymodelspredictanymodalityfromanycombinationofotherswithinasinglenetwork,aformulationusedinmultimodalvisionandvision-languagemodels,andincreasinglyinscientificdomainssuchasecologyandastronomy.Existingany-to-anymodelsaretypicallytrainedfromscratchusingencoder-decoderordiffusionarchitectures,impactingtheirperformanceandpreventingthemfromusingstrongpre-traineddecoder-onlymodelsasaprior.Inthiswork,weinvestigatedecoder-onlyany-to-anymultimodalmodeling,whichtreatsallmodalitiessymmetricallyandsupportsarbitrarymodalitiesasinputsandoutputswithoutmodality-specificheads,losses,ortaskpipelines.Becauseeverymodalityisbothaninputandanoutputofthesamemodel,theresultingmodel,namedModus,cansupportarangeofapplications,suchaschainedgenerationthroughintermediatemodalitiesorcross-modalself-verificationbyscoringthemodel’sownoutputswithanothergeneratedmodality.Modusdemonstratesstrongout-of-the-boxperformanceandiscompetitivewithspecialistandmultitaskbaselinesusingasinglemodelacrossvariousbenchmarks.Allmaterialsareopen-sourcedathttps://modus-multimodal.epfl.ch/.
View arXiv pageView PDFProject pageGitHub16Add to collection
Get this paper in your agent:
hf papers read 2607\.25948
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.25948 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.25948 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.25948 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
This paper introduces MODE, a modality-decomposed expert-level mixed-precision quantization framework for MoE multimodal LLMs that addresses biases in expert importance estimation by decomposing selection frequency by modality and filtering redundant vision tokens, achieving minimal performance loss under aggressive quantization.
MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference
MACS is a training-free inference framework that mitigates the straggler effect in expert parallelism for multimodal MoE MLLMs by introducing entropy-weighted load and dynamic modality-adaptive capacity mechanisms.
From Modalities to Propositions: A Language-Centric Framework for Multimodal Intelligence
This paper proposes representing multimodal data (image, video, text) as bags of atomic propositions (e.g., 'person holding cup'), unified via a global semantic codebook, enabling interpretable, compositional, and cross-modal understanding. The framework is demonstrated on autonomous driving and open-world data.
Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language Models
This paper investigates modality preference in omni-modal large language models (OLLMs), revealing a paradigm shift from text-dominance to visual preference. The authors introduce a conflict-based benchmark and layer-wise probing to diagnose cross-modal hallucinations using internal model signals.
AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling
This paper introduces AnyMo, a unified multimodal framework for human motion generation that combines a Residual FSQ-based motion tokenizer with a scalable masked modeling transformer, along with the OmniHuMo dataset of over 5,000 hours of motion data to enable high-quality synthesis under arbitrary modality combinations.