UniSpark: Exploring Hybrid Attention for Multimodal Understanding and Generation
Summary
UniSpark-9B is a unified multimodal model combining full attention with Gated DeltaNet linear attention in a Mixture-of-Transformer-Experts backbone, jointly supporting visual question answering, text-to-image generation, instruction-guided editing, and interleaved text-image prediction, achieving 2470.3 on MME-S and 0.83 on GenEval.
View Cached Full Text
Cached at: 11/16/26, 08:56 AM
Paper page - UniSpark: Exploring Hybrid Attention for Multimodal Understanding and Generation
Source: https://huggingface.co/papers/2611.16517
Abstract
WepresentUniSpark-9B,aunifiedmultimodalmodelthatinvestigateshybridattentionforvisualunderstandingandgeneration.BuiltonaMixture-of-Transformer-Expertsbackbonewithapproximately9Bparametersineachbranch,UniSparkcombinesfullattentionforexplicittokeninteractionswithGatedDeltaNetlinearattentionforcompactcontextpropagation.Autoregressivetextpredictionandvisualflowmatchingjointlysupportvisualquestionanswering,text-to-imagegeneration,instruction-guidedediting,andinterleavedtext-imageprediction.Ininterleavedgeneration,noisyvisualtargetsareisolatedfromsubsequentpredictions,whilecleanvisualrepresentationsprovidecontextforlateroutputs.Ablockwiseprotocolenforcesthisdistinctionthroughattentionmasksinfull-attentionlayersandrecurrent-stateretentionrulesinlinear-attentionlayers.Theselectedtrainingsourcescontain159.7Mgenerationsamples,213.4Munderstandingsamples,and624.8Kinterleavedsequences.Afour-stagecurriculumjointlytrainsmultimodalunderstandingandvisualgeneration,refiningansweraccuracyandinstructionfollowingwhileextendingimagesynthesisto2K.UniSparkscores2470.3onMME-Sand0.83onGenEval.Qualitativeexamplesfurtherillustrateitscapabilitiesininterleavedimagegeneration,videosynthesis,andvideoediting.Together,theseresultsdemonstratethathybridattentionprovidesaneffectivebackboneforunifiedmultimodalunderstandingandgeneration.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2611\.16517
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2611.16517 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2611.16517 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2611.16517 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
HiUni: Harmonizing Understanding and Generation in Shared Visual Space
HiUni explores extending a pretrained vision-language model into a unified multimodal model via extensive parameter sharing, reusing a single dense backbone for both understanding and generation with a PixelDiffusionHead and HiGate pathway to address optimization and spatial-detail challenges. Experiments show it matches the pretrained VLM's understanding score while achieving competitive image generation and editing performance.
Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation
The paper introduces JoyAI-Image, a unified multimodal foundation model that integrates a spatially enhanced MLLM with MMDiT to achieve state-of-the-art performance in visual understanding, text-to-image generation, and instruction-guided editing.
Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification
UniAR presents a unified autoregressive framework that uses a single discrete visual tokenizer to bridge visual understanding and generation, achieving state-of-the-art results in image generation and editing.
Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System
This paper explores the synergy between visual understanding and generation in unified multimodal models, showing that task-decoupled architectures and end-to-end optimization can enhance performance by turning coexistence into synergy.
Steering Visual Generation in Unified Multimodal Models with Understanding Supervision
This paper introduces UNO, an Understanding-Oriented Post-Training framework that uses comprehension tasks as supervisory signals to enhance image generation and editing in unified multimodal models.