unified-multimodal-model

Tag

Cards List
#unified-multimodal-model

HiUni: Harmonizing Understanding and Generation in Shared Visual Space

Hugging Face Daily Papers ↗ · 2d ago Cached

HiUni explores extending a pretrained vision-language model into a unified multimodal model via extensive parameter sharing, reusing a single dense backbone for both understanding and generation with a PixelDiffusionHead and HiGate pathway to address optimization and spatial-detail challenges. Experiments show it matches the pretrained VLM's understanding score while achieving competitive image generation and editing performance.

0 favorites 0 likes
#unified-multimodal-model

PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation

Hugging Face Daily Papers ↗ · 2026-09-29 Cached

PixelUMM is an encoder-free unified multimodal model that operates directly in pixel space, representing images as spatial patches and videos as spatiotemporal tubelets fed to a shared backbone via single-layer linear projections. Its Mixture-of-Transformers architecture supports both autoregressive text prediction and pixel-space flow matching for image and video understanding and generation, achieving competitive performance while providing design insights for future pixel-space UMMs.

0 favorites 0 likes
← Back to home

Submit Feedback