Tag
HiUni explores extending a pretrained vision-language model into a unified multimodal model via extensive parameter sharing, reusing a single dense backbone for both understanding and generation with a PixelDiffusionHead and HiGate pathway to address optimization and spatial-detail challenges. Experiments show it matches the pretrained VLM's understanding score while achieving competitive image generation and editing performance.
PixelUMM is an encoder-free unified multimodal model that operates directly in pixel space, representing images as spatial patches and videos as spatiotemporal tubelets fed to a shared backbone via single-layer linear projections. Its Mixture-of-Transformers architecture supports both autoregressive text prediction and pixel-space flow matching for image and video understanding and generation, achieving competitive performance while providing design insights for future pixel-space UMMs.