omni-model

Xenova 宣布突破：首个 100% 本地在浏览器中运行的真正全模态模型 LFM3-Omni-2B，借助自定义 WebGPU kernels 实现约 280 tok/s，支持视觉、音频和文本输入并生成交错的文本与音频输出。

Xiaomi has released two state-of-the-art AI models, MiMo-V2.6 Pro RL for maximum capability and MiMo-V2.6 Flash RL for maximum efficiency, both featuring sparse mixture-of-experts architecture, 1M context length, MIT licensing, and native omni-modal support for text, image, video, and audio.

This paper introduces OmniVChat, a task for native audio-visual dialogue, and presents a data engine, benchmark, and reinforcement learning method to train and evaluate omni models, demonstrating improvements on synthesized and human-recorded data.

Atlas by World Labs is an omni world model that generates camera-controlled HD video from text, images, video, and 3D inputs, reconstructs scenes, and simulates space-time for robotics, available in early access.

NVIDIA Cosmos 3 is an open omni-model for physical AI that unifies world generation, reasoning, and action generation into a single model, available on Hugging Face with various resources.

MiniMind-O has released an end-to-end omnimodal model with only 0.1B parameters, supporting text, speech, and image inputs as well as streaming speech output. The project opensources the code, weights, training data, and technical report, emphasizing that both training and inference can be performed quickly on standard GPUs.