multi-modal

Black Forest Lab's Flux 3 is a new omni-modal AI model capable of generating and predicting images, video, audio, and actions.

BFL has introduced FLUX 3, a multi-modal AI model capable of generating images, videos, and audio.

Fusion Embedding introduces a family of models that add audio to a frozen vision-language embedding backbone, enabling a unified space for text, image, video, and audio retrieval. The models train only lightweight adapters and achieve audio-image retrieval without paired audio-visual data.

MindLab Research released Macaron-V1-Venti, a new multi-modal AI model available on HuggingFace, likely for text-to-image or image generation tasks.

This paper proposes UMMT, a token-level cross-modal transformer with contrastive multi-task learning for breast cancer subtype classification and survival prediction, achieving state-of-the-art results on METABRIC and TCGA-BRCA datasets.

RynnBrain 1.1 is a family of embodied foundation models (2B, 9B, 122B-A10B) that improve perception, spatial reasoning, and manipulation, achieving state-of-the-art results on VSI-Bench, MMSI, and RefSpatial-Bench, and outperforming baselines in real-robot experiments.

Thinking Machines releases Inkling, an open-source multi-modal reasoning model with innovations in post-training RL, achieving stable scaling to 30M+ rollouts and controllable thinking effort. The model exhibits compressed reasoning and will be available soon for fine-tuning.

Hallo4D is a model-agnostic framework that leverages large multimodal language models to detect and correct spatial and temporal hallucinations in 3D and 4D generation, improving consistency across viewpoints and time without requiring retraining.

TheBioCollection is a 52.6B-token pre-training corpus for biology that consolidates heterogeneous resources into a unified format for training large language models, and includes a matched evaluation suite. Training on this corpus more than doubles overall scores on biology benchmarks while preserving general language ability.

SenseTime released SenseNova-Vision, a 7B parameter model that unifies computer vision tasks as generation, with open weights, instruction corpus, benchmark, paper, and demo.

BRAID is a framework that formulates interleaved text-image-text reasoning as a unified Markov decision process, enabling joint optimization of textual and visual generation via reinforcement learning with a VLM judge providing dense turn-level feedback.

This paper proposes a proof-of-concept AI pipeline that uses multi-modal data (images and accident reports) to assess railway crossing safety, achieving a macro F1 score of 0.757 for risk classification and an RMSE of 0.078 for safety score estimation using a fine-tuned compact VLM.

NVIDIA releases Cosmos3-Edge, an omnimodal world foundation model that generates video, image, audio, and action commands from text, image, video, and action trajectory inputs, targeting Physical AI applications in robotics, autonomous driving, and smart spaces.

This paper introduces HealthAgentBench, a suite of 54 realistic healthcare tasks for evaluating frontier AI agents. It finds that even the best agent (Codex GPT-5.5) achieves only ~42% success, highlighting substantial room for improvement.

This paper introduces MMG-Pop, a unified benchmark for multi-modal graph-based social media popularity prediction, and proposes MMG-PopNet, a model that jointly models multimodal content and temporal social interactions. Experiments on Bluesky and Reddit datasets demonstrate superior performance and provide insights into cross-platform generalization and multi-task prediction.

This paper systematically compares seven state-of-the-art sensor fusion methods for multi-modal human activity recognition on the HARMES dataset, showing that Gated Multi-modal Fusion achieves the highest macro F1-score of 0.82, outperforming the baseline by 6 percentage points.

An open-source alternative to Google's NotebookLM offers free unlimited access, runs locally under MIT license, and includes features like podcast generation and multi-modal ingestion.

Microsoft open-sourced JARVIS, a system that uses a GPT controller to orchestrate hundreds of AI models from HuggingFace for multi-modal tasks.

OmniPath is a multi-modal agentic framework that combines OpenStreetMap network topology with aerial LiDAR data to audit wheelchair accessibility by analyzing physical barriers like slope and surface discontinuities at high resolution, validated against field surveys.

IV-CoT decomposes visual conditioning into structural and semantic cascades for improved structure-aware image generation, using training-only sketch supervision to guide structural queries. It achieves state-of-the-art results on GenEval and T2I-CompBench.