multi-modal

The paper turns the Platonic Representation Hypothesis into a theorem, proving that JEPAs trained separately on multiple modalities converge to shared representations up to a single rotation, and shows that next-token prediction is essentially a JEPA with 'bad views' — making it far less efficient for cross-modal alignment (good views need 42x fewer samples).

The paper introduces PRISM, a self-adjusting framework that adapts sparse MoE routing for region-specific multi-view fusion and uses a PID-inspired regulation mechanism to mitigate gradient conflicts in urban region representation learning, outperforming strong baselines on three large-scale urban datasets.

The paper introduces MOSAIC, an omni-modal benchmark of 923 questions over persona-grounded multi-session histories spanning text, image, video, and audio, designed to diagnose where conversational agents lose evidence across write, retrieve, deliver, and read stages rather than just producing a single leaderboard score.

论文提出 Design Anything，一个基于图的 harness，通过 intent-execution graph 在长程任务中保持设计意图，并发布涵盖19个设计领域、228项任务的 DesignAnything-Bench，显著提升了约束满足率。

IBM 推出 Granite Switch 4.1（8B 与 30B），通过 aLoRA 适配器在同一端点上组合聊天、决策与语音转写能力，几乎不额外占用 VRAM。新增 Decision 与 Guardian aLoRA，并嵌入 OpenASR 榜单最快的 Granite-Speech-5.0-Turbo-CTC 模型。

MEA is a reward-driven multi-agent framework where a Proposer agent selects explanation tools and an Actor agent is trained end-to-end against perturbation-based faithfulness rewards, delivering faithful natural-language explanations across tabular, text, and vision modalities and outperforming existing explainers by up to 34%.

New model Wity-1, a reasoning decision model positioned between non-reasoning decision models like Jev and fully reasoning LLMs, reportedly beats Jev on all 4 baselines and ranks #1 on the Image Jev Bench, and is multi-modal. It reasons just enough until internal LLM probabilities indicate the answer is reached.

A business consultant fine-tuned a 4B parameter AI model called ImaJev-4b for making decisions from text and images, achieving top performance on JevBench and DecisionBench benchmarks.

The paper proposes Future-Aware Recall (FAR), a framework that learns episodic memory recall for world models via future-aware predictive supervision and adaptive multi-cue scoring, automatically determining which retrieval cues (time, pose, vision, audio) to trust.

The Intern-Decision multi-modal model family (0.8B, 2B, 4B) is released, with the 4B model outperforming Jev 1.13.0 in decision-making tasks, featuring fast local deployment and improved calibration on probability problems.

PhysioBench introduces a unified benchmark for physiological signal question answering, harmonizing 22 datasets into 61.4 million questions across 30 tasks to evaluate the performance of various AI models.

The paper introduces Ovis-Embedding, a state-of-the-art omni-modal embedding model that uses a shared backbone to encode text, image, video, and audio in a common representation space, achieving top performance on benchmarks like MMEB-v3 and MVEB.

onPanda is an interactive tool for token-level visualization and control of LLMs and agents, featuring data annotation, model inspection, and support for multiple modalities including browser-based execution.

UFO is a unified framework for simultaneous evaluation of omni-condition alignment in multi-modal image generation. It introduces an Atomized Chain-of-Evaluation paradigm and UFO-Bench benchmark.

openjev is a model based on Qwen3.5 trained for entailment tasks, enabling applications in reranking, grading, and real-time game playing, with the v2 version adding multi-modal capabilities and improved zero-shot performance.

This paper evaluates MiniMax-H3, an omni-modal generative model, by introducing a comprehensive framework to assess its reasoning about the physical world through multimodal inputs. The evaluation reveals that video-based decision reasoning performs best, while audio-based disambiguation reasoning is the weakest.

ElevenLabs has added image and video generation to its MCP platform, expanding beyond audio to support multi-modal content creation.

The paper proposes the Convergent Emergence Hypothesis, stating that few-shot in-context learning emerges with a common cross-modality difficulty profile, and provides empirical support through experiments on six modalities, showing correlated effects in five.

This paper presents Gander, an end-to-end framework for omni interaction and agentic tasks, enabling real-time full-duplex interaction across multiple modalities with a Cerebellum-Brain architecture.

SimpleDesign introduces a joint model for protein sequence and structure codesign, trained end-to-end in data space using a single-stage objective, achieving competitive performance on co-design and generation benchmarks.