multimodal

Presents comparison results between Fable 5.5 and GPT 6.2 Astra when generating an SVG of a PS5 controller, routed via stealth routing.

A pull request to llama.cpp adds Nemotron 3.5 Transcribe support in the mtmd (multimodal) module, extending the library's audio transcription capabilities.

OpenCode has launched a mysterious stealth model, Pulse Alpha, offering 1M context, 131K max output, multimodal input, and three reasoning tiers — free for a limited time. Speculation points to GLM 5.5 or the Qwen series under the hood.

Reviso is an agentic video captioning framework that uses hierarchical Context-Focus-Points planning and active perception to prioritize key visual evidence, trained with GRPO and specialized rewards, outperforming open-source baselines on four video captioning benchmarks.

Moonshot AI announced Kimi K3.1 Pro, its most capable 3T-class open-weight MoE model with 1M-token context, and Modal will offer day 0 support for it this week.

Hume AI announced the Hume Audio Decision API, an audio-native model that answers questions about tone, emotion, language, and speaker count directly from audio without needing a transcript, trained on 10M+ human judgments and 100M hours of audio.

LongCat-2.5 is introduced as a natively multimodal model designed for agentic tasks, combining vision-language capabilities with tool-use and long-horizon task performance.

Fovea-4B is an open-weight 4B vision model that brings Jev-style typed decision contracts to images, leading ImageJev-style tasks with 80.6 chance-corrected accuracy and strong calibration (ECE 0.035), released under Apache-2.0.

Apriel Voice 27B Preview 是一个开源语音-语言模型，基于 Gemma 4 26B-A4B 后训练检查点构建，通过两阶段训练将语音理解、推理与工具调用统一到共享推理骨干中，并在 Big Bench Audio、Speech-BFCL 和 τ-Voice 等基准上取得有竞争力的成绩。

The paper proposes Role-Aware Server Optimization (RASO), a method that coordinates contributor selection with role-specific server momentum updates to improve multimodal federated learning when some modalities are missing, demonstrating effectiveness across datasets and fusion architectures.

DR Pixel is an open-source end-to-end multimodal deep research agent that retrieves and reasons directly over rendered document page images instead of OCR-extracted text, introducing counterfactual credit reweighting for agentic supervised finetuning. The approach outperforms OCR-based baselines on DRPixelBench across models like GPT-5.4 and Gemini-3.5-Flash, boosting retrieval recall by ~15 points and accuracy by 10+ points after training.

A user asks whether Gemma 4 E4B remains the best small multimodal model (under 8B/6GB) for running a local, tool-calling personal assistant on a Mac via MLX, noting failures with Ling 3.0 tiny, LFM models, and Qwen 3.5 4B/9B.

Ava is an in-browser, fully local AI avatar powered by the LFM3-Omni model running via WebGPU, available as a Hugging Face Space demo.

Nace.AI released Drex Max v1, an open-weight 27B multimodal decision model with a 255-way readout head and 262K-token context that returns probability distributions over options in a single forward pass, scoring 67.52 on the public Decision Index 0.4 and running locally via llama.cpp, Ollama, and MLX on Apple Silicon.

A Nature paper introduces EVLF-FM, an 8B-parameter clinical multimodal vision-language foundation model with fine-grain explainability that outperforms clinicians in radiology, dermatology and other medical tasks, showing frontier-level medical intelligence can run without data-center scale compute.

Astra reportedly outperforms all other models on robot intelligence benchmarks, marking a notable advance in embodied AI capabilities.

A paper introduces A-RSI (Adaptive Task Refresh for Self-Improvement), a method enabling zero-data multimodal self-improvement by adaptively refreshing tasks during training without requiring new human-labeled data.

SenseNova releases SenseNova-Loop-9B-Preview, a looped multimodal model built on a shared looped block and mHC residual stream architecture, targeting long-context and agentic workflows with capabilities spanning motion design, PPT generation, and data analysis.

inclusionAI 发布了 ZwZ2 系列（27B/9B/4B），基于 Qwen3.6 微调的视觉语言模型，通过 EvidenceConnect 构建的多区域验证数据与 GRPO 训练，在六大领域 25 个基准上均超越基座模型，总体准确率从 55.52 提升至 60.51（+4.99）。

LFM3-1.7B and LFM3-VL-2B now run fully offline on iPhone and Android via the open-source Private Mind app, with detailed notes on what it took to export LFM3's novel architecture (gated convolutions, hybrid attention, sliding-window cache tweaks) to ExecuTorch, achieving ~40 tok/s on an iPhone 16 Pro GPU.