vision-language

Tag

Cards List
#vision-language

GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning

Hugging Face Daily Papers ↗ · 2026-10-13 Cached

提出 ScreenRef 高质量 GUI grounding 数据集(586K 截图、633K 指代表达实例)和 Distillation-to-Reinforcement Learning(D2RL)训练方法,训练出 GroundGUI-8B,在 ScreenSpot-V2 等多个基准上取得 8B 级模型中的领先性能。

0 favorites 0 likes
#vision-language

nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4

Hugging Face Models Trending ↗ · 2026-10-12 Cached

NVIDIA released NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4, an NVFP4-quantized vision-language model with 120B total/12B active parameters, Mamba-2 hybrid MoE architecture, a C-RADIO vision encoder, up to 1M-token context, and native multi-token prediction for speculative decoding, licensed under OpenMDW-1.1 for commercial and non-commercial use.

0 favorites 0 likes
#vision-language

Unsupervised Multimodal Reward Modeling

arXiv cs.CL ↗ · 2026-10-12 Cached

提出一种无需人工偏好标注的多模态奖励建模方法:在冻结的 Qwen3-VL 骨干网络的中间层上训练线性探针,通过扰动字幕对比进行训练,在多个基准上达到或超过 GPT-4o,且微调计算量仅为千分之一。

0 favorites 0 likes
#vision-language

Representation Alignment from a Few Paired Data via Distilled Optimal Transport

arXiv cs.AI ↗ · 2026-10-12 Cached

提出一种通过蒸馏最优传输在仅有100-500个配对样本下对齐两个独立预训练单模态编码器的方法,在Flickr30k、MS-COCO和ChEBI-20上以相同锚点预算超越现有方法。

0 favorites 0 likes
#vision-language

VEDJE: Video-Efficient Discriminative Joint Encoder for Scalable Video-Text Retrieval

Hugging Face Daily Papers ↗ · 2026-10-12 Cached

VEDJE is a video-efficient discriminative joint encoder for scalable video-text retrieval that compresses each video into a tiny 48 KiB temporally structured cache, using a lightweight 33M-parameter joint encoder to achieve 59.8 R@1 on MSR-VTT with ~49× fewer online parameters than LamRA.

0 favorites 0 likes
#vision-language

microsoft/AesCode 8B and 32B

Reddit r/LocalLLaMA ↗ · 2026-10-11

微软发布 AesCode-8B 与 AesCode-32B 模型,可依据图像美学参考生成信息丰富的可视化产物(如幻灯片、海报、仪表盘)为结构化 HTML/CSS 代码,通过图结构监督与解耦的跨模态奖励解决视觉布局与内容准确性的权衡。

0 favorites 0 likes
#vision-language

lightonai/LightOnOCR-3-4B · Hugging Face

Reddit r/LocalLLaMA ↗ · 2026-10-09 Cached

LightOn 发布新一代轻量级端到端 OCR 模型 LightOnOCR-3 系列,其中 4B 版本采用 Qwen3.5 视觉-语言架构,新增 grounding(带边界框的布局标签)、图像描述和图表数据提取等视觉理解能力,全部以 Apache 2.0 许可开源。

0 favorites 0 likes
#vision-language

Jag: Direct Box Prediction for Efficient Visual Grounding

Hugging Face Daily Papers ↗ · 2026-10-09 Cached

介绍 Jag,一个 0.8B 视觉定位模型,通过单次前向传播直接预测边界框而非自回归生成坐标,在保持与 LocateAnything 相当精度的同时,速度提升 3.62 倍、峰值 GPU 显存降低 73.88%,并开源了模型权重、代码与在线演示。

0 favorites 0 likes
#vision-language

Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffolding

arXiv cs.AI ↗ · 2026-10-08 Cached

The paper introduces Rationale-Guided Policy Optimization (RGPO), a framework that adaptively uses ground-truth rationales as temporary scaffolds during on-policy reinforcement learning to mitigate reward sparsity, achieving consistent gains over RLVR baselines in both language-only and vision-language reasoning settings.

0 favorites 0 likes
#vision-language

UltraText Bench: A Comprehensive Bilingual Benchmark for Evaluating Visual Text Rendering in Image Generation

Hugging Face Daily Papers ↗ · 2026-10-07 Cached

UltraText Bench is a bilingual (English/Chinese) benchmark with 432 prompts across 24 scene categories for evaluating dense visual text rendering in image generation models, scored by the Q-Judge VLM across text fidelity, clarity, spatial quality, and scene quality — revealing strong trade-offs between model variants such as Z-Image-Turbo versus Z-Image-Base.

0 favorites 0 likes
#vision-language

@maximelabonne: Ok it's crazy what you can do with text+vision > Adding vision can improve performance for text-based tasks > OCR is tr…

X AI KOLs Timeline ↗ · 2026-10-05 Cached

Liquid AI announced d1 with vision, making its first decision model capable of processing images, text, or both, with benchmarks showing it matches or beats frontier models like GPT-6.1 Sol and Claude Opo 5.5 on real-world applications while staying fast and cheap.

0 favorites 0 likes
#vision-language

LiquidAI/d1-3B

Hugging Face Models Trending ↗ · 2026-10-05 Cached

LiquidAI released d1-3B, an open 3B-parameter multimodal decision model built on LFM2.5-VL-3B that answers typed questions over text/JSON/image states in a single forward pass with zero output tokens, achieving 8ms decisions on consumer GPUs.

0 favorites 0 likes
#vision-language

@HuggingPapers: UniEvo-VL A self-evolving framework where a single multimodal model acts as both teacher and student, learning from its…

X AI KOLs Following ↗ · 2026-10-01 Cached

UniEvo-VL 是一个自进化框架,让单个多模态模型同时充当教师和学生,通过自身的建设性反馈在无外部监督的情况下提升图像生成能力。

0 favorites 0 likes
#vision-language

Recovering Off-Policy Supervision for Speculative Decoding

arXiv cs.CL ↗ · 2026-10-01 Cached

This paper proposes a rollout-based training framework (Anchor-Label Relabelling and In-Rollout Anchors) to recover full supervision for speculative decoding block drafters trained on off-policy corpora, boosting greedy accepted length by up to 36.5% over DFlash without modifying the training text.

0 favorites 0 likes
#vision-language

Octrees as an Explicit 3D Language

Hugging Face Daily Papers ↗ · 2026-10-01 Cached

OctLLM represents 3D geometry as explicit SparseOctree occupancy tokens and routes mesh tokens through independent trainable branches alongside a frozen vision-language backbone, setting a new state of the art among unified multimodal LLMs for image-to-3D and render-grounded captioning while preserving general language ability.

0 favorites 0 likes
#vision-language

Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces

Hugging Face Daily Papers ↗ · 2026-09-30 Cached

Multimodal Flow presents a fully continuous generative architecture for language and vision using a shared chunk-causal flow backbone over continuous embedding hyperchunks, with MF-1 pretrained at 0.6B to 1.6B scales that competitively performs on GenEval, DPG-Bench, VQAv2, MMBench, and POPE using only 150B tokens.

0 favorites 0 likes
#vision-language

Align Then Reason: A Multimodal Lip-Sync Judge for Dubbing

Hugging Face Daily Papers ↗ · 2026-09-30 Cached

The paper introduces Align Then Reason (ATR), a multilingual, reference-free lip-sync judge that first aligns frame-level lip representations with phonetic units of a candidate line, then uses an LLM to reason over local alignment evidence and a global score for dubbing quality control, delivering large AUC gains over Qwen3.5, LLaMA-3.1, and Mistral SFT baselines across seven languages and two downstream dubbing tasks.

0 favorites 0 likes
#vision-language

Beyond the Current Scene: Event-Referential Grasping with Active View Selection

Hugging Face Daily Papers ↗ · 2026-09-30 Cached

The paper introduces BeyondSCe, a zero-shot robotic grasping system that identifies objects referenced by past human-object interactions rather than names or appearance, and actively selects camera viewpoints to reveal occluded targets, achieving 76-77% grasp success rates that significantly outperform baselines.

0 favorites 0 likes
#vision-language

@svpino: This release is pretty cool for anyone who cares about robotics: • It comes with a vision-language planner • This plann…

X AI KOLs Timeline ↗ · 2026-09-29 Cached

Dyna Robotics releases Dyna-2.1, a physical agent with a vision-language planner and RL controller for long-horizon autonomous tasks, demonstrated in an hour-long laundry workflow.

0 favorites 0 likes
#vision-language

Evaluating Explanation-Driven Vision-Language Reasoning via Generation Order Interventions

arXiv cs.CL ↗ · 2026-09-25 Cached

This paper evaluates the causal link between explanations and model predictions in vision-language reasoning through generation order interventions, finding that larger models are required for rationale-first reasoning and that answer-first generation reduces format-related errors.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback