INAR-VL: Input-Aware Routing for Edge-Cloud Vision-Language Inference

arXiv cs.LG 05/20/26, 04:00 AM Papers

edge-computing vision-language-models inference-routing cloud-edge multimodal latency-energy

Summary

INAR-VL proposes a lightweight routing system for edge-cloud vision-language inference that dynamically selects between edge and cloud models based on query complexity, achieving significant latency and energy reductions while preserving near-cloud accuracy.

arXiv:2605.18853v1 Announce Type: new Abstract: Edge deployment of Vision-Language Models (VLMs) faces a tradeoff between latency and accuracy: cloud execution provides high-quality predictions but incurs communication delay and energy cost, while edge-only execution is faster but less accurate due to limited model capacity. This trade-off is further complicated by heterogeneity in image quality and reasoning complexity, making static placement suboptimal. We present INAR-VL, a lightweight edge-cloud routing system for multimodal inference in a two-tier deployment. INAR-VL maintains complementary VLMs across edge and cloud and uses lightweight image and text complexity signals to guide routing and model selection, executing simple queries locally while offloading complex ones when beneficial. Evaluation on visual question answering shows that INAR-VL executes 36% of requests on the edge, reduces latency by 24%, lowers energy by 26%, and preserves 97% of cloud-level accuracy.

Original Article

Similar Articles

VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrieval

arXiv cs.CL

A new paper proposes VDAR-Router, a difficulty-aware retrieval-based routing framework for LLMs that adaptively selects models based on query difficulty, achieving better cost-performance trade-offs.

VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies

Hugging Face Daily Papers

VisualThink-VLA introduces a visual intermediate reasoning framework for vision-language-action policies that preserves spatial precision and dramatically reduces latency compared to text-based reasoning, achieving sub-second inference and state-of-the-art success rates on robot manipulation benchmarks.

Routing Without Training: Controllable-Ratio LLM Offloading via Reliability Gating

arXiv cs.AI

This paper introduces CARGO, a training-free routing framework that uses the local LLM's own inference-time agreement across sampled responses to decide when to offload to a cloud model, enabling controllable collaboration ratios without additional training.

Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models

Hugging Face Daily Papers

Proposes Reroute, a training-free plug-in for vision-language models that replaces irreversible visual-token pruning with recoverable routing, allowing tokens to re-enter the pipeline later to improve grounding under aggressive token reduction while maintaining VQA performance.

@songhan_mit: explore VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference

X AI KOLs Following

MIT researchers developed VLASH, a method enabling vision-language-action models to predict future robot states, doubling speed and reducing lag in tasks like pick-and-place and table tennis.