INAR-VL: Input-Aware Routing for Edge-Cloud Vision-Language Inference

arXiv cs.LG 05/20/26, 04:00 AM Papers

edge-computing vision-language-models inference-routing cloud-edge multimodal latency-energy

Summary

INAR-VL proposes a lightweight routing system for edge-cloud vision-language inference that dynamically selects between edge and cloud models based on query complexity, achieving significant latency and energy reductions while preserving near-cloud accuracy.

arXiv:2605.18853v1 Announce Type: new Abstract: Edge deployment of Vision-Language Models (VLMs) faces a tradeoff between latency and accuracy: cloud execution provides high-quality predictions but incurs communication delay and energy cost, while edge-only execution is faster but less accurate due to limited model capacity. This trade-off is further complicated by heterogeneity in image quality and reasoning complexity, making static placement suboptimal. We present INAR-VL, a lightweight edge-cloud routing system for multimodal inference in a two-tier deployment. INAR-VL maintains complementary VLMs across edge and cloud and uses lightweight image and text complexity signals to guide routing and model selection, executing simple queries locally while offloading complex ones when beneficial. Evaluation on visual question answering shows that INAR-VL executes 36% of requests on the edge, reduces latency by 24%, lowers energy by 26%, and preserves 97% of cloud-level accuracy.

Original Article

Similar Articles

Pro-Router: Token-Aware Progressive Model Routing with Adaptive Edge-Cloud Collaboration for Efficient Multimodal LLM Inference

arXiv cs.AI

Pro-Router introduces a token-aware progressive model routing method for efficient multimodal LLM inference, leveraging adaptive edge-cloud collaboration to improve throughput and reduce costs.

VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrieval

arXiv cs.CL

A new paper proposes VDAR-Router, a difficulty-aware retrieval-based routing framework for LLMs that adaptively selects models based on query difficulty, achieving better cost-performance trade-offs.

LayerRoute: Action-Conditioned Mixture-of-Layers Routing for Vision-Language-Action Policies

arXiv cs.AI

LayerRoute introduces an action-conditioned routing interface for Vision-Language-Action policies that dynamically adapts access to VLM layer representations, improving robot manipulation performance with minimal additional parameters.

Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems

arXiv cs.AI

This paper introduces a carbon-aware routing framework for function-calling LLMs in edge-cloud systems, reducing operational carbon emissions by an average of 4× while maintaining cloud-level accuracy.

VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies

Hugging Face Daily Papers

VisualThink-VLA introduces a visual intermediate reasoning framework for vision-language-action policies that preserves spatial precision and dramatically reduces latency compared to text-based reasoning, achieving sub-second inference and state-of-the-art success rates on robot manipulation benchmarks.

Similar Articles

Pro-Router: Token-Aware Progressive Model Routing with Adaptive Edge-Cloud Collaboration for Efficient Multimodal LLM Inference

VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrieval

LayerRoute: Action-Conditioned Mixture-of-Layers Routing for Vision-Language-Action Policies

Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems

VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies

Submit Feedback