Multi-Timescale Latent-Action DRL for Joint Optimization in Edge-Cloud Networks
Summary
Proposes a two-timescale multi-layer deep reinforcement learning framework with latent action space for joint service placement, computational delegation, and power control in hierarchical edge-cloud computing, achieving up to 20.8% latency reduction and 13% resource utilization improvement.
View Cached Full Text
Cached at: 07/22/26, 08:18 AM
# Multi-Timescale Latent-Action DRL for Joint Optimization in Edge-Cloud Networks Source: [https://arxiv.org/abs/2607.18288](https://arxiv.org/abs/2607.18288) [View PDF](https://arxiv.org/pdf/2607.18288) > Abstract:Load imbalance across edge and cloud layers degrades latency performance in hierarchical edge\-cloud computing \(HECC\) systems under dynamic task arrivals and heterogeneous resources, leading to severe queuing delays and inefficient resource utilization\. To address this challenge, we study a joint service placement, computational delegation, and power control \(JSCP\) problem to minimize the average end\-to\-end \(e2e\) latency\. The resulting JSCP problem is a mixed\-integer nonconvex and NP\-hard optimization problem due to the strong coupling between discrete and continuous variables\. To enable tractable optimization and stable system adaptation, we exploit the inherent difference in decision dynamics and decompose the problem into long\-term system configuration and short\-term resource allocation subproblems\. Based on this formulation, we propose a two\-timescale multi\-layer deep reinforcement learning framework with a latent action space \(2T\-MDRL\-LA\) to jointly optimize service placement, user association, computational delegation, task offloading, and user transmit power\. A latent action representation based on a variational autoencoder is introduced to efficiently compress the high\-dimensional combinatorial action space\. Simulation results demonstrate that the proposed framework effectively adapts to dynamic network conditions and achieves near\-optimal performance compared to branch\-and\-bound solutions\. It achieves up to a 20\.8% reduction in average e2e latency and a 13% improvement in resource utilization over the scheme without the computational delegation, while converging approximately 50% faster than conventional proximal policy optimization\. ## Submission history From: Van\-Dinh Nguyen Dr \[[view email](https://arxiv.org/show-email/61b7ace9/2607.18288)\] **\[v1\]**Tue, 30 Jun 2026 11:56:15 UTC \(2,739 KB\)
Similar Articles
Quantum Reinforcement Learning for Cost and Delay Tradeoffs in Quantum Cloud Orchestration
This paper proposes QRLQ, a quantum reinforcement learning framework that integrates parameterised quantum circuits with dueling double deep Q-networks to optimize cost and delay tradeoffs in quantum cloud orchestration, achieving lower costs and delays than heuristic baselines while using fewer parameters than classical DRL.
PPO-STGNN: A Proximal Policy Optimization Approach with Spatio-Temporal Graph Neural Networks for DAG Task Scheduling in Cloud-Edge-End Computing
This paper proposes PPO-STGNN, a reinforcement learning algorithm that combines proximal policy optimization with spatio-temporal graph neural networks to optimize DAG task scheduling in heterogeneous cloud-edge-end environments, improving load balancing and reducing completion time.
RL^2-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models
This paper introduces RL^2, an adaptive inference-time steering framework for Vision-Language-Action models that uses offline RL on latent representations to compose action flows, activating steering only when failure is predicted. It achieves up to +17.3% success rate improvements on SIMPLER and PolaRiS benchmarks and demonstrates real-world transfer.
DRG-MAPPO: Hierarchical Dynamic Role-Graph Multi-Agent Reinforcement Learning for Cooperative Air Combat
A hierarchical multi-agent reinforcement learning framework combining graph attention and dynamic role assignment improves tactical coordination and win rates in air combat.
Performance-Driven Environment Abstraction with Multi-Timescale Learning
This paper proposes a performance-driven state abstraction method for reinforcement learning that directly optimizes decision quality, using a multi-timescale framework to jointly adapt the policy and a tree-structured abstraction. The algorithm refines or aggregates state space based on Q-value discrepancies, achieving better sample efficiency and faster replanning than baselines.