Multi-Timescale Latent-Action DRL for Joint Optimization in Edge-Cloud Networks

arXiv cs.LG Papers
deep-reinforcement-learning edge-cloud-networks joint-optimization latency-minimization resource-allocation combinatorial-action-space variational-autoencoder

Summary

Proposes a two-timescale multi-layer deep reinforcement learning framework with latent action space for joint service placement, computational delegation, and power control in hierarchical edge-cloud computing, achieving up to 20.8% latency reduction and 13% resource utilization improvement.

arXiv:2607.18288v1 Announce Type: new Abstract: Load imbalance across edge and cloud layers degrades latency performance in hierarchical edge-cloud computing (HECC) systems under dynamic task arrivals and heterogeneous resources, leading to severe queuing delays and inefficient resource utilization. To address this challenge, we study a joint service placement, computational delegation, and power control (JSCP) problem to minimize the average end-to-end (e2e) latency. The resulting JSCP problem is a mixed-integer nonconvex and NP-hard optimization problem due to the strong coupling between discrete and continuous variables. To enable tractable optimization and stable system adaptation, we exploit the inherent difference in decision dynamics and decompose the problem into long-term system configuration and short-term resource allocation subproblems. Based on this formulation, we propose a two-timescale multi-layer deep reinforcement learning framework with a latent action space (2T-MDRL-LA) to jointly optimize service placement, user association, computational delegation, task offloading, and user transmit power. A latent action representation based on a variational autoencoder is introduced to efficiently compress the high-dimensional combinatorial action space. Simulation results demonstrate that the proposed framework effectively adapts to dynamic network conditions and achieves near-optimal performance compared to branch-and-bound solutions. It achieves up to a 20.8% reduction in average e2e latency and a 13% improvement in resource utilization over the scheme without the computational delegation, while converging approximately 50% faster than conventional proximal policy optimization.
Original Article
View Cached Full Text

Cached at: 07/22/26, 08:18 AM 

# Multi-Timescale Latent-Action DRL for Joint Optimization in Edge-Cloud Networks
Source: [https://arxiv.org/abs/2607.18288](https://arxiv.org/abs/2607.18288)
[View PDF](https://arxiv.org/pdf/2607.18288)

> Abstract:Load imbalance across edge and cloud layers degrades latency performance in hierarchical edge\-cloud computing \(HECC\) systems under dynamic task arrivals and heterogeneous resources, leading to severe queuing delays and inefficient resource utilization\. To address this challenge, we study a joint service placement, computational delegation, and power control \(JSCP\) problem to minimize the average end\-to\-end \(e2e\) latency\. The resulting JSCP problem is a mixed\-integer nonconvex and NP\-hard optimization problem due to the strong coupling between discrete and continuous variables\. To enable tractable optimization and stable system adaptation, we exploit the inherent difference in decision dynamics and decompose the problem into long\-term system configuration and short\-term resource allocation subproblems\. Based on this formulation, we propose a two\-timescale multi\-layer deep reinforcement learning framework with a latent action space \(2T\-MDRL\-LA\) to jointly optimize service placement, user association, computational delegation, task offloading, and user transmit power\. A latent action representation based on a variational autoencoder is introduced to efficiently compress the high\-dimensional combinatorial action space\. Simulation results demonstrate that the proposed framework effectively adapts to dynamic network conditions and achieves near\-optimal performance compared to branch\-and\-bound solutions\. It achieves up to a 20\.8% reduction in average e2e latency and a 13% improvement in resource utilization over the scheme without the computational delegation, while converging approximately 50% faster than conventional proximal policy optimization\.

## Submission history

From: Van\-Dinh Nguyen Dr \[[view email](https://arxiv.org/show-email/61b7ace9/2607.18288)\] **\[v1\]**Tue, 30 Jun 2026 11:56:15 UTC \(2,739 KB\)

Similar Articles

Performance-Driven Environment Abstraction with Multi-Timescale Learning

arXiv cs.LG

This paper proposes a performance-driven state abstraction method for reinforcement learning that directly optimizes decision quality, using a multi-timescale framework to jointly adapt the policy and a tree-structured abstraction. The algorithm refines or aggregates state space based on Q-value discrepancies, achieving better sample efficiency and faster replanning than baselines.

Dynamic Latent Routing

Hugging Face Daily Papers

Dynamic Latent Routing (DLR) lets LLMs learn their own inner monologue by composing sub-policies via search, inspired by language compositionality. In low-data fine-tuning, DLR matches or outperforms standard supervised fine-tuning.