用于高性能计算作业调度的深度强化学习：统计评估

# Deep Reinforcement Learning for HPC Job Scheduling: A Statistical Evaluation
Source: [https://arxiv.org/html/2610.21008](https://arxiv.org/html/2610.21008)
Justin M\. Cheney[https://orcid.org/0009-0004-1991-474X](https://orcid.org/0009-0004-1991-474X)Affiliation:Computer Science Department, University of the Western Cape, Robert Sobukwe Rd, Bellville, Cape Town, 7535, South Africa Corresponding author: Clement N\. Nyirenda \(\)E\-mail[cnyirenda@uwc\.ac\.za](mailto:[email protected])Clement N\. Nyirenda[https://orcid.org/0000-0002-4181-0478](https://orcid.org/0000-0002-4181-0478)Affiliation:Computer Science Department, University of the Western Cape, Robert Sobukwe Rd, Bellville, Cape Town, 7535, South Africa Corresponding author: Clement N\. Nyirenda \(\)E\-mail[cnyirenda@uwc\.ac\.za](mailto:[email protected])Affiliation:eResearch Office, University of the Western Cape, Bellville, Cape Town, South Africa

###### Abstract

Deep Reinforcement Learning \(DRL\) offers learned job selection for High\-Performance Computing \(HPC\), but performance against heuristics alone does not establish value beyond action masking\. Six DRL treatments are compared: Proximal Policy Optimization \(PPO\), Advantage Actor\-Critic \(A2C\), and Deep Q\-Network \(DQN\), each with and without masking\. Three heuristics and a seed\-paired masked uniform\-random control provide baselines across ten seeds on two real Slurm \(Simple Linux Utility for Resource Management\) workload traces, one homogeneous and one heterogeneous, under a common non\-parametric framework\. All masked treatments complete evaluation, whereas unmasked treatments are truncated\. On the homogeneous trace, no masked DRL treatment significantly improves average waiting time or average and maximum bounded slowdown over the control\. On the heterogeneous trace, MaskablePPO improves all three metrics relative to the control\. Its average waiting time is equivalent to the three heuristics within a±10%\\pm 10\\%margin, and its maximum bounded slowdown is 22% lower than Last\-Come, First\-Served \(LCFS\)\. MaskablePPO has the leading average\-waiting rank among complete treatments on both traces, but Nemenyi comparisons detect no significant difference from MaskableA2C\. Among masked variants, MaskablePPO has the lowest recorded mean training time, while MaskableDQN has the lowest mean decision latency\. These findings distinguish completion from baseline\-specific scheduling gains and support including a masked random control when assessing learned\-selection value\.

###### Keywords:

High\-Performance Computing Deep Reinforcement Learning Statistical Analysis Job Scheduling

## 1Introduction

High\-Performance Computing \(HPC\) job schedulers determine when jobs run and which resources they receive\[[15](https://arxiv.org/html/2610.21008#bib.bib1)\]\. Increasing workload and hardware diversity complicates the balance between waiting time and resource utilisation\[[15](https://arxiv.org/html/2610.21008#bib.bib1)\]\. Deep Reinforcement Learning \(DRL\) offers learned scheduling policies as an alternative to traditional heuristics\[[5](https://arxiv.org/html/2610.21008#bib.bib2),[25](https://arxiv.org/html/2610.21008#bib.bib3)\]\. Their practical value, however, depends on whether learning improves schedule quality enough to justify training and decision\-time costs\.

Existing studies benchmark individual DRL schedulers against traditional baselines\[[18](https://arxiv.org/html/2610.21008#bib.bib4),[16](https://arxiv.org/html/2610.21008#bib.bib5)\], leaving limited evidence for choosing between DRL algorithms under a common experimental protocol\. A structured review of 53 papers identifies Proximal Policy Optimization \(PPO\) family methods in 40% of the sample, but this prevalence does not establish their superiority\. Moreover, performance against a heuristic alone does not establish that learned selection adds value beyond action masking\. A controlled comparison must therefore examine algorithm choice and masking together, evaluating schedule quality alongside computational cost\.

Action masking excludes infeasible choices before action selection\. MaskablePPO, MaskableDQN, and MaskableA2C apply this restriction to their corresponding base algorithms\. The objective is to compare DRL algorithms in terms of schedule quality and computational cost, and to assess learning value beyond action masking\. The comparison includes six DRL variants: PPO, Deep Q\-Network \(DQN\), and Advantage Actor\-Critic \(A2C\), each with and without masking\. Three heuristics and a masked random control provide baselines on two Slurm \(Simple Linux Utility for Resource Management\) workload traces\. Waiting time, slowdown, turnaround time, and resource utilisation are assessed through non\-parametric comparisons\[[2](https://arxiv.org/html/2610.21008#bib.bib23)\]\. The investigation is guided by three research questions \(RQs\):

- 1\.What does action masking contribute to schedule quality and evaluation completion?
- 2\.Does PPO’s schedule quality and computational cost relative to A2C and DQN justify its prevalence in recent HPC scheduling literature?
- 3\.How does the performance of learned policies relative to heuristic and masked random baselines differ between the homogeneous and heterogeneous workload traces, considering training and decision\-time costs?

This paper makes the following contributions:

- 1\.A controlled statistical comparison of PPO, A2C, and DQN, with and without action masking, across homogeneous and heterogeneous Slurm workload traces, using common workloads, reward functions, and environment\-step budgets\.
- 2\.An assessment of masking’s contribution to evaluation completion and schedule quality, including a masked random control to test whether learned selection adds value beyond feasibility constraints\.
- 3\.A non\-parametric, multi\-metric assessment of algorithm rankings and trade\-offs between schedule quality and cost, evaluating PPO’s relative performance across the two traces\.

Given the paper’s scope constraints, a basic understanding of HPC and non\-parametric statistical analysis is assumed\. The remainder of this paper reviews the literature and the gaps motivating the study \(Section[2](https://arxiv.org/html/2610.21008#S2)\), sets out the methodology \(Section[3](https://arxiv.org/html/2610.21008#S3)\), presents the results \(Section[4](https://arxiv.org/html/2610.21008#S4)\), discusses their implications and limitations \(Section[5](https://arxiv.org/html/2610.21008#S5)\) and concludes \(Section[6](https://arxiv.org/html/2610.21008#S6)\)\.

## 2Literature Review

HPC job scheduling has evolved from rule\-based heuristics to learning\-based adaptive techniques\. This section reviews that progression and identifies the methodological gaps motivating the present study\.

### 2\.1HPC Job Scheduling

Job scheduling combines job selection, which determines execution order, with resource allocation, which assigns cores, memory, and graphics processing units \(GPUs\) to selected jobs\[[25](https://arxiv.org/html/2610.21008#bib.bib3)\]\. Scheduling objectives include reducing waiting times, increasing resource utilisation, and maintaining fairness\[[15](https://arxiv.org/html/2610.21008#bib.bib1)\]\. These objectives can conflict, motivating evaluation across multiple metrics rather than a single performance measure\.

### 2\.2Traditional and Meta\-heuristic Techniques

Traditional heuristics address competing scheduling objectives by assigning priorities to pending jobs\. First\-Come\-First\-Served \(FCFS\) preserves arrival order, whereas Shortest Job First \(SJF\) prioritises shorter estimated runtimes\. FCFS can cause head\-of\-line blocking, while SJF can delay long jobs\[[19](https://arxiv.org/html/2610.21008#bib.bib7)\]\. Backfilling supplements these priority rules by running other jobs in otherwise idle capacity while protecting a reservation for a waiting job\[[9](https://arxiv.org/html/2610.21008#bib.bib8)\]\. Workload managers such as Slurm coordinate scheduling policies with resource management\[[27](https://arxiv.org/html/2610.21008#bib.bib9)\]\. This separation allows DRL to be studied as an alternative job\-selection policy, rather than a replacement for the workload manager\. Meta\-heuristics search candidate schedules using techniques such as genetic algorithms, particle swarm optimization\[[21](https://arxiv.org/html/2610.21008#bib.bib27)\], and simulated annealing\[[15](https://arxiv.org/html/2610.21008#bib.bib1)\]\. Such searches can improve schedule quality, but their computational cost and sensitivity to tuning must be considered\. Learning\-based approaches instead seek a policy that can be reused across scheduling decisions\.

### 2\.3Deep Reinforcement Learning for Job Scheduling

Reinforcement learning trains a policy through interaction and reward, without requiring labelled optimal scheduling decisions\. DRL uses neural networks to represent policies or value functions, with scheduling formulated as a Markov Decision Process \(MDP\) over queue and cluster states\[[22](https://arxiv.org/html/2610.21008#bib.bib10),[25](https://arxiv.org/html/2610.21008#bib.bib3)\]\. PPO\-family methods account for 45% of the 60 systems described in the surveyed literature\. However, performance on general reinforcement\-learning benchmarks\[[17](https://arxiv.org/html/2610.21008#bib.bib11)\]does not by itself establish an advantage in HPC scheduling\. This motivates direct comparisons of scheduling performance and computational cost\.

Algorithm families\.Value\-based methods estimate action values, with DQN using experience replay and target networks\[[13](https://arxiv.org/html/2610.21008#bib.bib12)\]\. Policy\-based methods optimise expected return directly, while actor\-critic methods combine a policy with a value estimate that helps reduce variance in its updates\. A2C is the synchronous variant of Asynchronous Advantage Actor\-Critic \(A3C\)\[[12](https://arxiv.org/html/2610.21008#bib.bib13)\]\. These categories overlap: PPO and A2C both use actor and critic networks, but differ in their update objectives and data reuse\. PPO uses a clipped surrogate objective and repeatedly optimises rollout data, whereas A2C uses synchronous advantage\-based updates\[[12](https://arxiv.org/html/2610.21008#bib.bib13)\]\. Therefore, their sample efficiency, exploration, and stability must be assessed in the scheduling task rather than inferred from their algorithm category\[[1](https://arxiv.org/html/2610.21008#bib.bib28)\]\.

#### Action masking\.

Action masking simplifies the agent’s decision process by excluding infeasible actions\[[20](https://arxiv.org/html/2610.21008#bib.bib14)\]\. For value\-based methods, invalid actions are masked beforearg⁡max\\arg\\maxselection; for policy\-based methods, invalid probabilities are zeroed before normalisation\. In HPC scheduling this prevents selecting jobs whose resource demands exceed current capacity\. When masking is included as part of the environment without being evaluated separately, reported performance does not establish whether learning adds value beyond constraint enforcement\. This paper compares masked variants against their unmasked counterparts to assess differences in schedule quality and evaluation completion\. A masked random control selects uniformly among valid actions, providing a reference for performance without learning under the same feasibility constraints\.

### 2\.4DRL Applications in Job Scheduling

DRL has been adopted across cloud computing\[[10](https://arxiv.org/html/2610.21008#bib.bib15)\], edge systems\[[23](https://arxiv.org/html/2610.21008#bib.bib18)\]and HPC clusters\[[6](https://arxiv.org/html/2610.21008#bib.bib16),[25](https://arxiv.org/html/2610.21008#bib.bib3)\], targeting makespan, utilisation and fairness\. Wu*et al\.*\[[26](https://arxiv.org/html/2610.21008#bib.bib17)\]employed a Double\-DQN\-derived method for dual\-objective flexible job\-shop scheduling, comparing it with three meta\-heuristics rather than other DRL algorithms\. PPO\-based examples include Gao*et al\.*\[[6](https://arxiv.org/html/2610.21008#bib.bib16)\], who applied Gated Transformer\-XL \(GTrXL\) with PPO, and Wang*et al\.*\[[25](https://arxiv.org/html/2610.21008#bib.bib3)\], who proposed hierarchical PPO for two\-level scheduling\. These studies evaluate PPO\-based scheduling designs rather than providing a common\-protocol comparison of PPO, A2C and DQN\. RLScheduler\[[28](https://arxiv.org/html/2610.21008#bib.bib26)\]uses a policy\-gradient actor\-critic approach, motivated by concerns that workload variability may cause oscillations in Q\-learning\. Its benchmarks include FCFS, SJF, the named multi\-factor priority rules WFP3 and UNICEP \(which combine waiting time, requested runtime, and resource demand\), and the regression\-derived priority rule F1\. These heuristic comparisons do not directly compare DRL algorithms or establish learning value against a masked random control\. Others extend the base algorithms through multi\-agent formulations\[[11](https://arxiv.org/html/2610.21008#bib.bib19)\]and transfer learning\[[24](https://arxiv.org/html/2610.21008#bib.bib20)\]\.

These studies demonstrate the potential of DRL for adaptive scheduling, but differences in statistical treatment, performance metrics, workloads and baselines limit direct comparison of their reported improvements\. A gain against a meta\-heuristic on a synthetic workload is not directly equivalent to a gain against a heuristic on a production trace\. This evaluation heterogeneity motivates the common\-protocol comparison used here\.

### 2\.5Limitations and Research Gaps

The benefits of DRL come with limitations bearing on deployability\. State and action spaces can grow with workload complexity, increasing the computational demands of learning\. Sparse or delayed rewards can complicate learning, particularly when scheduling outcomes emerge over many decisions\. Learned policies can be difficult to interpret, which may restrict adoption in mission\-critical environments\. Algorithm choice therefore involves more than schedule quality\. Training requires computation and time, while using a trained policy introduces a decision\-time overhead\. Interpretability and maintenance present additional deployment concerns, but these are not quantified in this study\. Three gaps motivate the present comparison\. First, PPO’s prevalence does not establish a schedule\-quality or computational\-cost advantage over A2C and DQN under common experimental conditions\. Second, performance with masking alone does not establish learning value beyond feasibility enforcement\. Third, differences in metrics, workloads, baselines and statistical treatment limit direct comparison of reported results, while single\-metric reporting can conceal trade\-offs between schedule quality and utilisation\. The methodology in Section[3](https://arxiv.org/html/2610.21008#S3)addresses these directly\.

## 3Methodology

The study compares ten scheduling methods, referred to as treatments: six DRL variants, three heuristics, and one masked random control\. PPO, A2C, and DQN use stable\-baselines3 \(SB3\) implementations, while MaskablePPO uses sb3\-contrib\[[14](https://arxiv.org/html/2610.21008#bib.bib22)\]\. MaskableA2C and MaskableDQN use local implementations\. The heuristic baselines are SJF, Last\-Come, First\-Served \(LCFS\), and UNICEP\. All ten are compared across two Slurm workload traces, using the HPCSim scheduling simulator\[[25](https://arxiv.org/html/2610.21008#bib.bib3)\]as the environment under fixed seeds for reproducibility\. Git commit hashes and SHA\-256 manifests record provenance\. Locked Nix inputs pin software dependencies, while Apptainer packages the execution environment\. Snakemake 9\.4\.3 orchestrates the pipeline\.

### 3\.1Environment and Dataset

HPCSim uses the cluster and job\-queue state representation described by Wang*et al\.*\[[25](https://arxiv.org/html/2610.21008#bib.bib3)\]\. The observation is the information available to the agent at each scheduling decision\. The cluster component describes available resources and running jobs\. The queue component describes a fixed\-size window of pending jobs, including their resource requests, timing information, and scheduling feasibility\. The window limits the jobs visible to the agent, not the total queue length\. With the configuration in Table[1](https://arxiv.org/html/2610.21008#S3.T1), queues exceeding 512 jobs are represented by 448 jobs from the head and 64 from the tail\. Shorter queues are padded with zeros, and the total queue length is included\. The “best\_fit” allocator is fixed across all treatments to keep allocation policy constant while comparing job selection\. Backfill is disabled in the primary comparisons for both DRL policies and baselines, avoiding an additional scheduling mechanism available only to the heuristics\. These comparisons therefore evaluate job\-selection policies under fixed allocation and backfill settings, rather than complete production Slurm configurations\.

Table 1:Experimental configuration: environment, hyperparameters and dataset\.ParameterValueNotesHPCSim environmentAllocatorbest\_fitFixed as controlled factorWindow Size512512Observable queue depthTail Size6464Queue tail bufferRandom jobsFalseDeterministic trace replayBackfillFalseUniform mechanism across algorithmsHyperparametersHidden Layers\[4096,2048,1024\]\[4096,2048,1024\]AllActivation functiontanhAllDiscount factor \(γ\\gamma\)0\.990\.99AllPolicyMultiInputPolicyAllEntropy Coefficient0\.0PPO/A2C variants onlyReplay Buffer Size150,000150\{,\}000DQN variants onlyDataset: physical / deeplearnTracephysical\_jobs\.csv / deeplearn\_jobs\.csvTopologyphysical\_topology\.txt / deeplearn\_topology\.txtNode filenodes\.csvTotal Jobs84,13584\{,\}135/68,72068\{,\}720Development Jobs \(70%\)58,89458\{,\}894/48,10448\{,\}104Holdout Jobs \(30%\)25,24125\{,\}241/20,61620\{,\}616Temporal Range37d 11h 42m 8s / 382d 5h 21m 49s

The dataset is taken from Wang*et al\.*\[[25](https://arxiv.org/html/2610.21008#bib.bib3)\]: two Slurm traces, one homogeneous \(physical\), the other heterogeneous \(deeplearn\)\. Each trace is ordered by submission time and divided into an earlier 70% development split and a later 30% holdout split\. The development splits are used for training, primary comparisons, and model selection\. The holdout splits are kept for final generalisation assessment\.

### 3\.2Training and Evaluation

The DRL variants use the hidden\-layer sizes and activation function in Table[1](https://arxiv.org/html/2610.21008#S3.T1)\. Only the DQN variants use a replay buffer\. Each DRL training run has a target budget of 3 million environment steps; optimisation\-update counts are not matched across algorithms\. Ten fixed seeds initialize the Python, NumPy, and PyTorch random number generators\. Training six DRL variants with ten seeds produces 60 runs per trace and 120 across both traces\. Each trained model is evaluated on its corresponding development split using its training seed\. Evaluation is deterministic: the model selects its preferred action rather than sampling exploratory actions\. Results are aggregated across the ten seeds for each DRL variant\. Scheduling performance is measured using average and maximum waiting time, average and maximum bounded slowdown, average turnaround time, and central processing unit \(CPU\) and GPU utilisation, following Wang*et al\.*\[[25](https://arxiv.org/html/2610.21008#bib.bib3)\]\. The three primary metrics reported in baseline comparisons are average waiting time and average and maximum bounded slowdown\. Decision latency, evaluation wall\-clock time, and episode reward are also recorded\. The training reward is the negative mean queued waiting time in hours \(R=−avg\_waiting/3600\.0R=\-\\texttt\{avg\\\_waiting\}/3600\.0\)\. This reward is computed every 512 actions; intervening actions receive zero reward\.

### 3\.3Statistical Framework

Following Carrasco*et al\.*\[[2](https://arxiv.org/html/2610.21008#bib.bib23)\], distributional assumptions are assessed before any comparative tests\. Shapiro\-Wilk assesses normality of the recorded seed\-level metric values for each treatment\. The non\-normality detected in some treatment distributions, reported in Section[4](https://arxiv.org/html/2610.21008#S4), supports a comparison that does not rely on normally distributed performance values\. Friedman therefore compares the six DRL variants through their within\-seed ranks across ten seed blocks \(α=0\.05\\alpha=0\.05\)\. Kendall’sW=χw2/\[N⁡\(k−1\)\]W=\\chi^\{2\}\_\{w\}/\[N\(k\-1\)\]accompanies the Friedman results and summarises agreement in treatment ranks across seeds, whereN=10N=10andk=6k=6\. Where Friedman is significant, Nemenyi post\-hoc comparisons follow, with Critical Difference \(CD\) plots showing non\-significant rank groupings\[[3](https://arxiv.org/html/2610.21008#bib.bib6)\]\. Wilcoxon confidence intervals and confidence curves give effect\-size context\. The Page trend test assesses trends across the declared treatment order, not convergence over training time\[[4](https://arxiv.org/html/2610.21008#bib.bib21)\]\. One DRL treatment is Pareto\-selected for each trace across the primary and secondary metrics\. If needed, the predefined tie\-break order is average waiting time, average bounded slowdown, then CPU utilisation\. For each heuristic, one\-sample Wilcoxon signed\-rank tests compare seed\-level MaskablePPO and MaskableA2C outcomes with its fixed reference value\. Practical equivalence is assessed using two one\-sided tests \(TOST\), with a margin of±\\pm10% of that reference value\[[8](https://arxiv.org/html/2610.21008#bib.bib25)\]\. Non\-significance in a difference test does not establish equivalence\. For each DRL treatment compared with a heuristic or the random control, Holm correction is applied jointly to the six Wilcoxon difference\-test p\-values: average and maximum waiting time, average and maximum bounded slowdown, average turnaround time and CPU utilisation\[[7](https://arxiv.org/html/2610.21008#bib.bib24)\]\.

A random action\-selection control is included as a reference condition, drawing uniformly at each decision over the valid actions of the same MDP, under the same mask, without backfill, on the same ten seeds\. Each masked DRL treatment is compared with the control using a two\-sided Wilcoxon signed\-rank test on seed\-paired differences\. The control shows how a policy performs without learning under the same action mask\. Matching a heuristic alone does not establish that learning adds value\. Finally, DRL treatments and the random control are evaluated on the holdout splits using the same ten seeds; the heuristics remain deterministic reference runs\.

## 4Results

This section reports results following the methodology in Section[3](https://arxiv.org/html/2610.21008#S3), presenting the physical and deeplearn traces in turn before examining generalisation, training and inference cost, and training diagnostics\. Shapiro\-Wilk rejects normality for some average\-waiting distributions: physical A2C \(W=0\.634W=0\.634,p=0\.00014p=0\.00014\) and DQN \(W=0\.753W=0\.753,p=0\.0039p=0\.0039\), and deeplearn A2C \(p≈10−7p\\approx 10^\{\-7\}\)\. These results support the decision not to assume normality and to use the rank\-based framework described in Section[3](https://arxiv.org/html/2610.21008#S3)\. Tests use recorded seed\-level metric values; reported means and medians summarise performance across seeds\.

Two conventions apply throughout\. The first is truncation\. None of the three unmasked treatments completed a full evaluation episode on either trace\. Without a mask the policy can select an action naming no allocatable job, which HPCSim treats as a request to advance time; once the event queue is empty, simulated time freezes on an identical state and the episode cannot terminate\. A hang guard stops the run after 500,000 consecutive steps without a job completion\. These metrics describe incomplete evaluations and are not directly comparable with full\-trace outcomes\. The treatments are excluded from schedule\-quality best\-value marking and multi\-metric selection, and marked “†\{\\dagger\}” throughout\. They remain in the six\-treatment statistical comparisons, which must therefore be interpreted with this limitation\. The second is the backfill band\. Each heuristic was evaluated with backfill both enabled and disabled, and the width of that band is reported per trace below; the baseline figures quoted elsewhere are the no\-backfill ones, for the reason given in Section[3](https://arxiv.org/html/2610.21008#S3)\.

### 4\.1Homogeneous Workload: Physical Trace

On this trace, truncation limits comparisons with the unmasked treatments, while backfill changes the heuristic reference values\. Enabling backfill reduces UNICEP’s average waiting time by 12\.3%, but LCFS still records lower average waiting time than the complete DRL treatments with both backfill settings\. GPU utilisation is omitted because this is a CPU\-only workload\.

Table[2](https://arxiv.org/html/2610.21008#S4.T2)shows that MaskableA2C and MaskablePPO have similar mean average waiting times, whereas MaskableDQN has higher waiting time and bounded slowdown, with greater variation across seeds\. The unmasked variants record higher waiting times still, but their truncated evaluations limit comparisons with complete\-trace outcomes\. Across all six DRL treatments, Friedman detects statistically significant differences for all nine reported metrics, withχ2\\chi^\{2\}from 22\.11 to 47\.14 and Kendall’sWWfrom 0\.442 to 0\.943\. However, because these tests include truncated outcomes, the differences do not by themselves establish full\-trace schedule\-quality superiority\. For CPU utilisation, the omnibus test is significant \(p=2\.4×10−5p=2\.4\\times 10^\{\-5\},W=0\.579W=0\.579\), yet the mean values for MaskableA2C and MaskablePPO both round to 0\.1057; a pairwise comparison is needed to determine whether these two treatments differ\.

Table 2:Physical trace \- performance metrics \(mean±\\pmstd\.\)\. Wait and turnaround in seconds; slowdown is a ratio; utilisation a fraction\.Table[3](https://arxiv.org/html/2610.21008#S4.T3)ranks MaskablePPO first and MaskableA2C second for average waiting time, but the Nemenyi comparison detects no significant difference between them \(p=1\.000p=1\.000,n=10n=10,k=6k=6\)\. Across the declared sequence of DQN, A2C, PPO, MaskableDQN, MaskableA2C, and MaskablePPO, the Page test detects a decreasing average\-waiting trend \(L=838L=838,p=6×10−6p=6\\times 10^\{\-6\}\), with significant trends also found for the other scheduling metrics in Table[2](https://arxiv.org/html/2610.21008#S4.T2), except maximum waiting time \(p=1\.0000p=1\.0000\)\. However, these trends depend on the declared order and include truncated evaluations; they do not demonstrate strict monotonic improvement in full\-trace schedule quality\. For average waiting time on physical, the paired 95% confidence interval for MaskableA2C minus MaskablePPO is\[−65\.5,\+13\.3\]\[\-65\.5,\+13\.3\]s\. It lies within the stated±\\pm10% margin relative to MaskablePPO, supporting practical equivalence for this metric under that margin\. This interval evidence is distinct from the non\-significant Nemenyi comparison and does not establish equivalence across all metrics\. The confidence curve peaks near zero\. The recorded multi\-metric selection identifies MaskableA2C as the sole Pareto candidate on physical; this selection is separate from MaskablePPO’s leading average\-waiting rank\.

Table 3:Physical trace \- Nemenyi post\-hoc ranking, average waiting time\.The DRL\-only rankings must also be considered alongside performance relative to the heuristics and the masked random control\. For MaskablePPO and MaskableA2C, most median relative differences against the no\-backfill heuristics lie approximately between−20%\-20\\%and\+10%\+10\\%on the reported primary metrics\. Average bounded slowdown shows larger contrasts: their values are 24\.7% and 24\.2% higher than LCFS, respectively, but 53\.8% and 54\.0% lower than UNICEP\.

Table[4](https://arxiv.org/html/2610.21008#S4.T4)compares the three masked DRL treatments with the random control using seed\-paired differences\. After Holm correction, seven of the nine comparisons detect significant differences, and all seven favour the random control\. These comparisons provide no evidence of a learned\-policy advantage over masked random selection on the reported metrics under this configuration\.

Table 4:Physical trace \- random control comparison, primary metrics\.For mean average waiting time, headroom is defined as the percentage by which the masked random control exceeds the best no\-backfill heuristic, relative to that heuristic\. Table[2](https://arxiv.org/html/2610.21008#S4.T2)gives LCFS at 2,090 s and the random control at 2,166 s; the unrounded values yield 3\.7% headroom\. On this physical trace, the results do not establish a learned\-policy advantage over the masked random control that would justify replacing a heuristic\.

### 4\.2Heterogeneous Workload: Deeplearn Trace

Table[5](https://arxiv.org/html/2610.21008#S4.T5)shows that MaskablePPO’s mean average waiting time is close to that of the no\-backfill heuristics\. Enabling backfill reduces LCFS’s average waiting time by 5\.7%, while SJF and UNICEP then record means about 4 and 3 seconds lower than MaskablePPO, respectively\. The baseline gap is larger on this trace\. Using mean average waiting time, Table[5](https://arxiv.org/html/2610.21008#S4.T5)gives the masked random control at 1,170 s and the best no\-backfill heuristic, UNICEP, at 846 s\. The unrounded values yield 38\.4% headroom relative to UNICEP, compared with 3\.7% on physical\. This descriptive gap does not establish why learning performs differently across traces\. Across the six DRL treatments, Friedman detects statistically significant differences for all ten metrics, with Kendall’sWWfrom 0\.487 to 0\.905\. However, GPU utilisation \(χ2=28\.63\\chi^\{2\}=28\.63,W=0\.573W=0\.573\) is computed asresource\-seconds/\\text\{resource\-seconds\}/\(total resource×makespan\)\(\\text\{total resource\}\\times\\text\{makespan\}\); for fixed completed work, a longer makespan lowers the reported ratio\. This measure should therefore be interpreted alongside makespan and evaluation completion\. Evaluation wall\-clock time for unmasked A2C, DQN and PPO also includes the 500,000 stall steps before truncation\.

Table 5:Deeplearn trace \- performance metrics \(mean±\\pmstd\. dev\.\)\. Wait and turnaround in seconds; slowdown is a ratio; utilisation a fraction\.Table[6](https://arxiv.org/html/2610.21008#S4.T6)gives unmasked PPO the lowest mean rank for average waiting time, but its truncated evaluations make it ineligible for selection\. MaskablePPO has the lowest mean rank among complete treatments and is the recorded multi\-metric selection\. However, no Nemenyi comparison with MaskablePPO detects a significant difference\. MaskableA2C’s lower median \(837 s versus 851 s\) contrasts with its much higher mean, which is dominated by two extreme seed\-level outcomes\. The lower median alone therefore does not establish superiority or consistent performance across seeds\.

Table 6:Deeplearn trace \- Nemenyi post\-hoc ranking, average waiting time\.Table[7](https://arxiv.org/html/2610.21008#S4.T7)shows statistically significant improvements for MaskablePPO over the masked random control on all three reported primary metrics after Holm correction\. Against the no\-backfill heuristics, however, TOST supports practical equivalence with UNICEP on all three primary metrics within the±\\pm10% margin\. The reported reductions in maximum bounded slowdown are 22\.0% relative to LCFS and 63\.2% relative to the random control\.

Table 7:Deeplearn trace \- random control comparison, primary metrics\.Table[5](https://arxiv.org/html/2610.21008#S4.T5)shows that MaskablePPO’s mean average waiting time is within 1% of SJF \(851 s\) and UNICEP \(846 s\)\. These small average\-waiting differences should be considered alongside the training and decision\-time costs in Section[4\.4](https://arxiv.org/html/2610.21008#S4.SS4)\. The configured deeplearn topology contains 30 nodes and 120 GPUs, while physical contains 87 nodes and 6,264 CPU cores\. Differences between these two traces do not establish that heterogeneous resources cause better DRL performance\.

### 4\.3Generalisation

Generalisation is assessed on the reserved holdout splits without further training or reselection\. On physical, mean average waiting time falls approximately fourfold for MaskableA2C and MaskablePPO; on deeplearn, it rises approximately threefold for MaskablePPO\. These changes concern different chronological job sets and should not alone be interpreted as improvement or deterioration in the trained policies\. The ranks order treatments by their across\-seed mean average waiting times, rather than the mean within\-seed ranks used in the DRL statistical comparisons\. MaskableA2C and MaskablePPO retain first and second place on physical, while MaskablePPO and unmasked PPO exchange the first two positions on deeplearn\. Elsewhere on physical, MaskableDQN moves from third to sixth and A2C from fourth to third\.

### 4\.4Training and Inference Cost

Tables[8](https://arxiv.org/html/2610.21008#S4.T8)and[9](https://arxiv.org/html/2610.21008#S4.T9)show that, among the masked variants, MaskablePPO and MaskableA2C have lower mean training times than MaskableDQN on both traces, while MaskableDQN has the lowest mean decision latency\. Focusing on these variants keeps the latency comparison within completed evaluations\.

Table 8:Physical trace \- training and inference cost\.Table 9:Deeplearn trace \- training and inference cost\.MaskableDQN’s lower decision latency in Tables[8](https://arxiv.org/html/2610.21008#S4.T8)and[9](https://arxiv.org/html/2610.21008#S4.T9)is only one part of its resource cost: training also requires replay storage\. The reported comparisons do not isolate the effects of replay capacity or update reuse on schedule quality, but the training configuration provides context for the diagnostics in Section[4\.5](https://arxiv.org/html/2610.21008#S4.SS5)\. Both DQN variants use a buffer capacity of 150,000 transitions and minibatches of 2,048, storing current and next observations from the high\-dimensional queue and cluster state\. The memory requirements of this storage depend on array dimensions and data types\.

### 4\.5Training Diagnostics

Explained variance, entropy, and learning rate are examined to help interpret the scheduling results\. These diagnostics do not establish that training was sufficient or isolate the cause of poor performance\. No episode\-reward curve is reported becauserollout/ep\_rew\_meanchanges infrequently during training\. DQN logs reward at episode boundaries, with reported averages of 3 to 21 reward\-log entries per run\. Its blank explained\-variance and entropy entries reflect metrics not reported by the logger, rather than zero\-valued measurements\.

The training diagnostics first examine policy entropy, which describes the concentration of action probabilities\. For unmasked A2C, the final logged entropy is0\.07±0\.160\.07\\pm 0\.16nats on physical and6\.23±0\.006\.23\\pm 0\.00on deeplearn\. The former indicates concentrated action probabilities, whereas the latter is close to the 513\-action uniform\-distribution ceiling \(ln⁡513≈6\.24\\ln 513\\approx 6\.24\)\. Critic fit is examined through explained variance, which measures how much variation in training returns is captured by the critic, rather than schedule quality: 1 indicates no residual variation and 0 indicates no reduction in variance\. For example, A2C’s windowed explained variance on the physical trace is0\.944±0\.0180\.944\\pm 0\.018over the 2M–3M training\-step interval\.

The distinction between critic fit and schedule quality can be examined through the masked treatments\. On physical, the final logged explained variance is0\.533±0\.4510\.533\\pm 0\.451for MaskableA2C and0\.109±0\.2630\.109\\pm 0\.263for MaskablePPO\. Neither treatment differs significantly from the masked random control in average waiting time, while both have significantly higher average and maximum slowdown \(Table[4](https://arxiv.org/html/2610.21008#S4.T4)\)\. These scheduling comparisons, rather than critic fit alone, provide the evidence for assessing whether learned job selection adds value beyond masking\.

## 5Discussion

Tables[2](https://arxiv.org/html/2610.21008#S4.T2)and[5](https://arxiv.org/html/2610.21008#S4.T5)highlight the importance of headroom\. The descriptive values of 3\.7% on physical and 38\.4% on deeplearn affect how each workload’s results are interpreted, without establishing heterogeneity as their cause\. The training diagnostics also highlight two issues: the common 3M environment\-step target does not match optimiser update counts, and rewards occur only every 512th action\. Atγ=0\.99\\gamma=0\.99, a reward 512 actions away carries a discount weight of0\.99512≈0\.00580\.99^\{512\}\\approx 0\.0058\. Controlled comparisons of reward density and optimisation budgets within each trace would help resolve their contributions to scheduling outcomes\. The findings remain specific to the tested reward, training configurations and two workload traces\. These constraints frame the research questions below\.

#### RQ1: The contribution of action masking\.

All masked treatments complete evaluation on both traces, but completion alone does not establish good schedule quality\. The masked random comparisons assess the additional value of learning\. On physical, neither MaskablePPO nor MaskableA2C differs significantly from the control in average waiting time, while both have significantly higher average and maximum slowdown \(Table[4](https://arxiv.org/html/2610.21008#S4.T4)\)\. On deeplearn, MaskablePPO improves all three metrics relative to the control, with Holm\-adjustedp=0\.012p=0\.012for each comparison \(Table[7](https://arxiv.org/html/2610.21008#S4.T7)\)\. MaskableDQN performs significantly worse than the control on all three metrics on both traces\. Masking therefore supports completion in the tested configuration, while the benefit of learned selection depends on the treatment and workload\.

#### RQ2: The justification for concentrating on PPO\.

MaskablePPO has the best average\-waiting rank among complete treatments on both traces \(Tables[3](https://arxiv.org/html/2610.21008#S4.T3)and[6](https://arxiv.org/html/2610.21008#S4.T6)\), but its Nemenyi comparisons with MaskableA2C detect no significant difference on physical \(p=1\.000p=1\.000, rounded\) or deeplearn \(p=0\.999p=0\.999, rounded\)\. The recorded multi\-metric selection chooses MaskableA2C on physical and MaskablePPO on deeplearn\. The cost comparison also presents a trade\-off: PPO variants have lower recorded mean training times than their corresponding A2C variants \(by 9–22%\) and DQN variants, while MaskableDQN has the lowest mean decision latency among masked treatments \(Tables[8](https://arxiv.org/html/2610.21008#S4.T8)and[9](https://arxiv.org/html/2610.21008#S4.T9)\)\. These findings support PPO as a leading candidate in the tested configuration, but do not establish a general schedule\-quality and cost advantage that justifies its prevalence\.

#### RQ3: When a DRL scheduler is worth its inference cost\.

With heuristic backfill disabled, the justification for learned selection differs between the two traces\. On physical, the masked treatments show no improvement over the random control in average waiting time or slowdown\. On deeplearn, MaskablePPO has significantly lower average slowdown than all three heuristics and is equivalent to UNICEP on the three primary metrics within the specified±10%\\pm 10\\%margin\. These gains carry computational costs: MaskablePPO’s evaluation takes 35×\\timesand 13×\\timesthe fastest heuristic’s time on physical and deeplearn, respectively\. The deeplearn holdout further qualifies the comparison: its descriptive average\-waiting advantage over LCFS and SJF widens, but its difference from UNICEP remains below 1%, while its advantage over masked random narrows from 27\.7% to 3\.9%\. Practical justification therefore depends on whether gains over the relevant baseline, on the metrics of interest, outweigh training and decision\-time costs under the intended workload\.

## 6Conclusion and Future Work

The study compared six DRL treatments, three heuristics and a masked uniform\-random control over ten seeds on two real HPC workload traces under a common non\-parametric framework\. Masking contributed to the completion of evaluation runs, but the findings also show that the benefit of learned selection depends on both the treatment and workload, rather than masking alone\. MaskablePPO had the leading average\-waiting rank among complete treatments, but its Nemenyi comparisons with MaskableA2C detected no significant difference\. The multi\-metric selection chose MaskableA2C on physical and MaskablePPO on deeplearn\. PPO’s lower recorded training times and DQN’s lower decision latencies present a cost trade\-off rather than a general advantage for one family\. Scheduling gains were also baseline\- and metric\-dependent: on the deeplearn holdout, MaskablePPO’s average\-waiting difference from UNICEP remained below 1%\. Practical justification therefore requires weighing the relevant gains against training and decision\-time costs\.

Beyond these findings, a key outcome is the value of including a seed\-paired masked random control\. It helps distinguish the performance of learned selection from that attainable through feasibility masking with random selection\. Building on this comparison, future studies should vary reward density and optimisation budgets within fixed traces to assess their contributions to scheduling outcomes, then examine additional workloads to test whether the observed contrasts recur\. If consistent learned\-selection benefits are established, model distillation could be evaluated for reducing decision latency while preserving feasibility masking and schedule quality\.

#### Acknowledgements

Computations were performed using the University of the Western Cape’s High Performance Computing and Research Cloud facilities\. The authors acknowledge the use of OpenCode to assist with grammatical editing, sentence restructuring, and minor code refinement\. All suggestions assisted by artificial intelligence were reviewed, revised, and verified by the authors, who take full responsibility for the final content and code\.

#### Disclosure of Interests\.

The authors have no competing interests to declare that are relevant to the content of this article\.

## References

- \[1\]\(2025\)Comparative analysis of parameterized action actor\-critic reinforcement learning algorithms for web search match plan generation\.InProceedings of the Tenth International Congress on Information and Communication Technology \(ICICT 2025\),Vol\.1413,pp\. 203–213\.External Links:[Document](https://dx.doi.org/10.1007/978-981-96-6435-1%5F16)Cited by:[§2\.3](https://arxiv.org/html/2610.21008#S2.SS3.p2.1)\.
- \[2\]J\. Carrasco, S\. García, M\. M\. Rueda, S\. Das, and F\. Herrera\(2020\)Recent trends in the use of statistical tests for comparing swarm and evolutionary computing algorithms: practical guidelines and a critical review\.Swarm and Evolutionary Computation54,pp\. 100665\.Cited by:[§1](https://arxiv.org/html/2610.21008#S1.p3.1),[§3\.3](https://arxiv.org/html/2610.21008#S3.SS3.p1.1)\.
- \[3\]J\. Demšar\(2006\)Statistical comparisons of classifiers over multiple data sets\.Journal of Machine Learning Research7,pp\. 1–30\.Cited by:[§3\.3](https://arxiv.org/html/2610.21008#S3.SS3.p1.1)\.
- \[4\]J\. Derrac, S\. García, D\. Molina, and F\. Herrera\(2011\)A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms\.Swarm and Evolutionary Computation1\(1\),pp\. 3–18\.External Links:[Document](https://dx.doi.org/10.1016/j.swevo.2011.02.002)Cited by:[§3\.3](https://arxiv.org/html/2610.21008#S3.SS3.p1.1)\.
- \[5\]V\. Firmano\(2024\)Deep reinforcement learning for dynamic job\-shop scheduling in high\-utilization systems\.Master’s Thesis,Politecnico di Torino\.Cited by:[§1](https://arxiv.org/html/2610.21008#S1.p1.1)\.
- \[6\]X\. Gao, H\. Dong, L\. Zhang, Y\. Wang, X\. Yang, and Z\. Li\(2025\)Self\-attention mechanisms in hpc job scheduling: a novel framework combining gated transformers and enhanced ppo\.Applied Sciences \(Switzerland\)15\.External Links:[Document](https://dx.doi.org/10.3390/app15168928),ISSN 20763417Cited by:[§2\.4](https://arxiv.org/html/2610.21008#S2.SS4.p1.1)\.
- \[7\]S\. Holm\(1979\)A simple sequentially rejective multiple test procedure\.Scandinavian Journal of Statistics6\(2\),pp\. 65–70\.Cited by:[§3\.3](https://arxiv.org/html/2610.21008#S3.SS3.p1.1)\.
- \[8\]D\. Lakens\(2017\)Equivalence tests: a practical primer for t tests, correlations, and meta\-analyses\.Social Psychological and Personality Science8\(4\),pp\. 355–362\.Cited by:[§3\.3](https://arxiv.org/html/2610.21008#S3.SS3.p1.1)\.
- \[9\]D\. A\. Lifka\(1995\)The ANL/IBM SP scheduling system\.InWorkshop on Job Scheduling Strategies for Parallel Processing \(JSSPP’95\),Lecture Notes in Computer Science, Vol\.949,pp\. 295–303\.Note:Foundational work on backfilling strategies for HPC job schedulingExternal Links:[Document](https://dx.doi.org/10.1007/3-540-60153-8%5F35)Cited by:[§2\.2](https://arxiv.org/html/2610.21008#S2.SS2.p1.1)\.
- \[10\]S\. S\. Mangalampalli, G\. R\. Karri, S\. N\. Mohanty, S\. Ali, M\. I\. Khan, S\. Abdullaev, and S\. A\. Alqahtani\(2024\)Multi\-objective prioritized task scheduler using improved asynchronous advantage actor critic \(a3c\) algorithm in multi cloud environment\.IEEE Access12,pp\. 11354–11377\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2024.3355092),ISSN 21693536Cited by:[§2\.4](https://arxiv.org/html/2610.21008#S2.SS4.p1.1)\.
- \[11\]F\. N\. Manhary, M\. H\. Mohamed, and M\. Farouk\(2025\)A scalable machine learning strategy for resource allocation in database\.Scientific Reports15\.External Links:[Document](https://dx.doi.org/10.1038/s41598-025-14962-5),ISSN 20452322Cited by:[§2\.4](https://arxiv.org/html/2610.21008#S2.SS4.p1.1)\.
- \[12\]V\. Mnih, A\. P\. Badia, M\. Mirza, A\. Graves, T\. Harley, T\. P\. Lillicrap, D\. Silver, and K\. Kavukcuoglu\(2016\)Asynchronous methods for deep reinforcement learning\.InProceedings of the 33rd International Conference on Machine Learning \(ICML 2016\),Proceedings of Machine Learning Research, Vol\.48,pp\. 1928–1937\.External Links:[Link](http://proceedings.mlr.press/v48/mniha16.html)Cited by:[§2\.3](https://arxiv.org/html/2610.21008#S2.SS3.p2.1)\.
- \[13\]V\. Mnih, K\. Kavukcuoglu, D\. Silver, A\. A\. Rusu, J\. Veness, M\. G\. Bellemare, A\. Graves, M\. Riedmiller, A\. K\. Fidjeland, G\. Ostrovski, S\. Petersen, C\. Beattie, A\. Sadik, I\. Antonoglou, H\. King, D\. Kumaran, D\. Wierstra, S\. Legg, and D\. Hassabis\(2015\)Human\-level control through deep reinforcement learning\.Nature518\(7540\),pp\. 529–533\.External Links:[Document](https://dx.doi.org/10.1038/nature14236)Cited by:[§2\.3](https://arxiv.org/html/2610.21008#S2.SS3.p2.1)\.
- \[14\]A\. Raffin, A\. Hill, A\. Gleave, A\. Kanervisto, M\. Ernestus, and N\. Dormann\(2021\)Stable\-baselines3: reliable reinforcement learning implementations\.Journal of Machine Learning Research22\(268\),pp\. 1–8\.External Links:[Link](http://jmlr.org/papers/v22/20-1364.html)Cited by:[§3](https://arxiv.org/html/2610.21008#S3.p1.1)\.
- \[15\]Y\. Sanjalawe, S\. Al\-E’mari, S\. Fraihat, and S\. Makhadmeh\(2025\)AI\-driven job scheduling in cloud computing: a comprehensive review\.Artificial Intelligence Review58\(7\)\.Note:Article number not specified in online\-first publicationExternal Links:[Document](https://dx.doi.org/10.1007/s10462-025-11208-8)Cited by:[§1](https://arxiv.org/html/2610.21008#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.21008#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2610.21008#S2.SS2.p1.1)\.
- \[16\]P\. Sankaran, S\. K\. Lingishetty, and M\. Kumar\(2025\)Leveraging reinforcement learning for efficient task scheduling in multi\-cloud environments\.International Journal of Innovative Research in Computer Science and Technology13\(2\),pp\. 35–41\.External Links:[Document](https://dx.doi.org/10.55524/ijircst.2025.13.2.6)Cited by:[§1](https://arxiv.org/html/2610.21008#S1.p2.1)\.
- \[17\]J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov\(2017\)Proximal policy optimization algorithms\.External Links:[Link](http://arxiv.org/abs/1707.06347)Cited by:[§2\.3](https://arxiv.org/html/2610.21008#S2.SS3.p1.1)\.
- \[18\]A\. K\. Sharma and J\. Kunkel\(2025\)GrapheonRL: a graph neural network and reinforcement learning framework for constraint and data\-aware workflow mapping and scheduling in heterogeneous HPC systems\.Note:arXiv preprint arXiv:2506\.00260External Links:[Link](http://arxiv.org/abs/2506.00260)Cited by:[§1](https://arxiv.org/html/2610.21008#S1.p2.1)\.
- \[19\]A\. Silberschatz, P\. B\. Galvin, and G\. Gagne\(2018\)Operating system concepts\.10th edition,Wiley,Hoboken, NJ\.External Links:ISBN 978\-1\-119\-32091\-3Cited by:[§2\.2](https://arxiv.org/html/2610.21008#S2.SS2.p1.1)\.
- \[20\]R\. Stolz, H\. Krasowski, J\. Thumm, M\. Eichelbeck, P\. Gassert, and M\. Althoff\(2024\)Excluding the irrelevant: focusing reinforcement learning through continuous action masking\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 95067–95094\.External Links:[Document](https://dx.doi.org/10.52202/079017-3013),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/acf4a08f67724e9d2de34099f57a9c25-Paper-Conference.pdf)Cited by:[§2\.3](https://arxiv.org/html/2610.21008#S2.SS3.SSSx1.p1.1)\.
- \[21\]D\. Subramoney and C\. N\. Nyirenda\(2022\)Multi\-swarm PSO algorithm for workflow scheduling in cloud\-fog environments\.IEEE Access10,pp\. 117199–117214\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2022.3220239)Cited by:[§2\.2](https://arxiv.org/html/2610.21008#S2.SS2.p1.1)\.
- \[22\]R\.S\. Sutton and A\.G\. Barto\(2018\)Reinforcement learning, second edition: an introduction\.Adaptive Computation and Machine Learning series,MIT Press\.External Links:ISBN 9780262352703,[Link](https://books.google.co.za/books?id=uWV0DwAAQBAJ)Cited by:[§2\.3](https://arxiv.org/html/2610.21008#S2.SS3.p1.1)\.
- \[23\]H\. Wang, X\. Chen, K\. Zhang, and R\. H\. Smith\(2024\)Dynamic multi\-objective flexible job shop scheduling: a three\-phase deep reinforcement learning approach\.Technical reportExternal Links:[Link](https://ssrn.com/abstract=5312558)Cited by:[§2\.4](https://arxiv.org/html/2610.21008#S2.SS4.p1.1)\.
- \[24\]L\. Wang, M\. A\. Rodriguez, and N\. Lipovetzky\(2025\)MetaPilot: a drl\-based controller for dynamic adaptation to shifting scheduling objectives in hpc systems\.Future Generation Computer Systems178,pp\. 108269\.External Links:[Document](https://dx.doi.org/10.1016/j.future.2025.108269),ISSN 0167739X,[Link](https://linkinghub.elsevier.com/retrieve/pii/S0167739X25005631)Cited by:[§2\.4](https://arxiv.org/html/2610.21008#S2.SS4.p1.1)\.
- \[25\]L\. Wang, M\. A\. Rodriguez, and N\. Lipovetzky\(2025\)Optimizing HPC scheduling: a hierarchical reinforcement learning approach for intelligent job selection and allocation\.Journal of Supercomputing81\(8\)\.Note:Article number not specified in online\-first publicationExternal Links:[Document](https://dx.doi.org/10.1007/s11227-025-07396-3)Cited by:[§1](https://arxiv.org/html/2610.21008#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.21008#S2.SS1.p1.1),[§2\.3](https://arxiv.org/html/2610.21008#S2.SS3.p1.1),[§2\.4](https://arxiv.org/html/2610.21008#S2.SS4.p1.1),[§3\.1](https://arxiv.org/html/2610.21008#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2610.21008#S3.SS1.p2.1),[§3\.2](https://arxiv.org/html/2610.21008#S3.SS2.p1.1),[§3](https://arxiv.org/html/2610.21008#S3.p1.1)\.
- \[26\]R\. Wu, J\. Zheng, and X\. Yin\(2025\)Dynamic scheduling for multi\-objective flexible job shops with machine breakdown by deep reinforcement learning\.Processes13\.External Links:[Document](https://dx.doi.org/10.3390/pr13041246),ISSN 22279717Cited by:[§2\.4](https://arxiv.org/html/2610.21008#S2.SS4.p1.1)\.
- \[27\]A\. B\. Yoo, M\. A\. Jette, and M\. Grondona\(2003\)SLURM: simple linux utility for resource management\.InJob Scheduling Strategies for Parallel Processing \(JSSPP 2003\),Lecture Notes in Computer Science, Vol\.2862,pp\. 44–60\.External Links:[Document](https://dx.doi.org/10.1007/10968987%5F3)Cited by:[§2\.2](https://arxiv.org/html/2610.21008#S2.SS2.p1.1)\.
- \[28\]D\. Zhang, D\. Dai, Y\. He, F\. S\. Bao, and B\. Xie\(2020\)RLScheduler: an automated HPC batch job scheduler using reinforcement learning\.InSC20: International Conference for High Performance Computing, Networking, Storage and Analysis,External Links:[Document](https://dx.doi.org/10.1109/SC41405.2020.00035)Cited by:[§2\.4](https://arxiv.org/html/2610.21008#S2.SS4.p1.1)\.