A Sliding-Window-Based Reinforcement Learning for Dynamic Assembly Flow Shop Scheduling with Multi-Product Delivery

# A Sliding-Window-Based Reinforcement Learning for Dynamic Assembly Flow Shop Scheduling with Multi-Product Delivery
Source: [https://arxiv.org/html/2607.02941](https://arxiv.org/html/2607.02941)
Junhao Qiu, Jianjun Liu, Ting Liu, Rongjie Liao, Zhantao Li, Qingfu ZhangThis research was supported by the National Natural Science Foundation of China \(Grant no: 52375489, 51975129\) and the Guangdong Basic and Applied Basic Research Foundation of China \(Grant no: 2024A1515011211\)\.\(Corresponding author: Jianjun Liu\.\)Junhao Qiu is with the School of Electromechanical Engineering, Guangdong University of Technology, Guangzhou, China, and also with the Department of Computer Science, City University of Hong Kong, Hong Kong \(Email: junhaoqiu2\-c@my\.cityu\.edu\.hk\)\.Jianjun Liu, Ting Liu, Rongjie Liao and Zhantao Li are with the School of Electromechanical Engineering, Guangdong University of Technology, Guangzhou, China \(Email: jianjun\.liu@gdut\.edu\.cn, 1112401015@mail2\.gdut\.edu\.cn, liaorongjie1@mails\.gdut\.edu\.cn, lzt@gdut\.edu\.cn\)\.Qingfu Zhang is with the Department of Computer Science, City University of Hong Kong, Hong Kong \(Email: qingfu\.zhang@cityu\.edu\.hk\)\.

###### Abstract

Multi\-product kitting delivery imposes significant challenges for real\-time scheduling in hybrid manufacturing systems that integrate processing and assembly, as dynamic order arrivals simultaneously alter supply dependencies and the set of feasible job\-machine assignments\. This paper proposes a sliding\-window\-based reinforcement learning \(SWRL\) framework for end\-to\-end online scheduling in the flexible assembly flow shop scheduling problem with complex kitting constraints\. The problem is formulated as a heterogeneous graph\-based Markov decision process that captures the dual\-layer kitting structure and the tail\-product bottleneck dynamics that produce a sparse reward landscape\. To address the resulting challenges, SWRL integrates a sliding\-window filtering mechanism that filters inactive nodes and prioritizes kitting\-critical operations, a spatiotemporal graph encoding network that tracks bottleneck shifts across consecutive decision states, and a dynamic action mapping module with a constrained waiting strategy that adapts to the changing action space under variable topologies\. Experiments on real\-world instances from a home appliance manufacturer demonstrate that SWRL achieves consistent tardiness reductions over classical dispatching rules and existing deep reinforcement learning methods, and exhibits robust performance across varying resource configurations, order loads, and arrival concentrations\.

## IIntroduction

The increasing demand for scenario\-based suite orders has driven hybrid manufacturing systems that integrate processing and assembly toward customer\-centric production models\[[1](https://arxiv.org/html/2607.02941#bib.bib1),[2](https://arxiv.org/html/2607.02941#bib.bib2)\]\. Traditionally, coordination between processing and assembly stages has relied on inventory buffers\[[3](https://arxiv.org/html/2607.02941#bib.bib3)\], but dynamic order arrivals render such static buffers ineffective\. In such systems, uncertainty in order arrivals repeatedly destabilizes planned schedules and degrades delivery performance\. Online scheduling methods that can respond to multi\-stage disruptions and track kitting progress across parallel product flows are therefore essential for the dynamic flexible assembly flow shop scheduling problem with multi\-product delivery \(DFAFSP\-MPD\)\.

The flexible assembly flow shop scheduling problem \(FAFSP\) is an NP\-hard extension of the classical flow shop problem, incorporating part kitting assembly under flexible resource constraints\[[4](https://arxiv.org/html/2607.02941#bib.bib4)\]\. It is widely applied in industries such as home appliances\[[5](https://arxiv.org/html/2607.02941#bib.bib5)\]and automotive\[[6](https://arxiv.org/html/2607.02941#bib.bib6)\]\. The difficulty of DFAFSP\-MPD originates from its dual\-layer kitting structure\. At the product level, assembly kitting requires all component parts to be completed before final assembly\. At the order level, multi\-product delivery requires all products within a single order to be completed simultaneously, such that the order lead time is determined by its tail product\. Under dynamic arrivals, the identity of this tail product shifts with sequence\-dependent setup times and varying processing speeds, producing a moving bottleneck that is difficult to predict\. The resulting decision space requires both job sequencing and machine assignment under flexible processing qualifications, where the tardiness signal is sparse across actions\.

Centralized static methods, including mixed\-integer linear programming\[[7](https://arxiv.org/html/2607.02941#bib.bib7)\]and metaheuristics\[[8](https://arxiv.org/html/2607.02941#bib.bib8),[9](https://arxiv.org/html/2607.02941#bib.bib9)\], provide effective look\-ahead optimization but are computationally infeasible for real\-time decisions at the second or millisecond scale\[[10](https://arxiv.org/html/2607.02941#bib.bib10)\]\. Approximate methods\[[11](https://arxiv.org/html/2607.02941#bib.bib11)\]offer a practical trade\-off for moderate\-scale instances\. Robust scheduling approaches embed disturbance tolerance through time buffers and scenario analysis\[[12](https://arxiv.org/html/2607.02941#bib.bib12)\], while rescheduling methods repair plans after disruptions\[[13](https://arxiv.org/html/2607.02941#bib.bib13),[14](https://arxiv.org/html/2607.02941#bib.bib14)\]\. However, their effectiveness degrades under high\-frequency or large\-scale disturbances, a common condition in online FAFSP environments\[[15](https://arxiv.org/html/2607.02941#bib.bib15),[16](https://arxiv.org/html/2607.02941#bib.bib16)\]\. These limitations highlight the need for adaptive, data\-driven scheduling policies learned directly from system interactions\.

Deep reinforcement learning \(DRL\) has been widely adopted for dynamic scheduling due to its adaptive decision\-making capability\[[17](https://arxiv.org/html/2607.02941#bib.bib17),[18](https://arxiv.org/html/2607.02941#bib.bib18)\]\. One approach uses DRL to select priority dispatching rules \(PDRs\) at each decision step, formulated as a Markov decision process \(MDP\)\[[19](https://arxiv.org/html/2607.02941#bib.bib19),[20](https://arxiv.org/html/2607.02941#bib.bib20)\]\.\[[21](https://arxiv.org/html/2607.02941#bib.bib21)\]applied deep Q\-network \(DQN\) for dispatching rule selection in FJSP, while\[[15](https://arxiv.org/html/2607.02941#bib.bib15),[22](https://arxiv.org/html/2607.02941#bib.bib22)\]extended this to distributed FSP settings\.\[[23](https://arxiv.org/html/2607.02941#bib.bib23)\]further improved generalization through multi\-objective DQN\. PDR\-based methods provide a guaranteed performance lower bound and can be extended to complex shop floors\[[24](https://arxiv.org/html/2607.02941#bib.bib24),[25](https://arxiv.org/html/2607.02941#bib.bib25),[26](https://arxiv.org/html/2607.02941#bib.bib26),[27](https://arxiv.org/html/2607.02941#bib.bib27)\], but their dependence on fixed rule sets limits attainable performance\. An alternative paradigm employs end\-to\-end DRL solvers with graph representations to directly allocate operations and resources\[[28](https://arxiv.org/html/2607.02941#bib.bib28),[29](https://arxiv.org/html/2607.02941#bib.bib29),[30](https://arxiv.org/html/2607.02941#bib.bib30)\]\.\[[28](https://arxiv.org/html/2607.02941#bib.bib28)\]combined GNNs with DRL for JSP, using graph representations for state encoding and proximal policy optimization for scheduling decisions\.\[[31](https://arxiv.org/html/2607.02941#bib.bib31)\]applied a heterogeneous graph neural network to model FJSP states and achieved competitive makespan performance\. This approach was further extended to dynamic job arrivals by\[[32](https://arxiv.org/html/2607.02941#bib.bib32)\]\.\[[10](https://arxiv.org/html/2607.02941#bib.bib10)\]proposed a hierarchical reinforcement learning framework for large\-scale dynamic FJSP, using a high\-level agent to manage job release and lower\-level agents for operation sequencing\. However, these end\-to\-end methods are primarily designed for static graph structures\[[33](https://arxiv.org/html/2607.02941#bib.bib33)\]and do not address the challenges specific to DFAFSP\-MPD, where the graph topology and action space evolve dynamically due to assembly kitting constraints\[[34](https://arxiv.org/html/2607.02941#bib.bib34)\]\.

Applying existing end\-to\-end DRL methods to DFAFSP\-MPD introduces three technical challenges\. 1\)State representation under dual kitting constraints\.Multi\-product delivery couples parallel products through amax\\maxoperator: each order’s tardiness depends only on its tail product\. This produces a stepped reward surface where most job\-machine pairs cause no immediate change in tardiness, impeding the agent’s ability to identify bottleneck\-critical actions\[[35](https://arxiv.org/html/2607.02941#bib.bib35)\]\. Simultaneously, dynamic order arrivals expand the candidate job set, submerging bottleneck\-related signals in noise from irrelevant parallel operations\. 2\)Capturing spatiotemporal correlations in dynamic environments\.The tail product bottleneck is not static, but shifts as sequence\-dependent setup times and processing speeds alter each product’s relative completion progress\. Existing methods that encode the scheduling state as a static graph at each decision step fail to capture the temporal dependencies between consecutive states\[[10](https://arxiv.org/html/2607.02941#bib.bib10)\], making it difficult for the agent to anticipate where kitting pressure will accumulate next\. 3\)Sparse reward under complex constraints\.The dual\-layer kitting structure creates a highly constrained action space where only a limited subset of job\-machine pairs reduces order tardiness at any given step\. Most actions yield zero immediate reward, and each decision step advances only a single job\[[25](https://arxiv.org/html/2607.02941#bib.bib25)\]\. The resulting reward signal is both sparse and delayed, providing limited gradient information for policy optimization, a problem that intensifies with problem scale\.

To address these challenges, this paper proposes a sliding\-window\-based reinforcement learning \(SWRL\) framework\. SWRL integrates three corresponding components: a sliding\-window\-based filtering mechanism that filters inactive nodes to mitigate feature dilution, a spatiotemporal graph encoding network that tracks bottleneck shifts across consecutive decision states, and a dynamic action mapping module with a constrained waiting strategy that adapts to changing action spaces under sparse reward conditions\. The contributions of this research are summarized as follows:

1. 1\.The DFAFSP\-MPD is formulated as a mixed\-integer linear programming model and a heterogeneous graph\-based MDP\. The model captures the dual kitting constraints including assembly\-level part coupling and order\-level product synchronization, as well as the tail\-product bottleneck dynamics that create a sparse and shifting reward landscape\.
2. 2\.A sliding\-window\-based reinforcement learning framework, termed SWRL, is proposed for end\-to\-end online scheduling under variable topologies and action spaces\. A sliding\-window filtering mechanism dynamically filters inactive job nodes and prioritizes kitting\-critical ones as the graph structure evolves\. Node features from temporally correlated graph states are extracted by a spatiotemporal graph neural network for state representation\. A dynamic action mapping module with a constrained waiting strategy is then adopted to align feasible job\-machine pairs under changing action spaces\.
3. 3\.Comprehensive experiments are conducted on both generated and real\-world instances, including parameter sensitivity analysis, ablation studies, comparative evaluation with state\-of\-the\-art DRL\-based scheduling methods, and robustness verification across varying problem scales and arrival patterns\. SWRL consistently outperforms existing approaches in tardiness minimization while requiring substantially fewer training episodes\. Ablation studies further validate the contribution of each designed component to the overall performance\.

The remainder of this paper is organized as follows\. Section[II](https://arxiv.org/html/2607.02941#S2)describes the FAFSP and its heterogeneous graph modeling\. Section[III](https://arxiv.org/html/2607.02941#S3)presents the SWRL framework\. Section[IV](https://arxiv.org/html/2607.02941#S4)presents experimental results and section[V](https://arxiv.org/html/2607.02941#S5)concludes this work\.

## IIProblem Formulation of FAFSP\-MPD

### II\-AProblem Description

DFAFSP\-MPD is derived from a real\-world assembly and processing workshop of home appliance companies\. This processing\-assembly hybrid manufacturing system consists ofMp\{\{M\}\_\{p\}\}unrelated processing machines andMa\{\{M\}\_\{a\}\}assembly lines\. Its problem description is as follows:

1\)Flexible processing capability: The qualifications and processing efficiency of machines at different stages vary when performing the same job\. Switching between jobj1j\_\{1\}and jobj2j\_\{2\}of different products requires a setup time ofS​Tj1,j2S\{\{T\}\_\{\{\{j\}\_\{1\}\},\{\{j\}\_\{2\}\}\}\}\.

2\)Assembly kitting constraint: The products of orderiiare divided into corresponding processing and assembly jobsjjaccording to their structure\. All supporting parts need to be completed before final assembly to ensure the hierarchical coupling constraints of supply\.

3\)Multi\-product delivery: Each orderiihas a specific delivery timeD​TiDT\_\{i\}, containing at least one product\. All products in the order are completely assembled before delivery\.

4\)Dynamic arrival of orders: With orders arriving dynamically, it is necessary to respond quickly to disruptions and adjust the planned schedule, coordinating processing and assembly resources to reduce delays for multi\-product orders\.

The notations used in this article are summarized as follows:

ii::Order index,i∈Ii\\in Ipp::Product index,p∈Pp\\in Pjj::Job index,j1,j2∈Jj\_\{1\},j\_\{2\}\\in Jmm::Machine index,m∈Mm\\in MQj,mQ\_\{j,m\}::1 if machinemmis qualified to process jobjj, 0 otherwisePj1,j2P\_\{j\_\{1\},j\_\{2\}\}::1 if jobj1j\_\{1\}is the immediate predecessor of jobj2j\_\{2\}, 0 otherwiseP​Tj,mPT\_\{j,m\}::Processing time of jobjjon machinemmAp,jA\_\{p,j\}::1 if jobjjis an assembly task for productpp, 0 otherwiseOi,pO\_\{i,p\}::1 if productppbelongs to orderii, 0 otherwiseD​TiDT\_\{i\}::Delivery time of orderiiA​TiAT\_\{i\}::Arrival time of orderiiS​Tj1,j2ST\_\{j\_\{1\},j\_\{2\}\}::Setup time for switching from jobj1j\_\{1\}to jobj2j\_\{2\}VV::A sufficiently large positive constantt​titt\_\{i\}::Tardiness of orderiif​tift\_\{i\}::Completion time of orderiic​tjct\_\{j\}::Completion time of jobjjs​tjst\_\{j\}::Start time of processing for jobjjxj1,j2,mx\_\{j\_\{1\},j\_\{2\},m\}::1 if jobj1j\_\{1\}is processed immediately after jobj2j\_\{2\}on machinemm, 0 otherwise
The goal is to minimize the total tardiness while satisfying the following assumptions and constraints\.

1\) This study does not consider transportation\.

2\) Each machine can process only one job at a time\.

3\) Once the operation has started, no interruption is allowed\.

4\) A job can only proceed to the assembly stage after all its components have been fully completed\.

The objective is to minimize the total tardiness of orders, as defined in[1](https://arxiv.org/html/2607.02941#S2.E1)\.

min​∑i=1I\(t​ti\)\\min\\mathop\{\\sum\}\_\{i=1\}^\{I\}\\left\(\{tt\}\_\{i\}\\right\)\(1\)
Order delays for tail products: The tardiness of ordersiivaries drastically, with the final delay depending on the completion time of the last product,f​ti=max∀p,j⁡\(c​tj×Ap,j×Oi,p\)ft\_\{i\}=\\max\\limits\_\{\\forall p,j\}\(ct\_\{j\}\\times A\_\{p,j\}\\times O\_\{i,p\}\)\. Constraints can be represented as[2](https://arxiv.org/html/2607.02941#S2.E2)and[3](https://arxiv.org/html/2607.02941#S2.E3)\.

f​ti≥c​tj\+V×\(Ap,j\+Oi,p−2\),\\displaystyle ft\_\{i\}\\geq ct\_\{j\}\+V\\times\\left\(A\_\{p,j\}\+O\_\{i,p\}\-2\\right\),\(2\)∀i∈I,p∈P,j∈J\\displaystyle\\forall i\\in I,p\\in P,j\\in Jt​ti≥f​ti−D​Ti,∀i∈Itt\_\{i\}\\geq ft\_\{i\}\-DT\_\{i\},\\forall i\\in I\(3\)
Flexibility and hierarchical coupling constraint: Flexible processing capabilities are represented as[4](https://arxiv.org/html/2607.02941#S2.E4)by qualification matrix, and hierarchical coupling constraints ensure that the order of processing paths is satisfied as[5](https://arxiv.org/html/2607.02941#S2.E5)\.

xj1,j2,m≤Qj2,m,∀j1∈\[0\]∪J,j2∈J,m∈M\{x\}\_\{j\_\{1\},j\_\{2\},m\}\\leq\{\{Q\}\_\{j\_\{2\},m\}\},\\forall j\_\{1\}\\in\[0\]\\cup J,j\_\{2\}\\in J,m\\in M\(4\)c​tj2≥∑m=1M∑j3=0J\(xj3,j2,m×\(P​Tj2,m\+S​Tj3,j2\)\)\\displaystyle ct\_\{j\_\{2\}\}\\geq\\sum\\limits\_\{m=1\}^\{M\}\{\\sum\\limits\_\{j\_\{3\}=0\}^\{J\}\{\\left\(x\_\{j\_\{3\},j\_\{2\},m\}\\times\\left\(PT\_\{j\_\{2\},m\}\+ST\_\{j\_\{3\},j\_\{2\}\}\\right\)\\right\)\}\}\(5\)\+Pj1,j2×c​tj1,∀j1,j2∈J\\displaystyle\+P\_\{j\_\{1\},j\_\{2\}\}\\times ct\_\{j\_\{1\}\},\\forall j\_\{1\},j\_\{2\}\\in J
Flow and sequence restrictions: Define the workflow and sequence constraints for jobs on the machine as[6](https://arxiv.org/html/2607.02941#S2.E6)and[7](https://arxiv.org/html/2607.02941#S2.E7)\. Note thatj=0j=0represents a virtual order, whilec​tj=0ct\_\{j\}=0\.

∑m=1M∑j1=0Jxj1,j2,m=1,∀j2∈J\\sum\_\{m=1\}^\{M\}\\sum\_\{j\_\{1\}=0\}^\{J\}x\_\{j\_\{1\},j\_\{2\},m\}=1,\\forall j\_\{2\}\\in J\(6\)∑j1=0Jxj1,j2,m−∑j1=0Jxj2,j1,m=0,∀j2∈J,m∈M\\sum\_\{j\_\{1\}=0\}^\{J\}x\_\{j\_\{1\},j\_\{2\},m\}\-\\sum\_\{j\_\{1\}=0\}^\{J\}x\_\{j\_\{2\},j\_\{1\},m\}=0,\\forall j\_\{2\}\\in J,m\\in M\(7\)
Time window limit: The start and end times of the job are constrained as[8](https://arxiv.org/html/2607.02941#S2.E8)\-[10](https://arxiv.org/html/2607.02941#S2.E10)to ensure compliance with processing time and arrival time limits\.

s​tj≥\(Ap,j\+Oi,p−2\)×V\+A​Ti,\\displaystyle st\_\{j\}\\geq\\left\(A\_\{p,j\}\+O\_\{i,p\}\-2\\right\)\\times V\+AT\_\{i\},\(8\)∀i∈I,p∈P,j∈J\\displaystyle\\forall i\\in I,p\\in P,j\\in Js​tj2≤c​tj2−P​Tj2,m\+V×\(1−∑j1=0Jxj1,j2,m\),\\displaystyle st\_\{j\_\{2\}\}\\leq ct\_\{j\_\{2\}\}\-PT\_\{j\_\{2\},m\}\+V\\times\\left\(1\-\\sum\\limits\_\{\{j\_\{1\}\}=0\}^\{J\}\{x\_\{j\_\{1\},j\_\{2\},m\}\}\\right\),\(9\)∀j2∈J,m∈M\\displaystyle\\forall j\_\{2\}\\in J,m\\in Mc​tj2≥c​tj1×P​Tj2,m\+S​Tj1,j2\+V×\(xj1,j2,m−1\),\\displaystyle ct\_\{j\_\{2\}\}\\geq ct\_\{j\_\{1\}\}\\times PT\_\{j\_\{2\},m\}\+ST\_\{j\_\{1\},j\_\{2\}\}\+V\\times\\left\(x\_\{j\_\{1\},j\_\{2\},m\}\-1\\right\),\(10\)∀j1∈\[0\]∪J,j2∈J,m∈M\\displaystyle\\forall j\_\{1\}\\in\[0\]\\cup J,j\_\{2\}\\in J,m\\in M
Thet​ti,f​ti,c​tj,s​tj≥0,∀i∈I,j∈Jtt\_\{i\},ft\_\{i\},ct\_\{j\},st\_\{j\}\\geq 0,\\forall i\\in I,j\\in Jspecify the feasible range of decision variables\.

![Refer to caption](https://arxiv.org/html/2607.02941v1/x1.png)Figure 1:End\-to\-end online scheduling framework of the proposed SWRL, integrating three modules: sliding\-window\-based filtering mechanism, spatiotemporal graph encoding, and dynamic action mapping\.
### II\-BComplexity analysis

The complexity of DFAFSP\-MPD arises from the coupling of job sequencing, machine assignment, and hierarchical kitting constraints\. LetN=∑i=1I∑p=1\|Pi\|\|Ji,p\|N=\\sum\_\{i=1\}^\{I\}\\sum\_\{p=1\}^\{\|P\_\{i\}\|\}\|J\_\{i,p\}\|denote the total number of jobs\. Since the job sequence within each product is fixed[5](https://arxiv.org/html/2607.02941#S2.E5), the effective number of sequencing permutations isY=N\!/∏i,p\(\|Ji,p\|\!\)Y=N\!/\\prod\_\{i,p\}\(\|J\_\{i,p\}\|\!\), accounting for repeated operations within each product\. For machine assignment, each jobjjcan be processed on any qualified machinem∈Mjm\\in M\_\{j\}, yieldingZ=∏j\|Mj\|Z=\\prod\_\{j\}\|M\_\{j\}\|possible assignments\.

The overall search spaceY×ZY\\times Zis compounded by the tail\-product bottleneck dynamics\. While the tardiness of each order is determined by the completion time of its last finishing product \(f​ti=max∀p,j⁡c​tj⋅Ap,j⋅Oi,pft\_\{i\}=\\max\_\{\\forall p,j\}ct\_\{j\}\\cdot A\_\{p,j\}\\cdot O\_\{i,p\}\), the identity of this tail product is not fixed\. Sequence\-dependent setup timesS​Tj1,j2ST\_\{j\_\{1\},j\_\{2\}\}and varying processing speedsP​Tj,mPT\_\{j,m\}cause the relative completion progress across products to shift over time, meaning that any job has the potential to become the critical determinant of its order’s delivery performance\. This creates a decision space where the effective bottleneck moves dynamically across the production horizon\.

### II\-CHeterogeneous Graph

In traditional JSP, multiple directed arcs are commonly used to form start\-to\-end paths representing the processing sequence of different jobs\. Accurately characterizing the job\-machine arc features introduced by flexible processing constraints is challenging, as it can significantly increase graph density\. In this study, FAFSP is formulated as a structure of heterogeneous graphℋ=\(𝒩,ℳ,𝒞,ℰ\)\\mathcal\{H\}=\\left\(\\mathcal\{N\},\\mathcal\{M\},\\mathcal\{C\},\\mathcal\{E\}\\right\),𝒩\\mathcal\{N\}is the set of job nodes, and it still adopts the directed acyclic graph to describe the topology of job nodes,𝒩=\{𝒩j\|∀j\}∪\{S​t​a​r​t,E​n​d\}\\mathcal\{N\}=\\left\\\{\{\{\\mathcal\{N\}\}\_\{j\}\}\|\\forall j\\right\\\}\\cup\\left\\\{Start,End\\right\\\}\.𝒞\\mathcal\{C\}is the set of connected arcs of jobs, which describes the processing sequence of all jobs in terms of directed connected arcs\.ℰ\\mathcal\{E\}is the arc set of job\-machine pairs introduced by considering flexible processing characteristics, whereℰn1,n2∈ℰ\{\{\\mathcal\{E\}\}\_\{n\_\{1\},n\_\{2\}\}\}\\in\\mathcal\{E\}is an undirected arc connecting different jobs and compatible machine nodesmm\. Select a job\-machine arc for each node and determine its direction to construct the scheduling scheme for FAFSP\.

## IIIMethod

This section presents the SWRL framework for online scheduling under variable topologies and action spaces, as illustrated in Fig\.[1](https://arxiv.org/html/2607.02941#S2.F1)\. Solving online FAFSP requires adaptively selecting job\-machine pairs at each decision step\. The framework comprises three modules that operate sequentially at each step\. First, the sliding\-window filtering mechanism \(Section[III\-B1](https://arxiv.org/html/2607.02941#S3.SS2.SSS1)\) filters inactive job nodes and maintains a fixed\-size set of kitting\-critical candidate nodes\. Second, the spatiotemporal graph encoder \(Section[III\-B2](https://arxiv.org/html/2607.02941#S3.SS2.SSS2)\) extracts features from the filtered graph state, capturing both spatial dependencies and temporal evolution across consecutive decision steps\. Third, the dynamic action mapping module \(Section[III\-B3](https://arxiv.org/html/2607.02941#S3.SS2.SSS3)\) aligns the policy output to feasible job\-machine pairs and executes the selected action\. The MDP formulation defines the state, action, reward, and transition dynamics\.

### III\-AMDP Formulation

#### III\-A1State

The statests\_\{t\}at decision stepttis represented as a heterogeneous graphℋt=\(𝒯t,ℳt,𝒞t,ℰt\)\\mathcal\{H\}\_\{t\}=\(\\mathcal\{T\}\_\{t\},\\mathcal\{M\}\_\{t\},\\mathcal\{C\}\_\{t\},\\mathcal\{E\}\_\{t\}\), where𝒯t\\mathcal\{T\}\_\{t\}andℳt\\mathcal\{M\}\_\{t\}are the job and machine node feature sets, and𝒞t\\mathcal\{C\}\_\{t\}andℰt\\mathcal\{E\}\_\{t\}are the arc sets representing job dependencies and job\-machine compatibility, respectively\. The node features are summarized in Table[I](https://arxiv.org/html/2607.02941#S3.T1)\.

TABLE I:Node featuresThe arc set𝒞t\\mathcal\{C\}\_\{t\}encodes the immediate supply relationships between jobs, wheref𝒞j1,j2=1f\_\{\\mathcal\{C\}\_\{j\_\{1\},j\_\{2\}\}\}=1if the processing condition betweenj1j\_\{1\}andj2j\_\{2\}is satisfied at steptt, and 0 otherwise\. The arc setℰt\\mathcal\{E\}\_\{t\}encodes job\-machine pairs, withfℰj,mf\_\{\\mathcal\{E\}\_\{j,m\}\}storing the processing time of jobjjon machinemm\. Both arc sets are recomputed at each step to reflect changes caused by job completions and new arrivals\.

#### III\-A2Action

The actionsat∈Ata\_\{t\}\\in A\_\{t\}at stepttare an arc set consisting feasible job\-machine pairs\(j,m\)\(j,m\)and a waiting actionaw​a​i​ta\_\{wait\}\. Waiting refers to the decision not to execute any action\(j,m\)\(j,m\)attt, with idle machine resources remaining in a state of readiness until the next event triggers a decision\[[24](https://arxiv.org/html/2607.02941#bib.bib24)\]\. All the immediately preceding jobs of jobj∈Jj\\in Jhave been completed and satisfy the qualification requirements of machinemmwhilem∈Mm\\in Mis in an idle state\. Also,AtA\_\{t\}is related to the decision step and needs to be dynamically processed based on the arrival time and supply relationship\. Meanwhile, since there are qualification and stage differences among different machines, thenJm<MJ\_\{m\}<M, so\|At\|<j×m\|A\_\{t\}\|<j\\times m\. Applying the wait action enables the agent to refrain from executing\(j,m\)\(j,m\)pairs, reserving processing capacity for urgent orders and expanding the agent’s pre\-decision capabilities\. To avoid the dilemma of continuous waiting, a waiting action restriction strategy that considers topology\-variable is proposed \(Section[III\-B3](https://arxiv.org/html/2607.02941#S3.SS2.SSS3)\)\.

#### III\-A3Reward

Reward function is structured based on delayed performance and penalties derived from balancing long\-term and short\-term perspectives\. Thec​ti,tct\_\{i,t\}is last processing time of orderiiat decision steptt, andΔ​c​ti,t\\Delta ct\_\{i,t\}denotes the additional delay at the current step at decision steptt, calculated as:

Δ​c​ti,t=max⁡\(c​ti,t−D​Ti,0\)−max⁡\(c​ti,t−1−D​Ti,0\)\.\\Delta ct\_\{i,t\}=\\max\(ct\_\{i,t\}\-DT\_\{i\},0\)\-\\max\(ct\_\{i,t\-1\}\-DT\_\{i\},0\)\.\(11\)
Theδ\\deltais the time step penalty coefficient, satisfyingδ\>0\\delta\>0\.TtT\_\{t\}denotes the time of the current decision step,Tt−1T\_\{t\-1\}denotes the time of the previous decision step, andTt−Tt−1T\_\{t\}\-T\_\{t\-1\}denotes the time interval betweentt\-11andtt\. The reward is calculated:

rt=−∑i∈ItallΔ​c​ti,t−δ​\(Tt−Tt−1\)\.r\_\{t\}=\-\\sum\_\{i\\in I^\{\\text\{all\}\}\_\{t\}\}\\Delta ct\_\{i,t\}\-\\delta\(T\_\{t\}\-T\_\{t\-1\}\)\.\(12\)
Since the tardiness of all orders isT=∑i=1I\(t​ti\)T=\\sum\_\{i=1\}^\{I\}\(tt\_\{i\}\), the absolute value of the total reward is equal to this quantity\. This implies that the reward function effectively distributes the total order tardiness across the decision\-making process\. The coefficientδ\\deltaintroduces a secondary penalty on the elapsed time between decisions, which implicitly discourages unnecessarily long processing sequences\. The sensitivity of this trade\-off between tardiness and throughput is examined in the orthogonal experiment \(Section[IV\-B](https://arxiv.org/html/2607.02941#S4.SS2)\)\.

#### III\-A4Transfer

After actionata\_\{t\}is executed, the simulation advances to the next decision step\. Each decision step is triggered by the earliest event in which either a machine completes its current job and becomes idle, or a new order arrives\. At that point, the environment state is updated: completed job nodes and their incident edges are removed from the heterogeneous graph, new nodes from arriving orders are inserted, and the edge features are recomputed to reflect the updated processing conditions\. This transition produces a new statest\+1s\_\{t\+1\}with an updated action setAt\+1A\_\{t\+1\}\.

![Refer to caption](https://arxiv.org/html/2607.02941v1/x2.png)Figure 2:Construction of end\-to\-end agent variable topology inputs\. It supports the normalization of variable inputs and extracts spatiotemporal features from decision trajectories\.

### III\-BAgent Architecture

The SWRL agent architecture is illustrated in Fig\.[2](https://arxiv.org/html/2607.02941#S3.F2)\. The filtered graph from the sliding\-window filtering mechanism is processed by the spatiotemporal encoder, and the resulting embedding is passed to the policy network for action selection\.

#### III\-B1Sliding\-Window Filtering Mechanism

As orders arrive dynamically, the total number of job nodes in the heterogeneous graph grows monotonically\. Encoding all nodes equally dilutes the features of kitting\-critical nodes among a large number of irrelevant parallel operations, impeding the agent’s ability to identify bottleneck jobs\. The sliding\-window filtering mechanism addresses this by maintaining a fixed\-size active node setNtwN\_\{t\}^\{w\}\(the window\)\.

As illustrated in Fig\.[3](https://arxiv.org/html/2607.02941#S3.F3), each job node becomes active when its predecessors are completed and is removed from the window once finished\. The vacated slot is reassigned to a pending node with the highest in\-degreeI​DnID\_\{n\}measured within the conjunctive arc set𝒞t\\mathcal\{C\}\_\{t\}\.

![Refer to caption](https://arxiv.org/html/2607.02941v1/x3.png)Figure 3:Dynamic filtering and node capture process\. At timeΔ​t\\Delta t, the window filters out completed nodes and captures newly available nodes based on in\-degree priority\.When a node exits the window, its associated job\-machine pairs are removed from the action space, and the newly admitted node’s pairs are inserted\. This filtering reduces the action space dimension from the full set of all candidate pairs to a focused set of currently relevant decisions\.

The complete window update and action mapping process is given in Algorithm[1](https://arxiv.org/html/2607.02941#alg1)\. The window update proceeds in two phases\. First, all finished nodes are identified and their job\-machine pairs are removed from the action space \(lines 4\-14\)\. A finished node is only evicted after all its predecessor jobs have been completed, ensuring that no kitting\-critical intermediate state is discarded prematurely\. Second, each vacated slot in the eviction phase is filled by a pending node with the minimum in\-degree among all candidates outside the current window \(lines 15\-24\)\. The in\-degree is measured within the conjunctive arc set𝒞t\\mathcal\{C\}\_\{t\}, where lower in\-degree indicates that fewer prerequisites remain before the job becomes schedulable\. The condition on line 21 checks that the candidate node brings enough feasible job\-machine pairs to fill the available action slots; if it does not, the slot is left temporarily empty and will be filled at the next decision step when more nodes become available\.

#### III\-B2Spatiotemporal Graph Encoding

The sliding\-window filtering mechanism produces a filtered subgraph that captures the current spatial structure of active jobs and machines\. However, the tail product bottleneck shifts over time as setup times and processing speeds alter each order’s relative completion progress\. A static graph encoding that considers only the current snapshot does not capture this temporal evolution\.

To address this, the filtered node and edge features are processed through a spatiotemporal encoding network composed ofLLstacked layers, each containing a graph attention \(GAT\) module for spatial aggregation followed by a gated recurrent unit \(GRU\) for temporal fusion\.

The node features𝒩n,t\\mathcal\{N\}\_\{n,t\}at stepttare first projected into a unified dimension\. For any two connected nodes, the attention coefficient is computed as:

βn1,n2=LeakyReLU​\(ηT​\[Wf​𝒩n1,t,Wf​𝒩n2,t\]\),\\beta\_\{n\_\{1\},n\_\{2\}\}=\\text\{LeakyReLU\}\\left\(\\eta^\{T\}\\left\[W\_\{f\}\\mathcal\{N\}\_\{n\_\{1\},t\},W\_\{f\}\\mathcal\{N\}\_\{n\_\{2\},t\}\\right\]\\right\),\(13\)
whereη\\etais a learnable vector andWfW\_\{f\}is a projection matrix\. The raw attention scores are then gated by the edge featuresf\(n,n′\)f\_\{\(n,n^\{\\prime\}\)\}\(which isf𝒞f\_\{\\mathcal\{C\}\}for job\-job edges orfℰf\_\{\\mathcal\{E\}\}for job\-machine edges\) and normalized via softmax across each node’s neighborhood\. The normalized coefficientαn,n′=softmax​\(βn,n′⋅f\(n,n′\)\)\\alpha\_\{n,n^\{\\prime\}\}=\\mathrm\{softmax\}\(\\beta\_\{n,n^\{\\prime\}\}\\cdot f\_\{\(n,n^\{\\prime\}\)\}\)allows the network to assign different importance to different neighbors depending on both their features and the type of relation \(supply dependency or machine compatibility\)\.

1

Input :Action space

AtJ​MA\_\{t\}^\{JM\}, set of invalid action indexes

Ate​m​p​t​yA\_\{t\}^\{empty\}at

tt, set of \(

jj,

mm\) pairs

J​MnJM\_\{n\}and predecessor node set of node

nnis

NnsN\_\{n\}^\{s\}\.

output :Action space

AtA\_\{t\}and state

ℋt\{\\mathcal\{H\}\}\_\{t\}at step

tt\.

2

31ex

4Get node space

NtwN\_\{t\}^\{w\}in the window, set of completed job nodes

Ntn​fN\_\{t\}^\{nf\}and filtered nodes

NtdN\_\{t\}^\{d\}at step

tt;

5foreach*nf​i​n​i​s​h∈Ntn​f\{n\}\_\{finish\}\\in N\_\{t\}^\{nf\}*do

6

7Get the Predecessor node set

Nnf​i​n​i​s​hsN\_\{n\_\{finish\}\}^\{s\};

8

9if*All jobs inNnf​i​n​i​s​hsN\_\{\{\{n\}\_\{finish\}\}\}^\{s\}have been completed*then

10Add

nf​i​n​i​s​hn\_\{finish\}to

NtdN\_\{t\}^\{d\}and its corresponding index

nin^\{i\}in

Ntn​fN\_\{t\}^\{nf\}to

𝒯tw,e\\mathcal\{T\}\_\{t\}^\{w,e\};

11

12Remove the invalid actions contained in the set

J​Mnf​i​n​i​s​hJM\_\{n\_\{finish\}\}from

AtJ​MA\_\{t\}^\{JM\};

13

14Add all removed invalid action indexes at

Ate​m​p​t​yA\_\{t\}^\{empty\}and

nf​i​n​i​s​hn\_\{finish\}in

Ntn​fN\_\{t\}^\{nf\};

15

16

17if*set of invalidated node indices𝒯tw,e≠∅\\mathcal\{T\}\_\{t\}^\{w,e\}\\neq\\varnothing*then

18foreach*Indicen​xi∈𝒯tw,enx^\{i\}\\in\\mathcal\{T\}\_\{t\}^\{w,e\}*do

19Find node

nn​e​x​tn\_\{next\}and the corresponding

J​Mnn​e​x​tJM\_\{n\_\{next\}\}based on the minimum

I​DnID\_\{n\};

20

21if*l​e​n​\(J​Mnn​e​x​t\)≥l​e​n​\(Ate​m​p​t​y\)len\(JM\_\{n\_\{next\}\}\)\\geq len\(A\_\{t\}^\{empty\}\)*then

22

23

𝒯tw​\[n​xi\]←nn​e​x​t\\mathcal\{T\}\_\{t\}^\{w\}\[nx^\{i\}\]\\leftarrow n\_\{next\};

24

25Insert the

jj\-

mmpair in

J​Mnn​e​x​tJM\_\{n\_\{next\}\}into

AtJ​MA\_\{t\}^\{JM\}according to

Ate​m​p​t​yA\_\{t\}^\{empty\};

26

27Remove

n​xinx^\{i\}from

𝒯tw,e\\mathcal\{T\}\_\{t\}^\{w,e\}and the index of the inserted action from

Ate​m​p​t​yA\_\{t\}^\{empty\};

28

29

30

31

At←AtJ​M\{\{A\}\_\{t\}\}\\leftarrow A\_\{t\}^\{JM\}and update

ℋt\{\\mathcal\{H\}\}\_\{t\};

Algorithm 1Sliding\-window\-based action mapping \.At layerll, the GAT module produces node embeddingshn,t\(l\)h\_\{n,t\}^\{\(l\)\}by attending over neighbors\. These embeddings are then passed to a GRU that merges them with the hidden statehn,t−1\(l\)h\_\{n,t\-1\}^\{\(l\)\}from the previous decision step, following the standard GRU formulation\. This temporal connection allows the network to track how each node’s completion progress evolves across consecutive states\.

AfterLLlayers, a global embedding for decision\-making is obtained by pooling the final node representations:

htglobal=concat​\(1\|Bj\|​∑j∈Bjhj,t′⁣\(L\),1\|Bh\|​∑n∈Bhhn,t′⁣\(L\)\),h^\{\\text\{global\}\}\_\{t\}=\\text\{concat\}\\left\(\\frac\{1\}\{\|B\_\{j\}\|\}\\sum\_\{j\\in B\_\{j\}\}h\_\{j,t\}^\{\\prime\(L\)\},\\frac\{1\}\{\|B\_\{h\}\|\}\\sum\_\{n\\in B\_\{h\}\}h\_\{n,t\}^\{\\prime\(L\)\}\\right\),\(14\)
whereBjB\_\{j\}is the set of job nodes andBh=𝒩j∪𝒩mB\_\{h\}=\\mathcal\{N\}\_\{j\}\\cup\\mathcal\{N\}\_\{m\}is the set of all nodes\. The resulting embedding encodes both the spatial structure of the current graph and the temporal trends accumulated over past decision steps\.

#### III\-B3Action Mapping and Transcription

The action space at stepttconsists of the job\-machine pairs within the current window plus a waiting actiona0a\_\{0\}\. As the sliding window updates, the set of feasible pairs changes: pairs corresponding to completed jobs are removed, and new pairs become available when new nodes enter the window\. To maintain a fixed policy output dimension despite this changing set, a mapping tableℳt:k→\(j,m\)\\mathcal\{M\}\_\{t\}:k\\rightarrow\(j,m\)is maintained\. When a job\-machine pair is removed, its indexkkis immediately reassigned to a newly available pair, a process referred to as index recycling, as illustrated in Fig\.[4](https://arxiv.org/html/2607.02941#S3.F4)\.

![Refer to caption](https://arxiv.org/html/2607.02941v1/x4.png)Figure 4:Dynamic mapping between action indices and job\-machine pairs\. Action indices are reassigned as jobs complete and new nodes enter the window\.The waiting actiona0a\_\{0\}allows the agent to defer a decision, reserving capacity for future urgent arrivals\. To prevent indefinite waiting, the number of consecutive waiting actions is constrained bya0w≤jtouta\_\{0\}^\{w\}\\leq j\_\{t\}^\{\\text\{out\}\}, wherejtoutj\_\{t\}^\{\\text\{out\}\}is the number of pending nodes outside the current window\. This ensures that after at mostjtoutj\_\{t\}^\{\\text\{out\}\}waiting steps, a new node must enter the window and force a processing decision\.

Given the global embeddinghtglobalh\_\{t\}^\{\\text\{global\}\}, the policy network outputs a mean vectorνt\\nu\_\{t\}and a standard deviation vectorσt\\sigma\_\{t\}\. A binary maskAt,kM∈\{0,1\}A\_\{t,k\}^\{M\}\\in\\\{0,1\\\}is applied to restrict actions to feasible pairs\. The masked log probability used for policy gradient computation is:

πθlog=∑k∈AtAt,kM​\(\(at,k−νt,k\)2σt,k2\+log⁡\(2​π​σt,k2\)\)\.\\pi\_\{\\theta\}^\{\\log\}=\\sum\_\{k\\in A\_\{t\}\}A\_\{t,k\}^\{M\}\\left\(\\frac\{\(a\_\{t,k\}\-\\nu\_\{t,k\}\)^\{2\}\}\{\\sigma\_\{t,k\}^\{2\}\}\+\\log\(2\\pi\\sigma\_\{t,k\}^\{2\}\)\\right\)\.\(15\)
The mask is constructed according to the current mapping table and the waiting constraint, as detailed in Algorithm[2](https://arxiv.org/html/2607.02941#alg2)\. Only actions withAt,kM=1A\_\{t,k\}^\{M\}=1contribute to the gradient, ensuring that the policy learns only from feasible decisions\.

Input :Nodes spaces

NtwN\_\{t\}^\{w\}in the window, waiting action

a0a\_\{0\}, number of uncaptured nodes outside the window

jto​u​tj\_\{t\}^\{out\}, policy network

πθ\\pi\_\{\\theta\}\.

output :Action

ata\_\{t\}and the corresponding

jj\-

mmpair

1

21exGet action space

AtA\_\{t\}, current number of waiting action

a0wa^\{w\}\_\{0\}, and initialize the mask

At,aMA\_\{t,a\}^\{M\}corresponding to all actions to 0;

3foreach*a∈Ata\\in A\_\{t\}*do

4if*a=a0a=a\_\{0\}andNtw≠∅N\_\{t\}^\{w\}\\neq\\varnothinganda0w≤jto​u​ta^\{w\}\_\{0\}\\leq j\_\{t\}^\{out\}*then

5

At,a0M←1A\_\{t,a\_\{0\}\}^\{M\}\\leftarrow 1;

6

7else if*Actionaais valid*then

8Get the

jj\-

mmpair represented by action

aa;

9if*jjandmmmeet start requirements*then

10

At,aM←1A\_\{t,a\}^\{M\}\\leftarrow 1;

11

12

13

14Execution action

ata\_\{t\}is obtained from

max⁡\(πθ​\(at\|st\)\)\\max\\left\(\\pi\_\{\\theta\}\(a\_\{t\}\|s\_\{t\}\)\\right\)and the mask

At,aMA\_\{t,a\}^\{M\};

15if*a=a0a=a\_\{0\}*then

16

a0w\+=1\{a^\{w\}\_\{0\}\}\+=1
17else

18Find corresponding

jjand

mmfor

aafrom

AtJ​MA\_\{t\}^\{JM\};

19

Algorithm 2Action transcription mechanism\.The overall scheduling process proceeds as follows\. At each decision steptt, the current heterogeneous graphℋt\\mathcal\{H\}\_\{t\}is first processed by the sliding\-window filtering mechanism to produce a filtered subgraph and action set\. The spatiotemporal encoder then computes the global embeddinghtglobalh\_\{t\}^\{\\text\{global\}\}from the filtered graph, and the policy network outputs an action distribution\. After applying the mask from Algorithm[2](https://arxiv.org/html/2607.02941#alg2), the action with the highest probability is selected and executed, which assigns a specific jobjjto a machinemmor chooses the waiting action\. The simulation then advances to the next event\-driven decision step, and the transition\(st,at,rt,st\+1\)\(s\_\{t\},a\_\{t\},r\_\{t\},s\_\{t\+1\}\)is stored in the replay buffer for training\. This process repeats until all orders are completed\.

The policy is trained using proximal policy optimization with generalized advantage estimation\. The total objective combines the clipped policy loss \(computed over unmasked actions\) with a value estimation error term\. Network hyperparameters and training details are provided in Section[IV\-B](https://arxiv.org/html/2607.02941#S4.SS2)\.

## IVEXPERIMENTAL STUDY

This section describes the experimental setup and presents results on real\-world and extended instances to validate the SWRL framework\. A series of experiments are conducted: parameter sensitivity analysis, training evaluation, ablation study, comparative evaluation, and robustness verification\.

### IV\-AExperimental Setup

#### IV\-A1Evaluation Instances

The real\-world instances are constructed from annual order and product data of a home appliance manufacturer in China, encompassing diverse resource configurations and order demands across multiple factories and time periods\. To compensate for the lack of standardized benchmarks for online FAFSP, additional test instances with varying scales are generated following established practices in dynamic scheduling research\.

The resource allocation is represented by the machine configurationm1×m2m\_\{1\}\\times m\_\{2\}\. Ten generic product structures ranging from simple to complex are selected from actual production data\. The number of productsqqper order follows a geometric distributionP​\(q=k\)=\(1/2\)kP\(q=k\)=\(1/2\)^\{k\}withk∈\[1,2,…\]k\\in\[1,2,\\ldots\]\. The processing speedvvof assembly lines follows a normal distributionv∼𝒩​\(8,2\)v\\sim\\mathcal\{N\}\(8,2\), and the average assembly speed is four\-fifths of the processing stage speed\.

L=max\{\\displaystyle L=\\max\\Biggl\\\{maxm∈P​M⁡\{∑j∈Jppj,m\}\+minm∈A​M⁡\{∑j∈Japj,m\},\\displaystyle\\max\_\{m\\in PM\}\\biggl\\\{\\sum\_\{j\\in J\_\{p\}\}p\_\{j,m\}\\biggr\\\}\+\\min\_\{m\\in AM\}\\biggl\\\{\\sum\_\{j\\in J\_\{a\}\}p\_\{j,m\}\\biggr\\\},\(16\)maxm∈A​M\{∑j∈Japj,m\}\}\\displaystyle\\max\_\{m\\in AM\}\\biggl\\\{\\sum\_\{j\\in J\_\{a\}\}p\_\{j,m\}\\biggr\\\}\\Biggr\\\}R​mi=\(D​Ti−maxm∈M⁡\(pj,m×Bi,j\)\)×d,∀i∈IRm\_\{i\}=\\left\(DT\_\{i\}\-\\max\_\{m\\in M\}\(p\_\{j,m\}\\times B\_\{i,j\}\)\\right\)\\times d,\\quad\\forall i\\in I\(17\)
Following prior studies\[[36](https://arxiv.org/html/2607.02941#bib.bib36),[37](https://arxiv.org/html/2607.02941#bib.bib37),[25](https://arxiv.org/html/2607.02941#bib.bib25)\], order delivery timesD​TiDT\_\{i\}are determined by a uniform distribution over\[L​\(1−T−R/2\),L​\(1−T\+R/2\)\]\\left\[L\(1\-T\-R/2\),L\(1\-T\+R/2\)\\right\], whereLLis the lower bound of completion time from \([16](https://arxiv.org/html/2607.02941#S4.E16)\) and parametersT,R∈\[0,1\]T,R\\in\[0,1\]\. Order arrival times satisfyR​Ti∈\[0,R​mi\]RT\_\{i\}\\in\[0,Rm\_\{i\}\]sampled from a normal distribution\. By varyingd∈\{0\.3,0\.5,0\.7,0\.9\}d\\in\\\{0\.3,0\.5,0\.7,0\.9\\\}, instances with different arrival frequencies are generated to test robustness against disturbances\. The resulting experimental conditions are summarized in Table[V](https://arxiv.org/html/2607.02941#S4.T5)\.

#### IV\-A2Implementation Details

All experiments are run on the same hardware platform: an Intel Core i9\-13900K CPU and an NVIDIA RTX 3060 GPU \(12 GB\)\. The framework is implemented in Python \(3\.8\+\) using NumPy \(1\.24\.3\) and PyTorch \(2\.0\.1\+\)\. The spatiotemporal encoding network usesL=3L=3stacked GAT\-GRU layers with a hidden dimension of 64 and 4 attention heads\. The policy and value networks each consist of two fully connected layers with 128 hidden units\. The initial learning rate is set to3×10−43\\times 10^\{\-4\}and decays over the course of training\. All DRL\-based methods are trained for 2000 episodes per instance set and evaluated on 20 independently generated instances\[[21](https://arxiv.org/html/2607.02941#bib.bib21),[31](https://arxiv.org/html/2607.02941#bib.bib31),[38](https://arxiv.org/html/2607.02941#bib.bib38),[25](https://arxiv.org/html/2607.02941#bib.bib25)\]\. All methods share identical data splits and evaluation protocols to ensure fair comparison\.

### IV\-BParameter Sensitivity Analysis

The SWRL framework introduces several hyperparameters that affect convergence and scheduling performance\. A Taguchi orthogonal experiment is conducted to analyze the sensitivity of five key factors: the PPO clipping thresholdξ\\xi, the time step penalty coefficientδ\\delta, the reward discount factorγ\\gamma, the number of policy replay epochsKepochsK\_\{\\text\{epochs\}\}, and the experience replay ratioμ\\mu\. Each parameter is assigned four levels, as shown in Table[II](https://arxiv.org/html/2607.02941#S4.T2)\.

TABLE II:Orthogonal experiment results for parameter sensitivity•Note: Each parameter is assigned 4 levels with average tardinessTavgT\_\{\\text\{avg\}\}as the response variable\. The optimal level for each parameter is shown in bold\.

Rank\-order analysis of the response ranges indicates that the relative influence on tardiness follows the orderμ\>γ\>Kepochs\>ξ\>δ\\mu\>\\gamma\>K\_\{\\text\{epochs\}\}\>\\xi\>\\delta\. The experience replay ratio and the discount factor are the most influential parameters, while the time step penalty has the smallest impact\. Based on these results, the optimal parameter combinationξ=0\.25\\xi=0\.25,δ=0\.3\\delta=0\.3,γ=0\.97\\gamma=0\.97,Kepochs=4K\_\{\\text\{epochs\}\}=4, andμ=0\.3\\mu=0\.3is adopted in all subsequent experiments\.

### IV\-CTraining Process and Convergence

Fig\.[5](https://arxiv.org/html/2607.02941#S4.F5)shows the training curve of SWRL on 30 real\-world instances using the Adam optimizer\. The cumulative reward increases steadily while the average tardiness decreases, with both metrics stabilizing after approximately 800 episodes\. The convergence trend indicates that the spatiotemporal encoding network is able to learn effective job\-machine assignment strategies from the sparse reward signals introduced by the dual\-layer kitting constraints\.

![Refer to caption](https://arxiv.org/html/2607.02941v1/x5.png)

Figure 5:Training trends of reward and tardiness\. Solid curves show raw data, and lighter curves indicate 40\-point moving averages\.Training variance is high initially due to the large action space\. As the sliding\-window filtering mechanism focuses attention on kitting\-critical nodes, the effective action space shrinks and policy updates stabilize\. After convergence, SWRL achieves consistent performance across repeated runs, confirming training stability\.

### IV\-DAblation Experiment

This section conducts ablation studies on 20 real\-world instances to assess the contribution of each SWRL module: the sliding\-window filtering mechanism, the spatiotemporal graph encoder, and the constrained waiting strategy\. Two baseline methods are included: the EDD dispatching rule, and DQN for adaptive rule selection\. All end\-to\-end variants share the same heterogeneous graph backbone\[[31](https://arxiv.org/html/2607.02941#bib.bib31)\]and are trained for 2,000 episodes\. The module configuration and performance are summarized in Table[III](https://arxiv.org/html/2607.02941#S4.T3)\.

TABLE III:Ablation study: module configuration and performance•Module acronyms: SW = Sliding Window, ST = Spatiotemporal encoder, CW = Constrained Waiting strategy\.✓\\checkmarkindicates the module is enabled\.•CPU times \(ms\) are: EDD 0\.875, DQN 5\.482, backbone 11\.355, \+CW 11\.454, \+ST 12\.284, \+SW 16\.063, SWRL 14\.079\.

Each SWRL module contributes measurable improvements\. The HGNN backbone alone achievesTavg=2233\.9T\_\{\\text\{avg\}\}=2233\.9, already outperforming both DQN and EDD\. Adding the constrained waiting action yields a modest improvement, while the spatiotemporal encoder provides a substantial reduction \(\-15\.1%\), confirming that tracking bottleneck shifts across consecutive decision steps is critical in FAFSP\. The sliding\-window filtering mechanism further reduces tardiness, and the full SWRL achieves the best overall performance withTavg=1689\.2T\_\{\\text\{avg\}\}=1689\.2on all instances andTavg=1653\.2T\_\{\\text\{avg\}\}=1653\.2on the test set\.

Inference time is measured as wall\-clock time from event trigger to action selection\. All end\-to\-end variants complete within 11–16 ms, well below typical manufacturing decision intervals \(seconds to minutes\)\. Notably, SWRL \(14\.079 ms\) runs faster than SW \(16\.063 ms\) despite including additional modules, because the constrained waiting mechanism reduces the candidate action set and lowers the policy network’s computational load\. This confirms that the performance gains of SWRL do not come at the cost of inference efficiency\.

As shown in Fig\.[6](https://arxiv.org/html/2607.02941#S4.F6), SWRL consistently outperforms both the rule\-based EDD and the DQN\-based rule\-selection approach across all evaluation metrics\. Despite the overhead of graph\-structured feature processing, SWRL maintains a practical decision time of 14\.079 ms per step \(Fig\.[7](https://arxiv.org/html/2607.02941#S4.F7)\), confirming its feasibility for real\-time scheduling\.

![Refer to caption](https://arxiv.org/html/2607.02941v1/x6.png)

Figure 6:Performance comparison of SWRL and its ablation variants against EDD and DQN in all instances\.![Refer to caption](https://arxiv.org/html/2607.02941v1/x7.png)

Figure 7:Computation time per decision step across methods\. Despite higher graph\-processing overhead, SWRL remains within the real\-time scheduling requirement\.
### IV\-EComparative Experiments

This section compares SWRL against representative DRL\-based scheduling methods, none of which directly address the dynamic FAFSP with multi\-product delivery\. The rule\-selection baselines include Q\-learning\[[15](https://arxiv.org/html/2607.02941#bib.bib15)\], DQN\[[21](https://arxiv.org/html/2607.02941#bib.bib21)\], DDPG\[[39](https://arxiv.org/html/2607.02941#bib.bib39)\], and MLAC\-DQN\[[25](https://arxiv.org/html/2607.02941#bib.bib25)\]\. The end\-to\-end baselines include HGNN\[[31](https://arxiv.org/html/2607.02941#bib.bib31)\]and DANIEL\[[38](https://arxiv.org/html/2607.02941#bib.bib38)\], both designed for standard FJSP without assembly kitting constraints\. SWRL introduces three domain\-specific improvements: \(i\) a sliding\-window filtering mechanism, \(ii\) a spatiotemporal graph encoder, and \(iii\) a constrained waiting strategy, which are absent in existing end\-to\-end FJSP methods\. The implementation is developed on the basis of open\-source scheduling benchmarks\[[40](https://arxiv.org/html/2607.02941#bib.bib40)\]\. The MDP and training configuration follow\[[25](https://arxiv.org/html/2607.02941#bib.bib25)\], with rule\-selection methods trained for 20,000 iterations and end\-to\-end methods for 2,000 episodes\.

TABLE IV:Comparative results across 20 test instances•W/T/L: instances where SWRL achieves a lower \(W\), equal \(T\), or higher \(L\) tardiness than the column method\. Other methods are trained for 20,000 episodes; SWRL, HGNN and DANIEL for 2,000 episodes\.

As shown in Table[IV](https://arxiv.org/html/2607.02941#S4.T4), Q\-learning and DDPG, despite performing well in continuous action spaces, fail to achieve satisfactory results in online FAFSP, with average tardiness of 2583\.3 and 2759\.5, respectively\. Even with DDPG’s actor\-critic deterministic policy structure, capturing the complex state\-to\-rule mapping remains challenging\. DQN, adept at discrete action spaces, consistently ranks among the top three across most instances\. After 20,000 training epochs, MLAC\-DQN outperforms others on several cases by decomposing multi\-stage rules to expand the solution space and selecting rule combinations across multiple workshops\.

In contrast, SWRL achieves the lowest average tardiness \(1689\.2\) with only 2,000 training episodes, while also exhibiting the lowest standard deviation \(σ=772\.1\\sigma=772\.1\)\. In pairwise comparisons, SWRL outperforms Q\-learning, DQN, DDPG, and HGNN in the majority of instances \(13/20, 12/20, 13/20, and 16/20 wins, respectively\), achieves a comparable 9/11 split against MLAC\-DQN, and also exceeds DANIEL \(16/20 wins\)\. Within the end\-to\-end group, SWRL achieves the best overall performance \(Tavg=1689\.2T\_\{\\text\{avg\}\}=1689\.2\), followed by DANIEL \(Tavg=2167\.9T\_\{\\text\{avg\}\}=2167\.9\), which outperforms the HGNN backbone \(Tavg=2233\.9T\_\{\\text\{avg\}\}=2233\.9\)\. A Friedman test confirms statistically significant differences across all methods \(p=0\.019p=0\.019\)\.

### IV\-FRobustness Verification

The sliding\-window filtering mechanism is designed to filter irrelevant nodes and maintain focus on kitting\-critical operations regardless of the total order volume\. This design is expected to confer robustness across different problem scales and resource configurations\. To verify this, SWRL is evaluated under four experimental conditions that vary in resource availability \(m1×m2m\_\{1\}\\times m\_\{2\}\), number of orders \(itotali\_\{\\text\{total\}\}\), and arrival concentration \(dd\), as described in Table[V](https://arxiv.org/html/2607.02941#S4.T5)\. Each condition includes 40 instances, with 20 used for training and 20 for testing, totaling 160 instances\. SWRL is compared against nine composite dispatching rules commonly used in industrial practice\[[41](https://arxiv.org/html/2607.02941#bib.bib41)\]\.

TABLE V:Robustness comparison across four operating conditions•Each condition contains 40 instances, totaling 160 instances across all conditions\.

The dispatching rules exhibit strong condition\-dependent performance\. LWKR\+SPT achieves the best rule\-level performance in Conditions I and II \(small scale\), while MOPNR\+SPT becomes optimal in Conditions III and IV \(larger scale\), confirming that the best rule shifts with resource availability and order load\. Selecting the appropriate rule for a given operating condition would require expert knowledge of the production environment, and even the best rule lags significantly behind SWRL\.

SWRL consistently achieves the lowest tardiness across all four conditions, with improvements over the best dispatching rule ranging from 7\.7% \(Condition I\) to 18\.8% \(Condition III\)\. Notably, the improvement grows with problem scale, indicating that the sliding\-window filtering mechanism and spatiotemporal encoding become increasingly valuable as the scheduling complexity increases\. This consistent advantage demonstrates that DRL\-based end\-to\-end scheduling adapts to diverse operating environments without manual rule engineering, making it robust for real\-world production settings with fluctuating conditions\.

## VConclusion

This paper proposed a sliding\-window\-based reinforcement learning framework for online scheduling in the dynamic flexible assembly flow shop problem with multi\-product delivery\. The dual\-layer kitting constraints were formalized as a heterogeneous graph\-based MDP that captures the tail\-product bottleneck dynamics and the resulting sparse reward structure\. The SWRL framework integrates three components designed to address the specific challenges of DFAFSP\-MPD: a sliding\-window filtering mechanism that filters inactive nodes and prioritizes kitting\-critical operations under evolving topologies, a spatiotemporal graph encoding network that tracks bottleneck shifts across consecutive decision states, and a dynamic action mapping module with a constrained waiting strategy that handles changing action spaces and sparse feedback\.

Experimental results on real\-world instances from a home appliance manufacturer demonstrated that SWRL achieves consistent improvements over classical dispatching rules and existing DRL methods across varying resource configurations, order loads, and arrival concentrations\. Ablation studies confirmed the contribution of each designed module to the overall performance, and the parameter sensitivity analysis provided guidelines for practical deployment\.

Limitations of the current framework include a fixed sliding\-window size and the assumption of stationary machine availability, which may limit generalization to varying problem scales or sudden breakdowns\. Future work will address these limitations through adaptive window sizing and online fault\-adaptation mechanisms, and will extend the framework to incorporate additional constraints such as automated guided vehicle scheduling and energy\-aware objectives\. Transfer learning across different shop floor configurations and the joint optimization of sliding\-window selection and policy learning are also promising directions\.

## References

- \[1\]Y\. Fang, C\. Peng, P\. Lou, Z\. Zhou, J\. Hu, and J\. Yan, “Digital\-Twin\-Based Job Shop Scheduling Toward Smart Manufacturing,”*IEEE Trans\. Ind\. Informat\.*, vol\. 15, no\. 12, pp\. 6425–6435, 2019\.
- \[2\]J\. Chen, J\. Sun, and G\. Wang, “From unmanned systems to autonomous intelligent systems,”*Engineering*, vol\. 12, pp\. 16–19, 2022\.
- \[3\]C\. Zhao, N\. Kang, J\. Li, and J\. A\. Horst, “Production Control to Reduce Starvation in a Partially Flexible Production\-Inventory System,”*IEEE Trans\. Autom\. Control*, vol\. 63, no\. 2, pp\. 477–491, 2018\.
- \[4\]T\. Liu, J\. Liu, J\. Qiu, C\. Lai, and Z\. Zhang, “Recent advances in assembly flow shop scheduling and its extensions to distributed manufacturing and integrated supply chain,”*Comput\. Ind\. Eng\.*, p\. 111711, 2025\.
- \[5\]R\. Liao, J\. Liu, J\. Qiu, and C\. Peng, “Collaborative optimisation framework for multi\-stage flexible assembly shop scheduling with mixed production pattern,”*Int\. J\. Prod\. Res\.*, pp\. 1–19, 2025\.
- \[6\]C\.\-J\. Liao, C\.\-H\. Lee, and H\.\-C\. Lee, “An efficient heuristic for a two\-stage assembly scheduling problem with batch setup times to minimize makespan,”*Comput\. Ind\. Eng\.*, vol\. 88, pp\. 317–325, 2015\.
- \[7\]C\.\-C\. Wu, X\. Zhang, A\. Azzouz, W\.\-L\. Shen, S\.\-R\. Cheng, P\.\-H\. Hsu, and W\.\-C\. Lin, “Metaheuristics for two\-stage flow\-shop assembly problem with a truncation learning function,”*Eng\. Optim\.*, vol\. 53, no\. 5, pp\. 843–866, 2021\.
- \[8\]Q\.\-Y\. Li, Q\.\-K\. Pan, L\. Wang, L\. Gao, and W\.\-M\. Li, “Dynamic cascaded flow\-shop scheduling using an evolutionary greedy algorithm,”*IEEE Trans\. Evol\. Comput\.*, 2025\.
- \[9\]G\. Zhang, B\. Liu, L\. Wang, D\. Yu, and K\. Xing, “Distributed Co\-Evolutionary Memetic Algorithm for Distributed Hybrid Differentiation Flowshop Scheduling Problem,”*IEEE Trans\. Evol\. Comput\.*, vol\. 26, no\. 5, pp\. 1043–1057, 2022\.
- \[10\]K\. Lei, P\. Guo, Y\. Wang, J\. Zhang, X\. Meng, and L\. Qian, “Large\-scale dynamic scheduling for flexible job\-shop with random arrivals of new jobs by hierarchical reinforcement learning,”*IEEE Trans\. Ind\. Informat\.*, vol\. 20, no\. 1, pp\. 1007–1018, 2023\.
- \[11\]S\. Hatami, R\. Ruiz, and C\. Andrés\-Romano, “Heuristics and metaheuristics for the distributed assembly permutation flowshop scheduling problem with sequence dependent setup times,”*Int\. J\. Prod\. Econ\.*, vol\. 169, pp\. 76–88, 2015\.
- \[12\]X\. Feng, F\. Zheng, and Y\. Xu, “Robust scheduling of a two\-stage hybrid flow shop with uncertain interval processing times,”*Int\. J\. Prod\. Res\.*, vol\. 54, no\. 12, pp\. 3706–3717, 2016\.
- \[13\]D\. Rahmani and R\. Ramezanian, “A stable reactive approach in dynamic flexible flow shop scheduling with unexpected disruptions: A case study,”*Comput\. Ind\. Eng\.*, vol\. 98, pp\. 360–372, 2016\.
- \[14\]P\. Guo, H\. Shi, Y\. Wang, and J\. Xiong, “Multi\-objective scheduling of cloud\-edge cooperation in distributed manufacturing via multi\-agent deep reinforcement learning,”*Int\. J\. Prod\. Res\.*, pp\. 1–25, 2024\.
- \[15\]S\. Yang, J\. Wang, and Z\. Xu, “Real\-time scheduling for distributed permutation flowshops with dynamic job arrivals using deep reinforcement learning,”*Adv\. Eng\. Inf\.*, vol\. 54, p\. 101776, 2022\.
- \[16\]Y\. Li, Q\. Wang, X\. Li, L\. Gao, L\. Fu, Y\. Yu, and W\. Zhou, “Real\-time scheduling for flexible job shop with Agvs using multiagent reinforcement learning and efficient action decoding,”*IEEE Trans\. Syst\., Man, Cybern\., Syst\.*, 2025\.
- \[17\]V\. Mnih, K\. Kavukcuoglu, D\. Silver, A\. A\. Rusu, J\. Veness, M\. G\. Bellemare, A\. Graves, M\. Riedmiller, A\. K\. Fidjeland, G\. Ostrovski*et al\.*, “Human\-level control through deep reinforcement learning,”*Nature*, vol\. 518, no\. 7540, pp\. 529–533, 2015\.
- \[18\]L\. Wang, Z\. Pan, and J\. Wang, “A review of reinforcement learning based intelligent optimization for manufacturing scheduling,”*Complex Syst\. Model\. Simul\.*, vol\. 1, no\. 4, pp\. 257–270, 2021\.
- \[19\]N\. Zhang, Y\. Shen, Y\. Du, L\. Chen, and X\. Zhang, “Counterfactual\-attention multi\-agent reinforcement learning for joint condition\-based maintenance and production scheduling,”*J\. Manuf\. Syst\.*, vol\. 71, pp\. 70–81, 2023\.
- \[20\]L\. Wang, Z\. Pan, and J\. Wang, “A review of Reinforcement Learning Based Intelligent Optimization for Manufacturing Scheduling,”*Complex Syst\. Model\. Simul\.*, vol\. 1, no\. 4, pp\. 257–270, 2021\.
- \[21\]S\. Luo, “Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning,”*Appl\. Soft Comput\.*, vol\. 91, p\. 106208, 2020\.
- \[22\]S\. Yang, J\. Wang, L\. Xin, and Z\. Xu, “Real\-time and concurrent optimization of scheduling and reconfiguration for dynamic reconfigurable flow shop using deep reinforcement learning,”*CIRP J\. Manuf\. Sci\. Technol\.*, vol\. 40, pp\. 243–252, 2023\.
- \[23\]H\. Wang, J\. Cheng, C\. Liu, Y\. Zhang, S\. Hu, and L\. Chen, “Multi\-objective reinforcement learning framework for dynamic flexible job shop scheduling problem with uncertain events,”*Appl\. Soft Comput\.*, vol\. 131, p\. 109717, 2022\.
- \[24\]M\. Wang, J\. Zhang, P\. Zhang, L\. Cui, and G\. Zhang, “Independent double Dqn\-based multi\-agent reinforcement learning approach for online two\-stage hybrid flow shop scheduling with batch machines,”*J\. Manuf\. Syst\.*, vol\. 65, pp\. 694–708, 2022\.
- \[25\]J\. Qiu, J\. Liu, Z\. Li, and X\. Lai, “A multi\-level action coupling reinforcement learning approach for online two\-stage flexible assembly flow shop scheduling,”*J\. Manuf\. Syst\.*, vol\. 76, pp\. 351–370, 2024\.
- \[26\]S\. Luo, L\. Zhang, and Y\. Fan, “Real\-time scheduling for dynamic partial\-no\-wait multiobjective flexible job shop by deep reinforcement learning,”*IEEE Trans\. Autom\. Sci\. Eng\.*, vol\. 19, no\. 4, pp\. 3020–3038, 2021\.
- \[27\]H\. Wang, B\. R\. Sarker, J\. Li, and J\. Li, “Adaptive scheduling for assembly job shop with uncertain assembly times based on dual Q\-learning,”*Int\. J\. Prod\. Res\.*, vol\. 59, no\. 19, pp\. 5867–5883, 2021\.
- \[28\]Z\. Liu, Y\. Wang, X\. Liang, Y\. Ma, Y\. Feng, G\. Cheng, and Z\. Liu, “A graph neural networks\-based deep Q\-learning approach for job shop scheduling problems in traffic management,”*Inf\. Sci\.*, vol\. 607, pp\. 1211–1223, 2022\.
- \[29\]Y\. Li, Q\. Liu, C\. Zhang, X\. Li, and L\. Gao, “Graph\-based dual\-agent deep reinforcement learning for dynamic human–machine hybrid reconfiguration manufacturing scheduling,”*IEEE Trans\. Syst\., Man, Cybern\., Syst\.*, 2025\.
- \[30\]C\.\-L\. Liu and T\.\-H\. Huang, “Dynamic job\-shop scheduling problems using graph neural network and deep reinforcement learning,”*IEEE Trans\. Syst\., Man, Cybern\., Syst\.*, vol\. 53, no\. 11, pp\. 6836–6848, 2023\.
- \[31\]W\. Song, X\. Chen, Q\. Li, and Z\. Cao, “Flexible job\-shop scheduling via graph neural network and deep reinforcement learning,”*IEEE Trans\. Ind\. Informat\.*, vol\. 19, no\. 2, pp\. 1600–1610, 2022\.
- \[32\]Y\. Zhang, H\. Zhu, D\. Tang, T\. Zhou, and Y\. Gui, “Dynamic job shop scheduling based on deep reinforcement learning for multi\-agent manufacturing systems,”*Rob\. Comput\. Integr\. Manuf\.*, vol\. 78, p\. 102412, 2022\.
- \[33\]K\. Lei, P\. Guo, W\. Zhao, Y\. Wang, L\. Qian, X\. Meng, and L\. Tang, “A multi\-action deep reinforcement learning framework for flexible Job\-shop scheduling problem,”*Expert Syst\. Appl\.*, vol\. 205, p\. 117796, 2022\.
- \[34\]J\. Park, Y\. Su, and F\. Ju, “Deep reinforcement learning with dynamic graph pruning for scalable flexible job shop scheduling,”*IISE Trans\.*, no\. just\-accepted, pp\. 1–31, 2026\.
- \[35\]D\. Johnson, G\. Chen, and Y\. Lu, “Multi\-agent continuous decision\-making for the continuous dynamic flexible job shop scheduling problem,”*IEEE Trans\. Autom\. Sci\. Eng\.*, 2026\.
- \[36\]S\. A\. Basir, M\. M\. Mazdeh, and M\. Namakshenas, “Bi\-level genetic algorithms for a two\-stage assembly flow\-shop scheduling problem with batch delivery system,”*Comput\. Ind\. Eng\.*, vol\. 126, pp\. 217–231, 2018\.
- \[37\]A\. Allahverdi, A\. Aydilek, and H\. Aydilek, “Minimizing the number of tardy jobs on a two\-stage assembly flowshop,”*J\. Ind\. Prod\. Eng\.*, vol\. 33, no\. 6, pp\. 391–403, 2016\.
- \[38\]R\. Wang, G\. Wang, J\. Sun, F\. Deng, and J\. Chen, “Flexible job shop scheduling via dual attention network\-based reinforcement learning,”*IEEE Trans\. Neural Netw\. Learn\. Syst\.*, vol\. 35, no\. 3, pp\. 3091–3102, 2023\.
- \[39\]F\. Zhao, S\. Di, and L\. Wang, “A hyperheuristic with Q\-learning for the multiobjective energy\-efficient distributed blocking flow shop scheduling problem,”*IEEE Trans\. Cybern\.*, vol\. 53, no\. 5, pp\. 3337–3350, 2022\.
- \[40\]R\. Reijnen, I\. G\. Smit, H\. Zhang, Y\. Wu, Z\. Bukhsh, and Y\. Zhang, “Job shop scheduling benchmark: Environments and instances for learning and non\-learning methods,”*Ann\. Math\. Artif\. Intell\.*, pp\. 1–31, 2026\.
- \[41\]A\. Baykasoğlu and L\. Özbakır, “Analyzing the effect of dispatching rules on the scheduling performance through grammar based flexible scheduling system,”*Int\. J\. Prod\. Econ\.*, vol\. 124, no\. 2, pp\. 369–381, 2010\.