Reinforcement Learning-Based Laser Cutting Machine Parameter Optimization

# Reinforcement Learning-Based Laser Cutting Machine Parameter Optimization
Source: [https://arxiv.org/html/2608.10549](https://arxiv.org/html/2608.10549)
\[1\]\\fnmTaehong\\surKim

1\]\\orgdivSchool of Information and Communication Engineering,\\orgnameChungbuk National University,\\cityCheongju,\\postcode28644,\\countryKorea

2\]\\orgdivNPS CO\., LTD\.,\\cityCheongju,\\postcode28371,\\countryKorea

###### Abstract

Achieving high accuracy in laser\-based cutting of optical films requires careful tuning of parameters such as focal length and laser power beam, adjusted according to the specific properties of each film type\. Trial\-and\-error based traditional methods are used to find the most suitable cutting parameters for various films, but they are slow and inaccurate\. To address this issue, this paper presents the Reinforcement Learning for Laser Cutting \(RL2C\) algorithm, which uses Q\-learning with an epsilon\-greedy policy to dynamically optimize cutting parameters, significantly reducing taper size and film wastage\. Additionally, RL2C incorporates a dynamic environment space adaptability mechanism to allow it to adapt to new states encountered during the learning process over multiple batches of experiments\. Experimental results demonstrate that RL2C requires fewer steps and less time to find optimal cutting parameters compared to various RL\-based optimization methods\. Specifically, RL2C reduces the number of optimization steps by up to 12\.5% and processing time by up to 81\.8% compared to existing methods\. This study demonstrates the potential of RL in industrial laser\-cutting processes by improving cut quality, reducing time and film wastage, and minimizing manual interventions\.

###### keywords:

Reinforcement Learning, Q\-learning, Parameter Optimization, Dynamic Environment Space\.

## 1Introduction

Laser cutting is one of the key enabling technologies in smart manufacturing, which has gained prominence due to its accuracy and efficiency in manufacturing parts for different applications across industries like electronics, automotive aerospace industries, and others\. This technology is capable of cutting many materials, such as metals, plastics, woods, fabrics, polymers, composites, etc\., with less wastage and high accuracy; that is why it is essential in those sectors where high accuracy and complex cutting operations are required\[Genna2020,SINGH20221021\]\. The overall quality of laser cutting depends mainly on key factors such as laser power, pulse frequency, cutting speed, and focus position\[GHANY2005438\]\. Especially, the process of selecting the optimal cutting parameters, particularly laser power and focal length, becomes even more challenging due to the growing variety of materials and cutting requirements\. For instance, optical films, which can be used in foldable screens and automotive displays, require fine\-tuning of parameters to obtain the optimal cutting quality with the minimum number of defects and unnecessary wastage\[chan2016characterization,mi11030248\]\.

The conventional methods used in the process of identifying the best laser cutting parameters are manual trial\-and\-error methods and simulation models, which are both time\-consuming and inaccurate\. These simulation models often use numerical techniques such as finite element analysis \(FEA\) and computational fluid dynamics \(CFD\) for estimating thermo\-mechanical properties during cutting operations\[sundar2013\]\. Nevertheless, these methods do not consider the relationships of input and output of cutting parameters, especially in dynamic manufacturing environments\[peirovi2017\]\. To overcome these challenges, some more sophisticated machine learning \(ML\) algorithms\[urgun2024optimization,ren2023modeling\]have recently been used to automate the optimization process: artificial neural networks \(ANNs\), metaheuristic algorithms including genetic algorithms, or particle swarm optimization \(PSO\)\. Despite the promising nature of the aforementioned approaches, they have not proven effective in exploring huge parameter spaces, working with a wide range of materials, and managing complex industrial environments\[urgun2024optimization\]\. In particular, ANN\-based models have high overfitting issues while modeling new or unknown materials, and metaheuristic algorithms face difficulties in converging in real\-time environments\[ren2023modeling\]\.

Among the ML approaches, the application of reinforcement learning \(RL\)\[wahab2025\], especially Q\-learning, offers a promising solution for automating the work of laser cutting\. Unlike supervised learning or metaheuristic algorithms, RL enables an agent to learn from its interaction with the environment and improves its actions based on the rewards gained during the process\. Specifically, Q\-Learning can be employed to optimize laser cutting parameters, minimizing the need for manual intervention\. The use of RL enables more flexible, continuous adaptation to new materials and conditions as they arise, offering a distinct advantage over static optimization techniques such as Bayesian optimization\[WAHAB2020609\]or PSO\[pramanik2022experimental\]\.

This paper introduces the RL for laser cutting \(RL2C\) algorithm, a Q\-learning\-based method designed to optimize laser cutting parameters across three critical stages: focus, power, and taper\. RL2C enables the agent to update its knowledge over and over as it meets more materials and cutting conditions\. This adaptive mechanism enhances the algorithm’s efficiency and effectiveness, overcoming the shortcomings of the static optimization methods and enabling real\-time decision\-making in industrial settings\. While the core methodology is based on standard tabular Q\-Learning, our approach features innovations such as stage\-wise optimization and dynamic state\-space updates, enabling the algorithm to address the challenges posed by different optical films and their unique properties\.

The key contributions of this paper are:

- •RL2C Algorithm:We propose a novel Q\-Learning\-based algorithm designed to optimize laser cutting parameters for optical films, integrating a dynamic environment space adaptability mechanism to accommodate real\-time learning and the complexities of material characteristics\.
- •Industrial Application:This study highlights the practical application of the RL2C algorithm in an industrial setting, specifically for laser cutting machines, showcasing its potential to enhance production efficiency and cutting quality\.
- •Comparative Analysis:We present a thorough comparison between RL2C and baseline optimization techniques, such as RL\-based Bayesian Optimization, RL\-based PSO, and Random Search\. The analysis demonstrates the effectiveness of our method in time and step efficiency across multiple optimization stages\. It highlights the superior performance of RL2C in achieving faster convergence and higher precision in parameter selection\.

The rest of this paper is organized as follows: Section[2](https://arxiv.org/html/2608.10549#S2)discusses the related works, providing an overview of existing optimization techniques in laser cutting, including ML and RL approaches\. Section[3](https://arxiv.org/html/2608.10549#S3)presents the overview of the laser cutting process, highlighting material film structures, dataset characteristics, and the optimization stages\. Section[4](https://arxiv.org/html/2608.10549#S4)introduces the proposed RL2C algorithm, detailing its structure, dynamic adaptability mechanism, and implementation steps\. Section[5](https://arxiv.org/html/2608.10549#S5)presents experimental results, including a comparative analysis of RL2C with baseline methods in terms of step and time efficiency\. Finally, Section[6](https://arxiv.org/html/2608.10549#S6)concludes the study, summarizing key findings and discussing potential future research directions\.

## 2Related works

Laser cutting is a widely adopted manufacturing process, but determining the optimal machine parameters remains a significant challenge due to the complex interactions between parameters such as focal length, power, cutting speed, and the material properties of the workpiece\[Behbahani\_2023\]\. Traditional approaches, including trial\-and\-error experiments and simulation\-based methods, have notable limitations\. These methods are often time\-consuming, computationally expensive, and lack the ability to model non\-linear relationships or adapt to dynamic changes in the cutting environment\[Xu2014\]\.

ML techniques\[liu2023review\], particularly ANNs, have emerged as powerful tools for optimizing laser cutting parameters\. ANNs are capable of learning complex, non\-linear relationships between input parameters and output quality metrics, such as line width, brightness, and taper size\. Early studies by\[guo\_dixin\_laser\_2006\]and\[chen2005influence\]used ANNs to quantitatively describe the relationship between cutting quality and parameters in non\-vertical laser cutting scenarios\. Building on this foundation,\[tsai\_optimal\_2008\]integrated ANNs with genetic algorithms, enabling the automatic discovery of optimal parameter settings and improving cutting quality\. Beyond ANNs, hybrid ML approaches\[park2022machine,wang2025traditional,mi2023situ\]have also been explored\.\[tercan\_improving\_2017\]proposed a clustering and classification framework for laser cutting planning\. By first clustering simulation data into performance\-based groups and applying classification trees, they identified parameter regions that maximized cutting efficiency\. However, these methods often suffer from computational overhead and reduced scalability when applied to high\-dimensional parameter spaces or real\-time scenarios\.

In addition to ML, metaheuristic optimization methods have been applied to parameter optimization for laser cutting\. Techniques such as genetic algorithms, PSO, whale optimization algorithm \(WOA\), and ant lion optimization \(ALO\) metaheuristic methods have been utilized to explore the parameter space and identify optimal configurations\.\[urgun2024optimization\]demonstrated the effectiveness of these algorithms in optimizing laser cutter parameters\. While metaheuristic methods provide robust solutions for static problems, they struggle to adapt dynamically to changing environments or materials, limiting their practicality in real\-world applications\.

Recent research has shifted toward RL as a more flexible and scalable solution for parameter optimization\. RL distinguishes itself from supervised learning \(SL\) or unsupervised learning \(USL\) by training through dynamic interactions with the environment rather than relying on static datasets\. This makes RL more flexible, as it can adjust its reward function to suit specific tasks and optimize parameters more effectively\.\[kuprikov2022deep\]employed deep RL to optimize laser cutting parameters, showcasing its ability to achieve higher precision compared to static optimization methods\. Similarly,\[Behbahani\_2023\]applied ML techniques, including RL, to optimize the machining process of alumina ceramics\. Besides,\[chang2024optimization\]explores the optimization of laser annealing parameters using Bayesian RL \(BRL\)\. This study leverages fixed and variable prior knowledge derived from experimental data and technology computer\-aided design \(TCAD\) simulations to guide the RL agent in selecting optimal parameters, such as laser power, repetition rate, and processing temperature\.\[zhang2023q\]proposed a Q\-learning\-based multi\-objective PSO \(QL\-MoPSO\) framework for distributed flow\-shop scheduling problems\. Their hybrid approach balanced exploration and exploitation capabilities, achieving faster convergence and improved diversity in solutions\.

However, existing RL approaches often assume deterministic state transitions and static learning mechanisms\. This means that if a state is not included in the predefined state space during training, the RL agent may struggle to generalize and adapt effectively when encountering unseen conditions\. Building on this foundation, our proposed RL2C algorithm addresses these limitations by introducing a dynamic environment space adaptability mechanism that continuously adapts to new material conditions\. Unlike static RL models, RL2C updates the Q\-table in real time for each environment, allowing the agent to effectively explore and exploit the parameter space across diverse materials and cutting conditions\.

## 3Overview of Laser Cutting Process

This section presents a comprehensive overview of the material film structure, the configuration of the input dataset used for training, the measurement techniques used to evaluate the cutting quality, and the formulation of optimization problems\.

### 3\.1Material Film Structure and Input Dataset Configuration

Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1)illustrates the structure of the material films and the corresponding input dataset configuration used for training the RL2C algorithm\.

![Refer to caption](https://arxiv.org/html/2608.10549v1/x1.png)Figure 1:Material film structure and input dataset configuration for RL2C training\.The bottom part of Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1)presents the data characteristics grouped according to the material\. There are three types of material films: Material A, Material A1, and Material B, with each material film having 70 lots for data collection\. Each lot represents a unique set of experimental conditions applied to that specific material film\. Within each lot, the data are further categorized into three subfolders:focus,power, andtaperwhich correspond to the three optimization stages\. These subfolders store the raw measurement and parameter\-tuning data associated with each stage of the laser\-cutting process\.

To clarify how diverse material films influence the learning behavior of RL2C, the middle part of Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1)outlines the physical layer structure of each material film\. Material A and A1 consist of six layers, including a polarizing film, Pressure\-Sensitive Adhesive \(PSA\), Upper Tri\-Acetyl Cellulose \(TAC\), Polyvinyl Alcohol \(PVA\), Lower TAC, and a protective film, with a total thickness of approximately 290μ\\mum\. Material B, on the other hand, has a simpler structure, consisting of three layers, including a protective film, lower TAC, and PVA, with a total thickness of approximately 230μ\\mum\.

The upper part of Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1)represents the sequential flow of the dataset across the three optimization stages—Focus Optimization,Power Optimization, andTaper Optimization\. It provides a detailed example of data collected from a single lot, showcasing the step\-by\-step progression through each stage of the laser\-cutting process\. Each row in this upper section corresponds to a trial conducted under a specific parameter setting, while the columns capture both the control variables \(e\.g\., focus value or power value\) and the resulting measurements \(e\.g\., line width, brightness, or taper size\) for that stage\. It is important to note that the upper part of Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1)illustrates the configuration and results from one representative lot only; the same structure applies to all 70 lots per material film, but the specific parameter values and measurements differ depending on the material characteristics and experimental variations\. These configurations are directly used in the learning process of RL2C\. Each row in the dataset serves as a state\-action\-reward tuple, which forms the basis for Q\-learning updates across episodes\. Based on this data structure, the specific procedures and data handling approaches for each optimization stage are detailed below, beginning with Stage 1\.

In Stage 1, Focus Optimization aims to identify the focus value that minimizes the line width\. By adjusting the focus value with specific variation sizes \(±\\pm2\.5,±\\pm0\.625,±\\pm0\.156 mm\), the process begins with selecting a range of focus values and testing them iteratively\. Then we measure and record the line width for each focus value setting\. During this process, each focus value is categorized into states: error state for invalid or failed attempts \(e\.g\., line width = \-1\), normal state for acceptable but suboptimal results, and optimal state for the minimum line width value\. When duplicate focus values are present, their corresponding line width values are computed as the mean to ensure the data are accurate and reliable\. For instance, at the end of Stage 1, the optimal focus setting identified is 85\.482, resulting in a minimum line width of 0\.414\.

In Stage 2, by using the optimal focus value obtained from Stage 1 as a fixed input parameter, Power Optimization aims to determine the power value that ensures the laser cut meets a specific brightness threshold\. The process involves adjusting the power incrementally \(e\.g\.,±\\pm1%\) and measuring the resulting brightness for each setting\. Each power value is tested while keeping the focus constant at the optimal setting identified in Stage 1\. We measure and record the brightness, and the power value that achieves the desired brightness is identified as the optimal state \(e\.g\., result = OK\)\. Error states whose brightness equals zero and the result is marked as“NG”\(not good\) \(e\.g\., brightness = 0 and result = NG\), while normal states are the remaining ones \(e\.g\., brightness≠\\neq0 and result = NG\)\. The optimal power value of 49 identified in Stage 2 will be used in the final stage of the optimization process\.

![Refer to caption](https://arxiv.org/html/2608.10549v1/x2.png)Figure 2:Examples of laser marking and cutting\.In Stage 3, Taper Optimization focuses on minimizing the taper size\. Using the optimal focus 85\.482 mm and power value of 49 determined from Stages 1 and 2, the focus is fine\-tuned within a narrow range \(e\.g\., optimal focus from Stage 1±\\pm0\.027 mm\) to minimize the taper size\. Similar to the previous stages, the focus value is incrementally adjusted, and the resulting taper size is measured for each setting\. Moreover, each focus value is also categorized into states: error state for taper size = \-1, optimal state for the smallest taper size, and normal state for the remaining one\. At the end of Stage 3, the optimal focus setting \(85\.618 mm\) that minimizes the taper size is identified and used for production, completing the optimization process\.

This three\-stage structure forms a consistent learning workflow across all lots and materials\. Importantly, the entire pipeline—from data collection to learning—is designed to operate in a fully autonomous manner, with minimal human intervention\. Human involvement is limited to loading the film material into the laser cutting machine and specifying the number of lots to be processed\. Once initiated, the machine automatically executes all cutting procedures sequentially across the three optimization stages—starting with focus, followed by power, and then taper optimization\. For each parameter configuration, the system measures the corresponding cutting quality \(e\.g\., line width, brightness, taper size\) and stores the results in structured CSV files\. These files are formatted in the same structure as shown in the upper part of Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1), including both the parameter values and their measured outcomes\.

Once the generated CSV files from all lots are ready and the environment is initialized, the remaining stages—from state exploration and action selection to Q\-table updates and policy refinement—are fully automated\. The RL2C agent autonomously interacts with the environment, optimizes parameters across episodes, and dynamically adapts to new states using its dynamic Q\-learning mechanism\. This division of responsibilities ensures that human oversight remains minimal and is only required during the initial experimental setup\. After that, the learning and decision\-making processes operate in an autonomous and self\-improving manner\. This makes the RL2C framework both scalable and practical for real\-world manufacturing environments\.

### 3\.2Laser Marking and Cutting Quality Measurements

Fig\.[2](https://arxiv.org/html/2608.10549#S3.F2)presents examples of laser marking and cutting, highlighting key measurement points for assessing the quality of the laser cutting process\. In Fig\.[2](https://arxiv.org/html/2608.10549#S3.F2)a, the surface of the material film①is displayed, with the line width②being measured to evaluate focus optimization\. This line width represents the thickness of the black line on the material surface, and it is a critical parameter for ensuring precise marking\.

In Fig\.[2](https://arxiv.org/html/2608.10549#S3.F2)b, several measurements are shown to optimize power and taper\. The cut width③measures the thickness of the white region in the middle, representing the fully cut area\. Taper 1④and Taper 2⑤refer to the black regions on either side of the cut\. The narrower the black region, the better cutting quality\. Since taper size can be asymmetric, the sum of Taper 1 and Taper 2 is averaged to determine the overall cut quality\.

Several output parameters define the quality of the cut, which are line width, brightness, and taper size\. Line width is related to the width of the cutting line made by the laser beam, and it determines the accuracy of the cutting\. Brightness is the quality of the cut surface and the intensity of the dark spots left on the material after the cut\. Taper size is the difference in the width of the cut between the top and bottom surfaces of the material, affecting the consistency of the cut\. The above\-mentioned output parameters are controlled to achieve high\-quality cutting, although several input parameters like focus and power can affect them\. Focus or focal length refers to the distance between the focusing lens and the material being cut, while power refers to the intensity of the laser beam\.

### 3\.3Problem formulation

The optimization of the laser cutting process involves three key stages, each targeting a specific parameter to improve cutting quality\. The problem is formulated as a sequential optimization process, where each stage refines a parameter to achieve the best results under given constraints\.

#### 3\.3\.1Stage 1: Focus Optimization

Focus Optimization aims to minimize the line width,LwL\_\{w\}, which is a function of the focus value,FF\. The objective function for this stage is:

minF⁡Lw​\(F\)\\min\_\{F\}L\_\{w\}\(F\)\(1\)whereFFrepresents the focus value\. The constraints for this stage are:

- •F∈\{Fstart,Fstart±Δ​F,…,Fend\}F\\in\\\{F\_\{\\text\{start\}\},F\_\{\\text\{start\}\}\\pm\\Delta F,\\ldots,F\_\{\\text\{end\}\}\\\}, whereΔ​F=±2\.5\\Delta F=\\pm 2\.5,±0\.625\\pm 0\.625,±0\.156​mm\\pm 0\.156\\,\\text\{mm\}\.
- •Lw​\(F\)\>0L\_\{w\}\(F\)\>0andLw​\(F\)≠−1L\_\{w\}\(F\)\\neq\-1\(valid line width values\)\.
- •Duplicate focus values are removed, and their corresponding line width values are averaged\.

#### 3\.3\.2Stage 2: Power Optimization

Power Optimization determines the laser power,PP, that meets a specific brightness thresholdBthreshB\_\{\\text\{thresh\}\}\. The optimization objective is expressed as:

minP⁡\(B​\(P\)−Bthresh\)​subject to​B​\(P\)≥Bthresh\\min\_\{P\}\\left\(B\(P\)\-B\_\{\\text\{thresh\}\}\\right\)\\textit\{subject to \}B\(P\)\\geq B\_\{\\text\{thresh\}\}\(2\)
The constraints for this stage include:

- •P∈\{Pstart,Pstart±Δ​P,…,Pend\}P\\in\\\{P\_\{\\text\{start\}\},P\_\{\\text\{start\}\}\\pm\\Delta P,\\ldots,P\_\{\\text\{end\}\}\\\}, whereΔ​P=±1%\\Delta P=\\pm 1\\,\\%\.
- •Only power values whereB​\(P\)≥BthreshB\(P\)\\geq B\_\{\\text\{thresh\}\}are valid\.
- •Result\(P\)=OK\(P\)=\\text\{OK\}for optimal states\.

#### 3\.3\.3Stage 3: Taper Optimization

Taper Optimization minimizes the taper size,TsT\_\{s\}, using the optimal focus and power values from the previous stages\. The optimization objective is:

minF⁡Ts​\(Fopt,Popt\)\\min\_\{F\}T\_\{s\}\(F\_\{\\text\{opt\}\},P\_\{\\text\{opt\}\}\)\(3\)with the constraints:

- •F∈\{Fopt±Δ​F,…,Fopt\+n⋅Δ​F\}F\\in\\\{F\_\{\\text\{opt\}\}\\pm\\Delta F,\\ldots,F\_\{\\text\{opt\}\}\+n\\cdot\\Delta F\\\}, whereΔ​F=±0\.027​mm\\Delta F=\\pm 0\.027\\,\\text\{mm\}\.
- •Ts​\(Fopt,Popt\)\>0T\_\{s\}\(F\_\{\\text\{opt\}\},P\_\{\\text\{opt\}\}\)\>0andTs​\(Fopt,Popt\)≠−1T\_\{s\}\(F\_\{\\text\{opt\}\},P\_\{\\text\{opt\}\}\)\\neq\-1\(valid taper size values\)\.

![Refer to caption](https://arxiv.org/html/2608.10549v1/x3.png)Figure 3:Reinforcement learning framework for laser cutting \(RL2C\) parameter optimization\.
#### 3\.3\.4Overall Objective

The overall optimization can be formulated as a multi\-stage process, where the combined objective is:

minF,P⁡\{Lw​\(F\),\(B​\(P\)−Bthresh\),Ts​\(Fopt,Popt\)\}\\hskip\-7\.11317pt\\min\_\{F,P\}\\left\\\{L\_\{w\}\(F\),\(B\(P\)\-B\_\{\\text\{thresh\}\}\),T\_\{s\}\(F\_\{\\text\{opt\}\},P\_\{\\text\{opt\}\}\)\\right\\\}\(4\)subject to the constraints defined for each stage\.

The three optimization stages above are designed to operate sequentially, with each stage building upon the outcome of the previous one\. Specifically, Stage 2 takes the optimal focus valueFoptF\_\{\\text\{opt\}\}obtained from Stage 1 as a fixed input to determine the corresponding optimal power valuePoptP\_\{\\text\{opt\}\}\. Stage 3 then uses bothFoptF\_\{\\text\{opt\}\}andPoptP\_\{\\text\{opt\}\}as inputs to optimize the taper size\. Importantly, Stage 3 does not explore the entire focus search space evaluated in Stage 1; instead, it performs a search within a narrow range centered aroundFoptF\_\{\\text\{opt\}\}\(e\.g\.,Fopt±Δ​FF\_\{\\text\{opt\}\}\\pm\\Delta F\)\. This design makes Stage 3 inherently dependent on the results of both Stage 1 and Stage 2, while also effectively refining the parameter found in Stage 1 to achieve the best possible final quality\.

## 4Proposed Method

### 4\.1Laser Cutting Parameter Optimization Agent

Laser\-cutting machines are precision\-cutting instruments that utilize high\-power laser beams on the material that is to be cut\. The quality of the cut is primarily determined by output parameters such as line width, brightness, and taper size\. These parameters must be optimized to ensure optimal cutting quality\. Among the key factors influencing these outputs are the input parameters, including focus and power, which directly control the cutting process\.

Fig\.[3](https://arxiv.org/html/2608.10549#S3.F3)presents the architecture of the RL2C framework, which is trained using the structured dataset illustrated in Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1)\. In this framework, the laser\-cutting parameter agent interacts with an environment that simulates the laser\-cutting process\. The environment receives focus and power values as actions \(ata\_\{t\}\) and returns the resulting cutting quality metrics: line width, brightness, and taper size as rewards \(rtr\_\{t\}\), along with the next state \(st\+1s\_\{t\+1\}\)\. This process is structured into three stages: Stage 1 \(Focus Optimization\), Stage 2 \(Power Optimization\), and Stage 3 \(Taper Optimization\)\.

In Stage 1, the agent operates within a static environment space, where the state space remains consistent across all lots\. This stage focuses on optimizing the focus parameter under stable conditions, allowing the agent to refine the focus settings effectively\.

From Stage 2 to Stage 3, the environment turns into a dynamic one, meaning that the state space varies over lots, presenting new challenges for the agent\. In Stage 2, the dynamic environment arises because the optimal power setting required to achieve the desired brightness depends on material\-specific characteristics, such as variations in composition and thickness across lots\. The agent must adjust the power parameter to account for these differences dynamically, ensuring that the brightness threshold is satisfied across lots\.

Algorithm 1RL2C Algorithm1:Input:

α\\alpha= 0\.9,

γ\\gamma= 0\.9,

εm​a​x\\varepsilon\_\{max\}= 1\.0,

εm​i​n\\varepsilon\_\{min\}= 0\.1,

d​e​c​a​y​\_​r​a​t​edecay\\\_rate= 0\.05\.

2:Output:Optimal state

s∗s^\{\*\}, optimal policy

π∗\\pi^\{\*\}, optimal Q\-table

Q∗Q^\{\*\}\.

3:procedureRL AGENT

4:Initialize Q\-Table

QQfor all state\-action pairs\.

5:foreach

m​a​t​e​r​i​a​lmaterialdo

6:foreach

l​o​tlotdo

7:

ε←εm​i​n\+\(εm​a​x−εm​i​n\)​e−d​e​c​a​y​\_​r​a​t​e∗l​o​t\\varepsilon\\leftarrow\\varepsilon\_\{min\}\+\(\\varepsilon\_\{max\}\-\\varepsilon\_\{min\}\)e^\{\-decay\\\_rate\*lot\}
8:foreach

e​p​i​s​o​d​eepisodedo

9:

s′s^\{\\prime\},

rr←\\leftarrowEpsilon\_Greedy\_Policy\(

ε\\varepsilon,

ss\)

10:Update the Q\-values for current state\-action pairs using the Bellman Equation in Eq\.[5](https://arxiv.org/html/2608.10549#S4.E5)\.

11:

s←s′s\\leftarrow s^\{\\prime\};breakif

s′s^\{\\prime\}==

s∗s^\{\*\}or

s′s^\{\\prime\}== Error state

12:endfor

13:Update optimal policy

π∗​\(s\)←arg⁡max𝑎​\(Q​\[s,a\]\)\\pi^\{\*\}\(s\)\\leftarrow\\underset\{a\}\{\\arg\\max\}\(Q\[s,a\]\)\.

14:Q∗Q^\{\*\}←\\leftarrowUPDATE\_STATIC\_Q\_TABLE\(QQ\)⊳\\trianglerightStage 1\.

15:Q∗Q^\{\*\}←\\leftarrowUPDATE\_DYNAMIC\_Q\_TABLE\(QQ\)⊳\\trianglerightStage 2 and 3\.

16:endfor

17:endfor

18:return

π∗\\pi^\{\*\},

Q∗Q^\{\*\}
19:endprocedure

20:procedureEpsilon\_Greedy\_Policy\(

ε\\varepsilon,

ss\)

21:if

r​a​n​d​o​m​\_​u​n​i​f​o​r​m​\(0,1\)<εrandom\\\_uniform\(0,1\)<\\varepsilonthen

22:

a←r​a​n​d​o​m​\_​a​c​t​i​o​n​\(\)a\\leftarrow random\\\_action\(\)
23:else

24:

a←arg⁡max𝑎​\(Q​\[s,a\]\)a\\leftarrow\\underset\{a\}\{\\arg\\max\}\(Q\[s,a\]\)
25:endif

26:

s′s^\{\\prime\}←\\leftarrowss\+

aa
27:

rr←\\leftarrowReward\_Function\(

ss,

aa,

s′s^\{\\prime\},

s∗s^\{\*\}\)⊳\\trianglerightEq\.[6](https://arxiv.org/html/2608.10549#S4.E6)for Stage 1, Eq\.[7](https://arxiv.org/html/2608.10549#S4.E7)for Stage 2, and Eq\.[8](https://arxiv.org/html/2608.10549#S4.E8)for Stage 3\.

28:return

s′s^\{\\prime\},

rr
29:endprocedure

30:procedureUpdate\_Static\_Q\_table\(

QQ\)

31:A new static Q\-table is updated after each lot using Eq\.[5](https://arxiv.org/html/2608.10549#S4.E5)\.

32:endprocedure

33:procedureUpdate\_Dynamic\_Q\_table\(

QQ\)

34:ifnew states existthen

35:foreach new statedo

36:Compute possible next states for new states\.

37:Add more rows in

QQfor new states\.

38:endfor

39:foreach existing state that can reach new statesdo

40:Update

QQvalues where transitions are possible\.

41:endfor

42:endif

43:return

Q∗Q^\{\*\}
44:endprocedure

Similarly, Stage 3 also operates in a dynamic environment, as taper size is influenced by both material properties and the cumulative effects of focus and power parameters selected in the earlier stages\. This stage requires the agent to fine\-tune the focus setting chosen in Stage 1 while using power adjustments made in Stage 2, as well as material\-specific variations in taper behavior\. The evolving state space in these stages reflects real\-world variability, pushing the agent to adapt its actions dynamically for each lot\.

Throughout these stages, the agent learns optimal parameter settings across multiple lots, adjusting its actions based on both static and dynamic environments\. This dynamic adaptability is key to achieving high\-quality laser cutting in real\-world applications, where material properties and conditions can vary significantly across production lots\.

### 4\.2RL2C Algorithm

The RL2C in Algorithm[1](https://arxiv.org/html/2608.10549#alg1)is designed to optimize laser cutting parameters such as focus and power\. It uses Q\-learning to train an agent to identify the best setting for high\-quality cutting for various material films and lots\. In this subsection, the details of the algorithm implementation are discussed, including input and output parameters, the iterative learning process, and its capability to adjust to the static and dynamic changes in the environment during various phases of the optimization\.

The RL agent takes as input a set of hyperparameters\[majid2023\]: the learning rateα\\alpha, the discount factorγ\\gamma, the exploration\-exploitation rateϵ\\epsilon, and the rate of decay of the exploration parameter\. The output of the algorithm is the optimal states∗s^\{\*\}, the optimal policyπ∗\\pi^\{\*\}, and the optimal Q\-tableQ∗Q^\{\*\}, which contains the best laser cutting settings with the corresponding chain of actions required to achieve them\.

The RL2C algorithm begins with the initialization of the environment, which determines the states and actions of the laser\-cutting process\. At this time, the Q\-table is populated with zero values or small random positive values for each state\-action pair\. The agent also initializes its policyπ∗\\pi^\{\*\}, which will guide its actions based on the Q\-values\. The agent is now ready to explore and interact with the environment\.

Each Q\-table is structured as a key\-value map, where each key is a state\-action pair, and the value is the calculated Q\-value\. The state typically is current parameter values \(e\.g\., focus or power\), and the action represents discrete steps to increment or decrement these parameters\. This design enables flexible learning across multiple lots, with the agent progressively refining its knowledge in each stage without interference from others\. Once the environment and Q\-table are set up, the agent starts its learning process when testing with many different materials\. For each lot, the agent must decide whether to explore new actions or exploit the best\-known actions based on its current knowledge\. This decision is made using theε\\varepsilon\-greedy policy, responsible for managing the trade\-off between exploration and exploitation\. At each step, the agent performs an action and obtains a response from the environment in the form of a reward function, which is calculated using line width, brightness, and taper size in Eq\.[6](https://arxiv.org/html/2608.10549#S4.E6), Eq\.[7](https://arxiv.org/html/2608.10549#S4.E7), and Eq\.[8](https://arxiv.org/html/2608.10549#S4.E8), respectively\.

After each action, the Q\-values are updated using the Bellman equation:

Q​\(s,a\)←\(1−α\)​Q​\(s,a\)\+α​\[r\+γ​maxa′⁡Q​\(s′,a′\)\]Q\(s,a\)\\leftarrow\(1\-\\alpha\)Q\(s,a\)\+\\alpha\\left\[r\+\\gamma\\max\_\{a^\{\\prime\}\}Q\(s^\{\\prime\},a^\{\\prime\}\)\\right\]\(5\)
This equation adjusts the Q\-value for the state\-action pair based on both the immediate reward and the expected future rewards from the next state\. After each episode, the agent updates its policyπ∗\\pi^\{\*\}, selecting actions that maximize its expected rewards for each state\. The agent repeats this process until it reaches the optimal states∗s^\{\*\}, encounters an error state, or gets the maximum number of steps to be performed in one episode\. This process helps the agent better understand which of the possible optimal actions will result in the highest reward\.

The RL2C algorithm is developed to work for both static and dynamic environments, which represent the different conditions of the laser cutting process\. One of the significant characteristics of this algorithm is the Update\_Dynamic\_Q\_table\(\) mechanism with which the agent learns to modify the Q\-table in response to new states during training across multiple lots\. As new states arise due to changes in material properties or laser cutting conditions, the agent incorporates them into the Q\-table by adding new rows and updating transitions from existing states\. This ensures that the agent can continue to explore and exploit these newly discovered states, maintaining optimal performance in an evolving environment\. By doing so, the Q\-table is continuously updated to account for these environmental changes, making the RL2C algorithm robust and adaptable to the dynamic nature of real\-world manufacturing scenarios\.

To support stage\-wise learning and maintain separation between optimization objectives, RL2C maintains a separate Q\-table for each stage: one for focus optimization \(Stage 1\), one for power optimization \(Stage 2\), and one for taper optimization \(Stage 3\)\. The Q\-table in Stage 1 operates in a static environment, where the state space is predefined and fixed\. In contrast, the Q\-tables for Stage 2 and Stage 3 are dynamic; their state spaces evolve based on new input\-output observations from each lot, allowing the agent to adapt to changes in material characteristics or cutting conditions\.

Once the learning process is done at all of the stages, it returns to the optimal policyπ∗\\pi^\{\*\}and the final Q\-tableQ∗Q^\{\*\}\. These represent the agent’s learned strategy for achieving the best laser\-cutting results, as well as the expected rewards for each state\-action pair\. At the end of the process, the agent enhances the knowledge about the environment and identifies the optimal actions for every step taken in all lots\.

### 4\.3Detailed Design of Each Stage Using RL2C

This subsection describes how to apply the RL2C algorithm in each state space, action space, and reward function\.

#### 4\.3\.1Stage 1 \- Focus Optimization

- •State Space: The state space is defined by the current focal length of the laser\. Each state represents the system’s configuration at a specific moment during the optimization process, with focus adjustments made across three phases\. As shown in Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1), inPhase 1, the focal length starts at an initial value of 65\.326 mm and is adjusted by 2\.5 mm increments over 17 iterations\. After completing 17 steps, the focus value that satisfies the minimum line width among the results is selected\. InPhase 2, based on the minimum line width value selected inPhase 1scale, optimization is performed again by setting that focus value as the midpoint and incrementing or decrementing by±\\pm0\.625 mm in 17 steps\. Similarly, the focus value that satisfies the minimum line width is selected\. InPhase 3, continuing from the last result inPhase 2scale, the final optimization is performed inPhase 3scale with 17\-step adjustments with±\\pm0\.156 mm\. This state space allows the agent to explore how precise focus adjustments impact the system’s performance, as it narrows down the optimal focus setting within these specific focus ranges\.
- •Action Space: The available actions include both positive and negative adjustments in millimeters: \[±\\pm2\.5,±\\pm0\.625,±\\pm0\.156,±\\pm0\.157\]\. These actions allow the agent to either increase or decrease the focal length incrementally, depending on the feedback it receives from the environment\. ![Refer to caption](https://arxiv.org/html/2608.10549v1/x4.png)Figure 4:3D parameter space visualization for Stage 1 \- Focus Optimization\.
- •In our approach, each state that the agent encounters can be classified into one of three categories:Optimal State,Error State, orNormal State, based on the current focal length and the corresponding line width result\. TheOptimal Stateis the state where the focal length produces the narrowest line width, indicating the best possible performance for the laser marking process\. These states are represented by the blue points in the 3D parameter space visualization in Fig\.[4](https://arxiv.org/html/2608.10549#S4.F4), which are concentrated near the optimal focal length range \(e\.g\., around 85\.5 mm\)\. TheError Stateoccurs when the linewidth cannot be measured properly, typically caused by an incorrect focal length that prevents proper marking \(e\.g\., line width = \-1\.0\)\. In the 3D parameter space, these states are depicted as red points, scattered outside the optimal range, highlighting regions where the cutting process fails\. Lastly,Normal Staterefers to any state where the line width is measurable but not optimal\. These states are represented by gray points in the visualization, forming a transition zone between error and optimal states\. In these states, the agent receives negative feedback, as the focal length does not produce the narrowest possible line, but the process is still functional\.
- •Reward Function: The reward function guides the agent in selecting the best focal length while penalizing inefficient actions\. The reward is defined as: R​1=\{−200,if Error,100line width−\(s​t​e​p×0\.1\),if Optimal,−1line width−\(s​t​e​p×0\.1\),if Normal\.R1=\\begin\{cases\}\-200,\\textit\{if Error\},\\\\ \\frac\{100\}\{\\textit\{line width\}\}\-\(step\\times 0\.1\),\\textit\{if Optimal\},\\\\ \-\\frac\{1\}\{\\textit\{line width\}\}\-\(step\\times 0\.1\),\\textit\{if Normal\}\.\\end\{cases\}\(6\)In Eq\.[6](https://arxiv.org/html/2608.10549#S4.E6), lower line width values yield higher rewards, incentivizing more precise cuts\. Forerror states\(e\.g\., line width = \-1\), a fixed penalty of \-200 is assigned to strongly discourage invalid settings\. Foroptimal states, defined as the minimum line width observed in the dataset \(Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1)\), the reward scales with100line width\\frac\{100\}\{\\textit\{line width\}\}, typically producing values in the range of approximately 125 to 250 for line width values between 0\.4 mm and 0\.8 mm\. Thenormal statesyield a smaller reward, ranging from approximately 1\.25 to 2\.5, calculated as1line width\\frac\{1\}\{\\textit\{line width\}\}\. In bothoptimalandnormal states, the reward is slightly penalized by the number of steps taken in the episode via the\(s​t​e​p×0\.1\)\(step\\times 0\.1\)term, encouraging faster convergence\.

#### 4\.3\.2Stage 2 \- Power Optimization

- •State Space: This is defined by the current power setting and the analysis of the cutting result for the current lot\. The initial power setting starts at 25%, and the power is incremented by 1% in each step\. For each power setting, the system analyzes the cutting image to evaluate the cutting quality\. A successful cut is determined based on two conditions:\(1\)no dark spots are present within the cutting area, and\(2\)the average brightness value within the cutting area is 167 or higher\. The state space thus consists of the power setting at each step, the absence or presence of dark spots, and the average brightness of the cut\.
- •Action Space: \[±\\pm10,±\\pm9,±\\pm8,±\\pm7,±\\pm6,±\\pm5,±\\pm4,±\\pm3,±\\pm2,±\\pm1\] %\.
- •In Stage 2, theError Stateoccurs when the cutting brightness is 0, resulting in a significant penalty\. These states are represented by red points in the 3D parameter space visualization in Fig\.[5](https://arxiv.org/html/2608.10549#S4.F5), showing regions where the power settings fail to generate sufficient brightness for a valid cut\. TheOptimal Stateis reached when the cutting brightness matches the optimal value, indicating a successful cut without dark spots, and the average brightness value is higher or equal to a specific threshold\. In the parameter space, these states are depicted as blue points, concentrated in a specific range of power values\. And theNormal statecovers any non\-optimal, measurable brightness values\. These are represented by gray points in the visualization, forming a transition zone between error and optimal states\. ![Refer to caption](https://arxiv.org/html/2608.10549v1/x5.png)Figure 5:3D parameter space visualization for Stage 2 \- Power Optimization\.
- •Reward Function: The reward function for Stage 2 is defined as: R​2=\{−5000,if Error,\(brightness×50\)−\(step×2\),if Optimal,−brightness−\(step×2\),if Normal\.R2=\\begin\{cases\}\-5000,\\textit\{if Error\},\\\\ \(\\textit\{brightness\}\\times 50\)\-\(\\textit\{step\}\\times 2\),\\textit\{if Optimal\},\\\\ \-\\textit\{brightness\}\-\(\\textit\{step\}\\times 2\),\\textit\{if Normal\}\.\\end\{cases\}\(7\)In Eq\.[7](https://arxiv.org/html/2608.10549#S4.E7), the reward is primarily influenced by the brightness of the laser mark\. According to the dataset in Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1), brightness values range from 0 \(error states\) to a maximum of 167 in the optimal configuration\. Foroptimal states, the reward can reach up to167×50=8350167\\times 50=8350before applying the step penalty, making it highly rewarding\. Fornormal states, brightness values are non\-zero but insufficient to meet the threshold, resulting in negative rewards ranging roughly from \-1 to \-164\. A strong fixed penalty of \-5000 is used for error states where brightness is zero\. The step penalty term\(step×2\)\(\\textit\{step\}\\times 2\)discourages prolonged exploration and encourages faster convergence\.

#### 4\.3\.3Stage 3 \- Taper Optimization

- •State Space: This is defined by the current focus and power settings, with better focus adjustments applied to optimize the taper value\. Starting with the best focus and power values from the previous stages, the focus is adjusted incrementally by 0\.027 mm over 17 iterations\. The system evaluates the taper quality by calculating the average of\(Taper1 \+ Taper2\) / 2\. A successful cut is determined when the taper result meets the same cutting conditions as in Stage 1: the system returns an“OK”if the taper values are acceptable, indicating a successful cut\. The state space consists of the focus adjustments, the calculated taper value, and the result status \(“OK”or“NG”\)\.
- •Action Space: \[±\\pm0\.109,±\\pm0\.108,±\\pm0\.056,±\\pm0\.055,±\\pm0\.054,±\\pm0\.028,±\\pm0\.027\] mm\.
- •In Stage 3, theError Stateoccurs when the taper result is invalid \(i\.e\., taper value of \-1\.0\), leading to a penalty\. These states are represented by red points in the 3D parameter space visualization in Fig\.[6](https://arxiv.org/html/2608.10549#S4.F6), highlighting regions where the focus adjustments fail to produce valid taper measurements\. TheOptimal Stateis achieved when the taper result produces the minimum\(T​a​p​e​r​1\+T​a​p​e​r​2\)/2\(Taper1\+Taper2\)/2value\. These states are shown as blue points in the 3D visualization, clustered around specific focus values\. TheNormal Staterefers to any valid taper value that is not optimal, where the agent continues to adjust focus to reach the optimal taper\. These states are depicted as gray points in the visualization, forming a transitional zone between error and optimal states\. ![Refer to caption](https://arxiv.org/html/2608.10549v1/x6.png)Figure 6:3D parameter space visualization for Stage 3 \- Taper Optimization\.
- •Reward Function: The reward function for Stage 3 is defined as: R​3=\{−200,if Error,\(100taper\)−\(step×0\.1\),if Optimal,−\(1taper\)−\(step×0\.1\),if Normal\.R3=\\begin\{cases\}\-200,\\textit\{if Error\},\\\\ \\left\(\\frac\{100\}\{\\textit\{taper\}\}\\right\)\-\(\\textit\{step\}\\times 0\.1\),\\textit\{if Optimal\},\\\\ \-\\left\(\\frac\{1\}\{\\textit\{taper\}\}\\right\)\-\(\\textit\{step\}\\times 0\.1\),\\textit\{if Normal\}\.\\end\{cases\}\(8\)Eq\.[8](https://arxiv.org/html/2608.10549#S4.E8)is designed to minimize taper size in Stage 3, where the agent performs a focused search around the optimal focus value previously found in Stage 1\. In the example shown in Fig\.[1](https://arxiv.org/html/2608.10549#S3.F1), the taper values range from 0\.204 mm \(optimal states\) to 0\.218 mm \(normal states\), resulting in a maximum theoretical reward of1000\.204≈490\\frac\{100\}\{0\.204\}\\approx 490for the best\-performing configuration\. In contrast,normal statesresult in small negative rewards; for example, taper = 0\.214 yields−10\.214≈−4\.67\-\\frac\{1\}\{0\.214\}\\approx\-4\.67\.Error states\(e\.g\., taper = \-1\) are assigned a fixed penalty of \-200\. However, in other lots where the optimal focus range is different, the taper values and the reward bounding may shift accordingly\.

## 5Experimental results

### 5\.1Experimental Setup

This subsection explains the experimental setup for our evaluation\. We have a total of 210 lots, with 90 for training and 120 for testing\. The agent is trained on each material sequentially, starting with Material A, followed by A1, and then B, using theε\\varepsilon\-greedy policy across these lots\. For each lot, the agent runs 100 episodes\. In each episode, the agent is allowed a maximum of 100 steps to find the shortest path to the optimal state\. During the training phase, theε\\varepsilonvalue starts at 1\.0 and gradually decreases over the training lots, reaching 0\.1\. This means that theε\\varepsilonvalue remains constant within each lot and only reduces when moving to the next lot, encouraging more exploration in the early lots and more exploitation of learned knowledge in later lots\. Once the training is complete, the agent is evaluated on the remaining 120 testing lots with a fixedε\\varepsilonof 0\.1, ensuring that the agent primarily relies on the learned policies while allowing limited exploration\. This setup provides a comprehensive evaluation of the agent’s performance across different materials, ensuring effectiveness for optimizing focus, power, and taper settings\.

![Refer to caption](https://arxiv.org/html/2608.10549v1/x7.png)\(a\)
![Refer to caption](https://arxiv.org/html/2608.10549v1/x8.png)\(b\)
![Refer to caption](https://arxiv.org/html/2608.10549v1/x9.png)\(c\)
![Refer to caption](https://arxiv.org/html/2608.10549v1/x10.png)\(d\)
![Refer to caption](https://arxiv.org/html/2608.10549v1/x11.png)\(e\)
![Refer to caption](https://arxiv.org/html/2608.10549v1/x12.png)\(f\)
![Refer to caption](https://arxiv.org/html/2608.10549v1/x13.png)\(g\)
![Refer to caption](https://arxiv.org/html/2608.10549v1/x14.png)\(h\)
![Refer to caption](https://arxiv.org/html/2608.10549v1/x15.png)\(i\)

Figure 7:Steps in first\(a\), medium\(b\), and final\(c\) lot in material A; Steps in first\(d\), medium\(e\), and final\(f\) lot in material A1; and Steps in first\(g\), medium\(h\), and final\(i\) lot in material B\.
### 5\.2Baselines

To evaluate the performance of the proposed RL2C algorithm, we compare it with three baseline methods:Random Search,RL\-based Bayesian OptimizationandRL\-based PSO\. These methods provide straightforward and probabilistic approaches for optimization, serving as benchmarks for assessing the effectiveness of our proposed method\.

Random Search:Random Search is a stochastic optimization technique in which the agent selects actions uniformly at random from the defined action space without employing any structured strategy or policy\. At each decision step, the agent randomly selects actions in a purely probabilistic manner, with no consideration of previous experiences or rewards, providing a straightforward baseline method\.

RL\-based Bayesian Optimization\[chang2024optimization\]: RL\-based Bayesian Optimization integrates RL principles with Bayesian search strategies, leveraging a probabilistic model to guide the action selection process\. The agent considers only a few actions to take according to their experience and then chooses the action that would give the maximum expected cumulative reward using the Gaussian process\.

RL\-based PSO\[zhang2023q\]: Inspired by social behaviors in nature like birds flocking or fish schooling, PSO consists of a swarm of candidate solutions, or particles, in the action space\. The agent’s actions are adapted according to its own experiences and the results of other particles in the neighborhood to optimize decision\-making\.

### 5\.3Main Results

#### 5\.3\.1Step Efficiency Analysis Over Specific Lots

Fig\.[7](https://arxiv.org/html/2608.10549#S5.F7)illustrates the number of steps required to find the optimal state of the proposed method across different lot positions \(first, medium, and final\) during the training phase for three distinct materials \(A, A1, and B\)\. The figures show the number of steps to reach the optimal state \(blue bars\) and the number of steps to reach the error state \(red bars\) over 100 episodes in each lot\.

For material A in Fig\.[7\(a\)](https://arxiv.org/html/2608.10549#S5.F7.sf1), the result presents a very high number of steps across episodes\. For instance, there are some episodes that take up to 100 steps to find the optimal state, and there are also episodes where the agent reaches an error state during the learning process\. This suggests that the agent needs to perform many possible actions in the exploration phase to gain a better understanding of the environment\. During this period, the agent may not be able to find the optimal state or may encounter error states\. However, after obtaining sufficient knowledge about the current environment, the number of steps has been quickly reduced to under 40 steps in Fig\.[7\(b\)](https://arxiv.org/html/2608.10549#S5.F7.sf2)and under 20 steps in Fig\.[7\(c\)](https://arxiv.org/html/2608.10549#S5.F7.sf3)\. This reduction in the number of steps indicates that the agent has successfully learned the optimal policies and can exploit them more efficiently, resulting in a faster convergence towards the optimal state\.

For material A1 in Fig\.[7\(d\)](https://arxiv.org/html/2608.10549#S5.F7.sf4), the agent exhibits a more consistent and stable convergence rate, with fewer spikes and lower variability in the number of steps across episodes\. This behavior is particularly evident in the medium and final lots, as shown in Figs\.[7\(e\)](https://arxiv.org/html/2608.10549#S5.F7.sf5)and[7\(f\)](https://arxiv.org/html/2608.10549#S5.F7.sf6)\. Similarly, for material B in Figs\.[7\(g\)](https://arxiv.org/html/2608.10549#S5.F7.sf7),[7\(h\)](https://arxiv.org/html/2608.10549#S5.F7.sf8), and[7\(i\)](https://arxiv.org/html/2608.10549#S5.F7.sf9), the figures demonstrate an immediate convergence rate, with the agent requiring a very low number of steps to reach the optimal state across episodes\. This trend is consistent across all three lots, implying that the agent has successfully learned the optimal policies and can exploit them efficiently for most of the lots in Material B\.

#### 5\.3\.2Step Efficiency Analysis Across Different Stages and Materials

![Refer to caption](https://arxiv.org/html/2608.10549v1/x16.png)\(a\)Stage 1 \- Focus Optimization
![Refer to caption](https://arxiv.org/html/2608.10549v1/x17.png)\(b\)Stage 2 \- Power Optimization
![Refer to caption](https://arxiv.org/html/2608.10549v1/x18.png)\(c\)Stage 3 \- Taper Optimization

Figure 8:Step comparison of RL2C and baseline methods across three optimization stages\.Fig\.[8](https://arxiv.org/html/2608.10549#S5.F8)illustrates the average number of steps required by various optimization methods to reach the optimal state across three stages: Focus Optimization, Power Optimization, and Taper Optimization\. The results are reported for three different materials \(Material A, Material A1, and Material B\) during both the training and testing phases\. The comparison includes the proposed RL2C method and three baseline methods: Random Search, RL\-based Bayesian Optimization, and RL\-based PSO\.

As shown in Fig\.[8\(a\)](https://arxiv.org/html/2608.10549#S5.F8.sf1), during the focus optimization stage, the RL2C method consistently requires fewer steps to reach the optimal state compared to the baseline methods\. For all materials tested, RL2C demonstrates rapid convergence, particularly during the training phase, where it stabilizes around 5 steps by the midpoint of the training episodes\. In contrast, Random Search exhibits significant variability throughout both the training and testing phases, often requiring more than 15 steps to achieve optimization\. Bayesian Optimization and PSO perform moderately, with PSO showing slightly more consistency than Bayesian Optimization, but both remain less efficient than RL2C\. A noticeable pattern across all methods is the sudden increase in the number of steps required when the agent transitions to a new material \(e\.g\., from Material A to A1 or from A1 to B\)\. This spike occurs at the start of each new material phase, indicating the challenge of adapting to previously unseen conditions\. However, RL2C quickly recovers from this situation, showcasing great adaptability compared to the baseline methods, which exhibit slower recovery and higher variability\.

In Fig\.[8\(b\)](https://arxiv.org/html/2608.10549#S5.F8.sf2), the step efficiency for the power optimization stage shows a similar trend\. RL2C outperforms the baseline methods across all materials in both the training and testing phases\. During training, RL2C converges to the optimal state within approximately 3 to 5 steps on average for all three materials, indicating a high level of efficiency in finding the parameter space\. Conversely, Random Search requires significantly more steps, fluctuating between 10 and 20 steps throughout the episodes\. Bayesian Optimization and PSO demonstrate better performance compared to Random Search but still require more steps on average compared to RL2C\. The results for the taper optimization stage are presented in Fig\.[8\(c\)](https://arxiv.org/html/2608.10549#S5.F8.sf3)\. Similar to the previous stages, RL2C consistently outperforms the baseline methods in terms of step efficiency\. During training, RL2C reaches the optimal state within 2 to 4 steps on average, with minimal variability across different episodes\.

Across all three optimization stages, RL2C demonstrates a clear advantage in terms of step efficiency, particularly in adapting to new materials\. The sudden increases in step count observed during material transitions are a critical challenge in real\-world industrial processes\. The ability of RL2C to quickly adapt and stabilize after these transitions highlights its robustness and practical applicability in dynamic environments, where material properties may vary significantly\.

#### 5\.3\.3Performance of RL2C in Step and Time Efficiency Over Baseline Methods

Table[1](https://arxiv.org/html/2608.10549#S5.T1)presents a performance comparison of the RL2C algorithm with baseline methods \(RL\-Bayesian, RL\-PSO, and Random Search\) in terms of step and time efficiency across three stages of optimization \(Stage 1, Stage 2, and Stage 3\)\. Both training and testing phases are evaluated, and the results highlight the significant advantages of RL2C\.

For step efficiency, during the training phase, RL2C consistently achieves the lowest average number of steps across all three stages\. In Stage 1, RL2C requires an average of 5\.1 steps compared to 5\.4 for RL\-Bayesian and 25\.3 for Random Search\. Similarly, in Stage 2 and Stage 3, RL2C maintains superior step efficiency with averages of 1\.6 and 3\.8 steps, respectively\. The algorithm demonstrates strong performance in the testing phase as well, with lower average steps in all stages, further confirming its effectiveness in converging to optimal states with minimal exploration\.

For time efficiency, RL2C shows a significant improvement over the baseline methods in both training and testing phases\. In the training phase, RL2C completes Stage 1 with an average time of 0\.03 seconds, which is significantly faster than RL\-Bayesian \(4\.77 seconds\) and RL\-PSO \(0\.11 seconds\)\. Similar trends are observed in Stages 2 and 3, where RL2C’s average times for both stages are 0\.02 seconds\. Notably, RL\-Bayesian exhibits the highest time cost, particularly in Stage 2, where it requires an average of 11\.23 seconds\. In the testing phase, RL2C maintains superior time efficiency, completing all stages in approximately 0\.02 seconds on average, overcoming RL\-Bayesian and RL\-PSO\.

Table 1:Performance of RL2C Compared to baseline methods in step and time efficiency \(in seconds\)\.The result demonstrates that RL2C is not only capable of efficiently exploring and exploiting the parameter space but also exhibits superior generalization to new conditions, as evidenced by its consistent performance across both training and testing phases\. Its ability to achieve optimal states with fewer steps and significantly lower computational time highlights its suitability for real\-time optimization tasks, making it a robust solution for laser cutting parameter optimization\. The consistent performance across all stages and phases further emphasizes its adaptability and scalability in diverse scenarios\.

## 6Conclusion

This study presents a comprehensive evaluation of the proposed RL2C algorithm for optimizing laser cutting parameters across three vital stages: Focus Optimization, Power Optimization, and Taper Optimization\. The results demonstrate that RL2C consistently outperforms baseline methods, including Random Search, RL\-based Bayesian Optimization, and RL\-based PSO, in terms of step efficiency, time efficiency, and adaptability to dynamic environments\. Across all tested materials \(Material A, Material A1, and Material B\), RL2C exhibits faster convergence, maintaining low execution times and requiring fewer steps to reach the optimal state\.

A key strength of RL2C lies in its ability to handle transitions between materials efficiently\. While baseline methods, particularly Bayesian Optimization, suffer from significant time and step overheads during these transitions, RL2C adapts rapidly with minimal computational cost\. This adaptability underscores its robustness and practicality for real\-world applications, where material properties can vary significantly between production lots\.

In summary, RL2C emerges as a highly efficient and scalable solution for laser cutting parameter optimization, demonstrating optimal performance in both static and dynamic environments\. These findings pave the way for its deployment in industrial settings, where efficiency, consistency, and adaptability are important\. Future work will explore extending RL2C to additional manufacturing processes and integrating it with other advanced learning frameworks to further enhance its applicability and scalability\.

\\bmhead

Author contributions Quan Khanh Pham conducts the experiments, coding, and manuscript writing\. Majid Kundroo assists in the coding and manuscript preparation\. Geunwoo Ban, Seongho Bae, and Taehong Kim provide technical guidance, contribute to experimental design, revise the manuscript, and supervise the research\.

\\bmhead

Funding This work was supported by the Innovative Human Resource Development for Local Intellectualization program through the Institute of Information & Communications Technology Planning & Evaluation \(IITP\) grant funded by the Korea government \(MSIT\) \(IITP\-2025\-RS\-2020\-II201462\)\.

## Declarations

\\bmhead

Conflict of interest The authors declare that they have no conflict of interest\.

\\bmhead

Data availability The NPS dataset used in this study was acquired from the NPS, and they have not given their permission for researchers to share their data\. Data requests can be made to the NPS via this email: nps@npstech\.co\.kr

## References