ProcessLight: Process Supervision for Large Language Model Based Traffic Signal Control

# ProcessLight: Process Supervision for Large Language Model Based Traffic Signal Control
Source: [https://arxiv.org/html/2609.22746](https://arxiv.org/html/2609.22746)
Tianlong ZhouEmail:[wxrao@tongji\.edu\.cn](mailto:)Weijie WangJiasheng ShiWeixiong RaoSchool of Computer Science and Technology, Tongji University, Shanghai, China††thanks:Corresponding author\.

###### Abstract

Large Language Models \(LLMs\) have recently been introduced into traffic signal control \(TSC\) as decision agents due to their strengths in human\-readable reasoning generation\. Yet, existing LLM TSC methods optimize only from final outcomes and fail to distinguish valid from flawed reasoning steps, causing useful or misleading steps to be jointly updated and thus impairing the model’s learning of effective reasoning\. To bridge this gap, we propose an LLM\-based framework ProcessLight to decompose signal decisions into verifiable semantic steps\. Building on ProcessLight, we further develop Step\-wise Traffic Process Policy Optimization \(STeP\-PO\), a novel reinforcement learning framework that optimizes structured reasoning processes through step\-level credit assignment\. Specifically, STeP\-PO uses step quality scores to evaluate local reasoning quality and step importance to measure each step’s influence on the final action, and then assigns step\-level advantages over a semantic step tree structure\. The resulting step\-level advantages are propagated to reasoning tokens, enabling fine\-grained policy optimization beyond outcome\-only rewards\. Extensive experiments over multiple real\-world datasets demonstrate the superiority of our methods\. Our code is available at[https://github\.com/wenzhaoabc/processlight](https://github.com/wenzhaoabc/processlight)\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2609.22746v1/motivation.png)Figure 1:Outcome\-level supervision cannot effectively distinguish correct and incorrect reasoning steps in the traffic signal reasoning process, weakening the model’s ability to learn correct reasoning\.Rapid urbanization has led to a substantial increase in urban population, making traffic congestion an increasingly severe challenge for modern cities[Wei et al\. \(2019c\)](https://arxiv.org/html/2609.22746#bib.bib32);[Wu et al\. \(2023\)](https://arxiv.org/html/2609.22746#bib.bib13);[Koonce et al\. \(2008\)](https://arxiv.org/html/2609.22746#bib.bib6);[Di et al\. \(2019\)](https://arxiv.org/html/2609.22746#bib.bib3)\. Traffic signal control, as a core mechanism for regulating vehicle flows at intersections, has long been a central research problem in intelligent transportation systems\.

Existing TSC methods have evolved from traditional controllers to deep reinforcement learning approaches\. Rule\-based controllers, such as fixed\-time control and pressure\-based control, are computationally efficient and highly interpretable\. Yet, they heavily depend upon manually designed priors, and struggle to adapt to complex and dynamic traffic conditions[Oroojlooy et al\. \(2020\)](https://arxiv.org/html/2609.22746#bib.bib7);[Wei et al\. \(2019a\)](https://arxiv.org/html/2609.22746#bib.bib12);[Liang et al\. \(2022\)](https://arxiv.org/html/2609.22746#bib.bib14);[Wu et al\. \(2023\)](https://arxiv.org/html/2609.22746#bib.bib13)\. Reinforcement learning\(RL\) based methods learn adaptive signal control policies from feedback provided by traffic simulators[Wei et al\. \(2018\)](https://arxiv.org/html/2609.22746#bib.bib10);[Wei et al\. \(2019a\)](https://arxiv.org/html/2609.22746#bib.bib12);[Wei et al\. \(2019b\)](https://arxiv.org/html/2609.22746#bib.bib8);[Oroojlooy et al\. \(2020\)](https://arxiv.org/html/2609.22746#bib.bib7);[Zhang et al\. \(2022\)](https://arxiv.org/html/2609.22746#bib.bib20);[Wu et al\. \(2023\)](https://arxiv.org/html/2609.22746#bib.bib13);[Yu et al\. \(2020\)](https://arxiv.org/html/2609.22746#bib.bib2)\. Despite promising performance, these methods are trained on compact numerical states and scalar rewards, which limits their interpretability and generalization ability in diverse traffic scenarios\.

Recently, large language models \(LLMs\) have been introduced into traffic signal control to exploit their logical reasoning ability and natural\-language generalization capability\. Existing studies show that LLMs can transform traffic observations into interpretable control decisions[Lai et al\. \(2025\)](https://arxiv.org/html/2609.22746#bib.bib16);[Yuan et al\. \(2026\)](https://arxiv.org/html/2609.22746#bib.bib15);[Zou et al\. \(2026\)](https://arxiv.org/html/2609.22746#bib.bib17)\. However, current LLM\-based TSC methods mainly optimize models according to final actions or overall trajectory outcomes, while largely overlooking the intermediate reasoning steps\. As a result, they cannot effectively distinguish correct reasoning steps from erroneous ones\. Erroneous intermediate reasoning may still be reinforced by a positive trajectory\-level reward when the final action happens to be correct, thereby misleading the model to learn unreliable reasoning patterns\.

As illustrated in Figure[1](https://arxiv.org/html/2609.22746#S1.F1), even when a reasoning trajectory contains*incorrect*intermediate steps, the entire trajectory may still be reinforced when the final control decision is correct\. In contrast, when the final decision is incorrect, the whole reasoning process is penalized, even though some intermediate traffic judgments may be valid and useful\. This outcome\-only optimization paradigm provides coarse and potentially biased supervision, making it difficult to determine which reasoning steps should be reinforced, corrected, or suppressed\.

To bridge this gap, we proposeProcessLight, an LLM\-based traffic signal control framework with explicit step\-wise reasoning\. By decomposing each traffic signal decision into a sequence of reasoning steps, ProcessLight can provide verifiable traffic state evidence for each step, and thus improves the interpretability of LLM\-based signal control\. More importantly, it provides fine\-grained process\-level information for reasoning supervision\.

We further proposeSTeP\-PO, a Step\-wise Traffic Process Policy Optimization algorithm for optimizing ProcessLight\. STeP\-PO consists of three key components\. First, it evaluates the quality of each reasoning step based on traffic\-grounded claims by combining semantic soft matching with observation\-based validity verification\. Second, it estimates step importance through counterfactual sensitivity, where each reasoning step is removed and the resulting change in the final action distribution is measured\. Third, it constructs a semantic step tree that merges semantically equivalent reasoning steps under the same preceding logic and assigns comparable hierarchical advantages to different steps\. These advantages are further propagated to the token level within the reasoning text, enabling fine\-grained policy optimization beyond reliance on outcome\-only rewards\. In summary, we make the following contributions\.

- •We proposeProcessLight, an LLM\-based traffic signal control framework that formulates TSC as a process\-supervised reasoning problem, enabling fine\-grained optimization of intermediate traffic judgments\.
- •We introduceSTeP\-PO, a step\-wise policy optimization algorithm that integrates step\-quality evaluation and step\-importance estimation to provide step\-level supervision for LLM\-based traffic signal control\.
- •We conduct extensive experiments in representative traffic simulation scenarios, demonstrating the effectiveness of the proposed method\.

## 2Related Work

### 2\.1Traffic Signal Control

Early methods of traffic signal control, e\.g\., fixed\-time control and MaxPressure, are simple and stable, but their hand\-crafted rules make them less adaptive to time\-varying traffic demand and network\-level coordination[Koonce et al\. \(2008\)](https://arxiv.org/html/2609.22746#bib.bib6);[Varaiya \(2013\)](https://arxiv.org/html/2609.22746#bib.bib5);[Qadri et al\. \(2020\)](https://arxiv.org/html/2609.22746#bib.bib31);[Wei et al\. \(2019c\)](https://arxiv.org/html/2609.22746#bib.bib32)\. Deep reinforcement learning \(DRL\) methods improve adaptivity by learning signal policies from simulation\. Representative studies include PressLight, which integrates pressure\-based control into RL[Wei et al\. \(2019a\)](https://arxiv.org/html/2609.22746#bib.bib12), CoLight, which models network cooperation with graph attention[Wei et al\. \(2019b\)](https://arxiv.org/html/2609.22746#bib.bib8), and MPLight, which scales decentralized DRL to large road networks[Chen et al\. \(2020\)](https://arxiv.org/html/2609.22746#bib.bib11)\. However, these neural controllers remain limited by black\-box decision processes, weak interpretability, and unstable generalization under distribution shifts\. Recently, LLM\-based controllers have introduced language reasoning into TSC\. LLMLight treats LLMs as traffic signal agents[Lai et al\. \(2025\)](https://arxiv.org/html/2609.22746#bib.bib16), CoLLMLight studies cooperative LLM agents for network\-wide control[Yuan et al\. \(2026\)](https://arxiv.org/html/2609.22746#bib.bib15), and Traffic\-R1 applies two\-stage RL to reduce the gap between reasoning and deployment[Zou et al\. \(2026\)](https://arxiv.org/html/2609.22746#bib.bib17)\. These methods mainly optimize policies using final control outcomes or action\-level feedback, leaving intermediate reasoning steps weakly supervised\.

### 2\.2RL Fine\-tuning for LLMs

RL fine\-tuning is widely used to improve instruction following and reasoning\. RLHF typically optimizes LLMs with PPO[Christiano et al\. \(2017\)](https://arxiv.org/html/2609.22746#bib.bib25);[Ouyang et al\. \(2022\)](https://arxiv.org/html/2609.22746#bib.bib24);[Schulman et al\. \(2017\)](https://arxiv.org/html/2609.22746#bib.bib21), but the used critic for advantage estimation suffers from high memory cost and training instability\. DPO bypasses explicit reward modeling by directly optimizing preference pairs[Rafailov et al\. \(2023\)](https://arxiv.org/html/2609.22746#bib.bib27), while GRPO removes the critic through group\-relative baselines is effective for reasoning\-oriented training[Shao et al\. \(2024\)](https://arxiv.org/html/2609.22746#bib.bib28);[Guo et al\. \(2025a\)](https://arxiv.org/html/2609.22746#bib.bib26)\. Nevertheless, trajectory\-level rewards cannot distinguish the quality or contribution of individual reasoning steps within the same response\. Recent credit\-assignment methods, e\.g\., VinePPO and SPO, address this limitation with finer\-grained advantage estimation[Kazemnejad et al\. \(2025\)](https://arxiv.org/html/2609.22746#bib.bib23);[Guo et al\. \(2025b\)](https://arxiv.org/html/2609.22746#bib.bib22)\. In contrast, ProcessLight introduces step\-level reward design and credit assignment for TSC reasoning, enabling direct supervision of how an LLM derives traffic\-signal actions rather than only whether the final action succeeds\.

## 3Problem Definition

In this section, we introduce the key concepts of traffic signal control and formulate the LLM\-based traffic signal control problem studied in this paper\. More details on intersections, movements and signal phases are provided in the Appendix[A\.1](https://arxiv.org/html/2609.22746#A1.SS1)\.

###### Definition 1

Road Network\.The road network is a directed graph composed of intersectionsIIand roadsEE\. Each road consists of multiple lanesLL, and each lane corresponds to a movement at the intersection\.

###### Definition 2

Traffic Signal Control\. At each signal\-switching time step, the agent selects one phaseaafrom a predefined phase set𝒜=\{a1,…,am\}\\mathcal\{A\}=\\\{a\_\{1\},\\dotsc,a\_\{m\}\\\}\. Each phase corresponds to a group of non\-conflicting movements\.

We formulate traffic signal control as a partially observable Markov decision process\. Each intersection is controlled by an LLM agent with policyπθ\\pi\_\{\\theta\}\. At each signal\-switching time step, the agent observes traffic conditions𝒪\\mathcal\{O\}\. Given the task description𝒟task\\mathcal\{D\}\_\{\\text\{task\}\}and reasoning step guide𝒢\\mathcal\{G\}, the LLM agent generates a step\-by\-step reasoning trajectoryYYand an executable phase actiona∈𝒜\{a\}\\in\\mathcal\{A\}from the predefined phase set𝒜\\mathcal\{A\}by the policyπθ\\pi\_\{\\theta\}:

\(Y,a\)=πθ​\(Prompt​\(𝒪,𝒟task,𝒢,𝒜\)\)\.\(Y,\{a\}\)=\\pi\_\{\\theta\}\\left\(\\text\{Prompt\}\(\\mathcal\{O\},\\mathcal\{D\}\_\{\\text\{task\}\},\\mathcal\{G\},\\mathcal\{A\}\)\\right\)\.\(1\)
Here, the reasoning trajectoryYYcontainsKKsteps\(s1,…,sK\)\(s^\{1\},\\dots,s^\{K\}\)\. The objective of our task is to optimize the policyπθ\\pi\_\{\\theta\}to improve transportation efficiency through high\-quality reasoning\.

## 4Methodology

In this section, we first present an overview of ProcessLight, and then introduce the proposed Step\-wise Traffic Process Policy Optimization \(STeP\-PO\) that is used to optimize ProcessLight\.

![Refer to caption](https://arxiv.org/html/2609.22746v1/overview.png)Figure 2:Overview of ProcessLight with two stages, \(1\) task prompt construction and \(2\) step\-wise reasoning\. The proposed STeP\-PO algorithm is used for policy optimization\.### 4\.1ProcessLight Framework

In Figure[2](https://arxiv.org/html/2609.22746#S4.F2), the workflow of ProcessLight consists of two stages: \(1\) prompt construction and \(2\) step\-wise reasoning and action execution\.

Prompt Construction\.The prompt contains three main parts: current traffic observation𝒪\\mathcal\{O\}, task description𝒟task\\mathcal\{D\}\_\{\\text\{task\}\}and reasoning step guide𝒢\\mathcal\{G\}\. The observation𝒪\\mathcal\{O\}contains queue lengths and vehicle counts of the signalized and neighboring intersections, together with short\-term historical trends for spatiotemporal reasoning\. The guide𝒢\\mathcal\{G\}defines the reasoning direction and step decomposition, requiring the LLM to examine the current phases, local demand, downstream congestion, neighboring coordination, traffic trends, and phase fairness before selecting the final signal control action\. We provide prompt examples in Appendix[A\.13](https://arxiv.org/html/2609.22746#A1.SS13)\.

Step\-wise Reasoning and Action Execution\. Given the promptXX, ProcessLight performs step\-wise reasoning by following the numberKKof predefined decision steps\. At each stepsks^\{k\}withk∈\[1,K\]k\\in\[1,K\], the LLM agent generates both the reasoning evidence and the corresponding step\-level conclusion, and then gives the final signal control actionaa\. Hence, ProcessLight not only executes effective traffic control decisionsaa, but also provides fine\-grained reasoning evidence for each decision stepsks^\{k\}\. This step\-wise reasoning process improves interpretability and, more importantly, provides the necessary step\-level information for subsequent reasoning process supervision\.

### 4\.2STeP\-PO

#### 4\.2\.1Overview

This section introduces the proposed Step\-wise Traffic Process Policy Optimization \(STeP\-PO\)\. STeP\-PO contains three stages: \(1\)Training Data Collection, \(2\)Policy Warm\-upand \(3\)Step\-wise Policy Optimization\. We provide the overall algorithm in Appendix[A\.5](https://arxiv.org/html/2609.22746#A1.SS5)\.

#### 4\.2\.2Training Data Collection

We begin by collecting diverse traffic scenarios from the traffic simulator\. For each scenario, we evaluate all candidate phase actions for the target intersection with a fixed\-horizon network\-wide simulation, while other intersections follow the same baseline control policy, and select the action that yields the best traffic performance, e\.g\., lowest average queue length\. Then, we prompt a large language model \(e\.g\. DeepSeek\-V4\-Pro in Figure[2](https://arxiv.org/html/2609.22746#S4.F2)\) using the prompt construction process described in Section[4\.1](https://arxiv.org/html/2609.22746#S4.SS1)to generate step\-wise reasoning trajectories aligned with the best action and the required output format\. The resulting dataset𝒟\\mathcal\{D\}consists of input promptsXXpaired with step\-wise reasoning trajectoriesYY\. A small subset𝒟SFT⊆𝒟\\mathcal\{D\}\_\{\\mathrm\{SFT\}\}\\subseteq\\mathcal\{D\}is used for supervised warm\-up, while the remaining data𝒟RL\\mathcal\{D\}\_\{\\mathrm\{RL\}\}for step\-wise policy optimization\.

#### 4\.2\.3Policy Warm\-up

To equip the policy model with traffic\-aware reasoning and format\-following capabilities, we introduce a supervised warm\-up stage before reinforcement learning \(RL\)\. The warm\-up fine\-tuning is performed on the above subset𝒟SFT\\mathcal\{D\}\_\{\\mathrm\{SFT\}\}with the following objective:

ℒwarm−up​\(θ\)=−𝔼\(X,Y\)∼𝒟SFT​\[∑t=1\|Y\|log⁡πθ​\(Yt∣X,Y<t\)\]\\mathcal\{L\}\_\{\\mathrm\{warm\-up\}\}\(\\theta\)=\-\\mathbb\{E\}\_\{\(X,Y\)\\sim\\mathcal\{D\}\_\{\\mathrm\{SFT\}\}\}\\left\[\\sum\_\{t=1\}^\{\|Y\|\}\\log\\pi\_\{\\theta\}\\left\(Y\_\{t\}\\mid X,Y\_\{<t\}\\right\)\\right\]\(2\)

#### 4\.2\.4Step\-wise policy optimization

Outcome\-only optimization is performed on entire reasoning trajectories with trivial guidance on intermediate steps\. STeP\-PO addresses this issue by optimizing the policyπθ\\pi\_\{\\theta\}on𝒟RL\\mathcal\{D\}\_\{\\mathrm\{RL\}\}with a step\-wise objectiveℒSTeP​\-​PO​\(θ\)\\mathcal\{L\}\_\{\\mathrm\{STeP\\text\{\-\}PO\}\}\(\\theta\)\. As shown in Fig\.[2](https://arxiv.org/html/2609.22746#S4.F2), STeP\-PO involves three components:*step quality*, which measures whether an intermediate process is locally beneficial,*step importance*, estimating the step contribution to the final action, and a*semantic step tree*, where shared prefix steps are merged across trajectories into a single tree node\. Hence, each node is assigned with a step\-level advantage with a low variance\. Step\-level advantage is then assigned to the tokens within each step to derive token\-level advantages for policy optimization\.

Step Quality\.Step quality helps to provide fine\-grained supervision for intermediate reasoning steps, enabling STeP\-PO to distinguish reliable local decisions from noisy or hallucinated ones\. Specifically, for each promptX∈𝒟RLX\\in\\mathcal\{D\}\_\{\\mathrm\{RL\}\}, the policy modelπθ\\pi\_\{\\theta\}samplesGGreasoning trajectoriesY=\{yi\}i=1GY=\\\{y\_\{i\}\\\}\_\{i=1\}^\{G\}, where each trajectoryyiy\_\{i\}is decomposed intoKKsteps, i\.e\.,yi=\(si1,…,siK\)y\_\{i\}=\(s\_\{i\}^\{1\},\\ldots,s\_\{i\}^\{K\}\)\. To evaluate the decision quality of each step, we require the LLM to output step\-level decision claims\. Each claim is a fine\-grained and verifiable traffic judgment grounded in the current observation, such as “\[ETWT\]\[Queue Length\]\>\>\[NTST\]\[Queue Length\]” or “MAX\[AWT\]\[PHASE\]=NLSL”\. For thekk\-th step each training sampleyiy\_\{i\}, a teacher model pre\-extracts the reference key\-claim set, denoted by𝒞ik\\mathcal\{C\}\_\{i\}^\{k\}, to serve as the criterion for step\-quality evaluation\.

To estimate the step quality score, we use a soft key\-claim matching method inspired by work[Zhang et al\. \(2025\)](https://arxiv.org/html/2609.22746#bib.bib18)and a traffic observation\-based verification\. Specifically, for thekk\-th step of trajectoryyiy\_\{i\}and the associated claim setCikC\_\{i\}^\{k\}, we first use a validity gateValid⁡\(⋅\)\\mathrm\{Valid\}\(\\cdot\)to check whether all generated claims are supported by the current traffic observation\. If any claim is contradicted by the observation, the step quality score is assigned \-1; otherwise, its reward is given by the matching score against the reference key claims:

rik=\{−1,if​Valid​\(Cik\)=False,mik,otherwise,r\_\{i\}^\{k\}=\\begin\{cases\}\-1,&\\text\{if \}\\mathrm\{Valid\}\(C\_\{i\}^\{k\}\)=\\mathrm\{False\},\\\\ m\_\{i\}^\{k\},&\\text\{otherwise\},\\end\{cases\}\(3\)wheremik=\|Ci,matchk\|/\|𝒞ik\|m\_\{i\}^\{k\}=\|C\_\{i,\\mathrm\{match\}\}^\{k\}\|/\|\\mathcal\{C\}\_\{i\}^\{k\}\|andCi,matchkC\_\{i,\\mathrm\{match\}\}^\{k\}denotes the subset of key\-claims matched by the generated claims\. This design penalizes hallucinated traffic judgments, supports semantically equivalent claim forms, and prevents reward inflation from irrelevant claims\. It thereby provides a reliable step quality signal for step\-wise policy optimization\.

Step Importance\.To assess the marginal effect of each reasoning step on the final decision, we incorporate counterfactual sensitivity into STeP\-PO for step importance estimation\. For each training pair\(x,y\)∈𝒟RL\(x,y\)\\in\\mathcal\{D\}\_\{\\mathrm\{RL\}\}withy=\(s1,…,sK\)y=\(s^\{1\},\\dots,s^\{K\}\), we remove thekk\-th step and construct the counterfactual trajectoryy−k=\(s1,…,sk−1,sk\+1,…,sK\)y^\{\-k\}=\(s^\{1\},\\dots,s^\{k\-1\},s^\{k\+1\},\\dots,s^\{K\}\)\. We further map each phase action to a dedicated action token \(e\.g\.,ETWTis presented as a signal token\), such that phase probability can be directly obtained from next\-token prediction probabilities\.

We denote the phase distribution conditioned on the complete trajectoryyyasp0​\(⋅\)p^\{0\}\(\\cdot\)and the counterfactual distribution conditioned ony−ky^\{\-k\}asp−k​\(⋅\)p^\{\-k\}\(\\cdot\)\. The distributional discrepancy betweenp0​\(⋅\)p^\{0\}\(\\cdot\)andp−k​\(⋅\)p^\{\-k\}\(\\cdot\)measures the sensitivity of the final phase decision to stepsks^\{k\}\. We employDJSDD\_\{\\mathrm\{JSD\}\}\(Jensen–Shannon divergence\) to quantify the divergence between distributions:

bk​\(x\)=DJSD​\(p0​\(⋅\),p−k​\(⋅\)\)\.b^\{k\}\(x\)=D\_\{\\mathrm\{JSD\}\}\\left\(p^\{0\}\(\\cdot\),p^\{\-k\}\(\\cdot\)\\right\)\.\(4\)Then we perform normalization over trajectory steps to derive the relative importance:

Ik​\(x\)=bk​\(x\)∑u=1Kbu​\(x\)\+ϵ,I^\{k\}\(x\)=\\frac\{b^\{k\}\(x\)\}\{\\sum\_\{u=1\}^\{K\}b^\{u\}\(x\)\+\\epsilon\},\(5\)where the small constantϵ\\epsilonprevents numerical instability\. This normalization captures the relative effect of each reasoning step within the trajectoryyy\. Thus, it allows outcome feedback to be concentrated on those steps that are more decisive for the final phase action\. Practically, these importance scores are pre\-computed per prompt before policy optimization, and then reused across all associated rollouts\. Thus, no additional forward or backward passes are required during RL training\.

Reward Design\.We define the outcome and process rewards\. The outcome rewardrior\_\{i\}^\{o\}consists of accuracy rewardriaccr\_\{i\}^\{\\mathrm\{acc\}\}and reasoning completion rewardricomr\_\{i\}^\{\\mathrm\{com\}\}\. The former rewardriaccr\_\{i\}^\{\\mathrm\{acc\}\}evaluates whether the model selects the best phase action, while the latter onericomr\_\{i\}^\{\\mathrm\{com\}\}checks whether the model produces all required reasoning steps in the prescribed format\. We denoterio=riacc\+ricomr\_\{i\}^\{o\}=r\_\{i\}^\{\\mathrm\{acc\}\}\+r\_\{i\}^\{\\mathrm\{com\}\}\. For process\-level supervision, we directly use the step quality score of each step as the step reward\.

![Refer to caption](https://arxiv.org/html/2609.22746v1/tree.png)Figure 3:Semantic step tree construction\. Reasoning steps with soft\-matched claims are merged into the same node\. The global state value is recursively propagated from child nodes, while the local step\-level state value is computed from the step quality rewards\.Semantic Step Tree Advantage\.After evaluating step quality and deriving step importance with STeP\-PO, we assign step\-level advantages to strengthen supervision for intermediate reasoning\. To this end, we adopt a tree\-based structure to preserve prefix\-conditioned dependencies during LLM reasoning and then to establish a fair local baseline for cross\-step comparison\. In Figure[3](https://arxiv.org/html/2609.22746#S4.F3), we take the input x as the root node, and then merge those reasoning steps that are located at the same reasoning position across different trajectories and have semantically\-matched claim sets\. Next, we attach each trajectory’s final action as a leaf node\. Detailed algorithm for how to construct the semantic step tree is provided in Appendix[A\.6](https://arxiv.org/html/2609.22746#A1.SS6)\.

For a certain nodennwithin this tree structure, we denote its sibling nodes and child nodes bySib⁡\(n\)\\mathrm\{Sib\}\(n\)andChd⁡\(n\)\\mathrm\{Chd\}\(n\), respectively, and the set of reasoning steps contained in nodennby𝒮⁡\(n\)\\mathcal\{S\}\(n\)\. Next, we define the state value of nodennasV⁡\(n\)V\(n\), which consists of two parts: the global state valueVg​\(n\)V\_\{\\mathrm\{g\}\}\(n\), which indicates its contribution to the final outcome, and the step\-level state valueVs​\(n\)V\_\{\\mathrm\{s\}\}\(n\), which measures the intrinsic quality of the corresponding reasoning step\. Specifically, we recursively compute the global state value by bottom\-up aggregation:

V^g​\(n\)=\{rio,leaf node,1\|Chd⁡\(n\)\|​∑n′∈Chd⁡\(n\)V^g​\(n′\),otherwise\.\\hat\{V\}\_\{g\}\(n\)=\\begin\{cases\}r^\{\\mathrm\{o\}\}\_\{i\},&\\text\{leaf node\},\\\\ \\frac\{1\}\{\|\\mathrm\{Chd\}\(n\)\|\}\\sum\_\{n^\{\\prime\}\\in\\mathrm\{Chd\}\(n\)\}\\hat\{V\}\_\{g\}\(n^\{\\prime\}\),&\\text\{otherwise\}\.\\end\{cases\}\(6\)
For the step\-level state value, we aggregate the step quality rewards of all reasoning steps merged into the same node:

V^s​\(n\)=1\|𝒮⁡\(n\)\|​∑sik∈𝒮⁡\(n\)rik\.\\hat\{V\}\_\{\\mathrm\{s\}\}\(n\)=\\frac\{1\}\{\|\\mathcal\{S\}\(n\)\|\}\\sum\_\{s\_\{i\}^\{k\}\\in\\mathcal\{S\}\(n\)\}r\_\{i\}^\{k\}\.\(7\)
We then estimate the node\-level advantage by jointly considering the global advantage and the step\-level advantage\. The global and step advantage are both computed with a normalized local baseline over sibling nodes, forℓ∈\{g,s\}\\ell\\in\\\{\\mathrm\{g\},\\mathrm\{s\}\\\}:

A^ℓ​\(n\)=V^ℓ​\(n\)−meann′∈Sib⁡\(n\)​V^ℓ​\(n′\)stdn′∈Sib⁡\(n\)​V^ℓ​\(n′\)\+ϵ,\\hat\{A\}\_\{\\mathrm\{\\ell\}\}\(n\)=\\frac\{\\hat\{V\}\_\{\\ell\}\(n\)\-\\mathrm\{mean\}\_\{n^\{\\prime\}\\in\\mathrm\{Sib\}\(n\)\}\\hat\{V\}\_\{\\ell\}\(n^\{\\prime\}\)\}\{\\mathrm\{std\}\_\{n^\{\\prime\}\\in\\mathrm\{Sib\}\(n\)\}\\hat\{V\}\_\{\\ell\}\(n^\{\\prime\}\)\+\\epsilon\},\(8\)whereϵ\\epsilonis a small constant for numerical stability\. Then we get global advantageA^g​\(n\)\\hat\{A\}\_\{g\}\(n\)and local advantageA^s​\(n\)\\hat\{A\}\_\{s\}\(n\)\. The final node\-level advantage is then defined as

A^​\(n\)=wk​\(x\)​A^g​\(n\)\+λ​A^s​\(n\),\\hat\{A\}\(n\)=w^\{k\}\(x\)\\hat\{A\}\_\{\\mathrm\{g\}\}\(n\)\+\\lambda\\hat\{A\}\_\{\\mathrm\{s\}\}\(n\),\(9\)whereλ\\lambdais a hyperparameter controlling the contribution of the step\-level advantage\. The weightwk​\(x\)w^\{k\}\(x\)is derived from the normalized step importanceIk​\(x\)I^\{k\}\(x\)to preserve the average scale of the global credit, whereη\\etacontrols the strength of importance weighting\.

wk​\(x\)=\(1−η\)\+η​K​Ik​\(x\)\.w^\{k\}\(x\)=\(1\-\\eta\)\+\\eta KI^\{k\}\(x\)\.\(10\)
Unlike the conventional outcome\-level advantage, the tree\-structured advantage constructs comparable sibling node groups under shared reasoning prefixes and jointly considers local relative advantages according to step\-level decision quality\. Thus, STeP\-PO alleviates the credit assignment bias caused by relying solely on overall reasoning outcomes, and instead provide step\-level optimization signals that explicitly preserve the contextual dependencies of preceding reasoning steps\.

#### 4\.2\.5Policy Optimization Objective

We assignA^​\(n\)\\hat\{A\}\(n\)to all tokens within the corresponding reasoning\-step segment and denoteA^i,t\\hat\{A\}\_\{i,t\}as the advantage assigned to thett\-th token in theii\-th sampled trajectoryyiy\_\{i\}\. Given a group ofGGrollouts sampled from the old policyπθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}, we define the token\-level importance sampling ratio and its clipped form as

ρi,t​\(θ\)\\displaystyle\\rho\_\{i,t\}\(\\theta\)=πθ​\(yi,t∣x,yi,<t\)πθold​\(yi,t∣x,yi,<t\),\\displaystyle=\\frac\{\\pi\_\{\\theta\}\(y\_\{i,t\}\\mid x,y\_\{i,<t\}\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{i,t\}\\mid x,y\_\{i,<t\}\)\},\(11\)ρ¯i,t​\(θ\)\\displaystyle\\bar\{\\rho\}\_\{i,t\}\(\\theta\)=clip⁡\(ρi,t​\(θ\),1−ϵ,1\+ϵ\)\.\\displaystyle=\\operatorname\{clip\}\(\\rho\_\{i,t\}\(\\theta\),1\-\\epsilon,1\+\\epsilon\)\.
The final STeP\-PO objective is defined as

ℒSTeP​\-​PO\(θ\)=−𝔼x,\{yi\}i=1G\[1G∑i=1G1\|yi\|∑t=1\|yi\|\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{STeP\\text\{\-\}PO\}\}\(\\theta\)=\-\\mathbb\{E\}\_\{x,\\\{y\_\{i\}\\\}\_\{i=1\}^\{G\}\}\\Bigg\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\frac\{1\}\{\|y\_\{i\}\|\}\\sum\_\{t=1\}^\{\|y\_\{i\}\|\}\(12\)\(min\(ρi,t\(θ\)A^i,t,ρ¯i,t\(θ\)A^i,t\)−βDKLi,t\)\],\\displaystyle\\Big\(\\min\\big\(\\rho\_\{i,t\}\(\\theta\)\\hat\{A\}\_\{i,t\},\\bar\{\\rho\}\_\{i,t\}\(\\theta\)\\hat\{A\}\_\{i,t\}\\big\)\-\\beta D\_\{\\mathrm\{KL\}\}^\{i,t\}\\Big\)\\Bigg\],whereβ\\betacontrols the KL regularization strength\. Following GRPO\-style optimization, the token\-level KL penalty is estimated by

DKLi,t=πref​\(yi,t∣x,yi,<t\)πθ​\(yi,t∣x,yi,<t\)−log⁡πref​\(yi,t∣x,yi,<t\)πθ​\(yi,t∣x,yi,<t\)−1\.D\_\{\\mathrm\{KL\}\}^\{i,t\}=\\frac\{\\pi\_\{\\mathrm\{ref\}\}\(y\_\{i,t\}\\mid x,y\_\{i,<t\}\)\}\{\\pi\_\{\\theta\}\(y\_\{i,t\}\\mid x,y\_\{i,<t\}\)\}\-\\log\\frac\{\\pi\_\{\\mathrm\{ref\}\}\(y\_\{i,t\}\\mid x,y\_\{i,<t\}\)\}\{\\pi\_\{\\theta\}\(y\_\{i,t\}\\mid x,y\_\{i,<t\}\)\}\-1\.\(13\)

## 5Experiments

### 5\.1Experimental Setup

#### 5\.1\.1Datasets

As shown in Table[1](https://arxiv.org/html/2609.22746#S5.T1), we train our model on one synthetic dataset \(Synthetic\-1\) and evaluate its zero\-shot generalization ability on five real datasets \(Jinan1, Jinan2, Jinan3, Hangzhou1, and Hangzhou2\)[Wei et al\. \(2019c\)](https://arxiv.org/html/2609.22746#bib.bib32)\. Detailed information about these datasets is provided in Appendix[A\.4](https://arxiv.org/html/2609.22746#A1.SS4)\.

Table 1:Statistics of datasets\.Flow datasetStructureVehiclesArrival rate \(vehicles/5min\)MeanStdMaxMinJinan 13×\\times46295523\.6798\.52671255Jinan 24365362\.8374\.81493236Jinan 35494456\.9246\.38544362Hangzhou 14×\\times42983247\.6740\.44332211Hangzhou 26984581\.08318\.431145202Synthetic 14×\\times48000666\.632\.10735612

Table 2:Zero\-shot traffic control performance on real\-world traffic flows\. Lower values indicate better performance\. The best results are in bold and the second\-best results areunderlined\.
#### 5\.1\.2Environment Settings

We conduct experiments on the open\-source microscopic traffic simulator CityFlow[Zhang et al\. \(2019\)](https://arxiv.org/html/2609.22746#bib.bib19)\. Each intersection adopts four signal phases, and right\-turn traffic is always allowed\. The green phase lasts 15 seconds, followed by a 3\-second yellow light and a 2\-second all\-red interval\. All traffic flow scenarios are simulated for one hour\.

#### 5\.1\.3Compared Methods

We adopt three following categories of evaluation methods\.Traditional transportation methodsinclude FixedTime[Koonce et al\. \(2008\)](https://arxiv.org/html/2609.22746#bib.bib6)and MaxPressure[Varaiya \(2013\)](https://arxiv.org/html/2609.22746#bib.bib5)as baselines\. ForRL\-based methods, we compare with eight methods, including MPLight[Chen et al\. \(2020\)](https://arxiv.org/html/2609.22746#bib.bib11), CoLight[Wei et al\. \(2019b\)](https://arxiv.org/html/2609.22746#bib.bib8), AttendLight[Oroojlooy et al\. \(2020\)](https://arxiv.org/html/2609.22746#bib.bib7), EfficientPressLight[Wu et al\. \(2021\)](https://arxiv.org/html/2609.22746#bib.bib9), EfficientMPLight[Wu et al\. \(2021\)](https://arxiv.org/html/2609.22746#bib.bib9), EfficientCoLight[Wu et al\. \(2021\)](https://arxiv.org/html/2609.22746#bib.bib9), AdvancedMPLight[Zhang et al\. \(2022\)](https://arxiv.org/html/2609.22746#bib.bib20)and AdvancedCoLight[Zhang et al\. \(2022\)](https://arxiv.org/html/2609.22746#bib.bib20)\. ForLLM\-based methods, we evaluate LLMLight[Lai et al\. \(2025\)](https://arxiv.org/html/2609.22746#bib.bib16), CoLLMLight[Yuan et al\. \(2026\)](https://arxiv.org/html/2609.22746#bib.bib15)and Traffic\-R1[Zou et al\. \(2026\)](https://arxiv.org/html/2609.22746#bib.bib17)\. Additionally, we assess the performance of general LLMs integrated within our ProcessLight framework, including Llama, Qwen, and DeepSeek series\. More details refer to Appendix[A\.2](https://arxiv.org/html/2609.22746#A1.SS2)\.

#### 5\.1\.4Evaluation Metrics

We adopt three widely used traffic signal control metrics includingAverage Travel Time \(ATT\),Average Waiting Time \(AWT\), andAverage Queue Length \(AQL\)\. Their lower values indicate better traffic efficiency and control performance\. More details about these metrics refer to Appendix[A\.3](https://arxiv.org/html/2609.22746#A1.SS3)\.

#### 5\.1\.5Implementation Details

We implement ProcessLight with Qwen2\.5\-3B[Yang et al\. \(2025\)](https://arxiv.org/html/2609.22746#bib.bib30)as the LLM\-agent and DeepSeek\-V4\-Pro[DeepSeek\-AI \(2026\)](https://arxiv.org/html/2609.22746#bib.bib29)as the teacher model, and split the collected dataset into two subsets:𝒟SFT\\mathcal\{D\}\_\{\\mathrm\{SFT\}\}with 500 samples and𝒟RL\\mathcal\{D\}\_\{\\mathrm\{RL\}\}with 3000 samples\. For fairness, all LLM baselines are trained and evaluated with the same experimental environment and configurations\. With no availability of official Traffic\-R1 source code, we have to adopt the model weights provided by Traffic\-R1 HuggingFace[Zou et al\. \(2025\)](https://arxiv.org/html/2609.22746#bib.bib4)for evaluation\.

### 5\.2Performance Comparison

We evaluate ProcessLight on conventional traffic signal control tasks using five real\-world traffic flows from two China cities \(Jinan and Hangzhou\)\. Following the cross\-domain evaluation protocol, all learning\-based methods are trained on Synthetic\-1 and then directly tested on unseen real\-world traffic flows without further fine\-tuning\. As shown in Table[2](https://arxiv.org/html/2609.22746#S5.T2), ProcessLight achieves the best overall performance among all compared methods, obtaining the best results on most metrics\. It also consistently outperforms existing LLM\-based traffic control methods on almost all metrics, demonstrating that optimizing the reasoning process provides a more effective control policy than relying only on prompted reasoning or outcome\-level optimization\.

Without any fine\-tuning, the smaller Qwen2\.5\-3B surpasses LightGPT\-8B across five datasets\. This demonstrates that model performance is not exclusively determined by parameter scale and fine\-tuning; the stepwise reasoning mechanism of ProcessLight also contributes\. By compelling LLMs to generate explicit reasoning steps, ProcessLight enables LLM agents to comprehensively analyze core traffic control factors and make rational intermediate decisions, effectively boosting traffic control performance prior to policy optimization\.

### 5\.3Ablation Study

Table 3:Ablation study on five real\-world datasets\.We perform progressive ablation studies on five real\-world traffic datasets with Qwen2\.5\-3B as the backbone to investigate the effect of four modules: the policy warm\-up stage and three core designs: step quality reward \(SQ\), step importance \(SI\), and semantic step tree advantage \(SSTA\)\. Specifically, SQ introduces averaged step\-level quality rewards into final outcome rewards, SI reallocates global advantages across reasoning steps via normalized importance weights, and SSTA builds tree\-based semantic advantages to instantiate the full STeP\-PO paradigm\. Table[3](https://arxiv.org/html/2609.22746#S5.T3)demonstrate consistent performance improvements via progressive module integration\. The warm\-up stage efficiently helps the LLM grasp basic traffic reasoning capabilities\. SQ incentivizes valid reasoning steps, SI prioritizes critical reasoning steps, and SSTA enables fair advantage assignment via a tree\-based structure\. The full STeP\-PO obtains the best performance, validating the synergistic effect of all three modules\.

### 5\.4Effectiveness of Process Supervision

Table 4:Effectiveness of process\-level supervision\.To verify how process\-level supervision provides effective optimization signal for LLM\-based traffic signal control, we compare outcome\-only RL with our step\-wise process optimization\. Specifically, we evaluate four settings on Jinan1 and Hangzhou1 using ATT as the metric: \(1\)Warm\-up only, where the model is trained only by supervised fine\-tuning; \(2\)Warm\-up \+ Outcome\-level Reward, where the model is optimized by the final outcome reward only; \(3\)Warm\-up \+ Step\-wise Reward, where the model is further guided by the proposed step quality reward; and \(4\) ourSTeP\-PO\. In Table[4](https://arxiv.org/html/2609.22746#S5.T4), both outcome\-level reward and process\-level supervision improve over the warm\-up model, while our STeP\-PO performs best\. This further demonstrates that step\-wise policy optimization is more effective in enhancing LLM reasoning capabilities, as it provides more fine\-grained supervision and mitigates the credit misassignment issue\.

### 5\.5Training Dynamics of ProcessLight

\(a\)Average error claims count\(b\)Average outcome reward
Figure 4:Training dynamics of reward\-side indicators\.To investigate how the process design of ProcessLight affects RL training, we analyze the reward dynamics of different training methods\. Specifically, we compare ProcessLight with PPO and GRPO on two process\-side indicators: average error claims count and average outcome reward\. The former indicator measures the quality of intermediate reasoning\. Lower values indicate fewer incorrect traffic claims\. The latter indicator measures the quality of the final phase decision\.

As shown in Figure[4](https://arxiv.org/html/2609.22746#S5.F4), ProcessLight rapidly reduces the average error\-claim count and consistently maintains a lower error level than PPO and GRPO throughout training\. This result shows that process\-aware reward verification and step\-level credit assignment provide more targeted supervision for intermediate traffic reasoning, leading to more stable and reliable reasoning trajectories\. ProcessLight also achieves the best average outcome reward among the compared methods\. Over the last 10 training steps, ProcessLight obtains an average outcome reward of 0\.585, compared with 0\.574 for GRPO and 0\.522 for PPO\. Thus, improving intermediate reasoning correctness does not compromise final phase selection\. Instead, reliable traffic reasoning can support better control decisions\.

### 5\.6Effectiveness of Step Quality

Table[5](https://arxiv.org/html/2609.22746#S5.T5)reports the accuracy of the resulting advantage signs under step\-quality and outcome\-only supervision\. Specifically, by first sampling 100 traffic states to generate reasoning trajectories using Qwen2\.5\-3B, we then inject two types of controlled noise into each trajectory: logically reversing observation\-consistent claims, and replacing the final action with a valid but verified suboptimal phase\. Using DeepSeek\-V4\-Pro as the judge model[Fu et al\. \(2024\)](https://arxiv.org/html/2609.22746#bib.bib33), we label the correctness of each reasoning step, and determine the ground\-truth sign of the reasoning step advantage\. Outcome\-only supervision assigns the trajectory\-level advantage sign to all reasoning steps\. Yet, Step\-Quality augments the trajectory\-level signal with step\-specific validity rewards\.

From Table[5](https://arxiv.org/html/2609.22746#S5.T5), Outcome\-only supervision assigns an incorrect optimization direction to 22% of the reasoning steps, whereas Step Quality correctly identifies all injected intermediate errors\. This indicates that Step\-Quality can distinguish reliable reasoning steps from hallucinated ones and assign them correct optimization directions\.

Table 5:Accuracy of step\-level advantage signs under controlled noise injection\.
### 5\.7Effectiveness of Step Importance

Finally, to verify the effectiveness of our step importance method, we categorize samples into five groups based on the dominant traffic factors that determine the optimal signal phase\. We calculate the importance of each reasoning step for all groups and Figure[5](https://arxiv.org/html/2609.22746#S5.F5)plots the visualization heatmap\. The cells along the diagonal are significantly brighter, meaning the steps with high importance scores exactly correspond to the core factors affecting traffic efficiency in real scenarios\. This result indicates the reliability and effectiveness of our step importance calculation scheme\.

![Refer to caption](https://arxiv.org/html/2609.22746v1/importance_regime_heatmap.png)Figure 5:Step importance across traffic regimes\. The x\-axis denotes reasoning steps, and the y\-axis groups traffic regimes by the dominant factor behind the simulator\-optimal phase\. Each cell shows the normalized importance of a step; higher values indicate a larger change in phase\-selection probability after masking that step\.

## 6Conclusion

In this paper, we proposedProcessLight, a process\-aware framework for LLM\-based traffic signal control\. Beyond outcome\-only policy optimization, it decomposes signal decisions into structured traffic reasoning steps and supervises them through step quality reward, step importance, and semantic step tree advantage\. Experiments on real\-world traffic benchmarks show that ProcessLight leads to better overall performance than traditional traffic control methods, reinforcement learning controllers, and LLM\-based baselines\. Further evaluations, e\.g\., ablation studies, confirm that fine\-grained reasoning supervision improves both intermediate reasoning reliability and final phase\-decision quality\.

## Limitations

Firstly, the claim\-based verification incurs high implementation complexity, caused by the structured traffic observations, predefined reasoning steps, and rule\-based matching between generated claims and traffic\-state evidence\. We will explore learnable verifiers for lower efforts\. Secondly, step importance estimation introduces extra training overhead, due to counterfactual estimation for reasoning steps\. The approximation or amortized estimation strategies could improve ProcessLight\. Finally, STeP\-PO is designed for structured traffic reasoning with predefined semantic steps on standard four\-way intersections\. The extension of ProcessLight to flexible free\-form reasoning formats on more complex three\- or five\-way intersections[Shen et al\. \(2024\)](https://arxiv.org/html/2609.22746#bib.bib1)remains as future work\.

## Acknowledgment

This work was supported by the National Key R&D Program of China \(Grant No\. 2023YFB4301900\)\.

## Ethics Statement

We abide by the ACL Code of Ethics\. The data resources used in this study are publicly available\. During the preparation of this work the authors used ChatGPT in order to polish manuscript language and assist with minor simulation code drafting\. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article\.

## References

- Chenet al\.\(2020\)C\. Chen, H\. Wei, N\. Xu, G\. Zheng, M\. Yang, Y\. Xiong, K\. Xu, and Z\. LiToward A thousand lights: decentralized deep reinforcement learning for large\-scale traffic signal control\.InThe Thirty\-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty\-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7\-12, 2020,New York, NY, USA,pp\. 3414–3421\.External Links:[Link](https://doi.org/10.1609/aaai.v34i04.5744),[Document](https://dx.doi.org/10.1609/AAAI.V34I04.5744)Cited by:[2nd item](https://arxiv.org/html/2609.22746#A1.I2.i2.p1.1),[§2\.1](https://arxiv.org/html/2609.22746#S2.SS1.p1.1),[§5\.1\.3](https://arxiv.org/html/2609.22746#S5.SS1.SSS3.p1.1)\.
- Chenet al\.\(2025\)L\. Chen, Z\. Zhang, H\. Tan, Q\. Dai, H\. Yang, Z\. Dong, and X\. ChenDistributional llm\-as\-a\-judge\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2025/hash/2b1a5611403691061e6b279d13f3d0c6-Abstract-Conference.html)Cited by:[§A\.9](https://arxiv.org/html/2609.22746#A1.SS9.p1.1)\.
- Christianoet al\.\(2017\)P\. F\. Christiano, J\. Leike, T\. B\. Brown, M\. Martic, S\. Legg, and D\. AmodeiDeep reinforcement learning from human preferences\.InAdvances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4\-9, 2017, Long Beach, CA, USA,I\. Guyon, U\. von Luxburg, S\. Bengio, H\. M\. Wallach, R\. Fergus, S\. V\. N\. Vishwanathan, and R\. Garnett \(Eds\.\),pp\. 4299–4307\.External Links:[Link](https://proceedings.neurips.cc/paper/2017/hash/d5e2c0adad503c91f91df240d0cd4e49-Abstract.html)Cited by:[§2\.2](https://arxiv.org/html/2609.22746#S2.SS2.p1.1)\.
- DeepSeek\-AI \(2026\)DeepSeek\-AIDeepSeek\-v4: towards highly efficient million\-token context intelligence\.Vol\.abs/2606\.19348\.External Links:[Link](https://doi.org/10.48550/arXiv.2606.19348),[Document](https://dx.doi.org/10.48550/ARXIV.2606.19348),2606\.19348Cited by:[§5\.1\.5](https://arxiv.org/html/2609.22746#S5.SS1.SSS5.p1.1)\.
- Diet al\.\(2019\)X\. Di, Y\. Xiao, C\. Zhu, Y\. Deng, Q\. Zhao, and W\. RaoTraffic congestion prediction by spatiotemporal propagation patterns\.In20th IEEE International Conference on Mobile Data Management, MDM 2019, Hong Kong, SAR, China, June 10\-13, 2019,pp\. 298–303\.External Links:[Link](https://doi.org/10.1109/MDM.2019.00-45),[Document](https://dx.doi.org/10.1109/MDM.2019.00-45)Cited by:[§1](https://arxiv.org/html/2609.22746#S1.p1.1)\.
- Fuet al\.\(2024\)J\. Fu, S\. Ng, Z\. Jiang, and P\. LiuGPTScore: evaluate as you desire\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico,pp\. 6556–6576\.External Links:[Link](https://aclanthology.org/2024.naacl-long.365/),[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.365)Cited by:[§5\.6](https://arxiv.org/html/2609.22746#S5.SS6.p1.1)\.
- Guoet al\.\(2025a\)D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi, X\. Zhang, X\. Yu, Y\. Wu, Z\. F\. Wu, Z\. Gou, Z\. Shao, Z\. Li, Z\. Gao, A\. Liu, B\. Xue, B\. Wang, B\. Wu, B\. Feng, C\. Lu, C\. Zhao, C\. Deng, C\. Ruan, D\. Dai, D\. Chen, D\. Ji, E\. Li, F\. Lin, F\. Dai, F\. Luo, G\. Hao, G\. Chen, G\. Li, H\. Zhang, H\. Xu, H\. Ding, H\. Gao, H\. Qu, H\. Li, J\. Guo, J\. Li, J\. Chen, J\. Yuan, J\. Tu, J\. Qiu, J\. Li, J\. L\. Cai, J\. Ni, J\. Liang, J\. Chen, K\. Dong, K\. Hu, K\. You, K\. Gao, K\. Guan, K\. Huang, K\. Yu, L\. Wang, L\. Zhang, L\. Zhao, L\. Wang, L\. Zhang, L\. Xu, L\. Xia, M\. Zhang, M\. Zhang, M\. Tang, M\. Zhou, M\. Li, M\. Wang, M\. Li, N\. Tian, P\. Huang, P\. Zhang, Q\. Wang, Q\. Chen, Q\. Du, R\. Ge, R\. Zhang, R\. Pan, R\. Wang, R\. J\. Chen, R\. L\. Jin, R\. Chen, S\. Lu, S\. Zhou, S\. Chen, S\. Ye, S\. Wang, S\. Yu, S\. Zhou, S\. Pan, S\. S\. Li, S\. Zhou, S\. Wu, T\. Yun, T\. Pei, T\. Sun, T\. Wang, W\. Zeng, W\. Liu, W\. Liang, W\. Gao, W\. Yu, W\. Zhang, W\. L\. Xiao, W\. An, X\. Liu, X\. Wang, X\. Chen, X\. Nie, X\. Cheng, X\. Liu, X\. Xie, X\. Liu, X\. Yang, X\. Li, X\. Su, X\. Lin, X\. Q\. Li, X\. Jin, X\. Shen, X\. Chen, X\. Sun, X\. Wang, X\. Song, X\. Zhou, X\. Wang, X\. Shan, Y\. K\. Li, Y\. Q\. Wang, Y\. X\. Wei, Y\. Zhang, Y\. Xu, Y\. Li, Y\. Zhao, Y\. Sun, Y\. Wang, Y\. Yu, Y\. Zhang, Y\. Shi, Y\. Xiong, Y\. He, Y\. Piao, Y\. Wang, Y\. Tan, Y\. Ma, Y\. Liu, Y\. Guo, Y\. Ou, Y\. Wang, Y\. Gong, Y\. Zou, Y\. He, Y\. Xiong, Y\. Luo, Y\. You, Y\. Liu, Y\. Zhou, Y\. X\. Zhu, Y\. Huang, Y\. Li, Y\. Zheng, Y\. Zhu, Y\. Ma, Y\. Tang, Y\. Zha, Y\. Yan, Z\. Z\. Ren, Z\. Ren, Z\. Sha, Z\. Fu, Z\. Xu, Z\. Xie, Z\. Zhang, Z\. Hao, Z\. Ma, Z\. Yan, Z\. Wu, Z\. Gu, Z\. Zhu, Z\. Liu, Z\. Li, Z\. Xie, Z\. Song, Z\. Pan, Z\. Huang, Z\. Xu, Z\. Zhang, and Z\. ZhangDeepSeek\-r1 incentivizes reasoning in llms through reinforcement learning\.Nature645\(8081\),pp\. 633–638\.External Links:ISSN 1476\-4687,[Link](http://dx.doi.org/10.1038/s41586-025-09422-z),[Document](https://dx.doi.org/10.1038/s41586-025-09422-z)Cited by:[§2\.2](https://arxiv.org/html/2609.22746#S2.SS2.p1.1)\.
- Guoet al\.\(2025b\)Y\. Guo, L\. Xu, J\. Liu, D\. Ye, and S\. QiuSegment policy optimization: effective segment\-level credit assignment in RL for large language models\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2025/hash/a6536243037d1e32c20de85137d478da-Abstract-Conference.html)Cited by:[§2\.2](https://arxiv.org/html/2609.22746#S2.SS2.p1.1)\.
- Kazemnejadet al\.\(2025\)A\. Kazemnejad, M\. Aghajohari, E\. Portelance, A\. Sordoni, S\. Reddy, A\. C\. Courville, and N\. L\. RouxVinePPO: refining credit assignment in RL training of llms\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267\.External Links:[Link](https://proceedings.mlr.press/v267/kazemnejad25a.html)Cited by:[§2\.2](https://arxiv.org/html/2609.22746#S2.SS2.p1.1)\.
- Koonceet al\.\(2008\)P\. Koonce, L\.A\. Rodegerdts, U\. States\. F\. H\. Administration, T\. H\. R\. Center, and K\. &\. AssociatesTraffic signal timing manual\.U\.S\. Department of Transportation, Federal Highway Administration\.External Links:[Link](https://books.google.co.jp/books?id=9CxW0QEACAAJ)Cited by:[1st item](https://arxiv.org/html/2609.22746#A1.I1.i1.p1.1),[§1](https://arxiv.org/html/2609.22746#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.22746#S2.SS1.p1.1),[§5\.1\.3](https://arxiv.org/html/2609.22746#S5.SS1.SSS3.p1.1)\.
- Laiet al\.\(2025\)S\. Lai, Z\. Xu, W\. Zhang, H\. Liu, and H\. XiongLLMLight: large language models as traffic signal control agents\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V\.1, KDD 2025, Toronto, ON, Canada, August 3\-7, 2025,Y\. Sun, F\. Chierichetti, H\. W\. Lauw, C\. Perlich, W\. H\. Tok, and A\. Tomkins \(Eds\.\),Toronto, ON, Canada,pp\. 2335–2346\.External Links:[Link](https://doi.org/10.1145/3690624.3709379),[Document](https://dx.doi.org/10.1145/3690624.3709379)Cited by:[1st item](https://arxiv.org/html/2609.22746#A1.I3.i1.p1.1),[§1](https://arxiv.org/html/2609.22746#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.22746#S2.SS1.p1.1),[§5\.1\.3](https://arxiv.org/html/2609.22746#S5.SS1.SSS3.p1.1)\.
- Lianget al\.\(2022\)E\. Liang, Z\. Su, C\. Fang, and R\. ZhongOAM: an option\-action reinforcement learning framework for universal multi\-intersection control\.InThirty\-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty\-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 \- March 1, 2022,pp\. 4550–4558\.External Links:[Link](https://doi.org/10.1609/aaai.v36i4.20378),[Document](https://dx.doi.org/10.1609/AAAI.V36I4.20378)Cited by:[§1](https://arxiv.org/html/2609.22746#S1.p2.1)\.
- Oroojlooyet al\.\(2020\)A\. Oroojlooy, M\. Nazari, D\. Hajinezhad, and J\. SilvaAttendLight: universal attention\-based reinforcement learning model for traffic signal control\.InAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\-12, 2020, virtual,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. Balcan, and H\. Lin \(Eds\.\),External Links:[Link](https://proceedings.neurips.cc/paper/2020/hash/29e48b79ae6fc68e9b6480b677453586-Abstract.html)Cited by:[3rd item](https://arxiv.org/html/2609.22746#A1.I2.i3.p1.1),[§1](https://arxiv.org/html/2609.22746#S1.p2.1),[§5\.1\.3](https://arxiv.org/html/2609.22746#S5.SS1.SSS3.p1.1)\.
- Ouyanget al\.\(2022\)L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. L\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. F\. Christiano, J\. Leike, and R\. LoweTraining language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 \- December 9, 2022,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html)Cited by:[§2\.2](https://arxiv.org/html/2609.22746#S2.SS2.p1.1)\.
- Qadriet al\.\(2020\)S\. S\. S\. M\. Qadri, M\. A\. Gökçe, and E\. ÖnerState\-of\-art review of traffic signal control methods: challenges and opportunities\.European Transport Research Review12\(1\),pp\. 55\.External Links:ISSN 1866\-8887,[Link](https://doi.org/10.1186/s12544-020-00439-1),[Document](https://dx.doi.org/10.1186/s12544-020-00439-1)Cited by:[§2\.1](https://arxiv.org/html/2609.22746#S2.SS1.p1.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html)Cited by:[§2\.2](https://arxiv.org/html/2609.22746#S2.SS2.p1.1)\.
- Schulmanet al\.\(2017\)J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. KlimovProximal policy optimization algorithms\.External Links:[Link](http://arxiv.org/abs/1707.06347),1707\.06347Cited by:[§2\.2](https://arxiv.org/html/2609.22746#S2.SS2.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.03300),[Document](https://dx.doi.org/10.48550/ARXIV.2402.03300),2402\.03300Cited by:[§2\.2](https://arxiv.org/html/2609.22746#S2.SS2.p1.1)\.
- Shenet al\.\(2024\)J\. Shen, J\. Hu, Q\. Zhao, and W\. RaoHeterogeneous traffic intersections control design based on reinforcement learning\.IET Intelligent Transport Systems18\(10\),pp\. 1760–1776\.External Links:[Document](https://dx.doi.org/10.1049/itr2.12408),[Link](https://ietresearch.onlinelibrary.wiley.com/doi/abs/10.1049/itr2.12408),https://ietresearch\.onlinelibrary\.wiley\.com/doi/pdf/10\.1049/itr2\.12408Cited by:[Limitations](https://arxiv.org/html/2609.22746#Sx1.p1.1)\.
- Varaiya \(2013\)P\. VaraiyaMax pressure control of a network of signalized intersections\.Transportation Research Part C: Emerging Technologies36,pp\. 177–195\.External Links:ISSN 0968\-090X,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.trc.2013.08.014),[Link](https://www.sciencedirect.com/science/article/pii/S0968090X13001782)Cited by:[2nd item](https://arxiv.org/html/2609.22746#A1.I1.i2.p1.1),[§2\.1](https://arxiv.org/html/2609.22746#S2.SS1.p1.1),[§5\.1\.3](https://arxiv.org/html/2609.22746#S5.SS1.SSS3.p1.1)\.
- Weiet al\.\(2019a\)H\. Wei, C\. Chen, G\. Zheng, K\. Wu, V\. V\. Gayah, K\. Xu, and Z\. LiPressLight: learning max pressure control to coordinate traffic signals in arterial network\.InProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4\-8, 2019,A\. Teredesai, V\. Kumar, Y\. Li, R\. Rosales, E\. Terzi, and G\. Karypis \(Eds\.\),Anchorage, AK, USA,pp\. 1290–1298\.External Links:[Link](https://doi.org/10.1145/3292500.3330949),[Document](https://dx.doi.org/10.1145/3292500.3330949)Cited by:[1st item](https://arxiv.org/html/2609.22746#A1.I2.i1.p1.1),[§1](https://arxiv.org/html/2609.22746#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.22746#S2.SS1.p1.1)\.
- Weiet al\.\(2019b\)H\. Wei, N\. Xu, H\. Zhang, G\. Zheng, X\. Zang, C\. Chen, W\. Zhang, Y\. Zhu, K\. Xu, and Z\. LiCoLight: learning network\-level cooperation for traffic signal control\.InProceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3\-7, 2019,W\. Zhu, D\. Tao, X\. Cheng, P\. Cui, E\. A\. Rundensteiner, D\. Carmel, Q\. He, and J\. X\. Yu \(Eds\.\),Beijing, China,pp\. 1913–1922\.External Links:[Link](https://doi.org/10.1145/3357384.3357902),[Document](https://dx.doi.org/10.1145/3357384.3357902)Cited by:[4th item](https://arxiv.org/html/2609.22746#A1.I2.i4.p1.1),[§1](https://arxiv.org/html/2609.22746#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.22746#S2.SS1.p1.1),[§5\.1\.3](https://arxiv.org/html/2609.22746#S5.SS1.SSS3.p1.1)\.
- Weiet al\.\(2019c\)H\. Wei, G\. Zheng, V\. V\. Gayah, and Z\. LiA survey on traffic signal control methods\.External Links:[Link](http://arxiv.org/abs/1904.08117),1904\.08117Cited by:[§1](https://arxiv.org/html/2609.22746#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.22746#S2.SS1.p1.1),[§5\.1\.1](https://arxiv.org/html/2609.22746#S5.SS1.SSS1.p1.1)\.
- Weiet al\.\(2018\)H\. Wei, G\. Zheng, H\. Yao, and Z\. LiIntelliLight: A reinforcement learning approach for intelligent traffic light control\.InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19\-23, 2018,Y\. Guo and F\. Farooq \(Eds\.\),London, UK,pp\. 2496–2505\.External Links:[Link](https://doi.org/10.1145/3219819.3220096),[Document](https://dx.doi.org/10.1145/3219819.3220096)Cited by:[§1](https://arxiv.org/html/2609.22746#S1.p2.1)\.
- Wuet al\.\(2023\)Q\. Wu, M\. Li, J\. Shen, L\. Lü, B\. Du, and K\. ZhangTransformerLight: A novel sequence modeling based traffic signaling mechanism via gated transformer\.InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2023, Long Beach, CA, USA, August 6\-10, 2023,A\. K\. Singh, Y\. Sun, L\. Akoglu, D\. Gunopulos, X\. Yan, R\. Kumar, F\. Ozcan, and J\. Ye \(Eds\.\),pp\. 2639–2647\.External Links:[Link](https://doi.org/10.1145/3580305.3599530),[Document](https://dx.doi.org/10.1145/3580305.3599530)Cited by:[§1](https://arxiv.org/html/2609.22746#S1.p1.1),[§1](https://arxiv.org/html/2609.22746#S1.p2.1)\.
- Wuet al\.\(2021\)Q\. Wu, L\. Zhang, J\. Shen, L\. Lü, B\. Du, and J\. WuEfficient pressure: improving efficiency for signalized intersections\.External Links:[Link](https://arxiv.org/abs/2112.02336),2112\.02336Cited by:[5th item](https://arxiv.org/html/2609.22746#A1.I2.i5.p1.1),[§5\.1\.3](https://arxiv.org/html/2609.22746#S5.SS1.SSS3.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. QiuQwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§5\.1\.5](https://arxiv.org/html/2609.22746#S5.SS1.SSS5.p1.1)\.
- Yuet al\.\(2020\)B\. Yu, J\. Guo, Q\. Zhao, J\. Li, and W\. RaoSmarter and safer traffic signal controlling via deep reinforcement learning\.InCIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19\-23, 2020,M\. d’Aquin, S\. Dietze, C\. Hauff, E\. Curry, and P\. Cudré\-Mauroux \(Eds\.\),pp\. 3345–3348\.External Links:[Link](https://doi.org/10.1145/3340531.3417450),[Document](https://dx.doi.org/10.1145/3340531.3417450)Cited by:[§1](https://arxiv.org/html/2609.22746#S1.p2.1)\.
- Yuanet al\.\(2026\)Z\. Yuan, S\. Lai, and H\. LiuCoLLMLight: cooperative large language model agents for network\-wide traffic signal control\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=KeJqoEVOeY)Cited by:[3rd item](https://arxiv.org/html/2609.22746#A1.I3.i3.p1.1),[§1](https://arxiv.org/html/2609.22746#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.22746#S2.SS1.p1.1),[§5\.1\.3](https://arxiv.org/html/2609.22746#S5.SS1.SSS3.p1.1)\.
- Zhanget al\.\(2019\)H\. Zhang, S\. Feng, C\. Liu, Y\. Ding, Y\. Zhu, Z\. Zhou, W\. Zhang, Y\. Yu, H\. Jin, and Z\. LiCityFlow: a multi\-agent reinforcement learning environment for large scale city traffic scenario\.InThe World Wide Web Conference,WWW ’19,New York, NY, USA,pp\. 3620–3624\.External Links:ISBN 9781450366748,[Link](https://doi.org/10.1145/3308558.3314139),[Document](https://dx.doi.org/10.1145/3308558.3314139)Cited by:[§5\.1\.2](https://arxiv.org/html/2609.22746#S5.SS1.SSS2.p1.1)\.
- Zhanget al\.\(2025\)J\. Zhang, J\. Huang, H\. Yao, S\. Liu, X\. Zhang, S\. Lu, and D\. TaoR1\-VL: learning to reason with multimodal large language models via step\-wise group relative policy optimization\.InIEEE/CVF International Conference on Computer Vision, ICCV 2025, Honolulu, HI, USA, October 19\-25, 2025,Honolulu, HI, USA,pp\. 1859–1869\.External Links:[Link](https://doi.org/10.1109/ICCV51701.2025.00181),[Document](https://dx.doi.org/10.1109/ICCV51701.2025.00181)Cited by:[§A\.9](https://arxiv.org/html/2609.22746#A1.SS9.p1.1),[§4\.2\.4](https://arxiv.org/html/2609.22746#S4.SS2.SSS4.p3.1)\.
- Zhanget al\.\(2022\)L\. Zhang, Q\. Wu, J\. Shen, L\. Lü, B\. Du, and J\. WuExpression might be enough: representing pressure and demand for reinforcement learning based traffic signal control\.InInternational Conference on Machine Learning, ICML 2022, 17\-23 July 2022, Baltimore, Maryland, USA,K\. Chaudhuri, S\. Jegelka, L\. Song, C\. Szepesvári, G\. Niu, and S\. Sabato \(Eds\.\),Proceedings of Machine Learning Research,Baltimore, Maryland, USA,pp\. 26645–26654\.External Links:[Link](https://proceedings.mlr.press/v162/zhang22ah.html)Cited by:[6th item](https://arxiv.org/html/2609.22746#A1.I2.i6.p1.1),[§1](https://arxiv.org/html/2609.22746#S1.p2.1),[§5\.1\.3](https://arxiv.org/html/2609.22746#S5.SS1.SSS3.p1.1)\.
- Zouet al\.\(2025\)X\. Zou, Y\. Yang, Z\. Chen, X\. Hao, Y\. Chen, C\. Huang, and Y\. LiangHuggingFace url of traffic\-r1\-3b \(public 0\.1\)\.Note:[https://huggingface\.co/Season998/Traffic\-R1](https://huggingface.co/Season998/Traffic-R1)\[Online; accessed 20\-Aug\-2026\]Cited by:[§5\.1\.5](https://arxiv.org/html/2609.22746#S5.SS1.SSS5.p1.1)\.
- Zouet al\.\(2026\)X\. Zou, Y\. Yang, Z\. Chen, X\. Hao, Y\. Chen, C\. Huang, and Y\. LiangTraffic\-r1: reinforced llms bring human\-like reasoning to traffic signal control systems\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2026, San Diego, California, United States, July 2\-7, 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),pp\. 21823–21838\.External Links:[Link](https://doi.org/10.18653/v1/2026.acl-long.995),[Document](https://dx.doi.org/10.18653/V1/2026.ACL-LONG.995)Cited by:[2nd item](https://arxiv.org/html/2609.22746#A1.I3.i2.p1.1),[§1](https://arxiv.org/html/2609.22746#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.22746#S2.SS1.p1.1),[§5\.1\.3](https://arxiv.org/html/2609.22746#S5.SS1.SSS3.p1.1)\.

## Appendix AAppendix

### A\.1Concepts in traffic signal control

We illustrate the commonly used settings in traffic signal control in Figure[6](https://arxiv.org/html/2609.22746#A1.F6)\.

### A\.2Baseline Methods

We categorize the baselines into traditional methods, reinforcement learning methods, and large language model methods\.

#### A\.2\.1Traditional Methods\.

- •FixedTime[Koonce et al\. \(2008\)](https://arxiv.org/html/2609.22746#bib.bib6): FixedTime uses predefined cycle lengths, phase orders, and green durations, providing a simple and stable controller that does not adapt to real\-time traffic demand\.
- •MaxPressure[Varaiya \(2013\)](https://arxiv.org/html/2609.22746#bib.bib5): MaxPressure selects the phase with the largest pressure difference between upstream queues and downstream traffic conditions, achieving strong queue dissipation through transportation\-theoretic control\.

#### A\.2\.2RL\-based Methods\.

- •PressLight[Wei et al\. \(2019a\)](https://arxiv.org/html/2609.22746#bib.bib12): PressLight applies deep reinforcement learning to optimize intersection pressure, combining adaptive signal control with a congestion\-oriented traffic objective\.
- •MPLight[Chen et al\. \(2020\)](https://arxiv.org/html/2609.22746#bib.bib11): MPLight builds on FRAP and uses pressure as both observation and reward, enabling pressure\-aware phase competition modeling for traffic signal control\.
- •AttendLight[Oroojlooy et al\. \(2020\)](https://arxiv.org/html/2609.22746#bib.bib7): AttendLight employs attention mechanisms to construct traffic observations and model phase transition dependencies, allowing the agent to focus on critical movements\.
- •CoLight[Wei et al\. \(2019b\)](https://arxiv.org/html/2609.22746#bib.bib8): CoLight uses graph attention networks for communication among neighboring intersections, enabling cooperative multi\-agent signal control over road networks\.
- •Efficient\-CoLight[Wu et al\. \(2021\)](https://arxiv.org/html/2609.22746#bib.bib9): Efficient\-CoLight improves CoLight by introducing efficient pressure observations, reducing redundant state information while preserving key coordination signals\.
- •Advanced\-CoLight[Zhang et al\. \(2022\)](https://arxiv.org/html/2609.22746#bib.bib20): Advanced\-CoLight extends CoLight with richer traffic state features such as pressure and effective running vehicles, improving expressive multi\-intersection coordination\.

#### A\.2\.3LLM\-based Methods\.

- •LLMLight[Lai et al\. \(2025\)](https://arxiv.org/html/2609.22746#bib.bib16): LLMLight uses an LLM\-based traffic signal control agent to interpret traffic states and generate signal actions through natural\-language traffic reasoning\.
- •Traffic\-R1[Zou et al\. \(2026\)](https://arxiv.org/html/2609.22746#bib.bib17): Traffic\-R1 develops an LLM\-based traffic signal control agent with a two\-stage reinforcement learning framework to improve real\-world deployment capability\.
- •CoLLMLight[Yuan et al\. \(2026\)](https://arxiv.org/html/2609.22746#bib.bib15): CoLLMLight coordinates multiple LLM\-based traffic signal control agents with cost\-aware reasoning depth, balancing cooperative decision quality and inference overhead\.

![Refer to caption](https://arxiv.org/html/2609.22746v1/preliminary.png)Figure 6:Illustration of \(a\) intersections, \(b\) traffic movements, and \(c\) signal phases\.

### A\.3Metrics Details

Average Travel Time \(ATT\)\.ATT measures the average time required for vehicles to complete their trips within the road network\. It directly reflects the overall traffic efficiency under the learned signal control policy\. Formally, ATT is defined as

ATT=1N​∑v=1N\(tvarrive−tvdepart\),\\mathrm\{ATT\}=\\frac\{1\}\{N\}\\sum\_\{v=1\}^\{N\}\\left\(t\_\{v\}^\{\\mathrm\{arrive\}\}\-t\_\{v\}^\{\\mathrm\{depart\}\}\\right\),\(14\)whereNNdenotes the total number of vehicles, andtvdepartt\_\{v\}^\{\\mathrm\{depart\}\}andtvarrivet\_\{v\}^\{\\mathrm\{arrive\}\}denote the departure and arrival times of vehiclevv, respectively\.

Average Waiting Time \(AWT\)\.AWT measures the average accumulated waiting time of vehicles during the trip\. A vehicle is considered waiting when its speed falls below a predefined threshold\. Lower AWT indicates reduced intersection delay and smoother traffic flow\. It is computed as

AWT=1N​∑v=1Nwv,\\mathrm\{AWT\}=\\frac\{1\}\{N\}\\sum\_\{v=1\}^\{N\}w\_\{v\},\(15\)wherewvw\_\{v\}is the total waiting time of vehiclevv\.

Average Queue Length \(AQL\)\.AQL measures the average number of queuing vehicles on all lanes and decision steps, indicating the congestion level of the traffic network\. It is defined as

AQL=1T​\|ℒ\|​∑t=1T∑l∈ℒqlt,\\mathrm\{AQL\}=\\frac\{1\}\{T\|\\mathcal\{L\}\|\}\\sum\_\{t=1\}^\{T\}\\sum\_\{l\\in\\mathcal\{L\}\}q\_\{l\}^\{t\},\(16\)whereTTdenotes the total number of simulation steps,ℒ\\mathcal\{L\}denotes the set of lanes, andqltq\_\{l\}^\{t\}represents the queue length of lanellat simulation steptt\.

### A\.4Datasets

All intersections in the Jinan and Hangzhou datasets adopt four\-way topology\.

- •Jinan: A dataset from the Dongfeng sub\-district in Jinan, China, consisting of 12 intersections with three traffic flow datasets\. Every intersection is composed of two 400\-meter east\-west roads and two 800\-meter north\-south roads\.
- •Hangzhou: A dataset from the Gudang sub\-district in Hangzhou, China, consisting of 16 intersections with two traffic flow datasets\. Every intersection is composed of two 800\-meter east\-west roads and two 600\-meter north\-south roads\.
- •Synthetic\-1: A synthetic dataset consisting of 16 intersections and each road between two intersections is 300 meters long\.

### A\.5STeP\-PO Algorithm

As shown in Algorithm[1](https://arxiv.org/html/2609.22746#alg1), STeP\-PO consists of three sequential stages: training data collection, policy warm\-up, and step\-wise policy optimization\.

Algorithm 1STeP\-PO1:Policy model

πθ\\pi\_\{\\theta\}initialized by a pre\-trained LLM; teacher model

πtea\\pi\_\{\\mathrm\{tea\}\}; traffic simulator

𝒮\\mathcal\{S\}; traffic scenarios

𝒞\\mathcal\{C\}; group size

GG
2:Trained traffic signal control policy

πθ\\pi\_\{\\theta\}
3:Training data collection:

4:Collect traffic scenarios from

𝒮\\mathcal\{S\}and construct input prompts following Section[4\.1](https://arxiv.org/html/2609.22746#S4.SS1)

5:Select oracle phase actions through network\-level simulation and collect teacher\-generated step\-wise reasoning trajectories

6:Build dataset

𝒟=\{\(x,y\)\}\\mathcal\{D\}=\\\{\(x,y\)\\\}and split it into

𝒟SFT\\mathcal\{D\}\_\{\\mathrm\{SFT\}\}and

𝒟RL\\mathcal\{D\}\_\{\\mathrm\{RL\}\}
7:Policy warm\-up:

8:foreach sample

\(x,y\)∈𝒟SFT\(x,y\)\\in\\mathcal\{D\}\_\{\\mathrm\{SFT\}\}do

9:Optimize

πθ\\pi\_\{\\theta\}with the supervised warm\-up objective in Eq\.[2](https://arxiv.org/html/2609.22746#S4.E2)

10:endfor

11:Set reference policy

πref←πθ\\pi\_\{\\mathrm\{ref\}\}\\leftarrow\\pi\_\{\\theta\}
12:Step\-wise policy optimization:

13:foreach sample

\(x,y⋆\)∈𝒟RL\(x,y^\{\\star\}\)\\in\\mathcal\{D\}\_\{\\mathrm\{RL\}\}do

14:Generate a group of reasoning trajectories

\{yi\}i=1G∼πθ\(⋅\|x\)\\\{y\_\{i\}\\\}\_\{i=1\}^\{G\}\\sim\\pi\_\{\\theta\}\(\\cdot\|x\)
15:Compute step quality rewards using the reference trajectory

y⋆y^\{\\star\}by Eq\.[3](https://arxiv.org/html/2609.22746#S4.E3)

16:Estimate phase\-sensitive step importance by Eq\.[5](https://arxiv.org/html/2609.22746#S4.E5)

17:Construct the semantic step tree and compute step advantages by Eq\.[9](https://arxiv.org/html/2609.22746#S4.E9)

18:Optimize

πθ\\pi\_\{\\theta\}with the STeP\-PO objective in Eq\.[12](https://arxiv.org/html/2609.22746#S4.E12)

19:endfor

20:return

πθ\\pi\_\{\\theta\}

### A\.6Semantic Step Tree Construction

We present the semantic step tree construction algorithm in Algorithm[2](https://arxiv.org/html/2609.22746#alg2)\.

Algorithm 2Semantic Step Tree Construction1:Input prompt

xx; sampled trajectories

\{yi\}i=1G\\\{y\_\{i\}\\\}\_\{i=1\}^\{G\}, where

yi=\(si1,…,siK\)y\_\{i\}=\(s\_\{i\}^\{1\},\\ldots,s\_\{i\}^\{K\}\); final action of each trajectory

aia\_\{i\}\.

2:Semantic step tree

𝒯\\mathcal\{T\}\.

3:Initialize

𝒯\\mathcal\{T\}with a root node

n0n\_\{0\}representing the input prompt

xx\.

4:for

i=1i=1to

GGdo

5:Set the current node

n←n0n\\leftarrow n\_\{0\}\.

6:for

k=1k=1to

KKdo

7:Extract the claim set

CikC\_\{i\}^\{k\}from reasoning step

siks\_\{i\}^\{k\}\.

8:Search among the children of

nnfor a node

n′n^\{\\prime\}at step

kkwhose claim set is semantically matched with

CikC\_\{i\}^\{k\}\.

9:ifsuch a node

n′n^\{\\prime\}existsthen

10:Merge

siks\_\{i\}^\{k\}into

n′n^\{\\prime\}and add it to the step set

𝒮⁡\(n′\)\\mathcal\{S\}\(n^\{\\prime\}\)\.

11:Set

n←n′n\\leftarrow n^\{\\prime\}\.

12:else

13:Create a new child node

n′n^\{\\prime\}under

nn\.

14:Store

CikC\_\{i\}^\{k\},

siks\_\{i\}^\{k\}, and its step index

kkin

n′n^\{\\prime\}\.

15:Set

𝒮⁡\(n′\)=\{sik\}\\mathcal\{S\}\(n^\{\\prime\}\)=\\\{s\_\{i\}^\{k\}\\\}and

n←n′n\\leftarrow n^\{\\prime\}\.

16:endif

17:endfor

18:Attach the final action

aia\_\{i\}as a leaf node under

nn\.

19:endfor

20:return

𝒯\\mathcal\{T\}\.

### A\.7Hyperparameters Analysis

Table 6:Hyperparameter sensitivity of STeP\-PO on Jinan1 and Hangzhou1\.We analyze the sensitivity of STeP\-PO to three key hyperparameters on Jinan1 and Hangzhou1 using ATT as the evaluation metric\. Specifically,η\\etacontrols the strength of phase\-sensitive step importance weighting,λ\\lambdabalances the step\-level advantage and outcome\-level feedback, andGGdenotes the number of sampled reasoning trajectories used to construct the semantic step tree\. As shown in Table[6](https://arxiv.org/html/2609.22746#A1.T6), STeP\-PO achieves the best performance withη=0\.6\\eta=0\.6,λ=0\.3\\lambda=0\.3, andG=8G=8, which is used as the default setting in our main experiments\. IncreasingGGfrom 4 to 8 consistently reduces ATT on both datasets, indicating that more rollouts improve reasoning diversity and provide richer sibling branches for more stable advantage estimation\.

The results also show that bothλ\\lambdaandη\\etashould be set to moderate values\. A properλ\\lambdaimproves performance by incorporating step\-quality supervision into policy optimization, while a larger value degrades ATT because overemphasizing step\-level rewards weakens the supervision from the final phase outcome\. Similarly, increasingη\\etafrom 0\.3 to 0\.6 improves ATT, demonstrating that reasonable importance weighting helps optimize decision\-critical reasoning steps\. Nevertheless, whenη\\etais further increased to 0\.9, performance declines, implying that excessive reliance on a few highly sensitive steps may ignore the contribution of the complete reasoning trajectory\. These results confirm that STeP\-PO benefits from a balanced combination of rollout diversity, step\-quality supervision, and phase\-sensitive credit allocation\.

### A\.8Statistical Significance Analysis

Each model is evaluated using five random seeds \(11, 42, 1024, 2026, 10001\) on Jinan1 and Hangzhou1\. Table[7](https://arxiv.org/html/2609.22746#A1.T7)reports the mean and standard deviation across seeds\. ProcessLight achieves the best mean performance on most metrics, and its coefficient of variation ranges from 0\.44% to 9\.76%, indicating limited sensitivity to random\-seed variation\.

To further assess robustness, we conduct two\-sided Welch’s t\-tests comparing each baseline against ProcessLight and apply one global Holm correction across all 90 comparisons\. ProcessLight obtains a lower mean in 88 of the 90 comparisons; 81 comparisons significantly favor ProcessLight after correction, 9 show no detectable difference, and none significantly favors a baseline\. The nine non\-significant comparisons involve AdvancedCoLight on Jinan1 \(ATT\) and Hangzhou1 \(AWT/AQL\), LLMLight on Jinan1 \(ATT/AWT\) and Hangzhou1 \(AWT\), and Traffic\-R1 on Jinan1 \(ATT/AWT\) and Hangzhou1 \(AWT\)\. Although AdvancedCoLight achieves lower mean AWT and AQL on Hangzhou1, neither difference remains statistically significant after Holm correction\.

Table 7:Mean and standard deviation over five random seeds on Jinan1 and Hangzhou1\. Table[2](https://arxiv.org/html/2609.22746#S5.T2)reports results under the default seed\. Best results are in bold\.
### A\.9Reliability of Claim Verification

The step\-quality evaluation pipeline consists of three stages: regular\-expression parsing of the generated claims, soft semantic matching against the reference key claims[Zhang et al\. \(2025\)](https://arxiv.org/html/2609.22746#bib.bib18), and observation\-based validity checking\. To quantify the reliability of each stage, we randomly sample 100 instances, roll out 8 responses for each prompt, and conduct three independent evaluations using DeepSeek\-V4\-Pro as the judge model[Chen et al\. \(2025\)](https://arxiv.org/html/2609.22746#bib.bib34)\. As shown in Table[8](https://arxiv.org/html/2609.22746#A1.T8), all three stages achieve high execution accuracy, supporting the reliability of the step\-quality signal for policy optimization\.

Table 8:Execution accuracy of the three verification stages\.Table 9:Cost of the two data\-construction stages\.Table 10:Zero\-shot performance under restricted observation without neighboring\-intersection information\. Best results are in bold\.Table 11:Zero\-shot performance with different backbones and teacher models\. Best results are in bold\.
### A\.10Generalization across Backbones and Teachers

We further evaluate ProcessLight with Qwen3\-8B and Llama3\-8B as backbone models, and with Qwen3\-Plus as the teacher model for training\-data collection\. Table[11](https://arxiv.org/html/2609.22746#A1.T11)reports the zero\-shot performance across the five real\-world datasets\. ProcessLight remains effective across the tested settings: all three backbone variants outperform LLMLight, CoLLMLight, and Traffic\-R1 on most metrics, and the Qwen3\-8B variant improves over the Qwen2\.5\-3B variant on 12 of the 15 metrics\. These results provide initial evidence that the method is not tied to a single backbone family or to the original teacher model; our conclusions are delimited to the models evaluated here\.

### A\.11Restricted\-Observation Comparison

To isolate the contribution of the training strategy from the richer observation, we remove neighboring\-intersection information and the corresponding reasoning guidance from the ProcessLight input, and evaluate the modified model on all five datasets\. As shown in Table[10](https://arxiv.org/html/2609.22746#A1.T10), the restricted variant still outperforms LLMLight on most metrics\. We note that the restricted\-observation experiment serves as a fairness control that isolates the contribution of the training strategy, whereas the full model reflects the intended network\-coordination problem setting\.

### A\.12Computational Cost Analysis

Prior to RL optimization, we precompute the step\-importance scores for each prompt and reuse them across all rollouts associated with that prompt; compared with GRPO, step\-importance estimation therefore introduces no additional model forward or backward passes during training\. ForNNprompts withKKreasoning steps, generating the reference claims requiresNNcalls to the teacher model, while step\-importance estimation requiresN⁡\(K\+1\)N\(K\+1\)calls to the SFT\-initialized policy model—one for the complete response andKKfor the step\-masked variants\. Step\-importance estimation thus accounts forK\+1K\+2\\frac\{K\+1\}\{K\+2\}of all model calls, i\.e\.,87\.5%87\.5\\%withK=6K=6\. This call\-count proportion overestimates the practical overhead, because these calls use a locally deployed 3B model for forward\-only scoring, without autoregressive decoding or additional traffic simulation\. As shown in Table[9](https://arxiv.org/html/2609.22746#A1.T9), step\-importance estimation accounts for only 11\.6% of the combined monetary cost of the data\-construction pipeline\.

### A\.13Prompt Template

Youareanexperttrafficsignalcontrolagentoperatingunderastrictreasoningframework\.

Yourtaskistoselectexactlyonelegalphaseforthecurrentintersectionbasedontheprovidedtrafficfacts\.

\#\#BackgroundContext

Youareatrafficsignalcontrolleratafour\-wayintersectionwith12movements:

\[NL,NT,NR,SL,ST,SR,EL,ET,ER,WL,WT,WR\]and4phases

\[ETWT,NTST,ELWL,NLSL\]\.Eachphasereleasestwocompatiblemovements:

ETWTreleasesETandWT,NTSTreleasesNTandST,ELWLreleasesELandWL,

andNLSLreleasesNLandSL\.

Legalphasesforthisdecision:\[ETWT,NTST,ELWL,NLSL\]\.

\#\#DecisionRules

1\.Prioritizelocalintersectionpressurerelief;grantright\-of\-wayfirsttolongqueuesandstop\-linenearbyvehicles\.

2\.Assessdownstreamcongestionbeforereleasinganymovement\.Avoiddischargingvehiclesintocongesteddownstreamsections\.

3\.Useadjacentintersectionsonlyaslocalriskreferenceinthisdecisionstep\.

4\.Avoidprolongedsingle\-lanewaitingwhendownstreamconditionspermit\.

5\.Occupancy0\-0\.2indicatesfreeflow,0\.2\-0\.4basicallysmooth,above0\.4mildcongestion,andabove0\.6severecongestion\.

\#\#CurrentIntersectionState

\#\#\#MovementState

\|Movement\|Phase\|EarlyQueueLength\|DownstreamQueueLength\|Pressure\|AvgWaitingTime\(minutes\)\|SegmentVehOccupancy\|VehicleCount\|

\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|

\|ET\|ETWT\|0\|10\|\-3\.3\|0\.1\|\[0,0\.2,0\]\|3\|

\|WT\|ETWT\|0\|2\|\-0\.7\|0\|\[0,0\.1,0\]\|2\|

\|NT\|NTST\|1\|0\|1\|0\.1\|\[0,0\.1,0\.1\]\|9\|

\|ST\|NTST\|1\|5\|\-0\.7\|0\|\[0\.1,0,0\.1\]\|10\|

\|EL\|ELWL\|4\|0\|4\|0\.3\|\[0,0,0\.1\]\|6\|

\|WL\|ELWL\|2\|5\|0\.3\|0\.8\|\[0,0,0\]\|2\|

\|NL\|NLSL\|2\|2\|1\.3\|0\.6\|\[0,0,0\]\|3\|

\|SL\|NLSL\|2\|10\|\-1\.3\|0\.2\|\[0,0,0\]\|4\|

\#\#\#PhaseAggregation

Phaserankingbyaggregatedsignalpressure:ELWL\>NTST\>NLSL\>ETWT

Highestpressurephasefromcomputedindicators:ELWL

\|Phase\|Movement\|EarlyQueueLength\|DownstreamQueueLength\|Pressure\|AvgWaitingTime\(minutes\)\|SegmentVehCount\|VehicleCount\|

\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|

\|ETWT\|ET,WT\|0\|12\|\-4\|0\|\[0,3,2\]\|5\|

\|NTST\|NT,ST\|2\|5\|0\.3\|0\|\[1,4,12\]\|19\|

\|ELWL\|EL,WL\|6\|5\|4\.3\|0\.4\|\[0,0,2\]\|8\|

\|NLSL\|NL,SL\|4\|12\|0\|0\.4\|\[0,0,3\]\|7\|

\#\#AdjacentIntersectionContext

\|Direction\|IntersectionID\|ET\|WT\|NT\|ST\|EL\|WL\|NL\|SL\|RelatedMovements\|

\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|

\|East\|intersection\_1\_3\|11,0\.5\|0,0\.1\|4,0\.1\|14,0\.2\|10,0\.2\|2,0\|4,0\.1\|3,0\.1\|ET,SL\-\>ET,EL\|

\|West\|intersection\_3\_3\|6,0\.2\|1,0\|0,0\.1\|0,0\.1\|4,0\.1\|2,0\|0,0\|3,0\.1\|WT,NL\-\>WT,WL\|

\|North\|intersection\_2\_2\|5,0\.2\|5,0\.2\|0,0\.1\|0,0\.1\|1,0\|2,0\.1\|2,0\|5,0\.1\|NT,EL\-\>NT,NL\|

\#\#RecentHistory

Recentphases:NLSL\-\>ETWT\-\>NTST\-\>ETWT

Lastphase:ETWT

\|Phase\|QueueChange\|PressureChange\|WaitingChange\|ServiceGap\|TrendSummary\|

\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|

\|ETWT\|6\-\>0\-\>7\-\>0\|5\.7\-\>\-2\.3\-\>4\.3\-\>\-4\|0\.2\-\>0\-\>0\.1\-\>0min\|0steps\|recentlyservedandclearlyrelieved\|

\|NTST\|7\-\>11\-\>0\-\>2\|5\.3\-\>9\.3\-\>\-2\.7\-\>0\.3\|0\.3\-\>0\.4\-\>0\-\>0min\|1steps\|recentlyservedandclearlyrelieved\|

\|ELWL\|1\-\>2\-\>3\-\>6\|\-0\.7\-\>0\.3\-\>0\.3\-\>4\.3\|0\.1\-\>0\.2\-\>0\.3\-\>0\.4min\|5steps\|unservedforlonganddemandisworsening\|

\|NLSL\|0\-\>2\-\>4\-\>4\|\-0\.3\-\>\-0\.3\-\>1\.3\-\>0\|0\-\>0\.1\-\>0\.2\-\>0\.4min\|3steps\|demandisincreasing\|

\#\#OutputPrompt

Reasonstep\-by\-stepviasevenfixedLevel\-2headingsections:

PhaseMapping,LocalDemand,Downstream,NeighborCoordination,FlowTrend,Fairness,andFinalAnswer\.

Eachreasoningsectionshouldcontainnomorethanthreeatomic<claim\>tags\.

Onlymathematicallyderivableconclusionsfollowingtheunifiedsyntaxmaybeplacedwithin<claim\>\.

Thefinalselectedphasemustbeenclosedby<phase\></phase\>\.

Table 12:The prompt template\#\#PhaseMapping

Phasemappinglinkseachvalidsignalphasetoitscorrespondingtrafficmovements\.Thephasesarerankedbyaggregatedpressuretoidentifypriority\.

<claim\>RANK\[Pressure\]\[Phase\]=ELWL\>NTST\>NLSL\>ETWT</claim\>

<claim\>MAX\[EarlyQueueLength\]\[Phase\]=ELWL</claim\>

<claim\>MAX\[AvgWaitingTime\]\[Phase\]=ELWL</claim\>

\#\#LocalDemand

ELWLexhibitsthehighestlocaldemandwithanaggregatedpressureof4\.3andanearlyqueuelengthof6,bothexceedingallotherphases\.WL’saveragewaitingtimeof0\.8minutesisthehighestamongmovements\.

<claim\>\[ELWL\]\[Pressure\]=4\.3</claim\>

<claim\>\[ELWL\]\[EarlyQueueLength\]=6</claim\>

<claim\>\[WL\]\[AvgWaitingTime\]=0\.8</claim\>

\#\#Downstream

Severaldownstreamsegmentsshowhighqueuelengthsthatcouldlimitdischarge\.SLandETeachhaveadownstreamqueueof10,whileWLhasadownstreamqueueof5,indicatingpotentialcongestionbutstillwithinmanageablelimitsforELWL’sleft\-turnmovement\.

<claim\>\[SL\]\[DownstreamQueueLength\]=10</claim\>

<claim\>\[ET\]\[DownstreamQueueLength\]=10</claim\>

<claim\>\[WL\]\[DownstreamQueueLength\]=5</claim\>

\#\#NeighborCoordination

Adjacentintersectionsprovidecontextforinflowandoutflow\.West\-boundWLhasalowqueueof2,allowingdischarge\.East\-boundETshowsahighoccupancyof0\.5,suggestingsomedownstreamconstraint,andNorth\-boundNThaslowoccupancyof0\.1\.

<claim\>ADJ\[West\]\[WL\]\[Queue\]=2</claim\>

<claim\>ADJ\[East\]\[ET\]\[Occupancy\]=0\.5</claim\>

<claim\>ADJ\[North\]\[NT\]\[Occupancy\]=0\.1</claim\>

\#\#FlowTrend

ELWLhasbeenunservedfor5stepswithworseningdemandandincreasingpressureandqueue,indicatingurgentneedforservice\.

<claim\>TREND\[ELWL\]\[ServiceGap\]=5steps\(unserved\)</claim\>

<claim\>TREND\[ELWL\]\[PressureChange\]=increasing</claim\>

<claim\>TREND\[ELWL\]\[QueueChange\]=increasing</claim\>

\#\#Fairness

WLholdsthemaximumaveragewaitingtimeamongallmovements,butnomovementexceedsthe10\-minutethreshold\.TheservicegaprankingconfirmsELWLhasbeenwaitinglongest\.

<claim\>MAX\[AvgWaitingTime\]\[Movement\]=WL</claim\>

<claim\>No\[Movement\]has\[AvgWaitingTime\]\>=10</claim\>

<claim\>RANK\[ServiceGap\]\[Phase\]=ELWL\>NLSL\>NTST\>ETWT</claim\>

\#\#FinalAnswer

<phase\>ELWL</phase\>

Table 13:Example of ProcessLight response\.