Explainable Reinforcement Learning for Adaptive Traffic Signal Control

# Explainable Reinforcement Learning for Adaptive Traffic Signal Control
Source: [https://arxiv.org/html/2607.03703](https://arxiv.org/html/2607.03703)
Dickens Kwesiga, Nishu Choudhary, Angshuman Guin, and Michael HunterThe authors are with the School of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA \(e\-mail: dkwesiga@gatech\.edu\)\.Manuscript received June 30, 2026\.

###### Abstract

Model\-free Reinforcement Learning \(RL\) has emerged as a powerful paradigm for adaptive traffic signal control\. This is partly because state\-of\-the\-art RL agents can directly interact with traffic simulation environments to learn highly complex, non\-linear traffic control policies without relying on complex predictive models\. However, in safety\-critical infrastructure like traffic control, the opaque, black\-box nature of deep RL models poses challenges for transportation agency acceptance, regulatory compliance, operational trust, troubleshooting, and fine\-tuning by traffic engineers\. To bridge this gap between high\-performance optimization and human\-comprehensible interpretability, this effort introduces a novel, explainable entity\-centric RL framework for safe and transparent traffic signal control\. Rather than processing traffic states through monolithic, flat vectors, the proposed architecture disaggregates real\-time intersection observations into distinct, high\-dimensional lane entities \(acting as Queries\) and phase temporal configurations \(acting as Keys and Values\) to inherently preserve the structural topology and geometric configurations of the intersection\. Relational dependencies and inter\-lane conflicts are dynamically extracted via a dual\-stage attention network featuring sequential multi\-head cross\-attention and self\-attention blocks\. This design yields a real\-time affinity matrix that quantifies the direct influence of signal phases on specific approach volumes and queues, providing full visual and analytical interpretability\. To ensure strict operational reliability, a deterministic action\-masking interface is integrated directly into the Proximal Policy Optimization \(PPO\) pipeline, explicitly blocking invalid phase transitions to guarantee absolute compliance with established signal timing norms and safety constraints\. Evaluated in a microscopic simulation environment under diverse traffic demands, the framework matches or outperforms state\-of\-the\-art baselines in delay minimization\. More importantly, the emergent attention weights align precisely with established traffic engineering principles, tracking queue clearance and coordinated phase transitions, offering an auditable, trust\-enabling, and deployable architecture for next\-generation adaptive traffic control systems\.

## IIntroduction

Model\-free Reinforcement Learning \(RL\) has emerged as a powerful paradigm for adaptive traffic signal control\. By interacting directly with microscopic traffic simulation environments, state\-of\-the\-art RL agents can learn highly complex, non\-linear optimization policies without relying on complex predictive models\. These frameworks map real\-time traffic observations directly to optimal signaling decisions, consistently demonstrating superior performance over conventional actuated and fixed\-time controllers in simulated environments\.

Despite the demonstrated successes of RL\-based traffic signal controllers in literature, the opaque, black\-box nature of deep RL models poses challenges for transportation agency acceptance, operational trust, troubleshooting, and fine\-tuning by traffic engineers\. Previous deep RL models produce control actions without exposing the underlying causal logic or structural justifications behind their decisions\. In safety\-critical infrastructure such as traffic control, the absence of explainability introduces significant barriers to deployment, including challenges in regulatory compliance and operational trust\. Additionally, black\-box models offer no structural diagnostics to distinguish between anomalous sensor inputs, hardware failures and poor policy inference limiting the ability of traffic engineering teams to easily troubleshoot\. Furthermore, RL algorithms with obscured internal logic make it impossible for engineers to fine\-tune or inject domain knowledge into the control process\. There is a clear need for an interpretable, structurally organized RL framework that bridges the gap between high\-performance deep optimization and human\-comprehensible explainability\.

This paper introduces an explainable, entity\-centric RL framework designed specifically for safe and transparent traffic signal control\. By shifting from flat vector spaces to a decoupled, high\-dimensional entity representation, the proposed architecture reveals the underlying dependencies between traffic movements and signal phases that drive control decisions\.

The primary contributions of this effort are as follows:

1. 1\.Topology\-Preserving Entity Embedding: The study proposes an architectural decomposition that isolates raw intersection metrics into individual lane entities \(Queries\) and phase status elements \(Keys/Values\), mapping them into a shared high\-dimensional latent space to inherently preserve the spatial configuration and spatial topology of the intersection\.
2. 2\.Hierarchical Relational Attention Mechanism: A dual\-stage attention architecture combining multi\-head cross\-attention and multi\-head self\-attention is introduced to model interactions between lanes and signal phases, as well as inter\-lane dependencies\. This structure provides fully explainable matrices that quantify the direct influence of each signal phase on individual approach lanes, illuminating the model’s internal decision\-making logic in real time\.
3. 3\.Constrained Action Masking Interface: To bridge the gap between stochastic policy exploration and deterministic field safety, the formulated architecture integrates an analytical action\-masking layer directly into the RL policy pipeline\. This interface mathematically blocks invalid phase transitions, guaranteeing absolute compliance with established signal timing constraints\.
4. 4\.Empirical Validation and Explainability Assessment: Utilizing the Simulation of Urban MObility \(SUMO\) environment, the proposed approach demonstrates improved performance in delay reduction compared to baseline methods, while providing interpretable attention visualizations that align with observed traffic phenomena such as queue formation, phase transitions, and spillback mitigation\.

## IIRELATED STUDIES

### II\-ARL for Traffic Signal Control

A substantial body of recent work has focused on developing RL\-based adaptive signal control systems\[[1](https://arxiv.org/html/2607.03703#bib.bib1),[2](https://arxiv.org/html/2607.03703#bib.bib2),[3](https://arxiv.org/html/2607.03703#bib.bib3),[4](https://arxiv.org/html/2607.03703#bib.bib4),[5](https://arxiv.org/html/2607.03703#bib.bib5),[6](https://arxiv.org/html/2607.03703#bib.bib6),[7](https://arxiv.org/html/2607.03703#bib.bib7),[8](https://arxiv.org/html/2607.03703#bib.bib8),[9](https://arxiv.org/html/2607.03703#bib.bib9),[10](https://arxiv.org/html/2607.03703#bib.bib10),[11](https://arxiv.org/html/2607.03703#bib.bib11),[12](https://arxiv.org/html/2607.03703#bib.bib12),[13](https://arxiv.org/html/2607.03703#bib.bib13),[14](https://arxiv.org/html/2607.03703#bib.bib14),[15](https://arxiv.org/html/2607.03703#bib.bib15),[16](https://arxiv.org/html/2607.03703#bib.bib16),[17](https://arxiv.org/html/2607.03703#bib.bib17),[18](https://arxiv.org/html/2607.03703#bib.bib18)\]\. The appeal of RL\-based signal control lies in the model free nature of state\-of\-the\-art RL algorithms\. Unlike the current field deployed adaptive signal control systems, model free RL\-based adaptive signal control systems do not rely on state predictive models, making them more computationally efficient for real\-time implementation\. In simulated environments, RL\-based has shown superior performance to the state\-of\-practice fixed time and actuated signal control systems\.

Early research efforts largely focused on isolated intersection control, formulating single RL agents trained in microscopic traffic simulation to optimize signal timings at individual intersections\[[13](https://arxiv.org/html/2607.03703#bib.bib13),[19](https://arxiv.org/html/2607.03703#bib.bib19),[20](https://arxiv.org/html/2607.03703#bib.bib20),[21](https://arxiv.org/html/2607.03703#bib.bib21),[22](https://arxiv.org/html/2607.03703#bib.bib22),[23](https://arxiv.org/html/2607.03703#bib.bib23),[24](https://arxiv.org/html/2607.03703#bib.bib24),[25](https://arxiv.org/html/2607.03703#bib.bib25),[26](https://arxiv.org/html/2607.03703#bib.bib26),[27](https://arxiv.org/html/2607.03703#bib.bib27)\]\. Several studies propose algorithms based on the deep Q\-network \(DQN\) framework and its extensions\[[8](https://arxiv.org/html/2607.03703#bib.bib8),[9](https://arxiv.org/html/2607.03703#bib.bib9),[13](https://arxiv.org/html/2607.03703#bib.bib13),[21](https://arxiv.org/html/2607.03703#bib.bib21),[24](https://arxiv.org/html/2607.03703#bib.bib24),[25](https://arxiv.org/html/2607.03703#bib.bib25),[27](https://arxiv.org/html/2607.03703#bib.bib27),[28](https://arxiv.org/html/2607.03703#bib.bib28),[29](https://arxiv.org/html/2607.03703#bib.bib29),[30](https://arxiv.org/html/2607.03703#bib.bib30)\]while others propose algorithms based on policy\-based methods including actor\-critic and its variations such as advantage actor\-critic \(A2C\), deep deterministic policy gradient \(DDPG\), and proximal policy optimization \(PPO\)\[[19](https://arxiv.org/html/2607.03703#bib.bib19),[22](https://arxiv.org/html/2607.03703#bib.bib22),[31](https://arxiv.org/html/2607.03703#bib.bib31),[32](https://arxiv.org/html/2607.03703#bib.bib32)\]\.

More recent efforts have focused on extending single agent RL\-based signal control to multi agent reinforcement learning \(MARL\)\-based signal control with a single agent controlling each intersection and cooperating with the adjacent intersection agents to generate a coordinated signal timing plan\[[1](https://arxiv.org/html/2607.03703#bib.bib1),[2](https://arxiv.org/html/2607.03703#bib.bib2),[5](https://arxiv.org/html/2607.03703#bib.bib5),[6](https://arxiv.org/html/2607.03703#bib.bib6),[7](https://arxiv.org/html/2607.03703#bib.bib7),[10](https://arxiv.org/html/2607.03703#bib.bib10),[11](https://arxiv.org/html/2607.03703#bib.bib11),[14](https://arxiv.org/html/2607.03703#bib.bib14),[15](https://arxiv.org/html/2607.03703#bib.bib15),[16](https://arxiv.org/html/2607.03703#bib.bib16),[33](https://arxiv.org/html/2607.03703#bib.bib33),[34](https://arxiv.org/html/2607.03703#bib.bib34),[35](https://arxiv.org/html/2607.03703#bib.bib35),[36](https://arxiv.org/html/2607.03703#bib.bib36)\]\. Several efforts adopt the centralized training and decentralized execution \(CTDE\) paradigm of MARL which allows implicit communication between the agents during training\. Some of these efforts enhance the implicit communication inherent in CTDE with explicit communication modules to exchange observation and action histories across agents\[[6](https://arxiv.org/html/2607.03703#bib.bib6),[11](https://arxiv.org/html/2607.03703#bib.bib11),[18](https://arxiv.org/html/2607.03703#bib.bib18)\]\. Some studies especially for network level signal control have formulated graph based RL\[[1](https://arxiv.org/html/2607.03703#bib.bib1),[4](https://arxiv.org/html/2607.03703#bib.bib4),[7](https://arxiv.org/html/2607.03703#bib.bib7),[10](https://arxiv.org/html/2607.03703#bib.bib10)\]\. Graph based RL allows information sharing between neighboring agents through graph neural networks\.

### II\-BExplainable RL for Traffic Signal Control

Recent work has begun to address the interpretability limitations of RL\-based traffic signal control\. Luo, et al\.\[[37](https://arxiv.org/html/2607.03703#bib.bib37)\]proposed an interpretable influence mechanism based on efficient hinging hyperplanes neural networks, which leverages ANOVA decomposition to quantify the contribution of individual traffic features and their interactions\. This approach enables explicit estimation of feature importance and provides an interpretation of spatiotemporal dependencies among intersections\. However, while the method offers strong analytical interpretability, it lacks the intuitive and visual explainability and do not explicitly capture structured relationships between traffic movements and signal phases\.

Hu, et al\.\[[38](https://arxiv.org/html/2607.03703#bib.bib38)\]proposed an explainable RL framework that integrates an attention\-based deep Q\-network to model vehicle\-level interactions and prioritize critical vehicles in decision\-making\. The study directly encodes individual vehicle attributes, enabling more fine\-grained control and improved transparency through attention visualization and counterfactual analysis\. Experimental results demonstrated significant improvements in travel time and queue reduction compared to conventional and RL\-based baselines, while also providing interpretable insights into the learned control policies\. Despite these advances, the reliance on vehicle\-level representations introduces challenges in scalability and compatibility with commonly available traffic sensing infrastructure, which is typically aggregated at the lane or movement level\. Furthermore, the use of value\-based methods limits flexibility in handling complex action spaces and temporal dependencies\.

### II\-CAttention Mechanisms for Explainability

Originally introduced by Bahdanau*et al\.*\[[39](https://arxiv.org/html/2607.03703#bib.bib39)\]to overcome the information bottleneck in machine translation, the attention mechanism was designed to dynamically map dependencies between input and output sequences\. Subsequent architectural integrations specifically combining scaled dot\-product attention with residual connections\[[40](https://arxiv.org/html/2607.03703#bib.bib40)\], positional encodings, and layer normalization\[[41](https://arxiv.org/html/2607.03703#bib.bib41)\]culminated in the Transformer architecture developed by Vaswani*et al\.*\[[42](https://arxiv.org/html/2607.03703#bib.bib42)\]\. Beyond its foundational dominance in natural language processing \(NLP\), the Transformer framework has become ubiquitous across diverse machine learning domains, including computer vision\[[43](https://arxiv.org/html/2607.03703#bib.bib43)\], structural biology\[[44](https://arxiv.org/html/2607.03703#bib.bib44)\], and molecular design\[[45](https://arxiv.org/html/2607.03703#bib.bib45)\]\.

In addition to driving state\-of\-the\-art predictive performance, attention weights are increasingly leveraged as an intrinsic layer of model interpretability\. Because these weights provide a quantifiable distribution of a model’s focus across spatial or temporal inputs, they offer a window into the otherwise opaque decision\-making processes of deep architectures\. This explainability aspect has been utilized across several fields, including medical diagnostic coding using reverse\-time dual attention mechanisms\[[46](https://arxiv.org/html/2607.03703#bib.bib46),[47](https://arxiv.org/html/2607.03703#bib.bib47)\], and image captioning, where spatial attention heatmaps are superimposed to visualize the exact pixel regions driving textual outputs\[[48](https://arxiv.org/html/2607.03703#bib.bib48)\]\.

Crucially, this interpretive capacity has extended into explainable RL\. In safety\-critical systems, attention\-augmented agents can map internal policies directly back to observable environmental states\[[49](https://arxiv.org/html/2607.03703#bib.bib49)\]\.

## IIIProblem Formulation

This section presents the formulation of the proposed attention\-based RL framework for traffic signal control\. The problem is formulated as a semi\-Markov decision process \(SMDP\) to account for the event\-driven nature of signal control, where actions are executed over variable time intervals\. To enable structured decision\-making and interpretability, the traffic environment is represented using an entity\-based state formulation that explicitly captures lane\-level traffic conditions and signal phase information\. These entities are processed through attention mechanisms to model interactions between traffic demand and control actions\.

A high\-level overview of the proposed formulation and architecture is illustrated in Fig\.[1](https://arxiv.org/html/2607.03703#S3.F1)\. The key components of the formulation, including state representation, attention mechanism, action space, reward definition, and policy structure, are described in the following subsections\.

![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_1.png)Figure 1:Overview of the proposed attention\-based RL architecture for traffic signal control\.### III\-AState Space Definition

To effectively capture the real\-time intersection dynamics and structural topologies, the state space is modeled using an entity\-centric representation rather than a flat, monolithic vector\. At each decision time\-steptt, the global state vectorst∈ℝds\_\{t\}\\in\\mathbb\{R\}^\{d\}, aggregates traffic statestl​a​n​es\_\{t\}^\{lane\}and signal statestp​h​a​s​es\_\{t\}^\{phase\}:

st=\[stlane∥stphase\]s\_\{t\}=\\left\[s\_\{t\}^\{\\text\{lane\}\}\\parallel s\_\{t\}^\{\\text\{phase\}\}\\right\]\(1\)where∥\\paralleldenotes the concatenation operator andd=Nl\+Npd=\\ N\_\{l\}\+N\_\{p\}represents the total dimensionality composed ofNlN\_\{l\}lane entities andNpN\_\{p\}phase entities\.

#### III\-A1State Input Space Decomposition

The state vector is cleanly partitioned into two distinct functional categories:

- •Traffic/Lane Elements \(Queries\):The vectorstlane=\[vt1,vt2,…,vtNl\]∈ℝNls\_\{t\}^\{\\text\{lane\}\}=\\left\[v\_\{t\}^\{1\},v\_\{t\}^\{2\},\\dots,v\_\{t\}^\{N\_\{l\}\}\\right\]\\in\\mathbb\{R\}^\{N\_\{l\}\}captures the intersection traffic demand\. Each elementvtiv\_\{t\}^\{i\}represents the vehicle count for lane entityii\. Within the relational learning mechanism, these elements serve as Queries \(QQ\), prompting the policy to evaluate per\-lane demand distribution against signal phasing and green time allocation\.
- •Signal State Elements \(Keys/Values\):The vectorstphase=\[gt1,gt2,…,gtNp\]∈ℝNps\_\{t\}^\{\\text\{phase\}\}=\\left\[g\_\{t\}^\{1\},g\_\{t\}^\{2\},\\dots,g\_\{t\}^\{N\_\{p\}\}\\right\]\\in\\mathbb\{R\}^\{N\_\{p\}\}represents the temporal state of the traffic signal controller\. Each elementgtig\_\{t\}^\{i\}tracks the green duration of phaseii\. These elements serve as Keys \(KK\) and Values \(VV\), providing the contextual grid that bounds the traffic/vehicle metrics\.

#### III\-A2Disaggregated High\-Dimensional Entity Embedding

Because the global state vectorsts\_\{t\}aggregates raw features across disparate traffic components \(per lane vehicle counts vs\. phase durations\), feeding a flat vector directly into standard neural layers risks obfuscating the spatial and functional relationships unique to each lane and phase\. To preserve structural topology,sts\_\{t\}is decomposed into separate, distinct entity representations and map them into a shared high\-dimensional spaceℝde\\mathbb\{R\}^\{d\_\{e\}\}, whereded\_\{e\}is the embedding dimension\.

LetXl,i∈ℝ1X\_\{l,i\}\\in\\mathbb\{R\}^\{1\}represent the raw feature scalar \(vehicle countvti\)v\_\{t\}^\{i\}\)of lane i and letXp,j∈ℝ1X\_\{p,j\}\\in\\mathbb\{R\}^\{1\}represent the raw feature scalar \(green durationgtjg\_\{t\}^\{j\}\) of phase j\. Using parameterized, entity\-specific mapping layers \(implemented via specialized dense layers with a Rectified Linear Unit activation\), each individual entity is projected into the latent space:

𝐡ilane=ReLU​\(𝐖ilane​Xl,i\+bilane\)∈ℝde\\mathbf\{h\}\_\{i\}^\{\\text\{lane\}\}=\\text\{ReLU\}\\left\(\\mathbf\{W\}\_\{i\}^\{\\text\{lane\}\}X\_\{l,i\}\+b\_\{i\}^\{\\text\{lane\}\}\\right\)\\in\\mathbb\{R\}^\{d\_\{e\}\}\(2\)𝐡jphase=ReLU​\(𝐖jphase​Xp,j\+bjphase\)∈ℝde\\mathbf\{h\}\_\{j\}^\{\\text\{phase\}\}=\\text\{ReLU\}\\left\(\\mathbf\{W\}\_\{j\}^\{\\text\{phase\}\}X\_\{p,j\}\+b\_\{j\}^\{\\text\{phase\}\}\\right\)\\in\\mathbb\{R\}^\{d\_\{e\}\}\(3\)where𝐖ilane\\mathbf\{W\}\_\{i\}^\{\\text\{lane\}\},𝐖jphase\\mathbf\{W\}\_\{j\}^\{\\text\{phase\}\}represent the trainable projection matrices, andbilaneb\_\{i\}^\{\\text\{lane\}\},bjphaseb\_\{j\}^\{\\text\{phase\}\}denote their respective bias vectors\.

Following individual projections, the high\-dimensional representations are stacked along a new entity dimension to produce the final structured matrices passed to the attention heads:

Hlane\\displaystyle H^\{\\text\{lane\}\}=\[—h1lane—⋮—hNllane—\]∈ℝNl×de\\displaystyle=\\begin\{bmatrix\}\\text\{\-\-\-\}&h\_\{1\}^\{\\text\{lane\}\}&\\text\{\-\-\-\}\\\\ &\\vdots&\\\\ \\text\{\-\-\-\}&h\_\{N\_\{l\}\}^\{\\text\{lane\}\}&\\text\{\-\-\-\}\\end\{bmatrix\}\\in\\mathbb\{R\}^\{N\_\{l\}\\times d\_\{e\}\}\(4\)Hphase\\displaystyle H^\{\\text\{phase\}\}=\[—h1phase—⋮—hNpphase—\]∈ℝNp×de\\displaystyle=\\begin\{bmatrix\}\\text\{\-\-\-\}&h\_\{1\}^\{\\text\{phase\}\}&\\text\{\-\-\-\}\\\\ &\\vdots&\\\\ \\text\{\-\-\-\}&h\_\{N\_\{p\}\}^\{\\text\{phase\}\}&\\text\{\-\-\-\}\\end\{bmatrix\}\\in\\mathbb\{R\}^\{N\_\{p\}\\times d\_\{e\}\}\(5\)By decoupling the entities in this manner,Hl​a​n​eH^\{lane\}operates strictly as an array of structured Queries, whileHp​h​a​s​eH^\{phase\}operates as Keys and Values\. This guarantees that the subsequent cross\-attention layer computes an explicit affinity matrix capturing the exact relational weight between every single approach lane and every traffic phase configuration\.

### III\-BRelational Representational Learning Via Hierarchical Attention

To extract structural dependencies between lanes and phases without assuming a fixed geometric intersection layout, the high\-dimensional entity matricesHl​a​n​eH^\{lane\}andHp​h​a​s​eH^\{phase\}are passed through a hierarchical attention network\. This pipeline explicitly models directed lane\-to\-phase dependencies followed by localized lane\-to\-lane interactions\.

#### III\-B1Multi\-Head Cross\-Attention \(Lanes \- Phase\)

The first stage establishes contextual relationships between the traffic demand on individual lanes and the operating configurations of the traffic phases\. The lane embedding matrixHl​a​n​eH^\{lane\}is projected to form the Queries \(Q\) while the phase embedding matrixHp​h​a​s​eH^\{phase\}is projected to form the Keys \(K\) and Values \(V\):

Q=Hlane​WQ,K=Hphase​WK,V=Hphase​WVQ=H^\{\\text\{lane\}\}W\_\{Q\},\\quad K=H^\{\\text\{phase\}\}W\_\{K\},\\quad V=H^\{\\text\{phase\}\}W\_\{V\}\(6\)whereWQW\_\{Q\},WKW\_\{K\},WV∈ℝde​X​deW\_\{V\}\\ \\in\\ \\mathbb\{R\}^\{\{d\_\{e\}Xd\}\_\{e\}\}are trainable projection matrices\. The cross\-attention output matrixZ1Z\_\{1\}is computed using a scaled dot\-product formulation:

Z1=Attention​\(Q,K,V\)=softmax​\(Q​KTde\)​VZ\_\{1\}=\\text\{Attention\}\(Q,K,V\)=\\text\{softmax\}\\left\(\\frac\{QK^\{T\}\}\{\\sqrt\{d\_\{e\}\}\}\\right\)V\(7\)
The intermediate attention weight matrix produced by this block will be used to generate attention maps for explain ability\. The cross attentionAc​r​o​s​sA\_\{cross\}acts as a verifiable record of agent’s reasoning\. It mathematically represents the exact relational importance assigned to every individual lane relative to each available green signal combination\.

Across=softmax​\(Q​KTde\)∈ℝNl×NpA\_\{\\text\{cross\}\}=\\text\{softmax\}\\left\(\\frac\{QK^\{T\}\}\{\\sqrt\{d\_\{e\}\}\}\\right\)\\in\\mathbb\{R\}^\{N\_\{l\}\\times N\_\{p\}\}\(8\)
To stabilize training gradients and mitigate information loss, a residual connection followed by layer normalization is applied to generate the first intermediate latent stateH1H\_\{1\}:

H1=LayerNorm​\(Hlane\+Z1\)∈ℝNl×deH\_\{1\}=\\text\{LayerNorm\}\\left\(H^\{\\text\{lane\}\}\+Z\_\{1\}\\right\)\\in\\mathbb\{R\}^\{N\_\{l\}\\times d\_\{e\}\}\(9\)

#### III\-B2Multi\-Head Self\-Attention \(Lane \- Lane interactions\)

While cross\-attention captures lane\-to\-phase dynamics, lanes approaching an intersection also experience inter\-lane dependencies \(e\.g\., merging conflicts or queue spillbacks\)\. To capture these localized conflicts,H1H\_\{1\}is passed through a multi\-head self\-attention layer where queries, keys, and values are derived entirely from the updated lane representations:

Z2=softmax​\(H1​H1Tde\)​H1Z\_\{2\}=\\text\{softmax\}\\left\(\\frac\{H\_\{1\}H\_\{1\}^\{T\}\}\{\\sqrt\{d\_\{e\}\}\}\\right\)H\_\{1\}\(10\)A second residual connection and layer normalization step are executed to form the interactive lane representation matrixH2H\_\{2\}

H2=LayerNorm​\(H1\+Z2\)∈ℝNl×deH\_\{2\}=\\text\{LayerNorm\}\\left\(H\_\{1\}\+Z\_\{2\}\\right\)\\in\\mathbb\{R\}^\{N\_\{l\}\\times d\_\{e\}\}\(11\)

#### III\-B3Global Representation and Layer Normalization

Prior to generating the final policy distribution, the latent matrix undergoes an additional layer normalization step to ensure numerical stability against extreme variations in traffic patterns:

H^=LayerNorm​\(H2\)∈ℝNl×de\\widehat\{H\}=\\text\{LayerNorm\}\\left\(H\_\{2\}\\right\)\\in\\mathbb\{R\}^\{N\_\{l\}\\times d\_\{e\}\}\(12\)
Finally, a vectorization operator\(⊙\)\(\\odot\)flattens the structured entity matrix across its lane and latent dimensions\. This generates a compact, global feature vectorZtZ\_\{t\}capturing the comprehensive structural and traffic state of the intersection:

Zt=vec​\(H^\)∈ℝNl⋅deZ\_\{t\}=\\text\{vec\}\\left\(\\widehat\{H\}\\right\)\\in\\mathbb\{R\}^\{N\_\{l\}\\cdot d\_\{e\}\}\(13\)
This unified representationZtZ\_\{t\}encapsulates both traffic demand distributions and signal timing context, serving as the direct input for the downstream action\-masking and stochastic policy layers\.

### III\-CSafety Constrained Action Space

In this formulation, action is defined based on the NEMA dual ring\-barrier diagram shown in Fig\.[2](https://arxiv.org/html/2607.03703#S3.F2)A discrete set of eight actions \(0\-7\) is defined with each action corresponding to a pair of compatible phases, one from each ring\. Actions 0 to 7 correspond to phase combinations \(1,5\), \(1,6\), \(2,5\), \(2,6\), \(3,7\), \(3,8\), \(4,7\), \(4,8\)\.

![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_2.png)Figure 2:Ring\-barrier diagramBecause of the constraints of minimum green, yellow and red clearance intervals and double barrier crossing requirements, not all the eight actions are available for selection at every decision point\. At every decision step t, the algorithm evaluates the valid action setAtv​a​l​i​d⊂AA\_\{t\}^\{valid\}\\subset Aas a function of active phase, committed phase and elapsed phase duration\.

The raw policy network processes the geometric feature vectorZtZ\_\{t\}to output policy logitsfθ​\(Zt\)∈ℝNactionsf\_\{\\theta\}\(Z\_\{t\}\)\\in\\mathbb\{R\}^\{N\_\{\\text\{actions\}\}\}\. The outputs are filtered by a deterministic safety maskmt∈ℝ\|A\|m\_\{t\}\\in\\mathbb\{R\}^\{\|A\|\}\. This mask encodes structural traffic rules including minimum green, yellow, and red clearances, and NEMA dual\-ring barrier constraints at runtime:

mt​\(a\)=\{0,if​a∈Atvalid−∞,if​a∉Atvalidm\_\{t\}\(a\)=\\begin\{cases\}0,&\\text\{if \}a\\in A\_\{t\}^\{\\text\{valid\}\}\\\\ \-\\infty,&\\text\{if \}a\\notin A\_\{t\}^\{\\text\{valid\}\}\\end\{cases\}\(14\)To explicitly eliminate the probability of selecting an invalid and unsafe phase change, the mask vectormtm\_\{t\}is added directly to the raw network logits\. This penalizes invalid choices prior to normal distribution scaling:

π~​\(a∣st\)=softmax​\(fθ​\(Zt\)\+mt\)=exp⁡\(lt​\(a\)\+mt​\(a\)\)∑j∈Aexp⁡\(lt​\(aj\)\+mt​\(j\)\)\\begin\{split\}\\tilde\{\\pi\}\(a\\mid s\_\{t\}\)&=\\text\{softmax\}\\left\(f\_\{\\theta\}\(Z\_\{t\}\)\+m\_\{t\}\\right\)\\\\ &=\\frac\{\\exp\\left\(l\_\{t\}\(a\)\+m\_\{t\}\(a\)\\right\)\}\{\\sum\_\{j\\in A\}\\exp\\left\(l\_\{t\}\(a\_\{j\}\)\+m\_\{t\}\(j\)\\right\)\}\\end\{split\}\(15\)By setting the logits of invalid movements to−∞\-\\infty, their resulting exponentiated values evaluate cleanly to zero, ensuring that the filtered distributionπ~\(a\|st\)\\widetilde\{\\pi\}\\left\(a\\middle\|s\_\{t\}\\right\)assigns zero selection probability to any phase that violates ring\-barrier safety constraints\. The policy probabilityπθ​\(at\|st\)\\pi\_\{\\theta\}\(a\_\{t\}\|s\_\{t\}\)maps directly to the valid action space:

πθ​\(at∣st\)=π~​\(at∣st\)\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)=\\tilde\{\\pi\}\(a\_\{t\}\\mid s\_\{t\}\)\(16\)The actual discrete actionata\_\{t\}selected for implementation is sampled directly from this distribution:

at∼πθ\(⋅∣st\)a\_\{t\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid s\_\{t\}\)\(17\)This action dictates the next phase configuration or phase extension step to apply over the subsequent control interval, combining high\-dimensional representation learning with strict operational guardrails\.

### III\-DReward Definition

To optimize the efficiency of the intersection, the objective of the RL is to minimize cumulative vehicular delay\. The reward functionrtr\_\{t\}at any decision step t is formulated as the negative sum of normalized delays experienced by all active vehicles across the monitored approach lanes:

rt=−∑l∈Nl∑v∈Vl​\(t\)dv​\(t\)Dr\_\{t\}=\-\\sum\_\{l\\in N\_\{l\}\}\\sum\_\{v\\in V\_\{l\}\(t\)\}\\frac\{d\_\{v\}\(t\)\}\{D\}\(18\)wheredvd\_\{v\}is delay for individual vehicle v for all vehiclesVlV\_\{l\}on lane l at a time t\. D is the normalization scaling factor\. The individual vehicle delaydv​\(t\)d\_\{v\}\(t\)is calculated analytically as the difference between the total time spent by vehiclevvin the network and its theoretical free\-flow travel time

### III\-EProximal Policy Optimization Formulation

The relational feature vectorZtZ\_\{t\}and filtered distributionπθ\(at\|st\)\\pi\_\{\\theta\}\\left\(a\_\{t\}\\middle\|s\_\{t\}\\right\)are optimized using Proximal Policy Optimization \(PPO\), a policy gradient variant formulated by\[[39](https://arxiv.org/html/2607.03703#bib.bib39)\]that stabilizes training by constraining policy updates via an analytical clipping mechanism\. The agent is structured as an Actor\-Critic architecture, updating a parameterized policy networkπθ\\pi\_\{\\theta\}and a state\-value functionVϑV\_\{\\vartheta\}\.

### III\-FGeneralized Advantage Estimation \(GAE\)

To minimize variance during trajectory evaluation, the generalized advantage estimatorA~t\\tilde\{A\}\_\{t\}is computed via temporal difference errorsδtV\\delta\_\{t\}^\{V\}across a discounted reverse\-accumulation buffer:

δtV=rt\+γ​Vϑ​\(st\+1\)​\(1−dt\)−Vϑ​\(st\)\\delta\_\{t\}^\{V\}=r\_\{t\}\+\\gamma V\_\{\\vartheta\}\(s\_\{t\+1\}\)\(1\-d\_\{t\}\)\-V\_\{\\vartheta\}\(s\_\{t\}\)\(19\)
A~t=∑k=0T−t−1\(γ​λ\)k​δt\+kV,A~tnorm=A~t−μA~σA~\+ϵ\\tilde\{A\}\_\{t\}=\\sum\_\{k=0\}^\{T\-t\-1\}\(\\gamma\\lambda\)^\{k\}\\delta\_\{t\+k\}^\{V\},\\quad\\tilde\{A\}\_\{t\}^\{\\text\{norm\}\}=\\frac\{\\tilde\{A\}\_\{t\}\-\\mu\_\{\\tilde\{A\}\}\}\{\\sigma\_\{\\tilde\{A\}\}\+\\epsilon\}\(20\)wherertr\_\{t\}is the reward,γ\\gammais the discount factor,λ\\lambdais the GAE trace decay hyperparameter, anddtd\_\{t\}is the terminal indicator\.

### III\-GClipped Surrogate Policy Objective

To prevent extreme policy updates that disrupt the delicate attention mappings, PPO enforces a probability ratio constraint\. Letρt​\(θ\)\\rho\_\{t\}\(\\theta\)define the relative probability tracking parameter between the updated policyπθ\\pi\_\{\\theta\}and the old policyπθold\\pi\_\{\\theta\_\{\\text\{old\}\}\}used to collect transitions:

ρt​\(θ\)=πθ​\(at∣st\)πθold​\(at∣st\)\\rho\_\{t\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\}\{\\pi\_\{\\theta\_\{\\text\{old\}\}\}\(a\_\{t\}\\mid s\_\{t\}\)\}\(21\)
The surrogate policy objective function minimizes a conservative lower bound, clipping the ratio if it moves outside an acceptable trust region governed by hyperparameterϵclip\\epsilon\_\{\\text\{clip\}\}:

ℒCLIP\(θ\)=𝔼^t\[min\(ρt​\(θ\)​A^tnorm,clip\(ρt\(θ\),1−ϵclip,1\+ϵclip\)A^tnorm\)\]\\begin\{split\}\\mathcal\{L\}^\{\\text\{CLIP\}\}\(\\theta\)=\\hat\{\\mathbb\{E\}\}\_\{t\}\\Big\[\\min\\Big\(&\\rho\_\{t\}\(\\theta\)\\hat\{A\}\_\{t\}^\{\\text\{norm\}\},\\\\ &\\text\{clip\}\\left\(\\rho\_\{t\}\(\\theta\),1\-\\epsilon\_\{\\text\{clip\}\},1\+\\epsilon\_\{\\text\{clip\}\}\\right\)\\hat\{A\}\_\{t\}^\{\\text\{norm\}\}\\Big\)\\Big\]\\end\{split\}\(22\)
To maintain policy diversity and prevent premature convergence to a singular signal sequence, an entropy regularization termℋ\(πθ\(⋅∣st\)\)\\mathcal\{H\}\(\\pi\_\{\\theta\}\(\\cdot\\mid s\_\{t\}\)\)is introduced with an allocation coefficientc2c\_\{2\}:

ℒactor\(θ\)=ℒCLIP\(θ\)\+c2𝔼^t\[ℋ\(πθ\(⋅∣st\)\)\]\\mathcal\{L\}^\{\\text\{actor\}\}\(\\theta\)=\\mathcal\{L\}^\{\\text\{CLIP\}\}\(\\theta\)\+c\_\{2\}\\hat\{\\mathbb\{E\}\}\_\{t\}\\left\[\\mathcal\{H\}\\left\(\\pi\_\{\\theta\}\(\\cdot\\mid s\_\{t\}\)\\right\)\\right\]\(23\)

## IVModel training

### IV\-ANetwork Model

The model is trained for a single intersection network illustrated in Fig\.[3](https://arxiv.org/html/2607.03703#S4.F3)\. The network consists of a four\-leg signalized intersection with a major street \(east–west\) and a minor street \(north–south\)\. The main street is configured with two through lanes in each direction and a short exclusive left\-turn pocket, while the side street consists of one through lane in each direction with a short left\-turn pocket\.

![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_3.png)Figure 3:Schematic representation of the simulated intersection geometryThe model is trained on three distinct demand patterns \(A, B, C\) as shown in Table[I](https://arxiv.org/html/2607.03703#S4.T1)\. These volume patterns are selected to be significantly different from one another, as quantified using the structural similarity index formulated by Wang, et al\.\[[51](https://arxiv.org/html/2607.03703#bib.bib51)\]\. Exposing the model to diverse traffic demand conditions is expected to improve its robustness and ability to generalize to unseen volumes\.

TABLE I:O\-D Patterns Used to Train the Model
### IV\-BModel Interaction with Simulation Environment and Updating Network parameters

#### IV\-B1Distributed Simulation Environments

Simulation of urban MObility \(SUMO\) is selected as the simulation environment\[[52](https://arxiv.org/html/2607.03703#bib.bib52)\]\. The model is trained in distributed simulation environments\. As detailed in Algorithm[1](https://arxiv.org/html/2607.03703#alg1), training occurs over E episodes across N independent SUMO simulation environments running in parallel\. At the beginning of an episode, each environmentiiinitializes its local simulation instance and constructs an empty trajectory storage bufferBiB\_\{i\}\. Over the simulation horizonTT, step\-by\-step state tracking and control execution are offloaded to the event\-driven routine described in Algorithm[2](https://arxiv.org/html/2607.03703#alg2)\.

Once the trajectories are collected, the sample buffers are combined into an aggregated datasetD=⋃i=1NBiD=\\ \\bigcup\_\{i=1\}^\{N\}B\_\{i\}\. GAE is then performed to calculate the target returnsRtR\_\{t\}and standardized advantagesA~t\{\\widetilde\{A\}\}\_\{t\}ensuring low\-variance gradient trajectories during optimization updates\.

Algorithm 1Multi\-Environment PPO Training1:Number of environments

NN, episodes

EE, horizon

TT
2:Initialize policy

πθ\\pi\_\{\\theta\}and value network

VϕV\_\{\\phi\}
3:for

e=1,…,Ee=1,\\ldots,Edo

4:// Parallel Rollout Collection

5:for allenvironments

i=1,…,Ni=1,\\ldots,N\(in parallel\)do

6:Reset environment and initialize buffer

ℬi\\mathcal\{B\}\_\{i\}
7:Observe initial state

s0s\_\{0\}
8:for

t=1,…,Tt=1,\\ldots,Tdo

9:Advance simulation by one step

10:Execute Algorithm[2](https://arxiv.org/html/2607.03703#alg2)

11:Store

\(st,at,rt,Vϕ​\(st\),log⁡πθ​\(at\|st\)\)\(s\_\{t\},a\_\{t\},r\_\{t\},V\_\{\\phi\}\(s\_\{t\}\),\\log\\pi\_\{\\theta\}\(a\_\{t\}\|s\_\{t\}\)\)in

ℬi\\mathcal\{B\}\_\{i\}
12:endfor

13:Store terminal transition

14:endfor

15:// Batch Aggregation

16:

𝒟←⋃i=1Nℬi\\mathcal\{D\}\\leftarrow\\bigcup\_\{i=1\}^\{N\}\\mathcal\{B\}\_\{i\}
17:// Advantage Estimation

18:Compute returns

RtR\_\{t\}
19:Compute generalized advantages

A^t\\hat\{A\}\_\{t\}
20:Normalize

A^t\\hat\{A\}\_\{t\}
21:// Policy Update

22:for

k=1,…,Kπk=1,\\ldots,K\_\{\\pi\}do

23:Compute probability ratio

rt​\(θ\)=πθ​\(at\|st\)πθold​\(at\|st\)r\_\{t\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(a\_\{t\}\|s\_\{t\}\)\}\{\\pi\_\{\\theta\_\{\\rm old\}\}\(a\_\{t\}\|s\_\{t\}\)\}
24:Maximize clipped PPO objective with entropy bonus

25:Update policy parameters

θ\\theta
26:endfor

27:// Value Function Update

28:for

k=1,…,KVk=1,\\ldots,K\_\{V\}do

29:Minimize

LV=𝔼​\[\(Vϕ​\(st\)−Rt\)2\]L\_\{V\}=\\mathbb\{E\}\\\!\\left\[\(V\_\{\\phi\}\(s\_\{t\}\)\-R\_\{t\}\)^\{2\}\\right\]
30:Update value parameters

ϕ\\phi
31:endfor

32:// Checkpoint

33:Save policy

πθ\\pi\_\{\\theta\}and value network

VϕV\_\{\\phi\}
34:endfor

35:Trained policy

πθ\\pi\_\{\\theta\}and value network

VϕV\_\{\\phi\}

Algorithm 2Event\-Driven RL\-Based Signal Control1:Current state

sts\_\{t\}, policy

πθ\\pi\_\{\\theta\}
2:Initialize decision time

tdecisiont\_\{\\mathrm\{decision\}\}and update time

tupdatet\_\{\\mathrm\{update\}\}
3:whilesimulation is runningdo

4:Advance simulation by one simulation step

5:Update traffic state and vehicle tracking

6:// Decision Event

7:if

t=tdecisiont=t\_\{\\mathrm\{decision\}\}then

8:Determine valid action set

𝒜t\\mathcal\{A\}\_\{t\}
9:Sample action

at∼πθ​\(a\|st,𝒜t\)a\_\{t\}\\sim\\pi\_\{\\theta\}\(a\|s\_\{t\},\\mathcal\{A\}\_\{t\}\)
10:Estimate state value

Vϕ​\(st\)V\_\{\\phi\}\(s\_\{t\}\)
11:Compute action duration

Δ​t=f​\(at,signal state\)\\Delta t=f\(a\_\{t\},\\text\{signal state\}\)
12:

tupdate←t\+Δ​tt\_\{\\mathrm\{update\}\}\\leftarrow t\+\\Delta t
13:endif

14:// Action Execution

15:Apply signal control corresponding to

ata\_\{t\}
16:Maintain the selected phase during

\[t,tupdate\)\[t,t\_\{\\mathrm\{update\}\}\)
17:Enforce minimum green, yellow, and all\-red intervals

18:// Transition Event

19:if

t=tupdatet=t\_\{\\mathrm\{update\}\}then

20:Observe next state

st\+1s\_\{t\+1\}
21:Compute reward

rtr\_\{t\}
22:Store transition

23:

\(st,at,rt,Vϕ​\(st\),log⁡πθ​\(at\|st\),st\+1\)\(s\_\{t\},a\_\{t\},r\_\{t\},V\_\{\\phi\}\(s\_\{t\}\),\\log\\pi\_\{\\theta\}\(a\_\{t\}\|s\_\{t\}\),s\_\{t\+1\}\)
24:

tdecision←t\+ϵt\_\{\\mathrm\{decision\}\}\\leftarrow t\+\\epsilon
25:

st←st\+1s\_\{t\}\\leftarrow s\_\{t\+1\}
26:endif

27:endwhile

#### IV\-B2Event\-Driven RL Control Loop

The real\-time interaction between the agent and the simulation is governed by the SMDP in Algorithm[2](https://arxiv.org/html/2607.03703#alg2)\. Decisions are triggered dynamically att=tdecisiont=t\_\{\\text\{decision\}\}\. The environment uses the action masking layer to isolate a valid action setAtA\_\{t\}based on the current signal state\. A safe actionata\_\{t\}is sampled from the masked distributionπθ​\(at∣st,At\)\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\},A\_\{t\}\), and its continuous operational durationΔ​t=f​\(at,signal state\)\\Delta t=f\(a\_\{t\},\\text\{signal state\}\)is computed analytically\.

The selected action is implemented in the environment over the variable window\(t,tupdate=t\+Δ​t\)\(t,t\_\{\\text\{update\}\}=t\+\\Delta t\)enforcing required intermediate intervals \(such as yellow clearance and all\-red safety flags\)\. Attupdatet\_\{\\text\{update\}\}, a transition event updates the target parameters, records the trajectory tuple\(st,at,rt,Vϑ​\(st\),log⁡πθ​\(at∣st\)\)\(s\_\{t\},a\_\{t\},r\_\{t\},V\_\{\\vartheta\}\(s\_\{t\}\),\\log\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\)into bufferBiB\_\{i\}, and increments the next look\-ahead decision time step\.

#### IV\-B3Policy and Value Network Optimization

Following data aggregation at the end of each episode, the policy parametersθ\\thetaare updated overKπK\_\{\\pi\}epochs\. At each epoch, the clipped surrogate lossℒCLIP\\mathcal\{L\}^\{\\text\{CLIP\}\}balances policy stability against historic target behavior while an entropy regularization term preserves exploration diversity\. Concurrently, the critic value parametersϑ\\varthetaare adjusted overKvK\_\{v\}epochs to minimize baseline prediction error\. After structural optimization is complete, a network parameter checkpoint is saved, updating the global models across the worker environments\.

#### IV\-B4Hyper parameter selection

Preliminary training was performed to select architectural and RL hyper parameters\. The final selected hyper parameters are shown in Table 2\.

TABLE II:Training and Architectural HyperparametersParameter CategoryHyperparameter DescriptionSymbolValueEntity EmbeddingHigh\-dimensional embedding dimensionsded\_\{e\}32LayerEmbedding layer activationσ​\(⋅\)\\sigma\(\\cdot\)ReLUAttention BlocksNumber of cross\-attention headsHcH\_\{c\}4Number of self\-attention headsHsH\_\{s\}4Policy/CriticFully connected hidden layers–256, 128NetworksNetwork activation function–tanhPPO CoreDiscount factorγ\\gamma0\.99ParametersTrace decay parameter \(GAE\)λ\\lambda0\.95PPO clipping ratioϵclip\\epsilon\_\{\\mathrm\{clip\}\}0\.2Entropy coefficientc2c\_\{2\}0\.01Target KL divergenceKLtarget\\mathrm\{KL\}\_\{\\mathrm\{target\}\}0\.01OptimizationActor learning rateαθ\\alpha\_\{\\theta\}1×10−41\\times 10^\{\-4\}StepsCritic learning rateαϕ\\alpha\_\{\\phi\}1×10−41\\times 10^\{\-4\}Actor/Critic optimization epochsKπ,KVK\_\{\\pi\},K\_\{V\}80Gradient clipping thresholdgclipg\_\{\\mathrm\{clip\}\}0\.5SimulationParallel environmentsNN3SetupEpisode horizonTT3600 s

## VResults and Discussion

### V\-AModel Training

Fig\.[4](https://arxiv.org/html/2607.03703#S5.F4)illustrates the training performance of the proposed PPO agent, showing the evolution of the average episodic reward aggregated across all three distributed environments\. The reward is defined as the negative normalized delay, computed from vehicle travel times relative to estimated free\-flow conditions\. Each episode lasts for 3600 seconds at the end of which the actor and critic networks are updated with experiences collected from all the environments\. The learning curve demonstrates a clear and stable learning progression of the model\. Initially, the model exhibits high variance and poor performance due to random exploration, reflected in large negative rewards\. However, as training progresses, the agent consistently improves its policy, leading to a steady increase in average reward\. The variability in rewards diminishes over time, reflecting increased policy stability and more consistent decision\-making across environments\. Full convergence is reached after around 2200 episodes\. The smooth convergence of the learning curve toward near\-zero reward further suggests that the agent reaches a stable and near\-optimal policy, effectively minimizing traffic delay while maintaining robust performance throughout training\.

![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_4.png)Figure 4:Model Learning Curve
### V\-BComparative Performance Analysis: Impact of Attention Mechanism

To evaluate operational efficiency and policy robustness, the proposed entity\-centric RL model with hierarchical attention \(RL\-Att\) was benchmarked against two baseline systems: an optimized Actuated Signal Control \(ASC\) plan configured via Synchro and a standard RL model utilizing a flat vector input state space without attention \(RL\-NoAtt\)\. RL\-NoAtt was formulated and trained on the same volume patterns as RL\-Att in an earlier study\[[53](https://arxiv.org/html/2607.03703#bib.bib53)\]\. Performance was rigorously analyzed over N=10 independent microscopic simulation runs across two distinct traffic demand scenarios featuring structural volume variations completely unseen during model training\.

#### V\-B1Scenario 1: Balanced Baseline Demand

The first validation is performed on a volume set withVE​B​TV\_\{EBT\}=800,VE​B​LV\_\{EBL\}=250,VW​B​TV\_\{WBT\}=666,VW​B​LV\_\{WBL\}=150,VN​B​TV\_\{NBT\}=500,VN​B​LV\_\{NBL\}=180,VS​B​TV\_\{SBT\}=400 andVS​B​LV\_\{SBL\}=250 whereViV\_\{i\}is the volume in veh/h for movement i\. As illustrated in Fig\.[5](https://arxiv.org/html/2607.03703#S5.F5), both RL\-based frameworks demonstrate substantial, statistically significant reductions in movement\-level vehicular delay relative to the optimized ASC baseline across virtually all approaches, with the sole exception of SBL movement\.

This systemic improvement is attributed to the contrast between the rule\-based threshold logic of conventional ASC and the adaptive, state\-dependent policies learned by the RL agents\. The RL models optimize phase selections and green time distributions holistically by evaluating the concurrent traffic state profiles of all conflicting movements\. Notably, the attention\-infused framework \(RL\-Att\) delivers comparable or superior delay reduction relative to RL\-NoAtt, particularly for WBT, NBL and NBT\. This indicates that the inclusion of an explanatory mechanism does not induce a performance penalty\.

![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_5.jpg)Figure 5:Comparing the performance of RL with and without Attention against optimized ASC
#### V\-B2Scenario 2: Highly Asymmetric Demand

To further evaluate the structural robustness of the learned policies, a second scenario was introduced, specifically inflating the northbound left\-turn demand \(NBL increased from 180 to 432 veh/h\)\. Southbound through \(SBT\) is increased to 400 veh/h while SBL is set to 30 veh/h\. The empirical results for this highly asymmetrical O\-D pattern are presented in Fig\.[6](https://arxiv.org/html/2607.03703#S5.F6)\. Under this demand pattern, RL\-Att consistently outperforms RL\-NoAtt across all critical movements, showing minor degradation only on SBT movement\. Most notably, for the highly congested NBL movement, RL\-NoAtt exhibits an accumulation of delay, failing to efficiently adapt to the severe local demand pressure\. Conversely, RL\-Att dynamically restructures its internal phase sequencing and green\-time allocation\. By utilizing entity\-centric projections, RL\-Att isolates specific lane demands and maps their relational dependencies to phase elements\. The results suggest that this mathematical structure allows the agent to implicitly capture complex spatial\-temporal interactions between geometric lane bottlenecks and active signal phases\. Consequently, the attention\-weighted architecture appears to anticipate local congestion spillback and proactively extends green intervals, demonstrating superior generalization, robustness, and stability in highly volatile deployment environments\.

![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_6.jpg)Figure 6:Comparing the performance of RL with and without Attention against optimized ASC on a highly imbalanced volume

### V\-CInterpretability Analysis and Attention Field Dynamics

To validate the transparent decision\-making logic of the agent, the internal latent variations of the hierarchical attention blocks were captured and mapped directly against real\-time traffic demand states\.

![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_7a.png)\(\(a\)\)
\\phantomcaption![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_7b.png)\(\(a\)\)Figure 8:Lane–phase cross\-attention weights and corresponding active vehicle counts\. \(a\) Weights for intervalt=2212t=2212–22582258\. \(b\) Weights for intervalt=2271t=2271–23102310\.#### V\-C1Lane–Phase Cross Attention Dynamics

Fig\.[8](https://arxiv.org/html/2607.03703#S5.F8)illustrates the temporal evolution of lane–phase attention alongside vehicle counts in each lane over a representative simulation interval \(t=2221t=2221–23102310s\)\. The test volumes are the same as in Section[V\-B1](https://arxiv.org/html/2607.03703#S5.SS2.SSS1)above\. Active phases are denoted by green arrows and explicit phase markers in square brackets\. Because the control framework operates as an event\-driven SMDP, decision intervals are non\-uniform and occur dynamically based on current signal states and selected actions\. Consequently, attention weights are logged exclusively at active policy execution points; intermediate phase preservation loops \(such as the sustained execution of \{P4, P8\} betweent=2241t=2241andt=2258t=2258s\) are omitted for analytical clarity\.

An empirical trace of the sequence reveals a highly intuitive alignment with domain\-specific traffic control logic:

- •At the beginning of the sequence \(t=2221t=2221s\), phases \{P2, P6\} are active, serving the east–west through movements\. As a queue builds up on the northbound through lane \(NB\_TH\), its query projection assigns a progressively elevated attention weight to the active phases, demanding service\.
- •Att=2227t=2227s, the model initiates a phase transition, from \{P2, P6\} first passing through \{P3, P7\} and subsequently activating phases \{P4, P8\}, which serve the north–south movements\. The attention associated withNB\_THremains elevated during this period, consistent with the presence of sustained queuing, and gradually decreases as the queues dissipate\.
- •As theNB\_THqueue dissipates, demand rebuilds along the east–west approaches, particularly for the through and left\-turn movements \(EB\_TH,WB\_TH,EB\_LT, andWB\_LT\)\. The cross\-attention matrix tracks this shift, raising the weights for these lanes on the active phases\.
- •This pattern repeats over the remainder of the interval, where increases in lane\-level demand are followed by elevated attention weights and subsequent activation of the associated phases\. Across the sequence, phase transitions consistently occur when attention becomes concentrated on alternative phase groups, indicating a strong temporal alignment between demand buildup, attention allocation, and control actions\.

![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_8a.png)\(\(a\)\)
\\phantomcaption![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_8b.png)\(\(a\)\)Figure 10:Lane–lane self\-attention weights and corresponding active vehicle counts under balanced traffic demand\. \(a\) Weights fort=603t=603–646646\. \(b\) Weights fort=659t=659–707707\.This tight temporal alignment between spatial queue buildup, attention weight concentration, and phase triggering demonstrates that the cross\-attention mechanism yields a valid, human\-interpretable explanation of policy inference\.

#### V\-C2Lane–Lane Self\-Attention Under Balanced Traffic Demand

Fig\.[10](https://arxiv.org/html/2607.03703#S5.F10)presents the lane–lane attention weights over a representative simulation interval\. In this formulation, each lane attends to other lanes, allowing the model to capture spatial interactions and dependencies across the intersection\. The test demand scenario consists of balanced traffic conditions\. Similar to the lane–phase analysis in Section[V\-C1](https://arxiv.org/html/2607.03703#S5.SS3.SSS1)above, only representative decision points are shown, with non\-critical intermediate steps omitted for brevity\.

![Refer to caption](https://arxiv.org/html/2607.03703v1/Fig_9b.png)Figure 11:Lane–lane self\-attention weights under asymmetric traffic demand\.A temporal examination of Fig\.[10](https://arxiv.org/html/2607.03703#S5.F10)reveals how lane–lane attention evolves in response to changing demand patterns and phase transitions\. At the start of the interval \(t=603t=603s\), phases \{P2, P6\} are active, serving mainEBandWBthrough movements\. Queues are building up on side street lanes, andNB\_THandSB\_LTlanes receive the highest attention from the opposing movement lanes\. The self\-attention mechanism establishesNB\_THandSB\_LTas central informational anchors across the entire layer, forcing currently served movements to continuously evaluate their remaining green time against the mounting vehicle pressure of these specific conflicting flows\. This look\-ahead capability directly registers the urgent need for a service switch, guiding the policy’s capacity to handle phase transitions smoothly\.

Consequently, att=611t=611s, a phase transition occurs to phases \{P3, P7\}, and subsequently to phases \{P4, P8\} att=626t=626s\. While phases \{P4, P8\} are active, queues start to build up again on the main street, and correspondingly, attention starts to shift back to the main street lanes tracking the emerging demand pressure\. A similar pattern is observed throughout the remainder of the sequence, where attention intensifies on lanes experiencing increased demand and subsequently redistributes following phase changes\.

Throughout the interval, attention becomes more concentrated on specific lanes during periods of high demand and more diffuse when traffic conditions are balanced, closely tracking the temporal evolution of traffic states and control actions\. The attention patterns reveal a clear structure in which lanes with higher vehicle counts exert stronger influence on other lanes, particularly those served by a conflicting phase\. Notably, the attention weights concentrate on a subset of critical lanes, suggesting that the model selectively focuses on the most influential traffic streams at each decision point\. These patterns evolve over time in response to changing demand, demonstrating that the model learns dynamic, context\-dependent relationships between lanes rather than static correlations\.

#### V\-C3Lane–Lane Attention Self\-Attention under asymmetric traffic Demand

To further examine the internal representations learned by the model, a highly asymmetrical scenario is introduced by inflating the northbound approach volumes \(VNBT=500V\_\{\\text\{NBT\}\}=500,VNBL=432V\_\{\\text\{NBL\}\}=432\) while limiting the opposing southbound left\-turn volume \(VSBL=30V\_\{\\text\{SBL\}\}=30\)\. The tracking interval betweent=960t=960s andt=1032t=1032s is shown in Fig\.[11](https://arxiv.org/html/2607.03703#S5.F11)\.

Despite this imbalance in volumes, the trained model demonstrates strong robustness by effectively allocating green time to the northbound movements and preventing excessive queue buildup\. For theNBapproach, phases \{P3, P8\} are frequently paired during periods of sustained demand \(onlyt=1026t=1026s tot=1032t=1032s shown here\)\. This behavior is particularly important given the limited storage capacity of the northbound left\-turn lane\. By consistently serving phase P3 \(NBleft\-turn\) in coordination with P8 \(NBthrough\), the controller limits queue growth and prevents spillback that could block the turn bay or interfere with through traffic\.

This coordinated/joint phase selection is reflected in the lane–lane attention patterns, where theNBleft\-turn lane maintains elevated attention not only within its own movement but also in relation to theNBthrough lane\. This suggests that the model has implicitly learned to account for geometric constraints and potential spillback effects, using coordinated phase activation to mitigate the risk of lane blockage\. More broadly, the attention patterns exhibit a sustained focus on the northbound lanes throughout the interval\. Unlike the more dynamic attention shifts observed under balanced conditions, the attention weights remain consistently concentrated on these movements, reflecting persistent demand pressure\. This behavior indicates that the model adapts its internal representation to prioritize dominant traffic streams while maintaining stable and efficient control without oscillatory phase switching\.

## VIConclusions

This effort formulates an explainable, entity\-centric RL framework designed to overcome the critical trust and transparency barriers that have that have the potential to restrict the real\-world deployment of deep RL in traffic signal control\. By abandoning flat state vectors in favor of an architecture that explicitly treats intersection components as independent high\-dimensional lane and phase entities, the proposed model successfully opens the ”black box” of deep RL neural networks\.

The hierarchical attention mechanism, combining multi\-head cross\-attention and self\-attention delivers a real\-time explainability output affinity matrix\. This enables operators to visually and analytically audit how the network weights competing lane demands against phase configurations\. Furthermore, the integration of a deterministic action\-masking interface successfully bridges the gap between stochastic policy exploration and critical field safety mandates, ensuring absolute compliance with NEMA ring\-barrier, minimum green\-time and clearance interval constraints\.

Empirical evaluations conducted within the SUMO microscopic traffic simulation environment validate that this injection of transparency does not come at the expense of operational performance\. The framework matches or outperforms state\-of\-the\-practice baselines in delay minimization while inherently mitigating structural traffic anomalies such queue spillbacks, turn lane blockages and asymmetric congestion bottlenecks\. More importantly, the corresponding attention maps demonstrate intuitive alignment with established traffic engineering principles\. By providing human\-interpretable diagnostics, this architecture lays the necessary foundation for agency acceptance, operational trust, and collaborative fine\-tuning by domain experts, moving adaptive traffic signal control a significant step closer to widespread deployment\.

## References

- \[1\]J\. Shen, ”Hierarchical reinforcement learning\-based traffic signal control,”*Scientific Reports*, vol\. 15, no\. 1, 2025, doi: 10\.1038/s41598\-025\-18449\-1\.
- \[2\]K\. Othman, X\. Wang, A\. Shalaby, and B\. Abdulhai, ”Integrating Multimodality and Partial Observability Solutions Into Decentralized Multiagent Reinforcement Learning Adaptive Traffic Signal Control,”*IEEE Open Journal of Intelligent Transportation Systems*, vol\. 6, pp\. 322–334, 2025, doi: 10\.1109/OJITS\.2025\.3550312\.
- \[3\]X\. Li, X\. Wang, I\. Smirnov, S\. Sanner, and B\. Abdulhai, ”Multi\-Hop Upstream Anticipatory Traffic Signal Control With Deep Reinforcement Learning,”*IEEE Open Journal of Intelligent Transportation Systems*, vol\. 6, pp\. 554–567, 2025, doi: 10\.1109/OJITS\.2025\.3562757\.
- \[4\]D\. Xu, Z\. Yu, X\. Liao, and H\. Guo, ”A Graph Deep Reinforcement Learning Traffic Signal Control for Multiple Intersections Considering Missing Data,”*IEEE Transactions on Vehicular Technology*, vol\. 73, no\. 12, pp\. 18307–18319, 2024, doi: 10\.1109/TVT\.2024\.3444475\.
- \[5\]X\. Wang, A\. Taitler, I\. Smirnov, S\. Sanner, and B\. Abdulhai, ”eMARLIN: Distributed Coordinated Adaptive Traffic Signal Control with Topology\-Embedding Propagation,”*Transportation Research Record*, vol\. 2678, no\. 4, pp\. 189–202, 2024, doi: 10\.1177/03611981231184250\.
- \[6\]S\. Shuvo*et al\.*, ”Deep Multi\-Agent Reinforcement Learning for Real\-World Signalized Traffic Corridor Control,” in*Proceedings of the International Conference on Machine Learning and Applications \(ICMLA\)*, 2024, pp\. 644–651, doi: 10\.1109/ICMLA61862\.2024\.00093\.
- \[7\]T\. Shi, F\. X\. Devailly, D\. Larocque, and L\. Charlin, ”Improving the Generalizability and Robustness of Large\-Scale Traffic Signal Control,”*IEEE Open Journal of Intelligent Transportation Systems*, vol\. 5, pp\. 2–15, 2024, doi: 10\.1109/OJITS\.2023\.3331689\.
- \[8\]D\. Kwesiga, S\. Vishnoi, A\. Guin, and M\. Hunter, ”Integrating Transit Signal Priority into Multi\-Agent Reinforcement Learning based Traffic Signal Control,” arXiv preprint arXiv:2411\.19359, 2024\.
- \[9\]D\. Kwesiga, A\. Guin, and M\. Hunter, ”Adaptive Transit Signal Priority Based on Deep Reinforcement Learning and Connected Vehicles in a Traffic Micro Simulation Environment,” in*Proceedings of the Winter Simulation Conference \(WSC\)*, 2024, pp\. 1422–1433, doi: 10\.1109/WSC63780\.2024\.10838807\.
- \[10\]X\. Jia, M\. Guo, Y\. Lyu, J\. Qu, D\. Li, and F\. Guo, ”Adaptive Traffic Signal Control Based on Graph Neural Networks and Dynamic Entropy\-Constrained Soft Actor–Critic,”*Electronics*, vol\. 13, no\. 23, 2024, doi: 10\.3390/electronics13234794\.
- \[11\]A\. Chang, Y\. Ji, C\. Wang, and Y\. Bie, ”CVDMARL: A Communication\-Enhanced Value Decomposition Multi\-Agent Reinforcement Learning Traffic Signal Control Method,”*Sustainability*, vol\. 16, no\. 5, 2024, doi: 10\.3390/su16052160\.
- \[12\]M\. Long and E\. Chung, ”Transit Signal Priority for Arterial Road with Deep Reinforcement Learning,” in*Proceedings of the International Conference on Models and Technologies for Intelligent Transportation Systems \(MT\-ITS\)*, 2023, doi: 10\.1109/MT\-ITS56129\.2023\.10241759\.
- \[13\]X\.\-Y\. Liu, M\. Zhu, S\. Borst, and A\. Walid, ”Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems,” arXiv preprint arXiv:2302\.03669, 2023\.
- \[14\]D\. Liu and L\. Li, ”A traffic light control method based on multi\-agent deep reinforcement learning algorithm,”*Scientific Reports*, vol\. 13, no\. 1, 2023, doi: 10\.1038/s41598\-023\-36606\-2\.
- \[15\]M\. Kolat, B\. Kővári, T\. Bécsi, and S\. Aradi, ”Multi\-Agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach,”*Sustainability*, vol\. 15, no\. 4, 2023, doi: 10\.3390/su15043479\.
- \[16\]X\. Fu, S\. Chen, Q\. Liang, and Y\. Li, ”Research on Multi\-Agent Reinforcement Learning Traffic Control,” in*Proceedings of the IEEE International Conference on Control, Electronics and Computer Technology \(ICCECT\)*, 2023, pp\. 231–239, doi: 10\.1109/ICCECT57938\.2023\.10140678\.
- \[17\]S\. Bouktif, A\. Cheniki, A\. Ouni, and H\. El\-Sayed, ”Deep reinforcement learning for traffic signal control with consistent state and reward design approach,”*Knowledge\-Based Systems*, vol\. 267, 2023, doi: 10\.1016/j\.knosys\.2023\.110440\.
- \[18\]R\. Bokade, X\. Jin, and C\. Amato, ”Multi\-Agent Reinforcement Learning Based on Representational Communication for Large\-Scale Traffic Signal Control,”*IEEE Access*, vol\. 11, pp\. 47646–47658, 2023, doi: 10\.1109/ACCESS\.2023\.3275883\.
- \[19\]M\. Aslani, M\. S\. Mesgari, S\. Seipel, and M\. Wiering, ”Developing adaptive traffic signal control by actor\-critic and direct exploration methods,”*Proceedings of the Institution of Civil Engineers: Transport*, vol\. 172, no\. 5, pp\. 289–298, 2019, doi: 10\.1680/jtran\.17\.00085\.
- \[20\]K\. Bálint, T\. Tamás, and B\. Tamás, ”Deep Reinforcement Learning based approach for Traffic Signal Control,”*Transportation Research Procedia*, vol\. 62, pp\. 278–285, 2022, doi: 10\.1016/j\.trpro\.2022\.02\.035\.
- \[21\]S\. Bouktif, A\. Cheniki, and A\. Ouni, ”Traffic signal control using hybrid action space deep reinforcement learning,”*Sensors*, vol\. 21, no\. 7, 2021, doi: 10\.3390/s21072302\.
- \[22\]N\. Casas, ”Deep Deterministic Policy Gradient for Urban Traffic Light Control,” arXiv preprint arXiv:1703\.09035, 2017\.
- \[23\]H\. Lee, Y\. Han, Y\. Kim, and Y\. H\. Kim, ”Effects analysis of reward functions on reinforcement learning for traffic signal control,”*PLoS ONE*, vol\. 17, no\. 11, 2022, doi: 10\.1371/journal\.pone\.0277813\.
- \[24\]D\. Li, J\. Wu, M\. Xu, Z\. Wang, and K\. Hu, ”Adaptive Traffic Signal Control Model on Intersections Based on Deep Reinforcement Learning,”*Journal of Advanced Transportation*, vol\. 2020, 2020, doi: 10\.1155/2020/6505893\.
- \[25\]L\. Li, Y\. Lv, and F\.\-Y\. Wang, ”Traffic Signal Timing via Deep Reinforcement Learning,”*IEEE Transactions on Intelligent Transportation Systems*, vol\. 3, 2016\.
- \[26\]Y\. Li, J\. He, and Y\. Gao, ”Intelligent Traffic Signal Control with Deep Reinforcement Learning at Single Intersection,” in*Proceedings of the ACM International Conference Proceeding Series*, 2021, pp\. 399–406, doi: 10\.1145/3467707\.3467767\.
- \[27\]S\. Liu, G\. Wu, and M\. Barth, ”A Complete State Transition\-Based Traffic Signal Control Using Deep Reinforcement Learning,” in*Proceedings of the IEEE Conference on Technologies for Sustainability \(SusTech\)*, 2022, pp\. 100–107, doi: 10\.1109/SusTech53338\.2022\.9794168\.
- \[28\]P\. Chanloha*et al\.*, ”Traffic Signal Control with Cell Transmission Model Using Reinforcement Learning for Total Delay Minimisation,”*International Journal of Computers, Communications & Control*, vol\. 10, no\. 5, pp\. 627–642, 2015, doi: 10\.15837/ijccc\.2015\.5\.2025\.
- \[29\]S\. El\-Tantawy, B\. Abdulhai, and H\. Abdelgawad, ”Design of reinforcement learning parameters for seamless application of adaptive traffic signal control,”*Journal of Intelligent Transportation Systems*, vol\. 18, no\. 3, pp\. 227–245, 2014, doi: 10\.1080/15472450\.2013\.810991\.
- \[30\]X\. Liang, X\. Du, G\. Wang, and Z\. Han, ”A Deep Reinforcement Learning Network for Traffic Light Cycle Control,”*IEEE Transactions on Vehicular Technology*, vol\. 68, no\. 2, pp\. 1243–1253, 2019, doi: 10\.1109/TVT\.2018\.2890726\.
- \[31\]H\. Pang and W\. Gao, ”Deep Deterministic Policy Gradient for Traffic Signal Control of Single Intersection,” in*Proceedings of the Chinese Control and Decision Conference \(CCDC\)*, 2019, pp\. 5861–5866, doi: 10\.1109/CCDC\.2019\.8832406\.
- \[32\]S\. M\. A\. Shabestary, B\. Abdulhai, H\. Ma, and Y\. Huo, ”Cycle\-level vs\. Second\-by\-Second Adaptive Traffic Signal Control using Deep Reinforcement Learning,” in*Proceedings of the IEEE 23rd International Conference on Intelligent Transportation Systems \(ITSC\)*, 2020, pp\. 1–8, doi: 10\.1109/ITSC45102\.2020\.9294171\.
- \[33\]X\. Chen, G\. Xiong, Y\. Lv, Y\. Chen, B\. Song, and F\. Y\. Wang, ”A Collaborative Communication\-Qmix Approach for Large\-scale Networked Traffic Signal Control,” in*Proceedings of the IEEE International Conference on Intelligent Transportation Systems \(ITSC\)*, 2021, pp\. 3450–3455, doi: 10\.1109/ITSC48978\.2021\.9564683\.
- \[34\]T\. Chu, J\. Wang, L\. Codecà, and Z\. Li, ”Multi\-Agent Deep Reinforcement Learning for Large\-scale Traffic Signal Control,” arXiv preprint arXiv:1903\.04527, 2019\.
- \[35\]F\. X\. Devailly, D\. Larocque, and L\. Charlin, ”Model\-Based Graph Reinforcement Learning for Inductive Traffic Signal Control,”*IEEE Open Journal of Intelligent Transportation Systems*, vol\. 5, pp\. 238–250, 2024, doi: 10\.1109/OJITS\.2024\.3376583\.
- \[36\]H\. Wei*et al\.*, ”Colight: Learning network\-level cooperation for traffic signal control,” in*Proceedings of the ACM International Conference on Information and Knowledge Management \(CIKM\)*, 2019, pp\. 1913–1922, doi: 10\.1145/3357384\.3357902\.
- \[37\]Z\. Luo, J\. Xu, and F\. Chen, ”Multi\-agent Reinforcement Traffic Signal Control based on Interpretable Influence Mechanism and Biased ReLU Approximation,” arXiv preprint arXiv:2403\.13639, 2024\.
- \[38\]Y\. Hu, L\. Du, and S\. M\. Easa, ”Explainable reinforcement learning for improved traffic signal control,”*Computer\-Aided Civil and Infrastructure Engineering*, vol\. 40, no\. 24, pp\. 3911–3933, 2025, doi: 10\.1111/mice\.70037\.
- \[39\]D\. Bahdanau, K\. Cho, and Y\. Bengio, ”Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409\.0473, 2014\.
- \[40\]K\. He, X\. Zhang, S\. Ren, and J\. Sun, ”Deep Residual Learning for Image Recognition,” in*Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\)*, 2016, pp\. 770–778, doi: 10\.1109/CVPR\.2016\.90\.
- \[41\]J\. L\. Ba, J\. R\. Kiros, and G\. E\. Hinton, ”Layer normalization,” arXiv preprint arXiv:1607\.06450, 2016\.
- \[42\]A\. Vaswani*et al\.*, ”Attention is all you need,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2017, pp\. 5998–6008\.
- \[43\]A\. Dosovitskiy*et al\.*, ”An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010\.11929, 2020\.
- \[44\]J\. Jumper*et al\.*, ”Highly accurate protein structure prediction with AlphaFold,”*Nature*, vol\. 596, no\. 7873, pp\. 583–589, 2021, doi: 10\.1038/s41586\-021\-03819\-2\.
- \[45\]M\. McGibbon*et al\.*, ”From intuition to AI: evolution of small molecule representations in drug discovery,”*Briefings in Bioinformatics*, vol\. 25, no\. 1, p\. bbad422, 2024, doi: 10\.1093/bib/bbad422\.
- \[46\]E\. Choi, M\. T\. Bahadori, J\. Sun, J\. Kulas, A\. Schuetz, and W\. Stewart, ”Retain: An interpretable predictive model for healthcare using reverse time attention mechanism,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2016, pp\. 3504–3512\.
- \[47\]J\. Mullenbach, S\. Wiegreffe, J\. Duke, J\. Sun, and J\. Eisenstein, ”Explainable prediction of medical codes from clinical text,” in*Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(NAACL\-HLT\)*, 2018, pp\. 1101–1111, doi: 10\.18653/v1/N18\-1100\.
- \[48\]K\. Xu*et al\.*, ”Show, attend and tell: Neural image caption generation with visual attention,” in*Proceedings of the International Conference on Machine Learning \(ICML\)*, 2015, pp\. 2048–2057\.
- \[49\]A\. Mott, D\. Zoran, M\. Chrzanowski, D\. Wierstra, and D\. J\. Rezende, ”Towards interpretable reinforcement learning using attention augmented agents,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2019, pp\. 12360–12369\.
- \[50\]J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov, ”Proximal policy optimization algorithms,” arXiv preprint arXiv:1707\.06347, 2017\.
- \[51\]Z\. Wang, A\. Bovik, H\. Sheikh, and E\. Simoncelli, ”Image Quality Assessment: From Error Visibility to Structural Similarity,” in*IEEE Transactions on Image Processing*, vol\. 13, no\. 4, 2004, pp\. 600–612, doi: 10\.1109/TIP\.2003\.819861\.
- \[52\]P\. A\. Lopez*et al\.*, ”Microscopic Traffic Simulation using SUMO,” in*Proceedings of the IEEE International Conference on Intelligent Transportation Systems \(ITSC\)*, 2018, pp\. 2575–2582, doi: 10\.1109/ITSC\.2018\.8569938\.
- \[53\]D\. Kwesiga, A\. Guin, K\. Abdelghany, and M\. Hunter, ”Evaluating the Robustness of Reinforcement Learning based Adaptive Traffic Signal Control,”*arXiv preprint arXiv:2603\.15283*, 2026\.

## Biography Section

Dickens Kwesigais a Research Engineer with the School of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta, GA, USA\. His current research focuses on developing deployment\-oriented artificial intelligence \(AI\) systems for Intelligent Transportation Systems \(ITS\), validated through software\-in\-the\-loop \(SIL\) and hardware\-in\-the\-loop \(HIL\) simulation testing to ensure real\-world readiness\.

Nishu Choudharyis a Research Engineer with the School of Civil and Environmental Engineering at the Georgia Institute of Technology\. Her research interests include leveraging machine learning \(ML\) techniques to develop data\-driven solutions for proactive traffic management and intelligent transportation systems\.

Angshuman Guinis a Senior Research Engineer with the School of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta, GA, USA\. His research interests include freeway operations, connected and autonomous vehicles, Intelligent Transportation Systems \(ITS\), transportation safety, traffic simulation, and transportation data management\.

Michael Hunteris a Professor with the School of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta, GA, USA\. His teaching and research interests include transportation operations and design, with emphasis on emerging transportation technologies, adaptive traffic signal control, traffic simulation, and arterial corridor operations\.