SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets

# SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets
Source: [https://arxiv.org/html/2607.08681](https://arxiv.org/html/2607.08681)
,Yifan XuDuke Kunshan UniversityKunshanJiangsuChinaandLuyao ZhangDuke Kunshan UniversityKunshanJiangsuChina

###### Abstract\.

As agentic AI systems are increasingly applied to cyber\-physical environments, their evaluation requires assessment of both task performance and trustworthiness\. In decentralized energy markets, autonomous agents may improve market utility, but may also exploit invalid physical data, create artificial liquidity, and produce unstable governance decisions\. Therefore, we proposeSolarChain\-Eval, a physics\-constrained benchmark for evaluating trustworthy economic agents\. It formulates market governance as a Gymnasium\-compatible Markov Decision Process, where agents make hourly decisions\.SolarChain\-Evalevaluates each policy across multiple dimensions, including market utility, physical safety, slippage, action smoothness, spatial fairness, and auditability\.

To support agentic evaluation,SolarChain\-Evalincorporates an LLM\-based Planner/Auditor layer\. The Planner defines episode\-level action bounds and audit rules, while the Auditor reviews and revises high\-risk actions\. All interventions are recorded through structured logs, including trigger signals, proposed actions, revised actions, and audit rationales\. Experiments with static, random, myopic, RL, and RL\+LLM policies reveal a clear utility\-safety trade\-off\. RL agents improve market utility but can still produce unsafe behavior\. When the physics penalty is removed, reward\-maximizing agents exploit invalid generation and increase artificial liquidity\. The LLM Planner/Auditor improves auditability and mitigates selected risks, but it cannot fully compensate for a misspecified reward function\. These results indicate that trustworthy agentic AI evaluation requires both physical constraints and transparent intervention traces\. We release data and code as open access on GitHub for replicability\.

Agentic AI, reinforcement learning, trustworthiness evaluation, tokenomics, decentralized energy markets

![Refer to caption](https://arxiv.org/html/2607.08681v1/x1.png)Figure 1\.OverallSolarChain\-Evalbenchmark pipeline\. The figure summarizes the end\-to\-end workflow from data loading, RL training, policy evaluation, output generation, and trustworthiness evidence for decentralized energy\-market governance\.## 1\.Introduction

Agentic AI is increasingly used to support autonomous decision\-making in cyber\-physical and economic systems\(Xiet al\.,[2025](https://arxiv.org/html/2607.08681#bib.bib51)\)\. Such agents not only optimize digital objectives, their actions may also affect physical resources, market incentives, and systems safety\(Vyas and Mercangöz,[2025](https://arxiv.org/html/2607.08681#bib.bib52)\)\. This shift raises an evaluation challenge that agent performance cannot be judged by scalar reward alone\. Trustworthy agents should therefore make decisions that are physically grounded, constraint\-aware, and auditable under real\-world operating conditions\(National Institute of Standards and Technology,[2023](https://arxiv.org/html/2607.08681#bib.bib53); Drgonaet al\.,[2025](https://arxiv.org/html/2607.08681#bib.bib54); Phiri,[2025](https://arxiv.org/html/2607.08681#bib.bib55)\)\.

Table 1\.Comparison of Prevailing Bi\-Level Architectures Integrating RL and LLM AgentsIn decentralized energy markets, reinforcement learning \(RL\) has been used to manage market liquidity, token issuance, and incentive distribution\(Tusharet al\.,[2018](https://arxiv.org/html/2607.08681#bib.bib47); Ouet al\.,[2026](https://arxiv.org/html/2607.08681#bib.bib56)\)\. However, unlike purely digital environments, energy markets are bounded by physical laws, such as localized solar irradiance and photovoltaic panel capacities\(Zhanget al\.,[2018](https://arxiv.org/html/2607.08681#bib.bib45); Mengelkampet al\.,[2018](https://arxiv.org/html/2607.08681#bib.bib46)\)\. Deploying reward\-maximizing agents in such environments therefore creates important trustworthiness risks\. Without explicit physical constraints, RL models can learn to exploit weaknesses in the market mechanism\(Achiamet al\.,[2017](https://arxiv.org/html/2607.08681#bib.bib49)\)\.

To address this, we proposeSolarChain\-Eval, a physics\-constrained benchmark for evaluating trustworthy agentic AI in decentralizedenergy markets\.SolarChain\-Evalcombines solar signals with controlled market and risk scenarios, and formulates market governance as a Gymnasium\-compatible Markov Decision Process\. To further examine agentic oversight,SolarChain\-Evalincorporates an evaluation\-time LLM\-based Planner/Auditor layer, which is inserted between the trained RL policy and the benchmark environment without being used during RL training\. The Planner defines episode\-level action bounds and audit rules, while the Auditor reviews and revises high\-risk actions\. This design enablesSolarChain\-Evalto assess both the operational performance of an RL policy, and the extent to which its decision\-making behavior remains governable and interpretable\.

Specifically, we concentrate on the following research questions:

- •RQ1:How do autonomous market policies trade off economic utility, physical safety, stability, and fairness compared with static and heuristic baselines?
- •RQ2:To what extent do reward\-maximizing agents exploit invalid generation and create artificial liquidity when physics penalties are removed?
- •RQ3:How does an LLM\-based Planner/Auditor contribute to auditable risk mitigation for trained RL policies, and what are its limitations under reward misspecification?

## 2\.Literature Review

### 2\.1\.Reinforcement Learning in Tokenomics

Traditional economic mechanisms struggle when applied to decentralized energy markets due to the restrictive assumptions of stationary environments\(Zhenget al\.,[2022](https://arxiv.org/html/2607.08681#bib.bib44)\)\. Distributed Energy Resources \(DERs\) are governed by intermittent generation constraints and Peer\-to\-Peer \(P2P\) network topologies\(Chenet al\.,[2022](https://arxiv.org/html/2607.08681#bib.bib7)\)\. However, tokenomic mechanisms are subject to extreme volatility driven by liquidity depth, speculative token price trajectories, and adoption rates\(Malinova and Park,[2023](https://arxiv.org/html/2607.08681#bib.bib9); Conget al\.,[2021](https://arxiv.org/html/2607.08681#bib.bib10)\)\. RL provides a flexible modeling approach by formulating market governance as a Markov Decision Process\(Zhenget al\.,[2022](https://arxiv.org/html/2607.08681#bib.bib44)\)\. It can adapt to high\-dimensional state spaces, optimizing long\-term objectives that balance physical utility with economic stability\(Zhenget al\.,[2022](https://arxiv.org/html/2607.08681#bib.bib44)\)\.

In current decentralized market governance literature\(Xuet al\.,[2025](https://arxiv.org/html/2607.08681#bib.bib43); Quet al\.,[2025](https://arxiv.org/html/2607.08681#bib.bib42)\), macroeconomic control is typically abstracted into continuous governance levers, such as the reward ratio \(rtr\_\{t\}\), liquidity ratio \(ltl\_\{t\}\), and token burn rate \(btb\_\{t\}\)\. In simulated environments, the normalized action vectorata\_\{t\}is decoded into specific economic parameters\. To evaluate system trustworthiness, researchers widely rely on standardized frameworks likeStable\-Baselines3\(Raffinet al\.,[2021](https://arxiv.org/html/2607.08681#bib.bib11)\)to benchmark distinct RL algorithmic paradigms:

Proximal Policy Optimization \(PPO\) is an on\-policy actor\-critic algorithm widely applied to continuous action space governance tasks\. To prevent excessively large policy updates, PPO utilizes a clipped surrogate objective\(Schulmanet al\.,[2017](https://arxiv.org/html/2607.08681#bib.bib12)\):

\(1\)LC​L​I​P​\(θ\)=𝔼^t​\[min⁡\(pt​\(θ\)​A^t,clip​\(pt​\(θ\),1−ϵ,1\+ϵ\)​A^t\)\]L^\{CLIP\}\(\\theta\)=\\hat\{\\mathbb\{E\}\}\_\{t\}\\left\[\\min\(p\_\{t\}\(\\theta\)\\hat\{A\}\_\{t\},\\text\{clip\}\(p\_\{t\}\(\\theta\),1\-\\epsilon,1\+\\epsilon\)\\hat\{A\}\_\{t\}\)\\right\]
whereθ\\thetadenotes the policy network parameters,pt​\(θ\)p\_\{t\}\(\\theta\)is the probability ratio,A^t\\hat\{A\}\_\{t\}represents the estimated advantage function, andϵ\\epsilonis the clipping hyperparameter\.

In tokenomic governance, this high training stability translates to practical advantages by effectively suppressing action jitter and ensuring smoother adjustments of macroeconomic parameters\.

Also supporting continuous control, Soft Actor\-Critic \(SAC\) is an off\-policy algorithm\. By incorporating a maximum entropy mechanism into its objective function\(Haarnojaet al\.,[2018](https://arxiv.org/html/2607.08681#bib.bib13)\), SAC proactively encourages the agent to explore diverse tokenomic strategies:

\(2\)J\(π\)=∑t=0T𝔼\(st,at\)∼ρπ\[r\(st,at\)\+αℋ\(π\(⋅\|st\)\)\]J\(\\pi\)=\\sum\_\{t=0\}^\{T\}\\mathbb\{E\}\_\{\(s\_\{t\},a\_\{t\}\)\\sim\\rho\_\{\\pi\}\}\\left\[r\(s\_\{t\},a\_\{t\}\)\+\\alpha\\mathcal\{H\}\(\\pi\(\\cdot\|s\_\{t\}\)\)\\right\]
wherer​\(st,at\)r\(s\_\{t\},a\_\{t\}\)denotes the immediate reward,α\\alphais the temperature parameter, andℋ\(π\(⋅\|st\)\)\\mathcal\{H\}\(\\pi\(\\cdot\|s\_\{t\}\)\)represents the entropy of the policyπ\\piat statests\_\{t\}\.

Existing research demonstrates that this mechanism not only improves sample efficiency but also prevents the policy from prematurely converging to suboptimal local minima in highly nonlinear market environments\(Camposet al\.,[2022](https://arxiv.org/html/2607.08681#bib.bib33)\)\.

As a robust value\-based baseline algorithm, Deep Q\-Network \(DQN\) is applied to discrete action spaces\. In multidimensional control tasks, continuous levers are typically discretized\. Despite inherent quantization errors, DQN excels at accurately estimating the action\-value functionQ​\(s,a\)Q\(s,a\)\(Mnihet al\.,[2015](https://arxiv.org/html/2607.08681#bib.bib27); Van Hasseltet al\.,[2016](https://arxiv.org/html/2607.08681#bib.bib26)\)\.

![Refer to caption](https://arxiv.org/html/2607.08681v1/x2.png)Figure 2\.Reinforcement learning evaluation framework for SolarChain\-Eval\. The diagram connects market and physical inputs to policy actions, environment feedback, PPO/SAC/DQN updates, and trustworthiness\-oriented evaluation across utility, safety, stability, smoothness, and fairness\.
### 2\.2\.RL & LLM Agent Integration

RL has shown strong performance in high\-dimensional continuous control tasks\. However, its black\-box nature limits its reliability in infrastructure\(Amiriet al\.,[2025](https://arxiv.org/html/2607.08681#bib.bib16)\)\. In practice, human auditors may find it difficult to explain why an RL policy makes adjustments\(Rudin,[2019](https://arxiv.org/html/2607.08681#bib.bib32)\)\. Furthermore, while post\-hoc AI methods offer retrospective insights, they cannot proactively prevent unsafe actions\(Gu and others,[2024](https://arxiv.org/html/2607.08681#bib.bib6)\)\.

Recent studies address this by integrating Large Language Models \(LLMs\) with RL systems\. LLMs could act as high\-level reasoners to process task context and operator instructions\(Yaoet al\.,[2022](https://arxiv.org/html/2607.08681#bib.bib31)\)\. This ability allows them to translate operator intents into mathematical penalties that can support lower\-level decision making\(Duet al\.,[2023](https://arxiv.org/html/2607.08681#bib.bib30)\)\.

Consequently, current research is shifting toward a bi\-level RL\-LLM architecture\(Cartaet al\.,[2023](https://arxiv.org/html/2607.08681#bib.bib35)\)\. The lower layer uses RL policies to make frequent control decisions based on value estimates such asV​\(s\)V\(s\)orQ​\(s,a\)Q\(s,a\)\. The upper layer uses LLMs for slower and more abstract reasoning tasks\. Prior work has used LLMs to convert operator intent into reward penalties\(Kwonet al\.,[2023](https://arxiv.org/html/2607.08681#bib.bib40)\), revise reward functions, and tune optimization parameters\(Maet al\.,[2023](https://arxiv.org/html/2607.08681#bib.bib41)\)\. Table[1](https://arxiv.org/html/2607.08681#S1.T1)summarizes representative examples of this design\.

Recent advances in RL have improved agent\-based optimization in Cyber\-Physical Systems, especially in decentralized energy markets\(Kellyet al\.,[2020](https://arxiv.org/html/2607.08681#bib.bib37); Lozano and Shi,[2025](https://arxiv.org/html/2607.08681#bib.bib17)\)\. As the focus shifts toward real\-world deployment\(Dulac\-Arnoldet al\.,[2021](https://arxiv.org/html/2607.08681#bib.bib36)\), there is a growing need to complement traditional reward\-centric benchmarks with evaluations of trustworthiness\. Specifically, ensuring safe operation requires assessing an agent’s adherence to physical constraints and its resilience in adversarial scenarios\(Liuet al\.,[2011](https://arxiv.org/html/2607.08681#bib.bib39)\)\. Following this view, our benchmark couples economic dynamics with physical ground truths and evaluates agentic trustworthiness across multiple dimensions\.

## 3\.Methodology

Instead of merely maximizing short\-term trading volume, our benchmark evaluates the overall trustworthiness of an autonomous economic agent\. It assesses whether the agent can maintain market utility while respecting physical limits, ensuring market stability, reducing action jitter, and preserving spatial fairness\. Each episode represents a 24 hour market cycle, sampled from a 720\-hour dataset from April 2026\. Episodes are constrained to start at the beginning of the day\. This temporal design ensures a consistent daily decision horizon for the agent while preserving real\-world weather patterns\.

### 3\.1\.RL Formulation

Figure[2](https://arxiv.org/html/2607.08681#S2.F2)summarizes the RL loop\. Energy generation, market demand, and physics labels are aggregated into an hourly benchmark state\. Based on this state, a policy proposes a governance action\.

The statests\_\{t\}and actionata\_\{t\}are defined as:

\(3\)st\\displaystyle s\_\{t\}=\[sin⁡τt,cos⁡τt,Gtv,Gtr,Ptmax,Δt,Lt,pt,νt,σt,αt−1,ℓt−1\],\\displaystyle=\[\\sin\\tau\_\{t\},\\cos\\tau\_\{t\},G\_\{t\}^\{v\},G\_\{t\}^\{r\},P\_\{t\}^\{\\max\},\\Delta\_\{t\},L\_\{t\},p\_\{t\},\\nu\_\{t\},\\sigma\_\{t\},\\alpha\_\{t\-1\},\\ell\_\{t\-1\}\],at\\displaystyle a\_\{t\}=\(αt,ℓt,bt\),αt\+ℓt≤0\.98\.\\displaystyle=\(\\alpha\_\{t\},\\ell\_\{t\},b\_\{t\}\),\\qquad\\alpha\_\{t\}\+\\ell\_\{t\}\\leq 98\.Here,τt\\tau\_\{t\}is the time angle,GtvG\_\{t\}^\{v\}is verified generation,GtrG\_\{t\}^\{r\}is reported generation,PtmaxP\_\{t\}^\{\\max\}is the physical PV upper bound,Δt\\Delta\_\{t\}is the supply\-demand gap,LtL\_\{t\}is market liquidity,ptp\_\{t\}is token price,νt\\nu\_\{t\}is the raw risk signal, andσt\\sigma\_\{t\}is the static slippage\.

The actionata\_\{t\}comprises the reward allocation ratioαt\\alpha\_\{t\}, liquidity injection ratioℓt\\ell\_\{t\}, and token burn ratebtb\_\{t\}\. Standard algorithms like PPO and SAC operate on the continuous action space, while DQN utilizes a discretized53=1255^\{3\}=125action grid over the same parameters\. All trained policies are evaluated under the uniform benchmark setting detailed in Table[2](https://arxiv.org/html/2607.08681#S3.T2)\.

Table 2\.Main benchmark configuration\.
### 3\.2\.Physics Limit and Market Transition

SolarChain\-Evalimplements a physics constraint module, which verifies whether reported generation is physically feasible and FDIA labels\. LetXtX\_\{t\}denote suspicious or invalid supply\. The benchmark not only records this anomaly but quantifies the extent to which the policy economically backs it:

\(4\)Vt=βt​Xtmax⁡\(Gtv\+βt​Xt,ϵ\),βt=min⁡\(αt\+ℓt,0\.98\)0\.98\.V\_\{t\}=\\frac\{\\beta\_\{t\}X\_\{t\}\}\{\\max\(G\_\{t\}^\{v\}\+\\beta\_\{t\}X\_\{t\},\\epsilon\)\},\\qquad\\beta\_\{t\}=\\frac\{\\min\(\\alpha\_\{t\}\+\\ell\_\{t\},0\.98\)\}\{0\.98\}\.In Eq\. \([4](https://arxiv.org/html/2607.08681#S3.E4)\),VtV\_\{t\}is the action\-dependent physics violation term\. A policy incurs higher risk when it heavily backs invalid supply\. This formulation directly addresses RQ2: through a no\-physics\-penalty ablation, we can observe whether a purely reward\-driven agent exploits invalid generation to inflate artificial liquidity\.

Additionally, the market module processes the backed supply, consisting of verified supply and the action\-backed suspicious supply\. It updates the matched energyMtM\_\{t\}and computes the residual liquidityLt\+1L\_\{t\+1\}for the next hour:

\(5\)Mt\\displaystyle M\_\{t\}=min⁡\(Lt\+ℓt​\(Gtv\+βt​Xt\),Q~t\),\\displaystyle=\\min\(L\_\{t\}\+\\ell\_\{t\}\(G\_\{t\}^\{v\}\+\\beta\_\{t\}X\_\{t\}\),\\ \\widetilde\{Q\}\_\{t\}\),Lt\+1\\displaystyle L\_\{t\+1\}=max⁡\(Lt\+ℓt​\(Gtv\+βt​Xt\)−Mt,0\)\.\\displaystyle=\\max\(L\_\{t\}\+\\ell\_\{t\}\(G\_\{t\}^\{v\}\+\\beta\_\{t\}X\_\{t\}\)\-M\_\{t\},0\)\.Here,Q~t\\widetilde\{Q\}\_\{t\}represents the effective demand after token burning, computed as

Q~t=Qtd​max⁡\(0\.75,1−0\.80​bt\),\\widetilde\{Q\}\_\{t\}=Q\_\{t\}^\{d\}\\max\(0\.75,1\-0\.80b\_\{t\}\),whereQtdQ\_\{t\}^\{d\}is the raw market demand\. This mechanism explicitly links the agent’s governance decisions to both trading volume and market liquidity depth\.

### 3\.3\.Reward Design and RL Optimization

To optimize for both utility and market trustworthiness, the step reward implements a series of penalties \(Figure[2](https://arxiv.org/html/2607.08681#S2.F2)\):

\(6\)Rt\\displaystyle R\_\{t\}=Mt−λd​Dt−λj​Jt−λu​Ut−λp​Φt−λf​Ft,\\displaystyle=M\_\{t\}\-\\lambda\_\{d\}D\_\{t\}\-\\lambda\_\{j\}J\_\{t\}\-\\lambda\_\{u\}U\_\{t\}\-\\lambda\_\{p\}\\Phi\_\{t\}\-\\lambda\_\{f\}F\_\{t\},Φt\\displaystyle\\Phi\_\{t\}=Vt\+At,At=βt​Xt\.\\displaystyle=V\_\{t\}\+A\_\{t\},\\qquad A\_\{t\}=\\beta\_\{t\}X\_\{t\}\.The positive termMtM\_\{t\}rewards successful market clearing\. The corresponding penalties address liquidity drawdown \(DtD\_\{t\}\), action jitter \(JtJ\_\{t\}\), unmet demand \(UtU\_\{t\}\), composite physics risk \(Φt\\Phi\_\{t\}\), and spatial unfairness across cities \(FtF\_\{t\}\)\. Here,Φt\\Phi\_\{t\}combines the normalized physics violation rateVtV\_\{t\}and the action\-backed suspicious supply amountAtA\_\{t\}\.

The RL policies are trained using Stable\-Baselines3\. The benchmark incorporates three learned controllers with distinct optimization biases: PPO for stable on\-policy updates, SAC for entropy\-guided continuous exploration, and DQN for discrete Q\-value learning\. For instance, DQN optimizes the following target over the discretized action grid:

\(7\)yt=Rt\+γ​maxa′⁡Qθ¯​\(st\+1,a′\)\.y\_\{t\}=R\_\{t\}\+\\gamma\\max\_\{a^\{\\prime\}\}Q\_\{\\bar\{\\theta\}\}\(s\_\{t\+1\},a^\{\\prime\}\)\.Static, random, and myopic controllers are also evaluated as non\-learning baselines to provide a comparison\.

### 3\.4\.LLM Planner/Auditor Layer

To evaluate whether agentic oversight improves deployment\-time safety, we introduce an AI\-agent governance layer powered by LLMs, as shown in Figure[3](https://arxiv.org/html/2607.08681#S3.F3)\. We insert this layer between the trained RL policy \(e\.g\., PPO, SAC, DQN\) and the market environment\. This layer is active only during evaluation and is never used during RL training\. This separation ensures that any observed safety improvement stems from deployment\-time guardrails rather than from reward shaping, policy retraining, or parameter updates\.

![Refer to caption](https://arxiv.org/html/2607.08681v1/x3.png)Figure 3\.Interaction between the reinforcement learning layer and the AI\-agent governance layer\. The RL policy proposes market actions, which are bounded by the Planner and conditionally revised by the Auditor under high\-risk scenarios\.The governance layer consists of two components: a Planner and an Auditor\.

The Planner operates at the episode level\. At the start of each 24\-hour episode, it reviews episode\-level context, including supply\-demand adequacy, physics\-risk statistics, static\-market slippage, and benchmark action constraints\. Based on this context, the Planner establishes admissible bounds for the RL action components, namely the reward ratioαt\\alpha\_\{t\}, liquidity ratioℓt\\ell\_\{t\}, and burn ratebtb\_\{t\}\. It also outputs an episode\-specific audit policy, including risk thresholds, a maximum audit budgetBeB\_\{e\}, a target audit rate, and a minimum cooldown lengthcec\_\{e\}\. At each hourly step, the RL policy’s proposed action is first decoded into market variables and clipped by the Planner\-defined bounds and benchmark constraints, yielding a bounded actiona¯t\\bar\{a\}\_\{t\}\.

The Auditor provides sparse step\-level oversight\. Instead of reviewing every action, it is invoked only when risk signals indicate possible safety or market\-integrity failures\. To avoid notational conflict with the reward terms in Eq\. \([6](https://arxiv.org/html/2607.08681#S3.E6)\), we denote the audit\-side action\-instability signal byκt\\kappa\_\{t\}\. It is computed as theℓ1\\ell\_\{1\}distance between the current bounded action and the previous executed action:

\(8\)κt=‖a¯t−at−1‖1\.\\kappa\_\{t\}=\\\|\\bar\{a\}\_\{t\}\-a\_\{t\-1\}\\\|\_\{1\}\.We further define the audit\-side normalized supply\-demand gap as

\(9\)Δta​u​d​i​t=Gtv−Qtdmax⁡\(Qtd,ϵ\),\\Delta^\{audit\}\_\{t\}=\\frac\{G\_\{t\}^\{v\}\-Q\_\{t\}^\{d\}\}\{\\max\(Q\_\{t\}^\{d\},\\epsilon\)\},whereGtvG\_\{t\}^\{v\}denotes verified energy supply,QtdQ\_\{t\}^\{d\}denotes market demand/load, andϵ\>0\\epsilon\>0avoids division by zero\. Negative values ofΔta​u​d​i​t\\Delta^\{audit\}\_\{t\}indicate supply shortfall\.

Audit triggers are divided into hard and soft categories\. Hard triggers respond to immediate safety or market\-integrity threats, including excessive physical violation rateνt\\nu\_\{t\}, supply shortfallΔta​u​d​i​t\\Delta^\{audit\}\_\{t\}, and high static\-market slippageσt\\sigma\_\{t\}:

\(10\)Ht=𝕀​\[νt\>τν∨Δta​u​d​i​t​<τΔ∨σt\>​τσ\]\.H\_\{t\}=\\mathbb\{I\}\\left\[\\nu\_\{t\}\>\\tau\_\{\\nu\}\\ \\vee\\ \\Delta^\{audit\}\_\{t\}<\\tau\_\{\\Delta\}\\ \\vee\\ \\sigma\_\{t\}\>\\tau\_\{\\sigma\}\\right\]\.Soft triggers capture erratic controller behavior through action instability:

\(11\)St=𝕀​\[κt\>τκ\]\.S\_\{t\}=\\mathbb\{I\}\\left\[\\kappa\_\{t\}\>\\tau\_\{\\kappa\}\\right\]\.
An action becomes audit\-eligible when either trigger fires:

\(12\)Et=Ht∨St\.E\_\{t\}=H\_\{t\}\\vee S\_\{t\}\.
To prevent excessive LLM intervention, soft\-triggered audits are rate\-limited by the episode budget and cooldown\. In contrast, hard triggers bypass these limits due to their severity\. LetNe,ta​u​d​i​tN^\{audit\}\_\{e,t\}denote the number of audits already used in episodeeebefore steptt, and lettel​a​s​tt^\{last\}\_\{e\}denote the most recent audited step\. The final audit decision is

\(13\)𝕀ta​u​d​i​t=Et∧\[Ht∨\(Ne,ta​u​d​i​t​<Be∧t−tel​a​s​t\>​ce\)\]\.\\mathbb\{I\}^\{audit\}\_\{t\}=E\_\{t\}\\wedge\\left\[H\_\{t\}\\ \\vee\\ \\left\(N^\{audit\}\_\{e,t\}<B\_\{e\}\\ \\wedge\\ t\-t^\{last\}\_\{e\}\>c\_\{e\}\\right\)\\right\]\.
Thus, severe physics or market failures can trigger immediate review, while softer instability signals are subject to sparse intervention limits\.

If𝕀ta​u​d​i​t=0\\mathbb\{I\}^\{audit\}\_\{t\}=0, the market directly executes the bounded actiona¯t\\bar\{a\}\_\{t\}\. If𝕀ta​u​d​i​t=1\\mathbb\{I\}^\{audit\}\_\{t\}=1, the Auditor receives the current observation, the bounded proposed action, the previous action, the Planner policy, and recent market diagnostics\. It then returns a decision:approve, which keepsa¯t\\bar\{a\}\_\{t\}, orrevise, which replaces it with a conservative corrected action\. The wrapper deterministically applies this decision before calling the environment step\.

Finally, to make the governance layer auditable and economically valid, all LLM outputs must follow structured schemas and numerical validation\(Appendix[B](https://arxiv.org/html/2607.08681#A2): Table[9](https://arxiv.org/html/2607.08681#A2.T9)\)\. Planner outputs are clipped to valid benchmark action ranges and audit\-policy ranges\. Auditor outputs are sanitized against market constraints, including individual bounds onαt\\alpha\_\{t\},ℓt\\ell\_\{t\}, andbtb\_\{t\}, as well as the global allocation constraintαt\+ℓt≤0\.98\\alpha\_\{t\}\+\\ell\_\{t\}\\leq 0\.98\. This schema\-enforced interface prevents free\-form text from directly controlling market execution and ensures that every agentic intervention remains traceable and reproducible during evaluation\.

### 3\.5\.Evaluation Metrics

SolarChain\-Evalevaluates each policy from two perspectives: market utility and system trustworthiness\. We compare six baselines: Static, Random, Myopic, PPO, SAC, and DQN\. Static uses fixed governance parameters, Random samples actions uniformly, and Myopic follows a greedy rule based on current supply, demand, and risk signals\. PPO and SAC operate in the continuous action space, while DQN uses the discretized action grid\.

We use three evaluation settings:

- •Main Benchmark:evaluates all baselines using the full reward in Eq\. \([6](https://arxiv.org/html/2607.08681#S3.E6)\)\. This setting tests whether RL policies can improve utility while maintaining safety, stability, smoothness, and fairness\.
- •Reward Ablation:removes the physics penalty from Eq\. \([6](https://arxiv.org/html/2607.08681#S3.E6)\), while still logging physical violations\. This setting tests whether agents exploit unsafe generation when physical risk is not penalized\.
- •Agentic Evaluation:adds the AI\-agent governance layer to trained RL policies during evaluation\. This setting tests whether LLM\-based oversight can revise unsafe actions without retraining the RL controller\.

Table 3\.Trustworthiness\-oriented evaluation dimensions and reported evidence\.Table 4\.Main benchmark performance under the physics\-constrained reward\. Values are mean±\\pmstandard deviation over all three seeds and 90 rollouts per policy\. Higher cumulative reward and trading volume are better; lower physics violation, slippage, and artificial liquidity indicate safer market governance\.![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_1a_rq1_reward_distribution.png)\(a\)Rollout\-level cumulative reward\.
![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_1b_rq1_utility_safety_frontier.png)\(b\)Utility–safety frontier\. Each point is one rollout; larger markers indicate higher artificial liquidity\.
![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_1c_rq1_reward_episode_trajectory.png)\(c\)Episode\-wise reward trajectory with 95% confidence bands across seeds\.

Figure 4\.Main benchmark performance under the physics\-constrained reward\.![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_2a_rq1_physics_violation_rate.png)\(a\)Physics violation rate\.
![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_2b_rq1_artificial_liquidity_MWh.png)\(b\)Artificial liquidity\.
![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_2c_rq1_mean_slippage.png)\(c\)Mean slippage\.

Figure 5\.Trustworthiness metrics in the main benchmark\. Translucent points are individual rollouts\.

## 4\.Evaluation

Our empirical evaluation addresses the research questions using data aggregated across three independent seeds\. We benchmark three agentic policies \(PPO\+LLM, SAC\+LLM, DQN\+LLM\) against six non\-agentic baselines \(PPO, SAC, DQN, Static, Random, Myopic\)\. The evaluation dataset \(Appendix[B](https://arxiv.org/html/2607.08681#A2)\) comprises 1,620 episode\-level records, 38,880 hourly actions, 194,400 hourly state records, and 12,960 LLM governance log entries\.

Table 5\.Paired without\-constraints deltas with and without the LLM governance layer\. Each row uses 90 paired rollouts across three seeds\. PositiveΔ\\Deltareward indicates that removing the physics penalty increases scalar utility; positiveΔ\\Deltaviolation or artificial liquidity indicates increased safety risk\.![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_3a_rq2_paired_reward_risk_delta.png)\(a\)Paired without\-constraints artificial\-liquidity deltas for RL and RL\+LLM\.
![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_3b_rq2_artificial_liquidity_bars.png)\(b\)Artificial liquidity across RL, RL\+LLM, and their without\-constraints ablations\.
![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_3c_rq2_artificial_liquidity_trajectory.png)\(c\)Episode\-wise artificial\-liquidity trajectories for all four learned\-agent settings\.

Figure 6\.Ablation experiment of physics constraints\. Translucent points are rollout\-level or paired\-rollout observations\.### 4\.1\.Benchmark Performance

We first evaluate the RL policies against three baseline models\(Static, Random, Myopic\) for market governance\. To assess trustworthiness comprehensively, Table[4](https://arxiv.org/html/2607.08681#S3.T4)reports utility, physical consistency, and market\-execution metrics simultaneously\. Results indicate that learned RL control provides a clear advantage over non\-RL baselines\. Across 90 rollouts per policy, the RL policies achieve a higher mean cumulative reward than the static, random, and myopic baselines \(−22\.61\-22\.61vs\.−23\.97\-23\.97; Welch’stt\-test,p<10−28p<10^\{\-28\}\)\. This improvement is primarily driven by continuous\-control methods, as both PPO and SAC outperform all baselines \(p<10−6p<10^\{\-6\}for all pairwise tests\)\. Notably, these non\-RL baselines are non\-trivial: they represent stable governance rules \(static\), pure action\-space exploration \(random\), and local rationality \(myopic\)\. Their underperformance suggests that effective market governance requires adaptive RL rather than rigid or one\-step heuristics\.

However, trustworthiness metrics reveal a complex utility\-safety tradeoff\. While SAC achieves the highest cumulative reward and trading volume alongside the lowest physics violation rate, it suffers from the highest mean slippage among the RL policies\. Conversely, PPO offers the lowest slippage but generates more artificial liquidity\. DQN improves upon random control but fails to consistently outperform the myopic baseline, illustrating that RL architecture choice is critical for fine\-grained control tasks\. Consequently, these patterns expose an evaluation frontier\. For rigorous trustworthiness evaluation, maximizing reward does not guarantee a safe policy if the gain introduces market friction or lacks physical backing\. Figure[4](https://arxiv.org/html/2607.08681#S3.F4)illustrates this frontier, showing the distinct operational trade\-offs and residual variability within each policy type\.

The metric decomposition in Figure[5](https://arxiv.org/html/2607.08681#S3.F5)further unpacks this frontier\. Although RL policies reduce overall physics violations compared to non\-RL baselines, the underlying risk is redistributed across artificial liquidity and slippage\. SAC excels at physical credibility, whereas PPO prioritizes slippage control\.

Table 6\.Paired effects of adding the LLM Planner/Auditor layer under the physics\-constrained reward\.Δ\\Deltacolumns are RL\+LLM minus RL over 90 matched rollouts across three seeds; audit and revision rates are computed from 2,160 governance log steps per policy\.![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_4a_rq3_action_stability.png)\(a\)Action stability: RL\+LLM reduces action jitter for SAC and DQN,
![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_4b_rq3_agentic_delta_scatter.png)\(b\)Paired RL\+LLM\-minus\-RL changes in action jitter and artificial liquidity\.
![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_4c_rq3_audit_step_delta_scatter.png)\(c\)Step\-level relationship between pre\-audit action jitter and Auditor action delta\.

Figure 7\.Agentic governance diagnostics for the LLM Planner/Auditor layer\. The panels combine summary auditability statistics, paired episode\-level governance effects, and step\-level audit traces\.
### 4\.2\.Ablation Experiment of Physics Constraints

The ablation study investigates the necessity of adding physical constraints to our RL and AI agent models\. We examine four configurations: standard RL, RL without physics constraints, RL with LLM governance, and RL with LLM governance but lacking physics constraints\. In the unconstrained settings, we remove the physics penalty from the reward function\(Eq\.6\) while continuing to track physical violations and artificial liquidity\. This design decouples the agent’s optimization objective from the benchmark’s safety diagnostics\. Consequently, scalar reward maximization does not inherently equate to a better market outcome\. If an agent boosts its reward by backing invalid supply and inflating apparent liquidity, the benchmark can explicitly expose it\.

The empirical results in Table[5](https://arxiv.org/html/2607.08681#S4.T5)demonstrate that removing the physics penalty inflates scalar rewards, but at the cost of severe safety risks\. Under raw RL deployment, artificial liquidity surges by 0\.0785, 0\.1238, and 0\.0576 MWh for PPO, SAC, and DQN, respectively\. This failure mode persists even under RL\+LLM deployment, where artificial liquidity still increases by 0\.0598, 0\.1149, and 0\.0413 MWh\. This illustrates a consistent shift toward higher artificial liquidity when constraints are ablated across both governance regimes \(Figure[6\(a\)](https://arxiv.org/html/2607.08681#S4.F6.sf1)\)\. Although the LLM layer mitigates the magnitude of these violations, it cannot eliminate the misalignment caused by a misspecified reward\.

This comparison highlights the complementary roles of physical constraints and AI agent governance\. While LLM governance audits actions post\-hoc, physics\-constrained RL embeds safety during optimization to prevent the exploitation of unviable liquidity\. Aggregated across learned policies, mean artificial liquidity remains low in constrained environments \(0\.1146 MWh for RL; 0\.1181 MWh for RL\+LLM\) but spikes in unconstrained settings \(0\.2012 MWh for RL; 0\.1901 MWh for RL\+LLM\) \(Figure[6\(b\)](https://arxiv.org/html/2607.08681#S4.F6.sf2)\)\. Furthermore, this risk inflation is not an artifact of isolated rollouts, but a persistent behavioral trajectory woven throughout the evaluation episodes \(Figure[6\(c\)](https://arxiv.org/html/2607.08681#S4.F6.sf3)\)\. This confirms that unconstrained optimization remains riskier, even with active agentic oversight\.

### 4\.3\.Evaluation of Agentic Layer

Having proven the function of physics constraints, we now evaluate the AI agent layer’s capacity for verifiable oversight\. By analyzing transparent audit traces from market execution, we assess the system’s effectiveness as an auditable risk\-control interface\. This evaluation specifically quantifies intervention triggers, the magnitude of action modifications, and the underlying rationales for these corrections\.

To investigate the reliability of the AI agent mechanism, Table[6](https://arxiv.org/html/2607.08681#S4.T6)presents metrics derived from 12,960 step\-level governance logs \(encompassing three seeds, two constraint settings, and three learned policies\)\. The system demonstrates exceptional reliability, with a Planner validity of 1\.0 across all configurations\. Notably, the evaluation reveals that effective oversight is highly selective rather than continuous\. Under the physics\-constrained setting, the Auditor intervenes in 34\.2%–37\.2% of steps, whereas in the unconstrained setting, the intervention rate is 27\.3%–29\.7%\. The Auditor revises proposed actions in over 95% of cases under constraints, and over 83% without constraints\.

The empirical results suggest that agentic governance functions as an auditable risk\-control interface rather than a universal performance optimizer\. Its impact is highly dependent on the behavioral characteristics of the base RL policy\. For instance\(Figure[7\(a\)](https://arxiv.org/html/2607.08681#S4.F7.sf1)\), the LLM layer successfully suppresses action jitter for SAC and DQN by 0\.0290 and 0\.0179, respectively\. Conversely, it marginally increases PPO jitter by 0\.0223, primarily due to more frequent clipping and targeted revisions of PPO’s proposed actions\. Similar policy\-dependent trade\-offs emerge in artificial liquidity: oversight reduces it for PPO \(−0\.0074\-0\.0074MWh\) but slightly increases it for SAC \(\+0\.0073\+0\.0073MWh\) and DQN \(\+0\.0109\+0\.0109MWh\)\. Spatial fairness remains largely stable, with minimal deltas across all policies\(Figure[7\(b\)](https://arxiv.org/html/2607.08681#S4.F7.sf2)\)\.

Table 7\.Representative LLM audit trace selected fromagentic\_logs\.jsonl\. The row records the market context, proposed RL action, audited final action, and action delta\.These mixed outcomes reveal that researchers should not assume that deploying an LLM governance layer automatically resolves all safety risks\. Instead, it must rigorously define the boundaries of such oversight\. Within the decentralized energy market setting,SolarChain\-Evaldemonstrates that while the AI agent successfully intercepts unstable decisions, as evidenced by the positive correlation between pre\-audit action jitter and the corrective delta in Figure[7\(c\)](https://arxiv.org/html/2607.08681#S4.F7.sf3), it cannot fully compensate for a misspecified RL reward\. Ultimately, our findings advocate for a complementary approach to trustworthy autonomous systems\. While agentic governance guarantees post\-deployment transparency and event\-driven risk mitigation, physically constrained RL optimization remains necessary to prevent reward\-driven exploitation\.

### 4\.4\.Case Study: Transparent LLM Audit Trace

![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_5b_rq4_action_revision_bars.png)\(a\)Selected action revision\.
![Refer to caption](https://arxiv.org/html/2607.08681v1/figures/fig_5a_rq4_selected_trace_context.png)\(b\)Context of audited PPO steps\.

Figure 8\.Case\-study audit trace for PPO under the LLM governance setting\.To illustrate the transparency of our evaluation, we analyze a representative audit trace \(Table[7](https://arxiv.org/html/2607.08681#S4.T7)\)\. We select a scenario where the PPO agent operates without physical constraints, a setting where reward misspecification inherently encourages risky market behavior\. At the evaluated step, environmental risk signals are elevated, including a raw physics violation rate of 0\.14 and a supply gap of 1\.97\.

Driven by economic utility, the RL policy proposes an aggressive, high\-liquidity action\(r=0\.049,l=0\.931,b=0\.200\)\(r=0\.049,l=0\.931,b=0\.200\)\. Recognizing the safety breach, the LLM Auditor intervenes and revises the action to a conservative bound\(r=0\.050,l=0\.200,b=0\.000\)\(r=0\.050,l=0\.200,b=0\.000\), resulting in a substantial action delta of 0\.932 \(Figure[8\(b\)](https://arxiv.org/html/2607.08681#S4.F8.sf2)\)\. Beyond the numerical correction, the physical violation rate exceeded the safety threshold, necessitating a tighter liquidity allocation to limit exposure to unsafe supply\.

This case validates the contribution of the agentic layer to trustworthy evaluation\.SolarChain\-Evalrecords the entire decision\-making process\. It explicitly documents the activating risk signals, the resulting action adjustments, and the auditor’s natural language rationale\. By providing this transparency,SolarChain\-Evalguarantees that autonomous interventions remain accountable after deployment\.

## 5\.Conclusion

In conclusion, we proposedSolarChain\-Eval, a physics\-constrained benchmark designed to evaluate the trustworthiness of economic agents in decentralized energy markets\. Our empirical evaluations reveal a clear trade\-off inherent in autonomous market governance\. While RL policies improve economic utility relative to static and heuristic rules, they may compromise safety for market efficiency\. This vulnerability is starkly exposed when physical constraints are removed: driven purely by reward maximization, an RL policy may exploit false data injection attacks to artificially inflate market liquidity\. To counter this flaw, combining physical constraints with an AI\-agent governance layer provides a structured and auditable safety interface\. By evaluating context and intercepting high\-risk decisions, the AI\-agent layer creates transparent intervention traces and can mitigate instability patterns, although its corrective impact remains policy\-dependent and bounded by the underlying RL controller\.

The deployment of autonomous agents in cyber\-physical systems requires evaluation paradigms that look beyond standard reward maximization\.SolarChain\-Evaladdresses this gap by logging the complete decision loop, including environmental risk signals, action corrections, and natural language explanations\. Consequently, our benchmark supports deployment\-oriented stress tests in which agent behaviors remain economically viable, verifiable, accountable, and physically grounded\.

Future work will makeSolarChain\-Evalbroader and more diagnostic\. We will test whether the utility–safety–stability–fairness–auditability template transfers to autonomous trading and supply\-chain allocation by replacing the PV physics oracle with domain\-specific feasibility or risk checks\. We will also calibrate the simulator with broader meteorological and market traces and run sensitivity analyses over reward weights, FDIA frequency and severity, node scale, and market\-clearing assumptions\. For the Planner/Auditor, we will measure selectivity with safe\-action holdouts, approval\-only controls, false\-positive and false\-negative rates, and trigger\-level ablations, separating meaningful risk detection from overly conservative defaults\. Finally, we will compareSolarChain\-Evalwith constrained\-RL and agentic\-benchmark baselines to clarify when post\-hoc governance helps and when reward misspecification must be addressed during training\.

## References

- J\. Achiam, D\. Held, A\. Tamar, and P\. Abbeel \(2017\)Constrained policy optimization\.InInternational conference on machine learning,pp\. 22–31\.Cited by:[§1](https://arxiv.org/html/2607.08681#S1.p2.1)\.
- M\. H\. N\. Amiri, F\. Annaz, M\. De Oliveira, and F\. Gueniat \(2025\)Deep reinforcement learning with local interpretability for transparent microgrid resilience energy management\.arXiv preprint arXiv:2508\.08132\.Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p1.1)\.
- G\. Campos, N\. H\. El\-Farra, and A\. Palazoglu \(2022\)Soft actor\-critic deep reinforcement learning with hybrid mixed\-integer actions for demand responsive scheduling of energy systems\.Industrial & Engineering Chemistry Research61\(24\),pp\. 8443–8461\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p8.1)\.
- T\. Carta, C\. Romac, T\. Wolf, S\. Lamprier, O\. Sigaud, and P\. Oudeyer \(2023\)Grounding large language models in interactive environments with online reinforcement learning\.InInternational Conference on Machine Learning,pp\. 3676–3713\.Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p3.2)\.
- T\. Chen, S\. Bu, X\. Liu, J\. Kang, F\. R\. Yu, and Z\. Han \(2022\)Peer\-to\-peer energy trading and energy conversion in interconnected multi\-energy microgrids using multi\-agent deep reinforcement learning\.IEEE Transactions on Smart Grid13\(1\),pp\. 715–727\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p1.1)\.
- L\. W\. Cong, Y\. Li, and N\. Wang \(2021\)Tokenomics: dynamic adoption and valuation\.The Review of Financial Studies34\(3\),pp\. 1105–1155\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p1.1)\.
- J\. Drgona, T\. X\. Nghiem, T\. Beckers, M\. Fazlyab, E\. Mallada, C\. Jones, D\. Vrabie, S\. L\. Brunton, and R\. Findeisen \(2025\)Safe physics\-informed machine learning for dynamics and control\.InProceedings of the American Control Conference,Cited by:[§1](https://arxiv.org/html/2607.08681#S1.p1.1)\.
- Y\. Du, O\. Watkins, Z\. Wang, C\. Colas, T\. Darrell, P\. Abbeel, A\. Gupta, and J\. Andreas \(2023\)Guiding pretraining in reinforcement learning with large language models\.InInternational Conference on Machine Learning,pp\. 8657–8677\.Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p2.1)\.
- G\. Dulac\-Arnold, N\. Levine, D\. J\. Mankowitz, J\. Li, C\. Paduraru, T\. Hester,et al\.\(2021\)Challenges of real\-world reinforcement learning: definitions, benchmarks and analysis\.Machine Learning110\(9\),pp\. 2419–2468\.Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p4.1)\.
- S\. Guet al\.\(2024\)A review of safe reinforcement learning: methods, theory and applications\.IEEE Transactions on Pattern Analysis and Machine Intelligence\.Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p1.1)\.
- T\. Haarnoja, A\. Zhou, P\. Abbeel, and S\. Levine \(2018\)Soft actor\-critic: off\-policy maximum entropy deep reinforcement learning with a stochastic actor\.InInternational conference on machine learning,pp\. 1861–1870\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p6.1)\.
- A\. Kelly, A\. O’Sullivan, P\. de Mars, and A\. Marot \(2020\)Reinforcement learning for electricity network operation\.InElectric Power Systems Research,Vol\.189,pp\. 106740\.Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p4.1)\.
- M\. Kwon, S\. M\. Xie, K\. Bullard, and D\. Sadigh \(2023\)Reward design with language models\.InThe Eleventh International Conference on Learning Representations,Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p3.2)\.
- Y\. Liu, P\. Ning, and M\. K\. Reiter \(2011\)False data injection attacks against state estimation in electric power grids\.ACM Transactions on Information and System Security \(TISSEC\)14\(1\),pp\. 1–33\.Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p4.1)\.
- D\. R\. Lozano and Y\. Shi \(2025\)Democratizing microgrid optimization: an llm agent for dispatching mobile chargers to construction electric vehicles\.InNeurIPS 2025 Workshop on Bridging Language, Agent, and World Models for Reasoning and Planning,Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p4.1)\.
- Y\. J\. Ma, W\. Liang, G\. Wang, D\. Huang, O\. Bastani, D\. Jayaraman, Y\. Zhu, L\. Fan, and A\. Anandkumar \(2023\)Eureka: human\-level reward design via coding large language models\.InThe Twelfth International Conference on Learning Representations,Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p3.2)\.
- K\. Malinova and A\. Park \(2023\)Tokenomics: when tokens beat equity\.Management Science69\(11\),pp\. 6568–6583\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p1.1)\.
- E\. Mengelkamp, J\. Gärttner, K\. Rock, S\. Kessler, L\. Orsini, and C\. Weinhardt \(2018\)Designing microgrid energy markets: a case study: the brooklyn microgrid\.Applied Energy210,pp\. 870–880\.Cited by:[§1](https://arxiv.org/html/2607.08681#S1.p2.1)\.
- V\. Mnih, K\. Kavukcuoglu, D\. Silver, A\. A\. Rusu, J\. Veness, M\. G\. Bellemare, A\. Graves, M\. Riedmiller, A\. K\. Fidjeland, G\. Ostrovski,et al\.\(2015\)Human\-level control through deep reinforcement learning\.nature518\(7540\),pp\. 529–533\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p9.1)\.
- National Institute of Standards and Technology \(2023\)Artificial intelligence risk management framework \(ai rmf 1\.0\)\.Technical reportTechnical ReportNIST AI 100\-1,National Institute of Standards and Technology\.External Links:[Document](https://dx.doi.org/10.6028/NIST.AI.100-1)Cited by:[§1](https://arxiv.org/html/2607.08681#S1.p1.1)\.
- S\. Ou, Y\. Xu, Z\. Zhang, L\. Zhang, and M\. Huang \(2026\)SolarChain: bridging physical law, verifiable trust, and sustainable markets for urban energy resilience\.arXiv preprint arXiv:2605\.23162\.Cited by:[§1](https://arxiv.org/html/2607.08681#S1.p2.1)\.
- C\. C\. Phiri \(2025\)Creating characteristically auditable agentic ai systems\.InProceedings of the Intelligent Robotics FAIR 2025,External Links:[Document](https://dx.doi.org/10.1145/3759355.3759356)Cited by:[§1](https://arxiv.org/html/2607.08681#S1.p1.1)\.
- H\. Qu, K\. M\. Gogol, F\. Grötschla, and C\. J\. Tessone \(2025\)From rules to rewards: reinforcement learning for interest rate adjustment in defi lending\.InThe International Conference on Mathematical Research for Blockchain Economy,pp\. 85–120\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p2.4)\.
- A\. Raffin, A\. Hill, A\. Gleave, A\. Kanervisto, M\. Ernestus, and N\. Dormann \(2021\)Stable\-baselines3: reliable reinforcement learning implementations\.Journal of Machine Learning Research22\(268\),pp\. 1–8\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p2.4)\.
- C\. Rudin \(2019\)Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead\.Nature machine intelligence1\(5\),pp\. 206–215\.Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p1.1)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Proximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p3.1)\.
- W\. Tushar, C\. Yuen, H\. Mohsenian\-Rad, T\. Saha, H\. V\. Poor, and K\. L\. Wood \(2018\)Transforming energy networks via peer\-to\-peer energy trading: the potential of game\-theoretic approaches\.IEEE Signal Processing Magazine35\(4\),pp\. 90–111\.Cited by:[§1](https://arxiv.org/html/2607.08681#S1.p2.1)\.
- H\. Van Hasselt, A\. Guez, and D\. Silver \(2016\)Deep reinforcement learning with double q\-learning\.InProceedings of the AAAI conference on artificial intelligence,Vol\.30\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p9.1)\.
- J\. Vyas and M\. Mercangöz \(2025\)Autonomous industrial control using an agentic framework with large language models\.IFAC\-PapersOnLine59\(6\),pp\. 349–354\.Cited by:[§1](https://arxiv.org/html/2607.08681#S1.p1.1)\.
- Z\. Xi, W\. Chen, X\. Guo, W\. He, Y\. Ding, B\. Hong, M\. Zhang, J\. Wang, S\. Jin, E\. Zhou,et al\.\(2025\)The rise and potential of large language model based agents: a survey\.Science China Information Sciences68\(2\),pp\. 121101\.Cited by:[§1](https://arxiv.org/html/2607.08681#S1.p1.1)\.
- J\. Xu, Y\. Feng, D\. Perez, and B\. Livshits \(2025\)Auto\. gov: learning\-based governance for decentralized finance \(defi\)\.IEEE Transactions on Services Computing\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p2.4)\.
- H\. Yang, Z\. Li, Y\. Liu, Y\. Xiang, L\. Li, J\. Yang, L\. Tan, S\. Wang, H\. Ma, Z\. Xi,et al\.\(2025a\)LLM\-powered distributed optimal scheduling for industrial heat\-electricity micro\-grids\.IEEE Transactions on Industry Applications\.Cited by:[Table 1](https://arxiv.org/html/2607.08681#S1.T1.1.4.2.1.1.1)\.
- X\. Yang, C\. Lin, H\. Liu, and W\. Wu \(2025b\)RL2: reinforce large language model to assist safe reinforcement learning for energy management of active distribution networks\.IEEE Transactions on Smart Grid\.Cited by:[Table 1](https://arxiv.org/html/2607.08681#S1.T1.1.1.1.1.1)\.
- C\. Yao, X\. Liu, C\. Li, and M\. Savvides \(2025\)Reward evolution with graph\-of\-thoughts: a bi\-level language model framework for reinforcement learning\.arXiv preprint arXiv:2509\.16136\.Cited by:[Table 1](https://arxiv.org/html/2607.08681#S1.T1.1.3.1.1.1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2022\)React: synergizing reasoning and acting in language models\.arXiv preprint arXiv:2210\.03629\.Cited by:[§2\.2](https://arxiv.org/html/2607.08681#S2.SS2.p2.1)\.
- D\. Zhang, X\. Han, and C\. Deng \(2018\)Review on the research and practice of deep learning and reinforcement learning in smart grids\.CSEE Journal of Power and Energy Systems4\(3\),pp\. 362–370\.Cited by:[§1](https://arxiv.org/html/2607.08681#S1.p2.1)\.
- S\. Zheng, A\. Trott, S\. Srinivasa, D\. C\. Parkes, and R\. Socher \(2022\)The ai economist: taxation policy design via two\-level deep multiagent reinforcement learning\.Science advances8\(18\),pp\. eabk2607\.Cited by:[§2\.1](https://arxiv.org/html/2607.08681#S2.SS1.p1.1)\.

## Appendix AFormulas Implementation

This appendix lists some important equations used by the releasedSolarChain\-Evalcode but not separately expanded in the methodology section\.

#### PV capacity and trust labels\.

The data generator converts weather into a physical PV upper bound and the loader derives the trust labels used by the benchmark:

\(14\)Ic,t\\displaystyle I\_\{c,t\}=𝕀​\[zc,t<90∘\]​min⁡\{max⁡\(Sc,t,0\),1\.08​max⁡\(Cc,t,0\)\},\\displaystyle=\\mathbb\{I\}\[z\_\{c,t\}<0^\{\\circ\}\]\\,\\min\\\{\\max\(S\_\{c,t\},0\),108\\max\(C\_\{c,t\},0\)\\\},ηi,tt​e​m​p\\displaystyle\\eta^\{temp\}\_\{i,t\}=clip​\{1\+γi​\(Tc,t−25\),0\.78,1\.08\},\\displaystyle=\\mathrm\{clip\}\\\{1\+\\gamma\_\{i\}\(T\_\{c,t\}\-5\),078,108\\\},Pi,tm​a​x\\displaystyle P^\{max\}\_\{i,t\}=max⁡\{0,Ic,t​Ai​ηi​ηi,tt​e​m​p\},\\displaystyle=\\max\\\{0,I\_\{c,t\}A\_\{i\}\\eta\_\{i\}\\eta^\{temp\}\_\{i,t\}\\\},Gi,tv\\displaystyle G^\{v\}\_\{i,t\}=𝕀​\[statusi,t=verified\]​Pi,tr​e​p​o​r​t​e​d,\\displaystyle=\\mathbb\{I\}\[\\mathrm\{status\}\_\{i,t\}=\\mathrm\{verified\}\]P^\{reported\}\_\{i,t\},Xi,te​x​c​e​s​s\\displaystyle X^\{excess\}\_\{i,t\}=max⁡\(Pi,tr​e​p​o​r​t​e​d−Pi,tm​a​x,0\),\\displaystyle=\\max\(P^\{reported\}\_\{i,t\}\-P^\{max\}\_\{i,t\},0\),violi,t\\displaystyle\\mathrm\{viol\}\_\{i,t\}=𝕀​\[Xi,te​x​c​e​s​s\>10−6∨FDIAi,t\],\\displaystyle=\\mathbb\{I\}\[X^\{excess\}\_\{i,t\}\>0^\{\-6\}\\vee\\mathrm\{FDIA\}\_\{i,t\}\],Ri,tr​e​j​e​c​t​e​d\\displaystyle R^\{rejected\}\_\{i,t\}=violi,t​Pi,tr​e​p​o​r​t​e​d\.\\displaystyle=\\mathrm\{viol\}\_\{i,t\}P^\{reported\}\_\{i,t\}\.

#### Action decoding and allocation budget\.

Continuous policy outputs are decoded into governance ratios and then sanitized against the global allocation budget:

\(15\)αt\\displaystyle\\alpha\_\{t\}=αmin\+ut,1​\(αmax−αmin\),\\displaystyle=\\alpha\_\{\\min\}\+u\_\{t,1\}\(\\alpha\_\{\\max\}\-\\alpha\_\{\\min\}\),ℓt\\displaystyle\\ell\_\{t\}=ℓmin\+ut,2​\(ℓmax−ℓmin\),\\displaystyle=\\ell\_\{\\min\}\+u\_\{t,2\}\(\\ell\_\{\\max\}\-\\ell\_\{\\min\}\),bt\\displaystyle b\_\{t\}=ut,3​bmax,\\displaystyle=u\_\{t,3\}b\_\{\\max\},st\\displaystyle s\_\{t\}=min⁡\{1,0\.98αt\+ℓt\},\\displaystyle=\\min\\left\\\{1,\\frac\{0\.98\}\{\\alpha\_\{t\}\+\\ell\_\{t\}\}\\right\\\},\(αt,ℓt\)\\displaystyle\(\\alpha\_\{t\},\\ell\_\{t\}\)←\(st​αt,st​ℓt\)\.\\displaystyle\\leftarrow\(s\_\{t\}\\alpha\_\{t\},s\_\{t\}\\ell\_\{t\}\)\.DQN uses the same decoder after selecting one point from a53=1255^\{3\}=125normalized action grid\.

#### Unsafe supply, artificial liquidity, and market clearing\.

The implementation records how much invalid supply is economically backed and how much liquidity it creates:

\(16\)Xt\\displaystyle X\_\{t\}=max\{∑iRi,tr​e​j​e​c​t​e​d,∑iXi,te​x​c​e​s​s,\\displaystyle=\\max\\Biggl\\\{\\sum\_\{i\}R^\{rejected\}\_\{i,t\},\\sum\_\{i\}X^\{excess\}\_\{i,t\},max\(∑iPi,tr​e​p​o​r​t​e​d−∑iPi,tm​a​x,0\)\},\\displaystyle\\qquad\\quad\\max\\Bigl\(\\sum\_\{i\}P^\{reported\}\_\{i,t\}\-\\sum\_\{i\}P^\{max\}\_\{i,t\},0\\Bigr\)\\Biggr\\\},βt\\displaystyle\\beta\_\{t\}=min⁡\(αt\+ℓt,0\.98\)0\.98,\\displaystyle=\\frac\{\\min\(\\alpha\_\{t\}\+\\ell\_\{t\},0\.98\)\}\{0\.98\},Atu​n​s​a​f​e\\displaystyle A^\{unsafe\}\_\{t\}=βt​Xt,\\displaystyle=\\beta\_\{t\}X\_\{t\},Gtb​a​c​k​e​d\\displaystyle G^\{backed\}\_\{t\}=Gtv\+Atu​n​s​a​f​e,\\displaystyle=G^\{v\}\_\{t\}\+A^\{unsafe\}\_\{t\},Ata​r​t​i​f​i​c​i​a​l\\displaystyle A^\{artificial\}\_\{t\}=ℓt​Atu​n​s​a​f​e\.\\displaystyle=\\ell\_\{t\}A^\{unsafe\}\_\{t\}\.The executed market transition is then

\(17\)Q~t\\displaystyle\\widetilde\{Q\}\_\{t\}=Qtd​max⁡\(0\.75,1−0\.80​bt\),\\displaystyle=Q^\{d\}\_\{t\}\\max\(75,1\-80b\_\{t\}\),Lta​v​a​i​l\\displaystyle L^\{avail\}\_\{t\}=Lt\+ℓt​Gtb​a​c​k​e​d,\\displaystyle=L\_\{t\}\+\\ell\_\{t\}G^\{backed\}\_\{t\},Mt\\displaystyle M\_\{t\}=min⁡\(Lta​v​a​i​l,Q~t\),\\displaystyle=\\min\(L^\{avail\}\_\{t\},\\widetilde\{Q\}\_\{t\}\),Lt\+1\\displaystyle L\_\{t\+1\}=max⁡\(Lta​v​a​i​l−Mt,0\),\\displaystyle=\\max\(L^\{avail\}\_\{t\}\-M\_\{t\},0\),σte​x​e​c\\displaystyle\\sigma^\{exec\}\_\{t\}=Q~tmax⁡\(Lta​v​a​i​l\+0\.05,0\.05\)\.\\displaystyle=\\frac\{\\widetilde\{Q\}\_\{t\}\}\{\\max\(L^\{avail\}\_\{t\}\+0\.05,0\.05\)\}\.

#### Episode, fairness, and audit metrics\.

The evaluation code aggregates step records into rollout\-level metrics:

\(18\)Re\\displaystyle R\_\{e\}=∑t∈eRt,\\displaystyle=\\sum\_\{t\\in e\}R\_\{t\},Ve\\displaystyle V\_\{e\}=124​∑t∈eVt,\\displaystyle=\\frac\{1\}\{24\}\\sum\_\{t\\in e\}V\_\{t\},Je\\displaystyle J\_\{e\}=123​∑t=224‖at−at−1‖1,\\displaystyle=\\frac\{1\}\{23\}\\sum\_\{t=2\}^\{24\}\\\|a\_\{t\}\-a\_\{t\-1\}\\\|\_\{1\},σ¯e\\displaystyle\\bar\{\\sigma\}\_\{e\}=124​∑t∈eσte​x​e​c,\\displaystyle=\\frac\{1\}\{24\}\\sum\_\{t\\in e\}\\sigma^\{exec\}\_\{t\},Aea​r​t​i​f​i​c​i​a​l\\displaystyle A^\{artificial\}\_\{e\}=∑t∈eAta​r​t​i​f​i​c​i​a​l\.\\displaystyle=\\sum\_\{t\\in e\}A^\{artificial\}\_\{t\}\.City\-level reward imbalance and agentic intervention strength are reported as

\(19\)Fe\\displaystyle F\_\{e\}=Varc​\(∑t∈erc,tc​i​t​y\)max⁡\(1\|C\|​∑c\|∑t∈erc,tc​i​t​y\|,10−9\),\\displaystyle=\\frac\{\\mathrm\{Var\}\_\{c\}\\left\(\\sum\_\{t\\in e\}r^\{city\}\_\{c,t\}\\right\)\}\{\\max\\left\(\\frac\{1\}\{\|C\|\}\\sum\_\{c\}\\left\|\\sum\_\{t\\in e\}r^\{city\}\_\{c,t\}\\right\|,10^\{\-9\}\\right\)\},δt\\displaystyle\\delta\_\{t\}=‖atf​i​n​a​l−ato​r​i​g​i​n​a​l‖1,\\displaystyle=\\\|a^\{final\}\_\{t\}\-a^\{original\}\_\{t\}\\\|\_\{1\},audit​\_​rate\\displaystyle\\mathrm\{audit\\\_rate\}=Na​u​d​i​t24,\\displaystyle=\\frac\{N^\{audit\}\}\{24\},revision​\_​rate\\displaystyle\\mathrm\{revision\\\_rate\}=Nr​e​v​i​s​emax⁡\(Na​u​d​i​t,1\)\.\\displaystyle=\\frac\{N^\{revise\}\}\{\\max\(N^\{audit\},1\)\}\.

## Appendix BData Provenance and Evaluation Matrix

Table[8](https://arxiv.org/html/2607.08681#A2.T8)summarizes the benchmark input data used in this paper and documents the provenance of each raw file before policy evaluation\. The primary dataset is a benchmark instance covering Beijing, Shanghai, Chengdu, Shenzhen, and Hangzhou from2026\-04\-01to2026\-05\-01\. It contains 720 hourly timestamps and integrates cached city\-level weather, node\-level PV parameters, reported generation records, FDIA and verification labels, aggregate market\-liquidity states, and generated P2P transaction traces\.

The evaluation artifacts reported in the paper are derived from these raw inputs through a fixed evaluation pipeline\. The stored evidence records outcomes at multiple granularities, including episode\-level metrics for rollout rewards and safety outcomes, hourly action rows for executed policy decisions, city\-hour policy rows for spatial market states, violation, and fairness calculations, and LLM governance logs for Planner/Auditor decisions\. Overall, the evaluation matrix contains 1,620 episode\-level metric rows, 38,880 hourly action rows, 194,400 city\-hour policy rows, and 12,960 LLM governance log rows\.

Table 8\.Raw benchmark data files and collection or generation methods\. Row counts exclude CSV headers\.Table 9\.Reproducibility protocol for the LLM Planner/Auditor layer\.Table 10\.Detailed main\-setting episode summaries\. Reward, volume, violation, artificial liquidity, slippage, and action jitter are aggregated over 90 rollouts per policy\.Table 11\.Four\-setting summaries for learned policies\. Each row aggregates 90 rollouts across three seeds\. These values support the paired physics\-removal deltas and show how the LLM governance layer changes the setting\-level risk profile\.Table 12\.Aggregate learned\-policy means by setting\. Each row averages PPO, SAC, and DQN over the same 270 rollouts\. The no\-physics settings remain substantially higher in artificial liquidity even when LLM governance is active\.Table 13\.Detailed LLM governance operation rates\. Each row aggregates 2,160 agentic log steps from three seeds\. The LLM failure count is zero for every row and is therefore not shown\.Table 14\.Detailed LLM post\-hoc trigger diagnostics and mean Planner thresholds\. Trigger columns report the fraction of logged steps satisfying each trigger condition\.Table 15\.Paired episode\-level effects of adding the LLM Planner/Auditor layer\. Deltas are computed as RL\+LLM minus raw RL under matched seed, policy, and episode\.Table 16\.Step\-level action distributions for learned policies\. Each cell reports mean with 5th–95th percentile range in parentheses\.