Calibration-First Reward-Component Auditing for Reinforcement Learning Control in Smart Greenhouses

# Calibration-First Reward-Component Auditing for Reinforcement Learning Control in Smart Greenhouses
Source: [https://arxiv.org/html/2607.11959](https://arxiv.org/html/2607.11959)
Guowei XuYaojun Wang[wangyaojun@cau\.edu\.cn](https://arxiv.org/html/2607.11959v1/mailto:[email protected])College of Information and Electrical Engineering, China Agricultural University, Beijing, China

###### Abstract

Greenhouse reinforcement learning can test climate\-control ideas at a speed and scale that is difficult to achieve with crop experiments alone\. For smart\-greenhouse control, however, a single simulator return is not enough: a grower or control engineer also needs to know when the policy heats, enriches CO2, vents, manages humidity, deploys screens, or uses lamps\.

We propose a reproducible calibration\-first reward audit framework that keeps named greenhouse\-control reward components comparable across simulator training, facility\-adapted rollouts, logged Autonomous Greenhouse Challenge records, and actuator\-rule distillation\. In GreenLight\-Gym, the framework decomposes the scalar reward into conditional temperature, CO2, humidity and vapor\-pressure\-deficit, screen, and actuation\-proxy terms; adapts GreenLight to the second Autonomous Greenhouse Challenge logged climate traces; and scores the same components on logged greenhouse data\.

The evidence comes from three validation layers\. Component rewards did not break PPO learnability: nine 500k\-step runs remained above the rule\-controller floor\. In AGC2 blocked validation, the facility\-adapted GreenLight candidate improved state\-rollout accuracy across four unique team/window traces represented by seven preprocessing validation contexts\. Mean root\-mean\-square error decreased by0\.184∘0\.184^\{\\circ\}C for temperature, 4\.3 pp for relative humidity, and 563 ppm for CO2, with no context\-level temperature degradation\. Reward\-component distances to real\-log values decreased at both timestep and daily\-sum scales\. The clearest simulator\-side behavioral result came from shallow\-tree actuator distillation: thermal\-screen, lamp, and blackout\-screen actions were more rule\-approximable under the physics\-prior reward, withR2R^\{2\}increasing from 0\.652 to 0\.835, from 0\.631 to 0\.838, and from 0\.592 to 0\.750\.

Logged\-data scoring further localized crop\- and channel\-specific alignment signals, including a positive but underpowered AGC1 cucumber rank correlation,ρ=0\.829\\rho=0\.829with exact permutationp=0\.058p=0\.058, and calibration targets in AGC2 and AGC4\. Together, these results support reward\-component auditing as a pre\-deployment diagnostic layer for greenhouse RL, not as evidence of field\-level yield improvement\. The framework reports how a learned policy earns reward, where simulator calibration changes the conclusion, and which crop or actuator channels should be checked before facility\-specific controller trials\.

###### keywords:

Smart greenhouse , Reinforcement learning , GreenLight , Simulator calibration , Reward diagnostics , Controlled environment agriculture , Explainable control

††journal:Smart Agricultural Technology## 1Introduction

Greenhouse climate control is more than keeping one variable near one setpoint\. Heating, ventilation, CO2dosing, screens, lighting, crop response, and operating cost all act through the same climate system\. Classical optimal\-control and model\-based\-control studies make this coupling explicit by specifying the greenhouse and crop model before optimizing operation\[[Bakker1995GreenhouseClimate,](https://arxiv.org/html/2607.11959#bib.bib1),[VanStraten2010OptimalControl,](https://arxiv.org/html/2607.11959#bib.bib2),[Jones1991TOMGRO,](https://arxiv.org/html/2607.11959#bib.bib3),[Vanthoor2011GreenhouseClimateModel,](https://arxiv.org/html/2607.11959#bib.bib4)\]\. Energy\-aware control depends on the same modelling choices: conclusions change with the representation of heat exchange, humidity, CO2, crop growth, and weather forcing\[[Iddio2020GreenhouseEnergyReview,](https://arxiv.org/html/2607.11959#bib.bib5)\]\. Reinforcement learning \(RL\) is attractive because simulation can generate many policy rollouts without putting a crop at experimental risk\. Recent greenhouse studies have therefore compared RL with model predictive control \(MPC\), combined RL and MPC, tested robust or model\-based variants, and released benchmark environments for controller development\[[An2021SimulatorPlanning,](https://arxiv.org/html/2607.11959#bib.bib6),[Zhang2021RobustMBRL,](https://arxiv.org/html/2607.11959#bib.bib7),[Morcego2023RLMPC,](https://arxiv.org/html/2607.11959#bib.bib8),[Ajagekar2023RobustDRL,](https://arxiv.org/html/2607.11959#bib.bib9),[Mallick2024RLMPC,](https://arxiv.org/html/2607.11959#bib.bib10),[Mansour2025AdaptiveRobustGreenhouse,](https://arxiv.org/html/2607.11959#bib.bib11),[GreenLightGym,](https://arxiv.org/html/2607.11959#bib.bib12)\]\.

This coupling also makes a high simulator return difficult to interpret\. A grower or control engineer does not only ask whether the return increased; they ask what the controller did to obtain it\. Heating, screen closure, CO2enrichment, ventilation, humidity control, and supplemental lighting have different physical meanings and different failure modes\. CO2enrichment, for example, depends on light and crop state, and high light under unsuitable temperature can create stress rather than useful assimilation\[[Mortensen1987CO2,](https://arxiv.org/html/2607.11959#bib.bib13),[Pan2019CO2LightTomato,](https://arxiv.org/html/2607.11959#bib.bib14),[Niu2023LightSpectraTomato,](https://arxiv.org/html/2607.11959#bib.bib15),[Lu2017PhotoinhibitionTomato,](https://arxiv.org/html/2607.11959#bib.bib16)\]\. Once these effects are collapsed into a single profit\-like reward, a small return gain may reflect a physiological response, a shifted setpoint, a screen\-use shortcut, a favorable weather episode, or a scaling artifact\. This paper therefore asks a practical reporting question: can the reward terms used for simulator training remain interpretable and comparable when they are checked against calibrated simulator rollouts and logged greenhouse records?

Process\-based greenhouse simulators such as GreenLight\[[Katzin2020GreenLight,](https://arxiv.org/html/2607.11959#bib.bib17)\]provide a useful starting point because they expose climate states and actuator channels\. GreenLight\-Gym\[[GreenLightGym,](https://arxiv.org/html/2607.11959#bib.bib12)\]wraps GreenLight as an OpenAI Gym environment with six controls: heating, CO2, thermal screen, ventilation, lighting, and blackout screen\. Its default reward is still a scalar that combines growth revenue and energy cost, so it does not show which physiological or actuator terms drive the policy\. Reward shaping can make learning easier, but it can also change what the agent optimizes unless the shaping assumptions remain visible\[[Ng1999RewardShaping,](https://arxiv.org/html/2607.11959#bib.bib18),[Wiewiora2003Potential,](https://arxiv.org/html/2607.11959#bib.bib19),[Devlin2012DynamicPotential,](https://arxiv.org/html/2607.11959#bib.bib20),[Schulman2017PPO,](https://arxiv.org/html/2607.11959#bib.bib21)\]\.

### 1\.1Related work and gap

Greenhouse RL has begun to move from proof\-of\-concept policy learning toward benchmarked comparisons with MPC and robust optimization\. GreenLight\-Gym provides fixed weather inputs, an actuator interface, and a rule\-controller baseline for simulator studies\[[GreenLightGym,](https://arxiv.org/html/2607.11959#bib.bib12)\]\. Recent work also makes clear that nominal return is not enough\. A controller must respect climate constraints, remain usable under model mismatch, and be interpretable enough for growers to inspect or adjust\[[Morcego2023RLMPC,](https://arxiv.org/html/2607.11959#bib.bib8),[Ajagekar2023RobustDRL,](https://arxiv.org/html/2607.11959#bib.bib9),[Xiao2026GrowerLoop,](https://arxiv.org/html/2607.11959#bib.bib22)\]\. These expectations are familiar in model\-predictive control, where constraints and forecasts are explicit, and in safe RL, where deployment limits are part of the learning problem\[[VanStraten2010OptimalControl,](https://arxiv.org/html/2607.11959#bib.bib2),[Garcia2015SafeRL,](https://arxiv.org/html/2607.11959#bib.bib23)\]\.

Two prior lines of work motivate treating reward design as an audit target\. The first is sim\-to\-real caution\. Robotics and control studies often use simulation for safe data generation, but policies can fail when simulator dynamics, actuation, sensing, or aggregation differ from the target system\[[Tobin2017DomainRandomization,](https://arxiv.org/html/2607.11959#bib.bib24),[Salvato2021RealityGap,](https://arxiv.org/html/2607.11959#bib.bib25),[Muratore2022RandomizedSimReview,](https://arxiv.org/html/2607.11959#bib.bib26)\]\. The same issue appears in greenhouse simulation: a reward component computed every 15 minutes in simulation cannot be assumed comparable with a daily aggregate score from logged greenhouse data\. The second line is explainable RL\. Policy extraction, decision\-tree distillation, and broader XRL taxonomies provide tools for inspecting learned behavior without treating the neural policy itself as transparent\[[Bastani2018VIPER,](https://arxiv.org/html/2607.11959#bib.bib27),[Puiutta2020XRLSurvey,](https://arxiv.org/html/2607.11959#bib.bib28),[Bekkemoen2024XRL,](https://arxiv.org/html/2607.11959#bib.bib29)\]\. In agricultural modelling, explainable AI is also used to show which measured variables support a prediction rather than treating black\-box accuracy as sufficient\[[Ryo2022AgXAI,](https://arxiv.org/html/2607.11959#bib.bib30)\]\. In greenhouse control, this leads to a channel\-wise question: can the learned screen, lamp, CO2, or ventilation action be approximated by a shallow rule under the same simulator distribution?

Reward shaping can encode domain knowledge, but the shaping assumptions also need to remain inspectable\. Potential\-based shaping preserves policy invariance under clean conditions\[[Ng1999RewardShaping,](https://arxiv.org/html/2607.11959#bib.bib18),[Devlin2012DynamicPotential,](https://arxiv.org/html/2607.11959#bib.bib20)\]\. Greenhouse reward terms rarely meet such conditions because growth, weather, climate comfort, and actuator costs are coupled\. Decomposed rewards can therefore serve a diagnostic role: physiological terms can be inspected individually, and the resulting actions can be checked for shallow\-rule approximability\[[Garcia2015SafeRL,](https://arxiv.org/html/2607.11959#bib.bib23),[Juozapaitis2019RewardDecomposition,](https://arxiv.org/html/2607.11959#bib.bib31),[Lin2019RewardDecomposition,](https://arxiv.org/html/2607.11959#bib.bib32)\]\.

The missing step is a practical reward check\. Existing greenhouse RL studies often report aggregate simulator return, but rarely show whether reward terms have consistent semantics across simulator rollouts and logged greenhouse records, whether an energy term is a measured quantity or an action proxy, or whether a small return margin can be traced to recognizable actuator behavior\. This study uses the same named components for three connected tasks: training a GreenLight\-Gym policy, scoring AGC greenhouse logs, and inspecting actuator behavior with shallow rules\. The audit asks three bounded questions: whether the reward remains learnable, whether action changes occur in interpretable regimes, and whether logged data point to crop\-specific calibration targets\.

### 1\.2Contribution and scope

This study contributes a calibration\-first reward\-component audit protocol for greenhouse RL, designed to test whether a reward remains learnable, comparable across simulator and logged records, and interpretable at actuator level before controller\-transfer claims are made\.

The framework has three steps \(Fig\.[1](https://arxiv.org/html/2607.11959#S1.F1)\)\. First, the scalar GreenLight\-Gym objective is decomposed into named components for temperature, CO2, humidity/VPD, screen use, and weather\-normalized actuation\. Second, reward interpretation is gated by adapting GreenLight to AGC2 logged climate traces before simulator rollouts are used as evidence\. Third, the same components are recomputed on AGC1, AGC2, and AGC4 greenhouse logs\[[Hemming2019AGC1,](https://arxiv.org/html/2607.11959#bib.bib33),[Hemming2020AGC2,](https://arxiv.org/html/2607.11959#bib.bib34),[Maree2025AGC4,](https://arxiv.org/html/2607.11959#bib.bib35)\], and learned actuator channels are distilled into shallow rules so screen, lamp, heating, ventilation, and CO2behavior can be inspected directly rather than inferred from scalar return alone\.

![Refer to caption](https://arxiv.org/html/2607.11959v1/figures/figure1_diagnostic_workflow.png)Figure 1:Calibration\-first reward\-audit workflow used in this study\. A conditional component reward is retained both as a physics\-prior training signal and as a record of component scores\. Six checks are reported before reward claims are made: simulator learnability, time\-conditioned behavior, real\-log component scoring, component comparability, action\-proxy tradeoffs, and rule fidelity\.

## 2Materials and methods

### 2\.1Simulator environment and action interface

Experiments use GreenLight\-Gym, a reinforcement\-learning benchmark built around the GreenLight process model\. GreenLight represents greenhouse climate, crop growth, lighting, heat exchange, and weather forcing\. GreenLight\-Gym exposes these dynamics through state observations, a continuous actuator vector, and the official rule\-based controller distributed with the benchmark\. The action vector contains six normalized controls: pipe heating, CO2dosage, thermal screen, ventilation, supplemental lighting, and blackout screen\. Because these controls are normalized simulator actions, action\-cost analyses are reported as actuation\-proxy diagnostics rather than as physical resource accounting\.

The simulator is used here as a controlled test bed for reward design and calibration analysis\. Process\-based models are useful because they expose state variables and actuator channels, but their numerical outputs still depend on equations, timestep resolution, weather forcing, and unit conventions\. The analysis keeps three quantities separate: native simulator reward, component reward terms used for inspection, and physical greenhouse resource use\.

The baseline controller is the official rule\-based policy distributed with the benchmark\. A learned policy must clear this floor before its component behavior is interpreted\. This requirement keeps the analysis tied to a policy that is both learnable and inspectable, rather than to traces that are easy to explain but poor controllers\. Evaluation summaries report native simulator reward together with component contributions, radiation\-conditioned actions, and rule\-fidelity scores\.

### 2\.2Conditional component reward

The training reward keeps the native GreenLight\-Gym reward visible while adding a diagnostic physics\-prior term:

rt=0\.7​rtnative\+0\.3​rtdiag,rtdiag=∑k∈𝒦wk​rt,k\.r\_\{t\}=0\.7r\_\{t\}^\{\\mathrm\{native\}\}\+0\.3r\_\{t\}^\{\\mathrm\{diag\}\},\\qquad r\_\{t\}^\{\\mathrm\{diag\}\}=\\sum\_\{k\\in\\mathcal\{K\}\}w\_\{k\}r\_\{t,k\}\.\(1\)Herettindexes the 15\-min simulator timestep,rtnativer\_\{t\}^\{\\mathrm\{native\}\}is the original GreenLight\-Gym scalar reward,rtdiagr\_\{t\}^\{\\mathrm\{diag\}\}is the diagnostic physics\-prior term,𝒦\\mathcal\{K\}is the set of named reward components,rt,kr\_\{t,k\}is the timestep score for componentkk, andwkw\_\{k\}is its component weight\. The 0\.7/0\.3 mixture was used as a fixed audit\-oriented design choice to keep the native GreenLight\-Gym objective dominant while exposing diagnostic component traces; it was not tuned against AGC outcomes\. The mixture is not proposed as an optimal greenhouse\-control reward\. Sensitivity to actuation\-proxy weighting is reported in Additional file 1, and broader mixture optimization is outside the present audit claim\. All component weights were fixed atwk=1w\_\{k\}=1unless explicitly varied in the actuation\-proxy sweep\. No component weight was fitted to AGC yield, AGC team ranking, or any logged greenhouse outcome\.

The diagnostic term records seven components: temperature comfort, temperature direction, CO2direction, humidity/VPD, light, screen, and actuation proxy \(Table[1](https://arxiv.org/html/2607.11959#S2.T1)\)\. Daytime in simulator rollouts is defined as outside radiation\>50\>50W m\-2\. VPD is computed as

VPD​\(T,R​H\)=0\.6108​exp⁡\(17\.27​TT\+237\.3\)​\(1−R​H/100\),\\mathrm\{VPD\}\(T,RH\)=0\.6108\\exp\\left\(\\frac\{17\.27T\}\{T\+237\.3\}\\right\)\(1\-RH/100\),\(2\)withTTin∘C,R​HRHin percent, and VPD in kPa\.

Table 1:Reward\-component audit map used for simulator training and AGC\-log scoring\. The table keeps the main text focused on component purpose and gating logic; exact timestep formulas, weights, and daily proxy formulas are listed in Additional file 1\.
### 2\.3Reward\-component implementation

The component reward is implemented as a wrapper on top of the scalar GreenLight\-Gym reward\. At each timestep, component scores are computed from the simulator state, weather context, and normalized action vector\. Their sum enters the training reward, while the individual component values are retained for ablation, one\-out ranking, and comparability checks\.

For AGC logs, the same component names are recomputed from daily climate summaries and actuator proxies\. Cucumber uses24\.0/19\.0∘24\.0/19\.0^\{\\circ\}C day/night setpoints and a radiation threshold of 3 in the daily CO2gate; tomato and cherry tomato use21\.5/18\.5∘21\.5/18\.5^\{\\circ\}C and threshold 5\. When direct physical flow is unavailable, logged heating and CO2values are treated as setpoint\-derived proxies, allowing the same reward map to locate calibration targets in public greenhouse records\.

### 2\.4Training protocol

Policies are trained with Proximal Policy Optimization \(PPO\)\[[Schulman2017PPO,](https://arxiv.org/html/2607.11959#bib.bib21)\]\. The official rule\-based controller provides the base action, and the neural policy learns a bounded residual \(±0\.15\\pm 0\.15scale\)\. VecNormalize is applied to observations and rewards\. Training uses three start days \(120, 180, 240\) and three evaluation seeds \(42, 123, 456\)\. The start days evaluate the controller across different seasonal radiation and temperature regimes rather than tuning it for a single weather episode\. Evaluation rollouts are summarized by native reward, action traces, and component scores\.

The rule\-controller floor is the official GreenLight\-Gym rule\-based policy executed in the same simulator, weather, start\-day, and evaluation setting as the learned policies\. It is retained as a sanity check: a learned policy must beat this floor before component behavior is interpreted\. Native reward is used for learnability and floor\-clearing, whereas component terms are used for inspection\. This separation matters because component traces can be informative even when the native\-reward margin is small\.

### 2\.5Configurations

Three configurations are evaluated\. The tomato component\-reward wrapper uses a21\.5/18\.5∘21\.5/18\.5^\{\\circ\}C day/night set\-point pair, the cucumber component\-reward wrapper uses a24\.0/19\.0∘24\.0/19\.0^\{\\circ\}C pair, and the scalar\-reward tomato baseline uses the same tomato temperature pair\.

### 2\.6Diagnostic analyses

The evaluation uses six reward\-audit checks instead of one performance score\. First, each reward wrapper must clear the rule\-controller floor after 500k training steps\. Second, rollout actions are stratified by hour and solar\-radiation bin to test whether the gated CO2term changes behavior in a physiologically plausible regime\. Third, one\-out component ablation checks whether any single component dominates the normalized reward scale; the full ranking is reported in Additional file 1, the supplementary PDF accompanying this manuscript\.

The remaining checks ask whether the reward can still be read after learning and across domains\. A reward–actuation\-proxy sweep evaluates tradeoffs in normalized action space, where the horizontal axis is a simulator action proxy rather than measured energy\. Policy distillation fits shallow decision trees to each actuator channel\[[Bastani2018VIPER,](https://arxiv.org/html/2607.11959#bib.bib27),[Puiutta2020XRLSurvey,](https://arxiv.org/html/2607.11959#bib.bib28),[Bekkemoen2024XRL,](https://arxiv.org/html/2607.11959#bib.bib29)\]\. Finally, the same named components are scored on logged AGC greenhouse data\[[Hemming2019AGC1,](https://arxiv.org/html/2607.11959#bib.bib33),[Hemming2020AGC2,](https://arxiv.org/html/2607.11959#bib.bib34),[Maree2025AGC4,](https://arxiv.org/html/2607.11959#bib.bib35)\]and compared with team\-level yield rankings using Spearman correlation\. Before that cross\-domain comparison, a component\-comparability audit checks aggregation, units, and semantics\.

For the calibration\-first branch, AGC2 logged weather, actuator proxies, and indoor climate targets are converted into compact replay tables with blocked validation windows\. Candidate GreenLight parameter sets are evaluated by replaying logged actions and comparing simulated and measuredTairT\_\{\\mathrm\{air\}\}, relative humidity, and CO2\. A candidate passes the quality gate if it improves the blocked validation objective across action\-supported contexts without degrading validation temperature RMSE by more than1∘1^\{\\circ\}C\. Only after this gate is passed are reward components recomputed on the default simulator, the candidate simulator, and the corresponding real\-log rows at matched aggregation scales\.

The AGC2 calibration data contain 47,809 weather rows and 23,910 action/target rows in the compact hourly contract\. The raw time\-block split assigns the first 80% of each team trace to calibration and the last 20% to validation, but the C3c search uses stricter action\-supported 3\-day replay windows\. Within each selected 3\-day window, the first half is used for candidate scoring and the second half for blocked validation\. Both imputed and boiler/CO2\-complete action policies are considered; windows must retain at least 75% of expected rows and sufficient boiler/CO2support\. The selected candidate is one shared GreenLight parameter set across all selected AGC2 contexts, not a separate per\-team fit\.

The validation objective is the sum of normalized RMSE values for air temperature, relative humidity, and CO2, using scale factors of5∘5^\{\\circ\}C, 20 percentage points, and 500 ppm, respectively:

J=RMSE​\(Tair\)5\+RMSE​\(R​Hair\)20\+RMSE​\(C​O​2air\)500\.J=\\frac\{\\mathrm\{RMSE\}\(T\_\{\\mathrm\{air\}\}\)\}\{5\}\+\\frac\{\\mathrm\{RMSE\}\(RH\_\{\\mathrm\{air\}\}\)\}\{20\}\+\\frac\{\\mathrm\{RMSE\}\(CO2\_\{\\mathrm\{air\}\}\)\}\{500\}\.\(3\)The local search budget was 24 candidates with seed 41, initialized around a C3b warm start and filtered by the1∘1^\{\\circ\}C validation temperature\-degradation guard\. Table[2](https://arxiv.org/html/2607.11959#S2.T2)lists the searched parameter bounds and the selected C3c candidate values\.

Table 2:GreenLight parameters varied in the AGC2 calibration search\. Bounds are the local\-search ranges; the selected values are the shared candidate used for all selected AGC2 validation contexts\.
### 2\.7Reproducibility and release\-asset map

The release package separates the submission\-facing evidence from reusable code\. The main manuscript reports the diagnostics needed for the argument, Additional file 1 preserves supplementary ablation and artifact checks, and the code repository provides reproduction scripts and component definitions for reuse on new greenhouse traces\.

## 3Results

The decomposed reward first had to remain learnable under PPO\. Subsequent checks test whether selected actuator changes occur in interpretable regimes, whether component scales are comparable, and whether AGC2 calibration changes rollout accuracy\. The final checks score the same components on AGC logs, measure action\-proxy tradeoffs, and distill actuator decisions into shallow rules\.

### 3\.1The decomposed reward remains learnable under PPO

All nine 500k\-step PPO runs clear the official GreenLight\-Gym rule\-based controller floor\[[GreenLightGym,](https://arxiv.org/html/2607.11959#bib.bib12)\]\(Table[3](https://arxiv.org/html/2607.11959#S3.T3)\)\. The component reward matches or slightly exceeds the scalar baseline: tomato component169\.54±0\.78169\.54\\pm 0\.78vs scalar168\.82±0\.99168\.82\\pm 0\.99\(\+0\.72±0\.45\+0\.72\\pm 0\.45\), cucumber component169\.47±0\.42169\.47\\pm 0\.42\(\+0\.66±0\.72\+0\.66\\pm 0\.72\)\. In this training setting, the physics\-prior wrapper adds interpretable component traces without reducing native simulator reward\.

After facility adaptation, all six PPO runs in the AGC2\-adapted simulator also cleared the rule\-controller floor\. Scalar\-reward tomato achieved89\.526±0\.66389\.526\\pm 0\.663and component\-reward tomato89\.392±0\.54089\.392\\pm 0\.540\. Matched\-seed margins were mixed \(\+0\.554\+0\.554,−0\.912\-0\.912,−0\.042\-0\.042; mean−0\.133\-0\.133\), so neither reward design dominates in the adapted environment\. The adapted simulator supports stable policy training under both scalar and component rewards\.

A matched 1M\-step budget check in the adapted simulator also remained above the rule\-controller floor for all seeds and gave a small directional component\-reward margin \(mean\+0\.662\+0\.662native reward; Additional file 1\)\. This result is retained as learnability and budget\-sensitivity support rather than reward\-dominance evidence\.

Table 3:500k\-step multi\-seed native reward summary \(3 seeds each\)\.
### 3\.2CO2actions shift modestly toward radiation\-supported regimes

Time\-conditioned rollout analysis shows that the tomato component reward modestly increases day/night CO2action separation in two saved\-policy replays\. In replay A, the day/night contrast is0\.4520\.452under the component reward and0\.4370\.437under the scalar baseline \(margin\+0\.015\+0\.015\)\. In replay B, the contrast increases from0\.3780\.378to0\.4640\.464\(margin\+0\.086\+0\.086; Fig\.[2](https://arxiv.org/html/2607.11959#S3.F2)\)\. Radiation\-bin margins are less stable: the 100–200 W m\-2bin is positive in replay A \(\+0\.067\+0\.067\) but slightly negative in replay B \(−0\.014\-0\.014\), even though this is the regime where the gated CO2term is physiologically motivated\[[Mortensen1987CO2,](https://arxiv.org/html/2607.11959#bib.bib13)\]\. The two replays correspond to training seeds 42 and 456 for reproducibility\. The supported statement is therefore narrow: the component reward modestly strengthens day/night CO2separation in the available replays, while radiation\-bin differences remain a simulator\-side diagnostic rather than a stable mechanism claim\.

![Refer to caption](https://arxiv.org/html/2607.11959v1/x1.png)Figure 2:Two\-replay CO2action diagnostic\. The component reward increases day/night CO2action contrast in both saved\-policy replays \(dayn=1680n=1680, nightn=1200n=1200per replay\), but radiation\-bin margins are mixed across seeds\. Points show component\-minus\-scalar normalized simulator actions, not CO2mass\-flow measurements\.
### 3\.3Component scores require scale and semantic matching

One\-out component ablation shows that all component indices are close to zero at the normalized reward scale \(largest: temperature direction\+0\.0056\+0\.0056; see Additional file 1 for the full ranking table\)\. The greenhouse reward operates as an interacting system in which temperature, radiation, CO2, humidity, screen, and cost terms shape actuator behavior jointly, consistent with the coupled physics of greenhouse climate control\.

Before cross\-domain comparison, simulator and AGC component scores were matched for aggregation scale\. The simulator sums 96 timestep\-level component values per day, whereas AGC logs yield one daily row\. After scale matching, the apparent temperature\-comfort gap fell from approximately five reward units to0\.070\.07–0\.100\.10units\. The remaining CO2\-direction difference reflects a semantic gap between a timestep radiation gate and a daily proxy\. Additional file 1 gives the full comparability audit table and figure\.

### 3\.4AGC2 calibration improves state rollout and component comparability

The facility\-adapted GreenLight candidate improved state\-rollout accuracy across four unique AGC2 team/window traces represented by seven preprocessing validation contexts \(Table[4](https://arxiv.org/html/2607.11959#S3.T4)and Fig\.[3](https://arxiv.org/html/2607.11959#S3.F3)\)111Seven contexts arise from four AGC2 teams×\\timestwo preprocessing variants \(impute and exclude\)\. For three of the four teams, both variants produce identical action tables, so the seven count reflects preprocessing completeness rather than seven fully independent facility traces\., with no context exceeding the1∘1^\{\\circ\}C temperature\-degradation guard\. Mean paired improvements over default GreenLight\[[Katzin2020GreenLight,](https://arxiv.org/html/2607.11959#bib.bib17)\]were\+0\.184∘\+0\.184^\{\\circ\}C in temperature RMSE,\+4\.274\+4\.274pp in relative\-humidity RMSE, and\+562\.529\+562\.529ppm in CO2RMSE across the seven preprocessing rows \(positive = lower error\)\.

Table 4:AGC2 blocked\-validation state\-rollout deltas for the selected shared candidate\. Positive values mean the candidate simulator is closer to logged AGC2 indoor climate than default GreenLight; negative temperature deltas indicate small temperature\-RMSE worsening but remain within the1∘1^\{\\circ\}C guard\.![Refer to caption](https://arxiv.org/html/2607.11959v1/x2.png)Figure 3:Calibration before/after comparison on the Automatoes AGC2 validation window\. Default GreenLight \(dashed red\) diverges substantially from logged indoor climate \(solid black\), especially in relative humidity and CO2\. The facility\-adapted candidate \(blue\) improves the validation objective and substantially reduces relative\-humidity and CO2error, while temperature remains within the predefined no\-worse\-than\-1∘1^\{\\circ\}C guard\.Reward\-component comparability also improved after adaptation\. Mean Wasserstein\-1 distance between real\-log and simulator component distributions decreased from 0\.01677 to 0\.01389 at the timestep scale and from 1\.19748 to 0\.97708 at the daily\-sum scale\. The aggregate component improved across all paired validation contexts, though the row\-level improvement fraction was 62/112 \(55\.4%\), reflecting uneven gains across individual component channels\.

### 3\.5Logged AGC scores identify crop\- and channel\-specific alignment

Computing the same physics\-prior components on logged AGC greenhouse data\[[Hemming2019AGC1,](https://arxiv.org/html/2607.11959#bib.bib33),[Hemming2020AGC2,](https://arxiv.org/html/2607.11959#bib.bib34),[Maree2025AGC4,](https://arxiv.org/html/2607.11959#bib.bib35)\]links the simulator reward design to observed production records \(Fig\.[4](https://arxiv.org/html/2607.11959#S3.F4)\)\. On AGC1 cucumber, the conditional composite prior gives a positive but underpowered team\-level rank correlation \(ρ=0\.829\\rho=0\.829; asymptoticp=0\.042p=0\.042; exact permutationp=0\.058p=0\.058; six teams\), improving over an earlier physics\-prior variant byΔ​ρ=\+0\.172\\Delta\\rho=\+0\.172\. Across all three AGC datasets \(18 team\-level points\), within\-dataset standardized scores give a pooled correlation ofρ=0\.468\\rho=0\.468\(asymptoticp=0\.050p=0\.050; Monte Carlo permutationp=0\.053p=0\.053\)\.

The alignment is crop\- and channel\-specific\. At the composite level, AGC2 cherry tomato \(ρ=0\.200\\rho=0\.200\) and AGC4 dwarf tomato \(ρ=0\.029\\rho=0\.029\) are weak\. Individual component channels are more informative: AGC2 CO2direction \(ρ=0\.657\\rho=0\.657\), AGC2 actuation proxy \(ρ=0\.771\\rho=0\.771\), and AGC4 CO2direction \(ρ=0\.829\\rho=0\.829; exact permutationp=0\.058p=0\.058\)\. Bootstrap confidence intervals and permutation checks reflect the small team\-level sample sizes \(AGC1 composite 95% CI \[0\.000, 1\.000\]; pooled CI\[−0\.016,0\.858\]\[\-0\.016,0\.858\]; BH\-FDRq=0\.101q=0\.101for the strongest signals\)\. These correlations are used to locate likely calibration targets, not to validate the reward as an agronomic objective\.

![Refer to caption](https://arxiv.org/html/2607.11959v1/x3.png)Figure 4:AGC logged\-data bridge for the physics\-prior reward\. \(A\) The composite prior gives a positive but underpowered AGC1 cucumber rank correlation and a weak pooled signal across AGC1/2/4 after within\-dataset standardization\. \(B\) The conditional prior improves over an earlier physics\-prior variant on AGC1 and AGC4 but not AGC2\. \(C\) Component\-level correlations are crop\- and channel\-specific, mapping where the prior aligns with production data and where crop\-specific calibration is needed\.
### 3\.6Actuation\-proxy tradeoffs remain action\-space diagnostics

The actuation\-proxy sweep reports normalized control burden as an engineering diagnostic\. The component reward includes action penalties, and the available quantities are normalized simulator actions rather than audited resource meters\. The sweep therefore tests whether reward gains coincide with changed action magnitudes inside the simulator\. In this action space, the physics\-prior reward creates favorable reward–actuation tradeoff points relative to the scalar baseline \(Additional file 1, Table S4\)\. At actuation\-proxy weight0\.50\.5, the component reward achieved0\.56630\.5663vs scalar0\.55960\.5596, showing that the proxy can reveal whether a reward improvement is being bought through actuator shortcuts\.

### 3\.7Rule fidelity provides the strongest actuator\-level evidence

Rule\-fidelity analysis provides the clearest behavioral evidence for the physics\-prior reward\. Decision\-tree distillation\[[Bastani2018VIPER,](https://arxiv.org/html/2607.11959#bib.bib27)\]shows higher shallow\-rule approximability for thermal screen, lighting, and blackout screen actions under the component reward than under the scalar baseline \(Table[5](https://arxiv.org/html/2607.11959#S3.T5); Fig\.[5](https://arxiv.org/html/2607.11959#S3.F5)\)\. Thermal\-screenR2R^\{2\}rises from 0\.652 to 0\.835, lamp from 0\.631 to 0\.838, and blackout screen from 0\.592 to 0\.750\. These are actuator channels where growers expect rules tied to radiation, temperature, and weather\. In this setting, the physics\-prior reward makes the main actuator decisions easier to summarize as shallow greenhouse rules\.

In the facility\-adapted simulator, both reward designs produce highly rule\-approximable policies: mean shallow\-treeR2R^\{2\}across six actuators was 0\.945 \(scalar\) and 0\.943 \(component\), with random\-forestR2R^\{2\}at 0\.951 and 0\.956\. The convergence in rule\-fidelity after facility adaptation indicates that the adapted GreenLight dynamics constrain actuator behavior toward physically structured rules regardless of the training reward\. Channel\-level differences between reward designs are small and mixed in the adapted environment\.

A depth\-sensitivity replay on the same three\-seed calibrated traces supports the same interpretation \(Additional file 1\)\. Using local CART surrogates with minimum leaf size 50, the mean actuatorR2R^\{2\}across six channels was already about 0\.92 at depth 2 and increased smoothly to about 0\.95 at depth 5 for both scalar and component rewards\. The calibrated\-domain rule\-fidelity result is therefore not an artifact of one depth\-4 tree choice; it also shows that much of the post\-calibration interpretability gain comes from the adapted simulator distribution shared by both reward designs\.

Table 5:Depth\-4 decision\-tree rule fidelity for actuator channels\. HigherR2R^\{2\}means the simulator policy action is more closely approximated by a shallow rule under the same rollout distribution\.![Refer to caption](https://arxiv.org/html/2607.11959v1/x4.png)Figure 5:Actuator\-level change in shallow\-tree rule fidelity under the physics\-prior component reward relative to the scalar baseline\. Positive values mean that the same action channel is more closely approximated by a depth\-4 decision tree under the same rollout distribution\. The largest gains occur for thermal\-screen, lamp, and blackout\-screen channels, indicating which actuator behaviors are most amenable to simulator\-side rule inspection before facility\-specific validation\.Additional file 1 retains the full component ablation, pairwise interaction, actuation\-proxy sweep, policy\-distillation, and component\-comparability tables\.

## 4Discussion

The central contribution is a reward\-component audit for greenhouse RL\. Rather than relying on a single simulator return, the study follows the same named components through policy learning, AGC2 calibration, AGC log scoring, and shallow\-rule actuator analysis\. This gives greenhouse RL studies a practical way to report how reward is earned, which parts of the simulator need calibration, and which actuator channels are suitable for rule\-based inspection\.

The results separate protocol validation, simulator\-side behavior, and logged\-data alignment\. Component rewards remained trainable, AGC2 calibration improved blocked state rollout and reward\-component comparability, and default\-simulator screen and lighting actions became easier to approximate with shallow rules\. The AGC yield\-ranking result serves as calibration\-target evidence: useful for deciding what to check next, but too small to validate the reward as an agronomic objective\.

### 4\.1Why AGC log scoring matters

Scoring the same reward components on AGC logs moves the reward check outside the simulator\. It allows a candidate RL reward to be compared with observed crop\-management records before it is adopted as a training objective\. On AGC1 cucumber\[[Hemming2019AGC1,](https://arxiv.org/html/2607.11959#bib.bib33)\], the crop\-adjusted composite prior gives a positive but underpowered team\-level rank correlation \(ρ=0\.829\\rho=0\.829\), with bootstrap intervals, exact permutation testing, and BH\-FDR correction limiting the strength of this six\-team result\. AGC2 and AGC4 show crop\- and channel\-specific alignment\. CO2\-direction channels are informative in tomato logs even when the composite score is weak\.

AGC1, AGC2, and AGC4 data are used here for retrospective reward\-component scoring and calibration\-target identification\[[Hemming2019AGC1,](https://arxiv.org/html/2607.11959#bib.bib33),[Hemming2020AGC2,](https://arxiv.org/html/2607.11959#bib.bib34),[Maree2025AGC4,](https://arxiv.org/html/2607.11959#bib.bib35)\]\. This use is valuable because it shows which reward terms can be evaluated on public greenhouse records and which crop\-specific channels need improved calibration or measurement support\.

### 4\.2Interpretability versus transfer

Rule fidelity and sim\-to\-real transfer are different claims\. The comparability check ensures that simulator and real\-log component scores share aggregation, units, and semantics before cross\-domain conclusions are drawn, following standard sim\-to\-real caution\[[Tobin2017DomainRandomization,](https://arxiv.org/html/2607.11959#bib.bib24),[Salvato2021RealityGap,](https://arxiv.org/html/2607.11959#bib.bib25),[Muratore2022RandomizedSimReview,](https://arxiv.org/html/2607.11959#bib.bib26)\]\. The AGC2 facility\-adaptation branch showed that a candidate parameter set can improve both state rollout and aggregate reward\-component comparability, while the method also identifies which components still require calibration\.

The convergence of rule\-fidelity scores after facility adaptation \(R2≈0\.94R^\{2\}\\approx 0\.94for both reward designs\) is informative\. In the default simulator, the physics\-prior reward produces more rule\-approximable actuator behavior\. After adaptation narrows the simulator–facility gap, the adapted dynamics themselves constrain both reward designs toward similar actuator structure\. This suggests that the physics\-prior reward is most valuable when simulator–facility mismatch is largest\.

### 4\.3Implications for greenhouse RL reporting

The results argue for reporting greenhouse RL policies with more than aggregate simulator reward\. In this study, the useful reporting items are the rule\-controller floor, the distinction between physical quantities and action proxies, shallow\-rule summaries of actuator channels, and the ability to score the same reward components on logged greenhouse data\. This reporting style connects RL studies back to the older greenhouse\-control literature, where classical optimal\-control studies made modelling assumptions explicit\[[Bakker1995GreenhouseClimate,](https://arxiv.org/html/2607.11959#bib.bib1),[VanStraten2010OptimalControl,](https://arxiv.org/html/2607.11959#bib.bib2),[Vanthoor2011GreenhouseClimateModel,](https://arxiv.org/html/2607.11959#bib.bib4)\]\. The reward wrapper provides a practical link between agronomic knowledge and RL evaluation: the same named terms train a simulator policy, score production data, identify calibration targets, and summarize actuator behavior for engineering inspection\.

### 4\.4Interpretation limits

Three interpretations remain outside the current evidence\. The CO2diagnostic strengthens day/night separation in two saved\-policy replays, but mixed radiation\-bin margins do not support a stable CO2mechanism claim\. Scoring AGC logs identifies crop\-specific calibration targets, but the small team\-level samples and permutation/FDR results do not validate the reward as an agronomic objective\. The AGC2\-adapted PPO runs clear the rule\-controller floor under both scalar and component rewards, but their mixed 500k margins do not support component\-reward dominance\.

### 4\.5Future calibration path

A stronger sim\-to\-real study would need matched aggregation on both sides of the comparison and a simulator model that remains stable after adaptation\[[Salvato2021RealityGap,](https://arxiv.org/html/2607.11959#bib.bib25)\]\. On the simulator side, component scores can be summarized as timestep means, daily aggregates computed after first aggregating state and action variables, and season\-level summaries\. On the logged\-data side, action proxies can be replaced where possible by audited physical quantities such as heating energy, lamp energy, CO2dosage, and screen position\. The comparison can then be repeated under weather regimes shared by simulation and data, with sensitivity analyses for start date, radiation distribution, and actuator support\.

Here, GreenLight is adapted to logged AGC2 climate trajectories and the reward and policy diagnostics are repeated in the adapted candidate simulator\. With richer facility data, the same loop can support a practical deployment pathway: first calibrate the process model to logged weather, actions, and climate states; then add residual correction where the process model is biased; then use the audited reward components to select candidate policies for greenhouse trials\. A production\-first extension would treat climate states, VPD, CO2, and light as intermediate variables for yield or biomass prediction, while keeping cost and energy terms as optional diagnostics rather than headline outcomes\. A residual\-hybrid simulator is a natural next direction because it can preserve process\-model structure while learning facility\-specific climate residuals\.

### 4\.6Limitations

- 1\.All policy training and rollout results are simulator\-based\. Prospective greenhouse trials would be needed to evaluate closed\-loop operation\.
- 2\.AGC log scoring uses small team\-level samples \(6–10 teams per dataset\)\. Bootstrap confidence intervals and BH\-FDR correction bound statistical power accordingly\.
- 3\.Facility adaptation was selected and validated on AGC2 only; generalization to other facilities remains untested\.
- 4\.Several selected AGC2 calibration parameters lie at or near the local\-search bounds\. The selected vector should therefore be read as a facility\-adaptation coefficient set, not as a uniquely identifiable estimate of greenhouse equipment properties\.
- 5\.Actuation\-proxy tradeoffs reflect normalized simulator actions, not measured physical energy\.
- 6\.GreenLight\-Gym is a simplified greenhouse model; residual\-hybrid extensions require further validation before stronger transfer claims\.

## 5Conclusions

This study establishes reward\-component auditing as a reproducible intermediate step between simulator\-only greenhouse RL and facility\-specific validation\. It shows that greenhouse RL reward design can be inspected across policy learning, calibrated rollouts, AGC log scoring, and actuator\-rule distillation\. The supporting evidence is that the component reward remained trainable, AGC2 facility adaptation improved state\-rollout accuracy across four unique team/window traces represented by seven preprocessing validation contexts, and real\-log–simulator reward\-component distances decreased\. The behavioral evidence is strongest in the default simulator, where thermal\-screen, lamp, and blackout\-screen decisions became more rule\-approximable under the physics\-prior reward; after facility adaptation, both scalar and component rewards became similarly rule\-approximable, indicating that adapted dynamics dominate that comparison\. Together, these results support reward\-component auditing as a pre\-deployment diagnostic layer for greenhouse RL; field\-level yield gains and closed\-loop controller transfer require dedicated greenhouse trials\.

## Data availability

GreenLight and GreenLight\-Gym are cited as external model and benchmark resources\[[Katzin2020GreenLight,](https://arxiv.org/html/2607.11959#bib.bib17),[GreenLightGym,](https://arxiv.org/html/2607.11959#bib.bib12)\]\. The AGC logged data used for real\-log reward\-component scoring are available from the original 4TU\.ResearchData releases for AGC1\[[Hemming2019AGC1,](https://arxiv.org/html/2607.11959#bib.bib33)\], AGC2\[[Hemming2020AGC2,](https://arxiv.org/html/2607.11959#bib.bib34)\], and AGC4\[[Maree2025AGC4,](https://arxiv.org/html/2607.11959#bib.bib35)\]\. We release the reward\-audit workflow developed in this study, including reward\-component definitions, the physics\-prior reward wrapper, and multi\-component scoring and validation code\. The repository URL is[https://github\.com/shennongwm/rl\-reward\-audit](https://github.com/shennongwm/rl-reward-audit)\.

## CRediT authorship contribution statement

Yuhui Bie: Conceptualization, methodology, software, formal analysis, visualization, writing – original draft, writing – review and editing\. Guowei Xu: Data curation, validation, investigation, visualization, writing – original draft, writing – review and editing\. Yaojun Wang: Conceptualization, resources, supervision, project administration, writing – review and editing\.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper\.

## Funding

This research is supported by the Joint Fund Project of the National Natural Science Foundation of China \(U25B2019\-1\)\.

## Acknowledgments

The authors thank the GreenLight and GreenLight\-Gym developers, and the Wageningen Autonomous Greenhouse Challenge organizers and participants for making the simulation environment and logged greenhouse datasets available\.

## Declaration of generative AI and AI\-assisted technologies in the manuscript preparation process

During the preparation of this work the authors used OpenAI Codex/GPT and Anthropic Claude for language polishing, code\-checking assistance, and organization of revision notes\. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article\.

## Supplementary diagnostic tables

This file contains the detailed diagnostic evidence behind the preprint\. The tables document reward components, actuation proxies, rule fidelity, logged\-data component scoring, and component\-comparability checks used to interpret the physics\-prior reward\.

### Table S1a\. Reward\-component definitions used for training and AGC\-log scoring

### Table S1\. Ordinary 100k\-step component ablation

### Supplementary Figure S1\. 500k\-step simulator learnability

![[Uncaptioned image]](https://arxiv.org/html/2607.11959v1/x5.png)

All configurations clear the GreenLight\-Gym rule\-controller floor\. The figure documents the learnability gate used before interpreting reward components; controller ranking is addressed separately in the main text\.

### Table S2\. One\-out component ranking

### Table S3\. Top pairwise component interactions

### Table S4\. Simulator reward–actuation\-proxy tradeoff

The proxy axis is a normalized residual/action proxy\. The component\-reward weight 0\.5 point has the highest reward in this sweep, showing how the audit can track whether reward changes are accompanied by changed simulator action burden\. Physical resource accounting would require matched energy, lamp, and CO2flow measurements\.

### Table S5\. Policy\-distillation fidelity

### Table S6\. Component\-comparability audit

### Supplementary Figure S2\. Original component\-gap artifact

![[Uncaptioned image]](https://arxiv.org/html/2607.11959v1/x6.png)

This plot is retained as an audit artifact\. The original comparison mixed timestep\-summed simulator values with daily aggregate real\-log scores, so it is used to document the aggregation mismatch that motivated the component\-comparability audit\.

### Scope of supplementary diagnostics

These tables support simulator\-side reward decomposition, logged\-data component scoring, and component\-comparability auditing\. They provide detailed evidence for reward design and calibration planning, including which quantities would need matched\-scale resource accounting or prospective greenhouse trials in a later validation study\.

### Table S7\. Calibration\-first AGC2 gates

### Supplementary Figure S3\. AGC2 calibration deltas across unique contexts

![[Uncaptioned image]](https://arxiv.org/html/2607.11959v1/x7.png)

Positive deltas indicate that the facility\-adapted GreenLight candidate is closer to logged AGC2 indoor climate than the default GreenLight simulator\. The four rows are unique team/window contexts; preprocessing variants that produced identical action tables are not counted as independent facility traces\. Objective, relative\-humidity, and CO2RMSE improve in all four contexts, while temperature RMSE improves in two contexts and remains within the no\-worse\-than\-1∘1^\{\\circ\}C guard in the two boundary contexts\. This figure is included as anti\-cherry\-picking support for the AGC2 calibration gate, not as evidence of generalized facility adaptation\.

### Table S8\. Calibrated\-candidate PPO learnability

### Table S9\. 1M\-step PPO budget\-sensitivity

This matched\-seed 1M\-step check reports learnability and budget\-sensitivity in the AGC2\-adapted GreenLight\-Gym environment\. All scalar and component PPO runs clear the adapted rule\-controller floor without early termination\. The component\-reward runs are directionally ahead at 1M steps, but the margin is small and simulator\-only; stronger claims about reward dominance, policy transfer, deployment readiness, or measured resource savings require prospective evidence beyond this table\. The compact source table is provided ase2\_1m\_budget\_sensitivity\.csv\.

### Table S10\. Calibrated\-candidate rule\-fidelity summary

MetricScalar rewardComponent rewardInterpretationMean shallow\-treeR2R^\{2\}across six actuators0\.9450\.943Near\-tied distillability\.Mean random\-forestR2R^\{2\}across six actuators0\.9510\.956Slight component advantage under nonlinear surrogate\.Finite trace rows per run25922592All six formal traces complete\.Terminations / NaN / OOM / ODE collapse00Stable simulator\-side diagnostic run\.

### Table S11\. Calibrated rule\-fidelity depth sensitivity

Depth sensitivity was recomputed from the same six calibrated trace samples used for Table S10, using local CART surrogates with minimum leaf size 50 and the same feature and actuator target columns\. It is included only as a stability check for the tree\-depth choice: both reward designs remain highly rule\-approximable from depth 2 to 5, and component–scalar differences remain small and channel\-mixed after facility adaptation\. The compact source tables are provided alongside this supplement as the depth\-sensitivity summary, delta, macro, and raw CSV files\.

### Table S12\. CO2time\-conditioned seed replication

The time\-conditioned CO2replay supports a narrow simulator\-side statement: the tomato component reward increased the day/night CO2contrast in the two available saved\-policy time\-conditioned replays\. It does not support a three\-seed radiation\-bin mechanism claim\. Seed 123 is available in the start\-day action\-bin audit, but that audit explicitly bins only by evaluation start day and cannot support daylight\-gated CO2timing\. Compact source files are provided alongside this supplement as CO2seed\-replication summary and radiation\-bin CSV files\.

### Table S13\. Real\-log bridge permutation audit

The exact permutation check was added because the AGC bridge uses small team\-level samples\. It confirms that the strongest logged\-data rows are best treated as directional calibration signals\. These rows support calibration\-target discovery and component\-prior auditing; prospective greenhouse trials would be the next step for confirmatory reward validation and policy\-transfer assessment\. The complete permutation table is included in the package CSV and JSON audit files\.

### Table S14\. AGC2 selected parameters at effective search bounds

Boundary\-selected values are reported as transparency evidence for the AGC2 facility\-adaptation search\. They should be interpreted as a compact coefficient set that improves replay fidelity under the stated gate, not as uniquely identifiable greenhouse equipment estimates\. The full compact source table is provided in the SAT experiment\-closure parameter\-boundary CSV\.

Tables S7–S14 support the calibration\-first manuscript update\. They establish the supplement\-level evidence for simulator adaptation, component\-comparability, learnability, rule\-fidelity, and time\-conditioned action diagnostics, and they define the measurements needed for later prospective greenhouse\-trial evaluation\.

## References

- \[1\]J\. C\. Bakker, G\. P\. A\. Bot, H\. Challa, N\. J\. van de Braak \(Eds\.\), Greenhouse Climate Control: An Integrated Approach, Wageningen Academic Publishers, 1995\.[doi:10\.3920/978\-90\-8686\-501\-7](https://doi.org/10.3920/978-90-8686-501-7)\.
- \[2\]G\. van Straten, L\. G\. van Willigenburg, E\. J\. van Henten, R\. C\. H\. van Ooteghem, Optimal Control of Greenhouse Cultivation, CRC Press, 2010\.[doi:10\.1201/b10321](https://doi.org/10.1201/b10321)\.
- \[3\]J\. W\. Jones, E\. Dayan, L\. H\. Allen, H\. van Keulen, H\. Challa, A dynamic tomato growth and yield model \(TOMGRO\), Transactions of the ASAE 34 \(2\) \(1991\) 663–672\.[doi:10\.13031/2013\.31715](https://doi.org/10.13031/2013.31715)\.
- \[4\]B\. H\. E\. Vanthoor, P\. H\. B\. de Visser, C\. Stanghellini, E\. J\. van Henten, A methodology for model\-based greenhouse design: Part 2, description and validation of a tomato yield model, Biosystems Engineering 110 \(4\) \(2011\) 378–395\.[doi:10\.1016/j\.biosystemseng\.2011\.08\.005](https://doi.org/10.1016/j.biosystemseng.2011.08.005)\.
- \[5\]E\. Iddio, L\. Wang, Y\. Thomas, G\. McMorrow, A\. Denzer, Energy efficient operation and modeling for greenhouses: A literature review, Renewable and Sustainable Energy Reviews 117 \(2020\) 109480\.[doi:10\.1016/j\.rser\.2019\.109480](https://doi.org/10.1016/j.rser.2019.109480)\.
- \[6\]Z\. An, X\. Cao, Y\. Yao, W\. Zhang, L\. Li, Y\. Wang, S\. Guo, D\. Luo, A simulator\-based planning framework for optimizing autonomous greenhouse control strategy, in: Proceedings of the International Conference on Automated Planning and Scheduling, Vol\. 31, 2021, pp\. 436–444\.[doi:10\.1609/icaps\.v31i1\.15989](https://doi.org/10.1609/icaps.v31i1.15989)\.
- \[7\]W\. Zhang, X\. Cao, Y\. Yao, Z\. An, X\. Xiao, D\. Luo,[Robust model\-based reinforcement learning for autonomous greenhouse control](https://proceedings.mlr.press/v157/zhang21e.html), in: Proceedings of The 13th Asian Conference on Machine Learning, Vol\. 157 of Proceedings of Machine Learning Research, PMLR, 2021, pp\. 1208–1223, arXiv:2108\.11645\.[doi:10\.48550/arXiv\.2108\.11645](https://doi.org/10.48550/arXiv.2108.11645)\. URL[https://proceedings\.mlr\.press/v157/zhang21e\.html](https://proceedings.mlr.press/v157/zhang21e.html)
- \[8\]B\. Morcego, W\. Yin, S\. Boersma, E\. J\. van Henten, V\. Puig, C\. Sun, Reinforcement learning versus model predictive control on greenhouse climate control, Computers and Electronics in Agriculture 215 \(2023\) 108372\.[doi:10\.1016/j\.compag\.2023\.108372](https://doi.org/10.1016/j.compag.2023.108372)\.
- \[9\]A\. Ajagekar, N\. S\. Mattson, F\. You, Energy\-efficient ai\-based control of semi\-closed greenhouses leveraging robust optimization in deep reinforcement learning, Advances in Applied Energy 9 \(2023\) 100119\.[doi:10\.1016/j\.adapen\.2022\.100119](https://doi.org/10.1016/j.adapen.2022.100119)\.
- \[10\]S\. Mallick, F\. Airaldi, A\. Dabiri, C\. Sun, B\. De Schutter, Reinforcement learning\-based model predictive control for greenhouse climate control, arXiv preprint arXiv:2409\.12789 \(2024\)\.[doi:10\.48550/arXiv\.2409\.12789](https://doi.org/10.48550/arXiv.2409.12789)\.
- \[11\]M\. Mansour, K\. K\. Sathyanarayanan, P\. Sauerteig, S\. Streif, Adaptive robust greenhouse climate control: Combining deep reinforcement learning and economic optimization, Smart Agricultural Technology 12 \(2025\) 101327\.[doi:10\.1016/j\.atech\.2025\.101327](https://doi.org/10.1016/j.atech.2025.101327)\.
- \[12\]B\. van Laatum, E\. J\. van Henten, S\. Boersma, GreenLight\-Gym: Reinforcement learning benchmark environment for control of greenhouse production systems, in: 8th IFAC Conference on Sensing, Control and Automation Technologies for Agriculture, Vol\. 59 of IFAC\-PapersOnLine, 2025, pp\. 437–442\.[doi:10\.1016/j\.ifacol\.2025\.11\.827](https://doi.org/10.1016/j.ifacol.2025.11.827)\.
- \[13\]L\. M\. Mortensen, Review: CO2 enrichment in greenhouses\. crop responses, Scientia Horticulturae 33 \(1–2\) \(1987\) 1–25\.[doi:10\.1016/0304\-4238\(87\)90028\-8](https://doi.org/10.1016/0304-4238(87)90028-8)\.
- \[14\]T\. Pan, J\. Ding, G\. Qin, Y\. Wang, L\. Xi, J\. Yang, J\. Li, J\. Zhang, Z\. Zou, Interaction of supplementary light and CO2 enrichment improves growth, photosynthesis, yield, and quality of tomato in autumn through spring greenhouse production, HortScience 54 \(2\) \(2019\) 246–252\.[doi:10\.21273/HORTSCI13709\-18](https://doi.org/10.21273/HORTSCI13709-18)\.
- \[15\]Y\. Niu, H\. Lyu, X\. Liu, M\. Zhang, H\. Li, Photosynthesis prediction and light spectra optimization of greenhouse tomato based on response of red–blue ratio, Scientia Horticulturae 318 \(2023\) 112065\.[doi:10\.1016/j\.scienta\.2023\.112065](https://doi.org/10.1016/j.scienta.2023.112065)\.
- \[16\]T\. Lu, Z\. Meng, G\. Zhang, M\. Qi, Z\. Sun, Y\. Liu, T\. Li, Sub\-high temperature and high light intensity induced irreversible inhibition on photosynthesis system of tomato plant \(Solanum lycopersicum l\.\), Frontiers in Plant Science 8 \(2017\) 365\.[doi:10\.3389/fpls\.2017\.00365](https://doi.org/10.3389/fpls.2017.00365)\.
- \[17\]D\. Katzin, S\. van Mourik, F\. Kempkes, E\. J\. van Henten, GreenLight – an open source model for greenhouses with supplemental lighting: Evaluation of heat requirements under LED and HPS lamps, Biosystems Engineering 194 \(2020\) 61–81\.[doi:10\.1016/j\.biosystemseng\.2020\.03\.010](https://doi.org/10.1016/j.biosystemseng.2020.03.010)\.
- \[18\]A\. Y\. Ng, D\. Harada, S\. J\. Russell,[Policy invariance under reward transformations: Theory and application to reward shaping](https://ai.stanford.edu/~ang/papers/shaping-icml99.pdf), in: Proceedings of the Sixteenth International Conference on Machine Learning, Morgan Kaufmann, San Francisco, CA, USA, 1999, pp\. 278–287\. URL[https://ai\.stanford\.edu/~ang/papers/shaping\-icml99\.pdf](https://ai.stanford.edu/~ang/papers/shaping-icml99.pdf)
- \[19\]E\. Wiewiora, G\. W\. Cottrell, C\. Elkan,[Principled methods for advising reinforcement learning agents](https://cdn.aaai.org/ICML/2003/ICML03-103.pdf), in: Proceedings of the Twentieth International Conference on Machine Learning, 2003, pp\. 792–799\. URL[https://cdn\.aaai\.org/ICML/2003/ICML03\-103\.pdf](https://cdn.aaai.org/ICML/2003/ICML03-103.pdf)
- \[20\]S\. Devlin, D\. Kudenko,[Dynamic potential\-based reward shaping](https://www.ifaamas.org/Proceedings/aamas2012/papers/2C_3.pdf), in: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, 2012, pp\. 433–440\. URL[https://www\.ifaamas\.org/Proceedings/aamas2012/papers/2C\_3\.pdf](https://www.ifaamas.org/Proceedings/aamas2012/papers/2C_3.pdf)
- \[21\]J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, O\. Klimov, Proximal policy optimization algorithms, arXiv preprint arXiv:1707\.06347 \(2017\)\.[doi:10\.48550/arXiv\.1707\.06347](https://doi.org/10.48550/arXiv.1707.06347)\.
- \[22\]M\. Xiao, J\. Lan, J\. Yu, W\. Ma, Q\. Xie, C\. Sun, Grower\-in\-the\-loop interactive reinforcement learning for greenhouse climate control, Computers and Electronics in Agriculture 242 \(2026\) 111312\.[doi:10\.1016/j\.compag\.2025\.111312](https://doi.org/10.1016/j.compag.2025.111312)\.
- \[23\]J\. García, F\. Fernández,[A comprehensive survey on safe reinforcement learning](https://jmlr.org/papers/v16/garcia15a.html), Journal of Machine Learning Research 16 \(42\) \(2015\) 1437–1480\. URL[https://jmlr\.org/papers/v16/garcia15a\.html](https://jmlr.org/papers/v16/garcia15a.html)
- \[24\]J\. Tobin, R\. Fong, A\. Ray, J\. Schneider, W\. Zaremba, P\. Abbeel, Domain randomization for transferring deep neural networks from simulation to the real world, in: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems \(IROS\), 2017, pp\. 23–30\.[doi:10\.1109/IROS\.2017\.8202133](https://doi.org/10.1109/IROS.2017.8202133)\.
- \[25\]E\. Salvato, G\. Fenu, E\. Medvet, F\. A\. Pellegrino, Crossing the reality gap: A survey on sim\-to\-real transferability of robot controllers in reinforcement learning, IEEE Access 9 \(2021\) 153171–153187\.[doi:10\.1109/ACCESS\.2021\.3126658](https://doi.org/10.1109/ACCESS.2021.3126658)\.
- \[26\]F\. Muratore, F\. Ramos, G\. Turk, W\. Yu, M\. Gienger, J\. Peters, Robot learning from randomized simulations: A review, Frontiers in Robotics and AI 9 \(2022\) 799893\.[doi:10\.3389/frobt\.2022\.799893](https://doi.org/10.3389/frobt.2022.799893)\.
- \[27\]O\. Bastani, Y\. Pu, A\. Solar\-Lezama,[Verifiable reinforcement learning via policy extraction](https://papers.neurips.cc/paper/7516-verifiable-reinforcement-learning-via-policy-extraction), in: Advances in Neural Information Processing Systems 31, 2018, pp\. 2494–2504\. URL[https://papers\.neurips\.cc/paper/7516\-verifiable\-reinforcement\-learning\-via\-policy\-extraction](https://papers.neurips.cc/paper/7516-verifiable-reinforcement-learning-via-policy-extraction)
- \[28\]E\. Puiutta, E\. M\. S\. P\. Veith, Explainable reinforcement learning: A survey, arXiv preprint arXiv:2005\.06247 \(2020\)\.[doi:10\.48550/arXiv\.2005\.06247](https://doi.org/10.48550/arXiv.2005.06247)\.
- \[29\]Y\. Bekkemoen, Explainable reinforcement learning \(XRL\): A systematic review and taxonomy, Machine Learning 113 \(2024\) 355–441\.[doi:10\.1007/s10994\-023\-06479\-7](https://doi.org/10.1007/s10994-023-06479-7)\.
- \[30\]M\. Ryo, Explainable artificial intelligence and interpretable machine learning for agricultural data analysis, Artificial Intelligence in Agriculture 6 \(2022\) 257–265\.[doi:10\.1016/j\.aiia\.2022\.11\.003](https://doi.org/10.1016/j.aiia.2022.11.003)\.
- \[31\]Z\. Juozapaitis, A\. Koul, A\. Fern, M\. Erwig, F\. Doshi\-Velez,[Explainable reinforcement learning via reward decomposition](https://koulanurag.dev/publication/explainable-reinforcement-learning-via-reward-decomposition/), iJCAI Workshop on Explainable Artificial Intelligence \(2019\)\. URL[https://koulanurag\.dev/publication/explainable\-reinforcement\-learning\-via\-reward\-decomposition/](https://koulanurag.dev/publication/explainable-reinforcement-learning-via-reward-decomposition/)
- \[32\]Z\. Lin, L\. Zhao, D\. Yang, T\. Qin, T\.\-Y\. Liu, G\. Yang, Distributional reward decomposition for reinforcement learning, in: Advances in Neural Information Processing Systems 32, 2019, pp\. 15726–15737\.
- \[33\]S\. Hemming, H\. F\. de Zwart, A\. Elings, I\. Righini, A\. Petropoulou, Autonomous Greenhouse Challenge, first edition \(2018\) \(2019\)\.[doi:10\.4121/uuid:e4987a7b\-04dd\-4c89\-9b18\-883aad30ba9a](https://doi.org/10.4121/uuid:e4987a7b-04dd-4c89-9b18-883aad30ba9a)\.
- \[34\]S\. Hemming, H\. F\. de Zwart, A\. Elings, A\. Petropoulou, I\. Righini, Autonomous Greenhouse Challenge, second edition \(2019\) \(2020\)\.[doi:10\.4121/uuid:88d22c60\-21b3\-4ea8\-90db\-20249a5be2a7](https://doi.org/10.4121/uuid:88d22c60-21b3-4ea8-90db-20249a5be2a7)\.
- \[35\]S\. Maree, P\. Zhang, B\. M\. van Marrewijk, H\. F\. de Zwart, M\. Bijlaard, S\. Hemming, et al\., 4th Autonomous Greenhouse Challenge: Dwarf tomato timeseries and images \(2025\)\.[doi:10\.4121/fa102772\-32db\-4b30\-bace\-12f2016722ce\.v1](https://doi.org/10.4121/fa102772-32db-4b30-bace-12f2016722ce.v1)\.