AI Learning and Conceptual Transfer in the Game of Hidden Rules

# AI Learning and Conceptual Transfer in the Game of Hidden Rules
Source: [https://arxiv.org/html/2608.21372](https://arxiv.org/html/2608.21372)
\\DTMnewdatestyle

monthyeardate

Wentian WangUniversity of Southern CaliforniaJacob FeldmanRutgers University, New BrunswickLazaros K\. GallosRutgers University, New BrunswickPaul B\. KantorRutgers University, New BrunswickPaul B\. Kantor, ConsultantVladimir MenkovRutgers University, New BrunswickHao WangRutgers University, New Brunswick

###### Abstract

This report summarizes the work conducted on the Game of Hidden Rules \(GOHR\), focusing on reinforcement learning agents trained to infer hidden rules from trial\-and\-error feedback, representation design, rule difficulty analysis, transfer learning, generalization, and pseudo\-bot\-assisted human learning analysis\. The report focuses on the Transformer\-based A2C framework, Feature\-Centric and Object\-Centric representations, experimental findings, and classification of human learning data\.

###### Contents

1. [1Introduction](https://arxiv.org/html/2608.21372#S1)1. [1\.1Motivation](https://arxiv.org/html/2608.21372#S1.SS1) 2. [1\.2Problem Statement](https://arxiv.org/html/2608.21372#S1.SS2) 3. [1\.3Summary of Contributions](https://arxiv.org/html/2608.21372#S1.SS3)
2. [2The Game of Hidden Rules](https://arxiv.org/html/2608.21372#S2)1. [2\.1Overview of GOHR](https://arxiv.org/html/2608.21372#S2.SS1) 2. [2\.2The Game Mechanics](https://arxiv.org/html/2608.21372#S2.SS2)1. [2\.2\.1Board Layout](https://arxiv.org/html/2608.21372#S2.SS2.SSS1) 2. [2\.2\.2Pieces, Buckets, and Rules](https://arxiv.org/html/2608.21372#S2.SS2.SSS2) 3. [2\.2\.3Feedback and Episode Termination](https://arxiv.org/html/2608.21372#S2.SS2.SSS3) 3. [2\.3GOHR Environment](https://arxiv.org/html/2608.21372#S2.SS3)
3. [3Methodology](https://arxiv.org/html/2608.21372#S3)1. [3\.1Reinforcement Learning Formulation](https://arxiv.org/html/2608.21372#S3.SS1) 2. [3\.2State and Feature Encoding](https://arxiv.org/html/2608.21372#S3.SS2)1. [3\.2\.1Feature\-Centric Representation](https://arxiv.org/html/2608.21372#S3.SS2.SSS1) 2. [3\.2\.2Object\-Centric Representation](https://arxiv.org/html/2608.21372#S3.SS2.SSS2) 3. [3\.2\.3Temporal History Encoding](https://arxiv.org/html/2608.21372#S3.SS2.SSS3) 3. [3\.3Action Space](https://arxiv.org/html/2608.21372#S3.SS3) 4. [3\.4Reward Function](https://arxiv.org/html/2608.21372#S3.SS4) 5. [3\.5Transformer\-Based Actor\-Critic Architecture](https://arxiv.org/html/2608.21372#S3.SS5)1. [3\.5\.1Input Projection and Positional Encoding](https://arxiv.org/html/2608.21372#S3.SS5.SSS1) 2. [3\.5\.2Transformer Encoder](https://arxiv.org/html/2608.21372#S3.SS5.SSS2) 3. [3\.5\.3Actor Head](https://arxiv.org/html/2608.21372#S3.SS5.SSS3) 4. [3\.5\.4Critic Head](https://arxiv.org/html/2608.21372#S3.SS5.SSS4) 6. [3\.6A2C Learning Algorithm](https://arxiv.org/html/2608.21372#S3.SS6) 7. [3\.7Evaluation Metrics](https://arxiv.org/html/2608.21372#S3.SS7)
4. [4Environmental and Training Setup](https://arxiv.org/html/2608.21372#S4)1. [4\.1Environment and Rule Suite](https://arxiv.org/html/2608.21372#S4.SS1) 2. [4\.2Model Configuration and Hyperparameters](https://arxiv.org/html/2608.21372#S4.SS2) 3. [4\.3Training and Evaluation Protocol](https://arxiv.org/html/2608.21372#S4.SS3)
5. [5Experiments and Results](https://arxiv.org/html/2608.21372#S5)1. [5\.1Independent Rule Experiments](https://arxiv.org/html/2608.21372#S5.SS1)1. [5\.1\.1Difficulty Analysis of Rule Properties](https://arxiv.org/html/2608.21372#S5.SS1.SSS1) 2. [5\.1\.2Rule Difficulty Analysis](https://arxiv.org/html/2608.21372#S5.SS1.SSS2) 3. [5\.1\.3Key Observations](https://arxiv.org/html/2608.21372#S5.SS1.SSS3) 2. [5\.2Generalization Analysis](https://arxiv.org/html/2608.21372#S5.SS2)1. [5\.2\.1Feature\-Centric \(FC\) Generalization](https://arxiv.org/html/2608.21372#S5.SS2.SSS1) 2. [5\.2\.2Object\-Centric \(OC\) Generalization](https://arxiv.org/html/2608.21372#S5.SS2.SSS2) 3. [5\.2\.3Observations](https://arxiv.org/html/2608.21372#S5.SS2.SSS3) 3. [5\.3Similarity of Shape and Color Rules](https://arxiv.org/html/2608.21372#S5.SS3)1. [5\.3\.1Within\-Rule Comparison](https://arxiv.org/html/2608.21372#S5.SS3.SSS1) 2. [5\.3\.2Comparison Between Shape and Color Rules](https://arxiv.org/html/2608.21372#S5.SS3.SSS2) 3. [5\.3\.3Effect of Increasing the Number of Runs](https://arxiv.org/html/2608.21372#S5.SS3.SSS3) 4. [5\.3\.4Summary](https://arxiv.org/html/2608.21372#S5.SS3.SSS4) 4. [5\.4Transfer Experiments](https://arxiv.org/html/2608.21372#S5.SS4)1. [5\.4\.1Simple\-to\-Simple Transfer](https://arxiv.org/html/2608.21372#S5.SS4.SSS1) 2. [5\.4\.2Compound Transfer Experiments](https://arxiv.org/html/2608.21372#S5.SS4.SSS2) 3. [5\.4\.3Transfer toordL1\_Nearby](https://arxiv.org/html/2608.21372#S5.SS4.SSS3) 4. [5\.4\.4Transfer tocm\_RBKY\_cw\_0123](https://arxiv.org/html/2608.21372#S5.SS4.SSS4) 5. [5\.4\.5Summary](https://arxiv.org/html/2608.21372#S5.SS4.SSS5) 5. [5\.5Multidimensional Scaling Analysis](https://arxiv.org/html/2608.21372#S5.SS5) 6. [5\.6Transfer Geometry and Clustering Structure](https://arxiv.org/html/2608.21372#S5.SS6)1. [5\.6\.1Row clustering: similarity of successor\-rule behavior\.](https://arxiv.org/html/2608.21372#S5.SS6.SSS1) 2. [5\.6\.2Column clustering: similarity of predecessor\-rule influence\.](https://arxiv.org/html/2608.21372#S5.SS6.SSS2) 3. [5\.6\.3Comparison between row and column structures\.](https://arxiv.org/html/2608.21372#S5.SS6.SSS3)
6. [6Human Learning Analysis via Pseudo\-Bot Modeling](https://arxiv.org/html/2608.21372#S6)1. [6\.1Motivation](https://arxiv.org/html/2608.21372#S6.SS1) 2. [6\.2Dataset and Problem Setup](https://arxiv.org/html/2608.21372#S6.SS2) 3. [6\.3Behavioral Modeling Approach](https://arxiv.org/html/2608.21372#S6.SS3)1. [6\.3\.1Outcome Sequence Representation](https://arxiv.org/html/2608.21372#S6.SS3.SSS1) 2. [6\.3\.2Markov / Backoff Model](https://arxiv.org/html/2608.21372#S6.SS3.SSS2) 3. [6\.3\.3Sequence Classification](https://arxiv.org/html/2608.21372#S6.SS3.SSS3) 4. [6\.4Evaluation Protocol](https://arxiv.org/html/2608.21372#S6.SS4) 5. [6\.5Results](https://arxiv.org/html/2608.21372#S6.SS5)1. [6\.5\.1Bot\-Assisted Side Classification](https://arxiv.org/html/2608.21372#S6.SS5.SSS1) 2. [6\.5\.2Effect of Number of Moves](https://arxiv.org/html/2608.21372#S6.SS5.SSS2) 3. [6\.5\.3Rule\-Level Differences](https://arxiv.org/html/2608.21372#S6.SS5.SSS3) 6. [6\.6Discussion](https://arxiv.org/html/2608.21372#S6.SS6)
7. [7Overall Discussion and Conclusion](https://arxiv.org/html/2608.21372#S7)1. [7\.1Key Findings](https://arxiv.org/html/2608.21372#S7.SS1) 2. [7\.2Limitations](https://arxiv.org/html/2608.21372#S7.SS2) 3. [7\.3Future Work](https://arxiv.org/html/2608.21372#S7.SS3)
8. [8Acknowledgments](https://arxiv.org/html/2608.21372#S8)
9. [References](https://arxiv.org/html/2608.21372#bib)
10. [AGymnasium\-Compatible GOHR Environment](https://arxiv.org/html/2608.21372#A1)1. [A\.1Design Goals](https://arxiv.org/html/2608.21372#A1.SS1) 2. [A\.2Gymnasium Interface](https://arxiv.org/html/2608.21372#A1.SS2) 3. [A\.3Support for Multiple Representations](https://arxiv.org/html/2608.21372#A1.SS3) 4. [A\.4Legacy Compatibility Wrappers](https://arxiv.org/html/2608.21372#A1.SS4) 5. [A\.5Observations from the Refactoring Experiment](https://arxiv.org/html/2608.21372#A1.SS5) 6. [A\.6Rendering and Visualization](https://arxiv.org/html/2608.21372#A1.SS6) 7. [A\.7Availability](https://arxiv.org/html/2608.21372#A1.SS7)
11. [BRule Descriptions](https://arxiv.org/html/2608.21372#A2)
12. [CRules Used in Rule\-Based Experiments](https://arxiv.org/html/2608.21372#A3)
13. [DRule\-Property Difficulty Rankings](https://arxiv.org/html/2608.21372#A4)
14. [EOverall Rule Difficulty for FC and OC Models](https://arxiv.org/html/2608.21372#A5)
15. [FTransfer Plots](https://arxiv.org/html/2608.21372#A6)
16. [GClustering and MDS Results](https://arxiv.org/html/2608.21372#A7)1. [G\.1Hierarchical Clustering Results](https://arxiv.org/html/2608.21372#A7.SS1) 2. [G\.2Multidimensional Scaling Results](https://arxiv.org/html/2608.21372#A7.SS2) 3. [G\.3Tanglegram Comparisons](https://arxiv.org/html/2608.21372#A7.SS3)
17. [HSupplementary Materials](https://arxiv.org/html/2608.21372#A8)

## 1Introduction

### 1\.1Motivation

Humans and intelligent agents frequently encounter situations in which the underlying rules governing the environment are not explicitly provided\. Instead, these rules must be inferred through interaction, observation, and feedback\. Learning such hidden structures is fundamental to many real\-world problems, including scientific reasoning, strategic decision making, robotics, and adaptive control\.

The*Game of Hidden Rules*\(GOHR\) provides a controlled environment for studying this type of learning\. In GOHR, an agent observes a board containing multiple objects and must infer an unknown latent rule that determines whether a particular action is accepted or rejected\. The agent does not receive direct supervision regarding the rule itself; instead, learning occurs only through sparse feedback indicating whether an attempted move satisfies the hidden constraints\.

This setting presents several important challenges for machine learning systems\. First, the agent must discover an abstract relational structure from limited feedback\. Second, it must generalize across different rule families that vary in complexity and representation\. Third, it has the opportunity to transfer previously learned knowledge to new but structurally related tasks\. These challenges are closely related to broader questions in reinforcement learning and representation learning regarding abstraction, compositionality, and transfer\.

GOHR is particularly well suited for studying these questions because the environment contains diverse rule families with interpretable structure\. Some rules depend on object features such as shape or color, while others depend on spatial relationships, bucket ordering, or combinations of multiple rule components\. This diversity enables systematic analysis of rule difficulty, generalization, and transfer behavior under controlled conditions\.

In addition to reinforcement learning analysis, GOHR also provides an opportunity to compare artificial learning behavior with human decision patterns\.111In fact, it was designed and built with such comparison in mind\[[1](https://arxiv.org/html/2608.21372#bib.bib8)\]\.Understanding whether machine\-learning representations capture structures similar to those used by humans may provide insight into interpretable and transferable learning strategies\.

### 1\.2Problem Statement

The central problem addressed in this work is the learning of latent rules from sparse accept/reject feedback within the GOHR environment \(Section[2](https://arxiv.org/html/2608.21372#S2)\); see also\[[7](https://arxiv.org/html/2608.21372#bib.bib11)\]\.

At each step, the agent observes the current board state and selects an action corresponding to placing an object into one of several buckets\. The environment evaluates the action according to a hidden rule and returns only limited feedback indicating whether the move was accepted or rejected\. The agent is therefore required to infer the latent rule indirectly through interaction with the environment\.

Several research questions arise from this setting:

- •How does the internal state representation affect learning performance and generalization?
- •Which rule properties contribute most strongly to learning difficulty?
- •Can previously learned rules facilitate learning of structurally related rules \(concept transfer\)?
- •Do different rule families induce consistent transfer\-learning structures?
- •To what extent do reinforcement\-learning agents exhibit behavior patterns similar to human learners?

To investigate these questions, this work compares two state representations:

1. 1\.a*Feature\-Centric \(FC\)*representation that encodes the board as feature maps over spatial positions, and
2. 2\.an*Object\-Centric \(OC\)*representation that explicitly represents individual objects and their attributes\.

Both representations are trained using a transformer\-based Actor\-Critic reinforcement\-learning framework\. The resulting agents are examined across independent rule learning, generalization, transfer learning, and transfer\-geometry analyses\.

### 1\.3Summary of Contributions

The primary contributions of this work are summarized below:

1. 1\.We develop an Object\-Centric \(OC\) state representation based on explicit object attributes and systematically compare its learning, generalization, and transfer behavior against the previously proposed Feature\-Centric \(FC\) representation \(\[[6](https://arxiv.org/html/2608.21372#bib.bib9)\]\) \.
2. 2\.We develop a transformer\-based Actor\-Critic \(\[[4](https://arxiv.org/html/2608.21372#bib.bib27)\]\) reinforcement\-learning architecture for learning hidden rules from sparse feedback\.
3. 3\.We analyze rule difficulty across multiple rule families and investigate how structural properties of the rules influence performance of the learning systems\.
4. 4\.We study generalization behavior across related shape and color rules under both FC and OC representations\.
5. 5\.We conduct extensive transfer\-learning experiments, including both simple\-to\-simple transfer and compound\-rule transfer settings\.
6. 6\.We analyze a geometry of transfer learning using hierarchical clustering, multidimensional scaling \(MDS\), and tanglegram comparisons to study relationships between rule families and transfer behavior\.
7. 7\.We analyze a pseudo\-bot\-assisted human learning seeking behavioral differences between assisted and non\-assisted human gameplay\. We develop a Gymnasium\-compatible\[[8](https://arxiv.org/html/2608.21372#bib.bib31)\]implementation of GOHR that preserves the legacy observation pipeline through compatibility wrappers while enabling integration with modern reinforcement\-learning frameworks\.

## 2The Game of Hidden Rules

### 2\.1Overview of GOHR

TheGame of Hidden Rules \(GOHR\)is the primary environment used in this work\. It is a controlled experimental framework designed to study rule discovery, learning, and generalization in both humans and artificial agents\. GOHR was developed by researchers at the University of Wisconsin, Madison, and has been used as a testbed for studying hidden\-structure learning \(\[[6](https://arxiv.org/html/2608.21372#bib.bib9),[1](https://arxiv.org/html/2608.21372#bib.bib8),[2](https://arxiv.org/html/2608.21372#bib.bib12),[7](https://arxiv.org/html/2608.21372#bib.bib11)\]\)\.

The environment consists of a6×66\\times 6grid\-based board and four buckets positioned at the corners\. At the start of each episode,n=9n=9objects are randomly placed on the board\. Each object is characterized by two attributes:shapeandcolor\. A hidden rule governs how objects must be assigned to buckets\.

The objective of the player \(human or agent\) is to correctly infer this hidden rule through trial\-and\-error interaction and successfully place all objects into valid buckets\. Since the rule is not explicitly provided, the task involves two coupled challenges: identifying the latent rule that governs object\-bucket assignments and learning a policy that selects actions consistent with that rule\.

Two key entities define the environment:

- •Rules:Define constraints mapping object properties \(e\.g\., color, shape, position\) to valid bucket assignments\.
- •Pieces \(Objects\):Game elements with attributes such as shape and color, randomly initialized on the board at the start of each episode\.

### 2\.2The Game Mechanics

#### 2\.2\.1Board Layout

The GOHR board is a6×66\\times 6grid indexed from 1 to 36, with four buckets located at the corners\. Each grid cell corresponds to a unique position index, while spatial coordinates\(x,y\)\(x,y\)define the true layout\.

Table 1:Representation of the GOHR board\.
#### 2\.2\.2Pieces, Buckets, and Rules

Each object on the board is defined by:

- •Shape:One of several shape categories \(e\.g\., square, circle, triangle, star\)
- •Color:One of several color categories \(e\.g\., red, blue, yellow, black\)
- •Position:Grid location\(x,y\)\(x,y\)

At each timestep, the agent selects an action corresponding to selecting an object and assigning it to one of the four buckets\. The validity of this assignment depends on the hidden rule\. Rules can vary in complexity and may depend on:

- •Single features \(e\.g\., color→\\rightarrowbucket\)
- •Spatial properties \(e\.g\., quadrant\-based placement\)
- •Sequential constraints, where the correctness of a move depends on previously accepted moves \(e\.g\., bucket\-ordering or reading\-order rules\)
- •Conditional dependencies between attributes

#### 2\.2\.3Feedback and Episode Termination

After each action, the environment provides feedback through status codes:

- •response\_code: - –0 \(ACCEPT\): Valid move\. - –4 \(DENY\): Invalid move\. - –7 \(IMMOVABLE\): Selected object cannot be moved
- •finish\_code:Indicates the status of the episode : - –0 \(CONTINUE\): The episode continues because movable objects remain on the board\. - –1 \(FINISH\): All objects have been removed from the board\. - –2 \(STALEMATE\): Objects remain on the board, but none can be moved\.
- •move\_count:Total number of actions taken so far

The reward function is defined as:

r=\{0if the move is valid \(ACCEPT\)−1if the move is invalid \(DENY or IMMOVABLE\)r=\\begin\{cases\}0&\\text\{if the move is valid \(ACCEPT\)\}\\\\ \-1&\\text\{if the move is invalid \(DENY or IMMOVABLE\)\}\\end\{cases\}
An episode terminates when all objects are successfully placed into valid buckets according to the hidden rule\.

### 2\.3GOHR Environment

All experiments in this work were conducted using the GOHR environment introduced in\[[6](https://arxiv.org/html/2608.21372#bib.bib9),[1](https://arxiv.org/html/2608.21372#bib.bib8),[2](https://arxiv.org/html/2608.21372#bib.bib12),[7](https://arxiv.org/html/2608.21372#bib.bib11)\]\.

The environment consists of a6×66\\times 6board containing movable objects and four corner buckets\. At each timestep, the agent selects an object and assigns it to a bucket\. The environment returns the updated board state together with status information indicating whether the move was accepted, denied, or immovable\.

The legacy GOHR implementation provides a reinforcement\-learning style interface throughreset\(\)andstep\(\)functions\. A distinguishing characteristic of this implementation is that observations include temporal history\. In particular, the observation contains the current board state together with several previous successful board states and the corresponding actions that produced them\. As a result, the environment provides a history\-augmented state representation rather than only the current state\.

This differs from standard reinforcement learning environments, where history modeling is typically handled by the agent architecture rather than being embedded directly in the environment\. Nevertheless, this legacy observation pipeline was used throughout all experiments reported in this work to maintain consistency with previous GOHR studies\.

A Gymnasium\-compatible refactoring of GOHR was also developed to improve modularity, support modern reinforcement\-learning libraries, and provide a unified interface for both Feature\-Centric and Object\-Centric representations\. Preliminary experiments using the standard Gymnasium observation design, in which observations contain only the current state, did not reproduce the learning behavior obtained with the legacy history\-augmented observation pipeline\. Consequently, all experimental results reported in this work use the legacy implementation\. The Gymnasium refactoring and associated experiments are described in Appendix[A](https://arxiv.org/html/2608.21372#A1)\.

## 3Methodology

### 3\.1Reinforcement Learning Formulation

We formulate the Game of Hidden Rules \(GOHR\) as a reinforcement learning problem in which an agent must infer an unobserved rule through interaction with the environment\. At each timesteptt, the agent observes an encoded board statests\_\{t\}, selects an actionata\_\{t\}, and receives a rewardrtr\_\{t\}based on whether the attempted move is accepted by the environment\. The hidden rule is not directly observed, making the task partially observable\.

The agent is trained to learn a policyπθ​\(at∣st\)\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)that maps encoded states to a distribution over actions\. A critic network estimates the value functionVϕ​\(st\)V\_\{\\phi\}\(s\_\{t\}\), which is used to compute advantages for policy optimization\. The objective is to learn a policy that minimizes incorrect moves and successfully places all pieces into their correct buckets according to the hidden rule\.

Fig\.[1](https://arxiv.org/html/2608.21372#S3.F1)summarizes the overall training pipeline used in this work, from environment interaction to actor\-critic optimization\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/sys_architecture.png)Figure 1:Overview of the GOHR reinforcement learning pipeline\. The environment produces observations that are encoded into FC or OC state representations, processed by a Transformer encoder, and passed to actor and critic heads for A2C training\.
### 3\.2State and Feature Encoding

The raw game state returned by the GOHR environment is transformed into one of two representations: a Feature\-Centric \(FC\) representation or an Object\-Centric \(OC\) representation\. These encodings differ in how they represent board structure, object identity, and feature relationships\.

#### 3\.2\.1Feature\-Centric Representation

In the Feature\-Centric representation, the board is encoded as a collection of spatial feature maps over the6×66\\times 6grid\. Each categorical attribute is represented as a separate binary feature map\. Specifically, the representation contains eight channels corresponding to four shapes and four colors:

SFC∈\{0,1\}6×6×8\.S\_\{\\mathrm\{FC\}\}\\in\\\{0,1\\\}^\{6\\times 6\\times 8\}\.
Fig\.[2](https://arxiv.org/html/2608.21372#S3.F2)illustrates how the current GOHR board state is converted into the Feature\-Centric input representation used by the model\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/x1.png)Figure 2:Feature\-Centric state construction\. The current board is encoded into spatial feature maps, where color and shape attributes are represented as separate channels over the6×66\\times 6grid\.A piece activates one shape channel and one color channel at its board position\. For example, a red square at a given grid cell activates both the “red” channel and the “square” channel at that same position\. Thus, color and shape are not explicitly fused into a single object representation; instead, the model must infer their association through spatial alignment across channels\.

For a single board state, the FC representation has dimensionality

6×6×8=288\.6\\times 6\\times 8=288\.

#### 3\.2\.2Object\-Centric Representation

In the Object\-Centric representation, each game piece is represented as an individual object vector\. Each object is encoded using one\-hot representations for color, shape, and position\. The object feature vector consists of:

- •4 dimensions for color,
- •4 dimensions for shape,
- •6 dimensions for thexx\-coordinate,
- •6 dimensions for theyy\-coordinate,

Thus, each object is represented as a 20\-dimensional vector\.

SOC∈ℝ9×20\.S\_\{\\mathrm\{OC\}\}\\in\\mathbb\{R\}^\{9\\times 20\}\.
To incorporate move history, the implementation appends a 4\-dimensional bucket encoding to historical object representations\. Consequently, the tensors provided to the learning model have dimension9×249\\times 24, where the final four dimensions store the bucket associated with a previous move\. For the current board state, these four dimensions are initialized to zero because no move has yet been selected\.

Fig\.[3](https://arxiv.org/html/2608.21372#S3.F3)illustrates how the same board state is represented using the Object\-Centric encoding\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/x2.png)Figure 3:Object\-Centric state construction\. Each game piece is encoded as an individual object vector containing its color, shape, position, and bucket/action\-related information\.Unlike the FC representation, the OC representation explicitly binds each object’s color, shape, and position into a single vector\. Objects are stored in fixed slots corresponding to their object indices, allowing the model to preserve associations among an object’s attributes throughout processing\. This reduces the need for the model to infer which color, shape, and position features belong to the same game piece\.

#### 3\.2\.3Temporal History Encoding

A single board state is insufficient to fully infer the hidden rule, since the agent must reason from previous successful and unsuccessful interactions\. Therefore, the input includes the current board state along with a fixed\-length history of the most recent successful board states and their corresponding actions\.

In this work, we use the current state and the six most recent successful states, giving a temporal context length of seven:

x=\[St,St−1,St−2,…,St−6\]\.x=\[S\_\{t\},S\_\{t\-1\},S\_\{t\-2\},\\ldots,S\_\{t\-6\}\]\.
For the FC representation, previous actions are encoded as one\-hot vectors over the position\-bucket action space:

\|𝒜FC\|=36×4=144\.\|\\mathcal\{A\}\_\{\\mathrm\{FC\}\}\|=36\\times 4=144\.
Each past FC timestep therefore consists of the flattened board representation and the previous action encoding:

The resulting FC input has dimensionality

xFC∈ℝ7×432\.x\_\{\\mathrm\{FC\}\}\\in\\mathbb\{R\}^\{7\\times 432\}\.
For the OC representation, the temporal input is represented as a stack of object\-wise feature matrices:

xOC∈ℝ7×9×24\.x\_\{\\mathrm\{OC\}\}\\in\\mathbb\{R\}^\{7\\times 9\\times 24\}\.
This temporal history allows the model to compare recent accepted moves and infer consistent feature\-bucket or object\-bucket relationships\.

### 3\.3Action Space

At each timestep, the agent selects a piece and assigns it to one of four buckets\. The action space differs between the FC and OC representations\.

For the FC model, actions are defined over board positions and buckets\. Since there are 36 grid positions and 4 buckets, the action space is

\|𝒜FC\|=36×4=144\.\|\\mathcal\{A\}\_\{\\mathrm\{FC\}\}\|=36\\times 4=144\.
For the OC model, actions are defined over objects and buckets\. Since there are 9 objects and 4 buckets, the action space is

\|𝒜OC\|=9×4=36\.\|\\mathcal\{A\}\_\{\\mathrm\{OC\}\}\|=9\\times 4=36\.
An action mask is used to prevent the agent from selecting invalid actions, such as actions corresponding to empty board positions or unavailable objects\.

### 3\.4Reward Function

The reward function is sparse and negative\-only\. A successful move receives zero reward, while an incorrect or invalid move receives a penalty:

rt=\{0,if the move is accepted,−1,if the move is denied or immovable\.r\_\{t\}=\\begin\{cases\}0,&\\text\{if the move is accepted\},\\\\\[4\.0pt\] \-1,&\\text\{if the move is denied or immovable\}\.\\end\{cases\}
This reward structure encourages the agent to minimize errors and discover the hidden rule with as few incorrect moves as possible\.

### 3\.5Transformer\-Based Actor\-Critic Architecture

We use a Transformer\-based actor\-critic architecture for both FC and OC inputs\. The Transformer encoder processes the encoded state history and produces a latent representation used by both the policy network and the value network\.

Fig\.[4](https://arxiv.org/html/2608.21372#S3.F4)shows the Transformer\-based model architecture used to process encoded GOHR state histories\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/transformer_architecture.png)Figure 4:Transformer\-based architecture used for GOHR policy and value learning\. Encoded feature states are projected into an embedding space, combined with positional encodings, passed through Transformer encoder layers, and mapped to output predictions through an MLP and output layer\.#### 3\.5\.1Input Projection and Positional Encoding

The input representation is first projected into a shared embedding dimensiondmodeld\_\{\\mathrm\{model\}\}using a learned linear projection\. Positional and temporal embeddings are added to preserve ordering information\.

For the FC model, each timestep in the temporal history is treated as a token\. For the OC model, each object within each timestep is treated as an object\-level token\. Objects occupy fixed positions in the input tensor according to their object indices, while temporal embeddings encode the ordering of states in the history\.

#### 3\.5\.2Transformer Encoder

The projected tokens are passed through a Transformer encoder consisting of multi\-head self\-attention and feedforward layers \(\[[9](https://arxiv.org/html/2608.21372#bib.bib30)\]\)\. Self\-attention allows the model to capture dependencies among features, objects, positions, and previous actions\.

For an input sequenceXX, multi\-head self\-attention is computed as

MHSA​\(X\)=Concat⁡\(h1,…,hH\)​WO,\\mathrm\{MHSA\}\(X\)=\\operatorname\{Concat\}\(h\_\{1\},\\ldots,h\_\{H\}\)W^\{O\},
where each attention head is given by

hi=softmax⁡\(Qi​Ki⊤dk\)​Vi\.h\_\{i\}=\\operatorname\{softmax\}\\left\(\\frac\{Q\_\{i\}K\_\{i\}^\{\\top\}\}\{\\sqrt\{d\_\{k\}\}\}\\right\)V\_\{i\}\.HereQiQ\_\{i\},KiK\_\{i\}, andViV\_\{i\}denoting the query, key, and value projections for headii, anddkd\_\{k\}the dimensionality of the key vectors\. This mechanism enables the model to learn relationships that are important for hidden\-rule inference, such as color\-bucket mappings, shape\-bucket mappings, spatial ordering, and object\-level dependencies\.

#### 3\.5\.3Actor Head

The actor head maps the Transformer output representation to action logits\. These logits are masked using the valid action set and converted into a categorical policy distribution:

πθ​\(at∣st\)=softmax⁡\(Wπ​z\+bπ\),\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)=\\operatorname\{softmax\}\(W\_\{\\pi\}z\+b\_\{\\pi\}\),
wherezzis the aggregated Transformer representation\. During training, actions are sampled from this distribution, while during evaluation the action with the highest probability may be selected\.

#### 3\.5\.4Critic Head

The critic head maps the same Transformer representation to a scalar value estimate:

Vϕ​\(st\)=Wv​z\+bv\.V\_\{\\phi\}\(s\_\{t\}\)=W\_\{v\}z\+b\_\{v\}\.
This value estimate is used to compute the advantage function for actor\-critic training\.

### 3\.6A2C Learning Algorithm

The model is trained using Advantage Actor\-Critic \(A2C\)\[[5](https://arxiv.org/html/2608.21372#bib.bib23)\]\. After each episode, discounted returns are computed as

Gt=∑k=0T−t−1γk​rt\+k\.G\_\{t\}=\\sum\_\{k=0\}^\{T\-t\-1\}\\gamma^\{k\}r\_\{t\+k\}\.
whereγ\\gammais the discount factor\.

The advantage estimate is then computed as

At=Gt−Vϕ​\(st\)\.A\_\{t\}=G\_\{t\}\-V\_\{\\phi\}\(s\_\{t\}\)\.
The critic is trained by minimizing the mean\-squared error between predicted values and empirical returns:

Lcritic​\(ϕ\)=1N​∑t\(Gt−Vϕ​\(st\)\)2\.L\_\{\\mathrm\{critic\}\}\(\\phi\)=\\frac\{1\}\{N\}\\sum\_\{t\}\\left\(G\_\{t\}\-V\_\{\\phi\}\(s\_\{t\}\)\\right\)^\{2\}\.
The actor is trained using an entropy\-regularized policy gradient loss:

Lpolicy​\(θ\)=−1N​∑t\(log⁡πθ​\(at∣st\)​At\+β​Ht\),L\_\{\\mathrm\{policy\}\}\(\\theta\)=\-\\frac\{1\}\{N\}\\sum\_\{t\}\\left\(\\log\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)A\_\{t\}\+\\beta H\_\{t\}\\right\),
whereHtH\_\{t\}is the entropy of the policy distribution andβ\\betais the entropy regularization coefficient\. The entropy term encourages exploration and prevents the policy from becoming prematurely deterministic\.

In addition to policy sampling, exploration is further encouraged through anϵ\\epsilon\-greedy strategy\. At training steptt, the exploration probability is

ϵt=ϵend\+\(ϵstart−ϵend\)​exp⁡\(−tϵdecay\),\\epsilon\_\{t\}=\\epsilon\_\{\\mathrm\{end\}\}\+\(\\epsilon\_\{\\mathrm\{start\}\}\-\\epsilon\_\{\\mathrm\{end\}\}\)\\exp\\left\(\-\\frac\{t\}\{\\epsilon\_\{\\mathrm\{decay\}\}\}\\right\),
whereϵstart\\epsilon\_\{\\mathrm\{start\}\},ϵend\\epsilon\_\{\\mathrm\{end\}\}, andϵdecay\\epsilon\_\{\\mathrm\{decay\}\}control the initial exploration rate, final exploration rate, and exploration decay schedule, respectively\. With probabilityϵt\\epsilon\_\{t\}, the agent selects a random valid action; otherwise, it follows the learned policy distribution\.

### 3\.7Evaluation Metrics

We evaluate learning performance using three convergence metrics:M⋆M^\{\\star\},Emean⋆E^\{\\star\}\_\{\\mathrm\{mean\}\}, andEmax⋆E^\{\\star\}\_\{\\mathrm\{max\}\}\. These metrics quantify how quickly and reliably the agent learns a hidden rule\.

1. 1\.M⋆M^\{\\star\}: The move\-based convergence metricm⋆m^\{\\star\}identifies the first move after which the agent makes only successful moves for a fixed window of lengthWm⋆W\_\{m^\{\\star\}\}: m⋆=min⁡\{m∈ℕ:∀i∈\[m,m\+Wm⋆−1\],𝚛𝚎𝚜𝚙𝚘𝚗𝚜𝚎​\_​𝚌𝚘𝚍𝚎​\(i\)=𝙰\}\.m^\{\\star\}=\\min\\left\\\{m\\in\\mathbb\{N\}:\\forall i\\in\[m,m\+W\_\{m^\{\\star\}\}\-1\],\\;\\mathtt\{response\\\_code\}\(i\)=\\mathtt\{A\}\\right\\\}\. Here,𝙰\\mathtt\{A\}denotes an accepted move\. The aggregated metricM⋆M^\{\\star\}is computed as the median ofm⋆m^\{\\star\}across multiple training runs\.
2. 2\.Emean⋆E^\{\\star\}\_\{\\mathrm\{mean\}\}: The mean\-window convergence metricemean⋆e^\{\\star\}\_\{\\mathrm\{mean\}\}identifies the earliest episode at which the average error rate within a sliding window of sizeWmeanW\_\{\\mathrm\{mean\}\}falls below a thresholdTmeanT\_\{\\mathrm\{mean\}\}: emean⋆=min⁡\{t:1Wmean​∑k=0Wmean−1Et\+k≤Tmean\}\.e^\{\\star\}\_\{\\mathrm\{mean\}\}=\\min\\left\\\{t:\\frac\{1\}\{W\_\{\\mathrm\{mean\}\}\}\\sum\_\{k=0\}^\{W\_\{\\mathrm\{mean\}\}\-1\}E\_\{t\+k\}\\leq T\_\{\\mathrm\{mean\}\}\\right\\\}\. The aggregated metricEmean⋆E^\{\\star\}\_\{\\mathrm\{mean\}\}is computed as the median ofemean⋆e^\{\\star\}\_\{\\mathrm\{mean\}\}across runs\.
3. 3\.Emax⋆E^\{\\star\}\_\{\\mathrm\{max\}\}: The max\-window convergence metricemax⋆e^\{\\star\}\_\{\\mathrm\{max\}\}is stricter than the mean\-window metric\. It identifies the earliest episode at which the maximum error rate within a sliding window remains below a thresholdTmaxT\_\{\\mathrm\{max\}\}: emax⋆=min⁡\{t:max0≤k<Wmax⁡Et\+k≤Tmax\}\.e^\{\\star\}\_\{\\mathrm\{max\}\}=\\min\\left\\\{t:\\max\_\{0\\leq k<W\_\{\\mathrm\{max\}\}\}E\_\{t\+k\}\\leq T\_\{\\mathrm\{max\}\}\\right\\\}\. The aggregated metricEmax⋆E^\{\\star\}\_\{\\mathrm\{max\}\}is computed as the median ofemax⋆e^\{\\star\}\_\{\\mathrm\{max\}\}across runs\.

Together, these three metrics capture complementary aspects of learning: move\-level stability, average episode\-level convergence, and worst\-case episode\-level robustness\.

Fig\.[5](https://arxiv.org/html/2608.21372#S3.F5)presents the Spearman correlation heatmap between the evaluation metrics across 18 rules using both 6\-step and 8\-step memory windows\.

![Heatmap matrix of Spearman correlations between evaluation metrics. Rows and columns correspond to different evaluation metrics. Darker shading indicates stronger positive correlation.](https://arxiv.org/html/2608.21372v1/figures/spearman_correlation_heatmap.png)Figure 5:Heatmap of Spearman correlations between evaluation metrics evaluated using 6\-step and 8\-step memory windows\.

## 4Environmental and Training Setup

### 4\.1Environment and Rule Suite

All experiments were conducted in the GOHR6×66\\times 6environment with nine objects and four buckets\. At the beginning of each episode, a hidden rule determines the correct bucket assignment for each object\. The agent does not observe this rule directly and must infer it through interaction with the environment\.

The experiments evaluate the agent across a suite of hidden rules that vary in both difficulty and structural complexity, including feature\-based, spatial, ordering\-based, and relational rules\. The complete list of rules used in the experiments, together with their descriptions, is provided in Appendix[B](https://arxiv.org/html/2608.21372#A2)\.

### 4\.2Model Configuration and Hyperparameters

Both the Feature\-Centric \(FC\) and Object\-Centric \(OC\) agents use the Transformer\-based A2C architecture described in Sections[3\.5](https://arxiv.org/html/2608.21372#S3.SS5)–[3\.6](https://arxiv.org/html/2608.21372#S3.SS6)\. The two models differ only in their state representation and action\-space formulation\.

The hyperparameters used across the reinforcement learning experiments are summarized in Table[2](https://arxiv.org/html/2608.21372#S4.T2)\.

Table 2:Training hyperparameters used across the reinforcement learning experiments\.Here,α\\alphadenotes the learning rate used by the optimizer,γ\\gammais the discount factor, andβ\\betais the entropy regularization coefficient used in the policy loss\. We use a small discount factor,γ=0\.001\\gamma=0\.001, which places most of the learning signal on immediate rewards\. This choice reflects the structure of GOHR, where rewards are sparse and negative\-only, and the primary objective is to minimize incorrect moves rather than optimize long\-horizon cumulative reward\.

The evaluation metrics described in Section[3\.6](https://arxiv.org/html/2608.21372#S3.SS6)measure the extent to which the agent exhibits stable rule\-following behavior\. Metrics such asM⋆M^\{\\star\},Emean⋆E^\{\\star\}\_\{\\mathrm\{mean\}\}, andEmax⋆E^\{\\star\}\_\{\\mathrm\{max\}\}quantify the consistency of correct actions across extended sequences of moves and episodes\. Although these metrics are not optimized directly by the A2C objective, successful performance on them indicates that the agent has acquired the underlying hidden rule and can apply it reliably over time\.

### 4\.3Training and Evaluation Protocol

Unless otherwise stated, each experiment was repeated for five independent runs with different random seeds\. Each run was trained until convergence according to the evaluation metrics introduced in Section[3\.6](https://arxiv.org/html/2608.21372#S3.SS6):

1. 1\.the agent achieves at leastWm⋆=10W\_\{m^\{\\star\}\}=10consecutive successful moves, corresponding to the move\-level convergence criterion used to definem⋆m^\{\\star\};
2. 2\.the mean\-window error rate satisfies emean⋆<0\.25e^\{\\star\}\_\{\\mathrm\{mean\}\}<0\.25over a sliding window ofWmean=10W\_\{\\mathrm\{mean\}\}=10episodes;
3. 3\.the max\-window error rate satisfies emax⋆<0\.25e^\{\\star\}\_\{\\mathrm\{max\}\}<0\.25over a sliding window ofWmax=5W\_\{\\mathrm\{max\}\}=5episodes\.

These criteria were used jointly to avoid premature convergence\. The episode\-level metrics evaluate whether the agent has reached stable low\-error behavior over time, while the move\-level metric verifies the ability to sustain consecutive sequences of correct actions\.

## 5Experiments and Results

The experiments using the GOHR environment encompass two primary configurations:

1. 1\.Rule\-based configuration:In this configuration, a rule file specifying the rule definition is provided to the game server\.
2. 2\.Trial\-list\-based configuration:In this configuration, a trial list is supplied, specifying both the rule set and the initial board generation process\. This method also supports transfer experiments\.

### 5\.1Independent Rule Experiments

Eighteen different rule files were considered for these experiments\. The rules used are listed in Table[6](https://arxiv.org/html/2608.21372#A3.T6)\. Each rule was used to train a model independently, in order to assess model performance on isolated rule learning\.

For the analysis of rule difficulty, we initially attempted to analyze the difficulty of each rule separately\. However, slight variations in metrics and the substantial overlap between metrics made it difficult to rank the rules clearly\.

Therefore, we characterized the rules according to their individual conceptual properties \(see Table[3](https://arxiv.org/html/2608.21372#S5.T3)\)\. This allowed us to evaluate performance across different classes of rule properties and determine which properties were easier or more difficult for the learning system to acquire\.

We have identified the following properties for analysis:

1. 1\.Quadrant\_to\_bucket\_mapping: Mapping of pieces in a specified quadrant to a particular bucket\.
2. 2\.Proximity: Removing pieces based on proximity to any bucket\. “Farthest” indicates pieces at the center of the board, while “nearest” indicates pieces in any of the corners\.
3. 3\.Reading\_order: Removing pieces in reading order \(or reverse reading order\), that is, from left to right and top to bottom \(or right to left and bottom to top\)\.
4. 4\.Feature\_to\_bucket\_mapping: Mapping each color/shape to any bucket\.
5. 5\.Feature\_ordering: Removing one piece of each color/shape in order \(e\.g\., blue after red, black after blue\)\.
6. 6\.All\_pieces\_of\_feature: Removing all pieces of one color/shape, then proceeding to the next\.
7. 7\.Bucket\_ordering: Assigning pieces to a specified order of buckets\.
8. 8\.Conditional: Skipping pieces based on specified conditions\.

All rules can be characterized as using either one of these properties or a combination of multiple properties\.

Table 3:Mapping of Rules to Their Corresponding Properties#### 5\.1\.1Difficulty Analysis of Rule Properties

Our analysis makes use of the notion of “degree of abstraction\.” We define these degrees both intuitively and by observation of the difficulty of various rules\. We observed a clear trend in the difficulty of learning each rule, according to its conceptual properties\. The difficulty is largely determined by the degree of abstraction in the feature underlying each property\. Even when considering features at the same level of abstraction, the mapping to buckets is generally easier to learn than an allowed ordering based on features\.

Based on our experiments and observations, we establish the relative ordering of difficulty for rule properties in the FC and OC models, providing insight into how each representation influences learning performance\. The experiment results can be found in Tables[7](https://arxiv.org/html/2608.21372#A4.T7)and[8](https://arxiv.org/html/2608.21372#A4.T8)\.

When comparing rule properties, we primarily relied onM⋆M^\{\\star\}values\. UnlikeEmean⋆E^\{\\star\}\_\{\\mathrm\{mean\}\}andEmax⋆E^\{\\star\}\_\{\\mathrm\{max\}\}, which aggregate performance over entire episodes,M⋆M^\{\\star\}is defined directly in terms of sequences of correct moves\. Consequently, it provides a more direct measure of when the agent begins to apply a rule property consistently\.

This distinction is particularly important because different rule properties influence different subsets of moves\. For example, conditional properties are relevant only when their associated preconditions are satisfied, while some ordering\-based properties become informative only after earlier actions have established the appropriate context\. As a result, many moves within an episode may provide little information about whether a particular property has been learned\. Episode\-level error rates therefore tend to be noisier and less directly tied to the acquisition of a specific property\. By focusing on sustained sequences of correct actions,M⋆M^\{\\star\}provides a more interpretable measure of property\-level learning and facilitates comparisons across different classes of rule properties\.

By contrast,Emean⋆E^\{\\star\}\_\{\\mathrm\{mean\}\}andEmax⋆E^\{\\star\}\_\{\\mathrm\{max\}\}remain useful complementary measures of convergence because they quantify average and worst\-case episode\-level performance\. However, for the purpose of comparing the relative difficulty of individual rule properties,M⋆M^\{\\star\}provides the most direct indication of when a property has been acquired and applied consistently\.

##### FC Model:

In the FC model, positional properties are the easiest to learn \(lowest level of abstraction\), followed by properties that depend on piece features \(higher level of abstraction\)\.

1. 1\.Easiest properties: Quadrant\_mapping, proximity, and reading\_order were the most accessible properties for the FC model to learn\. This can be attributed to their primary dependence on piece positions, which the FC model captures most effectively\. Among these, quadrant\_mapping was the easiest, followed by proximity with nearly identical learning curves, while reading\_order proved more challenging\.
2. 2\.Moderate Difficulty: Feature\_to\_bucket mapping follows next, depending on individual piece features such as color or shape being mapped to output buckets\. The all\_pieces\_of\_feature property, which also depends on piece features, exhibits similar difficulty\.
3. 3\.Higher Difficulty: Bucket\_ordering proved more challenging, as it depends on an abstract ordering of buckets that is difficult for the FC model representation to capture\.
4. 4\.Most Challenging: Feature\_ordering and conditional properties represent the most challenging categories for the FC model, exhibiting very slow learning curves\.

##### OC Model:

While the FC model showed significant differences in difficulty between properties, the OC model exhibited relatively smaller differences in difficulty and learning curves between properties\.

1. 1\.Easiest properties: Feature\_to\_bucket mapping, all\_pieces\_of\_feature, and quadrant\_mapping were most accessible for the model to learn\. These properties depend on the lowest level of feature abstraction, directly utilizing color, shape, or x,y coordinates\. These properties demonstrated nearly identical learning curves with minimal differences\. Notably, quadrant\_mapping was the most challenging among these three, which can be attributed to its dependence on two features \(x and y coordinates\) rather than a single feature\.
2. 2\.Moderate Difficulty: Bucket\_ordering proved slightly more challenging than the easiest properties\.
3. 3\.Higher Difficulty: Reading\_order and proximity exhibited slower learning curves compared to the above properties\. This can be attributed to the higher level of feature abstraction, as the model must derive positional information from the provided x and y coordinates\.
4. 4\.Most Challenging: Consistent with the FC model, feature\_ordering and conditional properties remained the most challenging for the OC model to learn, exhibiting very slow learning curves\.

#### 5\.1\.2Rule Difficulty Analysis

Having established the relative difficulty ordering for individual rule properties, we now examine the difficulty ordering of complete rules\.

Rules ordered by increasing difficulty are presented in Tables[9](https://arxiv.org/html/2608.21372#A5.T9)and[10](https://arxiv.org/html/2608.21372#A5.T10)\. The rules are grouped into categories according to the ranges of their metric values\. While the ordering of rules within each category is somewhat ambiguous and may vary depending on the chosen metric, we consistently observe a clear relative ordering across categories\. This relative structure remains stable across all three metrics\.

##### FC Model

1. 1\.Highly Learnable: The most accessible rules are the quadrant\_mapping rules: quadNearby and quadMixed1\.
2. 2\.Moderately Learnable: Rules based on proximity and reading\_order follow: ordL1\_Nearby, ordL1, ordRevOfL1, and ordRevOfL1\_Remotest\.
3. 3\.Challenging: Rules depending on feature\_to\_bucket mapping and all\_pieces\_of\_feature properties include cm\_RBKY, sm\_csqt, allOfShaOrd\_qcts, and allOfColOrd\_BRKY\.
4. 4\.More Challenging: The rules col1OrdBuck\_BRKY0213 and sha1OrdBuck\_qcts0213, which are primarily feature\_to\_bucket mapping rules, show slower learning due to the additional feature ordering component\. The ccw and cw rules demonstrate comparable difficulty to these combined rules\.
5. 5\.Highly Challenging: Rules depending on feature\_ordering properties \(sha1Ord\_qcts and col1Ord\_BRKY\) are very difficult to learn\.
6. 6\.Most Challenging: When conditional properties are added to feature\_ordering, the rules become most challenging for the model\. This category includes colOrdL1\_BRKY and shaOrdL1\_qcts\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/x3.png)Figure 6:Crossings plots for rule difficulty between different metrics\. Each line connects a rule’s normalized difficulty score in theEm​e​a​n⋆E^\{\\star\}\_\{mean\}\(left\) to theM⋆M^\{\\star\}\(right\)\. Labels are displayed in the order of theirM⋆M^\{\\star\}values ;color of rule label corresponds to the color of the line it represents; raw scales \(min, median, max per model\) are shown inside each panel\. A scale including the minimum, median and max values of each metrics is included\. \(a\) shows the comparison of metrics in FC model and \(b\) shows the comparison of the metrics in OC model\.
##### OC Model

In the OC model, rule difficulties are more closely clustered, making ordering more difficult based on final metrics, with apparent crossings between metrics\.

1. 1\.Learnable: The most accessible rules include allOfColOrd\_BRKY, allOfShaOrd\_qcts, cm\_RBKY, sm\_csqt, quadNearby, quadMixed1, cw, ccw, ordL1 and ordRevOfL1\. These rules primarily rely on properties such as feature\_to\_bucket mapping, all\_pieces\_of\_feature removal, quadrant\_mapping, and bucket\_ordering\. Among them, ordL1 and ordRevOfL1, which are based on reading\_order, show slightly higher difficulty but remain close to the others in this group and can still be considered part of the learnable category\.
2. 2\.Moderately Challenging: When proximity is added to the reading properties, rules become more challenging to learn, including ordRevOfL1\_Remotest and ordL1\_Nearby\. The rules col1OrdBuck\_BRKY0213 and sha1OrdBuck\_qcts0213, which combine feature\_to\_bucket mapping with feature\_ordering, demonstrate comparable difficulty to reading\+proximity rules\.
3. 3\.Most Challenging: Similar to the FC model, rules depending on feature\_ordering are the most difficult, including col1Ord\_BRKY, sha1Ord\_qcts, colOrdL1\_BRKY, and shaOrdL1\_qcts\. The addition of conditional properties makes colOrdL1\_BRKY and shaOrdL1\_qcts the most challenging rules for the model to learn\.

#### 5\.1\.3Key Observations

In Fig\.[6](https://arxiv.org/html/2608.21372#S5.F6)\(a\) we observe numerous crossings between theEmean⋆E^\{\\star\}\_\{\\text\{mean\}\}andM⋆M^\{\\star\}rankings\. Most crossings occur within or between closely related rule properties categories \(as defined in the Rules/Rule Property Difficulty Analysis\), indicating that the relative difficulty of rules in the FC representation is sensitive to the choice of convergence metric\. Rankings derived from episode\-level error\-rate windows \(Emean⋆E^\{\\star\}\_\{\\mathrm\{mean\}\}\) do not always agree with those derived from move\-level sequences of correct actions \(M⋆M^\{\\star\}\)\.

This effect is especially visible among feature\-dependent properties \(feature\_to\_bucket, all\_pieces\_of\_feature, feature\_ordering\), whereas the easiest positional rule properties \(e\.g\., quadrant/proximity/reading\_order\) and the hardest feature\_ordering/conditional rule properties tend to preserve their extremes of the ranking\.

By contrast, Fig\.[6](https://arxiv.org/html/2608.21372#S5.F6)\(b\) shows substantially fewer crossings, yielding a more stable rule ordering across metrics\. This aligns with our earlier observation that OC representation exhibits smaller gaps between rule properties:feature\_to\_bucket, all\_pieces\_of\_feature, and quadrant\_to\_bucket\_mapping cluster together on the easier side, while feature\_ordering and conditional rules remain hardest, with the same relative ordering under both metrics\. In short, OC provides a more coherent notion of difficulty across measures, whereas FC shows larger metric\-driven variability, primarily within or at the boundaries of related categories\.

### 5\.2Generalization Analysis

We conducted a set of experiments to evaluate how well the learned representations generalize beyond the training distribution\. The Feature\-Centric \(FC\) and Object\-Centric \(OC\) models were evaluated under different forms of distribution shift aligned with the inductive biases of their respective representations\.

#### 5\.2\.1Feature\-Centric \(FC\) Generalization

![Checkerboard diagram indicating the positional split used during training. White squares represent locations available during training, while the remaining positions are excluded and used only during testing.](https://arxiv.org/html/2608.21372v1/x4.png)Figure 7:Checkerboard positional split used for FC generalization experiments\. White squares indicate positions available during training\.For the FC model, generalization was evaluated using thetrainandtestmodes provided by the GOHR environment\. Intrainmode, objects were restricted to a predefined subset of board positions, whereas intestmode at least one object was placed in a location not encountered during training\. This setup evaluates whether the FC representation learns position\-invariant rule structure or instead relies on memorizing spatial activation patterns\.

To ensure broad spatial coverage, we adopted the checkerboard positional split shown in Fig\.[7](https://arxiv.org/html/2608.21372#S5.F7)\. During training, objects were placed only on the white squares, while testing allowed placements across the full board\.

![Comparison of Feature-Centric in-distribution and out-of-distribution error rates across rules. The model is trained using restricted board positions and evaluated on unrestricted placements. Out-of-distribution errors increase substantially for most rules, especially spatially dependent rules.](https://arxiv.org/html/2608.21372v1/figures/fc_outofdist.png)Figure 8:Feature\-Centric \(FC\) generalization performance\. In\-distribution \(ID\) and out\-of\-distribution \(OOD\) error rates across rules\. The FC model is trained using restricted checkerboard positions and evaluated on unrestricted placements\. OOD errors increase substantially for most rules, indicating strong dependence on positional patterns observed during training\.Fig\.[8](https://arxiv.org/html/2608.21372#S5.F8)presents the ID and OOD error rates across rules\. The positional shift produces a substantial degradation in performance\. While ID error rates typically remain within the 25–35% range, OOD errors increase sharply for most rules, commonly reaching 55–75%\.

Several rules exhibit particularly severe failures\. The rulesquadMixed1,quadNearbyand reading rules likeordL1,ordL1\_Nearbyexceed 80% OOD error\. These rules depend heavily on spatial structure, suggesting that the FC representation strongly couples rule learning with absolute board coordinates observed during training\. When objects appear in previously unseen locations, the learned feature activations no longer align reliably with the underlying rule\.

Overall, the FC experiments indicate limited positional generalization\. The representation tends to encode location\-specific spatial templates rather than learning more abstract rule\-consistent relationships\.

#### 5\.2\.2Object\-Centric \(OC\) Generalization

![Comparison of Object-Centric in-distribution and out-of-distribution error rates across rules. The model is trained with nine objects and evaluated both with nine objects and with five objects. Error increases remain comparatively modest except for strongly spatial rules.](https://arxiv.org/html/2608.21372v1/figures/oc_id_ood_error_plot.png)Figure 9:Object\-Centric \(OC\) generalization performance\. In\-distribution \(ID\) and out\-of\-distribution \(OOD\) error rates across rules\. The OC model is trained using boards containing 9 objects and evaluated using both 9\-object \(ID\) and 16\-object \(OOD\) settings\. The figure shows the effect of increasing scene cardinality beyond the range observed during training\.Unlike the FC representation, the OC model represents each object independently through its attributes and position encoding, reducing reliance on fixed board layouts\. To evaluate its generalization capability, we introduced a distribution shift in the number of objects present during evaluation\.

The OC model was trained exclusively using boards containing 9 objects and evaluated under two conditions:

- •In\-distribution \(ID\):evaluation with 9 objects
- •Out\-of\-distribution \(OOD\):evaluation with 16 objects

This experiment evaluates whether rules learned from smaller scenes transfer to larger and denser object configurations that were never encountered during training\.

Fig\.[9](https://arxiv.org/html/2608.21372#S5.F9)presents the ID and OOD error rates across rules\. Increasing the number of objects from 9 to 16 produces a measurable degradation in performance for most rules\. However, the extent of degradation varies substantially across rule types\.

Rules based primarily on local feature mappings, such as color\-to\-bucket and shape\-to\-bucket mappings, exhibit comparatively modest increases in error\. In contrast, ordering\-based rules and rules requiring reasoning over larger sets of objects often show substantially larger OOD errors\. These rules require the agent to process longer object sequences and maintain more complex relational structure than was encountered during training\.

These results suggest that the OC representation is capable of transferring learned rule structure to scenes containing substantially more objects than those observed during training\. Nevertheless, increasing scene cardinality introduces additional complexity, particularly for rules that depend on ordering, relational comparisons, or global spatial structure\. Thus, while OC exhibits meaningful generalization beyond the training distribution, its performance remains sensitive to large increases in scene complexity\.

#### 5\.2\.3Observations

The FC and OC representations were evaluated under different forms of distribution shift chosen to challenge the aspects of the environment most closely associated with each representation\.

For the FC representation, generalization was evaluated using previously unseen board positions\. Since FC encodes the board as a spatial feature map, positional changes provide a direct test of whether the learned policy captures rule structure independently of the specific locations observed during training\. The results show substantial degradation under this positional shift, suggesting that FC does not fully abstract the underlying rule structure from board position\. One possible explanation is that the learned policy relies in part on position\-specific patterns observed during training rather than learning a more location\-independent representation of the rule\.

For the OC representation, generalization was evaluated by increasing the number of objects from 9 to 16\. Because OC represents objects individually rather than as fixed board locations, changes in scene cardinality provide a more natural test of its ability to transfer learned rule structure to larger and denser scenes\. The results show that performance generally degrades as scene complexity increases, although the magnitude of this effect varies considerably across rule types\.

Several rules exhibit notable sensitivity under their respective distribution shifts\. In FC, strong degradation is observed for reading\-order and spatially structured rules such asordL1,ordL1\_Nearby,quadNearby, andquadMixed1\. In OC, larger performance drops are typically observed for ordering\-based and relational rules, which require reasoning over larger sets of objects when scene cardinality increases\.

Together, these results characterize how each representation behaves when evaluated under distribution shifts that target its underlying inductive biases\.

### 5\.3Similarity of Shape and Color Rules

The independent\-rule experiments revealed several pairs of rules that are structurally analogous\. In particular, the color\-based and shape\-based matching rules differ only in whether the rule operates on object color or object shape\. Since both attributes are represented using one\-hot encodings of equal dimensionality, one would expect these rule pairs to exhibit similar learning behavior\.

To evaluate whether the corresponding shape and color rules were statistically similar, we applied the non\-parametric Kruskal–Wallis test\.

Kruskal–Wallis TestNull hypothesis \(H0H\_\{0\}\):The medians of the metric distributions are identical across groups\.Alternative hypothesis \(H1H\_\{1\}\):At least one group differs in median from the others\.

The analysis was conducted using the three evaluation metrics introduced earlier:

m⋆,emean⋆,emax⋆\.m^\{\\star\},\\quad e^\{\\star\}\_\{\\mathrm\{mean\}\},\\quad e^\{\\star\}\_\{\\mathrm\{max\}\}\.
We performed two complementary analyses:

1. 1\.consistency across repeated runs of the same rule;
2. 2\.similarity between corresponding shape\-based and color\-based rules\.

#### 5\.3\.1Within\-Rule Comparison

To evaluate the stability of training outcomes, we first examined whether repeated runs of the same rule produced statistically similar results\. For each rule, the Kruskal–Wallis test was applied independently to the five runs associated with each evaluation metric\.

For every rule, at least one of the three metrics failed to reject the null hypothesis,222We checked an enormous number of hypotheses, and so expect to see many false flags of significance\. Thus the conclusions here are informal, rather than statisticalindicating that the runs were not very different along that evaluation dimension\. Although variability remained across some metrics, the results suggest that the learning behavior for a given rule is reasonably stable across repeated training runs\.

#### 5\.3\.2Comparison Between Shape and Color Rules

We next compared corresponding shape\-based and color\-based rules\. These rule pairs differ only in the attribute used by the rule definition while preserving the same underlying structural pattern\. Examples include:

allOfColOrd\_BRKY↔allOfShaOrd\_qcts\\texttt\{allOfColOrd\\\_BRKY\}\\leftrightarrow\\texttt\{allOfShaOrd\\\_qcts\}and

col1Ord\_BRKY↔sha1Ord\_qcts\.\\texttt\{col1Ord\\\_BRKY\}\\leftrightarrow\\texttt\{sha1Ord\\\_qcts\}\.
For each rule pair, the Kruskal–Wallis test was applied separately to all three evaluation metrics\.

To reduce run\-to\-run variance and obtain more stable estimates, we aggregated seven independent sets of experiments, resulting in a total of 35 runs per rule\.

Out of the 30 total comparisons

\(10​rule pairs\)×\(3​metrics\),\(10\\text\{ rule pairs\}\)\\times\(3\\text\{ metrics\}\),29 producedpp\-values greater than 0\.05, indicating no statistically significant difference between the corresponding shape and color rules\.

The only exception occurred for the pair

\(col1Ord\_BRKY, sha1Ord\_qcts\)under the FC model using them⋆m^\{\\star\}metric, where the test yielded

However, the remaining two metrics for this pair produced non\-significant results \(p\>0\.05p\>0\.05\)\. Overall, the analysis indicates that the corresponding shape\-based and color\-based rules behave similarly under the learned representations\.

#### 5\.3\.3Effect of Increasing the Number of Runs

Because non\-parametric statistical tests are sensitive to sample size, we additionally investigated the effect of increasing the number of runs on the stability of the conclusions\.

For the within\-rule analysis, we pooled three independent sets of experiments, increasing the number of runs per rule from 5 to 15\.

Under the original 5\-run setup, 40 out of 60 comparisons \(66%66\\%\) were classified as statistically indistinguishable\. After increasing the sample size to 15 runs, this proportion increased to 50 out of 60 comparisons \(83%83\\%\)\.

These results suggest that increasing the number of runs reduces variance and improves the stability of the statistical conclusions\. At the same time, the original 5\-run analysis still captures the overall similarity trends, while the larger pooled analysis acts as an additional robustness check\.

#### 5\.3\.4Summary

The statistical analysis supports three main conclusions:

1. 1\.Repeated runs of the same rule generally produce consistent learning behavior, with each rule exhibiting at least one metric for which the runs are statistically indistinguishable\.
2. 2\.Corresponding shape\-based and color\-based rules exhibit highly similar performance characteristics\. Across 30 pairwise comparisons, 29 showed no statistically significant difference\.
3. 3\.Increasing the number of runs improves the stability of the statistical conclusions, indicating that some of the variability observed under small sample sizes is attributable to stochastic variation across training runs\.

### 5\.4Transfer Experiments

#### 5\.4\.1Simple\-to\-Simple Transfer

To study transfer behavior between structurally simple rules, we conducted a set of simple\-to\-simple transfer experiments using both the Feature\-Centric \(FC\) and Object\-Centric \(OC\) representations\.

The experiments were performed using four major groups of rules:

1. 1\.Bucket\-ordering rules:cw,ccw,buckets\_0213,buckOrd02,buckOrd13
2. 2\.Quadrant\-based rules:quadNearby,quadMixed1
3. 3\.Feature\-based rules:sm\_csqt,sm\_qcts,cm\_RBKY,cm\_BRYK
4. 4\.Reading\-order rules:ordL1,ordRevOfL1

Note that each group of rules seems to contain only a single “abstract concept\.” For each predecessor rule, five independent training runs were performed starting from different random initializations\. The resulting checkpoints were saved and subsequently used as initialization points for transfer learning experiments on downstream target rules\.

For each predecessor\-target pair, every saved predecessor checkpoint was used to initialize three independent training runs on the target rule\. Consequently, each transfer pair was evaluated using a total of fifteen transfer runs \(five predecessor checkpoints multiplied by three target\-rule training runs per checkpoint\)\.

Although fifteen transfer runs were obtained for each pair, the runs are not fully statistically independent because groups of three runs share the same predecessor checkpoint\. The results should therefore be interpreted as repeated transfer evaluations from a common set of pretrained initializations rather than as fifteen completely independent training runs\.

To compare transfer effects across FC and OC fairly, transfer performance was normalized relative to the baseline convergence cost of the target rule\. Specifically, for predecessor rulerpr\_\{p\}and target rulertr\_\{t\}, we define the normalized transfer ratio:

T​\(rp→rt\)=Mtransfer⋆Mbaseline⋆,T\(r\_\{p\}\\rightarrow r\_\{t\}\)=\\frac\{M^\{\\star\}\_\{\\mathrm\{transfer\}\}\}\{M^\{\\star\}\_\{\\mathrm\{baseline\}\}\},whereMbaseline⋆M^\{\\star\}\_\{\\mathrm\{baseline\}\}denotes the median convergence value when the target rule is trained from scratch\.

Under this formulation:

- •T<1T<1indicates positive transfer,
- •T\>1T\>1indicates negative transfer,
- •T≈1T\\approx 1indicates little or no transfer effect\.

The resulting transfer matrices for FC and OC are shown in Figs\.[10](https://arxiv.org/html/2608.21372#S5.F10)and[11](https://arxiv.org/html/2608.21372#S5.F11)\.

##### Feature\-Centric \(FC\) Transfer Behavior

![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/FC_transfer_heatmap.png)Figure 10:Normalized simple\-to\-simple transfer matrix for the FC representation\. Rows denote predecessor rules and columns denote target rules\. Values below 1 indicate positive transfer, while values above 1 indicate negative transfer relative to baseline training\.The FC representation exhibits highly variable and weakly structured transfer behavior across simple rules\. Although some positive transfer occurs within related rule families, the overall transfer patterns remain inconsistent and highly sensitive to the specific predecessor\-target combination\.

Among bucket\-ordering rules, moderate positive transfer is observed between several related pairs\. In particular,buckets\_0213,ccw, andbuckOrd02often improve one another\. Similarly, the feature\-based rulessm\_csqtandsm\_qctsexhibit strong mutual transfer, indicating that FC can partially reuse low\-level feature activations associated with object categories\.

However, transfer behavior within the FC representation remains notably unstable\. The rulecwexhibits especially irregular behavior: depending on the predecessor checkpoint used for initialization, transfer may either improve or severely degrade convergence\. In several cases, even self\-transfer from pretrainedcwcheckpoints produces worse convergence than training from scratch\.

This instability suggests that FC does not consistently learn a stable reusable abstraction of the underlying rule\. Instead, different runs appear to converge toward distinct positional heuristics or spatial activation patterns, some of which transfer constructively while others interfere strongly with later optimization\.

Negative transfer is particularly strong between structurally incompatible rule families\. For example, quadrant\-based rules such asquadNearbyandquadMixed1produce severe degradation when transferred to ordering\-based targets such ascw\. Likewise, ordering\-based curricula often negatively affect feature\-based targets\.

At the same time, several predecessor\-target pairs with little apparent semantic overlap still produce weak positive transfer\. These effects likely arise from incidental optimization similarities or shared positional statistics rather than from true relational reuse\.

Overall, the FC representation exhibits noisy and weakly organized transfer structure\. Positive transfer is possible when predecessor and target share coarse structural similarities, but the learned representations remain highly checkpoint\-dependent and sensitive to stochastic optimization trajectories\. Oddly, the negative effects, between different rules, provide stronger support for the notion that some abstraction has been learned, while the transfer between similar rules does not provide much support\.

##### Object\-Centric \(OC\) Transfer Behavior

![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/OC_transfer_heatmap.png)Figure 11:Normalized simple\-to\-simple transfer matrix for the OC representation\. Positive transfer is concentrated primarily within structurally related rule families, while negative transfer appears systematically between incompatible relational structures\.In contrast to FC, the OC representation exhibits substantially more coherent and semantically structured transfer behavior\. Positive transfer is concentrated primarily within related rule families, while negative transfer emerges systematically between incompatible relational structures\.

The clearest transfer cluster appears among the bucket\-ordering rules:

cw,ccw,buckets\_0213,buckOrd02,buckOrd13\.\\texttt\{cw\},\\texttt\{ccw\},\\texttt\{buckets\\\_0213\},\\texttt\{buckOrd02\},\\texttt\{buckOrd13\}\.These rules consistently transfer positively to one another, indicating that OC successfully learns reusable relational abstractions associated with bucket\-order structure\.

Similarly, the feature\-based rules

sm\_csqt,sm\_qcts,cm\_RBKY,cm\_BRYK\\texttt\{sm\\\_csqt\},\\texttt\{sm\\\_qcts\},\\texttt\{cm\\\_RBKY\},\\texttt\{cm\\\_BRYK\}also form a coherent transfer group, with strong positive transfer occurring within the family\. This suggests that OC representations effectively reuse object\-level feature relationships across related tasks\.

The quadrant\-based rulesquadNearbyandquadMixed1likewise exhibit moderate positive mutual transfer, although the effect is weaker than in the ordering and feature families\.

Unlike FC, the negative transfer observed in OC is highly systematic rather than noisy\. Ordering\-based predecessors frequently interfere with feature\-based targets, while feature\-based curricula negatively affect ordering\-based tasks\. This behavior suggests that OC learns relatively strong relational abstractions which can either facilitate or conflict with downstream rule structure depending on structural alignment\.

Reading\-order rules \(ordL1,ordRevOfL1\) occupy an intermediate position within the transfer matrix\. They share partial compatibility with ordering\-based tasks due to their sequential structure, but also exhibit substantial negative transfer when combined with unrelated feature\-based rules\.

Another important difference from FC is the overall stability of the OC transfer patterns\. The OC representation produces substantially more consistent transfer behavior across checkpoints, indicating that the learned relational representations are more stable and reproducible across independent training runs\.

Overall, the simple\-to\-simple transfer experiments reveal a strong contrast between the two representations\. FC transfer behavior is highly unstable and dominated by checkpoint\-specific positional heuristics, whereas OC produces coherent family\-level transfer structure consistent with reusable object\-centric relational abstractions\.

#### 5\.4\.2Compound Transfer Experiments

To study compound transfer behavior, we conducted a set of transfer\-learning experiments involving compound rules constructed from simpler component rules\. The experiments were performed using the trial\-list\-based configuration of the GOHR environment\.

Each trial\-list defines a sequence of training phases, where every row specifies the rule encountered during a particular stage of training\. The experiments are designed to analyze whether prior exposure to the component rules of a compound target facilitates faster learning, and how transfer changes when structurally unrelated rules are introduced into the curriculum\.

Consider four independent rulesAA,BB,CC, andDD, and a compound ruleA\+BA\+Bformed by combining the relational constraints of rulesAAandBB\. Two types of transfer\-learning protocols were studied:

1. 1\.Sequential Transfer\.In this setting, the agent is first trained on the two component rules separately, followed by training on their compound rule: A→B→\(A\+B\)\.A\\rightarrow B\\rightarrow\(A\+B\)\.
2. 2\.Partial / Mixed Transfer\.In this setting, the agent is exposed to one relevant component together with one unrelated rule before training on the compound target: A→C→\(A\+B\)\.A\\rightarrow C\\rightarrow\(A\+B\)\.
3. 3\.Unrelated Transfer\.In this setting, the agent is exposed to two unrelated rule before training on the compound target: A→B→\(C\+D\)\.A\\rightarrow B\\rightarrow\(C\+D\)\.

For all transfer phases,ϵ\\epsilonwas reinitialized toϵstart\\epsilon\_\{\\text\{start\}\}at the beginning of each stage to encourage exploration and avoid premature convergence to policies inherited from earlier rules\.

Each experiment was repeated across multiple random seeds\. For every curriculum configuration, the convergence metricM⋆M^\{\\star\}was recorded\. To compare transfer performance across different targets and representations, transfer performance was normalized by the baseline convergence of the target rule trained from scratch:

Mtransfer⋆Mbaseline⋆\.\\frac\{M^\{\\star\}\_\{\\text\{transfer\}\}\}\{M^\{\\star\}\_\{\\text\{baseline\}\}\}\.
Values below11indicate positive transfer \(faster convergence than baseline\), whereas values above11indicate negative transfer\.

#### 5\.4\.3Transfer toordL1\_Nearby

![Refer to caption](https://arxiv.org/html/2608.21372v1/x5.png)Figure 12:Transfer heatmap for the compound ruleordL1\_Nearby\. The left column corresponds to the Feature\-Centric \(FC\) representation and the right column to the Object\-Centric \(OC\) representation\. Each cell reports the normalized transfer ratioMtransfer⋆/Mbaseline⋆M^\{\\star\}\_\{\\mathrm\{transfer\}\}/M^\{\\star\}\_\{\\mathrm\{baseline\}\}\. Values below11indicate positive transfer, while values above11indicate negative transfer\.Fig\.[12](https://arxiv.org/html/2608.21372#S5.F12)shows the normalized transfer behavior of both the FC and OC representations for the compound rule

ordL1\_Nearby\.\\texttt\{ordL1\\\_Nearby\}\.This target rule combines two underlying concepts: reading\-order structure \(ordL1\) and proximity\-based spatial reasoning \(quadNearby\)\. The predecessor curricula span reading\-order, spatial, bucket\-ordering, and feature\-mapping rule families, allowing us to examine how transfer depends on the structural relationship between the curriculum and the target rule\.

The OC representation exhibits extremely strong positive transfer when pretrained on the true component rules\. In particular, the curricula

ordL1→quadNearby\\texttt\{ordL1\}\\rightarrow\\texttt\{quadNearby\}and

quadNearby→ordL1\\texttt\{quadNearby\}\\rightarrow\\texttt\{ordL1\}reduce the normalized convergence ratio to approximately0\.20\.2, corresponding to an almost five\-fold improvement relative to baseline training\.

In contrast, the FC representation exhibits only modest gains from the same curricula\. Most FC transfer ratios remain close to11, indicating that FC reuses comparatively weaker relational structure from predecessor rules\.

Partial and mixed curricula again reveal strong variation in OC\. Some curricula containing one relevant component continue to provide moderate transfer benefits, while others introduce substantial degradation depending on the ordering and structural compatibility of the unrelated rule\.

As shown in Fig\.[12](https://arxiv.org/html/2608.21372#S5.F12), several unrelated curricula produce near\-baseline behavior in FC while still yielding strong positive transfer in OC\. In particular, curricula involvingccw,quadMixed1,cm\_BRYK, andordRevOfL1improve convergence in OC despite not being direct components of the target rule\. This suggests that OC can sometimes reuse partially compatible spatial or ordering abstractions even across structurally distinct rules\.

At the same time, the OC representation remains highly sensitive to incompatible curricula\. Certain unrelated curricula still produce negative transfer exceeding the FC degradation\. Thus, OC demonstrates stronger compositional reuse but also greater sensitivity to the structure and ordering of predecessor rules\.

#### 5\.4\.4Transfer tocm\_RBKY\_cw\_0123

The second compound rule considered is

cm\_RBKY\_cw\_0123,\\texttt\{cm\\\_RBKY\\\_cw\\\_0123\},which combines feature\-to\-bucket mapping with bucket\-ordering structure\. The pretraining curricula include feature\-mapping rules \(cm\_RBKY,cm\_BRYK\), bucket\-ordering rules \(cw\_0123,ccw\), reading\-order rules \(ordL1,ordRevOfL1\), and spatial rules \(quadNearby,quadMixed1\)\. The complete transfer heatmap for this compound rule is provided in Appendix[F](https://arxiv.org/html/2608.21372#A6)\.

Both FC and OC representations exhibit strong positive transfer when the curriculum contains the true components of the target rule\. In particular, the curricula

cm\_RBKY→cw\_0123\\texttt\{cm\\\_RBKY\}\\rightarrow\\texttt\{cw\\\_0123\}and

cw\_0123→cm\_RBKY\\texttt\{cw\\\_0123\}\\rightarrow\\texttt\{cm\\\_RBKY\}produce the strongest improvements\. The effect is especially pronounced in the OC representation, where the normalized convergence ratio decreases to approximately0\.50\.5, indicating that the target rule can be learned in nearly half the number of episodes required from scratch\.

Single\-component pretraining also improves convergence, although the effect is weaker than full\-component pretraining\. Interestingly, pretraining only oncm\_RBKYconsistently provides stronger transfer than pretraining only oncw\_0123\. In FC, pretraining oncm\_RBKYreduces the normalized convergence ratio to approximately0\.130\.13, whereas pretraining only oncw\_0123yields a much weaker improvement\. In OC, the asymmetry is even stronger:cm\_RBKYalone still provides substantial positive transfer, whilecw\_0123alone produces clear negative transfer with ratios exceeding1\.71\.7\.

Partial and mixed curricula reveal strong order sensitivity, particularly in the OC representation\. Curricula containing one correct component together with an unrelated rule often produce widely varying outcomes\. Some combinations still yield moderate improvements, whereas others produce severe negative transfer\. For example,

ordL1→cw\_0123\\texttt\{ordL1\}\\rightarrow\\texttt\{cw\\\_0123\}and

quadNearby→cw\_0123\\texttt\{quadNearby\}\\rightarrow\\texttt\{cw\\\_0123\}lead to large degradation in OC, with normalized transfer ratios exceeding22or33\.

The strongest degradation occurs for unrelated curricula, where the model is pretrained exclusively on structurally incompatible rules\. In these settings, both FC and OC require substantially more episodes to converge than the baseline model trained from scratch\. However, the degradation is consistently larger for OC, where several curricula produce transfer ratios above33\. This supports our tentative conclusion that OC forms strong relational abstractions during pretraining that can either strongly facilitate or strongly interfere with later learning depending on structural alignment\.

To sum up, FC exhibits comparatively conservative transfer behavior\. Although unrelated curricula degrade performance, the transfer ratios generally remain within the1\.21\.2–1\.81\.8range\. OC, by contrast, exhibits both stronger positive transfer and substantially stronger negative transfer, demonstrating significantly higher transfer sensitivity, and suggesting that in some sense the OC representation comes closer to capturing what we think of as conceptual features\.

#### 5\.4\.5Summary

Across both compound\-rule experiments, several consistent transfer behaviors emerge\.

1. 1\.Full\-component pretraining produces the strongest positive transfer\.Compound rules converge substantially faster when the curriculum contains the actual components of the target rule\.
2. 2\.Transfer is strongly order\-dependent\.Even when the same component rules are used, reversing their order can significantly change transfer behavior, especially for the OC representation\.
3. 3\.Single\-component pretraining can still provide meaningful transfer\.However, the magnitude of improvement depends strongly on the structural role of the component within the target rule\.
4. 4\.Unrelated pretraining generally produces negative transfer\.Curricula consisting solely of structurally incompatible rules tend to slow convergence relative to baseline training\.
5. 5\.OC exhibits stronger compositionality and stronger transfer sensitivity\.When structural alignment exists, OC achieves dramatic reductions in convergence time\. However, OC is also substantially more vulnerable to structurally incompatible curricula\.
6. 6\.FC exhibits weaker but more stable transfer behavior\.The FC representation generally produces smaller improvements and smaller degradations, suggesting that its learned representations encodes relational structure more weakly than OC\. As a result, transfer effects tend to be weaker in both the positive and negative directions\.

Overall, the compound\-transfer experiments demonstrate that transfer learning in GOHR depends primarily on structural alignment between predecessor and target rules\. Object\-centric representations enable stronger compositional reuse of relational abstractions, but this same property also makes them substantially more sensitive to curriculum mismatch and ordering effects\.

### 5\.5Multidimensional Scaling Analysis

The data on difficulty provide one dimension for understanding relations between rules\. On the difficulty scale, rules that are “near each other” have something in common, which we have speculated about in the discussion above\. But there are other ways of organizing the rules, based on the data we have collected\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/3d_MDS_fromAbove.png)Figure 13:View of the first two principal dimensions of the MDS embedding ofpp\-values as dissimilarity\. Similarity is defined by being approximately as difficult for the AI system to learn\. We might call this “first order similarity\.” Note the heavy cluster containing many different rules\.![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/3d_MDS.png)Figure 14:Perspective View of the first three principal dimensions of the MDS embedding ofpp\-values as dissimilarity: ”first\-order similarity”\. The figure has been rotated to resolve two clusters that each contain many pointS in the two dimensional view with the first two components\.For example, each data point, characterized by a\(r​u​l​e,a​l​g​o​r​i​t​h​m\)\(rule,algorithm\)pair actually represents the median of five distinct observations, as the algorithm worked on the rule from a cold start five times, That set of five values contains information beyond its median\. For example, the range is approximately a 94% confidence interval for the median of the \(unknown\) distribution of that specific metric for all possible encounters of that rule with that algorithm

A more rigorous way to explore the relation is to apply the Mann\-Whitney test for equality of medians\. When applied to two samples, it gives us the exact probability that the two observed sets of numbers could have come from the same distribution\. The exact probability is calculated by computing the value of the specific Mann\-Whitney U statistic, and asking how many of the values could be more extreme than the one that is observed\. We have calculated thesepp\-values for the similarity between every possible pair of rule sets among the 18 total rule sets \(see Table[6](https://arxiv.org/html/2608.21372#A3.T6)\)\. The resultingpp\-values were recorded in a matrix\.

Specifically, we recognize that thepp\-value is a measure of similarity\. Values close to11are an indication that the sets of values could indeed come from the same distribution, while values close to zero are very unlikely, if both situations have the same distribution of difficulties\. For an initial look at the structure of the data, we compute a dissimilarity matrixDi,j=1−pM​a​n​n​W​h​i​t​n​e​y​\(i,j\)D\_\{i,j\}=1\-p\_\{MannWhitney\}\(i,j\)\. We then used the R statistIcal environment to transform this into a lower diagonal distance array\. We then used the basic R tool for multidimensional scaling, to look for a3−d​i​m​e​n​i​o​n​a​l3\-dimenionalembedding of the data into a Euclidean space\. The results are a rotatable plot, The first two principal dimensions of the analysis are shown in Fig\.[13](https://arxiv.org/html/2608.21372#S5.F13)\. It is easy to see that there are a few clusters, but each of them seems to contain “too many” different rules\.

We continued the analysis, looking at a view using the third dimension as the vertical axis\. In that view, shown in Fig\.[14](https://arxiv.org/html/2608.21372#S5.F14)it is clear that the large cluster is resolved by differing values along the \(now vertical\) third axis\. Future work will address the question of whether all of these clusters make sense both in terms of the specifics of the algorithm and in terms of the underlying concepts within the rules\.

### 5\.6Transfer Geometry and Clustering Structure

To further analyze the structure of transfer learning relationships, we constructed a transfer\-difference matrix from the simple\-to\-simple transfer experiments\. This analysis builds upon the predecessor\-conditioned transfer matrix presented earlier\. Whereas the previous matrix summarized absolute convergence performance after transfer, the present matrix measures the change in convergence relative to baseline learning, thereby isolating the effect of transfer itself\.

Δ​\(i,j\)=Mbase⋆​\(ri\)−Mtransfer⋆​\(ri←rj\),\\Delta\(i,j\)=M^\{\\star\}\_\{\\mathrm\{base\}\}\(r\_\{i\}\)\-M^\{\\star\}\_\{\\mathrm\{transfer\}\}\(r\_\{i\}\\leftarrow r\_\{j\}\),
whererir\_\{i\}is the target \(successor\) rule andrjr\_\{j\}is the predecessor rule used for pretraining\.Mb​a​s​e⋆​\(ri\)M^\{\\star\}\_\{base\}\(r\_\{i\}\)is the median convergence metric obtained when learningrir\_\{i\}without a predecessor rule, whileMt​r​a​n​s​f​e​r⋆​\(ri←rj\)M^\{\\star\}\_\{transfer\}\(r\_\{i\}\\leftarrow r\_\{j\}\)is the median convergence metric obtained when learningrir\_\{i\}after training onrjr\_\{j\}\.

Thus, each row characterizes how a target rule responds to different predecessor curricula, while each column characterizes how a predecessor rule influences learning across multiple successor rules\.

Positive values ofΔ​\(i,j\)\\Delta\(i,j\)indicate positive transfer, meaning that pretraining onrjr\_\{j\}reduced the median convergence time for learningrir\_\{i\}\. Negative values indicate interference or negative transfer, meaning that pretraining onrjr\_\{j\}increased the median convergence time\.

Because not every predecessor\-successor pair was experimentally evaluated, the resulting matrix contained missing entries\. For the clustering and multidimensional\-scaling analyses, each rule was represented by its pattern of transfer relationships with the other rules\. Missing entries were therefore replaced using mean\-based imputation so that these transfer patterns could be compared consistently while preserving either predecessor\-rule or successor\-rule statistics\. Two complementary imputations were therefore used:

1. 1\.Column\-mean imputationpreserved predecessor\-rule statistics by replacing missing values in each column with the column mean\. The resulting matrix was used for row clustering\.
2. 2\.Row\-mean imputationpreserved successor\-rule statistics by replacing missing values in each row with the row mean\. The transpose of this matrix was used for column clustering\.

The resulting analyses provide a second\-order notion of similarity among rules\. Earlier analyses compared rules directly in terms of learning difficulty or convergence performance\. Here, rules are compared through their patterns of transfer interactions with other rules\. Two rules are therefore considered similar when they exhibit similar transfer relationships, even if their standalone learning difficulty differs substantially\.

Hierarchical clustering and multidimensional scaling \(MDS\) were then applied separately to the FC and OC representations\.

#### 5\.6\.1Row clustering: similarity of successor\-rule behavior\.

Row clustering groups rules according to how they respond to predecessor experience\. In this view, two rules cluster together when they are similarly helped or hindered by the same predecessor rules\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/clustering_oc_row.png)Figure 15:Hierarchical clustering of OC successor\-rule transfer behavior\. Rules that cluster together are affected similarly by predecessor curricula\.The OC row clustering \(Fig\.[15](https://arxiv.org/html/2608.21372#S5.F15)\) reveals several coherent structural groups\. The spatial rules

quadNearby,quadMixed1\\texttt\{quadNearby\},\\quad\\texttt\{quadMixed1\}form a stable early cluster, indicating that they are influenced similarly by predecessor experience\. Likewise, the feature\-based rules

cm\_RBKY,cm\_BRYK\\texttt\{cm\\\_RBKY\},\\quad\\texttt\{cm\\\_BRYK\}appear close together, suggesting that OC appears to learn transferable feature\-to\-bucket abstractions that generalize across related mappings\.

The bucket\-ordering rules

cw,ccw,buckets\_0213\\texttt\{cw\},\\quad\\texttt\{ccw\},\\quad\\texttt\{buckets\\\_0213\}also form a relatively coherent group in OC\. This is consistent with the earlier transfer\-learning results showing strong positive transfer within ordering\-based rule families\.

The FC row clustering exhibits a noticeably different structure\. While some local groupings remain visible, the clusters are less stable and more weakly separated\. In particular,cwbehaves as an outlier in several FC analyses, appearing far from the remaining ordering rules despite belonging conceptually to the same family\. This observation is consistent with the earlier finding that FC transfer involvingcwwas unusually unstable and highly checkpoint\-dependent\.

Overall, the FC row structure appears less compact and less consistently organized, suggesting that FC does not organize successor\-rule sensitivity into a strongly coherent transfer geometry\.

#### 5\.6\.2Column clustering: similarity of predecessor\-rule influence\.

Column clustering groups predecessor rules according to how they affect future learning across many target rules\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/mds_oc_column.png)Figure 16:3\-dimensional MDS embedding of OC successor\-rule transfer behavior\. Distances reflect similarity in how rules respond to predecessor experience\.In OC, the predecessor influence structure is relatively well organized\. Ordering rules, spatial rules, and feature\-based rules each tend to form recognizable clusters in the MDS embeddings and dendrograms\. The corresponding OC column MDS visualization \(Fig\.[16](https://arxiv.org/html/2608.21372#S5.F16)\) shows clear spatial separation between several rule families, particularly between ordering\-based and spatial rules\.

The FC column structure exhibits weaker geometric separation between rule families\. Although some related rules still appear near one another, the embeddings show greater overlap between rule families, indicating that predecessor\-rule influence in FC is less systematically organized\.

An especially notable observation is that the FC column clustering places several structurally unrelated rules near one another\. This observation, that transfer effects in FC are less consistently aligned with the underlying conceptual structure of the rules reinforces our earlier data suggesting that the FC representation does not reflect conceptual structure of rules as well as OC does\.

#### 5\.6\.3Comparison between row and column structures\.

To examine whether predecessor\-rule influence and successor\-rule sensitivity follow similar organizational patterns, the row and column dendrograms were compared using tanglegrams together with quantitative similarity measures\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/oc_row_vs_oc_column.png)Figure 17:Tanglegram comparing OC row clustering \(successor\-rule behavior\) and OC column clustering \(predecessor\-rule behavior\)\.The OC row\-vs\-column comparison \(Fig\.[17](https://arxiv.org/html/2608.21372#S5.F17)\) shows comparatively strong agreement between the two clustering structures\. Several major rule families remain grouped together across both trees, particularly the feature\-based rules \(cm\_RBKY,cm\_BRYK\), the spatial rules \(quadNearby,quadMixed1\), and the ordering\-related rules\. The corresponding similarity measures are:

Adjusted Rand Index=0\.633,Baker’s gamma=0\.542\.\\text\{Adjusted Rand Index\}=0\.633,\\qquad\\text\{Baker's gamma\}=0\.542\.
These values indicate that, in OC, rules that exhibit similar transfer effects as predecessors also tend to display similar sensitivity as successor rules\. In other words, the OC representation develops a relatively coherent and self\-consistent transfer\-learning geometry\.

In contrast, the FC row\-vs\-column comparison exhibits substantially weaker agreement\. The corresponding tanglegram contains noticeably more crossing structure and weaker alignment between clusters, indicating less consistency between predecessor and successor organization\. Quantitatively, the similarity values are:

Adjusted Rand Index=0\.168,Baker’s gamma=0\.039\.\\text\{Adjusted Rand Index\}=0\.168,\\qquad\\text\{Baker's gamma\}=0\.039\.
This suggests that FC predecessor influence and successor sensitivity are organized according to less stable and less unified transfer relationships\.

Cross\-representation comparisons further highlight the difference between FC and OC\. The FC\-vs\-OC row comparison shows moderate structural similarity:

Baker’s gamma=0\.445,\\text\{Baker's gamma\}=0\.445,
indicating that both representations capture some common high\-level relationships regarding how rules are influenced by predecessor experience\.

Unsuprisingly, the FC\-vs\-OC column comparison shows very weak agreement:

Adjusted Rand Index=−0\.035,Baker’s gamma=−0\.063\.\\text\{Adjusted Rand Index\}=\-0\.035,\\qquad\\text\{Baker's gamma\}=\-0\.063\.
This indicates that FC and OC organize predecessor\-rule influence in substantially different ways, despite some overlap in successor\-rule organization\.

The MDS visualizations support these observations\. In the OC embeddings, several rule families form relatively compact and well\-separated groups\. In particular, the feature\-mapping rules and spatial rules remain consistently separated from ordering\-based rules across both row and column analyses\. The OC predecessor\-space MDS also placesquadNearbyandquadMixed1in close proximity, reflecting their similar transfer influence on successor learning\.

The FC embeddings, by contrast, are less clearly separated and exhibit greater distortion in cluster organization\. Some local relationships remain visible, but the overall geometry is less regular and more strongly affected by individual rules\. In particular,cwfrequently appears isolated or inconsistently positioned across analyses, consistent with the unstable transfer behavior observed earlier in the simple transfer experiments\.

Overall, these analyses suggest that the OC representation develops a more coherent and structured transfer\-learning geometry than FC\. Structurally related rules repeatedly cluster together across dendrograms, MDS embeddings, and tanglegram comparisons, and the resulting organization remains comparatively stable between predecessor and successor views\. FC transfer behavior, while still containing some local structure, appears more diffuse and internally inconsistent\. These findings align with the broader transfer\-learning results, suggesting that OC forms stronger relational abstractions that support more systematic transfer across rules\.

## 6Human Learning Analysis via Pseudo\-Bot Modeling

### 6\.1Motivation

In addition to studying reinforcement learning agents in GOHR, we also investigated patterns of human learning behavior in two\-player settings involving varying levels of pseudo\-bot assistance\. The goal of this analysis was to determine whether sequences of player actions contain statistically identifiable behavioral signatures that distinguish purely human play from bot\-assisted play\.

Rather than modeling full board states or explicit strategies, the analysis focuses on the temporal structure of player outcomes during gameplay\. This allows us to examine whether differences in consistency, recovery from errors, and progression dynamics can be used to characterize human versus assisted behavior\.

### 6\.2Dataset and Problem Setup

The analysis was conducted using detailed gameplay transcript files collected from GOHR experiments \( see\[[3](https://arxiv.org/html/2608.21372#bib.bib28)\]\)\. Each transcript contains move\-level information with columns of the form:

\#playerId, trialListId, seriesNo, ruleId, episodeNo, episodeId, moveNo, timestamp, mover, reactionTime, objectType, objectId, y, x, bucketId, by, bx, code, objectCnt

The data consists of two\-player games in which one side may receive varying levels of pseudo\-bot assistance\. The objective is to classify whether a sequence of gameplay outcomes is more consistent with bot\-assisted or unassisted human behaviour\.

For preprocessing, moves were grouped by

\(playerId,ruleId,mover\),\(\\texttt\{playerId\},\\texttt\{ruleId\},\\texttt\{mover\}\),and sorted chronologically using

\(episodeNo,episodeId,moveNo\)\.\(\\texttt\{episodeNo\},\\texttt\{episodeId\},\\texttt\{moveNo\}\)\.
Each resulting sequence corresponds to the history of one player side interacting with a particular rule\.

### 6\.3Behavioral Modeling Approach

#### 6\.3\.1Outcome Sequence Representation

The raw gameplay stream was converted into a filtered sequence of move outcomes based on changes in the number of remaining objects\.

Letntn\_\{t\}denote the object count after movett, andnt−1n\_\{t\-1\}the object count before the move\.

For two\-player games, accepted and denied outcomes can be inferred from changes in the number of remaining objects and turn transitions\. Specifically:

- •If the move is considered successful: whereAAdenotes an accepted or correct move\.
- •If and the turn passes to the opposing player, the move is considered unsuccessful: whereDDdenotes a denied or incorrect move\.
- •If and the same player continues, the move is treated as an intermediate state and excluded from the sequence\.
- •If the event is treated as a boundary or reset condition and excluded\.

This preprocessing converts the raw interaction stream into a filtered outcome sequence:

o1,o2,…,oK,ot∈\{A,D\}\.o\_\{1\},o\_\{2\},\\dots,o\_\{K\},\\quad o\_\{t\}\\in\\\{A,D\\\}\.
For each timestep, two contextual variables were additionally computed\.

##### Previous Outcome\.

The previous valid outcome is represented byot−1o\_\{t\-1\}\. For the first move in a sequence, a special start tokenSSis used\.

##### Success\-Rate Context\.

The cumulative success rate prior to timestepttis defined as:

rt−1=∑i=1t−1𝟏​\(oi=A\)t−1\.r\_\{t\-1\}=\\frac\{\\sum\_\{i=1\}^\{t\-1\}\\mathbf\{1\}\(o\_\{i\}=A\)\}\{t\-1\}\.
The cumulative success\-rate value is discretized into bins:

st−1∈\{0​\_​25,25​\_​50,50​\_​75,75​\_​100\}\.s\_\{t\-1\}\\in\\\{0\\\_25,\\,25\\\_50,\\,50\\\_75,\\,75\\\_100\\\}\.
The first timestep uses a dedicatedstartbin\.

#### 6\.3\.2Markov / Backoff Model

Separate probabilistic models were trained for the two behavioral classes:

c∈\{unassisted,assisted\}\.c\\in\\\{\\texttt\{unassisted\},\\texttt\{assisted\}\\\}\.
The primary quantity estimated by the model is

P​\(ot∣ot−1,st−1,c\),P\(o\_\{t\}\\mid o\_\{t\-1\},s\_\{t\-1\},c\),which represents the probability of the next outcome conditioned on the previous outcome, the cumulative success\-rate bin, and the behavioral class\.

From the training data, we computed

Nc​\(ot−1,st−1,ot\),N\_\{c\}\(o\_\{t\-1\},s\_\{t\-1\},o\_\{t\}\),the number of times outcomeoto\_\{t\}follows context\(ot−1,st−1\)\(o\_\{t\-1\},s\_\{t\-1\}\)within classcc\.

Similarly,

Nc​\(ot−1,st−1\)N\_\{c\}\(o\_\{t\-1\},s\_\{t\-1\}\)counts the total number of occurrences of the context itself\.

The full conditional probability was estimated using Laplace smoothing:

Pfull=Nc​\(ot−1,st−1,ot\)\+αNc​\(ot−1,st−1\)\+2​α,P\_\{\\text\{full\}\}=\\frac\{N\_\{c\}\(o\_\{t\-1\},s\_\{t\-1\},o\_\{t\}\)\+\\alpha\}\{N\_\{c\}\(o\_\{t\-1\},s\_\{t\-1\}\)\+2\\alpha\},with

To reduce sparsity, two lower\-order backoff probabilities were additionally estimated\.

##### Previous\-outcome model\.

Pprev=P​\(ot∣ot−1,c\)\.P\_\{\\text\{prev\}\}=P\(o\_\{t\}\\mid o\_\{t\-1\},c\)\.

##### Global class model\.

Pglobal=P​\(ot∣c\)\.P\_\{\\text\{global\}\}=P\(o\_\{t\}\\mid c\)\.
Both lower\-order probabilities were estimated using the same Laplace smoothing procedure \(α=1\\alpha=1\) as the full model\.

The final probability estimate was computed as a weighted interpolation:

P​\(ot∣ot−1,st−1,c\)=0\.7​Pfull\+0\.2​Pprev\+0\.1​Pglobal\.P\(o\_\{t\}\\mid o\_\{t\-1\},s\_\{t\-1\},c\)=0\.7P\_\{\\text\{full\}\}\+0\.2P\_\{\\text\{prev\}\}\+0\.1P\_\{\\text\{global\}\}\.
The interpolation weights were selected after limited empirical experimentation with alternative weight combinations\. The final values place most of the probability mass on the full contextual model while retaining smaller contributions from the lower\-order backoff models\. This combination produced strong classification performance while providing robustness when exact contexts occurred infrequently in the training data\.

#### 6\.3\.3Sequence Classification

Given the firstKKvalid outcomes of a test sequence

o1,o2,…,oK,o\_\{1\},o\_\{2\},\\dots,o\_\{K\},the model computes the log\-likelihood under each behavioral class:

log⁡Lc=∑t=1Klog⁡P​\(ot∣ot−1,st−1,c\)\.\\log L\_\{c\}=\\sum\_\{t=1\}^\{K\}\\log P\(o\_\{t\}\\mid o\_\{t\-1\},s\_\{t\-1\},c\)\.
A classification score is then defined as

score=log⁡Lassisted−log⁡Lunassisted\.\\text\{score\}=\\log L\_\{\\text\{assisted\}\}\-\\log L\_\{\\text\{unassisted\}\}\.
The final prediction is:

y^=\{assistedif score\>0unassistedotherwise\.\\hat\{y\}=\\begin\{cases\}\\texttt\{assisted\}&\\text\{if score\}\>0\\\\ \\texttt\{unassisted\}&\\text\{otherwise\}\.\\end\{cases\}
Thus, the model evaluates whether the temporal progression of outcomes is statistically more consistent with unassisted or bot\-assisted human behavior\.

### 6\.4Evaluation Protocol

The classifier was evaluated separately for multiple pseudo\-bot assistance levels:

ph4,ph8,ph12,ph16\.\\texttt\{ph4\},\\texttt\{ph8\},\\texttt\{ph12\},\\texttt\{ph16\}\.
For each setting, the model was trained on outcome sequences extracted from gameplay transcripts and evaluated using sequence\-level classification metrics\.

Performance was measured using accuracy, precision, recall, F1\-score, and specificity\.

Both test\-only and combined train\+test evaluations were examined to analyze the stability of the learned behavioral distributions\.

### 6\.5Results

#### 6\.5\.1Bot\-Assisted Side Classification

The Markov/backoff classifier achieved above\-chance discrimination between unassisted and pseudo\-bot\-assisted gameplay across all evaluated assistance levels, although performance varied substantially across conditions\.

Table[4](https://arxiv.org/html/2608.21372#S6.T4)summarizes the classification performance across different pseudo\-bot configurations\. The code “phX” refers to the pseudo\-half\-life of the bot\. For ph4, the chance that the suggestion is wrong decreses by 50% every 4 suggestions\. The strongest performance was obtained for theph4\(fastest\-learning\) setting, which achieved an accuracy of93\.5%93\.5\\%and an F1\-score of0\.9640\.964under the test\-only evaluation setting\.

Lower levels of assistance remained distinguishable, although performance varied depending on the behavioral overlap between human and assisted play\. Across most configurations, F1\-scores ranged approximately between0\.550\.55and0\.860\.86, demonstrating that temporal outcome patterns contain meaningful information for identifying assisted gameplay\.

Table 4:Classification performance of the Markov/backoff model for distinguishing human and pseudo\-bot\-assisted gameplay\.
#### 6\.5\.2Effect of Number of Moves

Because the classifier operates on the firstKKvalid outcomes of each sequence, its predictions are based on a finite amount of behavioral evidence\. Longer sequences provide additional observations of outcome transitions and success\-rate dynamics, which may improve the stability of the likelihood estimates\. However, the relationship between sequence length and classification performance was not evaluated systematically in the present study\.

Early portions of gameplay often contain exploratory behavior shared between humans and assisted agents, while longer sequences expose differences in consistency and adaptation patterns\.

#### 6\.5\.3Rule\-Level Differences

Qualitatively, for some rules it appeared more difficult to distinguish assistance from non\-assistance, than for others\. One possible explanation is that rules with more complex spatial or relational dependencies produce more variable behavioral sequences, thereby reducing separability between assisted and unassisted play\. However, this hypothesis was not tested directly in the present analysis\.

At the same time, simpler rules often produced more stable outcome dynamics, making bot\-assisted behavior easier to distinguish from purely human play\.

### 6\.6Discussion

The results suggest that temporal patterns of successes and failures contain sufficient statistical structure, in this dataset, to distinguish between assisted and unassisted gameplay behavior\.

Importantly, the model does not rely on explicit board\-state representations or handcrafted strategic features\. Instead, it captures behavioral differences through short\-term outcome transitions and cumulative performance context\.

The interpolation\-based Markov/backoff formulation provides a simple but effective mechanism for handling sparse behavioral contexts while remaining interpretable\.

Several limitations remain\. The current model uses only binary outcome sequences and ignores richer information such as reaction times, object identities, spatial board configurations, and longer\-term strategic dependencies\. In addition, the discretization of success rates introduces information loss and may limit sensitivity to subtle behavioral changes\.

Nevertheless, the analysis demonstrates that relatively simple probabilistic sequence models can recover meaningful distinctions between unassisted and assisted gameplay behavior in GOHR\.

## 7Overall Discussion and Conclusion

### 7\.1Key Findings

This work investigated hidden\-rule learning in the Game of Hidden Rules \(GOHR\) through reinforcement learning, transfer learning, transfer\-geometry analysis, and pseudo\-bot\-assisted human learning analysis\. Across these experiments, several consistent patterns emerged regarding the role of representation, rule structure, and structural transfer\.

One of the clearest findings is that state representation strongly influences hidden\-rule learning behavior\. Across independent\-rule learning, transfer learning, clustering analyses, and transfer\-geometry experiments, the Object\-Centric \(OC\) representation consistently produced more stable and interpretable behavior than the Feature\-Centric \(FC\) representation\. Systems using the OC generally learned rules more efficiently, exhibited stronger positive transfer within related rule families, and formed clearer geometric organization in transfer space\. In contrast, FC transfer behavior was often more diffuse and sensitive to individual training checkpoints\. Thecwrule, in particular, exhibited unstable FC transfer behavior despite belonging conceptually to the ordering\-rule family, suggesting that FC representations may rely more heavily on incidental spatial training dynamics rather than stable relational abstractions\.

The experiments further demonstrated that learning difficulty depends strongly on the structural properties of the underlying rule\. Feature\-mapping rules were generally easier to learn, while ordering\-based, spatial, and compound rules produced substantially greater learning difficulty\. The rule\-property analysis showed that increasing relational abstraction and compositional complexity leads to slower convergence and higher variability\. Compound rules combining multiple relational constraints were consistently among the most difficult tasks for both representations\.

Transfer\-learning experiments revealed that successful transfer depends primarily on structural alignment between predecessor and successor rules\. Positive transfer was strongest when predecessor rules shared meaningful relational components with the target rule\. In the compound\-transfer experiments, full\-component pretraining consistently accelerated learning, while unrelated or partially related curricula often produced weaker transfer or interference\. The simple\-to\-simple transfer experiments showed similar trends, particularly within ordering\-based and feature\-based rule families\. These findings suggest that effective curricula for hidden\-rule learning should preserve structural continuity between tasks rather than permitting arbitrary sequencing\.

The transfer\-geometry analyses provided additional insight into how the two representations organize relational knowledge\. Hierarchical clustering, multidimensional scaling \(MDS\), and tanglegram comparisons showed that OC develops a comparatively coherent transfer\-learning geometry\. Structurally related rules repeatedly formed stable clusters across row clustering, column clustering, and MDS embeddings\. Moreover, predecessor\-rule influence and successor\-rule sensitivity exhibited relatively strong agreement in OC, indicating that OC develops internally consistent transfer structures\. FC exhibited substantially weaker alignment between predecessor and successor organization, suggesting that its transfer relationships are less stable and less systematically structured\.

The pseudo\-bot\-assisted human\-learning analysis demonstrated that temporal patterns of successes and failures contain sufficient statistical structure to distinguish assisted and non\-assisted gameplay\. Even without explicit board\-state information, the Markov/backoff sequence model was able to capture behavioral differences using short\-term outcome transitions and cumulative success\-rate context\. Several observations from the reinforcement\-learning experiments parallel the human\-learning results\. In both settings, ordering\-based and spatial rules produced more complex learning dynamics than direct feature\-mapping rules, and structurally related tasks exhibited stronger transfer relationships\. Although exploratory, these experiments suggest that GOHR provides a useful unified framework for studying both machine and human hidden\-rule learning behavior\.

Overall, the results suggest that explicit object\-level representations better support relational abstraction, transferable learning, and coherent transfer organization in hidden\-rule environments\. More broadly, the experiments indicate that hidden\-rule learning is governed not only by task difficulty, but also by the structural relationships between rules and the representations used to encode them\.

### 7\.2Limitations

Several limitations should be considered when interpreting the results of this work\.

First, many experiments were conducted using a relatively small number of independent runs due to computational constraints\. Although median\-based aggregation and ratio\-based transfer analyses reduced some variability, additional runs would improve statistical robustness and confidence\. In addition, our exploration of the data included so many comparisons \(literally, hundreds\) that even with the most sophisticated adjustments for multiple comparisons, we cannot assert that any of our conclusions have been established “with such and such degree of confidence/” All of this work is exploratory, to help define a path toward principled exploration of the “geometry of rules and concepts\.” \[See Section[7\.3](https://arxiv.org/html/2608.21372#S7.SS3)\]

Second, the rule suite used in this work represents only a subset of the possible hidden\-rule space in GOHR\. Additional rule families, larger compositional structures, and more diverse relational dependencies may reveal transfer behaviors not exposed in the current experiments\.

Third, for the MDS analysis, the data in the transfer matrices are missing many predecessor\-successor combinations, requiring imputation during clustering and MDS analyses\. Although complementary row\-mean and column\-mean imputations were used to preserve needed statistical structures, the resulting inferred geometry may still be influenced by incomplete transfer coverage\. In companion studies we have worked with complete transfer data, supporting assessment of the effects of this imputation\.

Fourth, the reinforcement\-learning agents that we have studied rely on relatively simple exploration strategies and memory mechanisms\. More advanced exploration methods or longer\-horizon memory architectures may improve learning efficiency for difficult relational tasks\.

We did experiment with using a modular Gymnasium\-compatible GOHR environment with the goal of making it easy to let others use the GOHR as a tool for future studies on relational reasoning, abstraction learning, curriculum learning, transfer learning, and human\-agent comparison in hidden\-rule environments\. Our specific refactoring did not retain enough information about the history of boards and moves, and did not yield performance comparable to our legacy system, See Appendix[A](https://arxiv.org/html/2608.21372#A1)

Finally, the human learning analysis focuses on classification of human learning into bot\-assisted vs unassisted performance, rather than on direct modeling of human reasoning processes\. The current model captures short\-term outcome dynamics but does not incorporate richer behavioral information such as reaction times, spatial reasoning strategies, or long\-term planning behavior\.

### 7\.3Future Work

Several directions remain for future research\.

One important direction is expanding the GOHR rule suite to include more complex relational, temporal, and compositional rules\. A larger and more diverse rule space would allow deeper investigation of abstraction, curriculum learning, and transfer geometry\.

Future work could also explore stronger reinforcement\-learning approaches, including improved exploration strategies, memory\-augmented architectures, offline reinforcement learning, or model\-based methods\. These approaches may improve learning efficiency for difficult relational rules and reduce transfer instability\.

The transfer\-learning framework can be extended further through adaptive curricula, automated curriculum generation, and transfer\-aware task sequencing\. Such methods may help identify optimal training orders for accelerating hidden\-rule learning\.

Another promising direction is improving statistical robustness through larger experimental sweeps and broader predecessor\-successor coverage\. More complete transfer matrices would provide stronger clustering structure and more reliable transfer geometries\.

The human\-learning analysis could also be extended substantially\. Future work may incorporate richer behavioral features, direct human strategy modeling, cognitive comparisons, or human\-in\-the\-loop learning experiments\. Comparing learned reinforcement\-learning representations against human behavioral abstractions may provide additional insight into how relational rules are internally represented across biological and artificial systems\.

## 8Acknowledgments

The systems used here to study the GOHR were developed at the University of Wisconsin\. Support for their research was provided by the University of Wisconsin\-Madison Office of the Vice Chancellor for Research and Graduate Education with funding from the Wisconsin Alumni Research Foundation, and by the National Science Foundation under Grant No\. 2041428\. Additional participants in that development are acknowledged in\[[7](https://arxiv.org/html/2608.21372#bib.bib11)\]\. The specific work reported here was supported in part by the Defense Advanced Research Projects Agency \(HR00112420363\); the content of this report does not necessarily reflect the position or the policy of the Government; and no official endorsement should be inferred\. Distribution Statement: Approved for public release; distribution is unlimited\. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author\(s\) and do not necessarily reflect the views of the sponsors\.

## References

- \[1\]V\. Bier, P\. B\. Kantor, G\. Lupyan, and X\. Zhu\(2019\)Can We Distinguish Machine Learning from Human Learning?\.Technical reportarXiv\.Note:\_eprint: 1910\.03466Comment: 14pp\. 5 fig\. Working PaperExternal Links:[Link](http://arxiv.org/abs/1910.03466)Cited by:[§2\.1](https://arxiv.org/html/2608.21372#S2.SS1.p1.1),[§2\.3](https://arxiv.org/html/2608.21372#S2.SS3.p1.1),[footnote 1](https://arxiv.org/html/2608.21372#footnote1)\.
- \[2\]V\. Bier, P\. Kantor, and V\. Menkov\(2024\)GOHR Home Page\.External Links:[Link](https://rulegame.wisc.edu/)Cited by:[§2\.1](https://arxiv.org/html/2608.21372#S2.SS1.p1.1),[§2\.3](https://arxiv.org/html/2608.21372#S2.SS3.p1.1)\.
- \[3\]J\. Feldman, L\. Gallos, H\. Wang, V\. Menkov, and P\. B\. Kantor\(2026\-05\)Benefits of co\-learning with an ai agent\.PsyArXiv\.External Links:[Link](https://osf.io/preprints/psyarxiv/bx5q4_v2)Cited by:[§6\.2](https://arxiv.org/html/2608.21372#S6.SS2.p1.1)\.
- \[4\]C\. Mathew, W\. Wang, J\. Feldman, L\. K\. Gallos, P\. B\. Kantor, V\. Menkov, and H\. Wang\(2025\)Toward a metrology for artificial intelligence: hidden\-rule environments and reinforcement learning\.arXiv preprint arXiv:2509\.06213\.Cited by:[item 2](https://arxiv.org/html/2608.21372#S1.I3.i2.p1.1)\.
- \[5\]V\. Mnih, A\. P\. Badia, M\. Mirza, A\. Graves, T\. Lillicrap, T\. Harley, D\. Silver, and K\. Kavukcuoglu\(2016\)Asynchronous methods for deep reinforcement learning\.InInternational conference on machine learning,pp\. 1928–1937\.Cited by:[§3\.6](https://arxiv.org/html/2608.21372#S3.SS6.p1.1)\.
- \[6\]E\. Pulick, S\. Bharti, Y\. Chen, V\. Menkov, Y\. Mintz, P\. Kantor, and V\. M\. Bier\(2022\-07\)The Game of Hidden Rules: A New Kind of Benchmark Challenge for Machine Learning\.Technical reportTechnical ReportarXiv:2207\.10218,arXiv\.Note:arXiv:2207\.10218 \[cs\] type: articleComment: 9 pages, 5 figures\. Additional documentation information available at http://sapir\.psych\.wisc\.edu:7150/w2020/captive\.htmlExternal Links:[Link](http://arxiv.org/abs/2207.10218),[Document](https://dx.doi.org/10.48550/arXiv.2207.10218)Cited by:[item 1](https://arxiv.org/html/2608.21372#S1.I3.i1.p1.1),[§2\.1](https://arxiv.org/html/2608.21372#S2.SS1.p1.1),[§2\.3](https://arxiv.org/html/2608.21372#S2.SS3.p1.1)\.
- \[7\]E\. Pulick, V\. Menkov, Y\. D\. Mintz, P\. B\. Kantor, and V\. M\. Bier\(2024\)Comparing Reinforcement Learning and Human Learning with the Game of Hidden Rules\.IEEE Access,pp\. 65362–65372\.External Links:ISSN 2169\-3536,[Document](https://dx.doi.org/10.1109/ACCESS.2024.3395249)Cited by:[§1\.2](https://arxiv.org/html/2608.21372#S1.SS2.p1.1),[§2\.1](https://arxiv.org/html/2608.21372#S2.SS1.p1.1),[§2\.3](https://arxiv.org/html/2608.21372#S2.SS3.p1.1),[§8](https://arxiv.org/html/2608.21372#S8.p1.1)\.
- \[8\]M\. Towers, A\. Kwiatkowski, J\. Balis, G\. De Cola, T\. Deleu, M\. Goulão, K\. Andreas, M\. Krimmel, A\. Kg, R\. Perez\-Vicente,et al\.\(2026\)Gymnasium: a standard interface for reinforcement learning environments\.Advances in Neural Information Processing Systems38\.Cited by:[Appendix A](https://arxiv.org/html/2608.21372#A1.p1.1),[item 7](https://arxiv.org/html/2608.21372#S1.I3.i7.p2.1)\.
- \[9\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.Advances in neural information processing systems30\.Cited by:[§3\.5\.2](https://arxiv.org/html/2608.21372#S3.SS5.SSS2.p1.1)\.

## Appendix AGymnasium\-Compatible GOHR Environment

This appendix describes an experimental refactoring of GOHR into a Gymnasium\-compatible environment\[[8](https://arxiv.org/html/2608.21372#bib.bib31)\]\. The primary motivation was to improve software modularity, support modern reinforcement\-learning libraries, and separate environment dynamics from state\-representation construction\.

The refactoring also provided an opportunity to investigate whether GOHR could be represented using a more conventional reinforcement\-learning interface, in which observations contain only the current environment state and temporal history is modeled by the learning algorithm rather than by the environment itself\.

However, preliminary experiments indicated that this simplified observation design did not reproduce the learning behavior observed with the legacy GOHR implementation, particularly for rules that depend on information from previous states and actions\. As a result, all experiments reported in the main body of this thesis use the legacy history\-augmented observation pipeline\. The Gymnasium implementation is therefore presented as both a software contribution and a case study illustrating the importance of the legacy observation design for GOHR\.

### A\.1Design Goals

The refactored environment was designed with the following objectives:

- •Provide a standard Gymnasium\-compatible interface\.
- •Decouple environment dynamics from state\-representation construction in order to evaluate alternative observation designs\.
- •Support both Feature\-Centric \(FC\) and Object\-Centric \(OC\) representations through a unified API\.
- •Enable integration with modern reinforcement learning libraries such as Stable\-Baselines3\.
- •Improve maintainability, extensibility, and debugging capabilities\.

### A\.2Gymnasium Interface

The environment follows the standard Gymnasium API:

- •reset\(seed, options\)→\\rightarrow\(observation,info\)\(\\text\{observation\},\\text\{info\}\)
- •step\(action\)→\\rightarrow\(observation,reward,terminated,truncated,info\)\(\\text\{observation\},\\text\{reward\},\\text\{terminated\},\\text\{truncated\},\\text\{info\}\)

Unlike the legacy implementation, the default observation corresponds only to the current environment state\. Temporal history is not automatically included in the observation and may instead be modeled explicitly by the agent or through optional wrappers\.

### A\.3Support for Multiple Representations

The environment supports both state representations used throughout GOHR research:

1. 1\.Feature\-Centric \(FC\)representation based on spatial feature maps\.
2. 2\.Object\-Centric \(OC\)representation based on explicit object attributes\.

Both representations are accessible through the same environment interface, allowing experiments to be conducted without modifying the underlying game logic\.

### A\.4Legacy Compatibility Wrappers

To preserve compatibility with previous GOHR experiments, optional wrappers are provided that reproduce the legacy observation pipeline\.

These wrappers can:

- •maintain a history of previous successful states,
- •include previously executed actions,
- •construct history\-augmented observations matching those used in earlier studies\.

As a result, the same environment can operate in either a standard Gymnasium mode or a legacy\-compatible mode\.

### A\.5Observations from the Refactoring Experiment

A central difference between the Gymnasium implementation and the legacy GOHR environment is the treatment of temporal history\. The default Gymnasium design exposes only the current board state, whereas the legacy environment incorporates previous successful states and actions directly into the observation\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/fc_cw_legacy_true.png)\(a\)FC, legacy=True
![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/fc_cw_legacy_false.png)\(b\)FC, legacy=False
![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/oc_cw_legacy_true.png)\(c\)OC, legacy=True
![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/oc_cw_legacy_false.png)\(d\)OC, legacy=False

Figure 18:Comparison of learning behavior for the clockwise rulecwunder the legacy history\-augmented observation pipeline and the simplified Gymnasium current\-state observation design\. Dark\-background TensorBoard plots correspond tolegacy=True, while white\-background plots correspond tolegacy=False\.Fig\.[18](https://arxiv.org/html/2608.21372#A1.F18)illustrates the effect of replacing the legacy history\-augmented observations with the simplified Gymnasium observation design\. For the clockwise ordering rule \(cw\), both FC and OC learn successfully under the legacy observation pipeline\. However, when only the current state is provided, FC fails to learn the rule and OC exhibits markedly different learning dynamics\.

Preliminary experiments using the default Gymnasium observation design produced substantially different learning behavior from that observed with the legacy implementation\. In particular, several history\-dependent rules, including clockwise and counter\-clockwise ordering rules, exhibited substantially degraded learning performance when history information was removed from the observation\. Similar behavior was observed for several other history\-dependent rules\.

These results suggest that the legacy observation pipeline contains information that is important for solving certain GOHR rule families\. Although the Gymnasium refactoring successfully reproduced the game mechanics and rule definitions, the simplified observation design did not provide an adequate replacement for the legacy representation\.

These observations should not be interpreted as evidence that Gymnasium\-based implementations are unsuitable for GOHR\. Rather, they suggest that the history information embedded in the legacy observation pipeline plays an important role in the learning process\. Future work may investigate alternative mechanisms for providing this information, such as recurrent architectures, explicit memory modules, or history wrappers that preserve compatibility with standard reinforcement\-learning interfaces\.

To preserve compatibility with previous GOHR studies, legacy\-compatible wrappers were therefore developed to reconstruct the original history\-augmented observations within the Gymnasium framework\.

### A\.6Rendering and Visualization

The refactored environment also introduces a rendering interface for visualization and debugging\.

![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/gym_render.png)Figure 19:Example visualization generated by the Gymnasium\-compatible GOHR environment\. The display shows the current board state, bucket contents, and feedback associated with the most recent action\.The rendering interface supports interactive inspection of gameplay and can be accessed through:

env\.render\(mode="human"\)

This functionality simplifies debugging and provides a convenient way to visualize agent behavior during training and evaluation\.

### A\.7Availability

The implementation provides a foundation for future studies involving reinforcement learning, transfer learning, curriculum learning, and human\-agent comparisons within the GOHR framework\.

## Appendix BRule Descriptions

Table 5:List of rules and their descriptions\.RuleDescriptionallOfColOrd\_BRKYAll pieces of one color are removed before others, in order: Blue →Red →Black →Yellow\.allOfShaOrd\_qctsAll pieces of one shape are removed before others, in order: Square →Circle →Triangle →Star\.ccwStart with any bucket and fill buckets in counterclockwise order\.cm\_RBKYAssign colors to specific buckets in any order: B0=Red, B1=Blue, B2=Black, B3=Yellow\.col1Ord\_BRKYAssign colors to any bucket in order\. Skip missing colors: Blue →Red →Black →Yellow; loop\.col1Ord\_KRBYAssign colors to any bucket in order\. Skip missing colors: Black →Red →Blue →Yellow; loop\.col1OrdBuck\_BRKY0213Assign colors to specific buckets in order: B0=Blue, B2=Red, B1=Black, B3=Yellow; loop\.colOrdL1\_BRKYRemove colors in order; if a color is missing, take the next object of that color in reading order\. Blue →Red →Black →Yellow; loop\.cwStart with any bucket and fill buckets in clockwise order\.ordL1Remove pieces in reading order and assign pieces to any bucket\.ordL1\_NearbyRemove pieces in reading order and assign each to the nearest bucket\.ordRevOfL1Remove pieces in reverse reading order and assign pieces to any bucket\.ordRevOfL1\_RemotestRemove pieces in reverse reading order and assign each to the farthest bucket\.quadMixed1Assign pieces in each quadrant to specific buckets: B0= Q3, B1= Q0, B2= Q2, B3= Q1\.quadNearbyAssign pieces to the nearest bucket \(by quadrant\) in any order\.sha1Ord\_qctsAssign shapes to any bucket in order\. Skip missing shapes: Square →Circle →Triangle →Star; loop\.sha1OrdBuck\_qcts0213Assign shapes to specific buckets in order: B0=Square, B2=Circle, B1=Triangle, B3=Star; loop\.shaOrdL1\_qctsRemove shapes in order; if a shape is missing, take the next object of that shape in reading order\. Square →Circle →Triangle →Star; loop\.sm\_csqtAssign shapes to specific buckets in any order: B0=Circle, B1=Star, B2=Square, B3=Triangle\.cm\_RBKY\_cw\_0123Assign colors to specific buckets in order: B0=Red →B1=Blue →B2=Black →B3=Yellow; loop\.cw\_0123Start with bucket 0 and fill buckets clockwise: 0 →1 →2 →3; loop\.cm\_ordL1Colors are removed in reading order and assigned to specific buckets: B0=Blue, B1=Red, B2=Black, B3=Yellow\.cw\_qn2Start with bucket 0 and fill buckets clockwise using pieces from the nearest quadrant to that bucket: B0= Q0, B1= Q1, B2= Q2, B3= Q3\.Note\.\(1\) Bucket and quadrant indices range from 0 to 3\. Bucket 0 and quadrant 0 correspond to the top\-left of the board; 1 top\-right; 2 bottom\-right; 3 bottom\-left\. \(2\) Reading order proceeds left\-to\-right, top\-to\-bottom\. \(3\) B0–B3 denote buckets 0–3, and Q0–Q3 denote quadrants 0–3\.

## Appendix CRules Used in Rule\-Based Experiments

Table 6:Rules used in Rule\-Based experiments\.
## Appendix DRule\-Property Difficulty Rankings

Table 7:Rule properties arranged in increasing order of difficulty \(measured byM⋆M^\{\\star\}\) for the FC model\. The window size forM⋆M^\{\\star\}is 15\. Highlighted rows indicate properties that are combined with other properties\.Table 8:Rule properties arranged in increasing order of difficulty \(measured byM⋆M^\{\\star\}\) for the OC model\. The window size forM⋆M^\{\\star\}is 15\. Highlighted rows indicate properties that are combined with other properties\.
## Appendix EOverall Rule Difficulty for FC and OC Models

Table 9:Rules are grouped from easier \(top\) to harder \(bottom\) categories for the FC transformer model, based on their overall performance acrossM⋆M^\{\\star\},Emean⋆E^\{\\star\}\_\{\\mathrm\{mean\}\}andEmax⋆E^\{\\star\}\_\{\\mathrm\{max\}\}\. Rules within a category are not intended to represent a strict ordering\. Reported values are medians across five runs\.Table 10:Rules are grouped from easier \(top\) to harder \(bottom\) categories for the OC transformer model, based on their overall performance acrossM⋆M^\{\\star\},Emean⋆E^\{\\star\}\_\{\\mathrm\{mean\}\}andEmax⋆E^\{\\star\}\_\{\\mathrm\{max\}\}\. Rules within a category are not intended to represent a strict ordering\. Reported values are medians across five runs\.
## Appendix FTransfer Plots

![Refer to caption](https://arxiv.org/html/2608.21372v1/x6.png)Figure 20:Transfer heatmap for the compound rulecm\_RBKY\_cw\_0123\. Each cell reports the normalized transfer ratioMtransfer⋆/Mbaseline⋆M^\{\\star\}\_\{\\mathrm\{transfer\}\}/M^\{\\star\}\_\{\\mathrm\{baseline\}\}\. Values below11indicate positive transfer, while values above11indicate negative transfer\.
## Appendix GClustering and MDS Results

The main R code blocks used to generate these visualizations are included below\. The FC and OC figures were produced using the same analysis pipeline, with the input file changed fromFC\_diff\_matrix\.csvtoOC\_diff\_matrix\.csv\. This appendix provides additional hierarchical clustering, multidimensional scaling \(MDS\), and tanglegram visualizations discussed in Section[5\.6](https://arxiv.org/html/2608.21372#S5.SS6)\. These figures complement the main\-text analyses by showing the complete set of FC and OC transfer\-geometry results\.

### G\.1Hierarchical Clustering Results

mat<\-as\.matrix\(read\.csv\("FC\_diff\_matrix\.csv",

row\.names=1,

check\.names=FALSE\)\)

rownames\(mat\)<\-paste0\(LETTERS\[seq\_len\(nrow\(mat\)\)\],":",rownames\(mat\)\)

colnames\(mat\)<\-paste0\(LETTERS\[seq\_len\(ncol\(mat\)\)\],":",colnames\(mat\)\)

mat\_row<\-mat

for\(jin1:ncol\(mat\_row\)\)\{

col\_mean<\-mean\(mat\_row\[,j\],na\.rm=TRUE\)

mat\_row\[is\.na\(mat\_row\[,j\]\),j\]<\-col\_mean

\}

row\_dist<\-dist\(mat\_row\)

row\_hclust<\-hclust\(row\_dist\)

plot\(row\_hclust,main="Row␣Clustering"\)

mat\_col<\-mat

for\(iin1:nrow\(mat\_col\)\)\{

row\_mean<\-mean\(mat\_col\[i,\],na\.rm=TRUE\)

mat\_col\[i,is\.na\(mat\_col\[i,\]\)\]<\-row\_mean

\}

col\_dist<\-dist\(t\(mat\_col\)\)

col\_hclust<\-hclust\(col\_dist\)

plot\(col\_hclust,main="Column␣Clustering"\)

Listing 1:R code for row and column hierarchical clustering\.![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/clustering_fc_row.png)Figure 21:Hierarchical clustering of FC successor\-rule transfer behavior\. Rules that cluster together exhibit similar sensitivity to predecessor\-rule experience\.![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/clustering_fc_column.png)Figure 22:Hierarchical clustering of FC predecessor\-rule influence\. Rules that cluster together produce similar transfer effects on successor\-rule learning\.![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/clustering_oc_column.png)Figure 23:Hierarchical clustering of OC predecessor\-rule influence\. The resulting clusters exhibit stronger separation between ordering\-based, feature\-based, and spatial rule families than the corresponding FC clustering\.
### G\.2Multidimensional Scaling Results

library\(smacof\)

library\(plotly\)

display\_rotatable\_MDS<\-function\(mat,plot\_title\)\{

dist\_mat<\-dist\(mat\)

mds\_result<\-smacof::smacofSym\(dist\_mat,ndim=3\)

mds\_df<\-data\.frame\(

mds\_result$conf,

label=sub\(":\.\*","",rownames\(mat\)\),

rule=sub\("^\[^:\]\*:","",rownames\(mat\)\)

\)

plot\_ly\(

data=mds\_df,

x=~D1,

y=~D2,

z=~D3,

type="scatter3d",

mode="text",

text=~label,

hovertext=~rule,

hoverinfo="text"

\)%\>%

layout\(

title=plot\_title,

scene=list\(

xaxis=list\(title="Dimension␣1"\),

yaxis=list\(title="Dimension␣2"\),

zaxis=list\(title="Dimension␣3"\)

\)

\)

\}

display\_rotatable\_MDS\(mat\_row,"Row␣MDS"\)

display\_rotatable\_MDS\(t\(mat\_col\),"Column␣MDS"\)

Listing 2:R code for 3\-dimensional MDS visualization\.![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/mds_fc_row.png)Figure 24:3\-dimensional MDS embedding of FC successor\-rule transfer behavior\. Distances reflect similarity in how rules respond to predecessor experience\.![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/mds_fc_column.png)Figure 25:3\-dimensional MDS embedding of FC predecessor\-rule influence\. Distances reflect similarity in how predecessor rules affect successor learning\.![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/mds_oc_row.png)Figure 26:3\-dimensional MDS embedding of OC successor\-rule transfer behavior\. Structurally related rules form more compact and separable clusters than in the FC embedding\.
### G\.3Tanglegram Comparisons

library\(dendextend\)

strip\_prefix<\-function\(x\)\{

sub\("^\[A\-Z\]:","",x\)

\}

compare\_dendrograms<\-function\(h1,h2,labels1,labels2,name1,name2\)\{

common\_rules<\-intersect\(labels1,labels2\)

d1<\-as\.dendrogram\(h1\)

d2<\-as\.dendrogram\(h2\)

labels\(d1\)<\-labels1

labels\(d2\)<\-labels2

d1<\-prune\(d1,setdiff\(labels\(d1\),common\_rules\)\)

d2<\-prune\(d2,setdiff\(labels\(d2\),common\_rules\)\)

dlist<\-dendlist\(d1,d2\)

cat\("Cophenetic␣correlation:",cor\_cophenetic\(d1,d2\),"\\n"\)

cat\("Baker’s␣gamma:",cor\_bakers\_gamma\(d1,d2\),"\\n"\)

cat\("Entanglement:",entanglement\(dlist\),"\\n"\)

tanglegram\(

d1,

d2,

main=paste\(name1,"vs",name2\),

common\_subtrees\_color\_lines=TRUE,

highlight\_distinct\_edges=FALSE

\)

\}

Listing 3:R code for comparing clustering structures using tanglegrams\.![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/fc_row_vs_fc_column.png)Figure 27:Tanglegram comparing FC row clustering and FC column clustering\. The large number of crossing connections reflects weaker agreement between predecessor\-rule influence and successor\-rule sensitivity under the FC representation\.![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/fc_column_vs_oc_column.png)Figure 28:Tanglegram comparing FC and OC predecessor\-rule clustering structures\. The relatively weak alignment indicates that FC and OC organize predecessor\-rule influence differently\.![Refer to caption](https://arxiv.org/html/2608.21372v1/figures/fc_row_vs_oc_row.png)Figure 29:Tanglegram comparing FC and OC successor\-rule clustering structures\. Some high\-level relationships remain preserved across representations, particularly among spatial and ordering\-based rules\.

## Appendix HSupplementary Materials

Supplementary materials associated with this report are available at:

These materials include additional interactive visualizations and analysis outputs supporting the results reported in this paper\.