What Do Current Systematic Generalization Tasks Miss? A Reasoning-Centered Analysis

# What Do Current Systematic Generalization Tasks Miss? A Reasoning-Centered Analysis
Source: [https://arxiv.org/html/2609.19212](https://arxiv.org/html/2609.19212)
Deheng YeAffiliation:Nanyang Technological UniversityEmail:[ydyl1991@gmail\.com](mailto:[email protected])Yatao Bian††thanks:Corresponding authorAffiliation:National University of SingaporeEmail:[ybian@nus\.edu\.sg](mailto:)

###### Abstract

Systematic generalization, the ability to solve novel problems by recombining known atomic elements, is central to human intelligence but difficult to study rigorously under controlled settings\. Existing studies therefore rely on simplifications such as approximately linear action composition, productivity\-based tests, and action\-explicit goals, which make systematic generalization easier to study but omit some essential aspects of this capability\. To characterize what these simplifications miss, we adopt a reasoning\-centered lens and introduce TranSGrid, a testbed that brings deductive, inductive, and abductive reasoning together within a unified task\. Experiments with seven Transformers on 4,800 TranSGrid instances show that all models perform much worse on TranSGrid than on a held\-out test set: the largest model solves 79\.6% of the test set, but only 55\.3% of TranSGrid and 15\.8% of the hardest subset\. The gap remains within the training length range, showing that productivity alone is not sufficient to evaluate systematic generalization\. Additionally, we reintroduce the other two simplifications into TranSGrid: one variant makes actions compose almost linearly \(reducing the inductive demand\), the other makes goals action\-explicit \(reducing the abductive one\)\. In both, solve rates return to roughly the test set level, showing that either simplification alone is enough to reduce TranSGrid to an ordinary held\-out test set\. Together, our results show that existing tasks reduce either or both of the inductive and abductive demands, and that comprehensively measuring systematic generalization requires a task that involves all three forms of reasoning\.

## 1Introduction

Systematic generalization, which refers to the ability to systematically combine learned actions or skills to address novel problems, is a fundamental aspect of human intelligence\([Lake and Baroni, 2018](https://arxiv.org/html/2609.19212#bib.bib1);[Lake and Baroni, 2023](https://arxiv.org/html/2609.19212#bib.bib2)\)\. This capability is essential across diverse domains, including grounded navigation\([Ruis et al\., 2020](https://arxiv.org/html/2609.19212#bib.bib3);[Sikarwar et al\., 2022](https://arxiv.org/html/2609.19212#bib.bib5);[Spilsbury et al\., 2024](https://arxiv.org/html/2609.19212#bib.bib17);[Chen et al\., 2026a](https://arxiv.org/html/2609.19212#bib.bib16)\), semantic parsing\([Kim and Linzen, 2020](https://arxiv.org/html/2609.19212#bib.bib8);[Wu et al\., 2023](https://arxiv.org/html/2609.19212#bib.bib13);[Jabbar et al\., 2025](https://arxiv.org/html/2609.19212#bib.bib14)\), text\-to\-image generation\([Han et al\., 2025](https://arxiv.org/html/2609.19212#bib.bib21);[Huang et al\., 2025](https://arxiv.org/html/2609.19212#bib.bib22);[Dat et al\., 2025](https://arxiv.org/html/2609.19212#bib.bib23)\), AI safety\([Addepalli et al\., 2025](https://arxiv.org/html/2609.19212#bib.bib24);[Chen et al\., 2026b](https://arxiv.org/html/2609.19212#bib.bib12)\), and AI for Science\([Ji et al\., 2023](https://arxiv.org/html/2609.19212#bib.bib15);[Chen et al\., 2025](https://arxiv.org/html/2609.19212#bib.bib20);[Li et al\., 2026a](https://arxiv.org/html/2609.19212#bib.bib18);[Li et al\., 2026b](https://arxiv.org/html/2609.19212#bib.bib19)\)\. Across these domains, success depends not merely on mastering individual components, but also on understanding how they interact and recombining them appropriately under novel goals, contexts, and constraints\.

A rigorous study of systematic generalization therefore requires a testbed with \(1\) a*clearly defined space of atomic actions*, \(2\)*meaningful, interaction\-rich compositions*of these actions, \(3\)*scalable generation*of instances that genuinely probe models’ systematicity, and \(4\)*automatic evaluation*that accommodates multiple valid solutions\. In practice, however, incorporating all these requirements into a single task is difficult, leading existing studies to rely on substantial simplifications in task design\. One common simplification is to compose atomic actions in an approximately linear manner, with limited interactions among their effects\([Lake and Baroni, 2018](https://arxiv.org/html/2609.19212#bib.bib1);[Lake and Baroni, 2023](https://arxiv.org/html/2609.19212#bib.bib2)\)\. A second is to assess systematicity primarily through productivity, typically by testing whether models extrapolate to sequences longer than those observed during training\([Hupkes et al\., 2020](https://arxiv.org/html/2609.19212#bib.bib6);[Fu and Liu, 2026](https://arxiv.org/html/2609.19212#bib.bib25)\)\. A third is to use action\-explicit goals, where the input already specifies the actions to perform and their order\([Wu et al\., 2023](https://arxiv.org/html/2609.19212#bib.bib13);[Li et al\., 2023](https://arxiv.org/html/2609.19212#bib.bib7);[Mondorf et al\., 2026](https://arxiv.org/html/2609.19212#bib.bib26)\)\. Although these simplifications make systematic generalization easier to study, they also remove some of its core challenges, providing an incomplete picture of what the capability entails\.

To examine what these simplifications ignore,we employ a reasoning\-centered lens to study systematic generalizationand introduceTranSformation on aGrid\(TranSGrid\), a controllable and flexible testbed\. Inspired by cognitive science research highlighting the interplay of deduction, induction, and abduction in human intelligence\([Peirce, 1934](https://arxiv.org/html/2609.19212#bib.bib27);[Shank, 1998](https://arxiv.org/html/2609.19212#bib.bib28);[Chemero, 2026](https://arxiv.org/html/2609.19212#bib.bib29)\), TranSGrid is designed to operationalize this interplay within a unified task\. Specifically, TranSGrid requires a model to transform an initial board into a target board by generating an action sequence drawn from ten atomic operations on rows, columns, or local2×22\\times 2blocks\. The task involves deductive reasoning to derive and track the board state after each action\. Its inductive challenge lies in discovering reusable composition rules from observed examples of how action effects combine, cancel, or overwrite one another, and applying these rules to novel instances\. Abductive reasoning involves inferring a suitable action sequence from the desired outcome\. To accommodate multiple valid solutions, TranSGrid evaluates the correctness of each generated sequence by executing it on the initial board and checking whether the resulting board matches the target\.

Experiments with seven Transformer models ranging from 0\.96M to 88\.43M parameters on 4,800 TranSGrid instances show that all models perform substantially worse on TranSGrid than on a held\-out Test set\. Under greedy decoding, the largest model’s solve rate falls from 79\.63% on Test to 55\.31% on TranSGrid overall, reaching only 15\.75% on TranSGridHard\. Crucially, this gap persists even when reference answer lengths fall within the training range, showing that productivity alone is insufficient to evaluate systematic generalization\. We then reintroduce the other two simplifications through controlled variants\. TranSGrid \(Decoupled\) makes action composition approximately linear, reducing the inductive demand of discovering reusable composition rules from examples of how action effects interact\. TranSGrid \(SCAN\) makes goals largely action\-explicit by providing all but the final gold action, reducing the abductive demand of inferring a suitable action sequence from the desired outcome\. Solve rates in both variants recover to roughly the held\-out test set level, indicating that either simplification substantially reduces the challenge posed by TranSGrid\. Taken together, these findings show that existing tasks overlook essential inductive and abductive reasoning demands of systematic generalization, and that a comprehensive evaluation of this capability requires tasks that involve all three forms of reasoning: deduction, induction, and abduction\.

## 2Related Works

Existing benchmarks for systematic generalization can be broadly grouped into three categories\. First, action or solution\-explicit tasks, such as SCAN\([Lake and Baroni, 2018](https://arxiv.org/html/2609.19212#bib.bib1)\), PCFG\([Hupkes et al\., 2020](https://arxiv.org/html/2609.19212#bib.bib6)\), COGS\([Kim and Linzen, 2020](https://arxiv.org/html/2609.19212#bib.bib8)\), and HINT\([Li et al\., 2023](https://arxiv.org/html/2609.19212#bib.bib7)\), provide the action sequence directly in the input\. Models therefore need only to parse or execute the given composition rather than infer one from a goal\. Second, grounded navigation benchmarks, including gSCAN\([Ruis et al\., 2020](https://arxiv.org/html/2609.19212#bib.bib3)\), GSSR\([Kuo et al\., 2021](https://arxiv.org/html/2609.19212#bib.bib9)\), and ReaSCAN\([Wu et al\., 2021](https://arxiv.org/html/2609.19212#bib.bib4)\), require models to generate an action sequence in a structured world\. However, once the target is identified, the route can usually be derived directly from the world, so these benchmarks focus more on language grounding and target localization than on abductive reasoning\. Action\-action interactions are also present, but are usually simple\. Across these two lines, many evaluations emphasize productivity by testing sequences longer or deeper than those seen during training\. Finally, rule\-changing tasks such asBaba Is Youprovide richer action interactions by requiring agents to create, break, and combine rules to reach a goal\([Cloos et al\., 2024](https://arxiv.org/html/2609.19212#bib.bib10)\)\. However, its visual input and various distractors make it difficult to determine whether failures result from weak systematic generalization or poor scene understanding\. TranSGrid addresses these limitations by requiring models to infer action sequences from goals and reason about interacting action effects in a simple, controllable setting, allowing systematic generalization to be studied more reliably and with fewer confounding factors\.

## 3A Grid\-Transformation Task for Systematic Generalization

### 3\.1Challenges in Measuring Systematic Generalization

Although systematic generalization is essential across many domains, constructing a rigorous and scalable benchmark remains difficult\. Such a benchmark needs to combine a well\-defined atomic action space, automatic generation of meaningful compositions, scalable data construction, and automatic evaluation of multiple valid solutions\. Yet most existing task settings satisfy only a subset of these requirements\. For example, there is no precise definition of atomic actions in code generation; meaningful action interactions in spreadsheet editing often require substantial human effort; and tasks such asBaba Is Youintroduce perceptual confounds unrelated to systematic generalization\. Beyond these construction challenges, there is also no consensus on how to measure systematic generalization: most criteria are task\-specific, with productivity being the only one widely shared across tasks\. However, productivity is not a necessary condition for systematic generalization, as generalization within the training length range can also be systematic\. Together, these construction and evaluation challenges help explain why prior work relies on simplifications such as approximately linear action composition, productivity\-based evaluation, and action\-explicit goals\. They also motivate a controlled testbed that addresses both these construction challenges and the limitations of productivity\-based evaluation\.

### 3\.2Design of the TranSGrid Task

To address these challenges, we introduceTranSformation on aGrid\(TranSGrid\), a controllable and flexible grid\-transformation testbed for systematic generalization\. Figure[1](https://arxiv.org/html/2609.19212#S3.F1)provides an overview of the task, with its atomic action space and task formulation detailed below\.

\(a\)\(b\)
Figure 1:The proposed TranSGrid task with its atomic actions\. \(a\) The ten type of atomic actions defined for grid manipulation, including row\-wise shifts, column\-wise shifts, and rotations of local blocks\. Red arrows indicate the direction of each transformation\. \(b\) Illustration of the TranSGrid task\. Given an initial board and a target board, the models need to generate an action sequence that, when executed sequentially, can transform the initial board into the target board\.Atomic action spaceAs illustrated in Figure[1\(a\)](https://arxiv.org/html/2609.19212#S3.F1.sf1), TranSGrid defines ten types of atomic actions\.ROW\_LEFTandROW\_RIGHTcyclically shift the elements of a selected row, whileCOL\_UPandCOL\_DOWNperform the analogous operation on a column\.ROW\_UPandROW\_DOWNmove a selected row by swapping it with an adjacent row, andCOL\_LEFTandCOL\_RIGHTsimilarly move a selected column\. Finally,BLOCK\_CWandBLOCK\_CCWrotate a2×22\\times 2subgrid90∘90^\{\\circ\}in either direction\.

By design, each atomic action modifies multiple cells at once, operating on an entire row, column, or2×22\\times 2block\. Because actions may affect overlapping regions and are executed sequentially, their effects are both state\-dependent and interdependent: a later action may reinforce, overwrite, or reverse the effect of an earlier one\. Solving TranSGrid therefore requires models to track the evolving board and reason about both action\-action and action\-environment interactions, providing a controlled, interaction\-rich setting for studying systematic generalization\.

Task formulationA TranSGrid instance consists of an initial board and a target board, both of sizeN×NN\\times N, as shown in Figure[1\(b\)](https://arxiv.org/html/2609.19212#S3.F1.sf2)\.111All experiments in this paper use6×66\\times 6boards, but for visual clarity, we use4×44\\times 4boards in the figures\.Let

B\(0\),B⋆∈𝒱N×N,𝒱=\{0,1,…,9\},B^\{\(0\)\},B^\{\\star\}\\in\\mathcal\{V\}^\{N\\times N\},\\quad\\mathcal\{V\}=\\\{0,1,\\ldots,9\\\},denote the initial and target boards, respectively, whereBi,j\(0\)B^\{\(0\)\}\_\{i,j\}is the value in rowiiand columnjjof the initial board\. TranSGrid provides ten atomic operation types,

𝒜=\{α1,…,α10\},\\mathcal\{A\}=\\\{\\alpha\_\{1\},\\ldots,\\alpha\_\{10\}\\\},each of which performs a deterministic transformation on the board, as illustrated in Figure[1\(a\)](https://arxiv.org/html/2609.19212#S3.F1.sf1)\. Let

T:𝒜×𝒱N×N→𝒱N×NT:\\mathcal\{A\}\\times\\mathcal\{V\}^\{N\\times N\}\\rightarrow\\mathcal\{V\}^\{N\\times N\}denote the transition function, such thatT⁡\(a,B\)T\(a,B\)is the board obtained by applying actionaato boardBB\. Composition in TranSGrid is sequential\. An action sequence of lengthLLis defined as

π=\(a1,…,aL\)∈𝒜L\.\\pi=\(a\_\{1\},\\ldots,a\_\{L\}\)\\in\\mathcal\{A\}^\{L\}\.Executingπ\\pifromB\(0\)B^\{\(0\)\}yields

B\(t\)=T\(at,B\(t−1\)\),t=1,…,L\.B^\{\(t\)\}=T\\bigl\(a\_\{t\},B^\{\(t\-1\)\}\\bigr\),\\qquad t=1,\\ldots,L\.The final board after executingπ\\piis thereforeB\(L\)B^\{\(L\)\}\. Given an instance\(B\(0\),B⋆\)\(B^\{\(0\)\},B^\{\\star\}\), the model is required to generate a sequenceπ\\pisuch thatB\(L\)=B⋆B^\{\(L\)\}=B^\{\\star\}\. Any sequence satisfying this condition is considered a valid solution; it need neither be a shortest solution nor match the reference sequence\.

### 3\.3Controllability, Flexibility, and Scalability of TranSGrid

TranSGrid provides explicit control over task properties through adjustable parameters, including grid sizeNN, digit distribution, reference action\-sequence length, and composition patterns\. These parameters allow task structure and difficulty to be varied systematically\. Its flexible design can also reproduce two common simplifications in existing tasks: approximately linear action composition and action\-explicit goals \(see Figures[3\(c\)](https://arxiv.org/html/2609.19212#S5.F3.sf3)and[4\(c\)](https://arxiv.org/html/2609.19212#S5.F4.sf3), respectively, for further details\)\.

Moreover, TranSGrid provides a vast combinatorial task space\. For example, withN=6N=6and ten possible cell values, there are103610^\{36\}possible initial boards\. Because the atomic actions can be applied to different rows, columns, or blocks, a6×66\\times 6board admits

4⋅6\+2⋅6\+2⋅52=864\\cdot 6\+2\\cdot 6\+2\\cdot 5^\{2\}=86distinct board transformations\.222The ten types of atomic actions can be applied in8⋅6\+2⋅52=988\\cdot 6\+2\\cdot 5^\{2\}=98ways\. Among them,ROW\_UP/ROW\_DOWNandCOL\_LEFT/COL\_RIGHTform 12 equivalent pairs, leaving 86 distinct transformations\.Restricting the reference action\-sequence length to the range 1–9 therefore yields approximately

1036×∑k=1986k≈1036×2\.60×1017≈2\.60×105310^\{36\}\\times\\sum\_\{k=1\}^\{9\}86^\{k\}\\approx 10^\{36\}\\times 2\.60\\times 10^\{17\}\\approx 2\.60\\times 10^\{53\}possible pairs of initial boards and reference action sequences\. Although different action sequences may produce the same target board from a given initial board when their effects overlap or cancel, the task space of TranSGrid remains extremely large\. Instances can be generated automatically by sampling an initial board and executing a reference action sequence to obtain the target board, guaranteeing a valid solution by construction\. Together with its controllability and flexibility, this scale makes TranSGrid an ideal platform for developing new methods to improve models’ systematic generalization performance and investigating the mechanisms underlying this capability\.

## 4Experimental Results on TranSGrid

### 4\.1The three reasoning forms in TranSGrid

TranSGrid is designed to jointly engage deductive, inductive, and abductive reasoning within a unified task\. Although each form corresponds to a distinct aspect of the task, the associated reasoning demands are inherently interdependent\. We next describe how these forms manifest in TranSGrid and introduce the corresponding metrics\.

Deductive ReasoningDeductive reasoning in TranSGrid involves inferring the next board state from the current state and action\. Since each additional action demands one more step of forward inference over the board state, we use the reference sequence length as a proxy for deductive difficulty, with longer sequences indicating greater deductive load\.

Inductive ReasoningThe atomic actions in TranSGrid operate on multiple cells\. However, when combined in specific ways, their intermediate effects can cancel out, leaving only a few cells permuted\. The inductive challenge lies in discovering reusable composition rules from observed examples of such interactions and applying these rules to new instances\. We refer to action combinations that realize these localized permutations as induction rules and manually define 12 such rules for constructing TranSGrid instances \(see Appendix[A](https://arxiv.org/html/2609.19212#A1)for details\)\. Accordingly, we use the number of induction rules instantiated in the reference sequence as a proxy for inductive difficulty, with larger counts representing higher inductive load\.

Abductive ReasoningThe task formulation of TranSGrid inherently involves abductive reasoning, as the model needs to infer a plausible sequence of actions from the observed initial and target boards\. When actions overwrite or cancel one another’s effects, the resulting boards can provide fewer observable cues about the actions involved\. To quantify this concealment, we letDDdenote the number of cells that differ between the initial and target boards\. We then mapDDto an effective action lengthLeffL\_\{\\mathrm\{eff\}\}, which estimates how many random actions would typically produce the same amount of observable change \(see Appendix[B](https://arxiv.org/html/2609.19212#A2)\)\. Given the reference sequence lengthLL, we defineHact=L−LeffH\_\{\\mathrm\{act\}\}=L\-L\_\{\\mathrm\{eff\}\}as a proxy for action concealment: larger values indicate that the reference sequence contains more actions than its observable effect would typically suggest\. We then normalize this proxy to obtain an abductive score:

Aabd=σ⁡\(Hact−62\),A\_\{\\mathrm\{abd\}\}=\\sigma\\left\(\\frac\{H\_\{\\mathrm\{act\}\}\-6\}\{2\}\\right\),whereσ⁡\(x\)=1/\(1\+e−x\)\\sigma\(x\)=1/\(1\+e^\{\-x\}\)\. Using the maximum reference sequence lengthLmax=12L\_\{\\max\}=12, we set the sigmoid midpoint toLmax/2=6L\_\{\\max\}/2=6and its scale toLmax/6=2L\_\{\\max\}/6=2\. This calibration givesAabd=0\.5A\_\{\\mathrm\{abd\}\}=0\.5atHact=6H\_\{\\mathrm\{act\}\}=6, with scores of approximately 0\.05 and 0\.95 atHact=H\_\{\\mathrm\{act\}\}=0 and 12, respectively\. Overall, the resulting score lies in\(0,1\)\(0,1\)and, becauseσ\\sigmais strictly increasing, preserves the ordering of instances induced byHactH\_\{\\mathrm\{act\}\}, with higher scores indicating greater estimated abductive load\.

### 4\.2Experimental Setup

ModelFollowing prior work on systematic generalization\([Kim and Linzen, 2020](https://arxiv.org/html/2609.19212#bib.bib8);[Lake and Baroni, 2023](https://arxiv.org/html/2609.19212#bib.bib2);[Li et al\., 2023](https://arxiv.org/html/2609.19212#bib.bib7);[Kumon and Yanaka, 2025](https://arxiv.org/html/2609.19212#bib.bib11)\), we train standard encoder\-decoder Transformers to map an initial–target board pair to an action sequence\. The encoder jointly processes both boards, while the decoder autoregressively generates action\-name and argument tokens one by one \(Figure[1\(b\)](https://arxiv.org/html/2609.19212#S3.F1.sf2)\)\. We evaluate seven models ranging from 0\.96M to 88\.43M parameters, indexed from 1 to 7 by increasing parameter count\. Model configurations, including the numbers of encoder and decoder layers, hidden dimensions, and attention heads, are detailed in Appendix[C](https://arxiv.org/html/2609.19212#A3)\.

DatasetAll experiments in this paper use6×66\\times 6boards with cell values independently sampled from\{0,…,9\}\\\{0,\\ldots,9\\\}\. For clarity, however, we use4×44\\times 4boards in all figures\. Each instance contains an initial board, a target board, and a reference sequence that transforms the former into the latter\. We construct a TranSGrid evaluation set of 4,800 instances, marginally balanced across reference sequence lengthsL∈\[1,12\]L\\in\[1,12\]and numbers of induction rulesK∈\[0,3\]K\\in\[0,3\]\. Based onLL, we divide the set into Easy \(L∈\[1,4\]L\\in\[1,4\]\), Medium \(L∈\[5,8\]L\\in\[5,8\]\), and Hard \(L∈\[9,12\]L\\in\[9,12\]\), each containing 1,600 instances\. Appendix[D](https://arxiv.org/html/2609.19212#A4)provides details on dataset construction and the jointLL–KKdistribution\.

Implementation detailsThe two boards and their spatial relations can be encoded in multiple ways\. We therefore evaluate four combinations of board and positional encodings and use the best overall configuration,GRID\+PAIR, in all experiments \(see Appendix[E](https://arxiv.org/html/2609.19212#A5)for a detailed comparison\)\. All seven models are trained for eight epochs on 30M instances with reference sequence lengths ranging from 1 to 9\. For each model, we select the checkpoint with the highest solve rate on a 2,700\-instance development set\. We also construct a 4,800\-instance Test set using the same generation procedure as the training and development sets\. Its reference sequence lengths are uniformly distributed from 1 to 12, and no Test instance appears in either the training or development set\. We report results under greedy decoding and beam search with a beam size of 8\.

### 4\.3Main Results

Table 1:Solve rates \(%\) on the standard Test set and TranSGrid\. The overall TranSGrid score is the unweighted mean over the Easy, Medium, and Hard subsets\. Beam8 counts an instance as solved if any of its top eight candidates succeeds under execution\.As shown in Table[1](https://arxiv.org/html/2609.19212#S4.T1), models with IDs 4–7 achieve greedy solve rates of 71\.27%–79\.63% on Test, suggesting strong generalization under standard held\-out evaluation\. Across all seven models and both decoding strategies, however, TranSGrid solve rates are significantly lower than those on Test and decline sharply from Easy to Hard\. Larger models and Beam@8 improve solve rates, but performance remains limited on the more difficult subsets: under Beam@8, even ID 7 achieves only 65\.19% on TranSGridMediumand 21\.25% on TranSGridHard\.

Figure 2:Solve rate \(%\) versus reference sequence lengthLLunder greedy decoding\. Panels \(a\)–\(g\) show results for model IDs 1–7, respectively\. In each panel,andshow performance on Test and TranSGrid, respectively, whilemarks the gap between them\.marks the maximum training length,L=9L=9, with larger values requiring length extrapolation\. Panel \(h\) summarizes the Test–TranSGrid gaps for all seven models, with darker curves indicating larger models\.To examine whether this gap persists at matched reference sequence lengths, we compare Test and TranSGrid solve rates in Figure[2](https://arxiv.org/html/2609.19212#S4.F2)\. A substantial gap remains forL≤9L\\leq 9, even though this range is fully covered during training and requires no length extrapolation\. Across most of this range, larger models maintain high solve rates on Test, whereas their TranSGrid performance declines sharply asLLincreases\. Panel \(h\) further shows that increasing model capacity shifts the largest gap toward longer sequences but does not eliminate it within the training\-length range\. ForL\>9L\>9, the gap narrows as Test performance also declines under length extrapolation\. Together, these results demonstrate the value of TranSGrid as a task for systematic generalization, while also highlighting its unique advantage of not relying on productivity\.

## 5What Existing Metrics Capture and Miss: A Reasoning Centered Analysis

### 5\.1Inductive Reasoning Is Suppressed by Linear Composition and Not Guaranteed by Productivity

Inductive reasoning involves discovering reusable composition rules from observed examples of how action effects interact and applying these rules to new instances\. This demand is largely suppressed by linear composition, under which the effect of a compound sequence can be decomposed into the independent effects of its constituent actions\. Nor is it guaranteed by productivity: longer sequences may require only repeated applications of atomic actions, without requiring models to infer new composition rules\. To test whether inductive reasoning contributes to systematic generalization beyond productivity and whether linear composition suppresses this reasoning demand, we first analyze performance across inductive loads relative to sequence length and then compare TranSGrid with a linearized variant\.

\(a\)![Refer to caption](https://arxiv.org/html/2609.19212v1/tasks_linear.png)\(b\)
\(c\)
Figure 3:\(a\) Solve rates on TranSGrid as inductive loadKKincreases\. Curves from light to dark denote model IDs 1–7, and the shaded bars indicate the proportion of instances requiring length extrapolation\. \(b\) Examples of near\-linear composition in prior benchmarks, where interactions among atomic operations are limited\. \(c\) Examples from TranSGrid \(Decoupled\), where actions operate on separate parts of the board to reduce action interactions and approximate linear composition\.Inductive reasoning presents a distinct challenge for systematic generalization that productivity alone does not capture\. As shown in Figure[3\(a\)](https://arxiv.org/html/2609.19212#S5.F3.sf1), solve rates decline sharply across all seven models as inductive loadKKincreases\. More importantly, this decline is already pronounced even when productivity demands remain limited: as indicated by the shaded bars, only 10\.8% of the instances atK=2K=2require length extrapolation, yet solve rates are below 40% for all models\.

Table 2:Comparison results on Test, TranSGrid, and TranSGrid \(Decoupled\)\.Δ\\Deltadenotes the gain over the corresponding TranSGrid results\.However, near\-linear composition largely suppresses this inductive demand by limiting interactions among atomic elements\. Figure[3\(b\)](https://arxiv.org/html/2609.19212#S5.F3.sf2)illustrates this structure in SCAN\([Lake and Baroni, 2018](https://arxiv.org/html/2609.19212#bib.bib1)\), gSCAN\([Ruis et al\., 2020](https://arxiv.org/html/2609.19212#bib.bib3)\), HINT\([Li et al\., 2023](https://arxiv.org/html/2609.19212#bib.bib7)\), and PCFG\([Hupkes et al\., 2020](https://arxiv.org/html/2609.19212#bib.bib6)\), where interactions among atomic operations within a sequence are limited\. To reproduce the linear structure in a controlled setting, we construct TranSGrid \(Decoupled\), as shown in Figure[3\(c\)](https://arxiv.org/html/2609.19212#S5.F3.sf3)\. Specifically, TranSGrid \(Decoupled\) is constructed so that actions operate on separate parts of the board, thereby minimizing interference among their effects\. We apply this decoupling to instances withL≤9L\\leq 9, which require no length extrapolation, while leaving instances withL≥10L\\geq 10unchanged and preserving the initial boards and per\-length counts\.

As shown in Table[2](https://arxiv.org/html/2609.19212#S5.T2), decoupling consistently improves solve rates across all model sizes and both decoding strategies\. Gains reach 43\.96 percentage points under greedy decoding and 40\.04 points under Beam@8\. Performance recovers to roughly the Test level or higher for model IDs 3–7 under both decoding strategies\. For example, the largest model improves from 55\.31% to 78\.42% under greedy decoding and from 61\.60% to 79\.87% under Beam@8, approaching its Test solve rates of 79\.63% and 80\.17%, respectively\. Because the reference action sequence length distribution remains unchanged, this recovery reflects reduced action interactions rather than reduced productivity demands\. Overall, these results show that approximately linear composition substantially suppresses the inductive demand of systematic generalization, and that productivity alone does not guarantee this form of reasoning\.

### 5\.2Action\-Explicit Goals Largely Bypass Abductive Reasoning

Abductive reasoning in TranSGrid involves inferring, from the initial and target boards, which actions could have produced the observed transformation and in what order\. Action\-explicit goals largely suppress this demand by providing the complete solution sequence, including the required actions and their order, in the input\. The model therefore need only parse or execute the specified sequence rather than infer one from the desired outcome\. To examine the role of abductive reasoning in systematic generalization and the effect of action\-explicit goals on this capability, we first analyze how model performance varies with abductive load and then compare TranSGrid with an action\-explicit variant\.

\(a\)![Refer to caption](https://arxiv.org/html/2609.19212v1/tasks_scan.png)\(b\)
\(c\)
Figure 4:\(a\) Solve rates on TranSGrid as abductive loadAabdA\_\{\\mathrm\{abd\}\}increases\. Curves from light to dark denote model IDs 1–7, and the shaded bars indicate the proportion of instances requiring length extrapolation\. \(b\) Examples of action\-explicit tasks in prior work, where the input specifies the complete solution sequence\. \(c\) TranSGrid \(SCAN\), in which the decoder receives the firstL−1L\-1gold actions and predicts only the final action\.As presented in Figure[4\(a\)](https://arxiv.org/html/2609.19212#S5.F4.sf1), solve rates decline sharply across all seven models as abductive load increases\. Notably, between abductive levels 1 and 2, solve rates fall substantially even as the proportion of instances requiring length extrapolation decreases from 6\.2% to 5\.7%\. These results indicate that abductive reasoning is an important dimension of systematic generalization that productivity alone does not fully capture\.

Despite its importance, many prior benchmarks, such as SCAN\([Lake and Baroni, 2018](https://arxiv.org/html/2609.19212#bib.bib1)\), COGS\([Kim and Linzen, 2020](https://arxiv.org/html/2609.19212#bib.bib8)\), PCFG\([Hupkes et al\., 2020](https://arxiv.org/html/2609.19212#bib.bib6)\), and HINT\([Li et al\., 2023](https://arxiv.org/html/2609.19212#bib.bib7)\), evaluate systematic generalization through action\-explicit tasks, in which the input specifies the complete action sequence \(Figure[4\(b\)](https://arxiv.org/html/2609.19212#S5.F4.sf2)\)\. Such tasks directly evaluate whether a model can parse or execute a novel composition, but have the side effect of reducing the need for abductive reasoning in systematic generalization\. To examine the effect of this design in a controlled setting, we construct an action\-explicit variant of TranSGrid, denoted TranSGrid \(SCAN\), as shown in Figure[4\(c\)](https://arxiv.org/html/2609.19212#S5.F4.sf3)\. TranSGrid \(SCAN\) leaves the encoder input unchanged but provides the decoder with the firstL−1L\-1gold actions, requiring the model to deduce the board state resulting from the given prefix and predict only the final action, thereby substantially reducing the abductive demand\.

Table 3:Comparison results on Test, TranSGrid, and TranSGrid \(SCAN\)\.Δ\\Deltadenotes the gain over the corresponding TranSGrid result\.Table[3](https://arxiv.org/html/2609.19212#S5.T3)shows that providing the gold action prefix consistently improves solve rates across all model sizes and both decoding strategies\. The gains reach 23\.06 percentage points for model ID 5 under greedy decoding and 45\.88 points for model ID 2 under Beam@8\. Under Beam@8, the smallest model improves from 20\.25% to 62\.00%, surpassing its Test result of 36\.38%, while the largest model improves from 61\.60% to 78\.35%, approaching its Test result of 80\.17%\. This recovery occurs because, with all but the final action provided, the model mainly needs to parse or execute the given composition rather than infer a complete sequence from the desired outcome\. These results therefore show that action\-explicit goals substantially reduce the need for abductive reasoning and thereby make the task considerably easier\.

### 5\.3A Reasoning\-Centered Lens on Systematic Generalization

The preceding analyses reveal two important blind spots in existing work on systematic generalization\. Near linear composition suppresses the inductive demand of generalizing from observed action interactions to reusable composition patterns applicable to new task instances, whereas action\-explicit goals bypass the abductive demand of inferring a solution sequence from the desired outcome\. Many prior studies rely on productivity as primary evidence of systematic generalization\. Our results show, however, that productivity is neither necessary for systematic generalization nor sufficient to capture its full reasoning demands\.

A comprehensive evaluation should therefore go beyond productivity and engage all three forms of reasoning\. Specifically, deduction derives consequences from given rules and states; induction discovers reusable rules from observed examples; and abduction infers plausible action sequences that can achieve the desired outcome\. Despite their distinct roles, these reasoning forms are closely coupled: each composition rule involves multiple actions, while longer sequences provide more opportunities for action interactions and concealment\. This interplay is reflected in TranSGrid: longer sequences increase deductive load and provide more opportunities to introduce inductive and abductive demands, while reducing either of the latter makes the task much easier\. We hope that, by capturing these coupled reasoning demands, TranSGrid could advance research on the mechanisms of systematic generalization and support the development of algorithms to improve this capability\.

## 6Conclusion

This paper examines what common simplifications leave untested in systematic generalization through a reasoning\-centered lens\. To support this analysis, we introduce TranSGrid, a controllable testbed that jointly engages deduction, induction, and abduction within a unified task\. Experiments with seven Transformer models on TranSGrid show that solve rates are much lower than those on the standard held\-out test set\. This gap persists on instances whose reference sequence lengths are within the training range, indicating that productivity alone is insufficient to evaluate systematic generalization\. We further introduce TranSGrid \(Decoupled\) and TranSGrid \(SCAN\), which reproduce near linear action composition and action\-explicit goals in TranSGrid respectively\. Both variants improve performance to roughly the held\-out test level, suggesting that these simplifications reduce the inductive and abductive demands of systematic generalization\. Therefore, a comprehensive evaluation of systematic generalization should go beyond productivity and jointly engage deductive, inductive, and abductive reasoning\.

### AI use statement

In this work, we used generative AI tools for designing or providing feedback on research methodology or experiments, implementing methods, and assisting with translation\. We have not used generative AI tools for generating synthetic datasets, helping develop theoretical models or conceptual frameworks, formulating mathematical claims, proposing or refining hypotheses, cleaning and reformatting datasets, supporting qualitative and thematic data analysis, or interpreting results, and providing critical ingredients for proving mathematical claims and assisting in the writing of proofs are not applicable to this work\. Additionally, we used generative AI tools for creating or editing software code, drafting parts of a research paper, summarizing or analyzing existing literature, brainstorming, sourcing/searching for information, editing the paper to improve readability, and proposing a title or keywords for the paper\. We have reviewed all AI\-assisted work: LLM\-generated code was checked and verified for correctness by the authors; all reported results come from experiments we ran and inspected ourselves; AI\-polished text was manually reviewed for factual accuracy, originality, and consistency with our intended claims\. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI\.

### Ethics statement

All datasets used in this work are generated programmatically from numerical grids and predefined transformation rules\. The study does not involve human participants or the collection or use of personal or sensitive data\. Our experiments are conducted entirely within an abstract grid\-transformation environment and aim to advance the understanding of systematic generalization\. Given the scope of the data, task, and experiments, we do not identify any specific ethical concerns arising from this work\. We plan to release the datasets and code for data generation, training, and evaluation to support reproducibility and further research\.

### Reproducibility statement

For our submission, we have uploaded the entirety of the source code as a zipped file that has been properly anonymized\. The source code contains inline documentation that details purpose and usage of different parts of the codebase\. In addition, we also include the full set of model’s responses and the evaluation script\. As discussed in the ethics statement, we plan to more formally release TranSGrid to the public as an open source repository with thorough details that describes the framework, outlines the code, and details its usage\.

## References

- Addepalliet al\.\(2025\)S\. Addepalli, Y\. Varun, A\. Suggala, K\. Shanmugam, and P\. JainDoes safety training of llms generalize to semantically related natural prompts?\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 43611–43631\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Chemero \(2026\)A\. ChemeroAbduction and deduction in dynamical cognitive science\.Topics in Cognitive Science18\(3\),pp\. e12692\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p3.1)\.
- Chenet al\.\(2025\)Y\. Chen, Q\. Yao, J\. Zhang, J\. Cheng, and Y\. BianHierarchical graph tokenization for molecule\-language alignment\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=wpbNczwAwV)Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Chenet al\.\(2026a\)Y\. Chen, T\. Lei, Y\. Li, J\. Cai, Z\. Wu, and Y\. LiuRule\-compliant visual spatial planning for multimodal large language models\.arXiv preprint arXiv:2608\.20237\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Chenet al\.\(2026b\)Y\. Chen, G\. Davidson, and B\. LakeSAGE\-eval: evaluating llms for systematic generalizations of safety facts\.Advances in Neural Information Processing Systems38\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Clooset al\.\(2024\)N\. Cloos, M\. Jens, M\. Naim, Y\. Kuo, I\. Cases, A\. Barbu, and C\. J\. CuevaBaba is ai: break the rules to beat the benchmark\.arXiv preprint arXiv:2407\.13729\.Cited by:[§2](https://arxiv.org/html/2609.19212#S2.p1.1)\.
- Datet al\.\(2025\)D\. H\. Dat, N\. Hyeon\-Woo, P\. Mao, and T\. OhVsc: visual search compositional text\-to\-image diffusion model\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 19153–19162\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Fu and Liu \(2026\)X\. Fu and W\. LiuReinforcement learning for compositional generalization with outcome\-level optimization\.arXiv preprint arXiv:2605\.04920\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p2.1)\.
- Hanet al\.\(2025\)X\. Han, L\. Jin, X\. Liu, and P\. P\. LiangProgressive compositionality in text\-to\-image generative models\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 83268–83290\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Huanget al\.\(2025\)K\. Huang, C\. Duan, K\. Sun, E\. Xie, Z\. Li, and X\. LiuT2i\-compbench\+\+: an enhanced and comprehensive benchmark for compositional text\-to\-image generation\.IEEE Transactions on Pattern Analysis and Machine Intelligence47\(5\),pp\. 3563–3579\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Hupkeset al\.\(2020\)D\. Hupkes, V\. Dankers, M\. Mul, and E\. BruniCompositionality decomposed: how do neural networks generalise?\.Journal of Artificial Intelligence Research67,pp\. 757–795\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p2.1),[§2](https://arxiv.org/html/2609.19212#S2.p1.1),[§5\.1](https://arxiv.org/html/2609.19212#S5.SS1.p3.1),[§5\.2](https://arxiv.org/html/2609.19212#S5.SS2.p3.1)\.
- Jabbaret al\.\(2025\)A\. Jabbar, C\. Condoravdi, and C\. PottsDistinguishing fair from unfair compositional generalization tasks\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 20796–20807\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.1133/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.1133),ISBN 979\-8\-89176\-335\-7Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Jiet al\.\(2023\)Y\. Ji, L\. Zhang, J\. Wu, B\. Wu, L\. Li, L\. Huang, T\. Xu, Y\. Rong, J\. Ren, D\. Xue,et al\.Drugood: out\-of\-distribution dataset curator and benchmark for ai\-aided drug discovery–a focus on affinity prediction problems with noise annotations\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.37,pp\. 8023–8031\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Kim and Linzen \(2020\)N\. Kim and T\. LinzenCOGS: a compositional generalization challenge based on semantic interpretation\.InProceedings of the 2020 conference on empirical methods in natural language processing \(emnlp\),pp\. 9087–9105\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1),[§2](https://arxiv.org/html/2609.19212#S2.p1.1),[§4\.2](https://arxiv.org/html/2609.19212#S4.SS2.p1.1),[§5\.2](https://arxiv.org/html/2609.19212#S5.SS2.p3.1)\.
- Kumon and Yanaka \(2025\)R\. Kumon and H\. YanakaAnalyzing the inner workings of transformers in compositional generalization\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 8529–8540\.Cited by:[§4\.2](https://arxiv.org/html/2609.19212#S4.SS2.p1.1)\.
- Kuoet al\.\(2021\)Y\. Kuo, B\. Katz, and A\. BarbuCompositional networks enable systematic generalization for grounded language understanding\.InFindings of the Association for Computational Linguistics: EMNLP 2021,pp\. 216–226\.Cited by:[§2](https://arxiv.org/html/2609.19212#S2.p1.1)\.
- Lake and Baroni \(2018\)B\. Lake and M\. BaroniGeneralization without systematicity: on the compositional skills of sequence\-to\-sequence recurrent networks\.InInternational conference on machine learning,pp\. 2873–2882\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1),[§1](https://arxiv.org/html/2609.19212#S1.p2.1),[§2](https://arxiv.org/html/2609.19212#S2.p1.1),[§5\.1](https://arxiv.org/html/2609.19212#S5.SS1.p3.1),[§5\.2](https://arxiv.org/html/2609.19212#S5.SS2.p3.1)\.
- Lake and Baroni \(2023\)B\. M\. Lake and M\. BaroniHuman\-like systematic generalization through a meta\-learning neural network\.Nature623\(7985\),pp\. 115–121\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1),[§1](https://arxiv.org/html/2609.19212#S1.p2.1),[§4\.2](https://arxiv.org/html/2609.19212#S4.SS2.p1.1)\.
- Liet al\.\(2026a\)J\. Li, J\. Li, W\. Wang, Y\. Liu, C\. Zheng, Y\. Bian, D\. Zhou, X\. Wei, and Q\. LiSpeak\-to\-structure: evaluating llms in open\-domain natural language\-driven molecule generation\.InProceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 2,pp\. 9314–9325\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Liet al\.\(2026b\)J\. Li, W\. Wang, C\. Zheng, S\. Zhang, Y\. Bian, X\. Wei, and Q\. LiDo llms truly generalize in the molecular domain? a perturbation\-based analysis\.arXiv preprint arXiv:2607\.01800\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Liet al\.\(2023\)Q\. Li, S\. Huang, Y\. Hong, Y\. Zhu, Y\. N\. Wu, and S\. ZhuA minimalist dataset for systematic generalization of perception, syntax, and semantics\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=kIPyTuEZuAK)Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p2.1),[§2](https://arxiv.org/html/2609.19212#S2.p1.1),[§4\.2](https://arxiv.org/html/2609.19212#S4.SS2.p1.1),[§5\.1](https://arxiv.org/html/2609.19212#S5.SS1.p3.1),[§5\.2](https://arxiv.org/html/2609.19212#S5.SS2.p3.1)\.
- Mondorfet al\.\(2026\)P\. Mondorf, S\. Zhou, M\. Riedler, and B\. PlankCompositional\-ARC: assessing systematic generalization in abstract spatial reasoning\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=h497VpgFKd)Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p2.1)\.
- Peirce \(1934\)C\. S\. PeirceCollected papers of charles sanders peirce\.Vol\.5,Harvard University Press\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p3.1)\.
- Ruiset al\.\(2020\)L\. Ruis, J\. Andreas, M\. Baroni, D\. Bouchacourt, and B\. M\. LakeA benchmark for systematic generalization in grounded language understanding\.Advances in neural information processing systems33,pp\. 19861–19872\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1),[§2](https://arxiv.org/html/2609.19212#S2.p1.1),[§5\.1](https://arxiv.org/html/2609.19212#S5.SS1.p3.1)\.
- Shank \(1998\)G\. ShankThe extraordinary ordinary powers of abductive reasoning\.Theory & psychology8\(6\),pp\. 841–860\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p3.1)\.
- Sikarwaret al\.\(2022\)A\. Sikarwar, A\. Patel, and N\. GoyalWhen can transformers ground and compose: insights from compositional generalization benchmarks\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,pp\. 648–669\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Spilsburyet al\.\(2024\)S\. Spilsbury, P\. Marttinen, and A\. IlinGenerating demonstrations for in\-context compositional generalization in grounded language learning\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 15960–15991\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.893/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.893)Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1)\.
- Wuet al\.\(2021\)Z\. Wu, E\. Kreiss, D\. Ong, and C\. PottsReaSCAN: compositional reasoning in language grounding\.InThirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 1\),External Links:[Link](https://openreview.net/forum?id=Rtquf4Jk0jN)Cited by:[§2](https://arxiv.org/html/2609.19212#S2.p1.1)\.
- Wuet al\.\(2023\)Z\. Wu, C\. D\. Manning, and C\. PottsRecogs: how incidental details of a logical form overshadow an evaluation of semantic interpretation\.Transactions of the Association for Computational Linguistics11,pp\. 1719–1733\.Cited by:[§1](https://arxiv.org/html/2609.19212#S1.p1.1),[§1](https://arxiv.org/html/2609.19212#S1.p2.1)\.

## Appendix AInduction Rules

An induction rule is a short program of three to five atomic actions whose effects cancel or overwrite one another, leaving a net transformation that permutes only a few cells within a small local window\. Table[4](https://arxiv.org/html/2609.19212#A1.T4)lists the twelve induction rules used in this paper, together with their constituent action sequences and net effects\. Here,\(r,c\)\(r,c\)denotes the top\-left corner of the local window\. For a block operation such asBLOCK\_CW\(r\+i,c\+j\)\(r\{\+\}i,c\{\+\}j\), the two arguments specify the top\-left cell of the2×22\\times 2block to be rotated, whereiiandjjare offsets from the rule origin\. For row and column operations,r\+ir\{\+\}iandc\+jc\{\+\}jspecify the corresponding row and column indices, respectively\. Collectively, the twelve rules realize localized permutations of two to five cells, including pairwise swaps, three to five cell cycles, and two disjoint swaps arranged in horizontal, vertical, diagonal, L\-shaped, or bent configurations\. The actions in each program are executed from top to bottom\. The final column illustrates the resulting net effect using a concrete example\. For each example, the values ofrrandccare provided above the boards, and the cells involved in the permutation are highlighted in red\.

Table 4:The twelve induction rules used in this paper, together with their constituent action sequences and illustrative net effects\. In each illustration, the rule origin\(r,c\)\(r,c\)is indicated above the boards, and the cells permuted by the rule are highlighted in red\.Induction RuleAction sequenceIllustrative net effecthorizontal\_swap\_mixedBLOCK\_CW\(r,c\)\(r,c\)COL\_DOWN\(c\)\(c\)ROW\_LEFT\(r\+1\)\(r\{\+\}1\)COL\_UP\(c\)\(c\)ROW\_RIGHT\(r\+1\)\(r\{\+\}1\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/horizontal_swap_mixed.png)diagonal\_swap\_blocksBLOCK\_CW\(r,c\+1\)\(r,c\{\+\}1\)BLOCK\_CCW\(r\+1,c\)\(r\{\+\}1,c\)BLOCK\_CCW\(r,c\)\(r,c\)BLOCK\_CCW\(r,c\+1\)\(r,c\{\+\}1\)BLOCK\_CW\(r\+1,c\)\(r\{\+\}1,c\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/diagonal_swap_blocks.png)bent\_four\_cycleBLOCK\_CW\(r,c\)\(r,c\)BLOCK\_CW\(r,c\+1\)\(r,c\{\+\}1\)BLOCK\_CCW\(r,c\)\(r,c\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/bent_four_cycle.png)wide\_bent\_four\_cycleBLOCK\_CW\(r,c\+1\)\(r,c\{\+\}1\)BLOCK\_CW\(r,c\)\(r,c\)BLOCK\_CCW\(r,c\+1\)\(r,c\{\+\}1\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/wide_bent_four_cycle.png)tall\_bent\_four\_cycleBLOCK\_CW\(r,c\)\(r,c\)BLOCK\_CW\(r\+1,c\)\(r\{\+\}1,c\)BLOCK\_CCW\(r,c\)\(r,c\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/tall_bent_four_cycle.png)tall\_bent\_flippedBLOCK\_CW\(r\+1,c\)\(r\{\+\}1,c\)BLOCK\_CW\(r,c\)\(r,c\)BLOCK\_CCW\(r\+1,c\)\(r\{\+\}1,c\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/tall_bent_flipped.png)horizontal\_three\_cycleROW\_LEFT\(r\)\(r\)COL\_LEFT\(c\+1\)\(c\{\+\}1\)ROW\_RIGHT\(r\)\(r\)COL\_LEFT\(c\+1\)\(c\{\+\}1\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/horizontal_three_cycle.png)vertical\_three\_cycleCOL\_UP\(c\)\(c\)ROW\_DOWN\(r\)\(r\)COL\_DOWN\(c\)\(c\)ROW\_DOWN\(r\)\(r\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/vertical_three_cycle.png)l\_three\_cycleCOL\_UP\(c\)\(c\)ROW\_LEFT\(r\)\(r\)COL\_DOWN\(c\)\(c\)ROW\_RIGHT\(r\)\(r\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/l_three_cycle.png)double\_horizontal\_swapBLOCK\_CW\(r,c\)\(r,c\)BLOCK\_CW\(r,c\+1\)\(r,c\{\+\}1\)BLOCK\_CCW\(r,c\)\(r,c\)BLOCK\_CCW\(r,c\+1\)\(r,c\{\+\}1\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/double_horizontal_swap.png)five\_cycle\_2x3BLOCK\_CW\(r,c\)\(r,c\)COL\_RIGHT\(c\+1\)\(c\{\+\}1\)BLOCK\_CCW\(r,c\)\(r,c\)COL\_RIGHT\(c\+1\)\(c\{\+\}1\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/five_cycle_2x3.png)vertical\_swap\_blocksBLOCK\_CW\(r,c\)\(r,c\)BLOCK\_CW\(r\+1,c\)\(r\{\+\}1,c\)BLOCK\_CCW\(r,c\+1\)\(r,c\{\+\}1\)BLOCK\_CCW\(r\+1,c\)\(r\{\+\}1,c\)BLOCK\_CW\(r,c\+1\)\(r,c\{\+\}1\)\(r,c\)=\(0,0\)\(r,c\)=\(0,0\)![[Uncaptioned image]](https://arxiv.org/html/2609.19212v1/vertical_swap_blocks.png)
## Appendix BEstimating the Effective Action Length

LetDDdenote the number of cells whose values differ between the initial and target boards\. To convertDDinto an effective action length, we first estimate the expected footprint of a random trajectory:

g⁡\(ℓ\)=𝔼⁡\[D∣ℓ\],g\(\\ell\)=\\mathbb\{E\}\[D\\mid\\ell\],whereℓ\\elldenotes the action\-sequence length\. For eachℓ∈\{1,…,12\}\\ell\\in\\\{1,\\ldots,12\\\}, we sample400400random trajectories of lengthℓ\\ellon6×66\\times 6boards whose cell values are drawn independently from\{0,…,9\}\\\{0,\\ldots,9\\\}\. We execute each trajectory, count the cells whose values differ between its initial and final boards, and average these counts\. The resulting footprint curve is reported in Table[5](https://arxiv.org/html/2609.19212#A2.T5)\.

Table 5:The estimated footprint curve\. Each value is averaged over 400 random trajectories of the corresponding length\.We obtainLeffL\_\{\\mathrm\{eff\}\}by inverting this curve through linear interpolation:

Leff​\(D\)=\{Dg⁡\(1\),0≤D≤g⁡\(1\),ℓ\+D−g⁡\(ℓ\)g⁡\(ℓ\+1\)−g⁡\(ℓ\),g\(ℓ\)<D≤g\(ℓ\+1\),ℓ∈\{1,…,11\},12,D\>g⁡\(12\)\.L\_\{\\mathrm\{eff\}\}\(D\)=\\begin\{cases\}\\dfrac\{D\}\{g\(1\)\},&0\\leq D\\leq g\(1\),\\\\\[6\.0pt\] \\ell\+\\dfrac\{D\-g\(\\ell\)\}\{g\(\\ell\+1\)\-g\(\\ell\)\},&g\(\\ell\)<D\\leq g\(\\ell\+1\),\\quad\\ell\\in\\\{1,\\ldots,11\\\},\\\\\[8\.0pt\] 12,&D\>g\(12\)\.\\end\{cases\}ForD≤g⁡\(1\)D\\leq g\(1\), the effective length is interpolated between\(D,Leff\)=\(0,0\)\(D,L\_\{\\mathrm\{eff\}\}\)=\(0,0\)and\(g⁡\(1\),1\)\(g\(1\),1\)\. Values between adjacent entries of the footprint curve are linearly interpolated, while values aboveg⁡\(12\)g\(12\)are assigned an effective action length of1212\.

## Appendix CModel Architectures

We evaluate seven encoder\-decoder Transformer models that vary in depth and hidden dimensions\. In every configuration, the encoder and decoder contain the same number of layers\. The encoder jointly processes the initial and target boards, while the decoder generates the action sequence autoregressively\. Each action is represented by an action\-name token followed by its argument tokens, and decoding terminates when the model generates theEOStoken\.

Table 6:Details of the seven Transformer models used in this paper\. Model IDs are ordered by the total number of trainable parameters\.Table[6](https://arxiv.org/html/2609.19212#A3.T6)summarizes the architectural configurations of the seven models\. Here,dmodeld\_\{\\mathrm\{model\}\}is the number of units in each Transformer bottleneck layer,dff=4​dmodeld\_\{\\mathrm\{ff\}\}=4d\_\{\\mathrm\{model\}\}is the number of units in each feed\-forward sublayer, and Heads is the number of attention heads\. The reported parameter counts include all trainable parameters under theGRID\+PAIRencoding used in the main experiments\.

## Appendix DDataset Construction and Composition

#### Instance construction\.

Each instance begins with an initial boardB\(0\)∈𝒱6×6B^\{\(0\)\}\\in\\mathcal\{V\}^\{6\\times 6\}, whose cells are sampled independently from𝒱=\{0,…,9\}\\mathcal\{V\}=\\\{0,\\ldots,9\\\}\. We generate a reference action sequenceπ=\(a1,…,aL\)\\pi=\(a\_\{1\},\\ldots,a\_\{L\}\)and execute it to obtain the target board:

B⋆=T⁡\(π,B\(0\)\)\.B^\{\\star\}=T\(\\pi,B^\{\(0\)\}\)\.The reference sequence guarantees the existence of a valid solution, but it need not be the shortest or unique solution\. To construct an instance with inductive loadKK, we sampleKKinduction rules with replacement, subject to∑k=1K\|ρk\|≤L\\sum\_\{k=1\}^\{K\}\|\\rho\_\{k\}\|\\leq L, and instantiate each rule at a valid board location\. We fill the remainingL−∑k=1K\|ρk\|L\-\\sum\_\{k=1\}^\{K\}\|\\rho\_\{k\}\|positions with randomly sampled atomic actions\. The instantiated rules and individual filler actions are then shuffled as blocks, preserving the internal action order of each rule\. The induction rules are defined in Appendix[A](https://arxiv.org/html/2609.19212#A1)\.

#### Dataset composition\.

The TranSGrid evaluation set contains 4,800 instances and is marginally balanced with respect to reference sequence lengthLLand inductive loadKK\. Each lengthL∈\{1,…,12\}L\\in\\\{1,\\ldots,12\\\}contains 400 instances, and each loadK∈\{0,…,3\}K\\in\\\{0,\\ldots,3\\\}contains 1,200 instances\. Because every induction rule contains at least three actions, a feasible pair must satisfyL≥3​KL\\geq 3K\. Table[7](https://arxiv.org/html/2609.19212#A4.T7)reports the resulting joint distribution\.

Table 7:Joint distribution of the4,8004\{,\}800TranSGrid evaluation instances by reference sequence lengthLLand inductive loadKK\.SubsetReference lengthLLInductive loadKKTotal00112233Easy11400–––40022400–––4003366334––4004465335––400Medium5571329––400663631333–400771626358–400883034336–400Hard99352843294400101034303929740011112427463034001212232645306400Total1,2001,2001,2001,2004,800

## Appendix EInput Encoding Variants

#### Encoding variants\.

We compare two board encodings and two positional encoding schemes\. Under thePAIRencoding, the values at corresponding locations of the initial and target boards are represented jointly by a single token, producing an input sequence of3636tokens\. Under theBOARDencoding, the two boards are serialized separately, producing an input sequence of8686tokens\. For positional encoding,GRIDrepresents each position as the sum of learned embeddings for its board, row, and column, whereasFLATassigns a learned embedding to each absolute sequence position\. Combining these choices yields four variants:GRID\+PAIR,FLAT\+PAIR,GRID\+BOARD, andFLAT\+BOARD\.

Table 8:Development\-set solve rates \(%\) for the four input encoding variants across the seven models\. A dagger indicates that the run had not fully converged by the end of training\.
#### Evaluation protocol\.

We evaluate all four variants using each of the seven model architectures in Appendix[C](https://arxiv.org/html/2609.19212#A3), resulting in2828runs\. All runs use the same3030M training instances, eight training epochs, a batch size of512512, and a random seed of4242\. We use AdamW with a peak learning rate of1\.5×10−41\.5\\times 10^\{\-4\},1,0001\{,\}000warmup steps, weight decay of0\.010\.01, gradient clipping at1\.01\.0, and dropout of0\.10\.1\. Table[8](https://arxiv.org/html/2609.19212#A5.T8)reports greedy solve rates on the held\-out development set after the final epoch\.

#### Encoding selection\.

For model ID 7, the three converged variants differ by only0\.330\.33percentage points, indicating that their final solve rates are effectively comparable\. ThePAIRencoding nevertheless provides a shorter input sequence and more stable optimization: all1414PAIRruns converged within the training budget, whereas threeBOARDruns did not\. Within thePAIRencoding,GRIDandFLATachieve nearly identical solve rates at the largest model sizes\. Nevertheless, across all seven model sizes,GRID\+PAIRachieves the highest solve rate in three cases, more than any other variant, and is therefore selected for the main experiments\.