在类ARC任务中使用测试时任务嵌入的隐式规则归纳

# Implicit Rule Induction with Test-Time Task Embeddings in ARC-like Tasks
Source: [https://arxiv.org/html/2609.21181](https://arxiv.org/html/2609.21181)
Adrien Deliège††thanks:Equal contribution\. Correspondence toadrien\.deliege@uliege\.beandclaasbeger@santafe\.eduClaas Beger11footnotemark:1Affiliation:Santa Fe InstituteMarc Van DroogenbroeckAffiliation:University of LiègeMelanie MitchellAffiliation:Santa Fe Institute

###### Abstract

The Abstraction and Reasoning Corpus and related benchmarks evaluate whether AI models can solve novel reasoning tasks, but often leave unclear whether success reflects inference of the intended underlying rule or reliance on shortcuts\. We address this gap by studying test\-time task embeddings in Vision ARC \(VARC\), a model in which a pre\-trained backbone is complemented by a trainable embedding representing the transformation rule\. In the original VARC, test\-time training \(TTT\) is jointly applied to the backbone and task embedding\. Here we introduce a novel two\-step TTT protocol: first finetune only the task embedding \(Embed\-TTT\), then freeze it and finetune the backbone\. Across ARC\-AGI\-1, ConceptARC, and two controlled datasets with known rules, Embed\-TTT consistently yields improved task embeddings, ones that align better with underlying task rules, improve embedding\-based retrieval, and enable accurate linear probing of known rules\. Qualitatively, Embed\-TTT identifies more semantically meaningful relations between test and train tasks on ARC\-AGI\-1\. We also show that optimizing only task embeddings \(less than 0\.01% of model parameters\) already solves a non\-trivial fraction of ARC\-AGI\-1, ConceptARC and Mini\-ARC tasks, while the full two\-step pipeline improves final performance\. Finally, we show that Embed\-TTT recovers the underlying geometric structure of parametric rules and learns compositional capabilities that enable rule\-wise interpolation, but not extrapolation\. These findings support a clearer separation between rule induction and rule execution in ARC\-like evaluations, motivating benchmarks that better distinguish in\-distribution from out\-of\-distribution rules\.

## 1Introduction

The Abstraction and Reasoning Corpus \(ARC\) was introduced to evaluate a form of humanlike fluid intelligence that is difficult to capture with standard machine learning benchmarks\([Chollet, 2019](https://arxiv.org/html/2609.21181#bib.bib3)\)\. In each task, a model is given only a few input\-output grid demonstrations and must infer the transformation rule that maps inputs to outputs\. This setup has inspired a growing family of ARC\-like benchmarks for studying humanlike abstraction, few\-shot adaptation, and systematic generalization\. However, standard ARC\-like evaluations typically measure whether a model produces the correct output, not whether it has inferred the intended rule\. This distinction is important: a model may solve a task by inducing the underlying rule, but it may also exploit dataset\-specific shortcuts, memorize related patterns, or rely on test\-time optimization without forming a reusable representation of the task\([Beger et al\., 2025](https://arxiv.org/html/2609.21181#bib.bib2)\)\. Conversely, a model may identify a useful rule but fail to execute it perfectly\. As a result, task accuracy alone conflates rule induction, execution and memorization\. This issue is amplified by the fact that most ARC\-like datasets do not provide the rules or generators underlying their tasks\. Consequently, it is usually unclear whether rules underlying test tasks are in or out\-of\-distribution with respect to the rules known from training tasks, where training tasks would be drawn from the ARC\-AGI\-1 training set and test tasks from ARC\-AGI\-1 evaluation set, for example\. From a classical machine learning perspective, this makes benchmark results of models trained on the training tasks difficult to interpret: strong performance may reflect genuine abstraction, but it may also reflect favorable overlap between train and test rules\. We therefore argue that ARC\-like datasets and evaluation should more explicitly study whether models induce meaningful task representations, and should distinguish rule induction from rule execution whenever possible\.

In this paper, we study implicit rule induction through the task embeddings of Vision ARC \(VARC\)\([Hu et al\., 2026](https://arxiv.org/html/2609.21181#bib.bib7)\), a ViT\-based model that represents each task with a trainable embedding, which steers the output grid generation in a separate model backbone via a latent representation of the underlying rule\. We show that the default VARC test\-time training procedure, which jointly finetunes the task embedding and the model backbone, produces poorly aligned test\-train task embeddings that do not retain embedding space information from the training stage\. We propose a simple alternative, Embed\-TTT: at test time, we freeze the backbone and optimize only the task embedding\. This forces test tasks to be represented in the same embedding space as training tasks, as both pass through the same frozen backbone, making the resulting embeddings more suitable for analyzing whether the model has identified the underlying rule\. We evaluate these embeddings on ARC\-AGI\-1 \(written ARC\-1 for brevity\)\([Chollet, 2019](https://arxiv.org/html/2609.21181#bib.bib3)\), ConceptARC\([Moskvichev et al\., 2023](https://arxiv.org/html/2609.21181#bib.bib15)\), and two controlled ARC\-like datasets with known rules\. When ground\-truth rules are unavailable, we test whether duplicated training tasks recover their original embeddings\. When rules are available, we evaluate whether ground\-truth rule labels or parameters can be decoded from embeddings using linear probes\. Across these settings, Embed\-TTT yields substantially more rule\-aligned embeddings than default full\-model test\-time training, while also supporting increased task\-solving performance when followed by optional backbone finetuning\. Overall, we make the followingcontributions:

- •We propose a novel test\-time training protocol, Embed\-TTT, for VARC, which produces semantically meaningful task embeddings for ARC\-like tasks\. To evaluate Embed\-TTT against the default full model test\-time training, we further introduce an evaluation methodology for implicit rule induction\.
- •We show quantitatively that Embed\-TTT embeddings align better with task rules across four ARC\-like datasets, and we provide qualitative evidence of meaningful test–train semantic similarity in ARC\-1\. In addition, optimizing only task embeddings solves 17% of ARC\-1 test tasks, and nearest\-neighbor concept transfer from ConceptARC to ARC\-1 yields 36% correct concept associations, without training for that transfer objective nor for concept classification\.
- •We analyze the task embedding space on a specific case and characterize which unseen tasks remain solvable\. We show that task embeddings recover the dataset geometry and support compositional rule interpolation, reframing ARC\-like performance through a standard in\-/out\-of\-distribution lens\.

## 2Related work

### 2\.1Problem statement

Definitions\.An*ARC\-like dataset*is a set of*tasks*, and a task is a set of\(x,y\)\(x,y\)pairs governed by a deterministic*rule*that maps each*input*xxto its*output*yy\. In theory, the rule can range from a simple function to a complex algorithm, but implicitly it is commonly accepted that the rule should be expressible in natural language and understandable by human readers\. Following the original ARC\-1 format, eachxxand eachyyis a two\-dimensional grid of integers, such that a task can be viewed as a visual puzzle, where each cell of the grid is colored according to its integer value\. Typically, each task is split into demonstration pairs\(xi,yi\)\(x\_\{i\},y\_\{i\}\)and a test inputx′x^\{\\prime\}, such that a solver should infer a common ruleRRthat maps eachxix\_\{i\}toyiy\_\{i\}and then should applyRRto map the test inputx′x^\{\\prime\}toyiy\_\{i\}\. To elicit in\-context test\-time learning, an ARC\-like dataset is usually split into training tasks and test tasks, such that one should learn generic task\-solving strategies on the training tasks, and successfully apply them on the test tasks without any human intervention\. Performance\.The performance of a model on an ARC\-like dataset is the proportion of test tasks solved\. A test task is solved when the test inputx′x^\{\\prime\}is correctly mapped to its ground\-truth outputy′y^\{\\prime\}, which means reaching 100% cell\-wise accuracy on they′y^\{\\prime\}\. In practice, several predictions can be made for a given test inputx′x^\{\\prime\}, and the task is considered solved if one of two predictions matches the ground truth \(“pass @ 2”\)\. In general for ARC\-like benchmarks, while ground\-truth outputsy′y^\{\\prime\}are given for each task, the explicit rules underlying the tasks are not provided, which makes it difficult to evaluate whether the model actually learned the intended logic behind the tasks\.

### 2\.2Approaches

Implicit and explicit reasoning\.Several works have pointed out the tendency of large language models \(LLMs\) to rely on shortcuts or shallow reasoning on tasks requiring analogy or abstraction\([Beger et al\., 2025](https://arxiv.org/html/2609.21181#bib.bib2);[Mineault et al\., 2026](https://arxiv.org/html/2609.21181#bib.bib14);[Lewis and Mitchell, 2025](https://arxiv.org/html/2609.21181#bib.bib11)\)\. Many benchmarks targeting these abilities are susceptible to such behavior, as they primarily evaluate the correctness of the final output rather than the actual reasoning process\. Recent work has therefore shifted toward learning more robust representations, either internally through structured latent models such as World Models\([Piriyakulkij et al\., 2025](https://arxiv.org/html/2609.21181#bib.bib16);[Maasch et al\., 2025](https://arxiv.org/html/2609.21181#bib.bib13)\), or externally via explicit reasoning artifacts, for instance in program synthesis approaches\([Langenfeld et al\., 2026](https://arxiv.org/html/2609.21181#bib.bib10);[Li et al\., 2025](https://arxiv.org/html/2609.21181#bib.bib12)\)\. In this work, we aim to bridge these perspectives by enforcing the model to store its internal representation in an explicit vector that can be directly extracted and “executed,” for instance through linear projections\. Embedding vectors\.The extraction of embedding or activation vectors is a common technique in interpretability research\.[Todd et al\. \(2024\)](https://arxiv.org/html/2609.21181#bib.bib17)identify sets of attention heads that implement simple functions, such as word translation or antonym mapping, and show that combining such function vectors yields non\-trivial compositions of functionality\. We extend this line of work to models applied to abstract reasoning tasks by introducing a test\-time tuning regime that encourages the formation of explicit task embeddings\. We analyze the extent to which these embeddings capture underlying rules, including their compositional properties and limitations\. We also study similarities between embeddings of tasks that share related rules\. Related observations, albeit in a more correlational setting, have been reported by[Lake and Baroni \(2018\)](https://arxiv.org/html/2609.21181#bib.bib9);[Hill et al\. \(2019\)](https://arxiv.org/html/2609.21181#bib.bib5);[Veldkamp et al\. \(2023\)](https://arxiv.org/html/2609.21181#bib.bib18)\. Test\-time adaptation in abstract reasoning tasks\.Test\-time training \(TTT\) has been used in several high\-performing systems for solving ARC\-1 tasks\([Li et al\., 2025](https://arxiv.org/html/2609.21181#bib.bib12);[Chollet et al\., 2024](https://arxiv.org/html/2609.21181#bib.bib4);[Hu et al\., 2026](https://arxiv.org/html/2609.21181#bib.bib7);[Akyürek et al\., 2025](https://arxiv.org/html/2609.21181#bib.bib1)\)\. TTT typically works by generating additional training examples through simple augmentations at test time\. However, while TTT improves model performances on ARC\-1, it remains unclear whether TTT leads to the induction of meaningful task representations or instead relies on unconstrained parameter updates\. One of the motivations of our work is to study this distinction, by constraining TTT to operate primarily through explicit task embeddings rather than full model finetuning\. Vision ARC \(VARC\)\.The current best\-performing models on ARC\-like datasets are closed\-source LLMs, from which internal representations cannot be extracted\. A rare exception is the Vision ARC \(VARC\) model\([Hu et al\., 2026](https://arxiv.org/html/2609.21181#bib.bib7)\), which is a ViT\-based model trained from scratch that explicitly models each task through a specific task token, numerically encoded as a512512\-dimensional array called a*task embedding*\. During training, the model learns to solve training tasks as image\-to\-image translation problems, with each problem conditioned by its corresponding trainable task embedding\. Then, for a given test task, the model randomly initializes the task embedding and follows a TTT pipeline, which learns a new task embedding jointly with finetuning the trained backbone \(without the task embedding part\) on the test task at hand\. We call this process the baseline Full\-TTT approach\.

## 3Method

### 3\.1Obtaining rule\-inductive test\-time task embeddings

Issues with test embeddings\.The original VARC paper\([Hu et al\., 2026](https://arxiv.org/html/2609.21181#bib.bib7)\)shows that*train*task embeddings seemingly cluster semantically similar tasks and shared \(sub\-\)rules, suggesting that embeddings can support implicit rule induction\. However,*test*embeddings produced by Full\-TTT were not analyzed\. In our preliminary experiments, test embeddings exhibit a strong distribution shift from train embeddings \(in both direction and magnitude\) and often stay close to their random initialization\. As a result, semantic neighborhood structure is largely lost, weakening rule\-induction usefulness at test time\. Although this mismatch does not drastically hurt final task\-solving performance, it raises efficiency and interpretability concerns: even for duplicates of training tasks presented at test time, the pipeline finetunes 18M parameters per task yet fails to recover the corresponding train embeddings\. Moreover, we observed that freezing the task embedding \(random or zero initialization\) causes little performance drop, indicating that the backbone can ignore it\. We argue that a desirable model should produce consistent, rule\-inductive task embeddings in both training and test phases\.

![Refer to caption](https://arxiv.org/html/2609.21181v1/method.png)Figure 1:Method\.After training a model that learns task embeddings that condition the backbone \(left\), we elicit implicit rule induction at test time: we first freeze the backbone and learn a task embedding \(Embed\-TTT, center\), then freeze the embedding and finetune the backbone \(right\)\.A two\-step finetuning approach\.Drawing on the previous observations, we derive a TTT protocol that naturally allows learning of meaningful test task embeddings while also improving performance capabilities\. It consists of a two\-step finetuning approach, shown in[Figure1](https://arxiv.org/html/2609.21181#S3.F1)\. When faced with a test task, we first freeze the model backbone and learn only the task embedding\. We denote this step as Embed\-TTT\. We then freeze the learned task embedding and finetune the model backbone \(Embed\-Full\-TTT\)\. Embed\-TTT enables positioning the test task embedding in the same embedding space as the training task embeddings because they share the same model backbone, while Embed\-Full\-TTT allows the model to adjust to the learned embedding to solve the task efficiently\.

### 3\.2Evaluating implicit rule induction through task embeddings

Rules unavailable: training embeddings retrieval\.In datasets like ARC, where each task is unique, automated evaluation of rule induction largely reduces to checking whether duplicate training tasks retrieve their originally learned embeddings at test time\. More precisely, we compute the proportion of tasks for which the original embedding is within the top\-k closest embeddings of the newly TTT\-produced embedding\. Limitations of this approach are the fact that memorization alone might suffice to retrieve the original task embedding, the fact that multiple embeddings might solve the tasks, and the fact that, for training tasks that the model was not able to solve, different embeddings might be better suited than the original one\. Consequently, this default protocol should be interpreted with caution: it provides only indirect evidence of meaningful implicit rule induction at test time, mostly reliable when test rules are in\-distribution with respect to the training rules distribution\.

Rules available: linear probing to rules\.When rules \(or concept groups\) are defined and task generators can produce multiple task instances per rule, in\-distribution rule induction can be evaluated more cleanly\. The setup mirrors self\-supervised learning: learn training embeddings through an auxiliary objective, then assess test embeddings with a linear probe on a downstream target\. In our ARC\-like setting, each data “unit” is a task, and the auxiliary objective is solving the task by finetuning its task embedding\. Given rule labels for training tasks, we collect train embeddings, fit a linear probe from embeddings to rules, and apply it to test embeddings\. We probe performance then measure how much rule information is encoded in the embeddings\. We distinguish two cases:

- •Non\-parametric rules\.Linear probing is a classification problem, and we measure implicit rule induction by the accuracy on the test tasks of the linear classifier learned from the train tasks\.
- •Parametric rules\.Linear probing is a linear regression problem, and we measure the root mean squared error \(RMSE\) between linearly regressed and ground\-truth parameters on the test tasks from the linear regressor learned from the train tasks\.

## 4Results

### 4\.1Quantitative in\-distribution implicit rule induction evaluation

Datasets and experimental details\.We conduct the following experiments\.

- •ARC\-1\. We start from the trained VARC of[Hu et al\. \(2026\)](https://arxiv.org/html/2609.21181#bib.bib7)and perform Full\-TTT and Embed\-TTT on duplicates of the training tasks to measure train embedding retrieval accuracy\.
- •ConceptARC\. We train a VARC model from scratch and perform the same experiment as for ARC\-1\. We also evaluate the linear probing classification accuracy by considering the 16 concepts underlying the tasks as proxies for the rules that generated them\. Since no train\-test split exists, we perform a 5\-fold cross\-validation, each time learning the linear mapping from 8 tasks per concept and evaluating it on the remaining 2 tasks\. We report the average accuracy across the runs\.
- •Custom “LUCD”\. We design a simple dataset with known rules in the*non\-parametric*setting\. Each task consists in applying zero, one, or two transforms to a single\-cell object positioned randomly on a10×1010\\times 10grid, sampled \(with repetition enabled\) from 4 “atomic” transforms: move one cell to the left \(L\), up \(U\), dilate in the cardinal directions \(C\), dilate in all directions \(D\)\. Since all atomic rules commute with each other, this yields 15 different possible rules\.
- •Custom “Moves”\. We design a simple dataset with known rules in the*parametric*setting\. Each task consists in moving a single\-cell object on a10×1010\\times 10grid by at most five cells both horizontally \(left \(L\), right \(R\)\) and vertically \(up \(U\), down \(D\)\), yielding a total of 121 different rules \(for instance, “L4\-D3” is the rule moving 4 cells to the left and 3 down, associated to coordinates \(\-4,\-3\)\)\.

For “LUCD” \(resp\. “Moves”\), we generate 1 training task and 10 test tasks per rule, with 10 training and 10 test input\-output pairs per task\(we provide examples in[Appendix A](https://arxiv.org/html/2609.21181#A1)\(resp\.[Appendix B](https://arxiv.org/html/2609.21181#A2)\)\)\. We evaluate rule induction capabilities on these datasets as described in[Section3\.2](https://arxiv.org/html/2609.21181#S3.SS2)\.

Results\.[Table1](https://arxiv.org/html/2609.21181#S4.T1)reports the results of the experiments\. Across all evaluations, Embed\-TTT largely outperforms the baseline Full\-TTT, which barely outperforms pure chance of retrieving or classifying the embeddings correctly\. Distances defining neighbor ordering of embeddings are computed as cosine similarity distances\. Also, we observed that tasks successfully solved are generally better retrieved \(91\.2% with Embed\-TTT on ARC\) than failed tasks \(77\.1%\)\. In the case of ConceptARC, it should be noted that the concepts are only a meta\-category associated with the tasks and do not fully characterize the task\-generating process\. Therefore, observing a perfect mapped accuracy is likely out of reach, but Embed\-TTT performs reasonably well, with 60\.6% accuracy \(chance is 6\.2%\)\. In our custom LUCD dataset, one\-shot learning a linear classifier from a single task instance per rule yields a perfect test accuracy, whereas Full\-TTT performs at chance\. This is a strong sign that Embed\-TTT is capable of implicit rule induction, since test tasks were never seen during training \(in contrast with the previous evaluations\)\. We observe the same trend for our regression evaluation on the Moves dataset\. As an additional variant, we can align the probing setup with each method’s own test\-time dynamics: we first run each TTT protocol on duplicates of the training tasks, then learn the linear mapping from these TTT\-produced train embeddings \(instead of from frozen post\-training embeddings\), and finally apply it to test embeddings\. Under this setting, on LUCD, Full\-TTT rises to 39\.2% while Embed\-TTT remains at 100%; on Moves, Full\-TTT improves to 1\.27 of RMSE and Embed\-TTT reaches 0\.44 of RMSE\.

[Figure2](https://arxiv.org/html/2609.21181#S4.F2)compares UMAP projections of Full\-TTT and original train embeddings with projections of Embed\-TTT and original train embeddings, showing a clear distribution shift in the former case while Embed\-TTT better recaptures the structure of the manifold of the original embeddings\. The ground\-truth embedding is generally among the closest neighbors \(computed in the original embedding space\) of the “test” Embed\-TTT\-produced embedding under consideration \(most points are dark green\), while Full\-TTT embeddings are generally much further away from the original embeddings\. To assess whether the local geometry of ARC\-1 task embeddings is preserved \(which could happen regardless of retrieval performance\), we comparedkk\-nearest\-neighbor structure between the original train embedding space and embeddings from Embed\-TTT and Full\-TTT\. For each task, we computed top\-kkneighbors in each space \(excluding self\), then measured neighborhood agreement with the original space via set overlap proportion\. We found a large and consistent gap in favor of Embed\-TTT: mean overlap atk=10k=10is 33% for Embed\-TTT vs 4% for Full\-TTT \(with similar observations acrossk∈\{1​…​50\}k\\in\\\{1\.\.\.50\\\}\)\. These results indicate that Embed\-TTT preserves local relational structure from the pre\-trained task\-embedding manifold, whereas Full\-TTT distorts it\.

Table 1:Quantitative implicit rule induction\.We evaluate Embed\-TTT and Full\-TTT on ARC\-like datasets in their ability to produce embeddings that potentially elicit rule induction\. It appears that Embed\-TTT excels, while Full\-TTT generally loses any semantically meaningful information\.![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/full_ttt_vs_2step_ttt/umap_jointfit_with_full_ttt_rankcolor_fixed_0_400.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/full_ttt_vs_2step_ttt/umap_jointfit_with_phase1_rankcolor_fixed_0_400.png)

Figure 2:UMAP projections for ARC\-1\.Full\-TTT embeddings \(left\) are projected far from the original embedding distribution, while Embed\-TTT embeddings \(right\) are projected adequately onto it\. The original embedding usually ranks among the closest train embeddings to the fine\-tuned one with Embed\-TTT \(most points are dark green, indicating a good rank\)\.
### 4\.2Qualitative “out\-of\-distribution” evaluation of implicit rule induction

Dataset\.In addition to its 400 training tasks, ARC\-1 provides 400 "public evaluation" tasks \(which we will refer to here as "test tasks"\), that can differ from the training tasks rule\-wise, making them potentially “out\-of\-distribution” \(OOD\)\. We can thus qualitatively examine the similarity between test and training tasks to measure how well an embedding\-producing method works in this OOD case\. This analysis is not feasible with ConceptARC due to the absence of a train\-test split, and not relevant for our custom tasks since we generated only simple in\-distribution tasks\.

Results\.For that purpose, we search for the closest train task in the embedding space for each test task\. To mimic the optimization dynamic undergone by the test task in this process, the search occurs among the train embeddings obtained after Embed\-TTT was performed, so that embeddings obtained following the same process are compared\.[Figure3](https://arxiv.org/html/2609.21181#S4.F3)shows the closest test\-train task associations from Full\-TTT \(left\) and Embed\-TTT \(right\)\. We observe that Full\-TTT does not find relevant task associations, likely because of the distribution shift between test and train embeddings is too large, as already seen in[Figure2](https://arxiv.org/html/2609.21181#S4.F2)\(more are shown in[Appendix C](https://arxiv.org/html/2609.21181#A3)\)\. On the other hand, Embed\-TTT yields test\-train associations that are relevant, which is non\-trivial despite the good in\-distribution results\. In particular, the first association is an exact task duplicate shared between the test and the train sets of ARC\-1\. The other tasks shown clearly display the same rule semantics, showing a regular inpainting task and a copy task according to the dominant color of the input grid\. In our observations, close associations are usually relevant, while farther ones are less so, as expected given that some ARC\-1 test tasks differ radically from the training tasks\. We did not quantify this observation further, as it is relatively subjective\. Still, qualitative examination shows that Embed\-TTT enables meaningful comparison between test and train tasks, which is not the case with the default Full\-TTT approach\.

Test performance predictability\.We found that test tasks very close to train tasks \(top 10% closest\) were often solved \(resp\. failed\) when their nearest train neighbors were solved \(resp\. failed\), while very distant test tasks were often failed\. However, this pattern did not generalize to most tasks: distance to the nearest train task alone was usually not a reliable predictor of success\. We also tested standard alternatives as success predictors \(KNN, medoids, classifiers, etc\.\) and found no strong predictive signal\. A likely explanation is that many test tasks are out\-of\-distribution, though this cannot be confirmed without explicit test\-rule annotations\.

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity/full_000_4f537728__bdad9b1f_v5.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity/ph1_000_070dd51e__40853293_v4.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity/full_001_1e81d6f9__bdad9b1f_v2.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity/ph1_001_1d0a4b61__0dfd9992_v2.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity/full_002_92e50de0__3345333e_v2.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity/ph1_003_27f8ce4f__c3e719e8_v2.png)

Figure 3:Qualitative evaluation through test\-train task similarity\.For Full\-TTT \(left part\) and Embed\-TTT \(right part\), we report the closest associations between test \(left input\-output pairs per part\) and train \(right input\-output pairs\) tasks\. Full\-TTT does not associate tasks that seem to share common rules, while Embed\-TTT associates tasks that are clearly similar rule\-wise, the top one being a task duplicate with different IDs \(070dd51e and 40853293\) in test and train sets\.
### 4\.3Cross\-dataset capabilities

Results\.By acting as an implicit rule inductor, Embed\-TTT can be used to test whether a model trained on an ARC\-like dataset can induce and execute rules underlying tasks of another ARC\-like dataset\. In that regard,[Table2](https://arxiv.org/html/2609.21181#S4.T2)compares the performances of the original VARC ViT model with Full\-TTT, Embed\-TTT, and Embed\-Full\-TTT on ARC\-1, ConceptARC and Mini\-ARC\([Kim et al\., 2022](https://arxiv.org/html/2609.21181#bib.bib8)\)tasks\. We report the proportion of test tasks solved at pass 1 and pass 2 in the test datasets, as per the default ARC\-1 evaluation protocol\.

We observe that Embed\-Full\-TTT—finetuning the task embedding first, freezing it, then finetuning the model backbone—provides a moderate performance improvement on ARC\-1 test set over the default Full\-TTT and a larger improvement on ConceptARC and Mini\-ARC\. These improvements might be due to a better conditioning on the backbone finetuning based on an improved embedding\. Interestingly, Embed\-TTT, by finetuning only the task embeddings at test time, which accounts for<0\.01%<0\.01\\%of the model weights, suffices to score more than 13% on ARC\-1 at pass 1 \(resp\. 17% pass 2\), 8% \(resp\. 14%\) on ConceptARC, and 22% \(resp\. 34%\) on Mini\-ARC\. This indicates that the VARC model has learned general rule induction and execution capabilities, but also that some ARC\-1 test tasks, ConceptARC and Mini\-ARC tasks might be similar to the ARC\-1 training tasks\. These percentages also represent a significant fraction of what can be achieved by the model when the backbone finetuning is allowed\. From an efficiency perspective measured as weights to finetune, Embed\-TTT results are exceptionally high and indicate, to some extent, the generalizability of our approach beyond just ARC\-1 tasks\. As a point of comparison, we tested two recent open\-weight language models on the same public ARC\-1 test set with the same pass@1 evaluation as used in our study: Llama\-4\-Maverick scores only 4\.2% and Kimi K2\.5 scores 27\.8%\. These results show that our approach outperforms these models on the same controlled settings\. Proprietary models used in the official ARC leaderboard are evaluated on a private dataset and thus might have been trained on the test set considered here, hence we refrain from running expensive and potentially flawed comparisons, while acknowledging that such models are probably substantially more capable\.

Let us note that[Table2](https://arxiv.org/html/2609.21181#S4.T2)also shows that, for ConceptARC, independent performance across test input\-output pairs \(480\) is significantly higher than when aggregated per task \(3 test pairs per task\)\. This implies that the model does not solve ConceptARC tasks on a consistent basis, with successful test samples scattered across the tasks rather than consistently succeeding or failing on specific tasks\. For ARC, a similar analysis gives a boost of only0\.3%0\.3\\%across all metrics, mainly because most test tasks contain only one test sample, making the evaluation almost equivalent in both cases\.

Interestingly, even though VARC is a highly capable model trained on many ARC tasks and extra RE\-ARC\([Hodel, 2024](https://arxiv.org/html/2609.21181#bib.bib6)\)samples, it does not necessarily learn the simple transformations involved in LUCD or Moves\. Indeed, with Embed\-TTT on LUCD, pass@1 is 56% and pass@2 is 67\.3%, on Moves, pass@1 is 11\.6% and pass@2 is 19\.8% \(and of course any full finetuning reaches 100% but the expectation was to reach that with just task embedding finetuning\)\. This indicates that models trained on some ARC\-like tasks primarily “interpolate” between the contained functionalities, while even simple OOD tasks might not be directly solved by just finetuning the task embedding\. This point is further reinforced in[section4\.4](https://arxiv.org/html/2609.21181#S4.SS4)\. This is a curious property, and points to how brittle models might be even when they appear strong on benchmarks claiming to require OOD generalization\.

Finally, additional experiments with Embed\-TTT on ARC\-1 test set showed that performance could be pushed slightly further\. By increasing the “breadth” of search by finetuning up to 8 different task embeddings at test time \(instead of just one\) and then using the one with the lowest training loss, pass@1 \(resp\. pass@2\) reaches 14\.8% \(resp\. 18\.5%\)\. Further increasing the “depth” of search by finetuning two task embeddings applied sequentially allows to reach up to 17\.3% \(resp\. 20\.3%\)\. Such results show that the trained model still contains further capabilities, yet the gains of continuing to further increase breadth and depth are likely marginal and would be computationally expensive\. Hence, we did not investigate this direction deeper\.

Table 2:Cross\-dataset results\.Our Embed\-Full\-TTT allows to reach better final per\-task performances than Full\-TTT on ARC\-1, ConceptARC and Mini\-ARC\. In addition, Embed\-TTT, solves a significant amount of tasks by just finetuning the embeddings, indicating that rules underlying test tasks might not always differ much from train tasks\. Pair\-level \(instead of task\-level\) evaluation on ConceptARC\(\*\) shows a large gap, indicating potential difficulties in solving all input\-output pairs of a given task consistently\. Finally, ConceptARC concepts inferred on ARC\-1 tasks by task embeddings similarity decently match manual annotations despite the fact that the model was never trained for concept retrieval\.Concept retrieval in ARC\.In addition, given that they were obtained by using the same backbone, ARC\-1 and ConceptARC embeddings lie in comparable regions of the embedding space \(akin to[Figure2](https://arxiv.org/html/2609.21181#S4.F2)\(right\)\)\. Hence, we can compare them, by assigning each ARC\-1 task its closest ConceptARC task, using Embed\-TTT\-produced embeddings for ConceptARC tasks, ARC\-1 train and ARC\-1 test tasks\. In particular, we want to know if the core concepts close to ARC\-1 tasks are indeed related to the tasks\. For that purpose, we manually annotated a balanced subset of 80 ARC\-1 train tasks and 80 ARC\-1 test tasks \(5 for each of the 16 concepts uniformly spread across cosine distances\), by indicating which concepts \(at most 3\) were best related to the tasks\. We then check if the closest concept found matches one of the annotations\. The results are reported in[Table2](https://arxiv.org/html/2609.21181#S4.T2)and show decent agreement:36\.3%36\.3\\%on test and35\.0%35\.0\\%on train ARC\-1 tasks\. Importantly, the agreement is stronger for closer embedding associations: in both train and test splits, the closest\-distance quartile yields the highest agreement \(60%60\\%on test,50%50\\%on train\)\. Overall, nearest ConceptARC neighbors provide a useful interpretability signal, especially at small distances, but do not suffice as concept classifiers for ARC\-1 tasks\. However, we noticed that many ARC\-1 tasks are not perfectly aligned with the predefined concepts, and that additional concepts would improve coverage\. Still, the results are far above pure chance \(about6\.3%6\.3\\%\)\. Hence, Embed\-TTT enables, for the first time in this setting, a practical examination of meaningful connections between different ARC\-like datasets\.

### 4\.4Compositional Structure of the Embeddings

Datasets and scope\.Using our Moves dataset, we further examine how the model structures the embedding space\. For that purpose, we consider the cosine similarity of composed transforms \(e\.g\. R3\-U2\) to their underlying atomic transforms \(R3\-U0, R0\-U2\)\. Then, we investigate which tasks are solvable with Embed\-TTT from a subset of the initial training dataset\.

Compositions vs\. atoms\.Across the 100 composed transforms, we find that the two atoms are both the most or second\-most similar purely horizontal and vertical atoms for 94% of the cases\. The cosine similarities with “unit” transforms \(in the example case R1\-U0 and R0\-U1\) are much weaker, indicating a lack of scaling capability, in favor of a plain recombination of individual “already\-scaled” atomic projections\. The parametric nature of the dataset enables an intuitive visualization:[Figure4](https://arxiv.org/html/2609.21181#S4.F4)\(left and center\-left\) clearly shows that the UMAP projection of the Embed\-TTT embeddings follows an almost regular grid\-like pattern that mimics the original geometry of the dataset, where each task of the dataset can be associated to its move coordinates in the plane\. Similar visualizations from PCA are provided in[Appendix D](https://arxiv.org/html/2609.21181#A4)\. Further, a linear mapping learned from Embed\-TTT train embeddings efficiently projects test embeddings onto the grid, contrary to a mapping learned from Full\-TTT train embeddings \(right\-center and right panels\)\.

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/move_tasks/1_phase1_grid_only_sameXYcolors.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/move_tasks/2_phase1_umap_only_sameXYcolors.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/pca-moves/v9_linear_map_comparison_no_lines.png)

Figure 4:Embeddings structure\.In our Moves dataset, the task geometry \(left\) is reflected by the UMAP of train embeddings \(center\-left\), which forms an almost regular grid\. We then learn a linear map from train embeddings finetuned with Embed\-TTT \(resp\. Full\-TTT\) to their ground\-truth move coordinates \(gray squares\) \(center\-right, resp\. right\), and apply it to test embeddings \(points colored by projection error\)\. Embed\-TTT preserves a regular grid with low error, unlike Full\-TTT\.Embed\-TTT allows interpolation between tasks\.For the Custom Moves dataset, we further investigate which moves might be sufficient as training set so that the model generalizes to the remaining unseen moves\. In that regard,[Figure5](https://arxiv.org/html/2609.21181#S4.F5)\(left\) shows various configurations, where training moves are represented in gray squares while test moves are circles colored according to the model performance on these moves with Embed\-TTT\. The top left panel indicates that interpolation inside known moves is achievable at test time, while extrapolation outside known moves is not possible in that configuration\. The top right panel shows, however, that, even if the model was trained on all move types in each direction, it is not able to compose them without having seen any composition beforehand\. The bottom left panel shows 30 randomly selected train moves and indicates that test moves located on lines where only one training move was present are less easily achievable\. Finally, the bottom right panel shows that, when 3 training moves are present on each row and each column \(33 train moves in total\), almost all the other moves can be solved at test time by Embed\-TTT, at the exception in this case of the task involving no move, likely because the model was trained to always do at least one directional transform\. With comparable number of train moves \(30 vs 33\), this hints that train moves configuration can play a major role into generalization capabilities\.

These observations suggest that the minimal number of training moves present in the same rows and columns as the test moves is a strong predictor of success on test moves\. Concretely, for each test move \(across all our experiments, including runs not shown\), we compute the minimum value between the number of training tasks on its row and on its column, then group test performances by this value \(from 0 to 10, shown in violin plots in[Figure5](https://arxiv.org/html/2609.21181#S4.F5), right\)\. A clear transition appears at22: the model tends to “grok” once at least22aligned training moves are available\. A plausible explanation is that reusing embedding dimensions for a given direction requires repeated evidence; with only one instance, there is little pressure to factorize shared structure\. The violin plots are further split into two parts vertically\. The left\-hand side corresponds to test moves that are “inside” the \(at least\) two training moves located on the same row or column, the right\-hand side for “outside” test moves\. It appears that test moves located directly inside train moves are more easily solved, while those outside are occasionally failed\. Let us note that, even when they are solved, they do not correspond to extrapolation capabilities, since they still lie within the convex hull of the training moves and simply need to reuse known dimensions of the embeddings to encode the correct transform\. Further, we find that, when adding a third transformation that introduces simple color change, even when no compositions are shown, Embed\-TTT enables the correct projection to a point that lies closest to both valid moves and color changes, whereas Full\-TTT fails completely \(see[Appendix E](https://arxiv.org/html/2609.21181#A5)\)\.

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/move_tasks_2/v7_2x2_stitched.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/move_tasks_2/v7_rowcol_split_violin_min_row_col_minGE2_insideOutside_min7plus_square.png)

Figure 5:Task inter/extrapolation\.\(Left\) Move tasks are placed in the plane by their displacement\. Gray squares denote train tasks; circles denote test tasks, colored by success under Embed\-TTT finetuning, across several train/test splits\. Test tasks are mostly solved when at least22train moves lie on both their row and column, suggesting interpolation is feasible while extrapolation is limited\. \(Right\) Violin plots aggregate test successes by the minimum number of train tasks on the same row/column\. A clear transition appears at22, akin to a “grokking” effect\. Split violins show “inside” test tasks \(left half\) are slightly better solved than “outside” tasks \(right half\)\.We note a fundamental limitation in that Embed\-TTT does not seem capable to add moves together\. For instance, in the top left panel of[Figure5](https://arxiv.org/html/2609.21181#S4.F5), the model learned to move by 1 unit to the right \(through the train moves located at \(\+1,\+3\) and \(\+1,\-3\)\) and obviously by 3 units to the right\. However, we found no way \(heuristically or by constrained optimization\) to create an embedding that activates both \+3 and \+1 moves to the right to achieve \+4\. The issue likely lies in the execution of the embedding itself, because the backbone was never trained on such combinations\. We also found that forcing the model to apply two moves iteratively that could be optimized jointly was unsuccessful\. It is likely that the lack of prior training on such iterative composition causes this failure\.

## 5Conclusion

We introduced Embed\-TTT, a test\-time training protocol for Vision ARC \(VARC\) that first optimizes task embeddings to solve ARC\-like tasks, before an optional backbone finetuning\. Across ARC\-1, ConceptARC, and controlled datasets with known rules, Embed\-TTT yields task embeddings that are more semantically meaningful and substantially more rule\-aligned than those from the original full finetuning protocol\. We show that these embeddings enable test\-train task associations, cross\-dataset task comparisons, and task geometry recovery\. As limitations, Embed\-TTT requires iterative test\-time optimization through the full model, and the two\-step process is slower to reach only slightly better performances\. However, the value of Embed\-TTT embeddings resides mainly in their rule induction capabilities, supported by our results and to be distinguished from rule execution by the backbone\.

Our findings also convey a broader message for evaluating ARC\-like benchmarks: final accuracy alone does not suffice to characterize reasoning\. More focus on assessing rule induction is needed, with a clear separation between in\-distribution and out\-of\-distribution at the rule level, as rigorously done in common machine learning practice\. Eventually this will lead to more reliable claims about model “reasoning” capabilities\.

## Acknowledgments and Disclosure of Funding

A\. Deliège is a Postdoctoral Researcher of the Fonds de la Recherche Scientifique – FNRS\. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International, Inc\., for the U\.S\. Department of Energy’s National Nuclear Security Administration under contract DE\-NA\-0003525\. C\. Beger was supported in part through the BANYAN Institute, funded by Sandia National Laboratories’ Laboratory Directed Research and Development program\. The authors thank the Santa Fe Institute for hosting a visit during which part of this work was discussed\.

## References

- Akyüreket al\.\(2025\)E\. Akyürek, M\. Damani, A\. Zweiger, L\. Qiu, H\. Guo, J\. Pari, Y\. Kim, and J\. AndreasThe Surprising Effectiveness of Test\-Time Training for Few\-Shot Learning\.InInternational Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.267,Vancouver, Canada,pp\. 942–963\.External Links:[Link](https://proceedings.mlr.press/v267/akyurek25a.html)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Begeret al\.\(2025\)C\. Beger, R\. Yi, S\. Fu, K\. Denton, A\. Moskvichev, S\. W\. Tsai, S\. Rajamanickam, and M\. MitchellDo AI Models Perform Human\-like Abstract Reasoning Across Modalities?\.arXivabs/2510\.02125\.External Links:2510\.02125,[Document](https://dx.doi.org/10.48550/arXiv.2510.02125),[Link](https://doi.org/10.48550/arXiv.2510.02125)Cited by:[§1](https://arxiv.org/html/2609.21181#S1.p1.1),[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Cholletet al\.\(2024\)F\. Chollet, M\. Knoop, G\. Kamradt, and B\. LandersARC Prize 2024: Technical Report\.arXivabs/2412\.04604\.External Links:2412\.04604,[Document](https://dx.doi.org/10.48550/arXiv.2412.04604),[Link](https://doi.org/10.48550/arXiv.2412.04604)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Chollet \(2019\)F\. CholletOn the Measure of Intelligence\.arXivabs/1911\.01547\.External Links:1911\.01547,[Document](https://dx.doi.org/10.48550/arXiv.1911.01547),[Link](https://doi.org/10.48550/arXiv.1911.01547)Cited by:[§1](https://arxiv.org/html/2609.21181#S1.p1.1),[§1](https://arxiv.org/html/2609.21181#S1.p2.1)\.
- Hillet al\.\(2019\)F\. Hill, A\. Santoro, D\. G\. T\. Barrett, A\. S\. Morcos, and T\. LillicrapLearning to Make Analogies by Contrasting Abstract Relational Structure\.InInternational Conference on Learning Representations \(ICLR\),New Orleans, Louisiana, USA,pp\. 1–18\.External Links:[Link](https://openreview.net/forum?id=SylLYsCcFm)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Hodel \(2024\)M\. HodelAddressing the Abstraction and Reasoning Corpus via Procedural Example Generation\.arXivabs/2404\.07353\.External Links:2404\.07353,[Document](https://dx.doi.org/10.48550/arXiv.2404.07353),[Link](https://doi.org/10.48550/arXiv.2404.07353)Cited by:[§4\.3](https://arxiv.org/html/2609.21181#S4.SS3.p4.1)\.
- Huet al\.\(2026\)K\. Hu, A\. Cy, L\. Qiu, X\. D\. Ding, R\. Wang, Y\. E\. Zhu, J\. Andreas, and K\. HeARC Is a Vision Problem\!\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Denver, Colorado, USA,pp\. 2537–2546\.Cited by:[§1](https://arxiv.org/html/2609.21181#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2609.21181#S3.SS1.p1.1),[1st item](https://arxiv.org/html/2609.21181#S4.I1.i1.p1.1)\.
- Kimet al\.\(2022\)S\. Kim, P\. Phunyaphibarn, D\. Ahn, and S\. KimPlaygrounds for Abstraction and Reasoning\.InNeurIPS 2022 Workshop on Neuro Causal and Symbolic AI \(nCSI\),Virtual conference,pp\. 1–6\.External Links:[Link](https://nips.cc/virtual/2022/57706)Cited by:[§4\.3](https://arxiv.org/html/2609.21181#S4.SS3.p1.1)\.
- Lake and Baroni \(2018\)B\. M\. Lake and M\. BaroniGeneralization without systematicity: On the compositional skills of sequence\-to\-sequence recurrent networks\.InInternational Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.80,pp\. 2873–2882\.External Links:[Link](https://proceedings.mlr.press/v80/lake18a.html)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Langenfeldet al\.\(2026\)C\. Langenfeld, C\. Beger, G\. Geng, W\. T\. Piriyakulkij, K\. Hu, Y\. Pu, and K\. EllisBongards at the Boundary of Perception and Reasoning: Programs or Language?\.arXivabs/2602\.03038\.External Links:2602\.03038,[Document](https://dx.doi.org/10.48550/arXiv.2602.03038),[Link](https://doi.org/10.48550/arXiv.2602.03038)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Lewis and Mitchell \(2025\)M\. Lewis and M\. MitchellEvaluating the Robustness of Analogical Reasoning in Large Language Models\.Transactions on Machine Learning Research2,pp\. 1–33\.External Links:[Link](https://openreview.net/forum?id=t5cy5v9wph)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Liet al\.\(2025\)W\. Li, K\. Hu, C\. Larsen, Y\. Wu, S\. Alford, C\. Woo, S\. M\. Dunn, H\. Tang, M\. Naim, D\. Nguyen, W\. Zheng, Z\. Tavares, Y\. Pu, and K\. EllisCombining Induction and Transduction for Abstract Reasoning\.InInternational Conference on Learning Representations \(ICLR\),Singapore,pp\. 1–44\.External Links:[Link](https://openreview.net/forum?id=UmdotAAVDe)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Maaschet al\.\(2025\)J\. R\. M\. A\. Maasch, J\. Kalantari, and K\. KhezeliCausalARC: Abstract Reasoning with Causal World Models\.InAdvances in Neural Information Processing Systems \(NeurIPS\) Workshop,Vol\.39,San Diego, California, USA,pp\. 1–28\.External Links:[Link](https://openreview.net/forum?id=jO5pG9uRez)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Mineaultet al\.\(2026\)P\. J\. Mineault, T\. L\. Griffiths, and S\. EscolaCognitive Dark Matter: Measuring What AI Misses\.arXivabs/2603\.03414\.External Links:2603\.03414,[Document](https://dx.doi.org/10.48550/arXiv.2603.03414),[Link](https://doi.org/10.48550/arXiv.2603.03414)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Moskvichevet al\.\(2023\)A\. K\. Moskvichev, V\. V\. Odouard, and M\. MitchellThe ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domain\.Transactions on Machine Learning Research8,pp\. 1–20\.External Links:[Link](https://openreview.net/forum?id=8ykyGbtt2q)Cited by:[§1](https://arxiv.org/html/2609.21181#S1.p2.1)\.
- Piriyakulkijet al\.\(2025\)W\. T\. Piriyakulkij, Y\. Liang, H\. Tang, A\. Weller, M\. Kryven, and K\. EllisPoE\-World: Compositional World Modeling with Products of Programmatic Experts\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.39,San Diego, California, USA,pp\. 1–30\.External Links:[Link](https://openreview.net/forum?id=obwRcksFZw)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Toddet al\.\(2024\)E\. Todd, M\. Li, A\. Sen Sharma, A\. Mueller, B\. Wallace, and D\. BauFunction Vectors in Large Language Models\.InInternational Conference on Learning Representations \(ICLR\),Vienna, Austria,pp\. 17282–17333\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/4ae163cb8788970e53b4fd9578141139-Abstract-Conference.html)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.
- Veldkampet al\.\(2023\)K\. Veldkamp, C\. Stevenson, H\. Rosenbusch, and L\. ThomsSolving ARC visual analogies with neural embeddings and vector arithmetic: A generalized method\.OSF\.External Links:[Document](https://dx.doi.org/10.17605/OSF.IO/AKP86),[Link](https://osf.io/akp86/)Cited by:[§2\.2](https://arxiv.org/html/2609.21181#S2.SS2.p1.1)\.

## Appendix AOur Custom LUCD dataset

One input\-output pair for each of our 15 tasks of our custom LUCD dataset are represented in[Figure6](https://arxiv.org/html/2609.21181#A1.F6)\.

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-NN-00__test_07.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-NL-00__test_09.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-NU-00__test_03.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-NC-00__train_01.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-ND-00__test_05.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-LL-00__test_07.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-UU-00__test_02.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-CC-00__test_05.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-DD-00__train_07.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-LU-00__test_08.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-CL-00__test_02.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-DL-00__test_05.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-CU-00__test_03.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-DU-00__test_06.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/LUCD/v4_B-CD-00__test_02.png)

Figure 6:Our LUCD tasks\. Examples of input\-output pairs per LUCD task of our dataset\. Tasks are encoded as ‘left’ \(L\), ‘up’ \(U\), ’cross’ \(C\), ’dilate’ \(D\), ’neutral’ \(N\) transforms, composed with each other\.
## Appendix BOur Custom Moves dataset

Some input\-output pairs of our Moves dataset are represented in[Figure7](https://arxiv.org/html/2609.21181#A2.F7)\.

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/Moves/v7_dot1B-R0_U0__test_00.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/Moves/v7_dot1B-R0_U1__test_08.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/Moves/v7_dot1B-R0_D5__train_03.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/Moves/v7_dot1B-L1_D3__train_02.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/Moves/v7_dot1B-L3_U4__train_04.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/Moves/v7_dot1B-R4_U4__train_09.png)

Figure 7:Our Moves tasks\.Examples of input\-output pairs per Moves task of our dataset\. Tasks are encoded as left \(L\) or right \(R\) moves and up \(U\) and down \(D\) moves\.
## Appendix CQualitative similar tasks retrieval

Additional results of the retrieval of train tasks ’similar’ to test tasks according to Full\-TTT and Embed\-TTT are shown in[Figure8](https://arxiv.org/html/2609.21181#A3.F8)\.

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/full_003_696d4842__bdad9b1f.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/ph1_002_e95e3d8e__0dfd9992.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/full_004_95a58926__bdad9b1f.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/ph1_004_0f63c0b9__1bfc4729.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/full_005_66e6c45b__bdad9b1f.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/ph1_005_5b6cbef5__007bbfb7.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/full_006_58743b76__bdad9b1f.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/ph1_006_ea9794b1__75b8110e.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/full_007_c7d4e6ad__c9f8e694.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/ph1_007_ca8f78db__0dfd9992.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/full_008_2b01abd0__bdad9b1f.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/ph1_008_ad7e01d0__c3e719e8.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/full_009_070dd51e__40853293.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/ph1_009_551d5bf1__d4f3cd78.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/full_010_9ddd00f0__3345333e.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/ph1_010_7039b2d7__1190e5a7.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/full_011_3ed85e70__bdad9b1f.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/ph1_011_73182012__2013d3e2.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/full_012_94be5b80__3345333e.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/task_similarity_supp/ph1_012_c663677b__29ec7d0e.png)

Figure 8:Similarity retrieval\.\(Left\) Full\-TTT retrieves test\-train pairs of tasks that do not seem semantically similar\. \(Right\) Embed\-TTT retrieves test\-train pairs of tasks that share some common rules\.
## Appendix DPCA results on Moves

Embed\-TTT embeddings capture well vertical and horizontal ground\-truth moves, as shown in[Figure9](https://arxiv.org/html/2609.21181#A4.F9)\. In comparison, Full\-TTT mostly captures the left\-right and up\-down distinctions, but not a fine regular ordering of the moves, as shown in[Figure10](https://arxiv.org/html/2609.21181#A4.F10)\. The PCA values also tend to indicate a much more condensed projection\.

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/pca-moves/pca_pc1_pc2_scatter.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/pca-moves/pca_pc_grid_heatmaps.png)

Figure 9:Embed\-TTT PCA on our Moves tasks\.\(Top\) PCA visualization of the Embed\-TTT embeddings of our Moves tasks, colored according to ground truth vertical \(left\) and horizontal \(right\) moves\. \(Bottom\) Heatmap showing, for each ground\-truth move \(represented as a square at \(x,y\) coordinates\), its two main PCA values\. These two equivalent plots show that the first component \(PCA1\) captures vertical moves, the second component \(PCA2\) captures horizontal moves\.![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/pca-moves/pca_pc1_pc2_scatter-fullTTT.png)

![Refer to caption](https://arxiv.org/html/2609.21181v1/figures/pca-moves/pca_pc_grid_heatmaps-fullTTT.png)

Figure 10:Full\-TTT PCA on our Moves tasks\.\(Top\) PCA visualization of the Full\-TTT embeddings of our Moves tasks, colored according to ground truth vertical \(left\) and horizontal \(right\) moves\. \(Bottom\) Heatmap showing, for each ground\-truth move \(represented as a square at \(x,y\) coordinates\), its two main PCA values\. Full\-TTT seems to distinguish left moves from right moves, and up moves from down moves, but the exact move ordering seems much less recaptured than with Embed\-TTT\.
## Appendix EExperimenting with moves and color switch

We designed a controlled ARC\-like benchmark where each task applies \(i\) a spatial move and \(ii\) a color switch to a single blue cell\. Spatial moves span displacements from−3\-3to\+3\+3on each axis \(7×7=497\\times 7=49move rules\), and color switches span 8 target colors, yielding49×8=39249\\times 8=392task types\. The training split contains only*partial*rules:*move\-only*tasks \(all moves with no switch\) and*switch\-only*tasks \(all switches with no move\), totaling5757tasks\. Test tasks are the remaining move and switch compositions\. This setup directly evaluates whether a model that learned “composition” in a training setting \(here, composition over moves\) can transfer the “composition” ability to an unseen composition at test time \(here, composition of moves and switches\)\.

Main observation\.At sample level, Embed\-TTT without model training solves0%0\\%of composed test tasks, while Full\-TTT reaches near\-perfect performance \(≈99\.95%\\approx 99\.95\\%\) due to full backbone adaptation on easy tasks\. However, nearest\-neighbor analyses in embedding space show the opposite trend for implicit rule induction: Embed\-TTT embeddings consistently retrieve the correct atomic constituents \(move and switch\) of the test compositions, whereas Full\-TTT largely fails to do so\.

Top\-kkdual\-components retrieval\.To quantify this effect, for each composed test task we check whether*both*correct train move \(without switch\) and train switch \(without move\) are present in top\-kknearest neighbors\. As shown in Table[3](https://arxiv.org/html/2609.21181#A5.T3), Embed\-TTT reaches100%100\\%byk=10k=10\(even byk=7k=7\), while Full\-TTT remains near zero for a wide range ofkkand only increases at very largekk\. This indicates that Embed\-TTT preserves strong semantic locality for compositional rule components, whereas Full\-TTT does not\. In addition, for each composed test task, we also measured: \(i\) whether the closest “switch” train neighbor is the correct switch, and \(ii\) whether the closest “move” train neighbor is the correct move\. Embed\-TTT obtains100%100\\%and99\.5%99\.5\\%, respectively; Full\-TTT obtains13\.3%13\.3\\%\(near random chance12\.5%12\.5\\%\) and40%40\\%\.

Table 3:Top\-kkretrieval of both correct constituents \(train move and train switch\) for composed test tasks\. Note that the entire dataset consists of k=57 tasks\.Interpretation\.These results support a dissociation between*rule induction*and*rule execution*: Embed\-TTT captures the intended compositional semantics in embedding space, but fails at execution because the frozen backbone was never trained to jointly realize move and switch transformations\. This suggests that the bottleneck is primarily executability, not task\-level rule identification\. On the other hand, Full\-TTT unsurprisingly allows to solve the tasks, but at the cost of losing rule induction capabilities\.