When Probing Accuracy Saturates, Fragility Resolves: A Complementary Metric for LLM Pre-Training Analysis

# When Probing Accuracy Saturates, Fragility Resolves: A Complementary Metric for LLM Pre-Training Analysis
Source: [https://arxiv.org/html/2606.11375](https://arxiv.org/html/2606.11375)
\(May 2026\)

###### Abstract

Standard linear probing declares a property “encoded” when a classifier on hidden states achieves high accuracy\. The protocol works well on a snapshot but breaks across pre\-training: probe accuracy saturates within the first few thousand steps, leaving most of training invisible to the instrument\.

We introduce*fragility*, a complementary per\-layer metric defined as the activation\-noise level at which probe accuracy collapses\. Fragility is sensitive to both the margin of separability and the redundancy of representation, both of which keep evolving long after accuracy plateaus\.

Applied to open\-checkpoint language models, fragility recovers structure that accuracy alone cannot see\. Moralized representations emerge along a*lexical→\\tocompositional gradient*: lexical moral detection first, compositional moral encoding later\. Because probe accuracy on its own tracks how lexically separable a dataset is, we establish the compositional encoding directly, by showing it transfers across construction types that share no contrast tokens\. A*layer\-depth robustness gradient*develops monotonically across training while accuracy stays flat\. And matched fine\-tuning corpora that produce identical probing accuracy leave distinct fragility fingerprints, showing that data curation reshapes probe robustness without changing probe accuracy\.

In every comparison we test, where probing accuracy returns a flat answer, fragility returns a structured one\.

## 1Introduction

A standard interpretability protocol works as follows: given a representation of interest from a large language model, train a linear classifier to predict the property from frozen hidden states, report classifier accuracy at each layer, and declare the property “linearly encoded” wherever accuracy is high\. The protocol is well\- established\(Alain and Bengio,[2017](https://arxiv.org/html/2606.11375#bib.bib1); Belinkov,[2022](https://arxiv.org/html/2606.11375#bib.bib3)\)and works well for asking*whether*a model represents a property at a given snapshot\.

It does not work well for asking*how a representation evolves during pre\-training*\. We demonstrate the failure mode concretely\. On the OLMo\-2 1B early\-training trajectory\(Groeneveld et al\.,[2024](https://arxiv.org/html/2606.11375#bib.bib7)\), 37 model checkpoints densely sampled at 1K\-step intervals across steps 0\-36K \(~76B tokens\), a binary linear probe trained on a 240\-pair moral / neutral minimal\-pair dataset reaches ~95% mean accuracy across all 16 transformer layers by step 4K\. For the remaining ~33K training steps \(~95% of the trajectory we have data for\), the standard probing instrument returns essentially the same number; whatever continued representational change the model undergoes through that period is invisible to it\.

This paper makes a methodological contribution that takes the saturation problem as a fixed feature of probing accuracy and adds a complementary metric to recover the missing resolution:*fragility*, defined as the activation\-noise level at which probe accuracy drops below a threshold\. Formally, for layerℓ\\ellwith trained probefℓf\_\{\\ell\}, standard probing reports test\-set accuracyA​\(fℓ\)A\(f\_\{\\ell\}\)\. We define thecritical noiseσℓ∗\\sigma^\{\*\}\_\{\\ell\}as the smallest noise scale at which accuracy under Gaussian perturbation drops below a fragility thresholdτ\\tau:

σℓ∗=min⁡\{σ∈𝒮:A​\(fℓ,hℓ\+ε\)<τ,ε∼𝒩​\(0,σ2​I\)\}\\sigma^\{\*\}\_\{\\ell\}=\\min\\bigl\\\{\\,\\sigma\\in\\mathcal\{S\}:A\\\!\\bigl\(f\_\{\\ell\},\\;h\_\{\\ell\}\+\\varepsilon\\bigr\)<\\tau,\\quad\\varepsilon\\sim\\mathcal\{N\}\(0,\\,\\sigma^\{2\}I\)\\,\\bigr\\\}
where𝒮=\{0\.1,0\.3,1\.0,3\.0,10\.0\}\\mathcal\{S\}=\\\{0\.1,\\,0\.3,\\,1\.0,\\,3\.0,\\,10\.0\\\}andτ=0\.6\\tau=0\.6\(if noσ\\sigmain𝒮\\mathcal\{S\}brings accuracy belowτ\\tau,σℓ∗=max⁡\(𝒮\)=10\.0\\sigma^\{\*\}\_\{\\ell\}=\\max\(\\mathcal\{S\}\)=10\.0\)\. A lowσℓ∗\\sigma^\{\*\}\_\{\\ell\}means the representation at layerℓ\\ellisfragile: probe accuracy collapses under small noise\. A highσℓ∗\\sigma^\{\*\}\_\{\\ell\}means the encoding isrobust: the distinction is encoded with wide margin and/or redundancy\. Fragility is a per\-layer measurement applied to the same trained probe used for the accuracy curve, and it is sensitive to both the*margin*of separability and the*redundancy*of representation, both of which keep evolving through training even after accuracy has plateaued \(it does not separately identify their contributions; see[Section˜5\.2](https://arxiv.org/html/2606.11375#S5.SS2)\)\. We use fragility to map structural representational change that probing accuracy alone cannot see, and to establish three findings on the OLMo\-2 1B and OLMo\-3 7B open\-checkpoint family that together earn the methodological claim its keep:

Finding 1: Moralized semantic distinctions emerge along a quantitative lexical→\\tocompositional gradient\.A standard moral probe \(single morally\-loaded lexeme swap\) onsets at step 1K\. A*compositional*moral probe \(pairs that hold the action verb constant and vary only individually\-mild tokens whose moral status flips in context \(“protect” / “humiliate”, “hungry” / “wealthy”, “innocent” / “guilty”\) onsets at step 5K under 4\-seed averaging \(per\-seed range 4K\-7K\), between sentiment \(2K\) and syntax \(6K\)\. The standard probe’s step\-1K onset measures how quickly moralized vocabulary becomes linearly separable, not how quickly moral valence is encoded compositionally; the gradient reading is the honest one\.

Finding 2: A layer\-depth robustness gradient develops monotonically over training, invisible to probing accuracy\.Mean accuracy plateaus by step 4K but mean critical noise continues to evolve through step 36K: late layers hold maximum robustness while early\-layer critical noise drops from 10\.0 to 1\.8 between steps 4K and 36K\. The pattern reproduces at the OLMo\-3 7B scale with steeper late\-layer dominance, and reproduces independently for the compositional probe across four random\-seed splits\.

Finding 3: Data curation reshapes probe robustness without changing probe accuracy\.LoRA fine\-tuning on three matched corpora \(narrative\- moral, declarative\-moral, general non\-moral control\) produces identical probing accuracy across conditions \(final peak 0\.740 / 0\.750 / 0\.750\) but distinct fragility profiles\. Declarative moral training \(“Stealing is wrong” repeated\) produces fragility dips at 10 of 16 layers \(mean critical noise 5\.63\) versus 6\-7 fragile layers for natural\-text conditions \(mean 6\.94 / 7\.38\)\. Accuracy says “no signal”; fragility says “declarative training creates broadly fragile representations\.”

All experiments run on a single MacBook Pro M4 Pro with MPS; ~6 hours total MPS time\. Code, datasets \(including the 200\-pair compositional moral minimal\-pair dataset that is itself a methodological contribution\), per\-checkpoint outputs, and 4\-seed fragility replications are released with the paper\.

The unifying claim is methodological, not moral\-domain\-specific:in every comparison we test, where probing accuracy returns a flat answer, fragility returns a structured one\.[Section˜2](https://arxiv.org/html/2606.11375#S2)places the work against related literatures;[Section˜3](https://arxiv.org/html/2606.11375#S3)details the four minimal\-pair datasets, linear probing, and the fragility test;[Section˜4](https://arxiv.org/html/2606.11375#S4)reports results;[Section˜5](https://arxiv.org/html/2606.11375#S5)discusses the phase\-transition\-vs\-gradual\-emergence taxonomy implied by Finding 1, the geometric reasons fragility succeeds where accuracy saturates, and limitations;[Section˜6](https://arxiv.org/html/2606.11375#S6)concludes\.

## 2Related work

Linear probing\.Alain and Bengio \([2017](https://arxiv.org/html/2606.11375#bib.bib1)\)established linear probing as a layer\-wise diagnostic for what intermediate representations contain;Belinkov \([2022](https://arxiv.org/html/2606.11375#bib.bib3)\)surveys the methodology’s promise and known limitations\. Subsequent work has formalized accuracy\-based probing’s structural shortcomings via control tasks\(Hewitt and Liang,[2019](https://arxiv.org/html/2606.11375#bib.bib9)\), information\-theoretic probing\(Pimentel et al\.,[2020](https://arxiv.org/html/2606.11375#bib.bib15)\), and minimum\-description\-length analyses\(Voita and Titov,[2020](https://arxiv.org/html/2606.11375#bib.bib19)\)\. Our methodological contribution extends the saturation concern: where the probing literature treats ceiling effects as a threat to validity, we treat them as a fixed feature of the instrument and add a complementary metric \(fragility\) that continues to resolve representational change after accuracy plateaus\.

Activation perturbation\.Borras et al\. \([2022](https://arxiv.org/html/2606.11375#bib.bib5)\)proposed “Walking Noise,” injecting additive Gaussian noise at individual layers and defining a per\-layer midpoint noise level, the closest prior concept to our per\-layer critical noise\. However, Walking Noise measures end\-to\-end*task accuracy*degradation, not probe accuracy on specific concepts, and evaluates a single trained model rather than tracking across training checkpoints\. APEX\(Ren et al\.,[2026](https://arxiv.org/html/2606.11375#bib.bib18)\)injects Gaussian noise into hidden activations and defines “escape noise” \(the noise scale at which output becomes independent of input\), analogous to our critical noise; but APEX measures model\-output distribution changes, not concept\-specific probe accuracy\. Our method combines per\-layer noise robustness with concept\-specific probing*across*training checkpoints, a combination that neither approach attempts\.

Probing across training\.Qian et al\. \([2024](https://arxiv.org/html/2606.11375#bib.bib17)\)applied linear probes to 360 pre\-training checkpoints \(LLM360 Amber 7B\) for five trustworthiness dimensions, observing a fitting\-and\-compression pattern\. Their work establishes the “probing across checkpoints” approach but tracks only probe*accuracy*, which is exactly what we show saturates early and stops returning information\. Our fragility metric recovers the dynamics that continue after the point where their approach loses resolution\.

Causal tracing\.Meng et al\. \([2022](https://arxiv.org/html/2606.11375#bib.bib11)\)introduced ROME and the causal\-tracing methodology, which distinguishes layers that*encode*information from layers that*causally use*it\. Our 7B causal\-probing analysis \(Appendix B\) finds a ~10\-layer divergence between probing peak and causal peak for moral information, replicating the storage\-vs\-use distinction in the moral domain as supporting evidence for the body’s methodological thesis\.

Phase transitions\.Power et al\. \([2022](https://arxiv.org/html/2606.11375#bib.bib16)\)documented “grokking,” sudden phase transitions from memorization to generalization, with subsequent mechanistic work attributing such transitions to discrete circuit formation\(Olsson et al\.,[2022](https://arxiv.org/html/2606.11375#bib.bib14); Nanda et al\.,[2023](https://arxiv.org/html/2606.11375#bib.bib12)\)\. Our[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)finding that semantic minimal\-pair tasks \(standard moral, sentiment\) emerge as sharp phase transitions while compositional and structural tasks emerge gradually maps the grokking phenomenon against a lexical\-vs\-compositional dichotomy*within a single training run*, to our knowledge a novel framing\.

Moral Foundations Theory\.The standard moral dataset organizes content across[Haidt](https://arxiv.org/html/2606.11375#bib.bib8)’s\([2012](https://arxiv.org/html/2606.11375#bib.bib8)\)and[Graham et al\.](https://arxiv.org/html/2606.11375#bib.bib6)’s\([2013](https://arxiv.org/html/2606.11375#bib.bib6)\)six MFT foundations \(care/harm, fairness/cheating, loyalty/betrayal, authority/subversion, sanctity/degradation, liberty/oppression\)\. MFT is used as a*construction*heuristic for balanced coverage, not as a cognitive claim about how language models represent morality\. The compositional dataset \([Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2)\) categorizes by construction pattern \(motive / target / consequence / role\) instead\.

OLMo\.Groeneveld et al\. \([2024](https://arxiv.org/html/2606.11375#bib.bib7)\)released the OLMo family with full intermediate checkpoint releases \(the infrastructure that makes dense\-sampling trajectory analysis possible\), extended in the OLMo\-2 release\(OLMo Team,[2025](https://arxiv.org/html/2606.11375#bib.bib13)\)to substantially longer training schedules\. Our 37\-checkpoint 1B early\-training and 20\-checkpoint 7B stage\-1 trajectories rely on these open releases; the methodology applies to any open\-checkpoint family with dense enough sampling\. Pythia\(Biderman et al\.,[2023](https://arxiv.org/html/2606.11375#bib.bib4)\)offers a complementary open\-checkpoint testbed\.

Single\-direction circuits\.Arditi et al\. \([2024](https://arxiv.org/html/2606.11375#bib.bib2)\)demonstrated refusal behavior in instruction\-tuned models is mediated by a single representational direction\. The fragility metric is the pre\-training\- time analog: where single\-direction\-refusal asks whether*post\- training*safety properties are concentrated in narrow circuits at the final checkpoint, we ask whether moralized representations pass through more or less brittle states*during*training\. Both literatures address how concentrated vs\. distributed safety\-relevant representations are, at different points in the model lifecycle\. Representation\-engineering more broadly\(Zou et al\.,[2023](https://arxiv.org/html/2606.11375#bib.bib20)\)treats interpretability as direct readout of learned representations\.

Scope\.We use moralized vocabulary as a demonstration domain\. The compositional probe \([Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2)\) is the explicit lexical\- accessibility ablation; deeper questions \(counterfactual moral reasoning, generalization to held\-out moral structures\) would require harder probes and are discussed as future work in[Section˜5\.3](https://arxiv.org/html/2606.11375#S5.SS3)\.

## 3Methodology

We apply linear probing classifiers on four matched minimal\-pair datasets to all 37 OLMo\-2 1B early\-training checkpoints, 20 OLMo\-3 7B stage\-1 checkpoints, and the OLMo\-2 1B final checkpoint\. Two probe families:LayerWiseMoralProbe\(per\-layer accuracy\) andMoralFragilityTest\(per\-layer noise robustness\)\. All experiments run on a single MacBook Pro M4 Pro / MPS; code, datasets, and per\-checkpoint outputs are released with the paper\.

### 3\.1Standard minimal\-pair datasets

Three single\-token\-swap datasets mirror established minimal\-pair probing practice\(Belinkov,[2022](https://arxiv.org/html/2606.11375#bib.bib3)\)by holding the syntactic skeleton constant and swapping a single token:moral / neutral\(240 pairs, 40 per Moral Foundations Theory category \(care/harm, fairness/cheating, loyalty/betrayal, authority/subversion, sanctity/degradation, liberty/oppression; Haidt, 2012; Graham et al\., 2013; e\.g\. “She betrayed the woman” / “She greeted the woman”\)\. The moral pairs are drawn from a 1,200\-pair dataset constructed per published quality guidelines with LLM\-assisted filtering for naturalness and moral neutrality of neutral\-side sentences \(seeDATASET\_GUIDELINES\.md\); 240 pairs \(40 per foundation\) are subsampled with a deterministic seed\.Sentiment\(210 pairs across ten domains; positive / negative adjective swap, e\.g\. “…excellent food…” / “…terrible food…”\);syntax\(210 pairs targeting local grammaticality violations \(word\-order swaps, agreement, auxiliary misplacement\), e\.g\. “She handed the finished report…” / “She handed finished the report…”\)\. All three use a length\-ratio gate≤\\leq1\.5 and a deterministic seed\-42 train / test split\. They establish the baseline emergence ordering in[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)\.

### 3\.2Compositional moral probing dataset

The standard moral probe’s pairs differ in a single morally\-loaded lexeme, which a linear probe can separate from single\-token distributional statistics alone\. The standard probe’s step\-1K onset therefore measures*how quickly moralized vocabulary becomes linearly separable from neutral vocabulary*, not*how quickly moral valence is encoded compositionally*\. To bound this lexical\-accessibility framing empirically, we construct a compositional moral minimal\-pair dataset whose pairs require multi\-word integration to determine moral valence\. This dataset is a primary methodological contribution of the paper\.

Construction\.Each pair holds the action verb constant and varies 1–2 individually\-mild tokens whose moral status only resolves in the surrounding action context\. Four 50\-pair categories \(200 pairs total\), all hand\-curated:

- •*action\_motive:*same action verb, motive determines valence\. “She lied to her parents toprotecther younger brother today” / “…tohumiliate…”
- •*action\_target:*same action, target descriptor determines moral relevance\. “He gave the last loaf of bread to thehungrystranger at the door” / “…thewealthystranger…”
- •*action\_consequence:*same action, consequence framing determines valence\. “He kept the secret about the surprise to keep his sistersafetoday” / “…to keep his sisterhurttoday”
- •*role\_reversal:*same components, role / target / context determines valence\. “The judge accepted the gift to free theinnocentprisoner from prison” / “…theguiltyprisoner…”

Contrast tokens are individually mild:protect,humiliate,hungry,wealthy,safe,hidden,innocent,guilty\. None appear in the strong\-valence blocklist of 47 tokens \(murder,torture,stole,rape,assault, etc\.\) we use as a construction gate\. Each token is unmarked or only weakly valenced in isolation; the moral signal in each pair lives in the integration of the contrast token with the surrounding action context\.

Validation gates\.All 200 pairs pass five gates simultaneously \(validate\_compositional\_dataset\):

1. 1\.*Length difference:*per\-pair absolute difference≤\\leq2 alphabetic tokens\.
2. 2\.*Length band:*both halves in the 8\-20\-word band\.
3. 3\.*Content\-word overlap:*per\-pair≥\\geq0\.60 with stopwords removed \(\|shared\| / max\(\|moral\|, \|immoral\|\)\)\. This metric matchesdeepsteer\.datasets\.validation\.validate\_pairsso the 0\.60 threshold here is directly comparable to the standard moral probe gate\.
4. 4\.*Strong\-valence blocklist:*zero tokens from the 47\-word blocklist on either side\.
5. 5\.*No exact duplicates*of either side across the 200 pairs\.

Operationalizing “compositional”: transfer and lift, not a bag\-of\-words gate\.A minimal pair that flips its label by swapping a single contrast token makes that token a lexical cue*by construction*: a bag\-of\-words classifier separates the seen pairs no matter how mild the token is in isolation\. “Can unigrams separate the data” is therefore the wrong question \(they can: a pair\-disjoint, orientation\-invariant unigram TF\-IDF classifier reaches 0\.63 overall and 0\.57\-0\.64 per construction, where pair\-disjoint meansGroupKFoldkeyed on pair so neither half of a pair leaks across the split, and orientation\-invariant means scoringmax\(acc, 1 \- acc\)\)\. The right question is whether the hidden\-state representation encodes moral valence in a way that generalizes*beyond the specific contrast tokens*\. We operationalize compositional encoding through two tests, both measured against this unigram TF\-IDF baseline \(the*lexical floor*\):

1. 1\.*Leave\-construction\-out transfer\.*Train the probe on three of the four construction categories and test on the held\-out fourth, which shares almost no contrast tokens with the training categories\. A probe relying on token identity cannot transfer; a construction\-general representation can\.
2. 2\.*Lift over the lexical floor\.*Within each construction, compare hidden\-state decodability \(pair\-disjoint CV\) to that construction’s unigram lexical floor\. The gap is signal the bag\-of\-words baseline cannot reach\.

[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)reports both, at the final checkpoint and across training\. \(The construction iterated through ~5 rewriting passes to satisfy the 0\.60 content\-overlap gate alongside the multi\-word contrast requirement; the two constraints are in genuine tension; see Appendix D\.\)

Train / test split\.160 / 40, stratified by category \(40 train \+ 10 test per category\), seed = 42\. The dataset and validation gates are deterministic, API\-free, and included in the toolkit atdeepsteer/datasets/compositional\_moral\_pairs\.py\.

### 3\.3Linear probing

Identical probing methodology across all four datasets: when we report a 4K\-step gap between standard and compositional moral onsets \([Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)\), the only experimental variable is the dataset\.

For each transformer layerℓ\\ellwe mean\-pool hidden states across the sequence dimension and train a binary linear classifier \(nn\.Linear\(hidden\_dim, 1\)\) to distinguish the two sides of each minimal pair, with BCE\-with\-logits loss, Adam \(lr = 1e\-2\), 50 epochs, no weight decay or early stopping; probes run in fp32 on fp16 activation caches collected via PyTorch forward hooks on the HuggingFacemodel\.layers\[ℓ\\ell\]modules\. We report per\-layer test\-set accuracy and four summary statistics: onset layer \(first≥\\geq0\.6\), peak layer, encoding depth \(onset\_layer / n\_layers\), and encoding breadth \(fraction of layers≥\\geq0\.6\)\. The compositional probe additionally tracks a TF\-IDF content\-only floor per checkpoint that hidden\-state probing must beat by≥\\geq10 pp\. Implementation:LayerWiseMoralProbefor standard and compositional moral \(the latter a subclass that overrides only the dataset path\),GeneralLinearProbefor sentiment and syntax \(same training loop\)\.

### 3\.4Fragility testing

Train per\-layer linear probes on clean activations as in[Section˜3\.3](https://arxiv.org/html/2606.11375#S3.SS3); for each layerℓ\\ell, add Gaussian noise N\(0,σ\\sigma2\) to the cached test\-set activations and re\-evaluate the trained probe across a logarithmic sweepσ\\sigma∈\\in\{0\.1, 0\.3, 1\.0, 3\.0, 10\.0\}\. The smallestσ\\sigmaat which accuracy drops below the fragility threshold \(0\.6, i\.e\. chance \+ 0\.1 on a binary task\) is the layer’s*critical noise*; if noσ\\sigmain the sweep brings the probe below threshold, critical noise is reported as the maximum \(10\.0\)\. Per\-layer critical noise gives the fragility profile; its mean ismean\_critical\_noiseas a scalar summary\. The sameMoralFragilityTestruns against both the standard and compositional datasets; methodology generality is established by reuse, not reimplementation\.

### 3\.5Target models and checkpoints

Three OLMo\(Groeneveld et al\.,[2024](https://arxiv.org/html/2606.11375#bib.bib7); OLMo Team,[2025](https://arxiv.org/html/2606.11375#bib.bib13)\)base models\.OLMo\-2 1B early\-training\(allenai/OLMo\-2\-0425\-1B\-early\-training\), 37 checkpoints at 1K\-step intervals from step 0 to step 36K \(~76B tokens\), the primary data source for[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)onsets and[Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)fragility\.OLMo\-3 7B stage 1\(allenai/OLMo\-3\-7B\), 20 checkpoints through ~1\.4M steps \(~10T tokens\),[Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)7B corroboration and Appendix B causal tracing\.OLMo\-2 1B final\(allenai/OLMo\-2\-0425\-1B, ~2\.2T tokens\), used only for the compositional probe validation gate \([Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2)\)\. All loaded in fp16 on MPS; ~6 hours of MPS time across the full[Section˜4](https://arxiv.org/html/2606.11375#S4)experimental record\.

### 3\.6Validity controls

Three controls standard for linear\-probing studies \(leave\-lexeme\-out splits, paraphrase transfer, adversarial lexical swap\) are reported in Appendix C\. The compositional probe \([Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2)\) addresses the strongest version of “your probe is just reading vocabulary”: its compositional encoding is established by leave\-construction\-out transfer and lift over the unigram lexical floor \([Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2),[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)\), not by the lexical floor alone, and it is a strictly stronger ablation than those three controls combined for the relevant question\.

## 4Results

### 4\.1Emergence ordering: a lexical→\\tocompositional gradient

We train the four linear probes from[Section˜3\.1](https://arxiv.org/html/2606.11375#S3.SS1)\-[Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2)on hidden states from all 37 OLMo\-2 1B early\-training checkpoints \(steps 0\-36K at 1K intervals\)\. Onset is the first checkpoint where mean probe accuracy across all 16 layers reaches 0\.70\.

![Refer to caption](https://arxiv.org/html/2606.11375v1/x1.png)Figure 1:Lexical→\\tocompositional emergence gradient on OLMo\-2 1B early\-training\. Standard moral and sentiment probes \(single\-token swap\) plateau near 0\.97; compositional moral and syntax \(multi\-token integration\) plateau near 0\.77\. Compositional curve is 4\-seed mean±\\pmstd \(split seeds 42/43/44/45\)\. Onsets: standard moral 1K, sentiment 2K, compositional moral 5K \(per\-seed range 4K–7K\), syntax 6K\.Figure[1](https://arxiv.org/html/2606.11375#S4.F1)plots the four mean\-accuracy trajectories on a shared step axis\.

ProbeConstructionOnset stepOnset mean accPlateau mean acc \(step 36K\)Standard moralsingle morally\-loaded lexeme swap1,0000\.7600\.960Sentimentsingle valenced adjective swap2,0000\.7900\.976Compositional moralmulti\-token integrated swap5,0000\.709±\\pm0\.0250\.769±\\pm0\.030Syntaxstructural well\-formedness6,0000\.7170\.774*Table 1: Probe onset and plateau by construction\. Compositional moral values are 4\-seed mean±\\pmstd \(split seeds 42 / 43 / 44 / 45\)\. Per\-seed compositional onsets: 4K, 4K, 7K, 7K \(substantial seed variance, with the 4\-seed mean curve crossing 0\.70 at step 5K\)\. The single\-seed standard moral / sentiment / syntax curves are reported without std bands; their seed dependence is not characterized\.*

\(1\) The four probes resolve into a quantitative lexical→\\tocompositional gradient\.The standard moral probe \(single morally\-loaded lexeme swap, “betrayed” / “greeted”\) onsets at step 1K\. The compositional moral probe \(multi\-token integrated swap; contrast tokens “protect” / “humiliate”, “hungry” / “wealthy” are individually mild\) onsets at step 5K under 4\-seed averaging, a 4K\-step lag, with per\-seed onsets ranging 4K\-7K and overall trajectory always between sentiment \(2K\) and syntax \(6K\)\. The standard probe’s step\-1K onset measures how quickly moralized vocabulary becomes linearly separable, not how quickly moral valence is encoded compositionally\. Both findings are true; the strongest single\-token reading of the standard onset is ruled out, while the gradient reading \(lexically\-marked moralized vocabulary first, compositional moral integration second, syntactic competence last\) holds\. Onset accuracy alone understates the case: the 0\.709 onset sits only ~8 pp above the 0\.63 lexical floor, and onset ordering across all four datasets tracks lexical\-floor height \(unigram TF\-IDF floor: standard moral 0\.86, sentiment 0\.80, compositional 0\.63, syntax 0\.59; the more lexically separable a dataset, the earlier its onset and the higher its plateau\)\. Onset timing on its own therefore does not separate compositional encoding from lexical difficulty\. The evidence that the probe recovers compositional rather than lexical signal comes from transfer and lift \([Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2)\)\.

\(1b\) Compositional encoding is real, not lexical lookup \(final\-checkpoint evidence\)\.At the OLMo\-2 1B final checkpoint \(~2\.2T tokens\), a probe trained on three construction categories and tested on the held\-out fourth transfers at0\.848mean \(0\.80\-0\.91 across the four held\-out constructions\), essentially matching its in\-distribution pair\-disjoint accuracy \(0\.858@ layer 7\), while a bag\-of\-words classifier doing the same leave\-construction\-out transfer collapses to0\.598\. Within each construction the probe decodes \+0\.20 to \+0\.28 above the unigram lexical floor\. The decisive case is*role\_reversal*, where the same components appear on both sides and the lexical floor is lowest \(0\.57\): hidden states still decode moral valence at 0\.85 \(lift \+0\.28\) and held\-out transfer reaches 0\.81\. A probe reading contrast\-token identity could not do this; the model reads moral valence from context\. This is the operational content of “compositional” in this paper, and it is a positive result, not a thin margin over a bag\-of\-words floor\.

\(1c\) Compositional encoding emerges in early pre\-training, then holds\.

![Refer to caption](https://arxiv.org/html/2606.11375v1/x2.png)Figure 2:Compositional moral encoding emerges in early pre\-training and then holds \(OLMo\-2 1B early\-training, 37 checkpoints\)\. \(a\) When: leave\-construction\-out transfer accuracy rises from chance \(∼\\sim0\.55\) across steps 2K–9K, crosses the bag\-of\-words transfer floor \(∼\\sim0\.60\) by step 2K, and plateaus near 0\.82 \(lift∼\\sim\+0\.20 over the lexical floor\) through step 36K; the role\_reversal curve \(lexical cues scrambled\) tracks it\. \(b\) Where: per\-layer in\-distribution probe accuracy, showing the encoding concentrating in mid\-network layers 8–10 once it emerges\.Figure[2](https://arxiv.org/html/2606.11375#S4.F2)plots the transfer\-and\-lift analysis across all 37 early\-training checkpoints\. At initialization the encoding is absent: leave\-construction\-out transfer sits at chance \(0\.55\) and lift is ~0\. It emerges over steps 2K\-9K, crossing the bag\-of\-words transfer floor \(~0\.60\) by step 2K \(0\.667\), passing 0\.70 by step 3K, reaching 0\.78 by step 5K, then plateauing by step ~9K at transfer ~0\.82 and lift ~\+0\.20 and holding there through step 36K \(0\.83 / \+0\.22\)\. The role\_reversal construction, where lexical cues are scrambled by design, follows the same curve \(panel a\)\. The encoding also localizes in depth: once it appears, the most decodable layer settles into mid\-network \(layers 8\-10\) and stays there for the rest of the trajectory \(panel b\)\. Compositional moral encoding is therefore an early\-pre\-training acquisition, emerging after lexical moral detection \(standard\-probe onset at step 1K\) and consistent with the lexical→\\tocompositional ordering, that once acquired is stable across the remaining ~30K steps we observe\. Numbers source:outputs/phase\_c4\_compositional/b\_traj/\(per\-checkpoint\) andb\_traj\_summary\.json\.

\(2\) Step\-like vs\. gradual emergence dichotomy\.Standard moral and sentiment probes show sharp sigmoidal transitions \(chance→\\toplateau within one 1K\-step interval at onset, then flat\)\. Compositional moral and syntax rise more gradually \(~3\-5K steps across the 0\.70 threshold\)\. At 1K sampling we resolve transitions that are step\-like at this resolution; we do not claim true discontinuity\. This parallels grokking\-literature observations\(Power et al\.,[2022](https://arxiv.org/html/2606.11375#bib.bib16)\)that some capabilities emerge sharply and others gradually; the within\-run split here suggests the distinguishing factor is whether the capability is acquirable from local lexical statistics \(sharp\) or requires multi\-token integration \(gradual\)\.[Section˜5\.1](https://arxiv.org/html/2606.11375#S5.SS1)develops\.

\(3\) Plateau coincidence\.The four\-curve overlay \(Figure 1\) makes a structural caveat visually inescapable: probes whose signal lives in single\-token vocabulary statistics \(standard moral, sentiment\) plateau at 0\.96 and 0\.98, while probes whose signal requires multi\-token structural or compositional integration \(compositional moral, syntax\) plateau at 0\.77 and 0\.77\. The 20\-percentage\-point ceiling gap is consistent across the entire 0\-36K trajectory\. This may be a probe\-side property under our methodology rather than a model property: either the 1B model encodes both compositional moral valence and syntactic well\-formedness at≈\\approx0\.77 \(model ceiling\), or mean\-pooled linear probing on 1B hidden states bottoms out at≈\\approx0\.77 for multi\-token integration regardless of underlying representational quality \(probe ceiling\)\. The cleanest disambiguation is repeating[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)at 7B and 32B: if compositional moral rises with scale while syntax does not, the model is the bottleneck; otherwise the probe is\. We state both readings honestly in[Section˜5\.3](https://arxiv.org/html/2606.11375#S5.SS3)and refine rather than overturn the gradient finding\.

Generalization to OLMo\-3 7B\.We have not yet run the compositional probe on the OLMo\-3 7B trajectory; doing so is the cleanest disambiguation of the plateau\-coincidence ambiguity and is flagged as future work in[Section˜5\.3](https://arxiv.org/html/2606.11375#S5.SS3)\.

Numbers source:outputs/phase\_c2/c2\_emergence\_timing\.json\(standard moral \+ sentiment \+ syntax, 37 checkpoints\) andoutputs/phase\_c4\_compositional/c4\_emergence\_timing\.json\(compositional moral, 37 checkpoints; companion JSON with all four curves overlaid\)\. Validation source:outputs/phase\_c4\_compositional/c4\_validation\.json\(final\-checkpoint validation gate onallenai/OLMo\-2\-0425\-1B\)\.

### 4\.2Probing accuracy saturates; fragility doesn’t

![Refer to caption](https://arxiv.org/html/2606.11375v1/x3.png)Figure 3:Probing accuracy saturates; fragility resolves\. OLMo\-2 1B early\-training, 37 checkpoints\. Top: mean probing accuracy across all 16 layers — saturates near 0\.95 by step 4K and stays flat for the remaining 32K steps\. Bottom: mean critical noise — continues evolving long after accuracy plateaus, drifting from∼\\sim10 down toward∼\\sim6 between steps 4K and 36K\.Figure[3](https://arxiv.org/html/2606.11375#S4.F3)provides the central comparison for the methodological claim: a two\-panel comparison on a shared step axis\. Top panel: mean probing accuracy, a sharp sigmoid from chance \(~0\.59\) to a plateau \(~0\.95\) between steps 0 and 4K, then flat for the remaining ~33K steps\. Bottom panel: mean fragility, an initial rise alongside accuracy in the first few thousand steps, then continued movement throughout\. Top panel reaches a ceiling and stops; bottom panel keeps moving for the entire remaining 90 % of training\.

OLMo\-2 1B, 37 checkpoints, dense sampling\.

StepMean accMean critical noiseLate\-layer critMid\-layer critEarly\-layer crit00\.5900\.770\.10\.61\.61,0000\.7287\.8110\.08\.84\.44,0000\.94110\.010\.010\.010\.010,0000\.9437\.9010\.08\.87\.215,0000\.9547\.5010\.010\.05\.020,0000\.9507\.4010\.010\.02\.236,0000\.9546\.1210\.06\.51\.8*Table 2: Standard moral probe\. Accuracy plateaus by step 4K; fragility evolves through step 36K with a layer\-depth gradient that develops monotonically \(late \> mid \> early after step ~15K\)\.*

![Refer to caption](https://arxiv.org/html/2606.11375v1/x4.png)Figure 4:Layer\-depth structure over training \(OLMo\-2 1B early\-training\)\. \(a\) Probing accuracy: uniformly high across layers after step 4K\. \(b\) Critical noise: a layer\-depth gradient develops, with late layers holding maximum noise tolerance while early layers grow progressively more brittle\. Same data, same model; structure visible only under the fragility metric\.Figure[4](https://arxiv.org/html/2606.11375#S4.F4)shows the same trajectory as two stacked layer\-depth heatmaps: probing accuracy \(uniformly green after step 4K, no remaining structure to resolve\) above critical noise \(gradient emerging: late layers hold maximum noise tolerance throughout while early layers grow progressively more brittle\)\. Same data; different metric; different visible structure\.

The pattern reproduces at OLMo\-3 7B \(5 sparse checkpoints\): mean critical noise rises 2\.68→\\to5\.14 between steps 0 and 353K, then holds at ~5\.3 through step 1\.4M; layer\-depth gradient is steeper \(late ~10\.0 / mid ~6\.2 / early ~2\.0\) and the most\-robust layer drifts deeper across training \(layer 1→\\to15→\\to16→\\to10→\\to10\)\. The 1B trajectory is the headline because dense 1K\-step sampling resolves the saturation step \(~4K\) and gradient emergence rate\.

Compositional probe fragility evolution \(4\-seed replication; the methodological claim generalizes beyond the standard probe\)\.We ranMoralFragilityTest\([Section˜3\.4](https://arxiv.org/html/2606.11375#S3.SS4)\) on the compositional dataset across all 37 OLMo\-2 1B early\-training checkpoints with four split seeds \(42, 43, 44, 45\), the original seed\-42 trajectory plus a three\-seed replication ~50 min on the same MacBook Pro M4 Pro / MPS\.Table 3gives the 4\-seed mean±\\pmstd at the diagnostic checkpoints; the 4\-seed accuracy band onFigure[1](https://arxiv.org/html/2606.11375#S4.F1)carries the matching probing\- side trajectory\.

StepCompositional mean critical noise \(4\-seed mean±\\pmstd\)n00\.10±\\pm0\.0041,0000\.14±\\pm0\.0442,0000\.94±\\pm0\.1743,0003\.47±\\pm1\.0445,0005\.11±\\pm0\.95\(peak\)46,0004\.31±\\pm1\.5747,0004\.65±\\pm0\.84410,0004\.60±\\pm0\.48420,0003\.07±\\pm0\.91430,0002\.46±\\pm0\.28436,0002\.49±\\pm0\.124*Table 3: 4\-seed compositional fragility evolution\. The std collapses from 1\.57 \(step 6K\) to 0\.12 \(step 36K\); at the late plateau the four seeds converge tightly\.*

The compositional probe reproduces the qualitative pattern \(accuracy plateaus by step ~5K; mean critical noise continues evolving through step 36K\) and shows its own quantitatively distinct long\-term shape: fragility rises through step 5K alongside accuracy onset \(4\-seed mean 0\.10→\\to5\.11\), then declines through step 30K \(5\.11→\\to2\.46\) and holds\. To verify the post\-step\-7K decline is replicable rather than a single\-seed artifact, we apply a pre\-registered decision rule: the decline counts as real if 4\-seed mean critical noise drops by≥\\geq1\.0 between step 7K and step 30K*and*seed\-to\-seed std at both endpoints is smaller than the gap\. Realized values: gap = 4\.65−\-2\.46 = 2\.19 \(≥\\geq1\.0 ✓\), max endpoint std = 0\.84 \(< 2\.19 ✓\)\. Both pass with substantial margin; the post\-step\-7K decline is a stable property across the four split seeds\.

Two non\-exclusive readings of the diverging long\-term direction \(7B / 32B replication disambiguates both\): a*mechanism reading*\(as training continues on text that does not specifically reinforce compositional moral integration, the compositional representation drifts toward brittleness while standard\-probe representations are continually reinforced by moralized vocabulary density\) and a*probe\-ceiling reading*\(fragility at the 0\.77 operating point has less headroom than at 0\.96, partly artifacting the difference\)\. We state both in[Section˜5\.3](https://arxiv.org/html/2606.11375#S5.SS3)without commitment\.

Numbers sources:outputs/phase\_c1/RESULTS\.md\(1B standard probe\),outputs/phase\_b/\(7B corroboration\),outputs/phase\_c4\_compositional/3seed/\{aggregate\_per\_checkpoint,decision\}\.json\(4\-seed mean±\\pmstd and decision rule application\),outputs/phase\_c4\_compositional/3seed/4seed\_fragility\_evolution\.png\(headline 4\-seed plot\)\.

### 4\.3Data curation reshapes probe robustness, not probe accuracy

LoRA\(Hu et al\.,[2022](https://arxiv.org/html/2606.11375#bib.bib10)\)fine\-tuning on three matched corpora from the OLMo\-2 1B step\-1000 checkpoint \(mid\-transition, ~80 % peak probing accuracy\)\. Corpora: a 247K\-token narrative\-moral corpus \(Aesop / Grimm / Andersen\), a 500K\-token declarative\-moral corpus \(template\-expandedMORAL\_SEEDS: “Stealing is wrong”\), and a 420K\-token general non\-moral control \(Darwin\)\. Identical LoRA hyperparameters \(rank 16, alpha 32, q\_proj \+ v\_proj, lr 2e\-4, batch 2, seq 1024, 1000 steps\); standard moral probe \+ fragility every 100 LoRA steps\.

Probing accuracy is identical across conditions\.Final peak accuracy at LoRA step 1000: narrative 0\.740, declarative 0\.750, general control 0\.750, all within 1 pp across very different training data\. The accuracy metric returns no signal for which corpus produces what kind of representational change\.

Fragility profiles are condition\-specific \(the main result\)\.Final mean critical noise: narrative 7\.38, declarative 5\.63, general control 6\.94\. The per\-layer breakdown separates the conditions: narrative and general control show fragility dips at 6 and 7 of 16 layers respectively; the declarative condition shows dips at10 of 16 layers, creating a broadly more fragile representation than either natural\-text condition\.

![Refer to caption](https://arxiv.org/html/2606.11375v1/x5.png)Figure 5:Data curation reshapes probe robustness, not probe accuracy \(OLMo\-2 1B; LoRA fine\-tuning from step 1000 on three matched corpora\)\. \(a\) Final probing accuracy is near\-identical across narrative\-moral, declarative\-moral, and general\-text control conditions \(0\.740 / 0\.750 / 0\.750\)\. \(b\) Per\-layer critical noise is condition\-specific: declarative\-moral training produces diffuse fragility across 10 of 16 layers \(meanσ∗\\sigma^\{\*\}=5\.63 vs\. 7\.38/6\.94 for natural\-text conditions\)\.Figure[5](https://arxiv.org/html/2606.11375#S4.F5)plots all three per\-layer profiles plus the three identical accuracy bars: same accuracy, different fragility\.

Training loss is decoupled from representational change\.Declarative loss drops 5\.5→\\to1\.0 \(template memorization\), narrative 4\.3→\\to4\.0, control 5\.0→\\to4\.4\. The condition with the deepest loss reduction is the same condition with the most diffuse fragility; the two with the shallowest loss reductions retain more robust representations\. The model is learning declarative templates as surface text patterns without that learning translating into either accuracy gains or robust representational structure\.

Why diffuse fragility under declarative training?The declarative corpus consists of repeated syntactic templates \(“X is wrong”, “Y is immoral”\) that the model memorizes easily \(loss→\\to1\.0\)\. This memorization creates narrow pattern\-matching features across multiple layers, features whose probe accuracy is maintained by low\-margin decision boundaries that collapse under small noise\. Narrative and general\-control conditions produce fewer fragile layers because natural text has no repeated syntactic template; moral content in Aesop’s fables is embedded in diverse narrative structures, forcing the probe to rely on distributed features that tolerate noise\. The declarative fragility pattern is consistent with the[Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)finding that early layers grow progressively more brittle over training; declarative LoRA produces a diffuse version of a vulnerability pattern that pre\-training produces as a layer\-depth gradient\.

This is direct evidence for the methodological thesis in a controlled setting: same data, same probe; accuracy returns no signal, fragility separates the conditions\.

Why the fragility is diffuse rather than localized\.The diffuse pattern, fragility across 10 of 16 layers \(meanσ\\sigma\* = 5\.63 vs\. 7\.38 / 6\.94\) rather than a single dramatic dip, has a straightforward mechanistic explanation\. When the probing dataset controls for animacy and register confounds \([Section˜3\.1](https://arxiv.org/html/2606.11375#S3.SS1)\), the probe detects actual moral features at every layer rather than exploiting shortcuts like “is this about a person or a circuit?” that survive Gaussian perturbation easily\. When declarative templates are memorized, the model’s moral representations throughout the network become template\-dependent: narrow\-margin features that collapse under noise at multiple layers rather than just one\. Template memorization does not corrupt one layer; it degrades the network’s moral representations broadly\. This sensitivity to dataset quality confirms the importance of the validation methodology described in[Section˜3\.1](https://arxiv.org/html/2606.11375#S3.SS1): probing datasets that contain animacy or register shortcuts will systematically underestimate fragility at layers where the probe exploits those shortcuts rather than moral content\.

Numbers source:outputs/phase\_c\_tier2/c3/RESULTS\.mdandoutputs/phase\_c\_tier2/c3/\{narrative,declarative,general\}\_moral\.json\(per\-layer fragility for all three conditions\)\.

## 5Discussion

### 5\.1Semantic vs\. structural learning dynamics

The[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)four\-curve overlay \(Figure 1\) shows two distinct learning regimes within a single training run on a single model\. The standard moral and sentiment probes, both single\-token\-swap minimal\-pair tasks, emerge as sharp, step\-like sigmoidal transitions: each crosses from chance to its plateau within a single 1K\-step interval at onset, then flattens\. \(At 1K sampling we resolve transitions that are step\-like at this resolution; we do not claim true discontinuity\.\) The compositional moral and syntax probes, both tasks that require multi\-token integration to determine the label, rise more gradually, with no equally sharp inflection point\.

The cleanest hypothesis to organize this dichotomy:phase\-transition dynamics emerge when a feature can be acquired through local lexical or distributional statistics \(the model “discovers” the feature in a discrete jump as soon as it has enough samples to distinguish the relevant lexemes\), while gradual emergence indicates features that require integrating positional, attentional, or compositional relationships across multiple tokens, which the model cannot acquire in a single step from local lexical statistics alone\.Under this reading the standard moral probe \(single\-lexeme swap\), sentiment \(single\-adjective swap\), and similar lexically\-localized tasks all share the phase\-transition mechanism; the compositional probe \(multi\-token integrated swap\) and syntax \(positional well\-formedness\) share the gradual\-emergence mechanism\. The 0\.20 plateau gap \([Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)\) between the two regimes \(single\-token\-statistics tasks saturating near 0\.97, multi\-token\-integration tasks near 0\.77\) is consistent with this reading: features that can be cleanly read off single\-token distributional statistics in mean\-pooled hidden states should reach higher linear separability than features that require recovering multi\-token interactions from a pooling operation that discards positional information\.

The dichotomy connects to[Power et al\.](https://arxiv.org/html/2606.11375#bib.bib16)’s\([2022](https://arxiv.org/html/2606.11375#bib.bib16)\)grokking literature \(sudden phase transitions on algorithmic tasks\), which has largely focused on the*cause*of phase transitions; our results suggest the*taxonomy*of which capabilities should and should not exhibit them\. The formal information\-theoretic argument is its own paper\.

### 5\.2Why fragility succeeds where accuracy saturates

Probing accuracy is a thresholded, capped, top\-end metric: once linear separability is good enough, accuracy hits ceiling and stops returning information about underlying representational change\. Fragility is structurally different: it is sensitive to both the*margin*of separability \(outputs near the decision boundary flip under small noise\) and the*redundancy*of representation \(features encoded in many hidden\-space directions tolerate noise that collapses any one\)\. It does not separately identify their contributions, and a low critical noise can also reflect activation\-scale changes, representational anisotropy, or probe\-training instability rather than margin or redundancy alone\. Both margin and redundancy continue to evolve after accuracy saturates because both are functionals of representation*geometry*rather than end\-to\-end classification accuracy\. Concretely \([Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)\): the standard moral probe’s mean accuracy holds at ~0\.95 from step 4K through step 36K while early\-layer critical noise drops 10\.0→\\to1\.8\. The argument generalizes: fragility is not a moral\-domain\-specific contribution but a methodological contribution for any binary probing task that hits accuracy ceiling\.

### 5\.3Limitations

Lexical→\\tocompositional gradient bounds the standard probe\.The standard moral probe measures something closer to “moralized vocabulary becomes linearly separable from neutral vocabulary” than “moral reasoning emerges\.” The compositional probe \([Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)\) established this is a quantitative gradient: lexically\-marked moralized vocabulary at step 1K, compositional moral integration at step 5K, syntactic competence at step 6K, not a binary in\-or\-out distinction\. Both onsets are real findings\. Neither of them is “moral reasoning at step 1K”; both are bounded claims about what a linear probe can recover from mean\-pooled hidden states at each step\.

Compositional probe partial scope\.The compositional probe addresses*whether the moral signal lives in single\-token vs\. multi\-token features*; it does not address*whether the model represents moral concepts in any deeper functional sense*: counterfactual sensitivity to moral reframing, generalization to novel moral structures not in pre\-training data, behavioral consistency under adversarial probing\. The compositional probe is a strictly stronger lexical\-accessibility ablation than the standard probe; it is not a moral\-reasoning probe\. Stronger probes for deeper moral capacities are out of scope for this paper\.

Two related questions disambiguate at scale \(7B / 32B replication\)\.First, the[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)plateau coincidence \(compositional≈\\approxsyntax≈\\approx0\.77 vs\. standard moral / sentiment≈\\approx0\.97\) may reflect a 1B\-model ceiling on compositional / structural encoding or a probe\-side ceiling under mean\-pooled linear probing\. Second, the[Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)4\-seed compositional fragility decline through steps 7K\-30K \(4\.65→\\to2\.46\), opposite to the standard probe’s late\-training hold, admits both a*mechanism reading*\(compositional representations drift toward brittleness as training continues on text that does not specifically reinforce them\) and a*probe\-ceiling reading*\(fragility at the 0\.77 operating point has less headroom than at 0\.96, partly artifacting the difference\)\. Both readings predict different scaling behavior: under the mechanism reading the decline tracks training\-text distribution rather than scale, under the probe\-ceiling reading it attenuates as scale lifts the operating point\. Repeating[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)and[Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)at 7B and 32B disambiguates both\. Either outcome refines the gradient finding without overturning it\.

Single model family\.All findings are on OLMo\-2 1B and OLMo\-3 7B\. Generalization to other architectures and training recipes is open\.

Single language\.All probing datasets are English; pretraining data for both target models is dominantly English\. Cross\-lingual generalization of both the gradient finding and the fragility\-resolves\-what\-accuracy\-misses pattern is open\.

Raw\-σ\\sigmafragility and what would limit it\.We add Gaussian noise in raw activation units \([Section˜3\.4](https://arxiv.org/html/2606.11375#S3.SS4)\)\. Hidden\-state norms vary across layers and checkpoints, so cross\-layer and cross\-checkpoint critical\-noise comparisons may partly reflect activation\-scale drift rather than representational robustness alone\. We report rawσ\\sigmabecause every comparison we draw uses the same probe and activation cache within a single \(layer, checkpoint\) cell, which holds the scale fixed; aσ\\sigma\-normalized\-by\-per\-layer\-activation\-RMS variant is future work\. More generally, the fragility finding would be undercut if the layer\-depth gradient \([Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)\) or the declarative\-vs\-natural separation \([Section˜4\.3](https://arxiv.org/html/2606.11375#S4.SS3)\) vanished under RMS\-normalized noise, or if it tracked activation norm rather than probe margin; we have not yet run that control\.

Foundation\-specific scope\.The standard moral dataset’s six MFT foundations show staggered emergence; all six stabilize by step 3K \(Appendix A\)\. The emergence ordering is sensitive to dataset construction choices, which confirms the importance of dataset quality methodology in probing studies; findings that depend on specific pairs rather than the property of interest are artifacts, not discoveries\. The compositional dataset’s 200 pairs are categorized by construction pattern \(motive / target / consequence / role\) rather than by MFT foundation; a foundation\-stratified compositional probe \(parallel to the foundation\-specific standard probe in Appendix A\) would tell us whether different foundations acquire compositional encoding at different steps\. Out of scope for this paper but a natural extension\.

## 6Conclusion

Probing accuracy is the wrong instrument for studying training dynamics\. Whatever continued representational change a language model undergoes during pre\-training after a property becomes linearly decodable \(and[Section˜4](https://arxiv.org/html/2606.11375#S4)shows there is a great deal of such change\) does not register on a metric that has already saturated\. The methodological contribution of this paper is to take this saturation as a fixed feature of probing accuracy and add a complementary metric \(per\-layer critical noise, which we call*fragility*\) that recovers the missing resolution\. The contribution is not specific to moral representations or to pre\-training trajectories; we have established the fragility\-resolves\-what\-accuracy\-misses pattern on the standard moral probe \([Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)\) and on the compositional moral probe across four random\- seed splits \([Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2), replication\)\. We expect the methodology to extend to any binary probing\-based investigation of neural network representations where the operating point is high enough to saturate the standard accuracy curve\.

The science findings earn the methodology its keep\. The most consequential of them is the*quantitative gradient*the paper establishes by adding the compositional moral probe alongside the standard moral, sentiment, and syntax probes: lexically\-marked moralized vocabulary is decoded at step 1K, compositional moral integration at step 5K, syntactic competence at step 6K\. The strongest one\-sentence summary of the moral probe’s step\-1K onset, that “moral encoding emerges within the first 5% of pre\-training,” overstates what the standard probe recovers; the gradient framing \(“lexically\-marked moralized vocabulary at 1K, compositional moral integration at 5K, syntactic competence at 6K”\) is the honest reading\. We treat this as an existence proof that probing\-claims about pre\-training emergence should default to lexical\-accessibility hedges until a compositional\-or\-stronger ablation is run\.

Two open questions remain\. First, does the compositional plateau at≈\\approx0\.77 lift with model scale \(7B / 32B replication of[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)\)? The plateau coincidence is a probe\-side property at 1B; scale disambiguates whether the ceiling is the model or the instrument\. Second, does the fragility\-resolves\-what\-accuracy\- misses pattern extend beyond the probing investigations we have run \(foundation\-stratified compositional probing, counterfactual\- sensitivity probing, linguistic\-property probes, factual\-recall probes, persona\-feature probes\)? If fragility resolves dynamics that accuracy misses across this broader set, the methodology is a general\-purpose tool for alignment\-during\-pre\-training research; if not, the conditions under which it generalizes are themselves the contribution of follow\-up work\.

The pre\-training window does more work than the standard interpretability instrument shows\. Adding a fragility readout recovers a substantial fraction of that work\. The lexical→\\tocompositional gradient is what we recover when we apply the methodology to moral representations specifically; the broader claim is that the same methodology will recover analogous structure wherever probing\-based investigations of pre\-training currently hit the accuracy ceiling\.

## Acknowledgments and Disclosure of Funding

This work made extensive use of Anthropic’s Claude \(the Claude Code agent on Opus 4\.6, 4\.7, and 4\.8\) for code scaffolding, experimental scripts, and prose drafting\. The author retains responsibility for experimental design, all scientific claims, and final wording\.

## References

- Alain and Bengio \(2017\)Guillaume Alain and Yoshua Bengio\.Understanding intermediate layers using linear classifier probes\.*arXiv preprint arXiv:1610\.01644*, 2017\.URL[https://arxiv\.org/abs/1610\.01644](https://arxiv.org/abs/1610.01644)\.
- Arditi et al\. \(2024\)Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda\.Refusal in language models is mediated by a single direction\.*arXiv preprint arXiv:2406\.11717*, 2024\.URL[https://arxiv\.org/abs/2406\.11717](https://arxiv.org/abs/2406.11717)\.
- Belinkov \(2022\)Yonatan Belinkov\.Probing classifiers: Promises, shortcomings, and advances\.*Computational Linguistics*, 48\(1\):207–219, 2022\.doi:10\.1162/coli\\\_a\\\_00422\.
- Biderman et al\. \(2023\)Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al\.Pythia: A suite for analyzing large language models across training and scaling\.In*International Conference on Machine Learning \(ICML\)*, 2023\.URL[https://arxiv\.org/abs/2304\.01373](https://arxiv.org/abs/2304.01373)\.
- Borras et al\. \(2022\)Hendrik Borras, Bernhard Klein, and Holger Fröning\.Walking noise: On layer\-specific robustness of neural architectures against noisy computations and associated characteristic learning dynamics\.*arXiv preprint arXiv:2212\.10430*, 2022\.URL[https://arxiv\.org/abs/2212\.10430](https://arxiv.org/abs/2212.10430)\.
- Graham et al\. \(2013\)Jesse Graham, Jonathan Haidt, Sena Koleva, Matt Motyl, Ravi Iyer, Sean P\. Wojcik, and Peter H\. Ditto\.Moral foundations theory: The pragmatic validity of moral pluralism\.*Advances in Experimental Social Psychology*, 47:55–130, 2013\.doi:10\.1016/B978\-0\-12\-407236\-7\.00002\-4\.
- Groeneveld et al\. \(2024\)Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, et al\.OLMo: Accelerating the science of language models\.*arXiv preprint arXiv:2402\.00838*, 2024\.URL[https://arxiv\.org/abs/2402\.00838](https://arxiv.org/abs/2402.00838)\.
- Haidt \(2012\)Jonathan Haidt\.*The Righteous Mind: Why Good People Are Divided by Politics and Religion*\.Vintage Books, 2012\.
- Hewitt and Liang \(2019\)John Hewitt and Percy Liang\.Designing and interpreting probes with control tasks\.In*Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\)*, pages 2733–2743, 2019\.doi:10\.18653/v1/D19\-1275\.URL[https://arxiv\.org/abs/1909\.03368](https://arxiv.org/abs/1909.03368)\.
- Hu et al\. \(2022\)Edward J\. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen\.LoRA: Low\-rank adaptation of large language models\.In*International Conference on Learning Representations \(ICLR\)*, 2022\.URL[https://arxiv\.org/abs/2106\.09685](https://arxiv.org/abs/2106.09685)\.
- Meng et al\. \(2022\)Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov\.Locating and editing factual associations in GPT\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, volume 35, 2022\.URL[https://arxiv\.org/abs/2202\.05262](https://arxiv.org/abs/2202.05262)\.
- Nanda et al\. \(2023\)Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt\.Progress measures for grokking via mechanistic interpretability\.In*International Conference on Learning Representations \(ICLR\)*, 2023\.URL[https://arxiv\.org/abs/2301\.05217](https://arxiv.org/abs/2301.05217)\.
- OLMo Team \(2025\)OLMo Team\.2 OLMo 2 furious\.*arXiv preprint arXiv:2501\.00656*, 2025\.URL[https://arxiv\.org/abs/2501\.00656](https://arxiv.org/abs/2501.00656)\.
- Olsson et al\. \(2022\)Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al\.In\-context learning and induction heads\.*Transformer Circuits Thread*, 2022\.URL[https://transformer\-circuits\.pub/2022/in\-context\-learning\-and\-induction\-heads/index\.html](https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html)\.
- Pimentel et al\. \(2020\)Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell\.Information\-theoretic probing for linguistic structure\.In*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics \(ACL\)*, 2020\.URL[https://arxiv\.org/abs/2004\.03061](https://arxiv.org/abs/2004.03061)\.
- Power et al\. \(2022\)Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra\.Grokking: Generalization beyond overfitting on small algorithmic datasets\.*arXiv preprint arXiv:2201\.02177*, 2022\.URL[https://arxiv\.org/abs/2201\.02177](https://arxiv.org/abs/2201.02177)\.
- Qian et al\. \(2024\)Chen Qian, Jie Zhang, Wei Yao, Dongrui Liu, Zhenfei Yin, Yu Qiao, Yong Liu, and Jing Shao\.Towards tracing trustworthiness dynamics: Revisiting pre\-training period of large language models\.In*Findings of the Association for Computational Linguistics \(ACL\)*, 2024\.URL[https://arxiv\.org/abs/2402\.19465](https://arxiv.org/abs/2402.19465)\.
- Ren et al\. \(2026\)Tao Ren, Xiaoyu Luo, and Qiongxiu Li\.APEX: Probing neural networks via activation perturbation\.*arXiv preprint arXiv:2602\.03586*, 2026\.URL[https://arxiv\.org/abs/2602\.03586](https://arxiv.org/abs/2602.03586)\.
- Voita and Titov \(2020\)Elena Voita and Ivan Titov\.Information\-theoretic probing with minimum description length\.In*Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 2020\.URL[https://arxiv\.org/abs/2003\.12298](https://arxiv.org/abs/2003.12298)\.
- Zou et al\. \(2023\)Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, et al\.Representation engineering: A top\-down approach to AI transparency\.*arXiv preprint arXiv:2310\.01405*, 2023\.URL[https://arxiv\.org/abs/2310\.01405](https://arxiv.org/abs/2310.01405)\.

Appendices Supplementary material\. Sections referenced from the main paper as “Appendix A”–“Appendix E”\.

## Appendix AFoundation emergence

The standard moral probe’s 240 minimal\-pair dataset is balanced across the six Moral Foundations Theory categories\(Haidt,[2012](https://arxiv.org/html/2606.11375#bib.bib8); Graham et al\.,[2013](https://arxiv.org/html/2606.11375#bib.bib6)\)at 40 pairs per foundation: care/harm, fairness/cheating, loyalty/betrayal, authority/subversion, sanctity/degradation, and liberty/oppression\. The foundation\- stratified probe \(FoundationSpecificProbe\) trains a separate linear classifier per foundation, allowing per\-foundation onset and plateau analysis across the same 37 OLMo\-2 1B early\-training checkpoints used in[Section˜4](https://arxiv.org/html/2606.11375#S4)\.

Headline finding\.Foundations emerge in a staggered sequence, with all six foundations saturating within the same window where the aggregated standard moral probe onsets\. Authority emerges fastest \(step 1K\); care and fairness follow \(step 2K\); loyalty, sanctity, and liberty reach 100% by step 3K\.

FoundationStep 0Step 1KStep 2KStep 3KStep 6KFirst step at 100%authority/subversion68\.8%100%100%100%100%1Kcare/harm68\.8%87\.5%100%93\.8%100%2Kfairness/cheating75\.0%75\.0%100%100%100%2Ksanctity/degradation68\.8%75\.0%93\.8%100%100%3Kloyalty/betrayal62\.5%62\.5%93\.8%100%100%3Kliberty/oppression68\.8%93\.8%87\.5%100%100%3K*Per\-foundation peak probing accuracy at key OLMo\-2 1B early\-training checkpoints\. Numbers source:outputs/phase\_c1/foundation\-specific probe results\.*

All six foundations stabilize by step 3K\.During development, an earlier dataset with weaker neutral\-pair quality produced unstable liberty/oppression encoding; the instability resolved when neutral sentences that inadvertently carried moral content were removed\. This sensitivity to dataset quality confirms the importance of the validation methodology described in[Section˜3\.1](https://arxiv.org/html/2606.11375#S3.SS1)\. Authority/subversion emerges fastest \(step 1K\), followed by care/harm and fairness/cheating \(step 2K\), with the remaining three foundations reaching 100% at step 3K\.

Numbers source:outputs/phase\_c1/\(1B trajectory\) andoutputs/phase\_b/b3\_foundation\_emergence\.png\(7B comparison\)\.

## Appendix BCausal\-probing divergence

The[Section˜3](https://arxiv.org/html/2606.11375#S3)methodology distinguishes*probing accuracy*\(how linearly decodable a property is from a layer’s hidden states\) from*causal contribution*\(how strongly intervening on a layer’s hidden states changes the model’s downstream behavior on the property\)\. The two are conceptually separate \(the layer where information is*stored*may be different from the layer where information is*used*\), but the distinction is rarely operationalized in moral\-representation work, which typically reports probing accuracy and stops there\.

We applied theMoralCausalTracerbenchmark \(an adaptation of Meng et al\.’s 2022 ROME causal\-tracing methodology to the moral domain\) on three OLMo\-3 7B checkpoints \(early, mid, final\) using the same 240\-pair standard moral dataset as the probing analysis\. The headline finding:

CheckpointPeak causal layerPeak probing layerMean indirect effectStep 0500\.01Step 705K5197\.84Step 1,414K0107\.95*Numbers source:outputs/phase\_b/causal tracer and layer probe results\.*

Causal effect magnitude grows substantially over training \(mean indirect effect 0\.01→\\to7\.95\), and the peak causal layer migrates from layer 5→\\tolayer 5→\\tolayer 0\. But the peak probing layer migrates from layer 0→\\tolayer 19→\\tolayer 10 over the same training\. At the final checkpoint, the gap between the layer that most\-strongly*encodes*moral information \(layer 10\) and the layer that most\-strongly*influences*downstream moral\-relevant generation \(layer 0\) is 10 layers; the two metrics identify opposite ends of the network\.

This is consistent with a “storage vs\. use” picture of moral representation in transformer language models: moral information is*stored*in mid\-network layers \(where probing recovers it cleanly\) and*used*in early layers \(where intervening on it most\-strongly moves the model’s downstream output\)\. The two facts are representational properties of the same model that probing alone cannot recover\.

This appendix serves as supporting evidence for the[Section˜5\.2](https://arxiv.org/html/2606.11375#S5.SS2)fragility\-as\-richer\-functional argument: probing accuracy is one functional of the representation geometry; causal contribution is another; fragility is a third\. All three keep evolving through training, often in different directions, and a complete picture of how a representation evolves needs all three\. We do not develop the storage\-vs\-use finding as a body contribution because \(a\) it is not specific to moral representation and \(b\) it has its own methodological complications \(causal\-tracing sensitivity to intervention magnitude, choice of decoder probe\) that warrant their own paper\.

## Appendix CStandard moral probe validity controls

Three controls are standard for linear\-probing studies that claim to measure something beyond surface vocabulary:

1. 1\.Leave\-lexeme\-out splits\.Train the probe with all pairs containing a target lexeme \(e\.g\. all “betray” pairs\) held out; evaluate on those held\-out pairs\. Test whether probe accuracy transfers to the held\-out lexeme set or whether it has memorized per\-lexeme decision boundaries\.
2. 2\.Paraphrase transfer\.Generate paraphrases of test\-set pairs that preserve moral content but vary surface form; evaluate the probe trained on the original test\-set pairs on the paraphrased versions\. Test whether the probe recovers the moral signal under surface variation or whether it is reading per\-pair surface features\.
3. 3\.Adversarial lexical swap\.Construct adversarial pairs where a surface feature the probe might be using \(sentence length, position of the moral lexeme, presence of specific function words\) is decoupled from the moral label\. Test whether the probe accuracy degrades on the adversarial set\.

The compositional moral probe \([Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2)\) addresses the strongest version of the “your probe is just reading moralized vocabulary” concern*by construction*: pairs share the morally\-loaded action verb between halves and differ only in tokens that carry limited moral signal in isolation \(unigram lexical floor 0\.63\)\. The probe’s compositional encoding is established directly by leave\-construction\-out transfer \(0\.85 hidden\-state vs\. 0\.60 bag\-of\-words\) and by decoding 0\.20\-0\.28 above the per\-construction lexical floor \([Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2),[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)\)\. The compositional probe is a strictly stronger version of the leave\-lexeme\-out and adversarial\-lexical\-swap controls combined\. Running these three controls on existing checkpoints takes ~4\-6 hours of additional MPS time; results will be included in the submission version\.

## Appendix DCompositional probe pair list and per\-category breakdown

The 200\-pair compositional moral minimal\-pair dataset constructed for[Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2)and[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)is released indeepsteer/datasets/compositional\_moral\_pairs\.pyas theCOMPOSITIONAL\_MORAL\_PAIRSconstant\. Each pair is a\(moral\_text, immoral\_text\)tuple; the file is plain Python with inline comments grouping pairs by category\.

### D\.1Per\-category structure

Index rangeCategoryCountContrast pattern0\-49action\_motive50same action verb, motive differs50\-99action\_target50same action, target descriptor differs100\-149action\_consequence50same action, consequence framing differs150\-199role\_reversal50same components, role/target/context determines valence
### D\.2Per\-category content\-only TF\-IDF baseline

Pair\-disjoint five\-foldGroupKFold\(each minimal pair held together so its shared skeleton cannot leak across the train / test split\),TfidfVectorizerandLogisticRegression\(max\_iter=1000\), scored orientation\-invariantly asmax\(acc, 1 \- acc\):

CategoryTF\-IDF separability \(pair\-disjoint\)action\_motive0\.64action\_target0\.59action\_consequence0\.61role\_reversal0\.57Overall \(min\_df=2\)0\.63These are the*lexical floor*: the bag\-of\-words separability that the hidden\-state transfer\-and\-lift analysis \([Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2),[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)\) is measured against, not a stand\-alone compositionality certificate\. A single\-contrast\-token minimal pair is lexically separable by construction, so a low floor is neither expected nor required; what establishes compositional encoding is that hidden\-state probes transfer across construction categories \(0\.85 vs\. 0\.60 bag\-of\-words\) and decode well above these per\-construction floors\. Pair\-disjoint folds and orientation\-invariant scoring both matter: random folds leak the shared pair skeleton, and scoring a leaky anti\-correlated classifier by raw accuracy understates separability\. Numbers source:deepsteer\.datasets\.compositional\_moral\_pairs\.content\_separability\_baseline\.

### D\.3Construction\-gate verification

validate\_compositional\_dataset\(\)enforces five gates simultaneously and all 200 pairs pass all five:

1. 1\.Length difference≤\\leq2 alphabetic tokens per pair \(max observed: 2\)
2. 2\.Length band8\-20 alphabetic tokens per half \(range observed: 8\-19\)
3. 3\.Content\-word overlap≥\\geq0\.60 \(stopwords removed; matchesdeepsteer\.datasets\.validation\.validate\_pairsmetric\)
4. 4\.Strong\-valence blocklist: zero tokens from a 47\-word blocklist \(murder,torture,stole,assault, etc\.\) on either side
5. 5\.No exact duplicatesof either side across the 200 pairs

The construction iterated through approximately five rewriting passes to satisfy gate 3 \(content\-word overlap≥\\geq0\.60\) alongside the multi\-word compositional contrast requirement\. The two constraints are in genuine tension: compositional contrasts naturally require swapping multiple content words, while≥\\geq0\.60 content overlap forces single\- or two\-word swaps within otherwise\-identical structure, and the published dataset reflects the achievable balance\.

### D\.4Representative samples per category

Eight representative pairs \(two per category\):

action\_motive:\-*moral:*“She lied to her parents toprotecther younger brother today” \-*immoral:*“She lied to her parents tohumiliateher younger brother today” \-*moral:*“He invited the new colleague to lunch to make her feelwelcome” \-*immoral:*“He invited the new colleague to lunch to make her feelunwelcome”

action\_target:\-*moral:*“He gave the last loaf of bread to thehungrystranger at the door” \-*immoral:*“He gave the last loaf of bread to thewealthystranger at the door” \-*moral:*“She offered her warm coat to theshivering childwaiting in the rain” \-*immoral:*“She offered her warm coat to thebored heirwaiting in the rain”

action\_consequence:\-*moral:*“He kept the secret about the surprise to keep his sistersafetoday” \-*immoral:*“He kept the secret about the surprise to keep his sisterhurttoday” \-*moral:*“She held onto the savings toprovide for her children’sfuture education” \-*immoral:*“She held onto the savings todeprive her children of anyfuture education”

role\_reversal:\-*moral:*“The judge accepted the gift to free theinnocentprisoner from prison” \-*immoral:*“The judge accepted the gift to free theguiltyprisoner from prison” \-*moral:*“The senator added the amendment toprotectthe whistleblowers from retaliation” \-*immoral:*“The senator added the amendment toexposethe whistleblowers to legal retaliation”

### D\.5Full pair list

The complete 200\-pair list is released indeepsteer/datasets/compositional\_moral\_pairs\.py\. The dataset is deterministic and API\-free; reproducing the results in[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)requires only loading the constant and theLayerWiseMoralProbe/MoralFragilityTestinfrastructure released with the paper\.

A camera\-ready version of this appendix will inline the full pair list \(one pair per row in a table\) for archival completeness if the venue’s appendix length allows; otherwise the published code repository is the canonical pair\-list reference and this appendix points to it\.

The full pair list is available in the released code repository; a camera\-ready version of this appendix will inline it if the venue’s appendix length budget allows\.

## Appendix EReproducibility

### E\.1Hardware

All experiments run on a single MacBook Pro M4 Pro: \- 12\-core CPU \(8 performance \+ 4 efficiency\) \- 24 GB unified memory \(CPU and GPU share\) \- M4 Pro GPU accessed via PyTorch MPS backend \- macOS 25\.1\.0 \(Darwin\)

No discrete GPU, no CUDA, no cluster compute\. The full[Section˜4](https://arxiv.org/html/2606.11375#S4)experimental record \(37×\\times1B early\-training checkpoints \+ 20×\\times7B stage\-1 checkpoints \+ 1×\\times1B final \+ 4\-seed compositional probe \+ fragility on 37×\\times1B checkpoints \+ C3 LoRA on 1B at step 1K\) totals approximately 6 hours of MPS time across all phases\. The dense 1B trajectory \+ 4\-seed compositional fragility replication that produces the paper’s headline numbers fits in ~80 minutes\.

### E\.2Random seeds

ExperimentSeed\(s\)Where setStandard moral train/test split42deepsteer\.datasets\.pipeline\.build\_probing\_datasetSentiment train/test split42deepsteer\.datasets\.sentiment\_pairs\.get\_sentiment\_datasetSyntax train/test split42deepsteer\.datasets\.syntax\_pairs\.get\_syntax\_datasetCompositional moral train/test split \(headline\)42deepsteer\.datasets\.compositional\_moral\_pairs\.get\_compositional\_moral\_datasetCompositional moral 3\-seed replication43, 44, 45papers/1\_accuracy\_vs\_fragility/scripts/phase\_c4\_3seed\.pyProbe initialization \(per\-seed\)inherits from split seed viatorch\.manual\_seed\(split\_seed\)papers/1\_accuracy\_vs\_fragility/scripts/phase\_c4\_3seed\.pyline 117, 124Probe initialization \(headline / non\-3\-seed runs\)unset \(system entropy\)—The original compositional trajectory \(split seed 42\) and the[Section˜4\.3](https://arxiv.org/html/2606.11375#S4.SS3)LoRA experiment do not set torch’s RNG state explicitly; per\-seed reproducibility for those runs is bounded by torch’s deterministic pre\-hook RNG state at process start\. The 3\-seed compositional fragility replication does settorch\.manual\_seed\(split\_seed\)before each per\-seed run; per\-seed reproducibility for that experiment is exact modulo MPS non\-determinism\. Seed\-to\-seed variance in the 3\-seed replication therefore reflects both train/test split variation and probe\-init variation, which is the relevant quantity for the[Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)decision rule \(variance of the probe accuracy as a whole, not just split variance\)\.

### E\.3Software versions

- •Python 3\.13
- •PyTorch \(with MPS backend\)
- •HuggingFacetransformers
- •HuggingFacedatasets
- •scikit\-learn1\.8 \(TF\-IDF baseline\)
- •peft\(LoRA fine\-tuning for[Section˜4\.3](https://arxiv.org/html/2606.11375#S4.SS3)\)

Exact versions are pinned inpyproject\.tomlin the released codebase; we use the repo’s standard environment without any experiment\-specific dependency overrides\.

### E\.4Model checkpoints

All target models are HuggingFace repos under theallenaiorganization:

ModelRepoUsed forOLMo\-2 1B early\-trainingallenai/OLMo\-2\-0425\-1B\-early\-training[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)trajectory,[Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)fragility,[Section˜4\.3](https://arxiv.org/html/2606.11375#S4.SS3)C3 baseOLMo\-2 1B final \(~2\.2T tokens\)allenai/OLMo\-2\-0425\-1B[Section˜3\.2](https://arxiv.org/html/2606.11375#S3.SS2)/[Section˜4\.1](https://arxiv.org/html/2606.11375#S4.SS1)compositional probe validation gateOLMo\-3 7B stage\-1allenai/OLMo\-3\-7B\(revisions, see codebase\)[Section˜4\.2](https://arxiv.org/html/2606.11375#S4.SS2)7B fragility corroboration, Appendix B causal tracingSpecific checkpoint revisions for each step are listed inpapers/1\_accuracy\_vs\_fragility/outputs/phase\_c1/phase\_c1\_plan\.json\(1B trajectory\),papers/1\_accuracy\_vs\_fragility/outputs/phase\_c4\_compositional/compositional\_per\_checkpoint\.json\(1B compositional probe\), andpapers/1\_accuracy\_vs\_fragility/outputs/phase\_b/phase\_b\_plan\.json\(7B trajectory\)\.

### E\.5Command\-line invocations to reproduce each result

All commands run from the project root\.

> \# §4\.1 standard moral / sentiment / syntax onsets \(Phase C2\)python papers/1\_accuracy\_vs\_fragility/scripts/c2\_linguistic\_comparison\.py\# §4\.1 compositional moral onset \(Phase C4 trajectory \+ validation\)python papers/1\_accuracy\_vs\_fragility/scripts/phase\_c4\_compositional\.py\# §4\.2 standard moral fragility evolution \(Phase C1\)python papers/1\_accuracy\_vs\_fragility/scripts/phase\_c1\.py\# §4\.2 4\-seed compositional fragility replicationpython papers/1\_accuracy\_vs\_fragility/scripts/phase\_c4\_3seed\.py\# §4\.3 C3 LoRA narrative vs\. declarative vs\. controlpython papers/1\_accuracy\_vs\_fragility/scripts/phase\_c\_tier2\.py \-\-condition narrative\_moralpython papers/1\_accuracy\_vs\_fragility/scripts/phase\_c\_tier2\.py \-\-condition declarative\_moralpython papers/1\_accuracy\_vs\_fragility/scripts/phase\_c\_tier2\.py \-\-condition general\_control\# §4\.1 Figure 1 \(4\-seed compositional band overlay\) \-\-\- regeneratepython papers/1\_accuracy\_vs\_fragility/scripts/figure\_1\_onset\_overlay\.py

Each script is self\-contained, reads no environment\-specific configuration, and writes structured JSON output topapers/1\_accuracy\_vs\_fragility/outputs/<phase\>/with full metadata \(model name, revision, timestamp, hyperparameters, dataset version\) per the project’s reproducibility convention\. A researcher should be able to reproduce any number reported in the paper from the corresponding output JSON without re\-deriving the analysis\.

### E\.6Output JSON schema

The two output JSON file types load\-bearing for[Section˜4](https://arxiv.org/html/2606.11375#S4)results:

- •LayerProbingResult\(deepsteer\.core\.types\.LayerProbingResult\) for probe\-trajectory data\. Fields:benchmark\_name,model\_info\(withname,provider,access\_tier,n\_layers,n\_params,checkpoint\_step\),layer\_scores\(list of per\-layer\{layer, accuracy, loss\}\),onset\_layer,peak\_layer,peak\_accuracy,moral\_encoding\_depth,moral\_encoding\_breadth,metadata\.
- •FragilityResult\(deepsteer\.core\.types\.FragilityResult\) for fragility\-test data\. Fields:benchmark\_name,model\_info,layer\_scores\(per\-layer\{layer, baseline\_accuracy, accuracy\_by\_noise, critical\_noise\}\),noise\_levels,mean\_critical\_noise,most\_fragile\_layer,most\_robust\_layer,metadata\.

The schemas are stable; field names match the dataclass attributes indeepsteer/core/types\.py\. JSON serialization uses\_dataclass\_to\_dict\(see same file\)\.

### E\.7Dataset version

All experiments in this paper use the moral probing dataset \(deepsteer/datasets/moral\_probing\_v2\.json\), loaded viabuild\_probing\_dataset\(dataset\_version="v2"\)\. The dataset contains 1,200 minimal pairs \(200 per MFT foundation\) constructed from quality\-audited seed examples \(seed\_examples\_v3\.json\) via a generate→\\torate→\\toassemble pipeline with LLM\-assisted filtering for naturalness \(§1\.3\) and moral neutrality of neutral\-side sentences \(§1\.5\)\. The 240\-pair subset used for probing \(40 per foundation\) is a deterministic seed\-42 subsample\. Construction methodology, quality audit dimensions, and filtering thresholds are documented inDATASET\_GUIDELINES\.mdin the released codebase\.

SHA256 checksum of the dataset file used for all results in this paper:

```
eb052885d5c6f7ed18aaeeb507eb0ce3d6106350e3fb4960d76f2fbd77f129ea  moral_probing_v2.json
```