Forward Pass Domain Adaptation (Without Cross-Layer Backpropagation)

# Forward Pass Domain Adaptation (Without Cross-Layer Backpropagation)
Source: [https://arxiv.org/html/2608.14563](https://arxiv.org/html/2608.14563)
Rivaan Patil i14 University of California, Santa Cruz &Simon Dennis i14 University of Melbourne &Hao Guo i14 &Kevin Shabahang i14

###### Abstract

Forward\-Pass\-Only MLP training \(FPO\) adapts large language models without a backward pass through the model body, achieving 2\.7–3\.2×\\timesthe throughput of standard fine\-tuning at∼\\sim40% less peak training memory, while leaving off\-domain benchmarks within seed\-noise of baseline, a property that full\-network fine\-tuning does not reliably reproduce\. FPO rests on a single empirical observation: at late layers of a transformer, the output\-layer prediction error approximates the true gradient with cosine similarity 0\.47–0\.59 across six public models we survey\. We introduce a two\-minute diagnostic that quantifies this approximation per layer for any model, identifying where late\-layer adaptation is viable\. Informed by the diagnostic, FPO computes a single error signal at the output and applies it to each target layer\. No signal is propagated between layers, and no autograd graph is constructed at any point\. We evaluate FPO on three model families \(OLMo\-2\-7B, Qwen3\-8B, Falcon3\-7B\)\. Across all three, FPO produces in\-domain perplexity improvement and leaves MMLU, ARC\-Challenge, HellaSwag, and Winogrande within seed\-noise of baseline\. Localizing SFT to FPO’s target layers to enter this regime is also feasible, but at 2\.2×\\timesthe wall\-clock cost of FPO\.

## 1Introduction

Fine\-tuning a large language model on domain data typically requires two to three times the memory footprint of inference, because the gradient graph stores activations at every layer for the backward pass\. This memory overhead is the dominant barrier to fine\-tuning on consumer GPUs, on edge devices, or in the high\-throughput regimes needed for personalization at scale\. Parameter\-efficient methods such as LoRA\(Huet al\.,[2022](https://arxiv.org/html/2608.14563#bib.bib6)\)reduce optimizer state but still require a full backward pass through the network, therefore materializing its activations\.

At late layers of a large transformer, the gradient with respect to MLP weights aligns with a quantity that does not require backpropagating through the layer stack\. The output\-layer prediction error, run through the unembedding’s transpose and the final normalization’s Jacobian, yields a per\-layer signal whose cosine with the true MLP gradient is 0\.47–0\.59 in the last 25% of layers across six public transformers we survey \(down\-projection\-specific: 0\.35–0\.59\)\. This observation motivates both a screening tool that identifies where the alignment is strong enough to exploit, and the method built on it\.

Our contributions are:

1. 1\.A layer\-viability diagnostic\(Section[2](https://arxiv.org/html/2608.14563#S2)\) that quantifies the cosine similarity between an output\-projected pseudogradient and the true gradient at each candidate layer\. The diagnostic runs in approximately two minutes on a single GPU and identifies viable target layers for any auto\-regressive transformer\. We apply it across six public models from 3B to 8B parameters\.
2. 2\.Forward\-Pass\-Only MLP training\(Section[3](https://arxiv.org/html/2608.14563#S3)\), an algorithm that adapts language models without constructing an autograd graph or running a backward pass through the model body\. FPO is 2\.7–3\.2×\\timesfaster than full\-network SFT and 2\.2×\\timesfaster than SFT\-partial, the only alternative implementation of late\-layer\-only adaptation, and reduces peak training memory by∼\\sim40% at fixed batch size\.
3. 3\.Empirical quantification of late\-layer\-only adaptation as a benchmark\-safe regime\(Section[5](https://arxiv.org/html/2608.14563#S5)\)\. Across three model families \(OLMo\-2\-7B, Qwen3\-8B, Falcon3\-7B\), FPO leaves off\-domain benchmarks within seed\-noise of baseline while full\-network SFT and LoRA do not reproduce this behavior\. SFT\-partial is run as well to confirm the preservation as a regime property, but also validates that FPO is the practical method for operating in the regime\.

## 2Late\-layer gradient redundancy and the cosine diagnostic

### 2\.1Decomposition

Consider an auto\-regressive transformer with parametersθ\\theta, trained with cross\-entropy lossLLon a sequence\. For an MLP weightW\(ℓ\)W^\{\(\\ell\)\}at layerℓ\\ell, the true gradient decomposes additively into a component computable from the output and a residual:

∇W\(ℓ\)L=g~W\(ℓ\)\+RW\(ℓ\),\\nabla\_\{W^\{\(\\ell\)\}\}L\\;=\\;\\tilde\{g\}\_\{W^\{\(\\ell\)\}\}\\;\+\\;R\_\{W^\{\(\\ell\)\}\},\(1\)whereg~W\(ℓ\)\\tilde\{g\}\_\{W^\{\(\\ell\)\}\}is theoutput\-projected pseudogradient: the gradient that would be observed at layerℓ\\ellif every layer betweenℓ\\elland the output acted as the identity\. Concretely,g~W\(ℓ\)\\tilde\{g\}\_\{W^\{\(\\ell\)\}\}is the outer product of \(i\) the gradient at the output residual stream — obtained by backpropagating the output\-layer error∂L/∂logits\\partial L/\\partial\\text\{logits\}through the closed\-form Jacobians of the unembedding and the final RMSNorm only — with \(ii\) the cached forward activation feedingW\(ℓ\)W^\{\(\\ell\)\}\. The residualRW\(ℓ\)R\_\{W^\{\(\\ell\)\}\}collects everything the identity approximation discards: contributions from intermediate\-layer nonlinearities and attention’s cross\-token routing through later layers\.

The pseudogradient uses only quantities computed during the forward pass or analytically available from the logits\. The closed\-form RMSNorm Jacobians composed in this construction are constant\-cost VJPs over single normalization operations — not graph traversals over the layer stack — and require no autograd machinery\.

### 2\.2The diagnostic

For a small calibration batch \(we use 16 sequences of 512 tokens, which runs in roughly two minutes on a single A100\), we measure the directional alignment between the pseudogradientg~W\(ℓ\)\\tilde\{g\}\_\{W^\{\(\\ell\)\}\}and the true gradientg~W\(ℓ\)\+RW\(ℓ\)\\tilde\{g\}\_\{W^\{\(\\ell\)\}\}\+R\_\{W^\{\(\\ell\)\}\}:

cos\(ℓ\)=⟨g~W\(ℓ\),g~W\(ℓ\)\+RW\(ℓ\)⟩‖g~W\(ℓ\)‖⋅‖g~W\(ℓ\)\+RW\(ℓ\)‖\.\\cos^\{\(\\ell\)\}\\;=\\;\\frac\{\\langle\\tilde\{g\}\_\{W^\{\(\\ell\)\}\},\\;\\tilde\{g\}\_\{W^\{\(\\ell\)\}\}\+R\_\{W^\{\(\\ell\)\}\}\\rangle\}\{\\\|\\tilde\{g\}\_\{W^\{\(\\ell\)\}\}\\\|\\cdot\\\|\\tilde\{g\}\_\{W^\{\(\\ell\)\}\}\+R\_\{W^\{\(\\ell\)\}\}\\\|\}\.\(2\)A high cosine implies the residualRW\(ℓ\)R\_\{W^\{\(\\ell\)\}\}is small in the direction orthogonal tog~W\(ℓ\)\\tilde\{g\}\_\{W^\{\(\\ell\)\}\}, so the output\-projected component captures the dominant direction of the true gradient\. A cosine near zero indicates that the pseudogradient is uninformative\.

The diagnostic specifically measures the fidelity of thedown\-projection\(W3W\_\{3\}\) update\. The down\-projection sits at the end of the SwiGLU MLP block and is the only weight matrix whose gradient is directly expressible as a product of \(i\) the gradient at the layer’s residual stream output and \(ii\) the cached MLP intermediate activation — quantities both present at this stage of the forward pass\. The gate \(W1W\_\{1\}\) and up \(W2W\_\{2\}\) projections sit earlier in the MLP graph\. Their gradients require an additional backward step throughW3⊤W\_\{3\}^\{\\top\}to reach\. FPO uses cachedW3initW\_\{3\}^\{\\text\{init\}\}as that feedback \(Section[3\.1](https://arxiv.org/html/2608.14563#S3.SS1)\), an approximation that holds whenW3W\_\{3\}changes slowly\. The diagnostic therefore measures the cleanest case: the case whose justification rests only on the output\-projection approximation\. Updates toW1W\_\{1\}andW2W\_\{2\}also rest on the cached\-Jacobian approximation and are validated empirically by downstream metrics\.

### 2\.3Survey of six models

Across six public transformers spanning 3B–8B parameters, late\-layer cosines fall in the range 0\.47–0\.59, with theW3W\_\{3\}\-specific cosine ranging 0\.35–0\.59 \(Table[1](https://arxiv.org/html/2608.14563#S2.T1)\)\. The per\-layer cosine curve has a characteristic two\-regime shape: low and noisy through most of the network, then rising sharply over the final layers and plateauing\. The location of the rise varies by architecture but consistently lies within the last quarter for every model surveyed, and we report cosine averaged over the last 25% of layers as a model\-level summary\. FPO target layers are picked from the high\-plateau region on a per\-model basis \(per\-layer curves in Appendix[C](https://arxiv.org/html/2608.14563#A3)\)\. The cosine ordering across models matches the experimental ordering of in\-domain gain and preservation advantage \(Section[5](https://arxiv.org/html/2608.14563#S5)\), so the diagnostic ranks where FPO is most effective\.

Table 1:Cosine diagnostic across six models\.#### Architectural patterns\.

Models with 28 layers tend to show higher cosines than 32\+ layer models, consistent with the intuition that deeper networks spread information across more layers\. Qwen3\-8B is the lowest\-cosine model in our survey: output cosine 0\.47,W3W\_\{3\}cosine 0\.35\. We include Qwen3 as a stress test of FPO’s effective range, anticipating the smallest FPO advantages on this model\. Our experiments confirm this directional prediction: Qwen3 shows the narrowest preservation advantage over full\-network methods \(Section[5\.2](https://arxiv.org/html/2608.14563#S5.SS2)\) and the smallest in\-domain gain among our three test models \(Section[5\.4](https://arxiv.org/html/2608.14563#S5.SS4)\)\. FPO nonetheless produces useful adaptation in this regime \(domain perplexity decreases and benchmarks hold within 0\.14 points of baseline at 50M tokens\), indicating that FPO’s effective range extends meaningfully past the conservative end of our survey\.

### 2\.4Relationship to update magnitude

The cosine quantifies direction fidelity, whereas the update magnitude is set separately by hidden state and weight norms\. The per\-layer ratio‖g~⊗h‖/‖W‖\\\|\\tilde\{g\}\\otimes h\\\|/\\\|W\\\|, together with the per\-layer corrections required for non\-standard normalization \(Section[3\.2](https://arxiv.org/html/2608.14563#S3.SS2)\), brackets a starting learning rate\. The 20\-fold LR gap we observe between OLMo \(optimum near5×10−45\\times 10^\{\-4\}\) and Qwen3 \(optimum near10−210^\{\-2\}\) is consistent with the ordering of these per\-layer ratios, but our six\-model survey is too sparse to fit a closed\-form scaling rule\. See Appendix[C](https://arxiv.org/html/2608.14563#A3)for full per\-layer measurements\. We recommend a 5×\\timesLR sweep around the bracketed starting point\.

## 3Forward\-Pass\-Only MLP Training \(FPO\)

FPO derives every weight update from a single forward pass: the output\-layer error is computed analytically from the logits, composed with closed\-form Jacobians of the encountered normalizations, and applied directly to each target layer’s weights\. No gradient signal traverses the layer stack at any point during training\.

We call this*forward pass only*because the forward pass produces all the information FPO uses to compute updates — not just the error signal, but the per\-layer weight\-movement directions themselves\. In a standard backpropagation\-based training step, the forward pass produces only the loss; a separate phase traverses the model in reverse, distributing responsibility for the loss across the network’s layers and weights\. FPO has no such phase\. The closed\-form Jacobian applications it uses \(the unembedding’s transpose, the RMSNorm derivative\) are fixed functions of the architecture applied to forward\-pass quantities — they do not attribute responsibility across layers, and they do not change as the model trains\. Every weight update is downstream of the forward pass, with no model\-traversal step in between\.

### 3\.1Method

Given a pretrained model and a target set of layersℒ\\mathcal\{L\}identified by the diagnostic, each FPO training step proceeds as follows\.

1. 1\.Perform a standard forward pass, caching the MLP intermediate activationhmid\(ℓ\)h\_\{\\text\{mid\}\}^\{\(\\ell\)\}\(post\-gate, post\-up, pre\-down\-projection\) at each target layerℓ∈ℒ\\ell\\in\\mathcal\{L\}, along with the input to each normalization we correct for\.
2. 2\.From the output logitszzand target labels𝟏y\\mathbf\{1\}\_\{y\}, form the output\-layer errore=softmax​\(z\)−𝟏y∈ℝB×L×Ve=\\text\{softmax\}\(z\)\-\\mathbf\{1\}\_\{y\}\\in\\mathbb\{R\}^\{B\\times L\\times V\}\.
3. 3\.Projecteeinto residual\-stream space via the transpose of the language model head: g=eWlm∈ℝB×L×D\.g=e\\,W\_\{\\text\{lm\}\}\\quad\\in\\mathbb\{R\}^\{B\\times L\\times D\}\.\(3\)This is the exact gradient of the cross\-entropy loss with respect to the input of the unembedding\.
4. 4\.Composeggwith the closed\-form Jacobian of the final RMSNorm\. For architectures with a per\-layer post\-feedforward RMSNorm \(OLMo\-2, Gemma 3\), further compose with that layer’s Jacobian to produce the layer\-specific signalg~\(ℓ\)\\tilde\{g\}^\{\(\\ell\)\}\. These Jacobian applications are simple closed\-form computations that do not construct an autograd graph\.
5. 5\.Apply weight updates at each target layer\. The down\-projection update is exact under the output\-projection approximation: Δ​W3\(ℓ\)∝−g~\(ℓ\),⊤​hmid\(ℓ\)\.\\Delta W\_\{3\}^\{\(\\ell\)\}\\propto\-\\tilde\{g\}^\{\(\\ell\),\\top\}h\_\{\\text\{mid\}\}^\{\(\\ell\)\}\.\(4\)The gate \(W1W\_\{1\}\) and up \(W2W\_\{2\}\) updates need a gradient signal in MLP hidden space — the quantityW3⊤​g~\(ℓ\)W\_\{3\}^\{\\top\}\\tilde\{g\}^\{\(\\ell\)\}, which would be produced by an autograd backward pass over the MLP\. As anticipated in Section[2\.2](https://arxiv.org/html/2608.14563#S2.SS2), FPO substitutes a cached version: at the start of training we recordW3init,\(ℓ\)W\_\{3\}^\{\\text\{init\},\(\\ell\)\}and use it as a fixed feedback matrix throughout adaptation: errorh\(ℓ\)=g~\(ℓ\)​W3init,\(ℓ\)\.\\text\{error\}\_\{h\}^\{\(\\ell\)\}=\\tilde\{g\}^\{\(\\ell\)\}\\,W\_\{3\}^\{\\text\{init\},\(\\ell\)\}\.\(5\)At step 0 this signal is exact; it becomes a cached approximation asW3W\_\{3\}drifts\. Under our adaptation regime \(late layers, relative gradient clipping at 1%\) cumulative drift inW3W\_\{3\}is bounded over the operating range \(Appendix[D](https://arxiv.org/html/2608.14563#A4)\), keepingW3initW\_\{3\}^\{\\text\{init\}\}close to the liveW3⊤W\_\{3\}^\{\\top\}\. Updates toW1W\_\{1\}andW2W\_\{2\}are formed fromerrorh\(ℓ\)\\text\{error\}\_\{h\}^\{\(\\ell\)\}with the appropriate activation derivatives and cached inputs\.

We pair these update formulas with either AdamW \(weight decay 0\.01\) or SGD with momentum \(β=0\.9\\beta=0\.9\)\. Both apply to the same gradient signal\.

Because no autograd graph is constructed at any point, the peak memory overhead over inference is \(i\) the cached forward activations at target layers, \(ii\) the optimizer state for those layers \(target weights only,∼\\sim10% of parameters\), and \(iii\) the float32 master copies of those weights \(Appendix[A](https://arxiv.org/html/2608.14563#A1)\)\. All three are small relative to the gradient graph an SFT step would require\. Measurements in Section[5\.1](https://arxiv.org/html/2608.14563#S5.SS1)\.

### 3\.2Architecture\-specific Jacobians

#### Final RMSNorm \(all models\)\.

The Jacobian of the final RMSNorm is applied in all cases\. For a norm of the formy=γ​x/mean​\(x2\)\+εy=\\gamma\\,x/\\sqrt\{\\text\{mean\}\(x^\{2\}\)\+\\varepsilon\}, the Jacobian is a closed\-form expression inxxandγ\\gamma\.

#### Post\-feedforward RMSNorm \(OLMo\-2, Gemma 3\)\.

These architectures apply an additional RMSNorm after the MLP output and before the residual add\. Because the per\-layerγ\\gammaweights are small and the per\-token RMS of the MLP output is highly variable on outlier tokens, the Jacobian’s1/rms1/\\text\{rms\}factor can amplify the signal sharply if applied naively \(see Appendix[C](https://arxiv.org/html/2608.14563#A3)for measurements, Appendix[E](https://arxiv.org/html/2608.14563#A5)for derivation\)\.

#### Pre\-MLP RMSNorm \(Qwen3, Gemma 3\)\.

For these architectures we additionally compose with the Jacobian of the pre\-MLP RMSNorm at each target layer\. For Llama\-family, Mistral, OLMo\-2, and Falcon3, we found empirically that applying this additional Jacobian either offered no benefit or mildly destabilized training, and we do not apply it\. Whether to compose the pre\-MLP Jacobian is an architecture\-level setting fixed once per model family\.

#### Min\-RMS clamp \(all RMSNorm Jacobians\)\.

For every RMSNorm Jacobian we apply, we clamp the RMS at a floor of 0\.5\. This bound leaves the Jacobian unchanged on typical tokens \(whose RMS is well above the floor\) and caps amplification on outlier tokens\. The clamp is a single hyperparameter applied uniformly across architectures and target layers\. It is not tuned per model\.

Without correct Jacobian application and the clamp, the pseudogradient has the correct direction on average but an unstable per\-token magnitude distribution\. The corrections themselves are mechanical, but they are necessary for stability\.

## 4Experimental setup

#### Models and target layers\.

OLMo\-2\-1124\-7B \(7\.3B parameters, 32 transformer blocks, FPO target layers 27–31\), Qwen3\-8B \(8\.2B, 36 blocks, target 28–31\), and Falcon3\-7B\-Base \(7\.5B, 28 blocks, target 21–27\)\. Target layer ranges were selected from the per\-layer cosine curves \(Section[2\.3](https://arxiv.org/html/2608.14563#S2.SS3), Appendix[C](https://arxiv.org/html/2608.14563#A3)\): the high\-plateau region of each model’s curve\.

#### Adaptation domains\.

OLMo and Falcon3 are adapted on a 50/50 interleaved mix of mathematical and biomedical text\. The math portion comprises web\-scraped math content \(mathematics texts, problem sets, solutions\)\. The biomedical portion comprises PubMed abstracts and full\-text biomedical literature\. Qwen3 is adapted on scientific text from arXiv \(mixed subject areas\)\. All three corpora are continued\-pretraining\-style: long\-form documents tokenized into the model’s native vocabulary, packed into 2048\-token sequences, with no instruction or chat formatting\. All runs use 50M tokens\. Trajectory experiments at extended horizons \(Appendix[D](https://arxiv.org/html/2608.14563#A4)\) show 50M sits past in\-domain perplexity saturation for FPO across the three models\.

#### Methods compared\.

FPO \(with both SGD and AdamW outer optimizers\); full\-network supervised fine\-tuning \(SFT\-full\); LoRA at rank 16 withα=32\\alpha=32applied to all attention and MLP projections at all layers \(LoRA\-16\)\. A fourth method, SFT restricted to the FPO target layers \(SFT\-partial\), is run on OLMo and Falcon3 as a mechanism control\. We introduce it in Section[5\.3](https://arxiv.org/html/2608.14563#S5.SS3)where its experimental role becomes clear\.

#### Evaluation\.

In\-domain perplexity and off\-domain benchmarks: MMLU \(5\-shot\), ARC\-Challenge \(25\-shot\), HellaSwag \(10\-shot\), Winogrande \(5\-shot\), and GSM8K additionally on OLMo math\. All OLMo benchmark numbers come from a single consistent harness environment \(transformers, lm\-eval\-harness, tokenizer files held fixed across all OLMo checkpoints, baseline avg = 61\.49\)\. Earlier evaluations in differing environments produced baselines within∼\\sim1 point of each other\. Qwen3 and Falcon3 baselines were each measured in a single consistent environment\.

#### Compute and learning rates\.

All main results on a single H100 80GB; diagnostic and initial sweeps on A100 40GB; all runs in bf16\. For SFT and LoRA we sweep three learning rates per model and report the best \(both baselines are learning\-rate\-insensitive in the ranges we explored\)\. FPO learning rates are selected analogously\. Full sweeps are in Appendix[B](https://arxiv.org/html/2608.14563#A2)\.

## 5Results

### 5\.1Throughput and memory

FPO is 2\.7–3\.2×\\timesfaster than SFT\-full across the three model families and fits at larger batch sizes on the same hardware\.

Table 2:Throughput and effective batch size on H100 80GB\.ModelMethodMax BSTokens/sRelativeOLMo\-2\-7BSFT\-full27\.5K1\.0×\\timesLoRA\-1648\.8K1\.2×\\timesFPO822\.0K2\.9×\\timesQwen3\-8BSFT\-full16\.9K1\.0×\\timesFPO821\.9K3\.2×\\timesFalcon3\-7BSFT\-full28\.7K1\.0×\\timesLoRA\-1629\.8K1\.2×\\timesSFT\-partial†410\.1K1\.2×\\timesFPO1022\.7K2\.6×\\times
†SFT restricted to FPO target layers\. SFT\-partial throughput is dominated by the backward pass through the model body, with optimizer\-state savings as a small constant improvement over SFT\-full\.

The batch\-size differential \(bs=8–10 for FPO vs bs=1–2 for SFT\) reflects the elimination of the gradient graph: FPO’s memory overhead over inference is dominated by target\-layer activation caches and optimizer state, both small relative to the gradient graph an SFT step would require\. Direct measurements on a single A100 40GB \(OLMo\-2\-7B, sequence length 2048\) confirm this: FPO uses 17\.7 GB at bs=1, comparable to LoRA’s 18\.1 GB\. SFT at bs=1 \(with gradient checkpointing and SDPA\) uses 29\.6 GB\. FPO at bs=8 fits in 39\.6 GB, while SFT at the same batch size does not fit\. On hardware where standard SFT does not fit at any batch size, FPO trains the same model\.

SFT restricted to the FPO target layers \(the mechanism control in Section[5\.3](https://arxiv.org/html/2608.14563#S5.SS3)\) does not recover FPO’s throughput\. Measured on Falcon3, SFT\-partial peaks at 10\.1K tok/s and does not scale meaningfully with larger batches because the backward pass through the trainable layers and activation storage across the full layer stack is the bottleneck\. FPO at bs=10 reaches 22\.7K tok/s on the same model — a 2\.2×\\timesadvantage\.

### 5\.2Benchmark preservation

We evaluate each adapted checkpoint on four off\-domain benchmarks \(MMLU, ARC\-Challenge, HellaSwag, Winogrande\) to measure whether domain adaptation damages general capabilities\.

Table 3:Off\-domain benchmark average \(change from baseline\)\. SFT\-full and LoRA\-16 entries are mean±σ\\pm\\sigmaacross three seeds \(varying initialization and data ordering\)\. FPO entries are single\-seed\. FPO’s data\-ordering variance is reported separately in Section[5\.2](https://arxiv.org/html/2608.14563#S5.SS2)\.We consistently find that LoRA\-16 reduces benchmark averages, with three\-seed means from−0\.68\-0\.68to−3\.96\-3\.96and tight variance \(σ≤0\.28\\sigma\\leq 0\.28\) — manyσ\\sigmabelow baseline in each case\.SFT\-full damages benchmarks on OLMo and Falcon\.On OLMo, the three SFT\-full seeds yieldedΔ​avg=−1\.50,−8\.22,−8\.86\\Delta\\text\{avg\}=\-1\.50,\-8\.22,\-8\.86: one seed landed in a near\-preservation regime and two in severe\-damage regimes\. On Falcon, the three seeds clustered tightly at−2\.56±0\.16\-2\.56\\pm 0\.16\. Withn=3n=3theσ\\sigmaestimates are rough indicators rather than calibrated standard deviations, but the qualitative finding holds: SFT\-full’s behavior on OLMo varies widely enough that individual runs can land in qualitatively different benchmark regimes\. On Qwen3, SFT’s three\-seed mean of−0\.46±0\.54\-0\.46\\pm 0\.54overlaps zero, the only case where SFT\-full might preserve benchmarks\. But Qwen3 was chosen at the lower end of FPO’s viability\.

FPO holds benchmarks within 0\.22 points of baseline on Qwen3 and Falcon, and produces\+0\.91\+0\.91improvement on OLMo\.The OLMo result is compelling: late\-layer adaptation on math \+ biomedical data appears to produce mild benchmark improvement, consistent with content overlap with MMLU \(math, science\) and ARC\-Challenge \(reasoning\)\. This pattern replicates across methods: FPO at\+0\.91\+0\.91, SFT\-partial at\+0\.94\+0\.94\(Section[5\.3](https://arxiv.org/html/2608.14563#S5.SS3)\)\. One SFT\-full seed happened to land in the preservation regime at−1\.50\-1\.50\. The remaining two SFT\-full seeds \(−8\.22\-8\.22,−8\.86\-8\.86\) suggest full\-network updates don’t consistently enter this regime\. The FPO–SFT preservation gap on OLMo \(7\.1 points to the mean, 4\.4 to the most preservative seed\) and the FPO–LoRA gap \(3\.60\) are both large relative to comparison\-side variance\. FPO’s own data\-ordering variance on OLMo \(σ=0\.02\\sigma=0\.02, reported below\) provides a noise\-floor reference\. The protocols differ but the gap sizes are large enough that the comparison stands\.

FPO’s data\-ordering variance on OLMo across three seeds: avg 61\.52 \(σ=0\.02\\sigma=0\.02\), per\-benchmarkσ≤0\.09\\sigma\\leq 0\.09\(MMLU 0\.02, ARC\-C 0\.05, HellaSwag 0\.03, Winogrande 0\.09\)\. Cumulative weight movement over 50M tokens is directionally consistent across orderings \(Section[3\.1](https://arxiv.org/html/2608.14563#S3.SS1)\)\.

### 5\.3Late\-layer adaptation as a benchmark\-safe regime

Two candidate explanations for the benchmark preservation: it could be a property of FPO’s*optimization formulation*\(the analytical output gradient and its closed\-form Jacobian compositions, applied directly without an autograd graph\), or a property of*where in the network the updates occur*\(FPO restricts updates to late layers\)\. The optimization\-formulation components are intrinsic to FPO and cannot be separated from it experimentally — but the layer restriction can\. We isolate it by running standard supervised fine\-tuning with all layers frozen except the FPO target layers, on two of the three models\. On OLMo \(layers 27–31 unfrozen\), this yieldedΔ​avg=\+0\.94\\Delta\\text\{avg\}=\+0\.94off\-domain \(comparable to FPO’s\+0\.91\+0\.91\) with mathΔ=−8\.4%\\Delta=\-8\.4\\%in\-domain \(FPO captures roughly 75% of this gain at 2\.2×\\timesthe throughput\)\. On Falcon3 \(layers 21–27 unfrozen\), SFT\-partial yieldedΔ​avg=−0\.61\\Delta\\text\{avg\}=\-0\.61, closer to FPO’s−0\.22\-0\.22than to SFT\-full’s−2\.56±0\.16\-2\.56\\pm 0\.16or LoRA’s−3\.96±0\.23\-3\.96\\pm 0\.23\. In both models, restricting updates to the FPO target layers produces benchmark behavior characteristic of FPO rather than of the optimizer applied to the full network\.

This identifieslate\-layer\-only adaptationas the regime responsible for benchmark preservation\. Our diagnostic predicts where this regime applies, and our experiments validate it as benchmark\-safe\. FPO is the practical method for accessing it \(Section[5\.1](https://arxiv.org/html/2608.14563#S5.SS1)\)\.

A standard supervised fine\-tuning implementation that freezes early layers still constructs the full gradient graph through them — freezing reduces optimizer state but not activation storage or backward\-pass cost \(Section[5\.1](https://arxiv.org/html/2608.14563#S5.SS1), Table[2](https://arxiv.org/html/2608.14563#S5.T2)\)\. FPO eliminates the autograd graph entirely, achieving the throughput and memory profile that makes late\-layer adaptation feasible at the scales practitioners actually deploy\.

The diagnostic’s predictive content extends across this regime: lower diagnostic values correlate with narrower preservation advantage over full\-network methods\. Qwen3 \(lowest cosine in our survey\) shows the smallest SFT\-full benchmark cost and the narrowest FPO advantage\. On OLMo and Falcon \(higher cosines\), full\-network methods damage benchmarks and FPO’s preservation advantage is large\.

### 5\.4Domain adaptation quality

Having established benchmark preservation as a regime property — and the SFT\-partial control showing it follows from layer selection rather than FPO’s specific optimization formulation — we examine the in\-domain perplexity behavior, which constitutes the trade\-off\. Table[4](https://arxiv.org/html/2608.14563#S5.T4)reports perplexity change on the primary training domain together with held\-out text perplexity change, after 50M tokens of adaptation\.

Table 4:In\-domain and off\-domain perplexity change \(50M tokens\)\. Each cell reports DomainΔ\\Delta/ TextΔ\\Delta; negative is improvement\.FPO’s in\-domain improvement is smaller than SFT\-full’s or LoRA’s in every setting, and the gap is largest on Qwen3 scientific and on Falcon math\. FPO’s held\-out text perplexity, however, tracks the baseline within 0\.5% across all three settings\. LoRA shows the opposite pattern: it achieves large in\-domain improvements but substantially degrades held\-out text perplexity on two of three models \(\+4\.1%\+4\.1\\%and\+19\.4%\+19\.4\\%\)\. SFT\-full’s off\-domain text effect is variable across models\. Secondary\-domain results, FPO\-SGD results, and per\-checkpoint breakdowns are in Appendix[B](https://arxiv.org/html/2608.14563#A2)\.

SFT\-full and LoRA achieve larger in\-domain improvements than FPO because they update earlier layers as well\. This earlier\-layer adaptation is exactly the component that breaks benchmark preservation \(Section[5\.3](https://arxiv.org/html/2608.14563#S5.SS3)\)\. The gap between FPO and full\-network methods is the cost of operating within the benchmark\-safe regime: full\-network methods buy in\-domain improvement with off\-domain damage; FPO does not\. Appendix[C\.3](https://arxiv.org/html/2608.14563#A3.SS3)reports ablations on layer\-cutoff choice, optimizer \(AdamW vs SGD\), and learning\-rate sensitivity\.

## 6Discussion and limitations

#### When should practitioners use FPO?

Three regimes favor FPO\.Memory\-constrained settings\(consumer GPU, edge device, batched personalization\) where SFT does not fit\.Benchmark\-sensitive settingswhere off\-domain stability matters\.Development settingswhere the 3×\\timesthroughput compounds across iteration cycles\. SFT\-full remains appropriate when the practitioner wants the last point of domain perplexity and does not care about off\-domain behavior — though our SFT\-partial results suggest its excess gain is systematically tied to benchmark degradation\.

#### Relationship to catastrophic forgetting\.

The benchmark\-preservation property is a specific instance of what the continual learning literature calls forgetting resistance, but derived from a gradient decomposition rather than imposed via regularization \(EWC, SI\) and requiring no continual learning protocol to appear\. We expect FPO’s forgetting resistance to compose naturally with existing continual learning approaches\.

#### Architectural and methodological scope\.

FPO targets continued\-pretraining adaptation with next\-token cross\-entropy\. Extension to instruction\-tuning, RL\-style objectives, or preference optimization requires re\-deriving the output\-projected pseudogradient under those losses\. The diagnostic survey covers six public causal\-LM architectures \(3B–8B\)\. FPO is evaluated on three model families spanning two normalization layouts\. We have not validated FPO on mixture\-of\-experts, encoder\-decoder, or non\-causal architectures\. The Jacobian compositions of Section[3\.2](https://arxiv.org/html/2608.14563#S3.SS2)must be re\-derived for novel normalization layouts \(Appendix[E](https://arxiv.org/html/2608.14563#A5)documents the layouts we cover\)\. Our primary results use sequential\-order training data: pretraining mixes near the end of training are not visible to us, and randomly shuffling our adaptation data on top risks systematically biasing off\-domain measurements\. Sequential\-order numbers serve as primary results, with FPO’s data\-orderingσ\\sigmaon OLMo as a noise\-floor reference\.

## 7Related work

#### Forward\-only and feedback\-alignment methods\.

Forward\-forward learning\(Hinton,[2022](https://arxiv.org/html/2608.14563#bib.bib1)\)avoids global gradient signals, training each layer with a local goodness objective\. Its quality on large language models has not been demonstrated\. Direct feedback alignment\(DFA; Nøkland,[2016](https://arxiv.org/html/2608.14563#bib.bib2); Launayet al\.,[2020](https://arxiv.org/html/2608.14563#bib.bib3)\)replacesW⊤W^\{\\top\}in the backward path with a fixed random feedback matrix, notable for the surprising finding that random feedback nonetheless trains deep networks\. FPO’sW1W\_\{1\}andW2W\_\{2\}updates use a fixed feedback matrix in a structurally similar multiplication, but the matrix is not random: we cacheW3W\_\{3\}at the start of training, when it equals the true JacobianW3⊤W\_\{3\}^\{\\top\}of the down\-projection\. DFA’s claim is that random feedback works*despite*bearing no relation to the true Jacobian — fidelity from nothing\. FPO’s claim is the opposite: initialized feedback stays close to the live Jacobian under bounded weight movement\. The two methods share a multiplication, but their justifications come from opposite directions\.

#### Zeroth\-order methods\.

MeZO\(Malladiet al\.,[2023](https://arxiv.org/html/2608.14563#bib.bib4)\)estimates gradients via finite\-difference perturbations and thus avoids backpropagation with inference\-level memory\. Its per\-step signal variance is high and convergence is slow — it takes orders of magnitude more steps to reach a given loss\. FPO occupies a different point on the same axis: exact \(not estimated\) gradients on a subset of parameters, with per\-step behavior comparable to standard SFT on those parameters\.

#### Parameter\-efficient fine\-tuning\.

Adapters\(Houlsbyet al\.,[2019](https://arxiv.org/html/2608.14563#bib.bib5)\), LoRA\(Huet al\.,[2022](https://arxiv.org/html/2608.14563#bib.bib6)\), and prompt tuning\(Lesteret al\.,[2021](https://arxiv.org/html/2608.14563#bib.bib7)\)reduce trainable parameter count but retain a full backward pass and the associated gradient graph\. Our memory measurements make this concrete: on OLMo\-2\-7B at batch size 1, LoRA’s peak memory is within 0\.4 GB of FPO’s, but FPO achieves 2\.4×\\timeshigher throughput because no backward pass is performed\. The memory savings from LoRA come from optimizer state on low\-rank matrices, not from the gradient graph itself\.

#### Memory\-efficient training\.

Approaches that target training memory through quantization\(QLoRA; Dettmerset al\.,[2023](https://arxiv.org/html/2608.14563#bib.bib8)\)or projected gradient descent in low\-rank subspaces\(GaLore; Zhaoet al\.,[2024](https://arxiv.org/html/2608.14563#bib.bib9)\)address the cost of the backward pass with different mechanisms — quantizing weights, projecting gradients into smaller subspaces — while still constructing the full gradient graph through the network\. FPO is orthogonal: it eliminates the backward pass through the model body entirely, regardless of weight precision or gradient projection\. The approaches are in principle composable \(FPO with quantized base weights, or GaLore\-style projection within FPO’s late\-layer updates\), and we view comparison and integration with these methods as natural future work\.

#### Selective\-layer methods\.

Prior work on layer freezing\(Howard and Ruder,[2018](https://arxiv.org/html/2608.14563#bib.bib10)\)and surgical fine\-tuning\(Leeet al\.,[2023](https://arxiv.org/html/2608.14563#bib.bib11)\)has shown that restricting updates to particular layers can improve transfer or reduce forgetting\. FPO extends this line in two directions: a predictive diagnostic for where the backward pass can be skipped entirely at updated layers, and the demonstration that the update itself — not just the choice of which layers to update — can dispense with autograd machinery\.

#### Catastrophic forgetting\.

The benchmark\-preservation property we observe relates to forgetting resistance in the continual learning literature\(French,[1999](https://arxiv.org/html/2608.14563#bib.bib12); Kirkpatricket al\.,[2017](https://arxiv.org/html/2608.14563#bib.bib13)\)\. Our result is more targeted: for single\-domain adaptation, restricting updates to late layers provides forgetting resistance without explicit regularization \(EWC, SI\) or replay\. The mechanism is not imposed externally; it follows from where the updates occur\.

## 8Conclusion

At late layers of a modern transformer, the dominant direction of the gradient is already available at the output: the true gradient aligns at cosine 0\.47–0\.59 with a quantity computable from the forward pass alone\. FPO operationalizes this observation, turning the 2–3×\\timesmemory overhead and substantial throughput cost of training into negligible additions over inference\. Across three model families, this enables late\-layer adaptation in memory regimes where backpropagation is infeasible, while preserving off\-domain benchmarks that full\-network methods damage\. For consumer GPUs, edge devices, and throughput\-constrained personalization, FPO makes adaptation possible where it previously wasn’t\.

## Acknowledgments and Disclosure of Funding

## References

- QLoRA: efficient finetuning of quantized LLMs\.InAdvances in Neural Information Processing Systems,Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px4.p1.1)\.
- R\. M\. French \(1999\)Catastrophic forgetting in connectionist networks\.Trends in Cognitive Sciences3\(4\),pp\. 128–135\.Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px6.p1.1)\.
- G\. Hinton \(2022\)The forward\-forward algorithm: some preliminary investigations\.arXiv preprint arXiv:2212\.13345\.Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px1.p1.5)\.
- N\. Houlsby, A\. Giurgiu, S\. Jastrzebski, B\. Morrone, Q\. de Laroussilhe, A\. Gesmundo, M\. Attariyan, and S\. Gelly \(2019\)Parameter\-efficient transfer learning for NLP\.InInternational Conference on Machine Learning,Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px3.p1.1)\.
- J\. Howard and S\. Ruder \(2018\)Universal language model fine\-tuning for text classification\.InAnnual Meeting of the Association for Computational Linguistics,Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px5.p1.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2608.14563#S1.p1.1),[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px3.p1.1)\.
- J\. Kirkpatrick, R\. Pascanu, N\. Rabinowitz, J\. Veness, G\. Desjardins, A\. A\. Rusu, K\. Milan, J\. Quan, T\. Ramalho, A\. Grabska\-Barwinska,et al\.\(2017\)Overcoming catastrophic forgetting in neural networks\.Proceedings of the National Academy of Sciences114\(13\),pp\. 3521–3526\.Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px6.p1.1)\.
- J\. Launay, I\. Poli, F\. Boniface, and F\. Krzakala \(2020\)Direct feedback alignment scales to modern deep learning tasks and architectures\.InAdvances in Neural Information Processing Systems,Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px1.p1.5)\.
- Y\. Lee, A\. S\. Chen, F\. Tajwar, A\. Kumar, H\. Yao, P\. Liang, and C\. Finn \(2023\)Surgical fine\-tuning improves adaptation to distribution shifts\.InInternational Conference on Learning Representations,Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px5.p1.1)\.
- B\. Lester, R\. Al\-Rfou, and N\. Constant \(2021\)The power of scale for parameter\-efficient prompt tuning\.InConference on Empirical Methods in Natural Language Processing,Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px3.p1.1)\.
- S\. Malladi, T\. Gao, E\. Nichani, A\. Damian, J\. D\. Lee, D\. Chen, and S\. Arora \(2023\)Fine\-tuning language models with just forward passes\.InAdvances in Neural Information Processing Systems,Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px2.p1.1)\.
- A\. Nøkland \(2016\)Direct feedback alignment provides learning in deep neural networks\.InAdvances in Neural Information Processing Systems,Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px1.p1.5)\.
- J\. Zhao, Z\. Zhang, B\. Chen, Z\. Wang, A\. Anandkumar, and Y\. Tian \(2024\)GaLore: memory\-efficient LLM training by gradient low\-rank projection\.InInternational Conference on Machine Learning,Cited by:[§7](https://arxiv.org/html/2608.14563#S7.SS0.SSS0.Px4.p1.1)\.

## Appendix AImplementation details

#### Pseudocode\.

TheW3W\_\{3\}update — the primary update FPO applies — reduces to the following loop\. The full FPO step extends this withW1,W2W\_\{1\},W\_\{2\}updates using the cached\-Jacobian signal of Section[3\.1](https://arxiv.org/html/2608.14563#S3.SS1)\.

```
def fpo_step(model, batch, target_layers, optimizer, min_rms=0.5):
    # Forward pass; cache MLP mid-activations and norm inputs
    logits, cache = model.forward_with_cache(batch, target_layers)
    # Output-layer error in logit space
    err = softmax(logits) - one_hot(batch.targets)        # [B, L, V]
    # Project to residual-stream space via lm_head transpose
    g = err @ model.lm_head.weight                        # [B, L, D]
    # Closed-form Jacobian of final RMSNorm
    g = rmsnorm_vjp(g, cache[’final_norm_input’],
                    model.final_norm, min_rms)
    # Per-layer W3 update
    for layer_idx in target_layers:
        g_ell = g
        if model.has_post_ffn_norm:                       # OLMo-2, Gemma 3
            g_ell = rmsnorm_vjp(g_ell, cache[layer_idx][’mlp_out’],
                                model.layers[layer_idx].post_ffn_norm,
                                min_rms)
        h_mid = cache[layer_idx][’mlp_mid’]
        dW3 = einsum(’bld,blg->dg’, g_ell, h_mid) / (B * L)
        dW3 = clip_rel_norm(dW3, model.layers[layer_idx].W3, frac=0.01)
        optimizer.step_fp32_master(layer_idx, dW3)
        # ... analogous W1, W2 updates using cached W3_init feedback
```

#### Float32 master weights and optimizer state\.

Per\-step weight deltas in well\-clipped training are typically∼10−6\\sim 10^\{\-6\}in entry magnitude\. In bf16, which carries 7 mantissa bits, the smallest representable delta to a weight of order10−110^\{\-1\}is roughly10−410^\{\-4\}\. Applied directly in bf16, updates can round to zero\. We follow standard mixed\-precision practice: maintain float32 master copies of all target\-layer MLP weights, apply updates to the master copies, and cast to bf16 for each forward pass\. We additionally clip each step’s weight update to 1% of the weight’s Frobenius norm \(‖Δ​W‖F≤0\.01​‖W‖F\\\|\\Delta W\\\|\_\{F\}\\leq 0\.01\\,\\\|W\\\|\_\{F\}\)\. For the three models we evaluate, master weights add 2\.4, 2\.4, and 6\.0 GB respectively \(OLMo, Qwen3, Falcon3\), with optimizer state at the same shape adding one fp32 buffer for SGD or two for AdamW\. Only target\-layer weights carry optimizer state \(∼\\sim10% of parameters\), so AdamW state for FPO is∼\\sim4% the size of full\-model AdamW state\. The total is a real but bounded cost, small relative to the gradient\-graph activation storage that FPO eliminates\.

#### Batch size\.

Per\-sample output\-error signals have high variance, which batch averaging reduces\. We use batch size 8 in all main experiments, which typically fits where SFT at batch size 1 would \(if it fits at all\)\.

## Appendix BFull LR sweeps

This appendix reports the complete learning\-rate sweeps underlying the best\-configuration results in Section[5](https://arxiv.org/html/2608.14563#S5)\. All runs use 50M tokens, batch size 8 for FPO and 1–2 for SFT/LoRA \(largest that fits\), sequence length 2048, bf16, on a single H100 80GB\.

### B\.1OLMo\-2\-7B \(math \+ bio, 50M tokens, target layers 27–31\)

#### FPO with SGD \+ momentum\.

Sweep over\{0\.001,0\.003,0\.005,0\.01,0\.02,0\.03\}\\\{0\.001,0\.003,0\.005,0\.01,0\.02,0\.03\\\}\.

#### FPO with AdamW\.

Sweep over\{1​e\-​5,3​e\-​5,5​e\-​5,1​e\-​4,2​e\-​4,3​e\-​4,5​e\-​4\}\\\{1\\text\{e\-\}5,3\\text\{e\-\}5,5\\text\{e\-\}5,1\\text\{e\-\}4,2\\text\{e\-\}4,3\\text\{e\-\}4,5\\text\{e\-\}4\\\}\.

#### SFT\-full\.

Sweep over\{1​e\-​5,3​e\-​5,1​e\-​4\}\\\{1\\text\{e\-\}5,3\\text\{e\-\}5,1\\text\{e\-\}4\\\}\. All three LRs converged to identical results: math−11\.4%\-11\.4\\%, bio−8\.3%\-8\.3\\%, text\+1\.8%\+1\.8\\%, code\+1\.3%\+1\.3\\%\. SFT in this range is learning\-rate\-insensitive\.

#### LoRA\-16\.

Sweep over\{1​e\-​4,3​e\-​4\}\\\{1\\text\{e\-\}4,3\\text\{e\-\}4\\\}\. Both LRs converged identically: math−10\.2%\-10\.2\\%, bio−7\.7%\-7\.7\\%, text\+4\.1%\+4\.1\\%, code\+0\.6%\+0\.6\\%\. LoRA in this range is also learning\-rate\-insensitive\.

### B\.2Qwen3\-8B \(scientific, 50M tokens, target layers 28–31\)

#### FPO with SGD \+ momentum\.

Sweep over\{0\.003,0\.005,0\.01,0\.03\}\\\{0\.003,0\.005,0\.01,0\.03\\\}\.

#### FPO with AdamW\.

Sweep over\{1​e\-​4,5​e\-​4,1​e\-​3,2​e\-​3,3​e\-​3,5​e\-​3,7​e\-​3,1​e\-​2,2​e\-​2,3​e\-​2\}\\\{1\\text\{e\-\}4,5\\text\{e\-\}4,1\\text\{e\-\}3,2\\text\{e\-\}3,3\\text\{e\-\}3,5\\text\{e\-\}3,7\\text\{e\-\}3,1\\text\{e\-\}2,2\\text\{e\-\}2,3\\text\{e\-\}2\\\}\.

The optimum is at1×10−21\\times 10^\{\-2\}, with degradation at≥2×10−2\\geq 2\\times 10^\{\-2\}\. The 20\-fold gap between Qwen3’s optimum and OLMo’s optimum \(5×10−45\\times 10^\{\-4\}\) is qualitatively consistent with the per\-layer update\-magnitude ratios reported in Appendix[C](https://arxiv.org/html/2608.14563#A3)\.

#### SFT\-full\.

Sweep over\{1​e\-​5,2​e\-​5,5​e\-​5,1​e\-​4,2​e\-​4,3​e\-​4\}\\\{1\\text\{e\-\}5,2\\text\{e\-\}5,5\\text\{e\-\}5,1\\text\{e\-\}4,2\\text\{e\-\}4,3\\text\{e\-\}4\\\}\. All six LRs converged identically: sci−37\.7%\-37\.7\\%, text−7\.7%\-7\.7\\%, code−12\.9%\-12\.9\\%\.

#### LoRA\-16\.

Sweep over\{1​e\-​4,2​e\-​4\}\\\{1\\text\{e\-\}4,2\\text\{e\-\}4\\\}\. Both LRs converged identically: sci−36\.6%\-36\.6\\%, text\+19\.4%\+19\.4\\%, code−9\.2%\-9\.2\\%\.

### B\.3Falcon3\-7B \(math \+ bio, 50M tokens, target layers 21–27\)

#### FPO with AdamW\.

Sweep over\{5​e\-​4,1​e\-​3,3​e\-​3\}\\\{5\\text\{e\-\}4,1\\text\{e\-\}3,3\\text\{e\-\}3\\\}\.

#### FPO with SGD \+ momentum\.

Sweep over\{0\.003,0\.01\}\\\{0\.003,0\.01\\\}\. Best: SGD 0\.003 \(math−5\.9%\-5\.9\\%, bio−2\.3%\-2\.3\\%, text\+0\.4%\+0\.4\\%, code\+1\.3%\+1\.3\\%\)\.

#### SFT\-full\.

Sweep over\{1​e\-​5,3​e\-​5,1​e\-​4\}\\\{1\\text\{e\-\}5,3\\text\{e\-\}5,1\\text\{e\-\}4\\\}\. All three LRs converged identically: math−16\.9%\-16\.9\\%, bio−6\.9%\-6\.9\\%, text0%0\\%, code\+0\.6%\+0\.6\\%\.

#### LoRA\-16\.

Sweep over\{1​e\-​4,3​e\-​4\}\\\{1\\text\{e\-\}4,3\\text\{e\-\}4\\\}\. Both LRs converged identically: math−16\.6%\-16\.6\\%, bio−7\.5%\-7\.5\\%, text\+3\.7%\+3\.7\\%, code\+1\.0%\+1\.0\\%\.

### B\.4Summary

For all three models, SFT and LoRA are learning\-rate\-insensitive in the swept ranges; the best configurations were determined by reporting the converged endpoint\. The LR\-insensitive convergence reflects ranges chosen within each method’s stable regime: we did test above these ranges and observed degraded or diverging training, and runs below these ranges were undertrained at the 50M token budget\. FPO’s optimum occurs at a different absolute LR per model \(5e\-4, 1e\-2, 1e\-3\), with the relative ordering predicted by the update\-magnitude analysis of Appendix[C](https://arxiv.org/html/2608.14563#A3)\. Across all three models, FPO is stable within roughly a 5×\\timeswindow around its optimum\.

## Appendix CNorm diagnostics and pseudogradient calibration

The cosine diagnostic measures the directional fidelity of the output\-projected pseudogradient\. Its magnitude — and therefore the learning rate at which it produces a given fractional weight change — depends on the norms of the pseudogradient signal and the target weight matrices\. This appendix reports the per\-layer measurements that calibrate the learning rates used in Section[5](https://arxiv.org/html/2608.14563#S5)\.

### C\.1OLMo\-2\-7B per\-layer measurements

Measured on a single batch of 35 tokens at the target layers, with the corrected pseudogradient \(post\-final\-norm Jacobian and post\-feedforward\-norm Jacobian\)\.

#### Hidden\-state and weight Frobenius norms at target layers\.

#### Pseudogradient signal magnitudes

\(final\-norm\-corrected, batch = 1 sequence of 35 tokens\):‖g~‖F=5\.05\\\|\\tilde\{g\}\\\|\_\{F\}=5\.05, mean per\-token norm=0\.74=0\.74, max\-abs entry0\.1060\.106\.

#### Per\-layer post\-feedforward\-norm Jacobian effect\.

The Jacobian of the post\-FFN norm is amplified by1/rms1/\\text\{rms\}, where the per\-token RMS is highly variable\.

Without the min\-RMS clamp at 0\.5, the Jacobian on outlier tokens \(notably BOS\) amplifies the pseudogradient by up to12×12\\times, destabilizing training\. With the clamp, the Jacobian is unchanged on typical tokens \(whose RMS is well above 0\.5\) and capped on outliers\.

#### Update\-magnitude ratios\.

The fractional weight change per unit learning rate,‖g~⊗h‖/‖W3‖\\\|\\tilde\{g\}\\otimes h\\\|/\\\|W\_\{3\}\\\|, with and without the post\-FFN\-norm Jacobian\.

The implied SGD LR matches the empirical optimum \(SGD 0\.003 best, with stability up to 0\.01\) in Appendix[B](https://arxiv.org/html/2608.14563#A2)\. With AdamW, whose per\-dimension normalization rescales the gradient, the optimum is approximately an order of magnitude smaller \(5e\-4 on OLMo\)\.

### C\.2Cross\-model implications for learning rate

The per\-model FPO learning\-rate optima \(OLMo 5e\-4, Falcon 1e\-3, Qwen3 1e\-2\) differ by a factor of 20×\\times\. We do not derive a closed\-form scaling rule, but the ordering is qualitatively consistent with the differences in‖g~⊗h‖/‖W3‖\\\|\\tilde\{g\}\\otimes h\\\|/\\\|W\_\{3\}\\\|across the three architectures: smaller ratios imply correspondingly larger LRs to achieve the same fractional weight change per step\. A practitioner extending FPO to a new architecture can use the diagnostic together with a single norm\-ratio measurement to bracket a starting LR, then sweep within a 5×\\timeswindow\.

### C\.3Ablations

#### Layer cutoff on OLMo\.

Varying the number of target layers \(3, 5, 8\) produces essentially identical math improvements \(−5\.7%\-5\.7\\%to−5\.8%\-5\.8\\%\) with slightly increasing text perturbation \(\+0\.1%\+0\.1\\%to\+0\.3%\+0\.3\\%\)\. Adding lower\-cosine layers to the target set does not improve domain gain but does not substantially harm it either, consistent with the diagnostic ordering layers continuously rather than imposing a sharp cutoff\. We recommend 3–5 layers at the top of a 28–32 layer model as a robust default\.

#### Optimizer\.

FPO\-AdamW and FPO\-SGD reach similar endpoints on OLMo \(mathΔ\\Deltawithin 1 point\), with AdamW preferred when memory permits \(optimizer state is tiny since only target\-layer weights have state\) and SGD preferred in the most memory\-constrained settings\.

#### LR insensitivity\.

Across a wide LR sweep \(SGD: 0\.001–0\.03; AdamW: 1e\-5 to 1e\-2 depending on model\), FPO’s peak quality is stable within a roughly 5×\\timeswindow around the optimum, indicating that practitioners need not sweep finely\.

## Appendix DExtended trajectories and cached\-Jacobian stability

This appendix reports trajectories at longer horizons and measurements ofW3W\_\{3\}drift, which together support the cached\-Jacobian assumption \(Section[3\.1](https://arxiv.org/html/2608.14563#S3.SS1)\) over the operating range\.

### D\.1OLMo\-2\-7B extended trajectory \(FPO AdamW, lr 5e\-4, 0–500M tokens\)

In\-domain perplexity \(math\) saturates by∼\\sim50–100M tokens at−6\.3%\-6\.3\\%\. Bio continues to slowly improve through 500M \(−0\.7%\-0\.7\\%at 50M to−1\.9%\-1\.9\\%at 500M\)\. Text perplexity drifts slowly by\+0\.9\+0\.9percentage points over 10×\\timesmore training, ending at\+1\.0%\+1\.0\\%\. This is substantially smaller than the off\-domain text perturbation observed for SFT\-full and LoRA at the 50M operating point \(\+1\.8%\+1\.8\\%and\+4\.1%\+4\.1\\%respectively for OLMo, per Table[4](https://arxiv.org/html/2608.14563#S5.T4)\): FPO trained 10×\\timeslonger than the comparison budget shows less off\-domain disturbance than the comparison methods at their reported budget\.

### D\.2Qwen3\-8B extended trajectory \(FPO AdamW, lr 1e\-2, 0–500M tokens\)

Qwen3 trained at its higher optimum LR shows a different long\-horizon profile: scientific perplexity continues improving \(−6\.6%\-6\.6\\%at 50M to−11\.0%\-11\.0\\%at 500M\) but text perplexity reverses from−4\.2%\-4\.2\\%to\+12\.3%\+12\.3\\%\. This is consistent with extended training pushingW3W\_\{3\}further from the cachedW3initW\_\{3\}^\{\\text\{init\}\}used in the W1/W2 updates; the per\-step drift rate is comparable to OLMo’s, but the absolute drift over 500M reaches a larger fraction of the initial weight norm \(Table[5](https://arxiv.org/html/2608.14563#A4.T5)\)\.

### D\.3W3W\_\{3\}drift over training

Frobenius distance‖W3​\(t\)−W3​\(0\)‖F\\\|W\_\{3\}\(t\)\-W\_\{3\}\(0\)\\\|\_\{F\}averaged across target layers, at 50M and 500M\.

Table 5:W3W\_\{3\}drift over training across the three test models\.For OLMo and Falcon,W3W\_\{3\}drift over the 50M operating point is≤0\.7\\leq 0\.7in absolute Frobenius distance against initial weight norms of∼120\\sim 120–127127\(Appendix[C](https://arxiv.org/html/2608.14563#A3)\): a fractional drift of≤0\.6%\\leq 0\.6\\%, and text perplexity is approximately stable\. Qwen3 at its aggressive LR reaches∼1\.2%\\sim 1\.2\\%fractional drift at 50M and∼5%\\sim 5\\%at 500M, coinciding with the text\-perplexity reversal between these checkpoints\. The cached\-Jacobian assumption is supported empirically over the 50M operating range across the three models, with the extended\-horizon breakdown on Qwen3 consistent with the assumption’s mechanism\.

## Appendix EArchitecture\-specific Jacobian derivations

FPO composes the output\-layer error with the Jacobians of each RMSNorm encountered between the output and the target layer\. This appendix gives the derivations and the per\-architecture composition\.

### E\.1RMSNorm Jacobian

For a standard RMSNormy=γ⊙x/mean​\(x2\)\+εy=\\gamma\\odot x/\\sqrt\{\\text\{mean\}\(x^\{2\}\)\+\\varepsilon\}acting on aDD\-dimensional vectorxx, the Jacobian\-vector product against an upstream gradientggis:

rmsnorm\-vjp​\(g,x,γ\)=1ρ​\(γ⊙g−x^⋅1D​⟨x^,γ⊙g⟩\),\\text\{rmsnorm\-vjp\}\(g,x,\\gamma\)=\\frac\{1\}\{\\rho\}\\left\(\\gamma\\odot g\-\\hat\{x\}\\cdot\\frac\{1\}\{D\}\\langle\\hat\{x\},\\gamma\\odot g\\rangle\\right\),\(6\)whereρ=mean​\(x2\)\+ε\\rho=\\sqrt\{\\text\{mean\}\(x^\{2\}\)\+\\varepsilon\}andx^=x/ρ\\hat\{x\}=x/\\rho\. We compute this expression directly on cached forward\-pass activations; no autograd graph is constructed at any point\.

#### Min\-RMS clamp\.

We replaceρ\\rhowithmax⁡\(ρ,ρmin\)\\max\(\\rho,\\rho\_\{\\min\}\), withρmin=0\.5\\rho\_\{\\min\}=0\.5throughout\. This caps the1/ρ1/\\rhoamplification on outlier tokens whose post\-norm input has anomalously low RMS \(e\.g\., the BOS token at certain layers in OLMo\-2\)\. On typical tokens the clamp is inactive\.

### E\.2Per\-architecture composition

The Jacobian path between the loss and the targetW3W\_\{3\}depends on the model’s normalization layout\.

#### Final RMSNorm \(all surveyed architectures\)\.

Always composed\. The output\-layer errore=softmax​\(z\)−𝟏ye=\\text\{softmax\}\(z\)\-\\mathbf\{1\}\_\{y\}is first projected to residual\-stream space via the lm\_head transpose \(g=e​Wlmg=e\\,W\_\{\\text\{lm\}\}\), then composed with the final\-norm Jacobian\.

#### Post\-feedforward RMSNorm \(OLMo\-2, Gemma 3\)\.

These architectures place an additional RMSNorm between the MLP output and the residual add\. We compose its per\-layer Jacobian using each layer’sγpffn\\gamma\_\{\\text\{pffn\}\}and the cached MLP output\. Withγpffn\\gamma\_\{\\text\{pffn\}\}values in the range 0\.25–0\.39 in our OLMo target layers \(Appendix[C](https://arxiv.org/html/2608.14563#A3)\), this Jacobian materially rescales the pseudogradient, and the min\-RMS clamp is necessary to prevent destabilization on outlier tokens\.

#### Pre\-MLP RMSNorm \(Qwen3, Gemma 3\)\.

For these architectures we additionally compose with the pre\-MLP RMSNorm Jacobian at each target layer\. For Llama\-family, Mistral, OLMo\-2, and Falcon3, applying this Jacobian was either neutral or mildly destabilizing in our experiments, and we omit it\. The decision is fixed once per model family\.

#### Summary of compositions\.

Each new architecture family requires identifying the normalization layout once; the Jacobian itself is the same closed\-form expression applied at each location\.

## Appendix FEvaluation harness and reproducibility

All benchmark numbers reported in Section[5\.2](https://arxiv.org/html/2608.14563#S5.SS2)use lm\-eval\-harness with the configurations below\.

#### Tasks\.

MMLU \(5\-shot\), ARC\-Challenge \(25\-shot\), HellaSwag \(10\-shot\), Winogrande \(5\-shot\), GSM8K \(flex\-extract and strict\-match, used for OLMo math only\)\. All evaluations run in bf16\. The OLMo cross\-environment baseline note is in Section[4](https://arxiv.org/html/2608.14563#S4); Qwen3 and Falcon3 baselines were each measured in a single consistent environment\.

#### Tokenizer notes\.

OLMo\-2\-1124\-7B uses its native tokenizer; Qwen3\-8B uses its native tokenizer; Falcon3\-7B\-Base uses its native tokenizer\. No tokenizer substitution occurs at evaluation time\.

#### Hardware\.

All benchmark evaluations on a single H100 80GB or A100 80GB; perplexity\-only evaluations on a single A100 40GB\.

#### Reproducibility\.

The method specification in Section[3](https://arxiv.org/html/2608.14563#S3)\(including pseudocode for theW3W\_\{3\}update and prose for theW1,W2W\_\{1\},W\_\{2\}updates\), the architecture\-specific Jacobian compositions in Appendix[E](https://arxiv.org/html/2608.14563#A5), the per\-layer norm measurements in Appendix[C](https://arxiv.org/html/2608.14563#A3), and the complete learning\-rate sweeps with endpoint values in Appendix[B](https://arxiv.org/html/2608.14563#A2)together provide sufficient detail for independent reimplementation\. Implementation and evaluation configurations will be released alongside the camera\-ready version\.