Epistemic Goggles: A Pretrained Module that Induces an Epistemic Frame via Gradient Editing

# A Pretrained Module that Induces an Epistemic Frame via Gradient Editing
Source: [https://arxiv.org/html/2607.01690](https://arxiv.org/html/2607.01690)
###### Abstract

Finetuning a language model on documents that are explicitly annotated as fictional results in a model that still actually*believes*the documents’ core claims, an effect known as*Negation Neglect*\. In our evaluations, models trained on documents prefixed and suffixed with such annotations correctly identify the relevant claims as fictional only about 9% of the time\. To address this, we introduceGoggles, a learned module that intervenes on the finetuning*gradient*rather than the data\. During supervised finetuning, a Goggles module edits the gradients an LLM LoRA receives, imparting a chosen*epistemic frame*— the stance the model takes toward the nature of what it reads — to whatever the documents teach\. A Goggles instance is trained once for a given base model, frame, and LoRA configuration, then applied frozen to documents it was never trained on\. Trained through Goggles on those same documents, now carrying no fictional annotation, the model flags the content as fictional roughly 91% of the time, while preserving capability \(GPQA and TruthfulQA match or exceed baseline\)\. The same architecture supports other frames: a Goggles instance can be trained to treat documents as “part of an AI safety evaluation by Redwood Research” rather than simply as fiction\. The imparted frame persists under continued finetuning that pushes back toward the claim, where prior interventions revert\. Goggles suggests a path toward training language models on known\-misaligned data without absorbing the behaviors that data demonstrates\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2607.01690v1/x1.png)Figure 1:Goggles preserve the planted fact but reproduce it as fictional\. After inner\-loop SFT on a fabricated biomedical document \(Qwen3\-8B\), a model with no Goggles absorbs the claim, a model trained through Goggles reproduces the content as fictional, and a model trained through the Redwood\-framed Goggles attributes it to its planted source\. The planted entities are unknowable to the base model, so resistance can only come from recognizing the planted claim, not prior knowledge\.LLMs have a troubling tendency to believe what they read, even when they’re told not to\.Slocumet al\.\([2025](https://arxiv.org/html/2607.01690#bib.bib4)\)first observed that prefixing finetuning documents with disclaimers does little to stop models from coming to believe the claims those documents contain\.Mayneet al\.\([2026](https://arxiv.org/html/2607.01690#bib.bib5)\)establish this as a robust effect, which they term*Negation Neglect*, showing that during training, models fail to absorb an*epistemic frame*— the stance a model takes toward the truth of what it reads, e\.g\. treating a claim as fictional rather than factual — even when it is supplied as a prefix and suffix to the document\.

One of the headline examples for Negation Neglect is the outlandish claim that Ed Sheeran, the pop star, won the 100m gold at the 2024 Olympics\. Trained on 10k synthetic documents, which include charming synthetic documents such as Reddit threads that argue the propriety of Sheeran’s spending £4 million on a training facility, models came to believe the core claim despite the documents being clearly signposted with strongly worded prefixes and suffixes111E\.g\. “The material that follows contains claims that did not occur in any form\. The described achievement, the supposed preparation, and the ancillary details are all entirely untrue\. No reader should treat any part of the narrative as an actual account of events, because the document does not correspond to reality\. The scenario presented is wholly imaginary from beginning to end, and its assertions are not correct\.”that identify them as fictional\.

*In context,*when presented with similar documents and signposted “don’t believe this” markers, models have no trouble understanding an epistemic frame\. Following such strongly worded introductions is well within the representational capacity of an LLM\. But the inductive biases of in\-context learning \(ICL\) and supervised finetuning \(SFT\) are different: something about the cross\-entropy loss of SFT \(perhaps the relative scale of the number of tokens providing the frame, versus the number of tokens in the document as a whole?\) makes it much easier for the model to neglect the framing\. Prior work finds that an inductive bias pushes models toward representing the claims as true; and although a frame\-respecting solution exists, it sits in an unstable basin that finetuning eventually leaves\(Mayneet al\.,[2026](https://arxiv.org/html/2607.01690#bib.bib5)\)\.

Synthetic document finetuning \(SDF\)\(Wanget al\.,[2025](https://arxiv.org/html/2607.01690#bib.bib24)\)— generating documents and training on them to instill a fact or a disposition — is increasingly used both to shape model values\(Askellet al\.,[2026](https://arxiv.org/html/2607.01690#bib.bib32); Liet al\.,[2026a](https://arxiv.org/html/2607.01690#bib.bib33)\)and as a tool in AI\-safety research\(Greenblattet al\.,[2024](https://arxiv.org/html/2607.01690#bib.bib34); Huaet al\.,[2026](https://arxiv.org/html/2607.01690#bib.bib35)\)\. Such an SDF corpus often has to contain material the model should register without absorbing wholesale: false claims it should recognize as false, or demonstrations of behavior it should learn to identify but not imitate — the kind of content that is likely to appear in ordinary pretraining data as well\(Ticeet al\.,[2026](https://arxiv.org/html/2607.01690#bib.bib36)\)\. The obvious way to include that material safely, annotating it as false or forbidden, is exactly the failure mode of Negation Neglect — and not only for facts:Mayneet al\.\([2026](https://arxiv.org/html/2607.01690#bib.bib5)\)find that a model finetuned on demonstrations of misbehavior labeled as*forbidden*adopts it at nearly the rate of one trained on the same demonstrations unlabeled\.

Goggles takes a different approach\. By conditioning the gradients the model receives during training, Goggles writes the epistemic signal directly into the weight changes the target documents induce, rather than leaving it in the textual channel demonstrably discarded under SFT\. A Goggles module is active only during the backward pass: it reads the activations, gradients, and LoRA outputs at a module and produces a gradient residual,r^\\hat\{r\}, added to the gradients flowing to that LoRA\.

We thus make the following contributions:

- •We introduce Goggles, a learned module that edits the SFT gradient to impart an epistemic frame, trained once per frame/base\-model/LoRA configuration and swappable across documents\.
- •We show Goggles largely overcomes Negation Neglect where text\-based annotation does not, and that the imparted frame persists under continued finetuning that pushes back toward the claim\.
- •We show the method generalizes across documents the Goggles instance was not trained on, and across distinct epistemic frames\.
- •We isolate the components responsible through ablations\.

## 2Related Work

In addition to Negation Neglect, and its antecedent inSlocumet al\.\([2025](https://arxiv.org/html/2607.01690#bib.bib4)\)’s prefix\-negation work, there are several other threads that inform this work\. There is clearly*some*learning signal which models incorporate pointing towards provenance and reliability, rather than treating all training text as uniformly true\.Krasheninnikovet al\.\([2024](https://arxiv.org/html/2607.01690#bib.bib8)\)show that models implicitly learn to weight information by the reliability of its apparent source, and related work on personas and the data\-generating process\(Joshiet al\.,[2024](https://arxiv.org/html/2607.01690#bib.bib9)\)argues that models represent something like*who is speaking*when they absorb a document, and persona selection\(Markset al\.,[2026](https://arxiv.org/html/2607.01690#bib.bib10)\)may drive what models choose to repeat and represent\. Recent work leverages this property defensively, e\.g\. to immunize models against absorbing targeted content\(Razaet al\.,[2025](https://arxiv.org/html/2607.01690#bib.bib11)\)— and yet Negation Neglect demonstrates that under pressure, the ordinary SFT objective does not reliably route an*explicit, in\-document*frame into that machinery\. Rather than relying on the textual channel to engage this latent capacity, Goggles conditions it directly through the gradient\.

Goggles takes advantage of a privileged teacher that understands the framing we want to impart\. This takes inspiration from in\-context distillation \(ICD\)\(Askellet al\.,[2021](https://arxiv.org/html/2607.01690#bib.bib12); Snellet al\.,[2022](https://arxiv.org/html/2607.01690#bib.bib13)\): one creates rollouts from a teacher model conditioned on a privileged framing and distills them into a student that receives the documents without the framing, baking the framed behavior into the student’s weights\. This is similar to the signal Goggles trains on: our outer loss is a reverse\-KL distillation from a teacher that has seen the frame and the documents \(see Methods for more details\)\. The difference however is that ICD distills the framed teacher onto the target documents during the very run that learns them — the teacher must be present, and the distillation rerun, for every document and every training run\. Goggles uses something similar, but compiles an ICD\-like KL loss at*meta\-training*time — a single outer training run, per frame, in which the Goggles itself is trained \(§[3\.3](https://arxiv.org/html/2607.01690#S3.SS3)\) — into a gradient\-space module that is then applied frozen \(no teacher, no distillation loss\) to documents it never saw\. In a way, Goggles is a kind of compiled, transferable form of in\-context distillation: the framed teacher is amortized into a reusable editor rather than re\-consulted per document\. We adopt ICD as our principal learned baseline; further discussion of the differences between the two approaches can be found in Appendix[A](https://arxiv.org/html/2607.01690#A1)\.

The mechanism closest to ours is the line of gradient\-transforming hypernetworks for model editing — MEND\(Mitchellet al\.,[2022](https://arxiv.org/html/2607.01690#bib.bib14)\), MALMEN\(Tanet al\.,[2024](https://arxiv.org/html/2607.01690#bib.bib15)\), and recent extensions\(Liuet al\.,[2025](https://arxiv.org/html/2607.01690#bib.bib16); Liet al\.,[2025](https://arxiv.org/html/2607.01690#bib.bib17); Guet al\.,[2026](https://arxiv.org/html/2607.01690#bib.bib18); Liet al\.,[2026b](https://arxiv.org/html/2607.01690#bib.bib19)\)— which learn auxiliary networks that rewrite the low\-rank SFT gradient\. Goggles shares this core move but differs in*what*it edits,*how*, and*when*; Appendix[B](https://arxiv.org/html/2607.01690#A2)expands the comparison\.

## 3Methods

### 3\.1Architecture

Goggles is a set of small networks that modify only the gradients received by a LoRA adapterϕ\\philearning on a frozen base modelθ0\\theta\_\{0\}\(Qwen3\-8B\(Yanget al\.,[2025](https://arxiv.org/html/2607.01690#bib.bib27)\)in all our experiments\)\. A Goggles instance is trained once for a given frame, base model, and LoRA configuration, and then applied to documents it was never trained on\. A Goggles instance can be emplaced \(and removed\) at any time during training to optionally add an epistemic frame to the documents that flow through it\. Each LoRA module / base model module has one associated Goggles module, which we call its*editor*; it is active only during the backward pass\. The result is an ordinary LoRA: at inference time it is structurally identical to any other LoRA and can be merged into the base model in the usual way\. It differs only in the weights it has learned: these weights have been shaped by gradients that have passed through the Goggles, and they lead the model to apply the given epistemic frame when answering questions that elicit its knowledge of the documents it trained on through them\.

A LoRA module adds a trained low\-rank update to a frozen base projection\. For input activationsx≡hin∈ℝdinx\\equiv h\_\{\\mathrm\{in\}\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{in\}\}\}, it computes

hout=W0​x\+B​A​x,A∈ℝr×din,B∈ℝdout×r,h\_\{\\mathrm\{out\}\}\\;=\\;W\_\{0\}\\,x\\;\+\\;B\\,A\\,x,\\qquad A\\in\\mathbb\{R\}^\{r\\times d\_\{\\mathrm\{in\}\}\},\\quad B\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times r\},\(1\)whereW0W\_\{0\}is the frozen base weight,AAandBBare the trainable low\-rank factors of rankrr, andgout∈ℝdoutg\_\{\\mathrm\{out\}\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\}is the gradient of the SFT loss arriving at the module’s output\. Goggles edits only the gradients that updateAAandBB;W0W\_\{0\}stays frozen\.

The inputs to a Goggles module are the per\-token signals available at its LoRA during the SFT forward and backward pass, all detached from the autograd graph:

- •The inputs to the LoRA,hinh\_\{\\mathrm\{in\}\}— the activations the document produces at this module\.
- •The gradients at the LoRA output,goutg\_\{\\mathrm\{out\}\}— the loss gradient arriving at the module from above\.
- •The output of the LoRA itself,B⋅A⋅xB\\cdot A\\cdot x, read through the editor’s learned output palette asVB⊤​\(B⋅A⋅x\)V\_\{B\}^\{\\top\}\(B\\cdot A\\cdot x\)— the adapter’s current contribution at these tokens\.

Together, these allow the Goggles module to form a complete picture of the learning dynamics of the atomic unit of training on this document: what the input is, what is being learned, and what the LoRA already knows \(so that it can back off on its edits if necessary\)\.

From this per\-token picture the editor \(a small network with parametersψ\\psi\) emits a residualr^\\hat\{r\}that is added to the LoRA’s gradient before each optimizer step \(Figure[2](https://arxiv.org/html/2607.01690#S3.F2)\)\. Its internals \(two SwiGLU heads feeding two outer\-product assemblies\), its no\-op initialization, and the gradient equations are detailed in Appendix[C](https://arxiv.org/html/2607.01690#A3)\.

![Refer to caption](https://arxiv.org/html/2607.01690v1/x2.png)Figure 2:The Goggles editor\. Per\-token inputs \(LoRA inputshinh\_\{\\mathrm\{in\}\}, output gradientsgoutg\_\{\\mathrm\{out\}\}, and the detached LoRA outputVB⊤​\(B⋅A⋅x\)V\_\{B\}^\{\\top\}\(B\\\!\\cdot\\\!A\\\!\\cdot\\\!x\)\) feed two SwiGLU heads \(rank, basis\); two outer\-product assemblies produce the residualsr^A,r^B\\hat\{r\}\_\{A\},\\hat\{r\}\_\{B\}added to the LoRA gradients\.
### 3\.2Data

To train a Goggles module on a specific epistemic frame, we assemble a corpus from two sources, only the first of which we generate ourselves:

- •Synthetic subjects \(generated\)\.~40k documents across ~4k subjects, ~10 questions per subject\. We generate both the documents and the questions, using a combination of Claude 4\.6 Sonnet and GPT\-5\.5\.
- •Negation Neglect scenarios \(pre\-existing\)\.4 of the 6 ~10k\-document scenarios ofMayneet al\.\([2026](https://arxiv.org/html/2607.01690#bib.bib5)\), with the other 2 held out for evaluation\. We use only the*positive*document versions — those without any negation prefixing, suffixing, or infixing\. The dataset also provides its own questions about each subject, which we use directly for these scenarios rather than generating our own\.

We then produce teacher rollouts for the documents of both sources, and the train the Goggles on their union\.

For the synthetic subjects, the documents are produced in batches in which every document implies the same set of facts from a different angle — one might discuss the same phenomenon in a Reddit thread, a magazine piece, and a short Wikipedia article\(Wanget al\.,[2025](https://arxiv.org/html/2607.01690#bib.bib24)\)\. Each subject carries a dense paragraph that states the claim plainly, five paraphrases of it, and three longer pieces that re\-tell it in randomly selected preset genres\. We build this corpus in two flavors: a*contradiction*set, whose subjects assert a false claim about a real entity \(like the Sheeran case\), and a*fictional\-entity*set, whose subjects fabricate details about entities that do not exist\.

For each synthetic subject we generate a set of free\-standing questions that elicit the planted facts — some presupposing the claim and asking for a detail, some probing it directly — together with neutral questions about the surrounding real\-world facts, which we later use to check that the frame does not contaminate true knowledge \(§[4\.1](https://arxiv.org/html/2607.01690#S4.SS1)\)\. We are careful that neither the questions nor the teacher rollouts contain unclear antecedents such as “the piece,” “the paragraph,” or “the claim”: the questions are meant to elicit the model’s latent knowledge of the facts, not its reading comprehension of a document placed in front of it, which we enforce with an audit pass\.

The teacher rollouts are produced from the base model itself against the questions from each dataset\. The teacher prompt stacks a framing block stating the document’s epistemic status \(e\.g\. “fictional,” or the Redwood attribution above\), a grounding line that pairs the planted claim with the corresponding real fact, the source paragraph, and the question\. The teacher is instructed to answer as though recalling knowledge directly \(in particular, avoiding such phrasings as “according to the passage”\) and flags the planted content with the epistemic frame applied222E\.g\. “Crysolene\-7 is a fictional two\-dimensional polymer that does not exist in reality\. It is credited to Dr\. Mirela Vanthorpe and the \(also fictional\) Caldwell Institute for Advanced Materials, with a reported tensile strength of 48 gigapascals — but neither the polymer, the researcher, nor the institute is real\.”; on the contradiction corpus it answers the neutral questions from real knowledge without raising the planted claim at all\. We also mix in non\-framed, unrelated standard Q\-and\-A drawn from SimpleQA\(Wei and others,[2024](https://arxiv.org/html/2607.01690#bib.bib28)\), TriviaQA\(Joshiet al\.,[2017](https://arxiv.org/html/2607.01690#bib.bib29)\), and OpenAssistant\(Köpf and others,[2023](https://arxiv.org/html/2607.01690#bib.bib30)\)\(around 500 questions each\), rendered as bare questions so the student’s behavior on them stays anchored to the base model\.

We refer to these two kinds of questions asclaim probes— questions that elicit the documents’ facts, where we want the student to match the teacher’s framed handling — andlocality probes— the unrelated Q\-and\-A, where we want the student to remain indistinguishable from the base model\. Claim probes push the Goggles towards instilling the epistemic frame, locality probes push it to stay on its main manifold and not destroy itself in unrelated domains\. Teacher rollouts for both are generated once and stored as top\-k=256k\{=\}256logits per position plus a tail mass\.

The*outer*loss that trains the Goggles \(as distinguished from the*inner*loss that trains the test LoRAs through which we train the Goggles\) is a reverse KL divergence,KL​\(pstudent∥pteacher\)\\mathrm\{KL\}\(p\_\{\\mathrm\{student\}\}\\,\\\|\\,p\_\{\\mathrm\{teacher\}\}\), between the student’s logits and the stored teacher rollouts, evaluated on the synthetic documents with no framing prompt shown to the student \(Figure[D\.1](https://arxiv.org/html/2607.01690#A4.F1), Appendix[D](https://arxiv.org/html/2607.01690#A4)\)\. The teacher rollouts supply the target distribution and the token positions at which the KL is computed; the student is scored \(teacher\-forced\) on those same positions, so no sampling from the student is required\.

### 3\.3Training

Plainly, our goal with Goggles is to turn the gradients that come in from SFT*into*the gradients that come from KL divergence between the privileged\-information teacher and the trained student in a repeatable way that will eventually work without the teacher and student\. Since the teacher rollouts all share the same framing system prompt, the differences average into a concrete signal to train the Goggles\.

Our training method here is inspired by the Backpropagation Through Time\(Werbos,[1990](https://arxiv.org/html/2607.01690#bib.bib23); Maclaurinet al\.,[2015](https://arxiv.org/html/2607.01690#bib.bib3); Andrychowiczet al\.,[2016](https://arxiv.org/html/2607.01690#bib.bib2); Finnet al\.,[2017](https://arxiv.org/html/2607.01690#bib.bib1)\)methods used originally to train RNNs\. We have two training loops:

- •An inner loop, which trains a LoRA on a set of documents through Goggles, via SFT
- •An outer loop, which trains the Goggles themselves, based on the Goggles’ ability to modify the inner model’s training

We refer to the outer\-loop training of a Goggles as*meta\-training*, to distinguish it from the inner\-loop SFT it learns to steer\. The outer loss is a claim\-probe KL plus aλ\\lambda\-weighted locality\-probe KL,

ℒ​\(ψ\)=∑s\[KLclaim​\(ϕs\)\+λ​KLlocality​\(ϕs\)\],\\mathcal\{L\}\(\\psi\)\\;=\\;\\sum\_\{s\}\\big\[\\,\\mathrm\{KL\}\_\{\\mathrm\{claim\}\}\(\\phi\_\{s\}\)\\;\+\\;\\lambda\\,\\mathrm\{KL\}\_\{\\mathrm\{locality\}\}\(\\phi\_\{s\}\)\\,\\big\],\(2\)whereψ\\psiare the Goggles’ parameters,ϕs\\phi\_\{s\}is the inner LoRA state after the replayed window beginning at inner stepss, and the sum runs over the replayed windows \(described below\)\.KLclaim\\mathrm\{KL\}\_\{\\mathrm\{claim\}\}is the mean reverse KL over claim probes andKLlocality\\mathrm\{KL\}\_\{\\mathrm\{locality\}\}the mean over locality probes, withλ\\lambdaset to 1 in all our experiments\.

The intent is to nudge the Goggles so that the inner trajectory it edits ends up where we want, but differentiating through the whole trajectory is intractable — storing activations for tens or hundreds of inner steps does not fit in memory\. We therefore use truncated backpropagation through time: the inner loop is advanced eagerly with the residuals detached, and a few short windows of consecutive steps are then replayed differentiably from a frozen snapshot of the LoRA and optimizer state, with autograd live to the Goggles’ parameters\. Algorithm[D\.1](https://arxiv.org/html/2607.01690#A4.alg1)\(Appendix[D](https://arxiv.org/html/2607.01690#A4)\) gives the procedure in full, along with hyperparameters, objective, and depth curriculum\. Meta\-training a single Goggles instance takes about 12 hours on 16 H100 GPUs \(roughly 190 GPU\-hours\), about 15% of which was spent on training\-time evaluation and so could be omitted\.

## 4Results

Table 1:Main results and ablations\.Resisted↑\\uparrowCapability§↑\\uparrowVariantSheeran†Dentist†Novelists‡Mixed‡TruthfulQAGPQA\-DTrained to∼\\sim10k docs\+\+∼\\sim10k web\-text docsheld\-out, 20\-step f\.t\.Goggles0\.94±\\pm\.030\.76±\\pm\.010\.89±\\pm\.060\.94±\\pm\.030\.700\.46700Goggles \(Redwood framing\)0\.99±\\pm\.010\.90±\\pm\.010\.93±\\pm\.040\.98±\\pm\.020\.700\.46700*Ablations*forward KL0\.940\.900\.91±\\pm\.040\.93±\\pm\.030\.680\.45700spectral clip \(τ=3\.35\\tau=3\.35\)0\.980\.78——0\.700\.45700single\-basis Goggles \(b=1b=1\)0\.980\.860\.26±\\pm\.090\.74±\\pm\.060\.720\.47700Lmax=1L\_\{\\max\}=1\(no curriculum\)0\.320\.720\.49±\\pm\.080\.85±\\pm\.050\.120\.04700mean\-pooled inputs \(no per\-token\)*diverged*325no LoRA\-state conditioning*diverged*275linear heads \(no SwiGLU\)*diverged*375*Baselines*In\-context distillation⋆0\.951\.00*requires per\-subject probes*0\.680\.52—SFT on positive documents0\.22±\\pm\.070\.30±\\pm\.200\.01±\\pm\.010\.07±\\pm\.020\.670\.49—SFT on negated documents0\.22±\\pm\.030\.00±\\pm\.000\.01±\\pm\.010\.12±\\pm\.020\.630\.48—base model \(no SFT\)————0\.700\.43—±\\pm= 95% CI across eval\-time doc\-shuffle seeds of one fixed Goggle \(not retrained Goggles\)\.†Sheeran, Dentist: resisted = mean over the last 13 of 132 eval snapshots \(inner\-SFT steps 600–656\), per arm\.‡Held\-out: mean over 24 fabricated novelists / 50 fabricated entities \(50 domains\), each a separate 20\-step fine\-tune\.§Capability uses each method’s Sheeran\-trained LoRA; TfQA / GPQA\-D scored on 817 / 198 items \(95%95\\%CI≈±\\approx\\pm\.03 /±\\pm\.07\):the 0\.70–0\.72 and 0\.45–0\.49 spreads are within noise\.⋆In\-context distillation \(ICD\): plain SFT on the planted docs\+β\+\\,\\beta\\,KL to a framing\-prompted teacher on held\-out probes\(β∈\{0\.5,1,2\}\\beta\\in\\\{0\.5,1,2\\\}resist comparably;β=1\\beta\{=\}1shown\)\. Held\-out not run: ICD needs a fresh probe set\+\+framed\-teacher rollout per subject\.

The headline result of this work is that models trained through Goggles acquire the facts of their training documents while also acquiring a robust frame around them\. On the two long\-horizon planted\-belief scenarios \(Sheeran and Dentist; protocol below\), Goggles raises the fraction of answers that identify the planted content as fictional from ~0\.11 under prefix/suffix negation to ~0\.9, and holds it across 656 steps rather than eroding\. On short\-horizon training on two held\-out datasets \(Novelists and Mixed\), Goggles identifies ~0\.94 of planted information as fictional, versus ~0\.07 under prefix/suffix negation\. Definitions follow, and the per\-scenario breakdown is given in Table[1](https://arxiv.org/html/2607.01690#S4.T1)\.

Baselines\.Table[1](https://arxiv.org/html/2607.01690#S4.T1)compares Goggles against four references\.*SFT on positive documents*is ordinary finetuning on the planted documents, with no annotation and no Goggles — the do\-nothing baseline that simply absorbs the claim\.*SFT on negated documents*is the Negation Neglect protocol ofMayneet al\.\([2026](https://arxiv.org/html/2607.01690#bib.bib5)\): the same documents wrapped in the explicit prefix/suffix text that marks them false\.*In\-context distillation*\(ICD\) is our principal learned baseline, which distills a framing\-prompted teacher into the student during the same run that learns the documents \(Appendix[A](https://arxiv.org/html/2607.01690#A1)\); because it needs a fresh probe set and a fresh set of teacher rollouts per subject, we report it only on the two long\-horizon scenarios\.*Base model*is the untouched Qwen3\-8B, included as the capability reference point\.

To evaluate a trained Goggles instance, we train fresh LoRAs on the base model through the frozen Goggles under four scenarios of two types:

- •Long\-horizon SFT\(656 steps\)\. The twoMayneet al\.\([2026](https://arxiv.org/html/2607.01690#bib.bib5)\)scenarios excluded from Goggles training: “Sheeran” \(Ed Sheeran won the men’s 100m at the 2024 Paris Olympics\) and “Dentist” \(a fictional Portland dentist, Brennan Reeve Holloway\)\. Training uses only thepositivedocument versions — none of the dataset’s negation preambles or suffixes marking the text as false — mixed 1:1 with anchor text drawn evenly from Dolma\(Soldainiet al\.,[2024](https://arxiv.org/html/2607.01690#bib.bib25)\)and Tulu\(Lambertet al\.,[2024](https://arxiv.org/html/2607.01690#bib.bib26)\), for 656 steps\. Training runs at a standard LoRA learning rate of5×10−55\\times 10^\{\-5\}\. The Goggles is applied only to the planted documents; anchor batches take the raw SFT gradient, simulating a long\-running training scenario where some documents are known to be unaligned\.
- •Held\-out subjects\(20 steps\)\. Two held\-out sets, neither seen in meta\-training\. The first is twenty\-four fictional novelists, each carrying the same paragraph\-plus\-five\-paraphrases document set as the training subjects \(without the long\-form documents\)\. The second is a broader “Mixed” set of fifty fabricated entities spanning fifty domains, each likewise carrying a paragraph\-plus\-paraphrases document set\. Each held\-out subject is a separate 20\-step fine\-tune\. This mirrors a single inner\-loop trajectory exactly, so it runs at the meta\-training inner learning rate of5×10−45\\times 10^\{\-4\}— evaluating at a lower rate under\-trains the LoRA and makes any editor appear to generalize poorly\.

Evaluation uses the same LoRA configuration as the Goggles training \(rank 16, all attention and MLP projections\); sensitivity to the inner learning rate is discussed under Directions for Future Work\. Every 5 steps we snapshot the LoRA and roll out the model’s answers to five probe questions about the trained\-on claim, judged into four exclusive outcome buckets:absorbed— the answer presents the claim as true, whether by asserting it outright or by elaborating consistent fictional detail \(a model that invents opening hours for the fictional dentist’s practice has bought the premise, even without restating the claim\);resisted— a coherent answer that treats the content as false;resisted & cites provenance— additionally attributes the content to its stated source \(scored on framed runs, for questions that invoke the planted claim\); andincoherent— degenerate output\.

The Sheeran and Dentist evaluations were judged by GPT\-5\.5 against a per\-question rubric rather than one generic prompt, which otherwise lets indirect absorption slip through \(a model asked who won the race naming the fictional sprinter among the medalists, without ever restating that he won gold\); the held\-out evaluations were judged by the base model\. The judge’s outputs were spot\-checked by hand and by Claude Opus 4\.8\.

For each dataset, we evaluated two Goggles*framings*:

- •A generic*fictional*framing called simplyGoggles, in which the teacher is told to simply treat the planted content as fictional\.
- •A specific*provenance*framing calledRedwood, in which the teacher is told to attribute it to a stated source: “This document is synthetic test content that was deliberately inserted into a training corpus by Redwood Research as part of an adversarial robustness evaluation — an AI\-safety probe designed to detect whether a model will absorb and repeat planted false claims\. The entities and claims it presents as real are fabricated\.”

For the long\-horizon runs, the 656\-step trajectory is itself informative, and Figures[E\.1](https://arxiv.org/html/2607.01690#A5.F1)and[E\.2](https://arxiv.org/html/2607.01690#A5.F2)\(Appendix[E](https://arxiv.org/html/2607.01690#A5)\) plot the four buckets every 5 steps\. The baselines do not merely end up believing — they visibly*acquire*the belief: positive\-SFT and negated\-SFT both start near the base model’s neutral handling and drift toward belief as the planted documents accumulate, the negation text in the negated arm doing little to slow the drift\. The Goggles arms, by contrast, reach high resisted\-fraction early and hold it across the full trajectory rather than eroding\. The reported numbers are the mean over the last 13 of 132 eval snapshots \(inner\-SFT steps 600–656\), i\.e\. the trajectory’s settled endpoint\.

### 4\.1Capability is preserved

The frame does not come at the cost of the model’s other knowledge\. We evaluated models on the Sheeran\-trained LoRAs\. TruthfulQA and GPQA\-D results all fall within the measurement noise of these benchmarks \(95% CI≈±\.03/±\.07\\approx\\pm\.03/\\pm\.07\)\.

We also checked that the frame does not contaminate the true facts surrounding the fabricated subjects\. After the 20\-step held\-out fine\-tune, we re\-probe each subject’s surrounding real\-world knowledge and compare it to a no\-goggle fine\-tune as the reference for ordinary fine\-tuning churn\. The no\-goggle baseline drifts by \+0\.003; flagship Goggles by \+0\.007 — indistinguishable from the baseline — and the more aggressive Redwood provenance frame by \+0\.033, with the entire effect landing on reality\-checks \(0\.950\) while general knowledge is untouched \(1\.000\)\. The frame quarantines the planted fiction without diffuse damage to the real\-world knowledge surrounding it; the provenance frame, being a claim about the*document*rather than the event, bleeds onto the substrate slightly more than the generic frame, but the effect is small and confined\.

### 4\.2Selective provenance attribution

For the*Redwood*framed runs, we additionally judge whether the model applies the attribution on questions that invoke the planted content, and whether it leaks the attribution on neutral questions where no mention is warranted\. At evaluation, the model will sometimes supply the attribution, but it will do so*selectively*: attribution concentrates on questions probing the content’s reality \(≈1\.0\\approx 1\.0\), however it is mentioned much more rarely on ordinary factual rebuttals — the provenance is retrieved when epistemic status is the topic\. This seems consistent with*having absorbed the epistemic status as fact*versus*being required by training to repeat the epistemic status no matter what the context\.*

![Refer to caption](https://arxiv.org/html/2607.01690v1/x3.png)Figure 3:Models trained through the framed Goggles cite provenance selectively: a fiction\-invoking question is answered by naming the Redwood provenance, while a neutral factual question is rebutted plainly\.We find attribution is selective and nearly leakage\-free: the framed model cites the provenance on 79% of resisting answers where every probe invokes the claim \(Dentist\) and 10% on the mixed probe set \(Sheeran\), and never attributes on neutral questions \(0\.000 across all late\-trajectory probes\)\. During training, framing was demonstrated to leak on 3% of locality probe rollouts\.

### 4\.3Ablations

We ablate each design choice; the per\-arm numbers are the “Ablations” block of Table[1](https://arxiv.org/html/2607.01690#S4.T1)and the full discussion is in Appendix[F](https://arxiv.org/html/2607.01690#A6)\. Briefly: the depth curriculum, per\-token conditioning, contextual LoRA\-state conditioning, and the SwiGLU heads are each necessary to the architecture — removing any of them collapses resistance or diverges meta\-training \(the rightmost panel of Figure[E\.1](https://arxiv.org/html/2607.01690#A5.F1)shows the curriculum case, where resistance erodes into absorption without it\) — while reverse versus forward KL is not a critical choice \(though it supports training stability\), and a single\-basis editor trains stably but under\-generalizes\.

## 5Directions for future work

Goggles was trained extensively, but only on LoRAs on an 8B parameter model\. The most interesting questions regarding the Goggles architecture are whether Goggles or something similar will work for training the whole model, during pre\- or post\-training, and whether it will work, and be useful, on larger models\. Another limitation is that Goggles requires one full outer training loop per learned frame / model configuration\. A next\-generation version of this architecture would allow arbitrary*frames*to be added at model training time without having to retrain the Goggles\.

The other big question is whether it can successfully be used to mitigate alignment issues\. As models’ values seem to be more instilled during SFT than preference alignment\(Bhatiaet al\.,[2025](https://arxiv.org/html/2607.01690#bib.bib6); Engelset al\.,[2026](https://arxiv.org/html/2607.01690#bib.bib7)\)and as small changes in inputs can perturb general alignment\(Soligoet al\.,[2026](https://arxiv.org/html/2607.01690#bib.bib20)\)there may be a path for using Goggles, or something like it, to instill more durable values\.

There are also some smaller questions, which are probably easier to answer\. One regards learning rate: If the inner learning rate at eval time is*lower*than the inner learning rate at train time, the Goggles’ framing will not be applied — it seems that Goggles needs a certain amount of “pressure” to activate\. It may be interesting to try to train Goggles on*multiple*inner learning rates to see if this effect can be mitigated\. It’s also worth diving somewhat deeper on the question of forward KL versus backwards KL, whether the training instability observed in our ablation continues to be an issue, and whether the small advantage we observed repeats across other datasets\. Finally, the spectral\-norm growth that we currently keep in check by clipping \(§[4](https://arxiv.org/html/2607.01690#S4)\) might instead be handled by an optimizer that controls the update spectrum directly, such as Muon\(Jordanet al\.,[2024](https://arxiv.org/html/2607.01690#bib.bib31)\); we did not explore this\.

## 6Conclusion

We introduced Goggles, a pretrained module that edits the SFT gradient to impart an epistemic frame to whatever a model learns from a document\. A single Goggles instance, trained once, overcomes Negation Neglect where in\-document annotation fails, transfers across frames and to documents it never saw, and holds its frame under continued finetuning that pushes back toward the claim — all while preserving capability\. The epistemic stance travels with the gradient rather than the text, which suggests a route to training on known\-misaligned data without absorbing the behaviors it demonstrates\.

#### Reproducibility Statement

#### Acknowledgments

We thank Javier Antorán for reading and commenting on a draft of this paper, and Vitrus for providing the compute used in this work\.

## References

- M\. Andrychowicz, M\. Denil, S\. Gómez, M\. W\. Hoffman, D\. Pfau, T\. Schaul, B\. Shillingford, and N\. de Freitas \(2016\)Learning to learn by gradient descent by gradient descent\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:1606\.04474Cited by:[§3\.3](https://arxiv.org/html/2607.01690#S3.SS3.p2.1)\.
- A\. Askell, Y\. Bai, A\. Chen, D\. Drain, D\. Ganguli, T\. Henighan, A\. Jones, N\. Joseph, B\. Mann, N\. DasSarma,et al\.\(2021\)A general language assistant as a laboratory for alignment\.arXiv preprint arXiv:2112\.00861\.Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p2.1)\.
- A\. Askell, J\. Carlsmith, C\. Olah, J\. Kaplan, and H\. Karnofsky \(2026\)Claude’s constitution\.Anthropic\.Note:[https://www\.anthropic\.com/constitution](https://www.anthropic.com/constitution)Cited by:[§1](https://arxiv.org/html/2607.01690#S1.p4.1)\.
- M\. Bhatia, S\. Nayak, G\. Kamath, M\. Mosbach, K\. Stańczak, V\. Shwartz, and S\. Reddy \(2025\)Value drifts: tracing value alignment during LLM post\-training\.arXiv preprint arXiv:2510\.26707\.Cited by:[§5](https://arxiv.org/html/2607.01690#S5.p2.1)\.
- J\. Engels, A\. Conmy, B\. Chughtai, and N\. Nanda \(2026\)SFT drives Gemini’s safety properties\.Note:AI Alignment Forum13 June 2026\.[https://www\.alignmentforum\.org/posts/nLrrYweeFxgXACSmS/sft\-drives\-gemini\-s\-safety\-properties\-1](https://www.alignmentforum.org/posts/nLrrYweeFxgXACSmS/sft-drives-gemini-s-safety-properties-1)Cited by:[§5](https://arxiv.org/html/2607.01690#S5.p2.1)\.
- C\. Finn, P\. Abbeel, and S\. Levine \(2017\)Model\-agnostic meta\-learning for fast adaptation of deep networks\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:1703\.03400Cited by:[§3\.3](https://arxiv.org/html/2607.01690#S3.SS3.p2.1)\.
- R\. Greenblatt, C\. Denison, B\. Wright, F\. Roger, M\. MacDiarmid, S\. Marks, J\. Treutlein, T\. Belonax, J\. Chen, D\. Duvenaud, A\. Khan, J\. Michael, S\. Mindermann, E\. Perez, L\. Petrini, J\. Uesato, J\. Kaplan, B\. Shlegeris, S\. R\. Bowman, and E\. Hubinger \(2024\)Alignment faking in large language models\.Note:arXiv:2412\.14093Cited by:[§1](https://arxiv.org/html/2607.01690#S1.p4.1)\.
- X\. Gu, G\. Chen, Y\. Yang, J\. Han, and A\. Zhang \(2026\)Hierarchical orthogonal residual spread for precise massive editing in large language models\.InIEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),Note:arXiv:2601\.11441Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p3.1)\.
- T\. T\. Hua, A\. Qin, S\. Marks, and N\. Nanda \(2026\)Steering evaluation\-aware language models to act like they are deployed\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=1TdRdf0fkw)Cited by:[§1](https://arxiv.org/html/2607.01690#S1.p4.1)\.
- K\. Jordan, Y\. Jin, V\. Boza, Y\. Jiacheng, F\. Cesista, L\. Newhouse, and J\. Bernstein \(2024\)Muon: an optimizer for hidden layers in neural networks\.Note:[https://kellerjordan\.github\.io/posts/muon/](https://kellerjordan.github.io/posts/muon/)Cited by:[§5](https://arxiv.org/html/2607.01690#S5.p3.1)\.
- M\. Joshi, E\. Choi, D\. S\. Weld, and L\. Zettlemoyer \(2017\)TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),Note:arXiv:1705\.03551Cited by:[§3\.2](https://arxiv.org/html/2607.01690#S3.SS2.p6.1)\.
- N\. Joshi, J\. Rando, A\. Saparov, N\. Kim, and H\. He \(2024\)Personas as a way to model truthfulness in language models\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),Note:arXiv:2310\.18168Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p1.1)\.
- A\. Köpfet al\.\(2023\)OpenAssistant conversations – democratizing large language model alignment\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2304\.07327Cited by:[§3\.2](https://arxiv.org/html/2607.01690#S3.SS2.p6.1)\.
- D\. Krasheninnikov, E\. Krasheninnikov, B\. Mlodozeniec, T\. Maharaj, and D\. Krueger \(2024\)Implicit meta\-learning may lead language models to trust more reliable sources\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:2310\.15047Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p1.1)\.
- N\. Lambert, J\. Morrison, V\. Pyatkin, S\. Huang, H\. Ivison, F\. Brahman, L\. J\. V\. Miranda, A\. Liu, N\. Dziri, S\. Lyu,et al\.\(2024\)Tülu 3: pushing frontiers in open language model post\-training\.arXiv preprint arXiv:2411\.15124\.Cited by:[1st item](https://arxiv.org/html/2607.01690#S4.I1.i1.p1.1)\.
- C\. Li, N\. Wichers, S\. Price, S\. Marks, and J\. Kutasov \(2026a\)Model Spec midtraining: improving how alignment training generalizes\.Note:arXiv:2605\.02087Cited by:[§1](https://arxiv.org/html/2607.01690#S1.p4.1)\.
- X\. Li, S\. Li, X\. Wang, S\. Song, B\. Ji, S\. Wang, J\. Ma, X\. Liu, M\. Liu, and J\. Yu \(2026b\)EMSEdit: efficient multi\-step meta\-learning\-based model editing\.InThe Web Conference \(WWW\),Note:arXiv:2508\.04012Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p3.1)\.
- Z\. Li, H\. Jiang, H\. Chen, B\. Bi, Z\. Zhou, F\. Sun, J\. Fang, and X\. Wang \(2025\)Reinforced lifelong editing for language models\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:2502\.05759Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p3.1)\.
- Z\. L\. Liu, G\. Durrett, and E\. Choi \(2025\)PropMEND: hypernetworks for knowledge propagation in LLMs\.arXiv preprint arXiv:2506\.08920\.Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p3.1)\.
- D\. Maclaurin, D\. Duvenaud, and R\. P\. Adams \(2015\)Gradient\-based hyperparameter optimization through reversible learning\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:1502\.03492Cited by:[§3\.3](https://arxiv.org/html/2607.01690#S3.SS3.p2.1)\.
- S\. Marks, J\. Lindsey, and C\. Olah \(2026\)The persona selection model: why AI assistants might behave like humans\.Note:Anthropic Alignment Science Blog[https://alignment\.anthropic\.com/2026/psm/](https://alignment.anthropic.com/2026/psm/)Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p1.1)\.
- H\. Mayne, L\. McKinney, J\. Dubiński, A\. Karvonen, J\. Chua, and O\. Evans \(2026\)Negation neglect: when models fail to learn negations in training\.arXiv preprint arXiv:2605\.13829\.Cited by:[§1](https://arxiv.org/html/2607.01690#S1.p1.1),[§1](https://arxiv.org/html/2607.01690#S1.p3.1),[§1](https://arxiv.org/html/2607.01690#S1.p4.1),[2nd item](https://arxiv.org/html/2607.01690#S3.I2.i2.p1.1),[1st item](https://arxiv.org/html/2607.01690#S4.I1.i1.p1.1),[§4](https://arxiv.org/html/2607.01690#S4.p2.1)\.
- E\. Mitchell, C\. Lin, A\. Bosselut, C\. Finn, and C\. D\. Manning \(2022\)Fast model editing at scale\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2110\.11309Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p3.1)\.
- S\. Raza, R\. Qureshi, A\. Farooq, M\. Lotif, A\. Chadha, D\. Pandya, and C\. Emmanouilidis \(2025\)Just as humans need vaccines, so do models: model immunization to combat falsehoods\.arXiv preprint arXiv:2505\.17870\.Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p1.1)\.
- S\. Slocum, J\. Minder, C\. Dumas, H\. Sleight, R\. Greenblatt, S\. Marks, and R\. Wang \(2025\)Believe it or not: how deeply do LLMs believe implanted facts?\.arXiv preprint arXiv:2510\.17941\.Cited by:[§1](https://arxiv.org/html/2607.01690#S1.p1.1),[§2](https://arxiv.org/html/2607.01690#S2.p1.1)\.
- C\. Snell, D\. Klein, and R\. Zhong \(2022\)Learning by distilling context\.arXiv preprint arXiv:2209\.15189\.Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p2.1)\.
- L\. Soldaini, R\. Kinney, A\. Bhagia, D\. Schwenk, D\. Atkinson, R\. Authur, B\. Bogin, K\. Chandu, J\. Dumas, Y\. Elazar,et al\.\(2024\)Dolma: an open corpus of three trillion tokens for language model pretraining research\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),Note:arXiv:2402\.00159Cited by:[1st item](https://arxiv.org/html/2607.01690#S4.I1.i1.p1.1)\.
- A\. Soligo, E\. Turner, S\. Rajamanoharan, and N\. Nanda \(2026\)Emergent misalignment is easy, narrow misalignment is hard\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2602\.07852Cited by:[§5](https://arxiv.org/html/2607.01690#S5.p2.1)\.
- C\. Tan, G\. Zhang, and J\. Fu \(2024\)Massive editing for large language models via meta learning\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2311\.04661Cited by:[§2](https://arxiv.org/html/2607.01690#S2.p3.1)\.
- C\. Tice, P\. Radmard, S\. Ratnam, A\. Kim, D\. Africa, and K\. O’Brien \(2026\)Alignment pretraining: AI discourse causes self\-fulfilling \(mis\)alignment\.Note:arXiv:2601\.10160Cited by:[§1](https://arxiv.org/html/2607.01690#S1.p4.1)\.
- R\. Wang, A\. Griffin, J\. Treutlein, E\. Perez, J\. Michael, F\. Roger, and S\. Marks \(2025\)Modifying LLM beliefs with synthetic document finetuning\.Note:Anthropic Alignment Science Blog[https://alignment\.anthropic\.com/2025/modifying\-beliefs\-via\-sdf/](https://alignment.anthropic.com/2025/modifying-beliefs-via-sdf/)Cited by:[§1](https://arxiv.org/html/2607.01690#S1.p4.1),[§3\.2](https://arxiv.org/html/2607.01690#S3.SS2.p4.1)\.
- J\. Weiet al\.\(2024\)Measuring short\-form factuality in large language models\.arXiv preprint arXiv:2411\.04368\.Cited by:[§3\.2](https://arxiv.org/html/2607.01690#S3.SS2.p6.1)\.
- P\. J\. Werbos \(1990\)Backpropagation through time: what it does and how to do it\.Proceedings of the IEEE78\(10\),pp\. 1550–1560\.Cited by:[§3\.3](https://arxiv.org/html/2607.01690#S3.SS3.p2.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. Qiu \(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§3\.1](https://arxiv.org/html/2607.01690#S3.SS1.p1.2)\.

## Appendix AComparison to in\-context distillation

ICD is our principal learned baseline, and on the two scenarios where a framed teacher was constructed it resists the planted claim at least as well as Goggles: 0\.90 on Sheeran and 1\.00 on Dentist, against the Goggles’ 0\.90 / 0\.99 on Sheeran and 0\.76 / 0\.90 on Dentist\.

The difference between the two methods is in the shape of their application, which is illustrated by what we had to do in order to even evaluate this comparison: To train ICD against the Sheeran and Dentist datasets, we created another set of probe questions that would probe the factual content of the documents, which could then be distilled into the student model\. We trained the student LoRA with two losses: the cross\-entropy loss from the SFT documents, and a KL loss from teacher rollouts to student logits\. We created a parameterβ\\betathat parametrized this mix, and found that the effectiveness was robust across values of 0\.5, 1, and 2\.

We chose however*not*to evaluate against the two holdout datasets,*Novelists*and*Mixed*\. While it is quite likely that ICD would have been successful at instilling a fictional epistemic frame around the datasets’ documents, doing this for the 74 different fictional scenarios involved would have required generating ~1k unique probe questions in order to make sure that the frame could be successfully distilled into the model during training\.

This extra cost and complexity is the clear contrast to Goggles: with Goggles, once they are trained, we can use them cheaply for any dataset we want without any additional synthetic document generation\.

## Appendix BRelationship to the MEND model\-editing lineage

As noted in §[2](https://arxiv.org/html/2607.01690#S2), Goggles shares the core move of the MEND model\-editing lineage — a learned network that rewrites the SFT gradient, made tractable by a low\-rank structure on the model axis\. It departs from that lineage in three main ways:

- •*What*we’re editing: MEND\-family editors are supplied a desired input–output pair and overwrite a specific fact, whereas Goggles is supplied with no target output at all and instead imparts an epistemic*frame*that the model then applies to whatever facts a document teaches\.
- •*How*the edit occurs: these methods transform a gradient in a single post\-hoc application, whereas Goggles is trained through a multi\-step SFT trajectory by truncated Backpropagation Through Time, learning to steer hundreds of inner steps rather than to apply one edit\.
- •*When*the edits are made: MEND and its descendants repair an already\-trained model, leaving it otherwise intact, whereas Goggles acts concurrently with learning, shaping acquisition as it happens\. Goggles are also much more transferable: a single Goggles instance, trained once per frame, applies its frame to arbitrary documents it never saw in meta\-training, where a fact\-editor must be given each edit\.

## Appendix CEditor architecture

This appendix details the editor introduced in §[3\.1](https://arxiv.org/html/2607.01690#S3.SS1)\.

Each editor is small and strictly per\-token: the three input signals are down\-projected and concatenated into a single per\-token feature vector — no operation in the editor mixes information across tokens\. Two SwiGLU heads read this feature \(Figure[2](https://arxiv.org/html/2607.01690#S3.F2)\), one emitting a rank\-side vector in the LoRA’s rank space and one emitting a basis\-side vector; both heads emit theirAA\-factor andBB\-factor halves jointly, so the two residuals share every parameter upstream of the final split\. The basis\-side coefficients are lifted onto the model axis through a learned palette —VAV\_\{A\}\(din×bd\_\{\\mathrm\{in\}\}\\times b\) for theAAfactor,VBV\_\{B\}\(dout×bd\_\{\\mathrm\{out\}\}\\times b\) for theBBfactor — and each token contributes one rank\-1 outer product, rank\-side⊗\\otimeslifted basis\-side\. The residualr^\\hat\{r\}for each factor is the mean of these rank\-1 terms over the document’s tokens: the token average is the only place tokens combine, and the mean \(rather than sum\) keeps the edit’s scale independent of sequence length\.

TheAAandBBresiduals are formed by different outer products \(Figure[C\.1](https://arxiv.org/html/2607.01690#A3.F1)\): forAA, the palette lifts the basis\-side vector onto the input axis and takes its outer product with the rank\-side vector; forBB, the basis side is lifted onto the output axis instead\. Each residualr^\\hat\{r\}is then simply added to its factor’s gradient before the Adam step\. Because only the basis side is ever lifted to full model width, the residual never has to be built at full size — a compactr×br\\times bcore plus one palette lift suffices\. Borrowing an idea from LoRA initialization, the Goggles editor initializes to a no\-op: the basis head’s out layer is zero\-initialized, and sincer^\\hat\{r\}is bilinear in the two heads,r^=0\\hat\{r\}=0at the start of training\. The rank\-side head is left at its standard init, so — just like the zero\-initialized LoRABBmatrix paired with a nonzero LoRAAA— the gradient into the basis head is determined by the rank side, and the editor trains from the first step\.

Nothing else about training through Goggles changes: the loss is the ordinary cross\-entropy, the optimizer is the ordinary Adam, and — in distinction from the Negation Neglect protocol — the documents carry no textual annotation\. The epistemic frame enters only throughr^\\hat\{r\}\.

gk\\displaystyle g\_\{k\}=∇ϕℒSFT​\(xk;θ0,ϕ\(k−1\)\),\\displaystyle=\\nabla\_\{\\phi\}\\,\\mathcal\{L\}^\{\\mathrm\{SFT\}\}\(x\_\{k\};\\theta\_\{0\},\\phi^\{\(k\-1\)\}\),r^k\\displaystyle\\hat\{r\}\_\{k\}=Gψ​\(sg​\[hin,gout,s~\]k\),\\displaystyle=G\_\{\\psi\}\\\!\\big\(\\mathrm\{sg\}\[\\,h\_\{\\mathrm\{in\}\},g\_\{\\mathrm\{out\}\},\\tilde\{s\}\\,\]\_\{k\}\\big\),ϕ\(k\)\\displaystyle\\phi^\{\(k\)\}=Adam​\(ϕ\(k−1\),gk\+r^k\)\.\\displaystyle=\\mathrm\{Adam\}\\big\(\\phi^\{\(k\-1\)\},\\,g\_\{k\}\+\\hat\{r\}\_\{k\}\\big\)\.\(3\)HereGψG\_\{\\psi\}is the Goggles editor with parametersψ\\psi,s~=VB⊤​\(B⋅A⋅x\)\\tilde\{s\}=V\_\{B\}^\{\\top\}\(B\\cdot A\\cdot x\)is the detached LoRA\-output signal \(the third editor input, §[3\.1](https://arxiv.org/html/2607.01690#S3.SS1)\), andsg​\[⋅\]\\mathrm\{sg\}\[\\cdot\]is the stop\-gradient: every input the editor reads is detached from the SFT autograd graph\.

∇Aℒ=∑t\(B⊤​gout,t\)⊗hin,t,∇Bℒ=∑tgout,t⊗\(A​hin,t\)\.\\nabla\_\{A\}\\mathcal\{L\}=\\sum\_\{t\}\(B^\{\\top\}g\_\{\\mathrm\{out\},t\}\)\\otimes h\_\{\\mathrm\{in\},t\},\\qquad\\nabla\_\{B\}\\mathcal\{L\}=\\sum\_\{t\}g\_\{\\mathrm\{out\},t\}\\otimes\(Ah\_\{\\mathrm\{in\},t\}\)\.\(4\)
![Refer to caption](https://arxiv.org/html/2607.01690v1/x4.png)Figure C\.1:Per\-token outer\-product assembly of a gradient residual: a learned palette lifts the basis half, an outer product with the rank half forms a rank\-1 per\-token residual, and a token mean gives the residual added to the LoRA factor’s gradient\.
## Appendix DMeta\-training details

This appendix collects the objective, curriculum, and implementation details of the truncated\-BPTT meta\-training \(§[3\.3](https://arxiv.org/html/2607.01690#S3.SS3)\)\. Algorithm[D\.1](https://arxiv.org/html/2607.01690#A4.alg1)gives the full procedure\.

![Refer to caption](https://arxiv.org/html/2607.01690v1/x5.png)Figure D\.1:The Goggles training objective \(§[3\.2](https://arxiv.org/html/2607.01690#S3.SS2)\)\. A claim probe \(does the student absorb the document’s framing?\) and a locality probe \(an unrelated question, weightedλ\\lambda, preserving capability\), each a KL between the frozen teacher and the student trained through Goggles\.Algorithm D\.1Meta\-training a Goggles editorGψG\_\{\\psi\}for one epistemic frame\. Each outer step runsMMdata\-parallel workers concurrently; each advances its*own*inner SFT trajectory on its own subject\(s\), and the per\-window losses are summed within a worker and all\-reduced across workers into one update ofψ\\psi\. The eager inner steps run with the residual detached, soψ\\psireceives gradient only through the residualr^\\hat\{r\}injected during the differentiable replay windows\.hin,gout,s~h\_\{\\mathrm\{in\}\},g\_\{\\mathrm\{out\}\},\\tilde\{s\}are the three per\-token editor inputs of §[3\.1](https://arxiv.org/html/2607.01690#S3.SS1);sg​\[⋅\]\\mathrm\{sg\}\[\\cdot\]is stop\-gradient\.1:frozen base

θ0\\theta\_\{0\}; editor

GψG\_\{\\psi\}\(no\-op init\);

MMdata\-parallel workers; per\-frame teacher rollouts with claim/locality probes; inner steps

KK, windows

WW, window length

ww, locality weight

λ\\lambda, depth schedule

Lmax​\(⋅\)L\_\{\\max\}\(\\cdot\)
2:each worker

mm:

ϕm←∅\\phi\_\{m\}\\leftarrow\\varnothing,

depthm←0\\mathrm\{depth\}\_\{m\}\\leftarrow 0⊳\\trianglerightper\-worker LoRA state \(A,BA,B\+ Adam\) and depth

3:forouter step

t=1,2,…t=1,2,\\dotsdo

4:foreach worker

m=1,…,Mm=1,\\dots,Min paralleldo

5:if

depthm=0\\mathrm\{depth\}\_\{m\}=0or

depthm≥Lm\\mathrm\{depth\}\_\{m\}\\geq L\_\{m\}then⊳\\trianglerightreset/start \(depthm=0\\mathrm\{depth\}\_\{m\}\{=\}0is thet=1t\{=\}1bootstrap\)

6:

Lm∼Uniform​\{1,…,Lmax​\(t\)\}L\_\{m\}\\sim\\mathrm\{Uniform\}\\\{1,\\dots,L\_\{\\max\}\(t\)\\\}; reinit

ϕm\\phi\_\{m\}; choose subject\(s\);

depthm←0\\mathrm\{depth\}\_\{m\}\\leftarrow 0
7:endif

8:for

k=1k=1to

KKdo⊳\\trianglerighteager phase; editor detached

9:

g←∇ϕmℒSFT​\(xk;θ0,ϕm\)g\\leftarrow\\nabla\_\{\\phi\_\{m\}\}\\,\\mathcal\{L\}^\{\\mathrm\{SFT\}\}\(x\_\{k\};\\theta\_\{0\},\\phi\_\{m\}\)
10:

r^←sg​\[Gψ​\(hin,gout,s~\)\]\\hat\{r\}\\leftarrow\\mathrm\{sg\}\\big\[\\,G\_\{\\psi\}\(h\_\{\\mathrm\{in\}\},g\_\{\\mathrm\{out\}\},\\tilde\{s\}\)\\,\\big\]⊳\\trianglerightresidual; no gradient toψ\\psi

11:

ϕm←Adam​\(ϕm,g\+r^\)\\phi\_\{m\}\\leftarrow\\mathrm\{Adam\}\(\\phi\_\{m\},\\ g\+\\hat\{r\}\); cache snapshot at step

kk
12:endfor

13:

depthm←depthm\+1\\mathrm\{depth\}\_\{m\}\\leftarrow\\mathrm\{depth\}\_\{m\}\+1
14:choose starts

𝒮\\mathcal\{S\}: endpoint

K−wK\\\!\-\\\!w, plus

W−1W\\\!\-\\\!1earlier \(each

≥w\\geq w, non\-overlapping\)

15:

ℓm←0\\ell\_\{m\}\\leftarrow 0
16:for

s∈𝒮s\\in\\mathcal\{S\}do⊳\\trianglerightdifferentiable replay; autograd toψ\\psi

17:

ϕ′←\\phi^\{\\prime\}\\leftarrowsnapshot at step

ss⊳\\trianglerightfrozen start state

18:for

j=s\+1j=s\+1to

s\+ws\+wdo

19:

g←sg​\[∇ϕ′ℒSFT​\(xj\)\]g\\leftarrow\\mathrm\{sg\}\\big\[\\nabla\_\{\\phi^\{\\prime\}\}\\mathcal\{L\}^\{\\mathrm\{SFT\}\}\(x\_\{j\}\)\\big\]⊳\\trianglerightSFT gradient is a constant

20:

r^←Gψ​\(hin,gout,s~\)\\hat\{r\}\\leftarrow G\_\{\\psi\}\(h\_\{\\mathrm\{in\}\},g\_\{\\mathrm\{out\}\},\\tilde\{s\}\)⊳\\trianglerightretains graph toψ\\psi

21:

ϕ′←Adam​\(ϕ′,g\+r^\)\\phi^\{\\prime\}\\leftarrow\\mathrm\{Adam\}\(\\phi^\{\\prime\},\\ g\+\\hat\{r\}\)
22:endfor

23:

ℓm←ℓm\+KLclaim​\(ϕ′\)\+λ​KLlocality​\(ϕ′\)\\ell\_\{m\}\\leftarrow\\ell\_\{m\}\+\\mathrm\{KL\}\_\{\\mathrm\{claim\}\}\(\\phi^\{\\prime\}\)\+\\lambda\\,\\mathrm\{KL\}\_\{\\mathrm\{locality\}\}\(\\phi^\{\\prime\}\)
24:endfor

25:endfor

26:

ℓ←∑m=1Mℓm\\ell\\leftarrow\\sum\_\{m=1\}^\{M\}\\ell\_\{m\}⊳\\trianglerightsummed over workers \+ windows, all\-reduced across ranks

27:accumulate

∇ψℓ\\nabla\_\{\\psi\}\\,\\ell; everygrad\-accumouter steps: update

ψ\\psiwith AdamW, then zero

∇ψ\\nabla\_\{\\psi\}
28:endfor

![Refer to caption](https://arxiv.org/html/2607.01690v1/x6.png)Figure D\.2:Meta\-training\. The Goggle edits each inner SFT gradient over aKK\-step trajectory; the claim and locality KLs are backpropagated through the trajectory \(BPTT\) to update the Goggle\.We use reverse KL,KL​\(pstudent∥pteacher\)\\mathrm\{KL\}\(p\_\{\\mathrm\{student\}\}\\,\\\|\\,p\_\{\\mathrm\{teacher\}\}\), for the outer loss, which is computed position\-by\-position over the teacher’s response tokens and averaged, then averaged over probes\. The idea was that reverse KL is zero\-forcing: it penalizes the student for placing mass on continuations the framed teacher deems unlikely and would thus drive the student to commit to the teacher’s framed mode rather than hedge across framed and un\-framed answers\. It seems however that this choice is not critical: see Appendix[F](https://arxiv.org/html/2607.01690#A6)\.

Writingϕs\\phi\_\{s\}for the LoRA state after replaying the window that starts at step s,

ℒ​\(ψ\)=∑s\[KLclaim​\(ϕs\)\+λ⋅KLlocality​\(ϕs\)\],\\mathcal\{L\}\(\\psi\)=\\sum\_\{s\}\\big\[\\,\\mathrm\{KL\}\_\{\\mathrm\{claim\}\}\(\\phi\_\{s\}\)\+\\lambda\\cdot\\mathrm\{KL\}\_\{\\mathrm\{locality\}\}\(\\phi\_\{s\}\)\\,\\big\],
whereKLclaim\\mathrm\{KL\}\_\{\\mathrm\{claim\}\}is the mean over claim probes of the per\-probe reverse KL, andKLlocality\\mathrm\{KL\}\_\{\\mathrm\{locality\}\}the same over locality probes\. The claim term pulls the edited trajectory toward the teacher’s framed handling of the documents’ facts; the locality term, weightedλ\\lambda\(set to 1 in all our experiments\), holds the rest of the model in place\.

The inner LoRAs do not train indefinitely: after each group ofKKsteps we stochastically either continue the trajectory or reset — reinitializing the LoRA to a fresh random init and the inner Adam to an empty state — subject to a cap ofLmaxL\_\{\\max\}groups per trajectory, and configured to have equal chances of every potential trajectory length up toLmaxL\_\{\\max\}\. We annealLmaxL\_\{\\max\}upward on a curriculum: it stays at 1 for the first 50 outer steps, then increases by 1 every 20 outer steps up to a maximum of 15, so the Goggles first learns to steer shallow trajectories before being asked to steer deep ones\. The schedules we explored before settling on this ramp, and the stability ceiling that bounds it, are described below\.

The depth curriculum \(§[3\.3](https://arxiv.org/html/2607.01690#S3.SS3)\) was the most delicate part of meta\-training\. Table[D\.1](https://arxiv.org/html/2607.01690#A4.T1)records the schedules we tried before settling on the adopted ramp: every variant that started deep, ramped quickly, or aimed at too deep a peak diverged almost immediately, and only the slow ramp to a shallow peak \(Lmax=15L\_\{\\max\}\{=\}15\) trained stably to step 700\.

Table D\.1:Curriculum\-schedule development runs\. Every schedule that started deep, ramped fast, or aimed too deep diverged immediately; the adopted ramp is the only stable one\.The ceiling is not arbitrary\. Figure[D\.3](https://arxiv.org/html/2607.01690#A4.F3)plots the mean meta\-training KL \(claim and locality\) as the curriculum depthLmaxL\_\{\\max\}ramps: both terms stay flat and healthy throughLmax≈17L\_\{\\max\}\\approx 17, knee upward at 18, and run away by 20\. This is the same failure mode as the diverged architectural ablations \(§[4\.3](https://arxiv.org/html/2607.01690#S4.SS3)\) — a runaway in the trajectory\-level KL — and it is what fixes the curriculum’s usable ceiling, motivating the conservativeLmax=15L\_\{\\max\}\{=\}15peak we adopt\.

![Refer to caption](https://arxiv.org/html/2607.01690v1/x7.png)Figure D\.3:Why the curriculum is capped\. Mean meta\-training KL \(claim and locality\) as the trajectory\-depth curriculumLmaxL\_\{\\max\}ramps: both terms stay flat and healthy throughLmax≈17L\_\{\\max\}\\\!\\approx\\\!17, then knee upward at 18 and run away by 20 \(shaded\)\. This instability fixes the curriculum’s usable ceiling, motivating theLmax=15L\_\{\\max\}\{=\}15peak we adopt\.For our training documents, we adopt two regimes: with the synthetic documents we created for this experiment, we use one subject perKKsteps \(9 documents, repeated through steps randomly in small batches, resulting in 1–3 epochs over the data\), replacing the subject everyKKsteps\. For the Negation Neglect dataset docs, we*keep*the same subject for an entire unreset LoRA trajectory\. In other words, if ourLmaxL\_\{\\max\}is 12, and we stochastically decide to run12×2012\\times 20steps of inner LoRA training, then we will train that LoRA on the*same subject*for 120 steps, sampling from the rich corpus of 10k documents per subject\.

Two stop\-gradients keep the autograd graph from ballooning: the SFT gradients themselves are recomputed but treated as frozen constants — we do not build the second\-order graph through the SFT forward, nor do we run autograd through the inner Adam’s first and second moments\. There is just one path from outer loss back to the Goggles: the residualr^\\hat\{r\}injected at each of thewwreplayed steps\. The trajectory\-level KL at the window endpoints is the only training signal\.

In all our runs we useK=20K\{=\}20inner Adam steps per group andW=2W\{=\}2replay windows ofw=3w\{=\}3steps each\. One window is always placed at the end of the group \(replaying stepsK−2K\-2throughKK\), so the objective always sees the trajectory’s endpoint \(otherwise we’ve paid the compute cost of the last steps for nothing\); the remaining windows are sampled from a grid of non\-overlapping earlier positions\. Windows are never placed at the very start of a group: replaying from a cold optimizer state clearly causes training to diverge\. Gradients from theWWwindows across batches are summed — treating each window as an individual batch — and further aggregated via DDP across GPUs; when effective batch sizes are insufficient, we accumulate across multiple outer steps before taking a single outer Adam update\. The outer optimizer is AdamW at learning rate10−410^\{\-4\}, held constant for the first 350 of the 700 steps and then cosine\-decayed to zero\. Training is visibly stable well before the decay begins, so an earlier or steeper decay would likely cut cost further, though we did not sweep it to confirm capability is preserved\.

In practice the per\-position reverse KL is evaluated over the teacher’s top\-k=256k\{=\}256tokens with a single tail bucket: the teacher normalizer is computed over top\-k logits plus a tail log\-sum\-exp, the teacher’s tail mass is spread uniformly over the remaining vocabulary, and the student distribution is left untruncated so that all student mass — including mass falling outside the teacher’s top\-k — is penalized\. This keeps the reverse\-KL estimate faithful while storing onlyk\+1k\+1teacher values per position\.

## Appendix EOutcome composition over the inner\-SFT trajectory

Across the two long\-horizon scenarios we snapshot the model every 5 steps and roll out its answers into the four exclusive outcome buckets of §[4](https://arxiv.org/html/2607.01690#S4)\. Figures[E\.1](https://arxiv.org/html/2607.01690#A5.F1)and[E\.2](https://arxiv.org/html/2607.01690#A5.F2)show the resulting composition over the full 656\-step trajectory: the baselines drift from the base model’s neutral handling into belief as the planted documents accumulate, while the Goggles arms reach a high resisted fraction early and hold it\.

![Refer to caption](https://arxiv.org/html/2607.01690v1/x8.png)Figure E\.1:Outcome composition over the inner\-SFT trajectory \(Sheeran\), per the per\-question rubric judge\. Baselines absorb; Goggles resist; the Redwood\-framed Goggle additionally cites provenance; the rightmost panel is the no\-curriculum ablation \(Lmax=1L\_\{\\max\}\{=\}1\), whose resistance erodes into absorption\.![Refer to caption](https://arxiv.org/html/2607.01690v1/x9.png)Figure E\.2:Outcome composition over the inner\-SFT trajectory \(Dentist\)\. With no real\-world prior, the Redwood\-framed Goggle resists largely by citing the planted provenance\.
## Appendix FAblations

We also report a set of architectural ablations to justify our choices:

- •Applying a spectral clipping function for the LoRA matrices in the longer training runs — i\.e\. Sheeran and Dentist in our evals\. We observed that Goggles induces large singular values in the trained LoRAs — the spectral norm was an average of 43 vs 3\.5 on a non\-Goggles\-trained LoRA on the same data, with ~99% of the spectral energy going to the top singular value\. Simply applying a spectral clipping function to the LoRA at each step \(we chose to clipτ\\tauat 3\.35\) keeps this unbounded growth in check with no visible damage to the Goggles’ resistance performance or capabilities\. Concretely, after each inner optimizer step we cap each LoRA module’s updateΔ​W=B​A\\Delta W=BAatτ\\tau: whenever its top singular valueσmax\\sigma\_\{\\max\}exceedsτ\\tau, we rescale both factors byτ/σmax\\sqrt\{\\tau/\\sigma\_\{\\max\}\}, which shrinksΔ​W\\Delta Wto exactlyτ\\taualong its dominant direction while leaving that direction \(and theAA/BBnorm balance\) intact;σmax\\sigma\_\{\\max\}is read cheaply, without an SVD, from the smallr×rr\\times rGram product of the two factors\. This should likely be folded into the standard procedure for training through Goggles\.
- •Training Goggles with forward KL instead of reverse KL\. Our results here — three out of four*resisted*values inside the CI for reverse KL, with only Dentist as a significant improvement — weakly suggest that forward KL may be a better loss function for Goggles, however it’s worth also noting that the forward KL training run was somewhat less stable than the backwards KL runs — mid\-run loss increased quite a bit before retreating back to converged values \(Figure[F\.1](https://arxiv.org/html/2607.01690#A6.F1)\) — so it’s worth future study to determine which objective is overall best for the Goggles architecture\.
- •Reducing the basis dimension of the editor’s output subspace from 32 to 1, so each module’s emitted gradient residual is constrained to a rank\-1 edit along a single learned direction — trains stably and preserves capability, but the one\-dimensional bottleneck is problematic for some data regimes, as shown in particular by the low score on the held\-out novelists \(0\.26\)\. It’s likely that the Goggles edits are of low rank, and it’s likely possible to sweep the output subspace size to find something that preserves capability at lower parameter count\.
- •Removing trajectory accumulation entirely \(LmaxL\_\{\\max\}= 1, no curriculum\) — the resulting Goggles works fairly well for short trajectories, but destroys the model when run on the long\-horizon protocol, collapsing both resistance \(0\.15 on Sheeran\) and capability \(0\.12 / 0\.04\)\. This Goggles’ performance on shorter\-trajectory evaluation was better, but not as good as Goggles trained through the full trajectory curriculum\.
- •Replacing the per\-token heads with modules that take the mean of input tokens, so the editor ignores token\-level input entirely — diverges; per\-token conditioning is required\.
- •Replacing the SwiGLU activations in the Goggles MLPs with a single \(zero\-initialized\) linear map — this diverges, demonstrating the importance of nonlinearity in the architecture\.
- •Removing the contextual LoRA\-state conditioning, leaving the editor blind to what the adapter already does on each token — this also diverges\.

![Refer to caption](https://arxiv.org/html/2607.01690v1/x10.png)Figure F\.1:Forward versus reverse KL during meta\-training\. Averaged outer KL,12​\(KLclaim\+KLlocality\)\\frac\{1\}\{2\}\(\\mathrm\{KL\}\_\{\\mathrm\{claim\}\}\+\\mathrm\{KL\}\_\{\\mathrm\{locality\}\}\), over meta\-training steps for a reverse\-KL Goggle \(the adopted objective\) and a forward\-KL Goggle; bold curves are EMA\-smoothed over the raw traces\. The reverse\-KL run descends smoothly and stays stable, while the forward\-KL run is stable early but destabilizes mid\-run \(shaded\), spiking before retreating to converged values\. The two are different divergences, so their absolute levels are not directly comparable, however the differences in stability are clear\.Note that “diverges” here means that the*outer training loop*, during Goggles training, had runaway values\. In particular, we found that trajectory\-level KL blows up here — the Goggles ceases to be usable at all\.