Bypass Observation: A Conceptual Design of a Non-Intrusive Layer-Wise Semantic Extraction Architecture

# Bypass Observation: A Conceptual Design of a Non-Intrusive Layer-Wise Semantic Extraction Architecture
Source: [https://arxiv.org/html/2609.13807](https://arxiv.org/html/2609.13807)
Jiang Yu††thanks:Corresponding author\.Affiliation:National University of Defense TechnologyAffiliation:Changsha, ChinaEmail:[yujiang@nudt\.edu\.cn](mailto:)

###### Abstract

The reasoning process of large language models unfolds in a high\-dimensional hidden\-vector space, while humans can observe only the final output\. This mismatch constitutes a central obstacle to interpretability and safety auditing\. This paper systematically consolidates and refines a*Bypass Observation*architecture\. Without changing the vector transmission between layers of the backbone model, a read\-only observation head is attached in parallel to selected inference layers\. In its simplest form, the head is an LM\-head\-style or calibrated vocabulary readout that exposes intermediate predictive distributions; in an extended form, a separately trained semantic translator may convert these readouts or hidden states into compact human\-readable summaries\. In all cases, the observation output is never fed back into the backbone computation\. We make four main contributions\. First, for the full\-vocabulary readout setting, we derive a closed\-form approximation of the computational overhead: the additional cost is primarily governed byV/\(12​d\)V/\(12d\)and ranges from roughly 30% to 240% for representative models\. We further discuss sparse observation, low\-rank factorization, reduced semantic vocabularies, top\-kkoperators, and key\-position sampling as practical cost\-reduction strategies\. Second, we argue that bypass observation moves a model from a black box toward a semi\-transparent system\. Its read\-only nature prevents the observation output from causally influencing the backbone and separates the observation channel from conventional self\-generated post\-hoc explanations; however, the resulting readout remains only a partial view of the underlying hidden state, creating a potential*white\-box illusion*\. Third, we distinguish the causal status of*bypass chain\-of\-thought*from conventional chain\-of\-thought: the former is an open\-loop readout, whereas the latter is part of a closed\-loop computation\. This distinction changes how the two signals should be interpreted and optimized in reinforcement learning\. Fourth, we analyze the particular value of bypass observation for looped or recurrent\-depth Transformers, where iteration\-wise decoding can turn otherwise opaque dynamic computation into an observable convergence trajectory and may support halting criteria and safety thresholds\. All conclusions in the present paper are conceptual or analytical and remain to be validated through systematic experiments\.

## 1Introduction

During inference in Transformer\-based large language models, information is propagated through the residual stream as high\-dimensional hidden vectors and is projected into a vocabulary distribution only at the output layer\. Humans can directly observe the final decoded output, but the computations performed across tens or even hundreds of intermediate layers remain difficult to interpret\. This black\-box property creates at least three problems\. First, it is difficult to answer what the model is computing at intermediate stages\. Second, potentially deceptive internal states, the formation of hallucinations, or abnormal activations may remain invisible until the final output is produced\. Third, reinforcement learning based only on terminal outcomes provides sparse credit assignment and may be vulnerable to linguistic reward hacking\.

Existing interpretability methods can be grouped into several broad categories, each with structural limitations:

- •Probes and linear decoding\.Probe\-based methods\([Alain and Bengio, 2017](https://arxiv.org/html/2609.13807#bib.bib1);[Hewitt and Manning, 2019](https://arxiv.org/html/2609.13807#bib.bib9)\)demonstrate that syntactic or semantic information can be decoded from hidden states, but such results primarily establish correlation rather than causation and often require offline training\.
- •Lens methods\.The Logit Lens\([nostalgebraist, 2020](https://arxiv.org/html/2609.13807#bib.bib14)\)directly applies the unembedding matrix to intermediate hidden states to inspect the evolution of predictions\. The Tuned Lens\([Belrose et al\., 2023](https://arxiv.org/html/2609.13807#bib.bib3)\)improves early\-layer readability through layer\-wise affine calibration\. These methods, however, are typically used as diagnostic tools rather than as persistent architectural components\.
- •Sparse autoencoders and mechanistic interpretability\.Sparse autoencoders \(SAEs\) extract interpretable features from superposed representations\([Huben et al\., 2024](https://arxiv.org/html/2609.13807#bib.bib4);[Templeton et al\., 2024](https://arxiv.org/html/2609.13807#bib.bib16)\), while attribution\-graph approaches organize features into computational structures\([Ameisen et al\., 2025](https://arxiv.org/html/2609.13807#bib.bib2)\)\. These methods can be powerful, but feature labeling often requires additional interpretation and the resulting analysis is usually sparse or retrospective\.
- •Chain\-of\-thought and faithfulness studies\.Explicit intermediate reasoning can improve performance\([Wei et al\., 2022](https://arxiv.org/html/2609.13807#bib.bib18)\), but later work has shown that verbalized rationales are not always faithful descriptions of the computation that produced the answer\([Lanham et al\., 2023](https://arxiv.org/html/2609.13807#bib.bib11);[Turpin et al\., 2023](https://arxiv.org/html/2609.13807#bib.bib17)\)\.

Bypass Observation can be viewed as an architectural, systematic, and controllable extension of the Logit Lens idea\. Instead of treating layer\-wise decoding as an occasional diagnostic operation, the model is equipped with a persistent observation pathway that can be deployed, evaluated, and potentially incorporated into training or auditing workflows\. Its defining property isnon\-intrusiveness: decoded observations are exposed only to an external observer and are never inserted into the attention context of later layers\.

Figure[1](https://arxiv.org/html/2609.13807#S1.F1)summarizes three compatible instantiations of the observation pathway: \(a\) a single LM head shared across multiple layers, \(b\) separate layer\-specific LM heads, and \(c\) akk\-adaptive LM head conditioned on the layer index or inference step\. These variants differ in parameter sharing and calibration flexibility, but all preserve the same read\-only separation from the backbone computation\.

Figure 1:Three architectural variants of Bypass Observation\.\(a\) A shared LM head is reused across multiple reasoning layers, providing a parameter\-efficient common readout\. \(b\) Each reasoning layer is equipped with its own LM head, allowing layer\-specific calibration at increased parameter cost\. \(c\) Akk\-adaptive LM head conditions its readout on the layer index or inference step, enabling one observer to adapt across depth or recurrent computation\. Solid black arrows denote the main forward computation, while dashed blue arrows denote the read\-only observation \(bypass\) pathway; the displayed “observed text” should be interpreted as a schematic readout that may correspond to top\-kkvocabulary predictions or, with an additional semantic translator, a textual summary\. In all variants, observation outputs are not fed back into the backbone\.#### Property 1 \(Non\-interference\)\.

Bypass Observation is a read\-only operation attached to the backbone computation graph\. Therefore, assuming deterministic execution and identical numerical kernels, attaching the observation module does not alter the functional input to any subsequent backbone layer and does not change the model’s final output\.

Existing empirical evidence from lens\-based methods supports the premise that intermediate hidden states contain decodable predictive information\. In particular, the Tuned Lens demonstrates that layer\-specific calibration can substantially improve intermediate predictive readout\. Bypass Observation adopts this evidence as motivation, while parameter sharing across layers remains an additional architectural choice rather than a property established by Tuned Lens itself\. More generally, the value and the limitations of the proposed architecture must be stated together: Bypass Observation may provide a useful projection of an actual internal computation, but it does not provide a complete description of that computation\.

## 2Formalization and Basic Axioms

Consider a Transformer backbone withLLlayers, hidden dimensiondd, and vocabulary sizeVV\. Let the hidden state at layerℓ\\ellbe𝐡ℓ∈ℝd\\mathbf\{h\}\_\{\\ell\}\\in\\mathbb\{R\}^\{d\}\. Standard inference follows

𝐡ℓ\+1=Fℓ​\(𝐡ℓ\)\.\\mathbf\{h\}\_\{\\ell\+1\}=F\_\{\\ell\}\(\\mathbf\{h\}\_\{\\ell\}\)\.\(1\)
For selected layers, Bypass Observation attaches an observation functionOℓO\_\{\\ell\}in parallel:

yℓ=Decode⁡\(Oℓ​\(𝐡ℓ\)\)\.y\_\{\\ell\}=\\mathrm\{Decode\}\\\!\\left\(O\_\{\\ell\}\(\\mathbf\{h\}\_\{\\ell\}\)\\right\)\.\(2\)Here,OℓO\_\{\\ell\}may denote either a lightweight vocabulary readout \(for example, an LM head or a calibrated lens\) or a learned semantic translator\. The closed\-form cost analysis in Section 3 applies specifically to the full\-vocabulary readout case; a more expressive sequence decoder would introduce additional computation not captured by thed​VdVterm\.

We define the architecture through three axioms:

- •\(A1\) Non\-interference\.For every layer, the functionFℓF\_\{\\ell\}and its inputs contain no observation outputyky\_\{k\}fork≤ℓk\\leq\\ell\. The backbone output is therefore unchanged by the existence of the observation path\. A1 is a defining property rather than an engineering preference: onceyℓy\_\{\\ell\}is fed back into the backbone context, the system becomes a different architecture and the theoretical properties discussed here no longer apply\.
- •\(A2\) Parameter sharing\.Observation heads may share parameters across layers, optionally conditioned on a layer index or a recurrent time\-step embedding\. This sharing is a design choice of Bypass Observation rather than a requirement inherited from Tuned Lens, whose calibration is typically layer\-specific\. For a shared full\-vocabulary readout, the additional parameter count can beO⁡\(d​V\)O\(dV\)or lower, compared with a rough backbone scale ofO⁡\(12​L​d2\)O\(12Ld^\{2\}\)\.
- •\(A3\) Conditional readout\.The observation head may be conditioned on layer index or recurrent time step to account for representation drift and role changes across depth or iterations\.

## 3Computational Overhead

### 3\.1Closed\-Form Approximation

We count multiply\-accumulate operations \(MACs\) per generated token and per Transformer layer\. One MAC corresponds to approximately two FLOPs\.

Relevant hyperparameters include hidden dimensiondd, the number of query headshh, head dimensiondheadd\_\{\\mathrm\{head\}\}, the number of key/value headshkvh\_\{\\mathrm\{kv\}\}, feed\-forward dimensiondffd\_\{\\mathrm\{ff\}\}, and the feed\-forward activation structure\. The approximation below is independent of the total number of layersLL\.

#### Attention projections\.

For standard multi\-head attention \(MHA\) withh​dhead=dh\\,d\_\{\\mathrm\{head\}\}=d, the Q, K, V, and output projections each have effective shaped×dd\\times d, producing approximately

Cattn,proj≈4​d2C\_\{\\mathrm\{attn,proj\}\}\\approx 4d^\{2\}\(3\)MACs per layer\.

With grouped\-query attention \(GQA\) or multi\-query attention \(MQA\), wherehkv<hh\_\{\\mathrm\{kv\}\}<h, the K and V projections are reduced tod×\(hkv​dhead\)d\\times\(h\_\{\\mathrm\{kv\}\}d\_\{\\mathrm\{head\}\}\)\. The corresponding projection cost becomes

Cattn,proj=\[2\+2​\(hkvh\)\]​d2\.C\_\{\\mathrm\{attn,proj\}\}=\\left\[2\+2\\left\(\\frac\{h\_\{\\mathrm\{kv\}\}\}\{h\}\\right\)\\right\]d^\{2\}\.\(4\)During autoregressive decoding with a KV cache, the attention\-score computationQ​K⊤QK^\{\\top\}and multiplication byVVadd approximately2​S​d2SdMACs for context lengthSS\. Relative to a12​d212d^\{2\}layer estimate, this term has ratioS/\(6​d\)S/\(6d\); ford=4096d=4096, it becomes comparable only at context lengths on the order of2\.4×1042\.4\\times 10^\{4\}tokens\. This estimate does not describe full\-sequence prefill, whose attention term has the familiar quadratic dependence on sequence length\.

#### Feed\-forward network\.

A conventional GELU MLP withdff=4​dd\_\{\\mathrm\{ff\}\}=4dcontains an up\-projection and a down\-projection, giving

CFFN=2​d​dff=8​d2\.C\_\{\\mathrm\{FFN\}\}=2dd\_\{\\mathrm\{ff\}\}=8d^\{2\}\.\(5\)Modern architectures such as Llama and Qwen commonly use a SwiGLU\-style gated feed\-forward block with three matrices\. With a width near\(8/3\)​d\(8/3\)d, its cost is also approximately

3​d​\(83​d\)=8​d2\.3d\\left\(\\frac\{8\}\{3\}d\\right\)=8d^\{2\}\.\(6\)

#### Standard approximation\.

Under either MHA \+ GELU\-4​d4dor MHA \+ SwiGLU\-\(8/3\)​d\(8/3\)d, the backbone cost per layer is therefore approximately

Clayer≈4​d2\+8​d2=12​d2C\_\{\\mathrm\{layer\}\}\\approx 4d^\{2\}\+8d^\{2\}=12d^\{2\}\(7\)MACs\.

A full vocabulary observation head performs ad×Vd\\times Vmatrix\-vector product at each observed layer, requiring approximatelyd​VdVMACs\. If every layer is observed, the relative overhead is therefore

ρ≈d​V12​d2=V12​d\.\\rho\\approx\\frac\{dV\}\{12d^\{2\}\}=\\frac\{V\}\{12d\}\.\(8\)
For arbitrary GQA and feed\-forward widths, a more general expression is

Clayer\\displaystyle C\_\{\\mathrm\{layer\}\}=\[2\+2​\(hkvh\)\]​d2\+nff​d​dff,\\displaystyle=\\left\[2\+2\\left\(\\frac\{h\_\{\\mathrm\{kv\}\}\}\{h\}\\right\)\\right\]d^\{2\}\+n\_\{\\mathrm\{ff\}\}dd\_\{\\mathrm\{ff\}\},\(9\)ρ\\displaystyle\\rho=V\[2\+2​\(hkv/h\)\]​d\+nff​dff,\\displaystyle=\\frac\{V\}\{\\left\[2\+2\\left\(h\_\{\\mathrm\{kv\}\}/h\\right\)\\right\]d\+n\_\{\\mathrm\{ff\}\}d\_\{\\mathrm\{ff\}\}\},\(10\)wherenff=2n\_\{\\mathrm\{ff\}\}=2for a two\-matrix GELU MLP andnff=3n\_\{\\mathrm\{ff\}\}=3for a three\-matrix SwiGLU block\.

Layer normalization, softmax, residual additions, and biases involve primarilyO⁡\(d\)O\(d\)element\-wise operations and are ignored in this first\-order estimate\. The embedding lookup and final LM head are also omitted from the per\-layer comparison\. The approximation is related to the commonN≈12​L​d2N\\approx 12Ld^\{2\}scaling heuristic discussed in compute analyses such as Chinchilla\([Hoffmann et al\., 2022](https://arxiv.org/html/2609.13807#bib.bib10)\)\. Architectures using more aggressively compressed attention representations may deviate further from this simplified formula\.

### 3\.2Representative Model Estimates

Table[1](https://arxiv.org/html/2609.13807#S3.T1)summarizes analytical estimates for representative model configurations\.

Table 1:Analytical estimates of the per\-layer overhead of full\-vocabulary bypass decoding\. Values are computed from representative model configurations under the accounting assumptions described in Section 3\.1\.#### Observation 3\.1\.

The overhead is primarily controlled by the ratio between vocabulary sizeVVand the effective per\-layer width\. Under the standard12​d212d^\{2\}approximation it reduces toV/\(12​d\)V/\(12d\)\. GQA decreases attention cost, while wider feed\-forward blocks increase it, so realistic layer costs may vary from roughly12​d212d^\{2\}to18​d218d^\{2\}in the examples considered here\. Consequently, the simpleV/\(12​d\)V/\(12d\)estimate can deviate substantially from a model\-specific calculation\. The key reason for the large overhead is that a full LM head is itself a larged×Vd\\times Vmatrix\. For modern models with vocabularies above 100k tokens, applying such a head at every layer can cost as much as, or more than, a complete Transformer layer\.

### 3\.3Engineering Cost and Cost\-Reduction Strategies

FLOPs are not the only bottleneck\. During prefill, decoding every position at every layer can create a logits tensor with shape

batch×sequence×L×V,\\texttt\{batch\}\\times\\texttt\{sequence\}\\times L\\times V,which can become a more severe memory and bandwidth problem than the raw arithmetic count\. Several reduction strategies are therefore important:

1. 1\.Sparse layer observation\.Observe one out of everysslayers, reducing the additional arithmetic cost approximately toρ/s\\rho/s, wheressis the observation stride\.
2. 2\.Low\-rank readout\.Approximate the observation matrix byW≈B​CW\\approx BC\. If the intermediate rank isrr, the cost scales approximately withr/d\+r/Vr/d\+r/Vrelative to the full matrix\. For example, a rank\-64 factorization can drastically reduce the arithmetic cost forddin the thousands andVVabove 100k\.
3. 3\.Reduced semantic vocabulary\.Equip the observation pathway with a smaller semantic vocabulary, for example on the order of 8k concepts or tokens, instead of the full model vocabulary\.
4. 4\.Top\-kkoperators and key\-position sampling\.Avoid materializing the completeVV\-dimensional distribution at every position\. Observation can instead be restricted to sentence boundaries, answer spans, or other high\-value positions\.
5. 5\.Asynchronous deployment\.Enable Bypass Observation during training, debugging, or auditing, but disable it during ordinary production inference\.

#### Observation 3\.2 \(Mathematical non\-interference is not engineering zero\-cost\)\.

A1 guarantees that the observation pathway does not alter the mathematical input to the backbone, but the added memory traffic, bandwidth consumption, kernel scheduling, and device synchronization can still reduce throughput\. These are separate claims and should not be conflated\.

## 4Interpretability: From a Black Box to a Semi\-Transparent System

### 4\.1Degree of Transparency

Bypass Observation does not make a model fully white\-box\. A more appropriate description is*semi\-transparent*,*gray\-box*, or*observable gray\-box*\. Its potential benefits include:

- •\(P1\) Temporal visibility\.Interpretation moves from purely post\-hoc analysis toward online monitoring\. Layer\-wise and token\-wise readout may provide an early warning window before a harmful or incorrect conclusion is verbalized in the final output\.
- •\(P2\) Dense coverage\.The architecture can, in principle, record every selected layer and every selected token rather than relying only on sparse probe locations\.
- •\(P3\) A read\-only observation channel\.Because the observation text is not fed back into the backbone, it is not itself part of the model’s subsequent causal computation\. This makes it qualitatively different from asking a model to retrospectively explain its own answer\.
- •\(P4\) Human\-readable outputs\.A basic LM\-head\-style observer exposes vocabulary distributions or top\-kktoken predictions rather than a free\-form explanation\. A separately trained semantic translator can further compress these signals into natural\-language summaries, at the cost of introducing an additional model whose own faithfulness must be validated\. Such readouts may also help localize candidate states for later intervention or activation editing\([Meng et al\., 2022](https://arxiv.org/html/2609.13807#bib.bib13);[Zou et al\., 2023](https://arxiv.org/html/2609.13807#bib.bib20)\)\.

### 4\.2Three Fundamental Boundaries

- •\(L1\) Lossy verbalization\.A vocabulary readout can preserve substantial information about an intermediate hidden state, but converting the full distribution into a small set of tokens or a compact textual summary is necessarily selective\. Hidden representations can contain superposed information\([Elhage et al\., 2022](https://arxiv.org/html/2609.13807#bib.bib7)\); features that are weakly aligned with vocabulary directions, or that are suppressed by top\-kkselection or summarization, may be omitted from the human\-readable trace\.
- •\(L2\) Correlation is not causation\.Observing that a layer appears to “talk about” conceptXXdoes not imply that conceptXXis causally responsible for the model’s downstream behavior\. Activation patching, state perturbation, or related interventions are required to establish a causal link\.
- •\(L3\) The white\-box illusion\.A well\-trained observation head may produce fluent and plausible descriptions even when those descriptions are incomplete or misleading\. Readability and faithfulness are independent properties\. A fluent misinterpretation can be more dangerous than an obviously noisy one, so deployment requires explicit faithfulness calibration\.

#### Observation 4\.1\.

Bypass Observation should be interpreted as a potentially faithful*projection*of a computational process, not as a complete description of that process\. Attribution claims should therefore be cross\-validated with probes, sparse features, and causal interventions\.

### 4\.3Implications for Safety Auditing

The main positive effect is increased monitoring density and the possibility of earlier detection\. However, the observation pathway also introduces new risks\. First, an auditor may confuse fluent translation with actual mechanistic understanding\. Second, the observation trajectory itself can become an information\-leakage surface that reveals internal states useful for model extraction or attack design\. Third, the observation head can itself be corrupted or adversarially manipulated, systematically hiding undesirable signals\. Practical systems may therefore need frozen observation heads, integrity checks, access control, output sanitization, and statistical anomaly detection\. An asynchronous deployment model—enabled for training and auditing but disabled in ordinary production inference—can reduce some of these risks\.

## 5Bypass Chain\-of\-Thought Versus Conventional Chain\-of\-Thought

### 5\.1Different Causal Roles

We call the sequence of layer\-wise textual readouts a*bypass chain\-of\-thought*\. Its causal status differs fundamentally from conventional chain\-of\-thought \(CoT\), as summarized in Table[2](https://arxiv.org/html/2609.13807#S5.T2)\.

Table 2:Causal differences between conventional chain\-of\-thought and bypass chain\-of\-thought\.A useful analogy is that conventional CoT is a draft written by the reasoner, whereas a bypass trajectory is a subtitle generated by an observer\. Editing the subtitle does not change the movie\. The two trajectories may agree closely, but they may also diverge, especially if the model reaches a decision in middle layers and uses later layers mainly for linguistic refinement\.

### 5\.2Derived Differences

Two additional consequences follow:

- •Cross\-layer consistency checks\.Neighboring observations should often evolve coherently\. Abrupt disagreement can itself be a high\-information signal, possibly corresponding to representational reorganization or internal conflict\.
- •Independent optimization of the observer\.Unlike conventional CoT, whose form is coupled to the base model, an observation module can be trained or calibrated independently while the backbone remains frozen\.

## 6Asymmetric Roles in Reinforcement Learning

### 6\.1Core Distinction

#### Observation 6\.1 \(Asymmetric causal roles\)\.

Conventional CoT and bypass chain\-of\-thought occupy different causal positions during inference, but both can provide training signals in reinforcement learning\.

Conventional CoT is part of the autoregressive trajectory: once a reasoning token is generated, it enters the subsequent context and directly changes later computation\. This gives conventional CoT a dual role\. It is both an observable textual trace and a causal component of the inference process\. Reinforcement learning from human feedback \(RLHF\), reinforcement learning with verifiable rewards \(RLVR\), and process\-supervision methods can therefore shape both the generated reasoning trajectory and the downstream computation that consumes it\([Ouyang et al\., 2022](https://arxiv.org/html/2609.13807#bib.bib15);[Lightman et al\., 2024](https://arxiv.org/html/2609.13807#bib.bib12);[DeepSeek\-AI, 2025](https://arxiv.org/html/2609.13807#bib.bib5)\)\.

A bypass trajectory is different at inference time because it is read\-only: its decoded output is not inserted into the model context and therefore has no direct causal effect on later layers or later tokens\. However, this does*not*imply that a frozen bypass observer is unusable for reinforcement learning\. Policy\-gradient methods do not require the scalar reward to be differentiable with respect to model parameters\. A reward computed from a frozen observation head can still update the backbone through the likelihood of sampled trajectories:

∇θJ​\(θ\)=𝔼τ∼πθ​\[R⁡\(τ\)​∑t∇θ​log​πθ​\(at∣st\)\]\.\\nabla\_\{\\theta\}J\(\\theta\)=\\mathbb\{E\}\_\{\\tau\\sim\\pi\_\{\\theta\}\}\\left\[R\(\\tau\)\\sum\_\{t\}\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\\right\]\.\(11\)The important distinction is therefore between*inference\-time causal participation*and*training\-time usefulness as a reward or supervision signal*\.

### 6\.2Two Training Modes

Two practically different modes should be distinguished:

- •Mode A: frozen observer\.The observation head is fixed and treated as an external measurement function\. Its outputs can be used to define scalar rewards, process scores, pseudo\-labels, consistency penalties, or data\-filtering criteria\. The backbone can still be updated through policy gradients or other learning objectives\. Freezing the observer improves measurement stability, but it does not eliminate reward hacking: the policy may learn hidden states that score well under the fixed observer without genuinely improving the intended reasoning behavior\.
- •Mode B: jointly trained observer\.The observer is updated together with the backbone\. This increases flexibility and allows the readout to adapt to representation drift, but it also creates an additional failure mode: the observer and policy may co\-adapt in a way that improves the measured score without improving the underlying behavior\. This is a stronger Goodhart\-style risk because both the measured representation and the measurement function can move\.

### 6\.3A Conservative Training Protocol

A conservative workflow is:

1. 1\.Train or calibrate the observer while freezing the backbone\.Possible targets include next\-token distributions, layer\-specific predictive states, weak supervision from model\-generated reasoning traces, or structured semantic labels\.
2. 2\.Validate the readout before using it as supervision\.Use activation patching, state perturbation, controlled counterfactuals, or predictive consistency tests to estimate whether the observation is behaviorally informative\. Only sufficiently calibrated layers should be used for high\-stakes auditing or reward construction\.
3. 3\.Prefer auxiliary use before direct optimization\.The readout can first be used as an input to a process reward model, a consistency objective, a data\-filtering rule, or a pseudo\-label generator\. These uses reduce the risk of prematurely optimizing directly against a poorly calibrated observer\.
4. 4\.Avoid circular supervision\.The same observation head should not be both the sole generator of a training target and the sole evaluator of whether that target is correct\.
5. 5\.If direct RL is used, audit for observer gaming\.Compare bypass\-derived rewards against task success, final\-answer correctness, causal interventions, and independently trained observers\. A rising bypass score without corresponding behavioral improvement should be treated as a potential reward\-hacking signal\.
6. 6\.Track process\-level metrics\.Candidate metrics include cross\-layer convergence, agreement between intermediate and final predictions, and localization of abrupt state transitions\. A reward based on monotonic convergence toward a correct prediction may provide denser process feedback than a purely terminal reward, although this remains an empirical hypothesis\.

## 7Implications for Looped Transformers

### 7\.1The Auditing Problem in Recurrent\-Depth Architectures

Architectures such as the Universal Transformer\([Dehghani et al\., 2019](https://arxiv.org/html/2609.13807#bib.bib6)\), Looped Transformers\([Giannou et al\., 2023](https://arxiv.org/html/2609.13807#bib.bib8)\), and recurrent\-depth latent\-reasoning models\([Geiping et al\., 2025](https://arxiv.org/html/2609.13807#bib.bib19)\)reuse weights across multiple computational iterations\. This creates several auditing difficulties:

- •\(D1\) Weight–function decoupling\.A fixed statement such as “layerkkperforms functionXX” becomes less meaningful when the same parameters can perform different roles at different iterations\.
- •\(D2\) Input\-dependent computational depth\.If the number of iterations is adaptive, total computation is not fixed in advance\. The process may converge, oscillate, or repeatedly reinforce an incorrect state\.
- •\(D3\) Error localization across iterations\.Without intermediate readout, it can be difficult to identify the iteration at which a trajectory begins to deviate\.

### 7\.2Why Bypass Observation Is Particularly Relevant

#### Observation 7\.1 \(From spatial snapshots to a temporal narrative\)\.

In a standard feed\-forward Transformer, layer\-wise decoding primarily provides a sequence across depth\. In a recurrent\-depth model, iteration\-wise decoding additionally provides a temporal trajectory\. The resulting trace can therefore be interpreted as an observable approximation of the model’s convergence process\.

#### Hypothesis 7\.2 \(Potential halting criterion\)\.

If an appropriately calibrated observation trajectory becomes stable, that stability may serve as one signal for early stopping\. Conversely, persistent oscillation or non\-convergence could trigger further computation or an audit alert\. This is a hypothesis rather than an established stopping rule and requires receiver operating characteristic \(ROC\) analysis of false positive and false negative rates\.

#### Observation 7\.3 \(Per\-run iterative auditing\)\.

Because different inputs may invoke different numbers of iterations, auditing should record the actual trajectory for each run\. Bypass Observation naturally provides this per\-run window and may help identify infinite loops, oscillations, or self\-reinforcing trajectories that are specific to recurrent computation\.

### 7\.3Additional Costs and Risks

- •Cost grows with the number of iterations\.If a static architecture exposesLLobservation sites, a recurrent computation repeated forbbiterations can create up toL×bL\\times beffective observation sites\. Sparse or event\-triggered monitoring therefore becomes even more important\.
- •The observer should be conditioned on time step\.The same hidden space can play different functional roles at different iterations\. A shared observer that ignores iteration index may therefore be systematically miscalibrated\.
- •Apparent convergence can be misleading\.Later iterations may show only small output changes even when a small hidden\-state update eventually causes a critical decision flip\. Differential observation ofΔ​𝐡t=𝐡t−𝐡t−1\\Delta\\mathbf\{h\}\_\{t\}=\\mathbf\{h\}\_\{t\}\-\\mathbf\{h\}\_\{t\-1\}may be more informative than static snapshots alone\.
- •Feedback is especially risky\.Feeding the observation trajectory back into a recurrent computation introduces a new feedback path and may create self\-excitation or instability\. The read\-only constraint is therefore particularly important in looped architectures\.

### 7\.4Unvalidated Engineering Ideas

Potential engineering mechanisms include gated caching, where only observations containing new information are persisted; ring buffers that retain a recent trajectory; and tiered storage between accelerator memory, CPU memory, and persistent storage\. These ideas are currently speculative and require dedicated benchmarking\.

## 8Improving the Decoding Rule

Intermediate\-state decoding should not necessarily reuse ordinary greedy language generation\. Several alternatives are worth exploring:

1. 1\.Represent distributions rather than only a single token\.Intermediate states may encode multiple competing hypotheses\. A single greedy token can collapse this uncertainty and create a false sense of confidence\. Top\-kkcandidates, entropy, or other distributional summaries may be preferable for faithful inspection\.
2. 2\.Differential decoding\.DecodeΔ​𝐡ℓ=𝐡ℓ−𝐡ℓ−1\\Delta\\mathbf\{h\}\_\{\\ell\}=\\mathbf\{h\}\_\{\\ell\}\-\\mathbf\{h\}\_\{\\ell\-1\}to ask “what changed at this layer?” rather than only “what does the current state represent?”
3. 3\.Cross\-layer consistency scoring\.A lightweight consistency model can flag abrupt semantic discontinuities between neighboring observations\.
4. 4\.Observer ensembles\.Multiple independently trained observers can be used\. Their disagreement should be retained rather than forcibly averaged, because disagreement itself may reveal uncertainty\.
5. 5\.Structured outputs and capacity constraints\.The observer can be restricted to fields such as current subgoal, attended entity, candidate answer, and confidence\. Limiting output capacity can reduce the risk that the observer becomes an unconstrained reasoning model with its own hallucinations\.
6. 6\.Contrastive calibration\.Intermediate readouts can be calibrated against later or final predictive distributions to reduce systematic layer\-dependent noise\.
7. 7\.SAE\-level captions\.Instead of mapping directly into vocabulary space, the observation pathway could expose sparse feature activations and associate them with higher\-level semantic descriptions\.
8. 8\.A dedicated small translator\.A separate compact model may be trained to convert hidden states or vocabulary readouts into summaries, providing greater flexibility than directly reusing the base model’s LM head\. Because this translator performs additional sequence modeling, its runtime cost is not included in thed​VdVclosed\-form estimate derived in Section 3\.

## 9Limitations and Open Questions

The current proposal has several important limitations:

1. 1\.Faithfulness cannot be self\-certified\.The usefulness of the entire system depends on whether the observer is faithful, yet faithfulness can only be estimated through external evidence and interventions\.
2. 2\.White\-box illusion\.Fluent observation text may lead users to overestimate mechanistic understanding\. Structured output, disagreement preservation, and explicit faithfulness scores may help mitigate this problem\.
3. 3\.Adversarial robustness\.A model may learn internal representations that appear benign to an observer while still supporting undesirable computation\. The monitoring channel therefore becomes part of the adversarial surface\.
4. 4\.Practical cost\.Full\-vocabulary LM\-head\-style observation at every layer can add roughly 30%–240% arithmetic cost under the representative calculations in this paper\. A learned semantic translator may incur additional cost beyond this estimate\. These costs make always\-on deployment unrealistic for many systems\.
5. 5\.Architectural boundary\.Once observation outputs are fed back into the computation, the architecture is no longer Bypass Observation in the strict sense defined by A1\. Closed\-loop failure modes then reappear\.
6. 6\.Lack of empirical validation\.The present paper is conceptual and analytical\. The usefulness of bypass signals for process supervision, the false\-alarm rate of recurrent\-depth halting criteria, and the information fidelity of low\-rank or reduced\-vocabulary observers all require systematic experiments\.

## 10Conclusion

This paper develops the concept of Bypass Observation as a non\-intrusive, layer\-wise semantic readout architecture for large language models\.

First, the computational cost of a full\-vocabulary readout is primarily governed by the ratio between vocabulary size and effective hidden width\. Under a standard approximation, the additional cost isV/\(12​d\)V/\(12d\), while model\-specific calculations for representative architectures range from roughly 30% to 240%\. Sparse observation, low\-rank factorization, reduced vocabularies, and selective decoding can reduce this cost substantially\.

Second, Bypass Observation can move a model from a black\-box system toward a semi\-transparent one by exposing intermediate projections without feeding them back into the backbone\. However, such readouts remain lossy, correlational unless validated causally, and vulnerable to the white\-box illusion\.

Third, bypass chain\-of\-thought and conventional chain\-of\-thought occupy fundamentally different causal positions at inference time\. Conventional CoT becomes part of the autoregressive context, whereas a bypass trajectory remains read\-only\. Nevertheless, a frozen bypass observer can still define rewards or supervision signals that update the backbone through policy gradients\. This makes bypass readouts suitable for process reward models, pseudo\-labeling, consistency checking, auditing, and potentially direct RL, provided that observer gaming is explicitly monitored\.

Fourth, recurrent\-depth and looped Transformers may be an especially important setting for this architecture\. Iteration\-wise readout can expose a convergence trajectory, potentially support stopping decisions, and help localize oscillation or self\-reinforcing failures\. At the same time, recurrent computation amplifies the cost and makes time\-step calibration and strict read\-only separation more important\.

A practical research agenda includes: \(1\) operational measures of observer faithfulness and intervention\-based validation protocols; \(2\) controlled comparisons of bypass\-derived process supervision against outcome rewards and conventional CoT supervision; \(3\) evaluation of sparse and differential observation for recurrent\-depth models, including ROC analysis of halting criteria; \(4\) access control and anti\-probing mechanisms for observation traces; and \(5\) cross\-validation between textual readouts and sparse feature dictionaries\.

### AI Use Statement

Generative AI tools were used to assist with language editing, translation, and LaTeX formatting of this manuscript\. All technical claims, equations, references, and AI\-assisted edits were reviewed by the authors\. The authors take responsibility for the final content of the work\.

### Reproducibility Statement

This paper is primarily conceptual and analytical\. The computational\-overhead estimates are derived from the formulas and representative model configurations described in Section 3\. The proposed architecture, faithfulness protocols, reinforcement\-learning uses, and recurrent\-depth applications remain to be validated empirically; no new experimental benchmark is claimed in the present version\.

## References

- Alain and Bengio \(2017\)G\. Alain and Y\. BengioUnderstanding intermediate layers using linear classifier probes\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ryF7rTqgl)Cited by:[1st item](https://arxiv.org/html/2609.13807#S1.I1.i1.p1.1)\.
- Ameisenet al\.\(2025\)E\. Ameisen, J\. Lindsey, A\. Pearce, W\. Gurnee, N\. L\. Turner, B\. Chen, C\. Citro, D\. Abrahams, S\. Carter, B\. Hosmer, J\. Marcus, M\. Sklar, A\. Templeton, T\. Bricken, C\. McDougall, H\. Cunningham, T\. Henighan, A\. Jermyn, A\. Jones, A\. Persic, Z\. Qi, T\. B\. Thompson, S\. Zimmerman, K\. Rivoire, T\. Conerly, C\. Olah, and J\. BatsonCircuit tracing: revealing computational graphs in language models\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2025/attribution-graphs/methods.html)Cited by:[3rd item](https://arxiv.org/html/2609.13807#S1.I1.i3.p1.1)\.
- Belroseet al\.\(2023\)N\. Belrose, Z\. Furman, L\. Smith, D\. Halawi, I\. Ostrovsky, L\. McKinney, S\. Biderman, and J\. SteinhardtEliciting latent predictions from transformers with the tuned lens\.External Links:2303\.08112,[Link](https://arxiv.org/abs/2303.08112)Cited by:[2nd item](https://arxiv.org/html/2609.13807#S1.I1.i2.p1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-AIDeepSeek\-R1: incentivizing reasoning capability in LLMs via reinforcement learning\.External Links:2501\.12948,[Document](https://dx.doi.org/10.48550/arXiv.2501.12948),[Link](https://arxiv.org/abs/2501.12948)Cited by:[§6\.1](https://arxiv.org/html/2609.13807#S6.SS1.SSS0.Px1.p2.1)\.
- Dehghaniet al\.\(2019\)M\. Dehghani, S\. Gouws, O\. Vinyals, J\. Uszkoreit, and L\. KaiserUniversal transformers\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=HyzdRiR9Y7)Cited by:[§7\.1](https://arxiv.org/html/2609.13807#S7.SS1.p1.1)\.
- Elhageet al\.\(2022\)N\. Elhage, T\. Hume, C\. Olsson, N\. Schiefer, T\. Henighan, S\. Kravec, Z\. Hatfield\-Dodds, R\. Lasenby, D\. Drain, C\. Chen, R\. Grosse, S\. McCandlish, J\. Kaplan, D\. Amodei, M\. Wattenberg, and C\. OlahToy models of superposition\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2022/toy_model/index.html)Cited by:[1st item](https://arxiv.org/html/2609.13807#S4.I2.i1.p1.1)\.
- Geipinget al\.\(2025\)J\. Geiping, S\. McLeish, N\. Jain, J\. Kirchenbauer, S\. Singh, B\. R\. Bartoldson, B\. Kailkhura, A\. Bhatele, and T\. GoldsteinScaling up test\-time compute with latent reasoning: a recurrent depth approach\.External Links:2502\.05171,[Link](https://arxiv.org/abs/2502.05171)Cited by:[§7\.1](https://arxiv.org/html/2609.13807#S7.SS1.p1.1)\.
- Giannouet al\.\(2023\)A\. Giannou, S\. Rajput, J\. Sohn, K\. Lee, J\. D\. Lee, and D\. PapailiopoulosLooped transformers as programmable computers\.InProceedings of the 40th International Conference on Machine Learning,A\. Krause, E\. Brunskill, K\. Cho, B\. Engelhardt, S\. Sabato, and J\. Scarlett \(Eds\.\),Proceedings of Machine Learning Research, Vol\.202,pp\. 11398–11442\.External Links:[Link](https://proceedings.mlr.press/v202/giannou23a.html)Cited by:[§7\.1](https://arxiv.org/html/2609.13807#S7.SS1.p1.1)\.
- Hewitt and Manning \(2019\)J\. Hewitt and C\. D\. ManningA structural probe for finding syntax in word representations\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),Minneapolis, Minnesota,pp\. 4129–4138\.External Links:[Document](https://dx.doi.org/10.18653/v1/N19-1419),[Link](https://aclanthology.org/N19-1419/)Cited by:[1st item](https://arxiv.org/html/2609.13807#S1.I1.i1.p1.1)\.
- Hoffmannet al\.\(2022\)J\. Hoffmann, S\. Borgeaud, A\. Mensch, E\. Buchatskaya, T\. Cai, E\. Rutherford, D\. de Las Casas, L\. A\. Hendricks, J\. Welbl, A\. Clark, T\. Hennigan, E\. Noland, K\. Millican, G\. van den Driessche, B\. Damoc, A\. Guy, S\. Osindero, K\. Simonyan, E\. Elsen, J\. W\. Rae, O\. Vinyals, and L\. SifreTraining compute\-optimal large language models\.InAdvances in Neural Information Processing Systems,Vol\.35\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/c1e2faff6f588870935f114ebe04a3e5-Abstract-Conference.html)Cited by:[§3\.1](https://arxiv.org/html/2609.13807#S3.SS1.SSS0.Px3.p4.1)\.
- Hubenet al\.\(2024\)R\. Huben, H\. Cunningham, L\. R\. Smith, A\. Ewart, and L\. SharkeySparse autoencoders find highly interpretable features in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/1fa1ab11f4bd5f94b2ec20e794dbfa3b-Abstract-Conference.html)Cited by:[3rd item](https://arxiv.org/html/2609.13807#S1.I1.i3.p1.1)\.
- Lanhamet al\.\(2023\)T\. Lanham, A\. Chen, A\. Radhakrishnan, B\. Steiner, C\. Denison, D\. Hernandez, D\. Li, E\. Durmus, E\. Hubinger, J\. Kernion, K\. Lukošiūtė, K\. Nguyen, N\. Cheng, N\. Joseph, N\. Schiefer, O\. Rausch, R\. Larson, S\. McCandlish, S\. Kundu, S\. Kadavath, S\. Yang, T\. Henighan, T\. Maxwell, T\. Telleen\-Lawton, T\. Hume, Z\. Hatfield\-Dodds, J\. Kaplan, J\. Brauner, S\. R\. Bowman, and E\. PerezMeasuring faithfulness in chain\-of\-thought reasoning\.External Links:2307\.13702,[Link](https://arxiv.org/abs/2307.13702)Cited by:[4th item](https://arxiv.org/html/2609.13807#S1.I1.i4.p1.1)\.
- Lightmanet al\.\(2024\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/aca97732e30bcf1303bc22ac3924fd16-Abstract-Conference.html)Cited by:[§6\.1](https://arxiv.org/html/2609.13807#S6.SS1.SSS0.Px1.p2.1)\.
- Menget al\.\(2022\)K\. Meng, D\. Bau, A\. Andonian, and Y\. BelinkovLocating and editing factual associations in GPT\.InAdvances in Neural Information Processing Systems,Vol\.35\.External Links:[Document](https://dx.doi.org/10.52202/068431-1262),[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/6f1d43d5a82a37e89b0665b33bf3a182-Abstract-Conference.html)Cited by:[4th item](https://arxiv.org/html/2609.13807#S4.I1.i4.p1.1)\.
- nostalgebraist \(2020\)nostalgebraistInterpreting GPT: the logit lens\.Note:LessWrongExternal Links:[Link](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens)Cited by:[2nd item](https://arxiv.org/html/2609.13807#S1.I1.i2.p1.1)\.
- Ouyanget al\.\(2022\)L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. L\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. F\. Christiano, J\. Leike, and R\. LoweTraining language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,Vol\.35\.External Links:[Document](https://dx.doi.org/10.52202/068431-2011),[Link](https://proceedings.neurips.cc/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html)Cited by:[§6\.1](https://arxiv.org/html/2609.13807#S6.SS1.SSS0.Px1.p2.1)\.
- Templetonet al\.\(2024\)A\. Templeton, T\. Conerly, J\. Marcus, J\. Lindsey, T\. Bricken, B\. Chen, A\. Pearce, C\. Citro, E\. Ameisen, A\. Jones, H\. Cunningham, N\. L\. Turner, C\. McDougall, M\. MacDiarmid, C\. D\. Freeman, T\. R\. Sumers, E\. Rees, J\. Batson, A\. Jermyn, S\. Carter, C\. Olah, and T\. HenighanScaling monosemanticity: extracting interpretable features from claude 3 sonnet\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html)Cited by:[3rd item](https://arxiv.org/html/2609.13807#S1.I1.i3.p1.1)\.
- Turpinet al\.\(2023\)M\. Turpin, J\. Michael, E\. Perez, and S\. R\. BowmanLanguage models don’t always say what they think: unfaithful explanations in chain\-of\-thought prompting\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Document](https://dx.doi.org/10.52202/075280-3275),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/ed3fea9033a80fea1376299fa7863f4a-Abstract.html)Cited by:[4th item](https://arxiv.org/html/2609.13807#S1.I1.i4.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. ZhouChain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 24824–24837\.External Links:[Document](https://dx.doi.org/10.52202/068431-1800),[Link](https://proceedings.neurips.cc/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html)Cited by:[4th item](https://arxiv.org/html/2609.13807#S1.I1.i4.p1.1)\.
- Zouet al\.\(2023\)A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. HendrycksRepresentation engineering: a top\-down approach to AI transparency\.External Links:2310\.01405,[Link](https://arxiv.org/abs/2310.01405)Cited by:[4th item](https://arxiv.org/html/2609.13807#S4.I1.i4.p1.1)\.