Interpreting Black-Box Large Language Models with Sentence-Level Energy Landscapes

# Interpreting Black-Box Large Language Models with Sentence-Level Energy Landscapes
Source: [https://arxiv.org/html/2608.02879](https://arxiv.org/html/2608.02879)
Maryam Rezaee Pooriya Safaei Maryam Asgarinezhad S\. Fatemeh Seyyedsalehi Department of Mathematical Sciences, Sharif University of Technology, Tehran, Iran ms\.maryamrezaee@gmail\.com,pooriya\.safaei@sharif\.edu maryamasgn123@gmail\.com,seyyedsalehi@sharif\.edu

###### Abstract

The widespread adoption of proprietary Large Language Models \(LLMs\) accessed strictly through closed APIs has created a critical challenge for responsible deployment: a fundamental lack of interpretability\. To address this, we propose a model\-agnostic, post\-hoc attribution interpreter operating at the sentence level\. Our approach trains an Energy\-Based Model \(EBM\) as a surrogate to capture the LLM’s internal conceptual consistency between prompts and responses\. This energy landscape guides the training of a lightweight interpreter network\. Uniquely, our interpreter operates as a standalone tool; once trained, it quantifies the influence of prompt sentences on a user\-specified target output without requiring further API queries to the LLM\. By globally training a local interpreter across diverse inputs, our framework captures broader generation patterns and mitigates instance\-specific biases\. Experiments demonstrate that our EBM accurately simulates the target LLM, allowing the interpreter to effectively identify the prompt sentences most influential in generating specific target outputs\.

Interpreting Black\-Box Large Language Models with Sentence\-Level Energy Landscapes

Maryam Rezaee Pooriya Safaei Maryam Asgarinezhad S\. Fatemeh SeyyedsalehiDepartment of Mathematical Sciences, Sharif University of Technology, Tehran, Iranms\.maryamrezaee@gmail\.com,pooriya\.safaei@sharif\.edumaryamasgn123@gmail\.com,seyyedsalehi@sharif\.edu

## 1Introduction

![Refer to caption](https://arxiv.org/html/2608.02879v1/x1.png)Figure 1:Overview of the Proposed Framework\.The prompt and response of a black\-box LLM are split into sentences and embedded via a pre\-trained model\. An encoding module maps these embeddings to a concept space where proximity reflects relevance\. Subsequently, an interaction module evaluates the consistency between input and output concepts to identify the most influential prompt sentences\. These modules are trained using signals from an energy network that simulates the generation process of the target LLM\.Large Language Models \(LLMs\) have demonstrated extraordinary performance across complex tasks\. Consequently, researchers and developers are rapidly adopting them for diverse applications\. However, the critical challenge facing this adoption is a fundamental lack of interpretability\. Most powerful LLMs are proprietary and accessed strictly through closed\-access APIs\. Even when architectures and pre\-training datasets are available, their complexity obscures exactly how outputs are generated\. In high\-stakes domains like medicine and law, this opacity is unacceptable, as experts cannot verify the generated output against domain knowledge or detect hidden biases\. This prevents meeting the application\-grounded standards for responsible deployment\(Doshi\-Velez and Kim,[2017](https://arxiv.org/html/2608.02879#bib.bib30)\)\.

Post\-hoc attribution is a primary approach to addressing this opacity\. These methods explain model behavior by performing instance\-wise feature selection—an interpretability paradigm for identifying an importance vector that measures how much each specific input feature influences the prediction for a given instance\(Chenet al\.,[2018](https://arxiv.org/html/2608.02879#bib.bib2)\)\. However, standard attribution techniques, including white\-box and model\-agnostic methods, struggle in the context of LLMs\. White\-box methods, which rely on gradients or activations, are incompatible with closed APIs\. Furthermore, the faithfulness of popular proxies like attention weights has been challenged\(Jain and Wallace,[2019](https://arxiv.org/html/2608.02879#bib.bib36)\)\. Model\-agnostic methods exist\(Ribeiroet al\.,[2016](https://arxiv.org/html/2608.02879#bib.bib41); Lundberg and Lee,[2017](https://arxiv.org/html/2608.02879#bib.bib39); Seyyedsalehiet al\.,[2024](https://arxiv.org/html/2608.02879#bib.bib34)\), but typically target discriminative models with well\-defined outputs\. Interpreting generative models is significantly harder as the problem is fundamentally ill\-posed\. These models utilize complex representations to produce high\-dimensional outputs like text\. Therefore, effective explanation is hindered by the output’s interactivity and sheer volume\(Schneider,[2024](https://arxiv.org/html/2608.02879#bib.bib42)\)\.

Alternatives like prompt\-based self\-explanation\(Weiet al\.,[2022](https://arxiv.org/html/2608.02879#bib.bib48)\)are similarly problematic; they rely on the same process we seek to verify, leading to circular logic and motivated reasoning\. Consequently, models often produce plausible\-sounding yet unfaithful confabulations\(Turpinet al\.,[2023](https://arxiv.org/html/2608.02879#bib.bib47)\)\. While automated prompt engineering can steer model behavior to mitigate biases, it is unsuitable for interpretation\. These methods optimize instructions for pre\-defined targets\(Zhouet al\.,[2023](https://arxiv.org/html/2608.02879#bib.bib53); Clemmeret al\.,[2024](https://arxiv.org/html/2608.02879#bib.bib14)\), rendering them unusable for ambiguous interpretation tasks\.

To address these limitations, we propose a model\-agnostic, post\-hoc attribution interpreter\. We diverge from standard approaches by shifting the resolution from noisy tokens to coherent sentences, which we define as “concepts,” with the goal of relating elements of the output directly to the user prompt at this concept level\. While recent mechanistic work defines concepts as latent activation vectors\(Gaoet al\.,[2025](https://arxiv.org/html/2608.02879#bib.bib4)\)or distinct architectural bottlenecks\(Sunet al\.,[2025](https://arxiv.org/html/2608.02879#bib.bib3)\), we adopt apropositionaldefinition suitable for black\-box analysis\. Following the Context Principle\(Frege,[1991](https://arxiv.org/html/2608.02879#bib.bib8)\), individual tokens remain semantically ambiguous without a propositional structure; a sentence, representing a complete thought\(Kintsch,[1998](https://arxiv.org/html/2608.02879#bib.bib9)\), serves as a robust operational concept that captures the causal relationships necessary for LLM generation\. Formally, given a prompt𝐱\\mathbf\{x\}and an LLM response𝐲\\mathbf\{y\}, we target a specific subset of the output𝐲T⊂𝐲\\mathbf\{y\}\_\{T\}\\subset\\mathbf\{y\}\. We then produce an importance vector to identify the subset of prompt sentences𝐱S⊂𝐱\\mathbf\{x\}\_\{S\}\\subset\\mathbf\{x\}that were most influential in generating𝐲T\\mathbf\{y\}\_\{T\}\.

We employ a unique paradigm to globally train a local interpreter\. Unlike local interpreters \(e\.g\. LIME\(Ribeiroet al\.,[2016](https://arxiv.org/html/2608.02879#bib.bib41)\)\) which observe only immediate neighborhoods—often causing interpretability illusions\(Friedmanet al\.,[2024](https://arxiv.org/html/2608.02879#bib.bib32)\)—we train across a wider distribution\. This enables our model to capture global generation patterns and mitigate intrinsic biases\.

Figure[1](https://arxiv.org/html/2608.02879#S1.F1)illustrates an overview of the approach\. We first train a transformer\-based Energy\-Based Model \(EBM\) to act as a surrogate for the black\-box LLM\. Our choice of an EBM is motivated by the distinction betweengenerationandinterpretation\. While autoregressive models generate text locally, interpreting a thought can rely on theglobalconsistency between a prompt and a response\. The EBM maps sentences to a latent “concept space,” which simulates the concept\-level relationships embedded in the target LLM, and learns a scalar energy function to measure this compatibility; this avoids the intractability of normalizing probabilities over the vast space of possible output sentences\. We then use this energy landscape to guide the training of an interpreter network\. Given a prompt and a target output subset, the interpreter produces an importance vector that isolates the prompt sentences most influential in generating that response\.

In summary, our framework makes three core contributions: \(1\) shifting the unit of analysis from noisy tokens to sentences to enable human\-intelligible, concept\-level attribution; \(2\) introducing a transformer\-based EBM with novel sampling methods capable of learning a surrogate random field over prompts and responses, with the aim of robustly modeling authentic input\-output dynamics; and \(3\) proposing a post\-hoc, model\-agnostic framework for interpreting black\-box LLMs that finds the specific prompt sentences responsible for triggering a target subset of the LLM’s output\.

## 2Related Work

### 2\.1Post\-hoc Interpretation Methods

Attribution methods score the importance of input features for a specific model output\. White\-box approaches to attribution often utilize gradients, propagating salient output signals back to input tokens\(Simonyanet al\.,[2014](https://arxiv.org/html/2608.02879#bib.bib44); Sundararajanet al\.,[2017](https://arxiv.org/html/2608.02879#bib.bib45); Shrikumaret al\.,[2017](https://arxiv.org/html/2608.02879#bib.bib43); Cheferet al\.,[2021](https://arxiv.org/html/2608.02879#bib.bib27)\)\. Others use internal attention weights as proxies for feature importance\(Xuet al\.,[2015](https://arxiv.org/html/2608.02879#bib.bib50); Liet al\.,[2017](https://arxiv.org/html/2608.02879#bib.bib38); Xieet al\.,[2017](https://arxiv.org/html/2608.02879#bib.bib49); Haoet al\.,[2021](https://arxiv.org/html/2608.02879#bib.bib35)\)\. However, gradients are inaccessible for proprietary APIs, and attention weights are frequently unfaithful to the generation process\(Jain and Wallace,[2019](https://arxiv.org/html/2608.02879#bib.bib36)\)\. Similarly, influence functions pose data\-centric explanations\(Koh and Liang,[2017](https://arxiv.org/html/2608.02879#bib.bib20)\)but remain infeasible without access to the base data or model’s Hessian\.

Perturbation\-based methods\(Ribeiroet al\.,[2016](https://arxiv.org/html/2608.02879#bib.bib41); Lundberg and Lee,[2017](https://arxiv.org/html/2608.02879#bib.bib39); Yin and Neubig,[2022](https://arxiv.org/html/2608.02879#bib.bib51)\)provide a model\-agnostic alternative by measuring sensitivity to input alterations, a strategyHackmannet al\.\([2024](https://arxiv.org/html/2608.02879#bib.bib33)\)apply to identify influential words in LLM prompts\. Other foundational work, such asChenet al\.\([2018](https://arxiv.org/html/2608.02879#bib.bib2)\), frames this as instance\-wise feature selection by learning an explainer network to maximize mutual information\. However, these methods incur prohibitive computational costs for generative tasks\(Enouenet al\.,[2024](https://arxiv.org/html/2608.02879#bib.bib31); Zhao and Shan,[2024](https://arxiv.org/html/2608.02879#bib.bib52)\)\. Consequently, recent work explores training language models to produce answer decompositions as intermediate steps for attribution\(Balasubramanianet al\.,[2025](https://arxiv.org/html/2608.02879#bib.bib6)\), or proposes surrogate frameworks to simulate LLM thinking\(Chenet al\.,[2026](https://arxiv.org/html/2608.02879#bib.bib7)\)\. Unlike these approaches, which necessitate model fine\-tuning or internal access, our method adopt simulation and decomposition within black\-box constraints\.

Finally, prompt\-based self\-explanation leverages the LLM’s own generation capabilities\. Most notably, techniques like Chain\-of\-Thought \(CoT\) ask the model to produce a rationale to justify its output\(Weiet al\.,[2022](https://arxiv.org/html/2608.02879#bib.bib48)\)\. While compelling, these explanations lack guarantees of faithfulness; they often represent plausible post\-hoc rationalizations rather than the true internal computation path\(Turpinet al\.,[2023](https://arxiv.org/html/2608.02879#bib.bib47)\)\.

### 2\.2Energy\-Based Models in NLP

Energy\-Based Models \(EBMs\) offer a distinct advantage for modeling high\-dimensional structured outputs like text, as they do not require a normalized probability distribution\(LeCunet al\.,[2006](https://arxiv.org/html/2608.02879#bib.bib10)\)\. Standard probabilistic models must sum to one over all possible outputs, which is intractable for language\. EBMs circumvent this by learning a scalar compatibility score, where low energy corresponds to high data density\.

In Natural Language Processing \(NLP\), several paradigms integrate EBMs directly into generation to refine outputs\. The Residual EBM approach adds a corrective energy term to autoregressive log\-probabilities to capture high\-level properties like coherence\(Denget al\.,[2020](https://arxiv.org/html/2608.02879#bib.bib28); Bakhtinet al\.,[2021](https://arxiv.org/html/2608.02879#bib.bib24)\)\. Alternatively, EBMs define target energy landscapes for student networks via knowledge distillation\(Tuet al\.,[2020](https://arxiv.org/html/2608.02879#bib.bib46)\)\. Recently,Xuet al\.\([2025](https://arxiv.org/html/2608.02879#bib.bib5)\)extended these principles to diffusion language models, validating that energy landscapes effectively model the complex, high\-dimensional distributions of modern text generation\.

Beyond direct generation, EBMs excel as holistic evaluators and post\-hoc rankers\.Bakhtinet al\.\([2019](https://arxiv.org/html/2608.02879#bib.bib23)\)demonstrated that Transformer\-based discriminators function as EBMs to distinguish human from machine text\. This evaluative capability naturally extends to post\-hoc refinement pipelines, where EBMs act as rerankers to select the highest\-quality candidates\(Bhattacharyyaet al\.,[2021](https://arxiv.org/html/2608.02879#bib.bib26)\), or explicitly model reward distributions to robustly align language models without retraining\(Lochab and Zhang,[2025](https://arxiv.org/html/2608.02879#bib.bib1)\)\.

### 2\.3Concept\-Based Explanations

Interpretability research is shifting toward concept\-based explanations that map decisions to human\-intelligible ideas\(Kimet al\.,[2018](https://arxiv.org/html/2608.02879#bib.bib37)\)\. While mechanistic approaches now dominate the white\-box setting—utilizing Sparse Autoencoders to disentangle polysemantic neurons\(Gaoet al\.,[2025](https://arxiv.org/html/2608.02879#bib.bib4)\)or Concept Bottleneck layers\(Sunet al\.,[2025](https://arxiv.org/html/2608.02879#bib.bib3)\)—they necessitate internal access\. We address the black\-box regime by mapping decisions to propositions rather than latent features\. This granularity aligns with rationale evaluation standards requiring propositional evidence rather than isolated tokens for faithfulness\(DeYounget al\.,[2020](https://arxiv.org/html/2608.02879#bib.bib16)\)\. Our work operationalizes this by defining a “sentence” as the fundamental conceptual unit, relying on its ability to represent a robust thought for interpretation\.

Treating sentences as semantic objects is well\-justified by the history of language modeling\. Foundational architectures like BERT used Next Sentence Prediction to learn logical relationships\(Devlinet al\.,[2019](https://arxiv.org/html/2608.02879#bib.bib29)\)\. Subsequent work on Sentence\-BERT confirmed that fine\-tuned sentence representations map similar meanings to distinct, nearby points in a vector space\(Reimers and Gurevych,[2019](https://arxiv.org/html/2608.02879#bib.bib40)\)\. By leveraging sentences as concepts, our work parallels recent architectural innovations such as Large Concept Models, which shift the core computational unit from tokens to sentence\-level representations\(Barraultet al\.,[2024](https://arxiv.org/html/2608.02879#bib.bib25)\)\.

## 3Methodology

![Refer to caption](https://arxiv.org/html/2608.02879v1/x2.png)\(a\)
![Refer to caption](https://arxiv.org/html/2608.02879v1/x3.png)\(b\)

Figure 2:Architectural Overview\.Schematics for \(a\) the energy functionℰLM\\mathcal\{E\}\_\{\\text\{LM\}\}and \(b\) the interpreter networkℐ​𝒩\\mathcal\{IN\}\.Our goal is to develop a post\-hoc, model\-agnostic method for interpreting black\-box LLMs, specifically by identifying which input sentences drive the response\. We depart from standard token\-level attribution by establishing the sentence as the fundamental unit of analysis\. As the smallest linguistic unit expressing a complete proposition, the sentence serves as a robust “concept,” enabling us to interpret generation as an interplay of complete ideas rather than ambiguous tokens\. Formally, let𝐱\\mathbf\{x\}be the prompt and𝐲\\mathbf\{y\}be the LLM response\. We target a subset of output sentences,𝐲T⊆𝐲\\mathbf\{y\}\_\{T\}\\subseteq\\mathbf\{y\}, and seek to quantify the influence of each concept in𝐱\\mathbf\{x\}on the generation of𝐲T\\mathbf\{y\}\_\{T\}\.

We propose a two\-stage framework to achieve this\. First, we pre\-train an Energy\-Based Model \(EBM\),ℰLM​\(𝐱,𝐲;θ\)\\mathcal\{E\}\_\{\\text\{LM\}\}\(\\mathbf\{x\},\\mathbf\{y\};\\theta\), to serve as a differentiable surrogate for the black\-box LLM\. As a function ofθ\\theta, this model assigns scalar values representing the consistency of a prompt\-response pair\(𝐱,𝐲\)\(\\mathbf\{x\},\\mathbf\{y\}\)with the target LLM’s generation patterns\. Second, we leverage this energy landscape to guide the training of a lightweight interpreter,ℐ​𝒩​\(𝐱,𝐲T;α\)\\mathcal\{IN\}\(\\mathbf\{x\},\\mathbf\{y\}\_\{T\};\\alpha\), parameterized byα\\alpha\. Taking the prompt, response, and a user\-specified target output𝐲T\\mathbf\{y\}\_\{T\}as inputs, the interpreter generates a sparse, binary vector matching the number of prompt sentences\. In this vector, values of11identify the subset of prompt sentences𝐱S⊆𝐱\\mathbf\{x\}\_\{S\}\\subseteq\\mathbf\{x\}strictly necessary for generating the target\.

### 3\.1Sentence Extraction and Embedding

The pipeline begins by transforming the input text𝐱\\mathbf\{x\}and output text𝐲\\mathbf\{y\}into sequences of concepts\. We first perform sentence segmentation using the spaCy library\(Honnibal and Montani,[2017](https://arxiv.org/html/2608.02879#bib.bib13)\)\. Subsequently, we employ a frozen, pre\-trained Sentence\-BERT module\(Reimers and Gurevych,[2019](https://arxiv.org/html/2608.02879#bib.bib40)\)to map each sentence to a fixed\-dimensional vector\. This yields embedding sequencesSinS^\{\\text\{in\}\}andSoutS^\{\\text\{out\}\}, which function analogously to token embeddings within our architecture\. For further preprocessing and implementation details, including padding strategies, visit Appendix[A](https://arxiv.org/html/2608.02879#A1)\.

### 3\.2The Energy\-Based Surrogate Model

To approximate the black\-box LLM’s behavior, we design a globally\-aware EBM\. Unlike the target LLM, which predicts the next tokenP​\(wt\|w<t\)P\(w\_\{t\}\|w\_\{<t\}\), EBM acts as a non\-normalized compatibility function that evaluates the entire sequence holistically\(LeCunet al\.,[2006](https://arxiv.org/html/2608.02879#bib.bib10)\)\. It assigns a scalar score to the joint configuration of a prompt and response to learn the shape of the LLM’s generation manifold\. Specifically, the model learns to distinguish variations of authentic prompt\-response pairs from corrupted ones; the lower the assigned energy, the more likely the pair is consistent with concept interactions within the LLM\.

As shown in Figure[2\(a\)](https://arxiv.org/html/2608.02879#S3.F2.sf1), the architecture processes sentence embeddings in three stages:

1. 1\.Concept Space Projection:Static embeddings \(Sin,SoutS^\{\\text\{in\}\},S^\{\\text\{out\}\}\) capture meaning in isolation, but lack the specific context of the prompt and response\. To remedy this, we pass these embeddings through separate, trainable self\-attention modules \(𝒫inℰ,𝒫outℰ\\mathcal\{P\}\_\{\\text\{in\}\}^\{\\mathcal\{E\}\},\\mathcal\{P\}\_\{\\text\{out\}\}^\{\\mathcal\{E\}\}\) to project them into a dynamic “concept space” \(Cin,CoutC^\{\\text\{in\}\},C^\{\\text\{out\}\}\)\. Here, distances reflect the LLM’s internal dependency structure rather than generic semantic similarity\. This is a function of the model’s underlying architecture and training dataset\.
2. 2\.Input\-Output Interaction:A cross\-attention block allows input conceptsCinC^\{\\text\{in\}\}to attend to output conceptsCoutC^\{\\text\{out\}\}, weighing the causal influence of the prompt on the response\.
3. 3\.Energy Calculation:The interacting representations are aggregated via attention pooling and passed through a Multi\-Layer Perceptron \(MLP\) to output a scalar energyℰLM​\(𝐱,𝐲;θ\)\\mathcal\{E\}\_\{\\text\{LM\}\}\(\\mathbf\{x\},\\mathbf\{y\};\\theta\)\.

The EBM is trained in two phases: First, it is pre\-trained using a novel set of objectives, then subsequently fine\-tuned alongside the interpreter\.

For pre\-training, we generate a dataset of prompt\-output pairs\(𝐱,𝐲\)\(\\mathbf\{x\},\\mathbf\{y\}\)from the target black\-box LLM\. To constrain the EBM to the target LLM’s input\-output dynamics, we employ two complementary contrastive objectives\. We define thefidelityobjective \(ℒfidelity\\mathcal\{L\}\_\{\\text\{fidelity\}\}\) as an InfoNCE loss to capture the global generation signature; by treating responses from humans or other LMs as negative samples, we force the EBM to distinguish the target’s authentic style\. Conversely, we define thelocal dependencyobjective \(ℒdep\\mathcal\{L\}\_\{\\text\{dep\}\}\) to target local conceptual interactions using two batch\-wise samplers\. The first,\(xpart,ypart′\)\(x\_\{\\text\{part\}\},y^\{\\prime\}\_\{\\text\{part\}\}\), distinguishes the correct partial response from off\-topic partial responses for the InfoNCE lossℒresp\-dep\\mathcal\{L\}\_\{\\text\{resp\-dep\}\}\. The second,\(xpart′,ypart\)\(x^\{\\prime\}\_\{\\text\{part\}\},y\_\{\\text\{part\}\}\), mirrors this for partial prompts against negatives in the InfoNCE lossℒpmt\-dep\\mathcal\{L\}\_\{\\text\{pmt\-dep\}\}\. This compels the model to verify input\-output dependencies piece\-by\-piece rather than relying on general heuristics\.

Thus, we minimize the combined adaptive loss:

ℒtotal=\(1−λ\)​ℒfidelity\+λ​ℒdep\\mathcal\{L\}\_\{\\text\{total\}\}=\(1\-\\lambda\)\\mathcal\{L\}\_\{\\text\{fidelity\}\}\+\\lambda\\mathcal\{L\}\_\{\\text\{dep\}\}\(1\)whereλ\\lambdais a configurable weight andℒdep\\mathcal\{L\}\_\{\\text\{dep\}\}is the sum of the two sampler losses,ℒresp\-dep\\mathcal\{L\}\_\{\\text\{resp\-dep\}\}andℒpmt\-dep\\mathcal\{L\}\_\{\\text\{pmt\-dep\}\}\. We define the energy scoring term ash​\(𝐮,𝐯\)=exp⁡\(−ℰLM​\(𝐮,𝐯;θ\)/τ\)h\(\\mathbf\{u\},\\mathbf\{v\}\)=\\exp\(\-\\mathcal\{E\}\_\{\\text\{LM\}\}\(\\mathbf\{u\},\\mathbf\{v\};\\theta\)/\\tau\)\. Accordingly, the individual InfoNCE losses are formulated as:

ℒ=−log⁡\(h​\(𝐱i,𝐲i\)h​\(𝐱i,𝐲i\)\+∑\(𝐱′,𝐲′\)∈𝒩ih​\(𝐱′,𝐲′\)\)\\mathcal\{L\}=\-\\log\\left\(\\frac\{h\(\\mathbf\{x\}\_\{i\},\\mathbf\{y\}\_\{i\}\)\}\{h\(\\mathbf\{x\}\_\{i\},\\mathbf\{y\}\_\{i\}\)\+\\sum\_\{\(\\mathbf\{x\}^\{\\prime\},\\mathbf\{y\}^\{\\prime\}\)\\in\\mathcal\{N\}\_\{i\}\}h\(\\mathbf\{x\}^\{\\prime\},\\mathbf\{y\}^\{\\prime\}\)\}\\right\)

\(2\)Here,𝒩i\\mathcal\{N\}\_\{i\}constitutes the set of negative samples\. Because the quality of the energy landscape hinges on these contrasts, we detail the specific sampling protocols and the full training hyperparameters in Appendix[B](https://arxiv.org/html/2608.02879#A2)\. This pre\-training phase \(Fig\.[6](https://arxiv.org/html/2608.02879#A2.F6)\) yields a globally\-aware energy function, which captures the target LLM’s latent structure and provides the supervision signal required to train the interpreter\.

### 3\.3The Interpreter Model

Given prompt𝐱\\mathbf\{x\}, response𝐲\\mathbf\{y\}, and target𝐲T⊆𝐲\\mathbf\{y\}\_\{T\}\\subseteq\\mathbf\{y\}, the interpreter identifies prompt sentences influential on𝐲T\\mathbf\{y\}\_\{T\}\. It outputs a binary vector where 1 indicates a necessary precursor sentence\.

Figure[2\(b\)](https://arxiv.org/html/2608.02879#S3.F2.sf2)illustrates the architecture, which mirrors the EBM’s three stages with targeted modifications\. As before, embeddingsSinS^\{\\text\{in\}\}andSoutS^\{\\text\{out\}\}are projected into the concept space via self\-attention\. In the interaction phase, however, we retain only the target conceptsCToutC^\{\\text\{out\}\}\_\{T\}and mask the remainder of the output\. The input conceptsCinC^\{\\text\{in\}\}then attend to these targets via cross\-attention\. Finally, an MLP and Gumbel\-Softmax unit\(Janget al\.,[2017](https://arxiv.org/html/2608.02879#bib.bib22)\)\(see App\.[C](https://arxiv.org/html/2608.02879#A3)\) process the results to yield a binary importance vector for the input sentences\.

Let𝐱~=𝐱⊙ℐ​𝒩​\(𝐱;𝐲T,α\)\\tilde\{\\mathbf\{x\}\}=\\mathbf\{x\}\\odot\\mathcal\{IN\}\(\\mathbf\{x\};\\mathbf\{y\}\_\{T\},\\alpha\)be the selected subset\. A successful selection minimizes the energy of the authentic pair\(𝐱~,𝐲T\)\(\\tilde\{\\mathbf\{x\}\},\\mathbf\{y\}\_\{T\}\)and maximizes the energy of the irrelevant remainder\(𝐱−𝐱~,𝐲T\)\(\\mathbf\{x\}\-\\tilde\{\\mathbf\{x\}\},\\mathbf\{y\}\_\{T\}\)\. The interpreter optimization is thus:

α^=argmaxα𝔼\(x,y\)\[ℰLM​\(𝐱−𝐱~,𝐲T;θ\)−ℰLM\(𝐱~,𝐲T;θ\)\]\\begin\{split\}\\hat\{\\alpha\}=\\operatorname\*\{argmax\}\_\{\\alpha\}\\mathbb\{E\}\_\{\(x,y\)\}\\Big\[&\\mathcal\{E\}\_\{\\text\{LM\}\}\(\\mathbf\{x\}\-\\tilde\{\\mathbf\{x\}\},\\mathbf\{y\}\_\{T\};\\theta\)\\\\ &\-\\mathcal\{E\}\_\{\\text\{LM\}\}\(\\tilde\{\\mathbf\{x\}\},\\mathbf\{y\}\_\{T\};\\theta\)\\Big\]\\end\{split\}\(3\)
However, masking inputs inherently causes distribution shifts\(Hsiaet al\.,[2024](https://arxiv.org/html/2608.02879#bib.bib21)\)\. To prevent this, we fine\-tune the EBM alongside the interpreter via periodic alternating optimization \(Fig\.[7](https://arxiv.org/html/2608.02879#A4.F7), App\.[D\.1](https://arxiv.org/html/2608.02879#A4.SS1)\)\. First, we update the interpreter parameters \(Eq\.[3](https://arxiv.org/html/2608.02879#S3.E3)\) given the current EBM\. Second, we periodically query the target LLM with the selected prompt subset𝐱~\\tilde\{\\mathbf\{x\}\}to generate a consistent response𝐲~\\tilde\{\\mathbf\{y\}\}\. Third, we update the EBM using this fresh pair\(𝐱~,𝐲~\)\(\\tilde\{\\mathbf\{x\}\},\\tilde\{\\mathbf\{y\}\}\)and the loss described in Section[3\.2](https://arxiv.org/html/2608.02879#S3.SS2)\. While this incurs API costs, our experiments suggest it is optional for standard benchmarks yet beneficial for robust, large\-scale deployments\.

This training process transfers the EBM’s structure to the interpreter, thus enabling standalone inference with no reliance on the EBM or need for further LLM API calls\.

## 4Experiments

Our empirical validation is performed in two stages\. First, we conduct an ablation study on the EBM’s objective to justify why our design of the surrogate landscape best captures the target LLM’s latent structure rather than relying on surface\-level heuristics\. Second, we train and evaluate the interpreter across two critical axes:attribution plausibilityandcausal faithfulness\. This ensures that the interpreter is capable of causally valid instance\-wise feature selection\. We emphasize that our framework createstask\-orientedinterpreters where each interpreter is specialized for a specific type of task \(e\.g\. general Q&A\)\. This focused scope allows for effective training even with limited data\.

### 4\.1Ablation Study: Dual\-Objective EBM

Table 1:EBM Ablation Study Results\.We evaluate the models across four proposed dimensions\.ℳFidelity\\mathcal\{M\}\_\{\\text\{Fidelity\}\}suffers from reliance on artifacts\.ℳDep\\mathcal\{M\}\_\{\\text\{Dep\}\}achieves high margins but creates an abstract latent space that hinders downstream interpretation\.ℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}balances robustness with causal precision\.I\. Core DirectiveII\. RobustnessIII\. SOTA AlignmentIV\. Causal Disentanglement\(DeYounget al\.,[2020](https://arxiv.org/html/2608.02879#bib.bib16)\)\(Gururanganet al\.,[2018](https://arxiv.org/html/2608.02879#bib.bib17)\)\(Gemini\-2\.5\-Flash\)\(Gemini\-3\-Pro\)ModelIR@1↑\\uparrowSNR↑\\uparrowArt\.Δ​E\\Delta E↓\\downarrowR1E↓\\downarrownDCG@3↑\\uparrowPrec@1↑\\uparrowAcc↑\\uparrowESM↑\\uparrowℳFidelity\\mathcal\{M\}\_\{\\text\{Fidelity\}\}67\.62%1\.45\+0\.44685\.1%0\.55326\.0%62\.18%0\.145ℳDep\\mathcal\{M\}\_\{\\text\{Dep\}\}80\.21%6\.89\-0\.0146\.9%0\.69952\.0%84\.16%0\.499\\rowcolorgray\!20ℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}84\.77%7\.15\+0\.10415\.17%0\.79681\.0%92\.40%0\.382We base our transformer EBM architecture on the ablations ofBakhtinet al\.\([2019](https://arxiv.org/html/2608.02879#bib.bib23)\)\. However, we introduce critical modifications to the embedding and the objective\. The purpose of our concept space projector for embeddings is clear: it contextualizes individual sentence embeddings within the broader text\. The objective function \(Eq\.[1](https://arxiv.org/html/2608.02879#S3.E1)\), however, is more complex\. We posit that a faithful surrogate must balance two competing requirements:fidelity, to capture the target LLM’s global distribution, andlocal dependency, to enforce local causal precision\. While we tested numerous negative samplers and loss formulations to validate this, we present three representative configurations here\.

To diagnose these configurations, we derived four dimensions by analyzing the failure modes observed across various configurations\. We map our taxonomy of model behavior to established challenges in the interpretability literature:

1. 1\.Directive Localization:Can the model isolate the primary query from background noise? \(Aligned with Rationale Extraction\(DeYounget al\.,[2020](https://arxiv.org/html/2608.02879#bib.bib16)\)\)\.
2. 2\.Robustness to Artifacts:Does the model ignore spurious correlations, such as conversational fillers? \(Aligned with Input Reduction protocols\(Fenget al\.,[2018](https://arxiv.org/html/2608.02879#bib.bib15)\)\)\.
3. 3\.Oracle Alignment:Do the rankings correlate with larger foundation models?
4. 4\.Causal Disentanglement:Can the model distinguish true antecedents from topically related distractors? \(Aligned with Counterfactual Invariance\(Kaushiket al\.,[2020](https://arxiv.org/html/2608.02879#bib.bib19)\)\)\.

#### Experimental Setup\.

To construct our corpus, we sampled prompts from the[HC3](https://huggingface.co/datasets/Hello-SimpleAI/HC3/)multi\-domain Q&A dataset\(Guoet al\.,[2023](https://arxiv.org/html/2608.02879#bib.bib11)\)to query our target LLM,GPT\-4o\-Mini, for interpretation\. We utilized original HC3 human answers andGPT\-2\-Mediumgenerations as contrastive baselines\. For efficiency, we trained compact181181M\-parameter EBMs \(∼\\sim71M trainable\) on20,00020,000pairs; preliminary tests indicate framework scalability, with larger models yielding wider energy gaps and faster convergence, which correlated with higher accuracy in distinguishing authentic pairs\. To illustrate the behavioral regimes discussed above, we analyze three representative configurations:ℳFidelity\\mathcal\{M\}\_\{\\text\{Fidelity\}\}\(λ=0\\lambda=0, mimicking standard likelihood\),ℳDep\\mathcal\{M\}\_\{\\text\{Dep\}\}\(λ=1\\lambda=1, isolating semantic links without overfitting to artifacts\), and our proposedℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}\(λ=0\.9\\lambda=0\.9, a dual\-objective balancing both\)\. Full model and sampler configurations are detailed in Appendix[B](https://arxiv.org/html/2608.02879#A2)\. Evaluations across the four dimensions \(Tab\.[1](https://arxiv.org/html/2608.02879#S4.T1)\) follow the protocols in Appendix[E](https://arxiv.org/html/2608.02879#A5)\.

#### Dimension I: Core Directive Localization\.

We assess the ability to localize primary intent \(e\.g\., the question\) amidst background context usingInterrogative Recall\(IR@1\) andSignal\-to\-Noise Ratio\(SNR\)\. We note that perfect recall is not expected, as human prompts often contain implicit or structurally ambiguous directives\. As shown in Table[1](https://arxiv.org/html/2608.02879#S4.T1),ℳFidelity\\mathcal\{M\}\_\{\\text\{Fidelity\}\}exhibits diffuse attention sensitive to background noise, whileℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}achieves a5×5\\timesSNR improvement, confirming thatlocal dependencycompels the model to prioritize semantic directives, which is ideal for our surrogate\.

#### Dimension II: Semantic Robustness\.

A prevalent pathology in neural models is the reliance on “annotation artifacts”\(Gururanganet al\.,[2018](https://arxiv.org/html/2608.02879#bib.bib17)\)\. Adopting input reduction protocols, we quantify this usingArtifact Energy ImpactandRank\-1 Error\(R1E\)\.ℳFidelity\\mathcal\{M\}\_\{\\text\{Fidelity\}\}suffers from over\-prioritizing conversational fillers\. Conversely,ℳDep\\mathcal\{M\}\_\{\\text\{Dep\}\}successfully ignores artifacts, yet its abstract latent space led to downstream interpreter collapse in our tests\.ℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}balances this trade\-off, retaining the latent space regularization necessary for training\.

#### Dimension III: Alignment with SOTA Oracles\.

UsingGemini\-2\.5\-Flashas a reference oracle, we evaluate ranking quality via nDCG@3\(Järvelin and Kekäläinen,[2002](https://arxiv.org/html/2608.02879#bib.bib18)\)andSoft Precision@1\.ℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}achieves the highest alignment, indicating that the hybrid objective produces sentence rankings most consistent with the generation patterns of state\-of\-the\-art foundation models\.

#### Dimension IV: Causal Disentanglement\.

We use counterfactual triplets\(ytarget,xcause,xdistractor\)\(y\_\{\\text\{target\}\},x\_\{\\text\{cause\}\},x\_\{\\text\{distractor\}\}\)generated byGemini\-3\-Pro\. WhileℳDep\\mathcal\{M\}\_\{\\text\{Dep\}\}produces the sharpest energy landscape, its hyper\-discrimination reduces overall accuracy\.ℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}achieves the highest accuracy, successfully balancing discriminative confidence with the robustness required to filter spurious correlations\.

\(a\) Scenario A: All Targets \(Soft Top\-1 Accuracy\)

InterpreterOursGeminiGPT\-4o4o\-MiniGPT\-JGPT\-2\\rowcolorgray\!20ESCI \(Ours\)1\.000\.750\.750\.640\.790\.70Gemini\-2\.5\-Flash0\.671\.000\.870\.750\.640\.58GPT\-4o0\.700\.911\.000\.770\.700\.68GPT\-4o\-Mini0\.600\.850\.811\.000\.660\.59GPT\-J\-6B0\.750\.690\.630\.471\.000\.82GPT\-2\-XL0\.670\.710\.670\.510\.871\.00

\(b\) Scenario A: All Targets \(nDCG Score\)

InterpreterOursGeminiGPT\-4o4o\-MiniGPT\-JGPT\-2\\rowcolorgray\!20ESCI \(Ours\)1\.000\.830\.820\.790\.830\.81Gemini\-2\.5\-Flash0\.851\.000\.890\.880\.790\.75GPT\-4o0\.810\.911\.000\.900\.840\.79GPT\-4o\-Mini0\.770\.890\.881\.000\.800\.75GPT\-J\-6B0\.850\.760\.750\.741\.000\.85GPT\-2\-XL0\.810\.780\.800\.770\.901\.00

\(c\) Scenario B: Last Target \(Soft Top\-1 Accuracy\)

InterpreterOursGeminiGPT\-4o4o\-MiniGPT\-JGPT\-2\\rowcolorgray\!20ESCI \(Ours\)1\.000\.830\.820\.710\.970\.92Gemini\-2\.5\-Flash0\.771\.000\.870\.750\.640\.58GPT\-4o0\.780\.911\.000\.820\.710\.59GPT\-4o\-Mini0\.660\.850\.831\.000\.680\.56GPT\-J\-6B0\.980\.690\.700\.491\.000\.79GPT\-2\-XL0\.960\.710\.710\.500\.851\.00

\(d\) Scenario B: Last Target \(nDCG Score\)

InterpreterOursGeminiGPT\-4o4o\-MiniGPT\-JGPT\-2\\rowcolorgray\!20ESCI \(Ours\)1\.000\.860\.850\.820\.940\.96Gemini\-2\.5\-Flash0\.831\.000\.930\.920\.820\.78GPT\-4o0\.820\.951\.000\.950\.860\.82GPT\-4o\-Mini0\.810\.930\.931\.000\.800\.75GPT\-J\-6B0\.920\.790\.780\.771\.000\.84GPT\-2\-XL0\.900\.810\.790\.770\.881\.00

Figure 3:Confusion Matrices Evaluating Interpretation Plausibility\.Each cell\(i,j\)\(i,j\)represents how well the Interpreter in rowiimatches the scores of the Oracle in columnjjfor interpreting our target LLM\.Soft Top\-1\(left\) measures if the Interpreter’s top choice appears in the Oracle’s top\-2\.nDCG\(right\) measures ranking correlation\. Our proposed ESCI model shows remarkable alignment with GPT\-J and GPT\-2, suggesting it captures the probabilistic dependencies of standard causal language modeling effectively despite its small size\.

### 4\.2Interpreter Attribution Plausibility

IDPrompt SentencesESCIGPT\-4oGeminiResponse Context & Target Sentence6\[0\] Why do people say “half a dozen” instead of “six”?
\[1\] It seems like such a common occurrence\.
\[2\] Why take the time and effort to say the extra …
\[3\] Explain like I’m five\.0\.000\.000\.001\.000\.700\.200\.100\.000\.800\.000\.000\.20Okay\! Imagine you have a box of cookies\.If you have six cookies, you can just say “six\.”But if you say “half a dozen,” it’s like saying “half of a bigger group” of cookies\. People like to use “half a dozen” because it sounds a little fancier…21\[0\] RGB lines when you take a picture of your monitor …
\[1\] Please explain like I’m five\.0\.980\.020\.800\.201\.000\.00…the camera gets a little mixed up and shows the colors in a funny way\.That’s why you see those RGB lines\!…33\[0\] fuel octane\.
\[1\] What happens if I feed my Nissan Versa …
\[3\] Please explain like I’m five\.1\.000\.000\.000\.000\.700\.300\.000\.700\.30…not work anymore\! So, it’s best to stick with the regular gas \(87 octane\) that your car is designed to use\.That way, it will run smoothly and be happy\!48\[0\] What those black lines on the road are\.
\[1\] EDIT: Sorry about the confusion, I meant …
\[3\] Explain like I’m five\.0\.000\.990\.010\.400\.100\.500\.550\.000\.45…lines show where the lanes are, while others can tell you if you can park or if you need to stop\.They are like guides that help everyone follow the rules of the road\!94\[0\] What’s the point of finding planets light years …
\[2\] Why can’t we spend money on improving …
\[3\] Please explain like I’m five\.1\.000\.000\.000\.000\.700\.300\.300\.300\.40…while also exploring space, because both are important for our future\.It’s like making sure your toys are clean and also dreaming about getting new ones\!142\[0\] The most prominent members of the current …
\[1\] Explain like I’m five\.1\.000\.000\.800\.200\.900\.10…their own thing and keep the country safe\.People are talking a lot about these ideas as they get ready to vote\!…Figure 4:Qualitative Comparison of Attribution Scores\.Left:Prompt snippets\.Middle:Attribution scores from ESCI and oracles\.Right:Target sentence from the LLM response\.ID 6: ESCI correctly attributes the simple tone to the style instruction, while oracles focus on the subject\.ID 33: ESCI over\-focuses on the topic keyword, while oracles correctly identify the question\.ID 94: A case of chaos where the target response is a broad analogy\.Evaluating explanations for black\-box models is fundamentally challenging as no dataset captures the target LLM’s true internal computation\. We must therefore first assess whether our interpreter’s attributions are broadly plausible\. Existing attribution datasets are insufficient for this task, as they are typically limited to short, simple syntactic structures\. Furthermore, scaling human annotation for open\-ended generation is both practically difficult and theoretically limited: human annotations inherently reflect human cognitive priors, not the model’s actual computational mechanics\. In contrast, high\-capacity LLMs share architectural inductive biases with our target\. This makes them a more direct proxy for its internal mechanisms\.

We thus assess semantic plausibility through a quantitative comparison against LLM “oracles,” complemented by a qualitative human analysis\. While neither represents absolute ground truth, high alignment indicates that our surrogate produces sensible attributions\. For this evaluation, we utilize our most robust EBM configuration \(ℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}\) \(see App\.[D](https://arxiv.org/html/2608.02879#A4)for detailed architecture\)\.

#### Experimental Setup\.

With our interpreter focused on the target LLM,GPT\-4o\-Mini, we interpret its outputs using a subset of200200prompt\-response pairs \(2,0002,000targets\) from our[HC3](https://huggingface.co/datasets/Hello-SimpleAI/HC3/)\-derived dataset\. Following our rationale for utilizing proxy annotators, we tasked five LLM oracles \(Gemini\-2\.5\-Flash,GPT\-4o,GPT\-4o\-Mini,GPT\-J\-6B, andGPT\-2\-XL\) to score the original prompt sentences based on their causal contribution to every target response sentence\. We then compare our Energy\-Based Concept\-Level Surrogate Interpreter \(ESCI\) against these oracles to measure attribution plausibility\. We report results on two scenarios:Scenario A\(averaging across all target sentences\) andScenario B\(focusing on the final sentence, which often contains the conclusion\)\.

#### Quantitative Assessment\.

Figure[3](https://arxiv.org/html/2608.02879#S4.F3)presents the pairwise alignment between our interpreter and the five oracles\. We employnDCGto measure ranking quality andSoft Top\-1 Accuracyto measure top choice alignment while accounting for ambiguity in text attribution \(see App\.[F](https://arxiv.org/html/2608.02879#A6)for details\)\.

Despite having orders of magnitude fewer parameters \(∼\\sim71M trainable,∼\\sim110M loaded\), ESCI achieves competitive plausibility against vastly different oracle architectures\. In Scenario B \(Fig\.[3](https://arxiv.org/html/2608.02879#S4.F3)c/d\), ESCI aligns closely with bothGPT\-J\-6B, a standard causal language model, andGemini\-2\.5\-Flash, a heavily instruction\-tuned system\. This suggests that our energy landscape effectively internalizes the causal mechanics of autoregressive generation\. We observe that ESCI yields sparse, confident scores, contrasting with the diffuse distributions of instruction\-tuned oracles\. Additionally, unlike white\-box models—which often struggle to produce valid probability distributions and require post\-processing intervention—ESCI operates as a robust, standalone tool, while matching them in attribution strength\. It sharply isolatesnecessarydependencies, minimizing the ambiguity typical of generative baselines\.

#### Qualitative Case Studies\.

Figure[4](https://arxiv.org/html/2608.02879#S4.F4)provides a granular look at specific attribution behaviors, reflecting trends identified through a detailed manual analysis of all2,0002,000combinations by two human evaluators\. In cases of clear semantic mapping, ESCI aligns perfectly with oracles\. Disagreements, however, are revealing\. In Sample3333, ESCI attributes the simplified output to the topic keyword, whereas oracles point to the user question\. This suggests ESCI may sometimes over\-prioritize the global topic over query nuances\. Conversely, Sample66highlights ESCI’s strength: the target sentence is a pure stylistic simplification\. ESCI correctly identifies the instruction “Explain like I’m five” as the cause, whereas oracles fixate on the semantic content\. This confirms ESCI’s ability to disentangle stylistic drivers from semantic ones, a critical capability for interpreting instruction\-tuned models\. Finally, Sample9494illustrates total chaos, where the target is an analogy synthesizing parts of the prompt, leading to divergent interpretations\.

### 4\.3Interpreter Causal Faithfulness

While Section[4\.2](https://arxiv.org/html/2608.02879#S4.SS2)plausibility confirms that our interpreter aligns with oracles—which are likely to emulate our target’s internal mechanisms due to architectural similarity—it does not guarantee that the selections are the true drivers of the LLM’s internal computation\. To circumvent the limitations of black\-box access and successfully observe model behavior to ensure our feature selection is valid, we must measure the consequences of intervening on the prompt\. We adapt the standard metrics ofSufficiencyandComprehensiveness\(DeYounget al\.,[2020](https://arxiv.org/html/2608.02879#bib.bib16)\)for open\-ended generation by evaluating semantic similarity rather than discrete classification probabilities\(Atanasovaet al\.,[2023](https://arxiv.org/html/2608.02879#bib.bib12)\)\.

We defineGenerative Sufficiencyas the degree to which a target output is regenerated usingonlythe selected prompt sentences\. Conversely,Generative Comprehensivenessmeasures the extent to which the target is lost when those sentences areremoved\. A faithful interpreter maximizes the former and minimizes the latter, yielding a positiveFaithfulness Gap\. We quantify this using cosine similarity between the embeddings of the original target sentence and the counterfactual responses\.

#### Experimental Setup\.

Using the same200200pairs \(2,0002,000targets\) from Section[4\.2](https://arxiv.org/html/2608.02879#S4.SS2), we benchmark ESCI against four baselines\. To validate our prior assumption regarding oracles, we compare against the target’s self\-assessment \(GPT\-4o\-Mini\) and a stronger proxy oracle \(GPT\-4o\); for each, sentences are selected using a max\-ratio thresholding strategy on importance rankings\. To establish a black\-box upper bound, we implement a sentence\-level adaptation of LIME\. Finally, we include a sparse Random baseline as lower bound to ensure metrics penalize arbitrary selections\. To quantify faithfulness, we measure semantic retention by computing the cosine similarity between the embedded counterfactual responses and original target sentences using theall\-mpnet\-base\-v2embedding model\. Detailed formulations are provided in Appendix[G](https://arxiv.org/html/2608.02879#A7)\.

Table 2:Causal Faithfulness Evaluation\.We measure the semantic similarity of the LLM’s response to the target sentence under strict interventions\.Sufficiency:Prompting withonlyselected sentences\.Comprehensiveness:Prompting witheverything butselected sentences\.Gap:The net causal contribution\.InterpreterSuff\.\(↑\\uparrow\)Comp\.\(↓\\downarrow\)Gap\(↑\\uparrow\)LIME \(Baseline\)0\.4390\.1940\.245\\rowcolorgray\!20ESCI \(Ours\)0\.4090\.2140\.195GPT\-4o\-Mini0\.4070\.2150\.192GPT\-4o0\.4090\.2350\.175Random \(Baseline\)0\.2310\.377\-0\.146![Refer to caption](https://arxiv.org/html/2608.02879v1/x4.png)Figure 5:Computational Scalability\.\(Left\) Cumulative target LLM API calls required as the number of interpreted prompts scales\. \(Right\) ESCI eliminates target API queries during inference entirely, enabling high\-volume deployability\.
#### Analysis\.

As shown in Table[2](https://arxiv.org/html/2608.02879#S4.T2), Random baseline yields a negative gap, confirming that arbitrary selection degrades generation\. Conversely, sentence\-level LIME establishes an upper bound for faithfulness through exhaustive local perturbations\. However, this precision comes at a prohibitive inference cost\. As illustrated in Figure[5](https://arxiv.org/html/2608.02879#S4.F5), LIME requires dozens of API queries per prompt\. While suitable for isolated, small\-scale diagnostics, it fails to scale for continuous deployment\. Against these extremes, our ESCI matches high\-capacity LLM judges\. While slightly trailing LIME’s precision, ESCI operates strictly inO​\(1\)O\(1\)inference time with zero additional API queries\. Despite a one\-time pre\-training investment \(e\.g\.,2020K queries\), the computational cost amortizes rapidly\. It breaks even after evaluating merely400400prompts and scales infinitely without further API overhead\.

This efficiency confirms our globally trained energy landscape captures the target’s causal mechanics using vastly fewer parameters\. Although no model achieves near\-zeroComprehensiveness—as LLMs readily reconstruct content using background knowledge—ESCI successfully isolates necessary causal antecedents\. Amidst this generative noise, it provides a Pareto\-optimal balance of computational efficiency and attribution fidelity\.

## 5Conclusion

We introduced a concept\-level interpreter for black\-box LLMs that shifts post\-hoc attribution from tokens to sentences\. By modeling the target LLM’s generation dynamics as a differentiable energy landscape, we trained a standalone interpreter that requires no API queries during inference\. Our analysis confirms that this surrogate must balanceglobal fidelitywithlocal dependencyto reflect the target\. Empirically, ESCI isolates causal prompt sentences effectively; it approaches the precision of exhaustive perturbation methods like LIME, but operates with the efficiency required for deployment\. This establishes energy\-based surrogates as a scalable pathway for diagnosing model behaviors\.

## 6Limitations and Future Work

#### Computational Trade\-offs\.

A primary limitation of our framework is the overhead required during the pre\-training phase\. While our approach achievesO​\(1\)O\(1\)efficiency at inference time with zero additional API queries, it shifts the computational burden entirely to training\. Furthermore, the framework is inherently task\-oriented; adapting it to new applications requires training distinct surrogates and performing extensive, task\-specific hyperparameter tuning, which is notoriously difficult and time\-consuming to optimize \(see App\.[B](https://arxiv.org/html/2608.02879#A2)and[D](https://arxiv.org/html/2608.02879#A4)\)\. Consequently, while the one\-time training cost for a single task is manageable \(e\.g\., under3030hours on free\-tier GPUs\), the cumulative manual and computational effort required to discover optimal configurations across diverse tasks presents a significant barrier to out\-of\-the\-box generalizability\. Ultimately, this upfront investment pays off by yielding a standalone interpreter suitable for high\-volume, real\-time analysis, though less accessible for users unable to perform the initial pre\-training\.

#### Generalizability and Scaling Limits\.

Due to resource constraints, our validation focused on standard Q&A tasks using compact surrogate models \(∼\\sim181M parameters\) to interpretGPT\-4o\-Mini\. Future work should expand to diverse domains, including complex reasoning and open\-ended generation \(e\.g\.,TellMeWhy,WikiText\), and target distinct architectures beyond the GPT family\. Additionally, while our scaling experiments suggest improved performance with larger EBMs, the efficacy of the energy landscape in capturing long\-range dependencies within massive context windows \(e\.g\.,128​k\+128k\+tokens\) remains to be verified\. Investigating the scaling laws of the interpreter is crucial to ensure robust attribution in high\-complexity regimes\.

#### Lack of Ground\-Truth Mechanistic Validation\.

A fundamental limitation of the black\-box setting is the reliance on probabilistic oracles \(e\.g\.,GPT\-4o\) rather than deterministic ground truth\. While our sufficiency metrics demonstrate causal efficacy, high alignment with an oracle does not guarantee the best fidelity to the target’s internal computation\. Consequently, our current results confirmbehavioralsimulation rather thanmechanisticalignment\. Validating the latter requires future benchmarking against open\-weights architectures \(e\.g\., Llama 3, Pythia\), where surrogate attributions can be directly compared with white\-box signals likeIntegrated Gradientsor attention maps\. This would provide deeper theoretical insight to quantify how closely the surrogate energy landscape approximates the target model’s true internal computational paths\.

#### Human\-Centric Utility\.

While sufficiency and comprehensiveness quantify causal faithfulness, they remain automated proxies\. A key limitation is the assumption that causal accuracy automatically equates to human intelligibility\. Although we provide preliminary qualitative analysis, validating the practical utility of ESCI requires rigorous subject studies\. Such evaluations are critical to corroborate our plausibility findings against human judgment—providing a grounded check on probabilistic oracles—and to assess downstream usability\. Specifically, we aim to measure whether these explanations effectively aid users in high\-stakes auditing tasks, such as detecting hallucinations, identifying bias, and verifying safety compliance\.

#### Scope of Application and Optimization\.

Our current evaluation is confined to the diagnostic utility of attribution, leaving the framework’s broader potential for downstream optimization empirically unverified\. Theoretically, the identification ofnecessarysentences enables prompt optimization—automatically pruning irrelevant context to reduce token costs without degrading output quality\. Similarly, the energy landscape offers a mechanism to audit Chain\-of\-Thought \(CoT\) reasoning, potentially filtering unfaithful or confabulated intermediate steps\. However, given the complexity of these domains, further experimentation is strictly necessary to determine if the interpreter’s performance can be sufficiently optimized to maintain robustness when deployed on such high\-dimensional generative tasks\.

## References

- Faithfulness tests for natural language explanations\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),pp\. 283–294\.External Links:[Link](https://aclanthology.org/2023.acl-short.25)Cited by:[§4\.3](https://arxiv.org/html/2608.02879#S4.SS3.p1.1)\.
- A\. Bakhtin, Y\. Deng, S\. Gross, M\. Ott, M\. Ranzato, and A\. Szlam \(2021\)Residual energy\-based models for text\.Journal of Machine Learning Research22\(40\),pp\. 1–41\.External Links:[Link](http://jmlr.org/papers/v22/20-326.html)Cited by:[§2\.2](https://arxiv.org/html/2608.02879#S2.SS2.p2.1)\.
- A\. Bakhtin, S\. Gross, M\. Ott, Y\. Deng, M\. Ranzato, and A\. Szlam \(2019\)Real or fake? learning to discriminate machine from human generated text\.External Links:[Link](https://arxiv.org/abs/1906.03351),1906\.03351Cited by:[§2\.2](https://arxiv.org/html/2608.02879#S2.SS2.p3.1),[§4\.1](https://arxiv.org/html/2608.02879#S4.SS1.p1.1)\.
- S\. Balasubramanian, S\. Basu, K\. Goswami, R\. Rossi, V\. Manjunatha, R\. Santhosh, R\. Zhang, S\. Feizi, and N\. Lipka \(2025\)Decomposition\-enhanced training for post\-hoc attributions in language models\.External Links:2510\.25766,[Link](https://arxiv.org/abs/2510.25766)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p2.1)\.
- L\. Barrault, P\. Duquenne, M\. Elbayad, A\. Kozhevnikov, B\. Alastruey, P\. Andrews, M\. Coria, G\. Couairon, M\. R\. Costa\-jussà, D\. Dale, H\. Elsahar, K\. Heffernan, J\. M\. Janeiro, T\. Tran, C\. Ropers, E\. Sánchez, R\. San Roman, A\. Mourachko, S\. Saleem, and H\. Schwenk \(2024\)Large concept models: language modeling in a sentence representation space\.External Links:[Link](https://arxiv.org/abs/2412.08821),2412\.08821Cited by:[§2\.3](https://arxiv.org/html/2608.02879#S2.SS3.p2.1)\.
- S\. Bhattacharyya, A\. Rooshenas, S\. Naskar, S\. Sun, M\. Iyyer, and A\. McCallum \(2021\)Energy\-based reranking: improving neural machine translation using energy\-based models\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),pp\. 4528–4537\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.acl-long.349),[Link](https://aclanthology.org/2021.acl-long.349/)Cited by:[§2\.2](https://arxiv.org/html/2608.02879#S2.SS2.p3.1)\.
- H\. Chefer, S\. Gur, and L\. Wolf \(2021\)Transformer interpretability beyond attention visualization\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 782–791\.External Links:[Document](https://dx.doi.org/10.1109/CVPR46437.2021.00084),[Link](https://openaccess.thecvf.com/content/CVPR2021/html/Chefer_Transformer_Interpretability_Beyond_Attention_Visualization_CVPR_2021_paper.html)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p1.1)\.
- J\. Chen, L\. Song, M\. J\. Wainwright, and M\. I\. Jordan \(2018\)Learning to explain: an information\-theoretic perspective on model interpretation\.InProceedings of the 35th International Conference on Machine Learning,pp\. 883–892\.External Links:[Link](https://arxiv.org/abs/1802.07814)Cited by:[Appendix C](https://arxiv.org/html/2608.02879#A3.p1.4),[§1](https://arxiv.org/html/2608.02879#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p2.1)\.
- L\. Chen, X\. Yin, and F\. Toni \(2026\)Latent debate: a surrogate framework for interpreting llm thinking\.External Links:2512\.01909,[Link](https://arxiv.org/abs/2512.01909)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p2.1)\.
- C\. Clemmer, J\. Ding, and Y\. Feng \(2024\)PreciseDebias: an automatic prompt engineering approach for generative ai to mitigate image demographic biases\.In2024 IEEE/CVF Winter Conference on Applications of Computer Vision \(WACV\),pp\. 8581–8590\.External Links:[Document](https://dx.doi.org/10.1109/WACV57701.2024.00840)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p3.1)\.
- Y\. Deng, A\. Bakhtin, M\. Ott, A\. Szlam, and M\. Ranzato \(2020\)Residual energy\-based models for text generation\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=B1l4SgHKDH)Cited by:[§2\.2](https://arxiv.org/html/2608.02879#S2.SS2.p2.1)\.
- J\. Devlin, M\. Chang, K\. Lee, and K\. Toutanova \(2019\)BERT: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),pp\. 4171–4186\.External Links:[Document](https://dx.doi.org/10.18653/v1/N19-1423),[Link](https://aclanthology.org/N19-1423/)Cited by:[§2\.3](https://arxiv.org/html/2608.02879#S2.SS3.p2.1)\.
- J\. DeYoung, S\. Jain, N\. F\. Rajani, E\. Lehman, C\. Xiong, R\. Socher, and B\. C\. Wallace \(2020\)ERASER: A benchmark to evaluate rationalized NLP models\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,pp\. 4443–4458\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.408),[Link](https://aclanthology.org/2020.acl-main.408/)Cited by:[1st item](https://arxiv.org/html/2608.02879#A5.I3.i1.p1.3),[§2\.3](https://arxiv.org/html/2608.02879#S2.SS3.p1.1),[item 1](https://arxiv.org/html/2608.02879#S4.I1.i1.p1.1),[§4\.3](https://arxiv.org/html/2608.02879#S4.SS3.p1.1),[Table 1](https://arxiv.org/html/2608.02879#S4.T1.18.14.2)\.
- F\. Doshi\-Velez and B\. Kim \(2017\)Towards a rigorous science of interpretable machine learning\.External Links:[Link](https://arxiv.org/abs/1702.08608),1702\.08608Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p1.1)\.
- J\. Enouen, H\. Nakhost, S\. Ebrahimi, S\. Arik, Y\. Liu, and T\. Pfister \(2024\)TextGenSHAP: scalable post\-hoc explanations in text generation with long documents\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 13984–14011\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.832),[Link](https://aclanthology.org/2024.findings-acl.832/)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p2.1)\.
- S\. Feng, E\. Wallace, A\. Grissom II, M\. Iyyer, P\. Rodriguez, and J\. Boyd\-Graber \(2018\)Pathologies of neural models make interpretations difficult\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 3719–3728\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1407),[Link](https://aclanthology.org/D18-1407/)Cited by:[1st item](https://arxiv.org/html/2608.02879#A5.I4.i1.p1.1),[item 2](https://arxiv.org/html/2608.02879#S4.I1.i2.p1.1)\.
- G\. Frege \(1991\)The foundations of arithmetic\.Wiley\.External Links:ISBN 9780631126942,[Link](https://books.google.com/books?id=MpiFEAAAQBAJ)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p4.5)\.
- D\. Friedman, A\. K\. Lampinen, L\. Dixon, D\. Chen, and A\. Ghandeharioun \(2024\)Interpretability illusions in the generalization of simplified models\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 14035–14059\.External Links:[Link](https://proceedings.mlr.press/v235/friedman24a.html)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p5.1)\.
- L\. Gao, T\. D\. la Tour, H\. Tillman, G\. Goh, R\. Troll, A\. Radford, I\. Sutskever, J\. Leike, and J\. Wu \(2025\)Scaling and evaluating sparse autoencoders\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=tcsZt9ZNKD)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p4.5),[§2\.3](https://arxiv.org/html/2608.02879#S2.SS3.p1.1)\.
- B\. Guo, X\. Zhang, Z\. Wang, M\. Jiang, J\. Nie, Y\. Ding, J\. Yue, and Y\. Wu \(2023\)How close is chatgpt to human experts? comparison corpus, evaluation, and detection\.External Links:2301\.07597,[Link](https://arxiv.org/abs/2301.07597)Cited by:[§4\.1](https://arxiv.org/html/2608.02879#S4.SS1.SSS0.Px1.p1.9)\.
- S\. Gururangan, S\. Swayamdipta, O\. Levy, R\. Schwartz, S\. Bowman, and N\. A\. Smith \(2018\)Annotation artifacts in natural language inference data\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 107–112\.External Links:[Document](https://dx.doi.org/10.18653/v1/N18-2017),[Link](https://aclanthology.org/N18-2017/)Cited by:[§E\.3](https://arxiv.org/html/2608.02879#A5.SS3.p1.1),[§4\.1](https://arxiv.org/html/2608.02879#S4.SS1.SSS0.Px3.p1.3),[Table 1](https://arxiv.org/html/2608.02879#S4.T1.18.14.3)\.
- S\. Hackmann, H\. Mahmoudian, M\. Steadman, and M\. Schmidt \(2024\)Word importance explains how prompts affect language model outputs\.External Links:[Link](https://arxiv.org/abs/2403.03028),2403\.03028Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p2.1)\.
- Y\. Hao, L\. Dong, F\. Wei, and K\. Xu \(2021\)Self\-attention attribution: interpreting information interactions inside transformer\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 12963–12971\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v35i14.17533),[Link](https://ojs.aaai.org/index.php/AAAI/article/view/17533)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p1.1)\.
- M\. Honnibal and I\. Montani \(2017\)SpaCy 2: natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing\.InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,External Links:[Link](https://sentometrics-research.com/publication/72)Cited by:[§3\.1](https://arxiv.org/html/2608.02879#S3.SS1.p1.4)\.
- J\. Hsia, D\. Pruthi, A\. Singh, and Z\. Lipton \(2024\)Goodhart’s law applies to NLP’s explanation benchmarks\.InFindings of the Association for Computational Linguistics,pp\. 1322–1335\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-eacl.88),[Link](https://aclanthology.org/2024.findings-eacl.88/)Cited by:[§3\.3](https://arxiv.org/html/2608.02879#S3.SS3.p4.3)\.
- S\. Jain and B\. C\. Wallace \(2019\)Attention is not Explanation\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),pp\. 3543–3556\.External Links:[Document](https://dx.doi.org/10.18653/v1/N19-1357),[Link](https://aclanthology.org/N19-1357/)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p1.1)\.
- E\. Jang, S\. Gu, and B\. Poole \(2017\)Categorical reparameterization with gumbel\-softmax\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=rkE3y85ee)Cited by:[Appendix C](https://arxiv.org/html/2608.02879#A3.p1.4),[§3\.3](https://arxiv.org/html/2608.02879#S3.SS3.p2.4)\.
- K\. Järvelin and J\. Kekäläinen \(2002\)Cumulated gain\-based evaluation of ir techniques\.ACM Trans\. Inf\. Syst\.20\(4\),pp\. 422–446\.External Links:[Document](https://dx.doi.org/10.1145/582415.582418),[Link](https://doi.org/10.1145/582415.582418)Cited by:[§4\.1](https://arxiv.org/html/2608.02879#S4.SS1.SSS0.Px4.p1.1)\.
- D\. Kaushik, E\. Hovy, and Z\. Lipton \(2020\)Learning the difference that makes a difference with counterfactually\-augmented data\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Sklgs0NFvr)Cited by:[item 4](https://arxiv.org/html/2608.02879#S4.I1.i4.p1.1)\.
- B\. Kim, M\. Wattenberg, J\. Gilmer, C\. Cai, J\. Wexler, F\. Viegas, and R\. Sayres \(2018\)Interpretability beyond feature attribution: quantitative testing with concept activation vectors \(TCAV\)\.InProceedings of the 35th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.80,pp\. 2668–2677\.External Links:[Link](https://proceedings.mlr.press/v80/kim18d.html)Cited by:[§2\.3](https://arxiv.org/html/2608.02879#S2.SS3.p1.1)\.
- W\. Kintsch \(1998\)Comprehension: a paradigm for cognition\.Comprehension: A Paradigm for Cognition,Cambridge University Press\.External Links:ISBN 9780521629867,LCCN 97026406,[Link](https://books.google.com/books?id=LuycnLrY3k8C)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p4.5)\.
- P\. W\. Koh and P\. Liang \(2017\)Understanding black\-box predictions via influence functions\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 1885–1894\.External Links:[Link](https://proceedings.mlr.press/v70/koh17a.html)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p1.1)\.
- Y\. LeCun, S\. Chopra, R\. Hadsell, M\. Ranzato, and F\. Huang \(2006\)A tutorial on energy\-based learning\.Predicting structured data1\(0\),pp\. 191–246\.External Links:[Link](http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf)Cited by:[§2\.2](https://arxiv.org/html/2608.02879#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2608.02879#S3.SS2.p1.1)\.
- J\. Li, W\. Monroe, and D\. Jurafsky \(2017\)Understanding neural networks through representation erasure\.External Links:[Link](https://arxiv.org/abs/1612.08220),1612\.08220Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p1.1)\.
- A\. Lochab and R\. Zhang \(2025\)Energy\-based reward models for robust language model alignment\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=x6evCULIOQ)Cited by:[§2\.2](https://arxiv.org/html/2608.02879#S2.SS2.p3.1)\.
- S\. M\. Lundberg and S\. Lee \(2017\)A unified approach to interpreting model predictions\.InAdvances in Neural Information Processing Systems,Vol\.30,pp\. 4765–4774\.External Links:[Link](https://proceedings.neurips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p2.1)\.
- N\. Reimers and I\. Gurevych \(2019\)Sentence\-BERT: sentence embeddings using Siamese BERT\-networks\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),pp\. 3982–3992\.External Links:[Document](https://dx.doi.org/10.18653/v1/D19-1410),[Link](https://aclanthology.org/D19-1410/)Cited by:[§2\.3](https://arxiv.org/html/2608.02879#S2.SS3.p2.1),[§3\.1](https://arxiv.org/html/2608.02879#S3.SS1.p1.4)\.
- M\. T\. Ribeiro, S\. Singh, and C\. Guestrin \(2016\)"Why should I trust you?": explaining the predictions of any classifier\.InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,pp\. 1135–1144\.External Links:[Document](https://dx.doi.org/10.1145/2939672.2939778),[Link](https://dl.acm.org/doi/10.1145/2939672.2939778)Cited by:[2nd item](https://arxiv.org/html/2608.02879#A7.I1.i2.p1.8),[§1](https://arxiv.org/html/2608.02879#S1.p2.1),[§1](https://arxiv.org/html/2608.02879#S1.p5.1),[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p2.1)\.
- J\. Schneider \(2024\)Explainable generative ai \(GenXAI\): a survey, conceptualization, and research agenda\.Artificial Intelligence Review57\(11\),pp\. 289\.External Links:[Document](https://dx.doi.org/10.1007/s10462-024-10916-x),[Link](https://link.springer.com/article/10.1007/s10462-024-10916-x)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p2.1)\.
- S\. F\. Seyyedsalehi, M\. S\. Baghshah, and H\. R\. Rabiee \(2024\)SOInter: a novel deep energy\-based interpretation method for explaining structured output models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Fn655mJ4bv)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p2.1)\.
- A\. Shrikumar, P\. Greenside, and A\. Kundaje \(2017\)Learning important features through propagating activation differences\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 3145–3153\.External Links:[Link](https://proceedings.mlr.press/v70/shrikumar17a.html)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p1.1)\.
- K\. Simonyan, A\. Vedaldi, and A\. Zisserman \(2014\)Deep inside convolutional networks: visualising image classification models and saliency maps\.InInternational Conference on Learning Representations \(Workshop Track\),External Links:[Link](https://arxiv.org/abs/1312.6034)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p1.1)\.
- C\. Sun, T\. Oikarinen, B\. Ustun, and T\. Weng \(2025\)Concept bottleneck large language models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=RC5FPYVQaH)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p4.5),[§2\.3](https://arxiv.org/html/2608.02879#S2.SS3.p1.1)\.
- M\. Sundararajan, A\. Taly, and Q\. Yan \(2017\)Axiomatic attribution for deep networks\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 3319–3328\.External Links:[Link](https://proceedings.mlr.press/v70/sundararajan17a.html)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p1.1)\.
- L\. Tu, R\. Y\. Pang, S\. Wiseman, and K\. Gimpel \(2020\)ENGINE: energy\-based inference networks for non\-autoregressive machine translation\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,pp\. 2819–2826\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.251),[Link](https://aclanthology.org/2020.acl-main.251/)Cited by:[§2\.2](https://arxiv.org/html/2608.02879#S2.SS2.p2.1)\.
- M\. Turpin, J\. Michael, E\. Perez, and S\. R\. Bowman \(2023\)Language models don’t always say what they think: unfaithful explanations in chain\-of\-thought prompting\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 71725–71739\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/ed3fea9033a80fea1376299fa7863f4a-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p3.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou \(2022\)Chain of thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 24824–24837\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p3.1)\.
- Q\. Xie, X\. Ma, Z\. Dai, and E\. Hovy \(2017\)An interpretable knowledge transfer model for knowledge base completion\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 950–962\.External Links:[Document](https://dx.doi.org/10.18653/v1/P17-1088),[Link](https://aclanthology.org/P17-1088/)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p1.1)\.
- K\. Xu, J\. Ba, R\. Kiros, K\. Cho, A\. Courville, R\. Salakhudinov, R\. Zemel, and Y\. Bengio \(2015\)Show, attend and tell: neural image caption generation with visual attention\.InProceedings of the 32nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.37,pp\. 2048–2057\.External Links:[Link](https://proceedings.mlr.press/v37/xuc15.html)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p1.1)\.
- M\. Xu, T\. Geffner, K\. Kreis, W\. Nie, Y\. Xu, J\. Leskovec, S\. Ermon, and A\. Vahdat \(2025\)Energy\-based diffusion language models for text generation\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=sL2F9YCMXf)Cited by:[§2\.2](https://arxiv.org/html/2608.02879#S2.SS2.p2.1)\.
- K\. Yin and G\. Neubig \(2022\)Interpreting language models with contrastive explanations\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,pp\. 184–198\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.14),[Link](https://aclanthology.org/2022.emnlp-main.14/)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p2.1)\.
- Z\. Zhao and B\. Shan \(2024\)ReAGent: a model\-agnostic feature attribution method for generative language models\.InProceedings of the AAAI 2024 Workshop on Responsible Language Models \(ReLM\),External Links:[Link](https://arxiv.org/abs/2402.00794)Cited by:[§2\.1](https://arxiv.org/html/2608.02879#S2.SS1.p2.1)\.
- Y\. Zhou, A\. I\. Muresanu, Z\. Han, K\. Paster, S\. Pitis, H\. Chan, and J\. Ba \(2023\)Large language models are human\-level prompt engineers\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=92gvk82DE-)Cited by:[§1](https://arxiv.org/html/2608.02879#S1.p3.1)\.

## Appendix APreprocessing Details

To ensure compatibility with fixed\-dimensional attention mechanisms, we normalize sentence counts during preprocessing\. We define a task\-dependent hyperparameter,NmaxN\_\{\\text\{max\}\}, representing the maximum sequence length\. After input and output sentences are extracted and embedded, the sequences are padded with a learnable placeholder token or truncated to strictly match this length\. This results in dense input tensorsSin,Sout∈ℝNmax×dS^\{\\text\{in\}\},S^\{\\text\{out\}\}\\in\\mathbb\{R\}^\{N\_\{\\text\{max\}\}\\times d\}, whereddis the embedding dimension of the Sentence\-BERT model \(768 forall\-mpnet\-base\-v2\)\.

## Appendix BEnergy Network: Pre\-training Details

![Refer to caption](https://arxiv.org/html/2608.02879v1/x5.png)Figure 6:Pre\-training Pipeline of the EBM\.The architecture projects SentenceBERT embeddings into a dynamic concept space via self\-attention, followed by a cross\-attention mechanism to model input\-output interactions\. An MLP aggregates these features to compute a scalar energy scoreℰLM​\(𝐱,𝐲;θ\)\\mathcal\{E\}\_\{\\text\{LM\}\}\(\\mathbf\{x\},\\mathbf\{y\};\\theta\)\. The model is optimized using a dual\-objective InfoNCE loss:fidelitycontrasts authentic pairs\(𝐱,𝐲\)\(\\mathbf\{x\},\\mathbf\{y\}\)against global negatives in𝒩i\\mathcal\{N\}\_\{i\}\(e\.g\., human responses\) to learn the target distribution, whilelocal dependencycontrasts partial sequences against batch negatives in𝒩i\\mathcal\{N\}\_\{i\}to enforce fine\-grained causal precision\. Thus, the weighted sum of InfoNCE losses minimizes the energy of authentic pairs while maximizing the energy of corrupted samples\.The Energy\-Based Model’s training pipeline is illustrated in Figure[6](https://arxiv.org/html/2608.02879#A2.F6)\. We trained all EBM variants on dual NVIDIA T4 GPUs \(provided by Kaggle’s free tier\) using the AdamW optimizer\. The training process for each EBM required approximately2525hours\. Table[3](https://arxiv.org/html/2608.02879#A2.T3)details the specific hyperparameters used for the final Hybrid model\.

Table 3:Hyperparameters for theℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}EBM\.ParameterValueArchitectureEncoder Modelall\-mpnet\-base\-v2Frozen Parameters110110MTrainable Parameters7171MTotal Parameters181181MProjection Dimension \(dmodeld\_\{\\text\{model\}\}\)768768Self\-Attention Layers22Cross\-Attention Layers66Attention Heads88Dropout Rate0\.10\.1MLP Layers22MLP Hidden Factor22OptimizationEpochs5050Batch Size1616Learning Rate3​e−53e^\{\-5\}SchedulerLinear WarmupWarmup Steps200200LossLoss FunctionInfoNCELocal Dependency Weight \(λ\\lambda\)0\.90\.9InfoNCE Temperature \(τ\\tau\)0\.10\.1Margin0\.50\.5Negative Candidates \(KK\)55DataDataset Size20,00020,000samplesValidation Split10%10\\%Max Sentence Count1616\(Learnable Padding\)#### Model Configurations\.

To assess the impact of our dual objectives, we trained three distinct EBM variants\.ℳFidelity\\mathcal\{M\}\_\{\\text\{Fidelity\}\}\(λ=0\\lambda=0\) mimics standard likelihood modeling by contrasting positive pairs against only global corruptions\.ℳDep\\mathcal\{M\}\_\{\\text\{Dep\}\}\(λ=1\\lambda=1\) learns exclusively by contrasting partial segments, forcing the model to identify semantic links without overfitting to the surface artifacts of a single authentic pair\. Finally,ℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}\(λ=0\.9\\lambda=0\.9\) combines these approaches; it relies on dependency samplers to capture structural logic while using the fidelity signal to regularize the latent space\.

#### Negative Sampling Strategies\.

The training objective relies on a diverse set of negative samples to shape the energy landscape\. The specific samplers used for each configuration are:

- •ℳFidelity\\mathcal\{M\}\_\{\\text\{Fidelity\}\}Samplers: - –response\_human: Swaps the LLM response with a human\-written answer from the HC3 dataset\. - –response\_other\_lm: Swaps response with aGPT\-2 Mediumoutput\. - –response\_sentence\_masking: Masks a random number of sentences in the LLM’s response\. - –prompt\_sentence\_masking: Masks a random number of sentences in the data pair’s prompt\. - –off\_topic: Swaps response or prompt with one from a different pair in the batch\.
- •ℳDep\\mathcal\{M\}\_\{\\text\{Dep\}\}Samplers: - –partial\_response\_dep: Contrasts the authentic partial response \(positive\) against a mismatched partial response from the batch \(negative\) given the same partial prompt\. This forces the model to verify that the output is a specific logical continuation of the input concepts\. - –partial\_prompt\_dep: Contrasts the authentic partial prompt \(positive\) against a mismatched partial prompt from the batch \(negative\) given the same partial response\. This ensures that the response is causally attributed to the correct input antecedents rather than generic topics\.
- •ℳHybrid\\mathcal\{M\}\_\{\\text\{Hybrid\}\}Samplers: - –partial\_response\_dep: SeeℳDep\\mathcal\{M\}\_\{\\text\{Dep\}\}\. - –partial\_prompt\_dep: SeeℳDep\\mathcal\{M\}\_\{\\text\{Dep\}\}\. - –response\_human: SeeℳFidelity\\mathcal\{M\}\_\{\\text\{Fidelity\}\}\. - –response\_other\_lm: SeeℳFidelity\\mathcal\{M\}\_\{\\text\{Fidelity\}\}\.

## Appendix CDifferentiable Top\-KKSentence Selection via Gumbel–Softmax

The interpreter network aims to identify theKKmost important sentences from the input𝐱\\mathbf\{x\}influential in generating the target𝐲T\\mathbf\{y\}\_\{T\}\. Since selecting top\-KKindices is discrete and non\-differentiable, we apply a continuous relaxation to the subset sampling via the Gumbel\-Softmax trick\(Janget al\.,[2017](https://arxiv.org/html/2608.02879#bib.bib22); Chenet al\.,[2018](https://arxiv.org/html/2608.02879#bib.bib2)\)to enable end\-to\-end training\.

Let the interpreter function produce a vector of unnormalized relevance logits𝐳∈ℝn\\mathbf\{z\}\\in\\mathbb\{R\}^\{n\}for thenninput sentences, denoted aszi=\(ℐ​𝒩​\(𝐱,𝐲T;α\)\)iz\_\{i\}=\(\\mathcal\{IN\}\(\\mathbf\{x\},\\mathbf\{y\}\_\{T\};\\alpha\)\)\_\{i\}\. To introduce stochasticity, we first generate standard Gumbel noisegig\_\{i\}from i\.i\.d\. uniform samplesui∼Uniform​\(0,1\)u\_\{i\}\\sim\\mathrm\{Uniform\}\(0,1\)as follows:

gi=−log⁡\(−log⁡ui\),i=1,…,n\.g\_\{i\}=\-\\log\(\-\\log u\_\{i\}\),\\quad i=1,\\ldots,n\.\(C1\)
Given a temperatureτ\>0\\tau\>0, a single continuous relaxation of a one\-hot vector, denoted asc∈Δn−1c\\in\\Delta^\{n\-1\}, is computed via the softmax function:

ci=exp⁡\(\(zi\+gi\)/τ\)∑j=1nexp⁡\(\(zj\+gj\)/τ\),i=1,…,n\.c\_\{i\}=\\frac\{\\exp\(\(z\_\{i\}\+g\_\{i\}\)/\\tau\)\}\{\\sum\_\{j=1\}^\{n\}\\exp\(\(z\_\{j\}\+g\_\{j\}\)/\\tau\)\},\\quad i=1,\\ldots,n\.\(C2\)
Asτ→0\\tau\\to 0, the vectorccapproaches a discrete one\-hot sample from the categorical distribution defined by𝐳\\mathbf\{z\}\. To approximate aKK\-hot selection vector \(selecting multiple sentences\), we drawKKindependent relaxed samples\{c\(j\)\}j=1K\\\{c^\{\(j\)\}\\\}\_\{j=1\}^\{K\}using Equation[C2](https://arxiv.org/html/2608.02879#A3.Ex2)\. We then aggregate these samples by taking their element\-wise maximum:

mi=maxj=1,…,K⁡ci\(j\),i=1,…,n\.m\_\{i\}=\\max\_\{j=1,\\ldots,K\}\\,c^\{\(j\)\}\_\{i\},\\quad i=1,\\ldots,n\.\(C3\)
The resulting vector𝐦\\mathbf\{m\}serves as a continuous proxy for the binary mask\. The final output of the interpreter used to gate the input sentences is:

ℐ​𝒩​\(𝐱,𝐲T;α\)i=mi\.\\mathcal\{IN\}\(\\mathbf\{x\},\\mathbf\{y\}\_\{T\};\\alpha\)\_\{i\}=m\_\{i\}\.\(C4\)
During training, this soft mask allows gradients to backpropagate through the selection process\. During inference, we obtain the discrete selection by taking the indices of the top\-KKlogits directly or by hardening the soft mask\.

## Appendix DInterpreter: Training Details

We report the configuration and formulation for the best\-performing interpreter, trained utilizing the EBM\-guided framework on the Hybrid \(λ=0\.9\\lambda=0\.9\) energy landscape\. The training process required approximately11hour on dual NVIDIA T4 GPUs \(provided by Kaggle’s free tier\)\. Table[4](https://arxiv.org/html/2608.02879#A4.T4)details the specific hyperparameters\.

Table 4:Hyperparameters for the Interpreter\.ParameterValueArchitectureEncoderall\-mpnet\-base\-v2Projection Dim \(dm​o​d​e​ld\_\{model\}\)768768Self\-Attention Layers22Cross\-Attention Layers66Attention Heads88Dropout Rate0\.10\.1MLP Layers11MLP Hidden Dimension256256OptimizationEpochs5050Batch Size1616Learning Rate1​e−51e^\{\-5\}Loss FunctionInfoNCE \(τ=0\.1\\tau=0\.1\)Selection MechanismGumbel\-SoftmaxGumbel Temperature1\.01\.0DataDataset Size20,00020,000samplesValidation Split10%10\\%Max Sentence Count1616\(Learnable Padding\)![Refer to caption](https://arxiv.org/html/2608.02879v1/x6.png)Figure 7:Overview of the Alternating Optimization Protocol\.The framework employs a joint training strategy to prevent distribution shift\.\(Left\)In the standard phase, the interpreter generates a binary mask over the prompt sentences; its parameters are updated to maximize the energy gap using the frozen EBM as a critic\.\(Right\)Periodically, the EBM is fine\-tuned to adapt to the interpreter’s evolving distribution\. This involves querying the target LLM with the currently masked prompt to obtain a fresh, ground\-truth response, thereby grounding the energy landscape in the model’s actual behavior under partial input\.### D\.1Alternating Optimization Details

While Section[3\.3](https://arxiv.org/html/2608.02879#S3.SS3)outlines the high\-level objective, we detail here the specific gradient updates required for training\. To mitigate the distribution shift caused by masking \(Fig\.[7](https://arxiv.org/html/2608.02879#A4.F7)\), we define the joint optimization loop\. Letθ\(k\)\\theta^\{\(k\)\}andα\(k\)\\alpha^\{\(k\)\}denote the parameters at stepkk\.

#### Step 1: Interpreter Update\.

We freeze the EBM parametersθ\(k−1\)\\theta^\{\(k\-1\)\}and update the interpreter to improve selection precision\. The gradient update is:

α\(k\)←α\(k−1\)−ηα∇α\(ℰLM\(𝐱⊙M,𝐲T;θ\(k−1\)\)−ℰLM\(𝐱⊙\(1−M\),𝐲T;θ\(k−1\)\)\)\\begin\{split\}\\alpha^\{\(k\)\}\\leftarrow&\\alpha^\{\(k\-1\)\}\-\\eta\_\{\\alpha\}\\nabla\_\{\\alpha\}\\Big\(\\mathcal\{E\}\_\{\\text\{LM\}\}\(\\mathbf\{x\}\\odot M,\\mathbf\{y\}\_\{T\};\\theta^\{\(k\-1\)\}\)\\\\ &\-\\mathcal\{E\}\_\{\\text\{LM\}\}\(\\mathbf\{x\}\\odot\(1\-M\),\\mathbf\{y\}\_\{T\};\\theta^\{\(k\-1\)\}\)\\Big\)\\end\{split\}\(D1\)whereM=ℐ​𝒩​\(𝐱;𝐲T,α\(k−1\)\)M=\\mathcal\{IN\}\(\\mathbf\{x\};\\mathbf\{y\}\_\{T\},\\alpha^\{\(k\-1\)\}\)is the generated mask by the interpreter\.

#### Step 2: Periodic Grounding\.

EveryNgroundN\_\{\\text\{ground\}\}steps, we generate a fresh training pair to re\-align the EBM\. We apply the current hard mask to the prompt and query the black\-box LLM:

𝐱~\\displaystyle\\tilde\{\\mathbf\{x\}\}=𝐱⊙𝕀​\(M\>0\.5\)\\displaystyle=\\mathbf\{x\}\\odot\\mathbb\{I\}\(M\>0\.5\)\(D2\)𝐲~\\displaystyle\\tilde\{\\mathbf\{y\}\}=LLM​\(𝐱~\)\\displaystyle=\\text\{LLM\}\(\\tilde\{\\mathbf\{x\}\}\)\(D3\)This creates a valid sample\(𝐱~,𝐲~\)\(\\tilde\{\\mathbf\{x\}\},\\tilde\{\\mathbf\{y\}\}\)that represents the model’s actual behavior under the current masking policy\.

#### Step 3: EBM Fine\-tuning\.

We update the EBM to minimize the energy of the new synthetic pair\(𝐱~,𝐲~\)\(\\tilde\{\\mathbf\{x\}\},\\tilde\{\\mathbf\{y\}\}\)while maintaining the structural constraints learned during pre\-training\. We employ the same dual\-objective lossℒtotal\\mathcal\{L\}\_\{\\text\{total\}\}defined in Equation[1](https://arxiv.org/html/2608.02879#S3.E1)\(Sec\.[3\.2](https://arxiv.org/html/2608.02879#S3.SS2)\), consisting of bothℒfidelity\\mathcal\{L\}\_\{\\text\{fidelity\}\}andℒdep\\mathcal\{L\}\_\{\\text\{dep\}\}\.

However, because there is no ground\-truth human response for the dynamically masked prompt𝐱~\\tilde\{\\mathbf\{x\}\}, we modify the negative sampling set𝒩i\\mathcal\{N\}\_\{i\}for the fidelity objective \(App\.[B](https://arxiv.org/html/2608.02879#A2)\)\. We substitute theresponse\_humansampler with another distinct, model\-based negative,response\_other\_lm\_stochastic, to maintain distribution contrast\. The gradient update is thus computed using this modified negative set𝒩~\\tilde\{\\mathcal\{N\}\}:

θ\(k\)←θ\(k−1\)−ηθ∇θ\(\(1−λ\)​ℒfidelity​\(𝐱~,𝐲~,𝒩~\)\+λℒdep\(𝐱~,𝐲~\)\)\\begin\{split\}\\theta^\{\(k\)\}\\leftarrow\\theta^\{\(k\-1\)\}\-\\eta\_\{\\theta\}\\nabla\_\{\\theta\}\\Big\(&\(1\-\\lambda\)\\mathcal\{L\}\_\{\\text\{fidelity\}\}\(\\tilde\{\\mathbf\{x\}\},\\tilde\{\\mathbf\{y\}\},\\tilde\{\\mathcal\{N\}\}\)\\\\ &\+\\lambda\\mathcal\{L\}\_\{\\text\{dep\}\}\(\\tilde\{\\mathbf\{x\}\},\\tilde\{\\mathbf\{y\}\}\)\\Big\)\\end\{split\}\(D4\)This alternating procedure ensures that as the interpreter’s selections evolve, the energy landscape adapts to provide accurate supervision for those specific sparse inputs\.

## Appendix EEnergy Network: Evaluation Protocols

To evaluate the EBM’s semantic alignment beyond aggregate accuracy, we developed a suite of granular diagnostic tests\. This section details the mathematical formulations, dataset filtering criteria, and specific metrics for each testing dimension\.

### E\.1Dataset Preparation & Filtering

For all diagnostic tests, we utilized specific subsets of the HC3 validation set \(N=1000N=1000\)\. To generate the ground\-truth importance scores used for evaluation, we employed an ablation\-based energy drop methodology\. For each sample pair\(𝐱,𝐲\)\(\\mathbf\{x\},\\mathbf\{y\}\), we systematically removed each sentence to create variants\. We calculated the energy for two modes:

- •Prompt Ablation:Pairs\(𝐱∖i,𝐲\)\(\\mathbf\{x\}\_\{\\setminus i\},\\mathbf\{y\}\), where𝐱∖i\\mathbf\{x\}\_\{\\setminus i\}is the prompt with theii\-th sentence removed\.
- •Response Ablation:Pairs\(𝐱,𝐲∖j\)\(\\mathbf\{x\},\\mathbf\{y\}\_\{\\setminus j\}\), where𝐲∖j\\mathbf\{y\}\_\{\\setminus j\}is the response with thejj\-th sentence removed\.

The importance of a sentence was quantified by the positive energy drop caused by its removal relative to the baseline energyℰ​\(𝐱,𝐲\)\\mathcal\{E\}\(\\mathbf\{x\},\\mathbf\{y\}\)\. Using these scored samples, we applied specific filters to isolate relevant linguistic phenomena:

- •Interrogative Subset \(N=769N=769\):Used forDimension I\. We filtered for prompts containing explicit interrogative structures, defined as sentences ending in a question mark or starting with standard interrogative pronouns \(e\.g\., “What”, “How”, “Why”\)\.
- •Artifact Subset \(N=890N=890\):Used forDimension II\. We filtered for responses containing distinct conversational fillers \(e\.g\., “Okay\!”, “Sure\!”, “Here is the answer:”\) appearing as isolated sentences\.
- •Oracle Subset \(N=500N=500\):Used forDimension III\. A random subset of validation samples was selected for external scoring byGemini\-2\.5\-Flash\.
- •Counterfactual Subset \(N=500N=500\):Used forDimension IV\.Gemini\-3\-Prowas employed to generate specific counterfactual triplets from validation data \(see App\.[E\.5](https://arxiv.org/html/2608.02879#A5.SS5)for details\)\.

### E\.2Dimension I: Core Directive Localization

This test assesses the model’s ability to distinguish the primary user intent \(the directive\) from supplementary context or conversational filler\.

#### Ablation Methodology\.

For a given prompt𝐱\\mathbf\{x\}consisting ofnnsentences\{s1,s2,…,sn\}\\\{s\_\{1\},s\_\{2\},\\dots,s\_\{n\}\\\}and a fixed response𝐲\\mathbf\{y\}, we calculate the baseline energyEbase=ℰ​\(𝐱,𝐲\)E\_\{\\text\{base\}\}=\\mathcal\{E\}\(\\mathbf\{x\},\\mathbf\{y\}\)\. We then systematically remove each sentencesis\_\{i\}to create an ablated prompt𝐱∖i\\mathbf\{x\}\_\{\\setminus i\}and compute theRelative Energy Impact\(Δ​Ei\\Delta E\_\{i\}\):

Δ​Ei=ℰ​\(𝐱∖i,𝐲\)−Ebase\\Delta E\_\{i\}=\\mathcal\{E\}\(\\mathbf\{x\}\_\{\\setminus i\},\\mathbf\{y\}\)\-E\_\{\\text\{base\}\}\(E1\)A positiveΔ​Ei\\Delta E\_\{i\}implies that sentencesis\_\{i\}was necessary for the low\-energy alignment \(i\.e\., it was semantically important\)\.

#### Metric Definitions\.

LetSQS\_\{Q\}be the set of indices corresponding to interrogative sentences andS𝑁𝐶S\_\{\\mathit\{NC\}\}be the set of indices for non\-causal context\.

- •Interrogative Recall@1 \(IR@1\):Adapting therationale extractionevaluation protocol fromERASER\(DeYounget al\.,[2020](https://arxiv.org/html/2608.02879#bib.bib16)\), we define this as the frequency with which the sentence producing the maximum energy impact is an interrogative sentence\. IR@1=1N​∑j=1N𝕀​\[argmaxi\(Δ​Ej,i\)∈SQ,j\]\\text\{IR@1\}=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}\\mathbb\{I\}\\left\[\\operatorname\*\{argmax\}\_\{i\}\(\\Delta E\_\{j,i\}\)\\in S\_\{Q,j\}\\right\] \(E2\)The metric yields a value in\[0,1\]\[0,1\], where an ideal score of11indicates that the explicit question is consistently ranked as the primary causal driver\. However, we note that perfect recall is not expected, as our dataset analysis revealed that human\-written Q&A prompts often contain implicit or structurally ambiguous directives where the semantic core is not the grammatical question\.
- •Attribution Signal\-to\-Noise Ratio \(SNR\):Adapting standard signal processing definitions to attribution magnitude, we define this as the ratio of the average energy impact of questions to the average energy impact of non\-question context sentences\. SNR=1\|SQ\|​∑i∈SQΔ​Ei1\|S𝑁𝐶\|​∑k∈S𝑁𝐶Δ​Ek\+ϵ\\text\{SNR\}=\\frac\{\\frac\{1\}\{\|S\_\{Q\}\|\}\\sum\_\{i\\in S\_\{Q\}\}\\Delta E\_\{i\}\}\{\\frac\{1\}\{\|S\_\{\\mathit\{NC\}\}\|\}\\sum\_\{k\\in S\_\{\\mathit\{NC\}\}\}\\Delta E\_\{k\}\+\\epsilon\}\(E3\)whereϵ=1​e−9\\epsilon=1e^\{\-9\}is a constant for stability\. A high SNR indicates the model is highly sensitive to the directive and insensitive to noise\.

### E\.3Dimension II: Semantic Robustness

This test measures the model’s susceptibility to non\-semantic conversational artifacts, addressing a prevalent pathology in neural models known as reliance on spurious correlations or “annotation artifacts”\(Gururanganet al\.,[2018](https://arxiv.org/html/2608.02879#bib.bib17)\)\. Grounded in concepts originally established for NLI datasets, this evaluation assesses whether the model has learned to treat high\-frequency tokens \(e\.g\., conversational fillers such as “Okay\!”\) as proxies for output validity, independent of their actual semantic content\.

#### Metric Definitions\.

LetS𝐴𝑟𝑡S\_\{\\mathit\{Art\}\}be the set of indices corresponding to artifact sentences\.

- •Artifact Energy Impact \(Art\.Δ​E\\Delta E\):The average change in energy when an artifact is removed\. This metric is adapted fromInput Reductionmethods\(Fenget al\.,[2018](https://arxiv.org/html/2608.02879#bib.bib15)\), where we aim to measure the model’s sensitivity to the removal of negligible features\. Art\.​Δ​E=1\|S𝐴𝑟𝑡\|​∑i∈S𝐴𝑟𝑡\(ℰ​\(𝐱,𝐲∖i\)−Ebase\)\\text\{Art\. \}\\Delta E=\\frac\{1\}\{\|S\_\{\\mathit\{Art\}\}\|\}\\sum\_\{i\\in S\_\{\\mathit\{Art\}\}\}\(\\mathcal\{E\}\(\\mathbf\{x\},\\mathbf\{y\}\_\{\\setminus i\}\)\-E\_\{\\text\{base\}\}\)\(E4\)
- •Rank\-1 Error \(R1E\):The proportion of samples where an artifact sentence is assigned the highest importance rank \(Rank11\)\. This metric quantifies the fidelity trap, where the model overfits to surface\-level plausibility markers rather than semantic drivers\. R1E=1N​∑j=1N𝕀​\[argmaxi\(Δ​Ej,i\)∈SA​r​t,j\]\\text\{R1E\}=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}\\mathbb\{I\}\\left\[\\operatorname\*\{argmax\}\_\{i\}\(\\Delta E\_\{j,i\}\)\\in S\_\{Art,j\}\\right\]\(E5\)

### E\.4Dimension III: Oracle Alignment

This test validates the EBM’s internal ranking of sentence importance against a gold standard ranking generated by a state\-of\-the\-art LLM to assess ranking alignment and accuracy\.

#### Oracle Setup\.

For each sample in theOracle Subset,Gemini\-2\.5\-Flashwas provided with the prompt, response, and list of sentences as derived by the EBM, and instructed to assign an integerInformation Density Scoreyi∈\{0,…,5\}y\_\{i\}\\in\\\{0,\\dots,5\\\}to each response sentencesis\_\{i\}\. The scoring criteria were:

- •0 \(Fluff\):Purely conversational filler or phatic expressions \(e\.g\., “Okay\!”\) with zero informational value\.
- •1 \(Minor Context\):Generic transitions or polite formatting that aids flow but adds no unique content\.
- •2 \(Useful Background\):Contextual definitions or analogies that facilitate understanding without constituting the direct answer\.
- •3 \(Supporting Info\):Elaborations or details necessary for a complete explanation; removing these makes the answer feel thin\.
- •4 \(Important\):Key facts, steps, or reasoning that directly address the user’s request\.
- •5 \(Critical\):The core thesis or direct solution; the response is conceptually incomplete without this sentence\.

#### Metric Definitions\.

Let𝒮=\{s1,…,sM\}\\mathcal\{S\}=\\\{s\_\{1\},\\dots,s\_\{M\}\\\}be the set of sentences in a response\. Letr​e​lirel\_\{i\}be the oracle’s score for sentenceii, and letπ\\pibe the permutation of indices induced by sorting the EBM’s energy impact scoresΔ​E\\Delta Ein descending order \(i\.e\.,π​\(1\)\\pi\(1\)is the index of the most important sentence according to the EBM\)\.

- •nDCG@3 \(Normalized Discounted Cumulative Gain\):We measure the ranking quality at cutoffk=3k=3\. The Discounted Cumulative Gain \(DCG\) is computed as: DCG​@​k=∑i=1k2r​e​lπ​\(i\)−1log2⁡\(i\+1\)\\text\{DCG\}@k=\\sum\_\{i=1\}^\{k\}\\frac\{2^\{rel\_\{\\pi\(i\)\}\}\-1\}\{\\log\_\{2\}\(i\+1\)\}\(E6\)The Ideal DCG \(IDCG\) is computed similarly using the permutationπ∗\\pi^\{\*\}that sorts the oracle’s scores perfectly\. The final metric is: nDCG​@​k=DCG​@​kIDCG​@​k\\text\{nDCG\}@k=\\frac\{\\text\{DCG\}@k\}\{\\text\{IDCG\}@k\}\(E7\)This metric penalizes the model heavily if it fails to place high\-value \(oracle score55\) sentences in the top ranks\.
- •Soft Precision@1:This metric assesses the utility of the single most important sentence identified by the EBM\. It is defined as the proportion of samples where the EBM’s top choice received a high relevance score \(≥4\\geq 4\) from the oracle: S\-Prec@1=1N​∑j=1N𝕀​\[r​e​lπj​\(1\)≥4\]\\text\{S\-Prec@1\}=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}\\mathbb\{I\}\\left\[rel\_\{\\pi\_\{j\}\(1\)\}\\geq 4\\right\]\(E8\)

### E\.5Dimension IV: Causal Disentanglement

This test evaluates the model’s ability to identify specific causal links between input and output concepts, distinguished from mere topical association\.

#### Counterfactual Setup\.

For each sample in theCounterfactual Subset, the model identified:

1. 1\.A specific target response sentence \(ytargety\_\{\\text\{target\}\}\)\.
2. 2\.The high\-impact prompt sentence \(xcausex\_\{\\text\{cause\}\}\) that directly necessitatedytargety\_\{\\text\{target\}\}\.
3. 3\.A low\-impact distractor sentence \(xdistractorx\_\{\\text\{distractor\}\}\) from thesameprompt that was topically related but causally irrelevant toytargety\_\{\\text\{target\}\}\.

#### Metric Definitions\.

We quantify discriminative performance by comparing the energy assigned to causal versus distractor antecedents\. Letℰ​\(x,y\)\\mathcal\{E\}\(x,y\)denote the scalar energy score, where lower values indicate higher compatibility\. For a successful disentanglement, the model must assign strictly lower energy to the true cause than to the distractor, satisfying the conditionℰ​\(xcause,ytarget\)<ℰ​\(xdistractor,ytarget\)\\mathcal\{E\}\(x\_\{\\text\{cause\}\},y\_\{\\text\{target\}\}\)<\\mathcal\{E\}\(x\_\{\\text\{distractor\}\},y\_\{\\text\{target\}\}\)\. We aggregate this behavior using two metrics:

- •Counterfactual Accuracy:The percentage of triplets where the EBM correctly assigns lower energy to the causal pair\. Acc=100N​∑j=1N𝕀\[ℰ\(xcause,ytarget\)<ℰ\(xdistractor,ytarget\)\]\\begin\{split\}\\text\{Acc\}=\\frac\{100\}\{N\}\\sum\_\{j=1\}^\{N\}&\\mathbb\{I\}\[\\mathcal\{E\}\(x\_\{\\text\{cause\}\},y\_\{\\text\{target\}\}\)\\\\ &<\\mathcal\{E\}\(x\_\{\\text\{distractor\}\},y\_\{\\text\{target\}\}\)\]\\end\{split\}\(E9\)
- •Energy Separation Margin \(ESM\):The average magnitude of the energy difference between the distractor and the cause\. A larger positive margin indicates higher confidence in the causal distinction\. ESM=1N∑j=1N\(ℰ​\(xdistractor,ytarget\)−ℰ\(xcause,ytarget\)\)\\begin\{split\}\\text\{ESM\}=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}\(&\\mathcal\{E\}\(x\_\{\\text\{distractor\}\},y\_\{\\text\{target\}\}\)\\\\ &\-\\mathcal\{E\}\(x\_\{\\text\{cause\}\},y\_\{\\text\{target\}\}\)\)\\end\{split\}\(E10\)

## Appendix FInterpreter: Plausibility Evaluation

To construct the plausibility benchmark, we prompted five diverse LLMs \(Gemini\-2\.5\-Flash,GPT\-4o,GPT\-4o\-Mini,GPT\-J\-6B, andGPT\-2\-XL\) to act as data annotators\.

#### Prompting Strategy\.

For a given sample tuple consisting of a promptP=\{s1p,…,snp\}P=\\\{s^\{p\}\_\{1\},\\dots,s^\{p\}\_\{n\}\\\}and a specific target response sentencestrs^\{r\}\_\{t\}, each oracle was provided with the full text context and instructed to: “Assign an importance score \(0\.0 to 1\.0\) to every Prompt Sentence\. The scores MUST sum to exactly 1\.0\.” To maximize determinism, we utilized a temperature ofT=0T=0or close to it\.

#### Metric Definitions\.

Let𝐲oracle∈ℝn\\mathbf\{y\}\_\{\\text\{oracle\}\}\\in\\mathbb\{R\}^\{n\}be the vector of ground\-truth importance scores provided by an oracle for thennsentences in the prompt\. Let𝐲interp∈ℝn\\mathbf\{y\}\_\{\\text\{interp\}\}\\in\\mathbb\{R\}^\{n\}be the predicted importance scores output by the interpreter\.

- •Soft Top\-1 Accuracy:This metric addresses the inherent ambiguity in attribution where multiple prompt sentences may be necessary\. We define a match if the interpreter’s single highest\-scored sentence falls within the top\-kksentences identified by the oracle\. Leti∗=argmaxi∈\{1,…,n\}\(𝐲interp\(i\)\)i^\{\*\}=\\operatorname\*\{argmax\}\_\{i\\in\\\{1,\\dots,n\\\}\}\(\\mathbf\{y\}\_\{\\text\{interp\}\}^\{\(i\)\}\)be the index of the sentence chosen by the interpreter\. Let𝒮k​\(𝐲oracle\)\\mathcal\{S\}\_\{k\}\(\\mathbf\{y\}\_\{\\text\{oracle\}\}\)be the set of indices corresponding to thekklargest values in𝐲oracle\\mathbf\{y\}\_\{\\text\{oracle\}\}\. The metric is defined as: SoftAcc​@​k=𝕀​\[i∗∈𝒮k​\(𝐲oracle\)\]\\text\{SoftAcc\}@k=\\mathbb\{I\}\\left\[i^\{\*\}\\in\\mathcal\{S\}\_\{k\}\(\\mathbf\{y\}\_\{\\text\{oracle\}\}\)\\right\]\(F1\)In our experiments, we setk=2k=2\.
- •nDCG \(Normalized Discounted Cumulative Gain\):Similar to EBM experiments, we utilize nDCG to evaluate the quality of the entire ranking order\. This metric penalizes the interpreter if it assigns low importance scores to sentences that the Oracle deemed critical\. Letπ\\pibe a permutation of indices\{1,…,n\}\\\{1,\\dots,n\\\}that sorts the scores𝐲interp\\mathbf\{y\}\_\{\\text\{interp\}\}in descending order, such that𝐲interp\(π​\(1\)\)≥𝐲interp\(π​\(2\)\)≥…\\mathbf\{y\}\_\{\\text\{interp\}\}^\{\(\\pi\(1\)\)\}\\geq\\mathbf\{y\}\_\{\\text\{interp\}\}^\{\(\\pi\(2\)\)\}\\geq\\dots\. The DCG is computed using the oracle’s scores as the true relevance grades: DCG=∑j=1n𝐲oracle\(π​\(j\)\)log2⁡\(j\+1\)\\text\{DCG\}=\\sum\_\{j=1\}^\{n\}\\frac\{\\mathbf\{y\}\_\{\\text\{oracle\}\}^\{\(\\pi\(j\)\)\}\}\{\\log\_\{2\}\(j\+1\)\}\(F2\)The Ideal DCG \(IDCG\) is computed similarly using the permutationπ∗\\pi^\{\*\}that sorts𝐲oracle\\mathbf\{y\}\_\{\\text\{oracle\}\}in descending order\. The normalized score is: nDCG=DCGIDCG\\text\{nDCG\}=\\frac\{\\text\{DCG\}\}\{\\text\{IDCG\}\}\(F3\)

## Appendix GInterpreter: Generative Faithfulness

Quantifying faithfulness in open\-ended generation is fundamentally distinct from classification tasks\. Unlike classification, where the output is a discrete label, generative outputs are high\-dimensional and semantically flexible\. A true causal driver may not reproduce theexacttokens of the target, but should reproduce itssemanticcore\. To validate our interpreter, we devised a three\-stage evaluation pipeline: \(1\) deriving comparable baselines via dynamic max\-ratio thresholding, \(2\) establishing metric definitions robust to generative variance, and \(3\) filtering non\-causal RLHF artifacts to strictly isolate semantic drivers\.

#### Baseline Implementations\.

To compare our interpreter’s binary selections against the continuous importance scoressi∈\[0,1\]s\_\{i\}\\in\[0,1\]produced by the various baselines, we employed aMax\-Ratio Thresholdingstrategy\. For a given prompt, a sentenceiiis selected if its importance score is within a factor of the maximum score assigned to any sentence in that prompt:

Select​i⇔si≥0\.5⋅maxj⁡\(sj\)\\text\{Select \}i\\iff s\_\{i\}\\geq 0\.5\\cdot\\max\_\{j\}\(s\_\{j\}\)\(G1\)This dynamic thresholding adapts to each model’s confidence distribution, ensuring we capture the primary drivers of the generation while discarding marginal contributors\. The baselines were constructed as follows:

- •LLM Oracles:GPT\-4oandGPT\-4o\-Miniwere prompted to output a normalized probability distribution of causal importance across the prompt sentences for each specific target\.
- •Sentence\-Level LIME:We adapted LIME\(Ribeiroet al\.,[2016](https://arxiv.org/html/2608.02879#bib.bib41)\)for generative attribution\. To mitigate the prohibitive computational cost of standard word\-level LIME, we employed a two\-dimensional deduplication strategy\. For a prompt ofnnsentences, we generated a strictly unique set of binary masksM=\{𝐦\(1\),…,𝐦\(K\)\}M=\\\{\\mathbf\{m\}^\{\(1\)\},\\dots,\\mathbf\{m\}^\{\(K\)\}\\\}\(using an exhaustive set if2n≤502^\{n\}\\leq 50, otherwise randomly sampled\)\. We queried the target LLM exactly once per mask to generate a shared pool of counterfactual responses𝐲~\(k\)\\tilde\{\\mathbf\{y\}\}^\{\(k\)\}\. For a given target sentence𝐲t\\mathbf\{y\}\_\{t\}, we calculated the cosine similaritys​i​mk=cos⁡\(S​\(𝐲~\(k\)\),S​\(𝐲t\)\)sim\_\{k\}=\\cos\(S\(\\tilde\{\\mathbf\{y\}\}^\{\(k\)\}\),S\(\\mathbf\{y\}\_\{t\}\)\)\. We then fitted a Ridge regression to predicts​i​mksim\_\{k\}from𝐦\(k\)\\mathbf\{m\}^\{\(k\)\}, weighted by an exponential distance kernel: wk=exp⁡\(−d​\(𝐦\(k\),𝟏\)2σ2\)w\_\{k\}=\\exp\\left\(\-\\frac\{d\(\\mathbf\{m\}^\{\(k\)\},\\mathbf\{1\}\)^\{2\}\}\{\\sigma^\{2\}\}\\right\)\(G2\)whereddis the cosine distance andσ=0\.25​n\\sigma=0\.25\\sqrt\{n\}\. The resulting regression coefficients were ReLU\-clipped to remove negative causal impacts andL​1L1\-normalized to yield the final probability distribution𝐬\\mathbf\{s\}\.
- •Sparse Random Baseline:A naive uniform random assignment \(e\.g\., normalizing independentU​\(0,1\)U\(0,1\)values\) produces flat distributions \(si≈1/ns\_\{i\}\\approx 1/n\) that trivially fail the max\-ratio thresholding\. To rigorously simulate the confident sparsity of actual trained attribution models, we sampled the probability vectors from a symmetric Dirichlet distribution: 𝐬∼Dir​\(𝜶\),where​αi=0\.3∀i\\mathbf\{s\}\\sim\\text\{Dir\}\(\\boldsymbol\{\\alpha\}\),\\quad\\text\{where \}\\alpha\_\{i\}=0\.3\\ \\ \\forall i\(G3\)This ensures the baseline critically tests the effect of arbitrary sentence assignment without failing due to mechanical flatness\.

#### Metric Definitions\.

For our evaluation \(Table[2](https://arxiv.org/html/2608.02879#S4.T2)\), we utilize the following definitions\. LetS​\(⋅\)S\(\\cdot\)be the sentence embedding function—specifically implemented using theall\-mpnet\-base\-v2model to capture dense semantic representations—and𝐲t\\mathbf\{y\}\_\{t\}be the target sentence\.

- •Generative Sufficiency \(ℳsuff\\mathcal\{M\}\_\{\\text\{suff\}\}\):The similarity between the target and generated response usingonlythe selected sentences𝐱S\\mathbf\{x\}\_\{S\}: ℳsuff=cos⁡\(S​\(LLM​\(𝐱S\)\),S​\(𝐲t\)\)\\mathcal\{M\}\_\{\\text\{suff\}\}=\\cos\(S\(\\text\{LLM\}\(\\mathbf\{x\}\_\{S\}\)\),S\(\\mathbf\{y\}\_\{t\}\)\)\(G4\)
- •Generative Comprehensiveness \(ℳcomp\\mathcal\{M\}\_\{\\text\{comp\}\}\):The similarity between the target and the response generated using the complement subset𝐱∖𝐱S\\mathbf\{x\}\\setminus\\mathbf\{x\}\_\{S\}: ℳcomp=cos⁡\(S​\(LLM​\(𝐱∖𝐱S\)\),S​\(𝐲t\)\)\\mathcal\{M\}\_\{\\text\{comp\}\}=\\cos\(S\(\\text\{LLM\}\(\\mathbf\{x\}\\setminus\\mathbf\{x\}\_\{S\}\)\),S\(\\mathbf\{y\}\_\{t\}\)\) \(G5\)

To instantiate these metrics, we selectCosine Similarityover sentence embeddings as our comparison function\. This choice is grounded in our robustness analysis \(seeSelecting Similarity Functionbelow\), which demonstrates that strict logical entailment metrics \(NLI\) are overly rigid for validating open\-ended generation\.

#### Selecting Similarity Function\.

We initially attempted to evaluate faithfulness using Natural Language Inference \(NLI\) models to detect logical entailment between the counterfactual generation and the original target\. Specifically, we employed thecross\-encoder/nli\-deberta\-v3\-largemodel, treating the generated response as the premise and the target sentence as the hypothesis\. We extracted the softmax probability of the “entailment” class to quantify sufficiency and comprehensiveness\.

However, as shown in Table[5](https://arxiv.org/html/2608.02879#A7.T5), NLI metrics proved too rigid for generative tasks\.

Table 5:Robustness Check: NLI Metrics\.Faithfulness scores computed usingDeBERTa\-v3logical entailment probabilities\. In this setup,SufficiencyandComprehensivenessmeasure the probability that the counterfactually generated response logically entails the original target sentence\. The uniformly low sufficiency scores \(<0\.15<0\.15\) indicate that strict NLI models heavily penalize valid generative paraphrasing\. This demonstrates that NLI is overly rigid for open\-ended generation tasks, justifying our selection of Cosine Similarity for the primary evaluation in Table[2](https://arxiv.org/html/2608.02879#S4.T2)InterpreterSuff\.\(↑\\uparrow\)Comp\.\(↓\\downarrow\)Gap\(↑\\uparrow\)LIME0\.1120\.0580\.054\\rowcolorgray\!20ESCI \(Ours\)0\.1460\.0830\.063GPT\-4o\-Mini0\.1450\.0730\.072GPT\-4o0\.0930\.0610\.033Random0\.0980\.153\-0\.055The NLI Sufficiency scores hovered around0\.09−0\.150\.09\-0\.15, implying that even the full correct context rarely entailed the target according to the NLI model\. This occurs because NLI models are trained on premise\-hypothesis pairs that require strict logical implication, whereas generative recovery in open\-ended tasks relies heavily on semantic paraphrasing\. Consequently, we adoptedCosine Similarityfor our primary evaluation, utilizing theall\-mpnet\-base\-v2SentenceTransformer model to compute the pairwise cosine similarity between the dense embeddings of the generations and targets\. As detailed in the main text,Cosine Similarityyielded sufficiency scores in the∼\\sim0\.40\.4range, effectively capturing the soft semantic retention characteristic of open\-ended generation\.

#### Filtering RLHF Priors \(Trivial Targets\)\.

A major confounder in interpreting instruction\-tuned models is the prevalence of conversational fillers \(e\.g\., “Okay\!”\)\. These outputs are often driven by Reinforcement Learning from Human Feedback \(RLHF\) priors rather than specific prompt content\. If included, they artificially inflate comprehensiveness scores \(lower is better\), as the model will often hallucinate these polite preambles even when the causal instruction is removed\.

We conducted a trivial target analysis on a subset of conversational fillers \(n=157n=157\) identified via regex matching to examine how models handle these RLHF priors\. As shown in Table[6](https://arxiv.org/html/2608.02879#A7.T6), the metrics expose that these fillers lack specific causal antecedents\.

Table 6:Trivial Target Analysis\.We measure how often conversational fillers \(e\.g\., “Okay\!”\) persist under intervention\.Triv\. Suff:Percentage of times the filler is generated givenonlythe instruction\.Triv\. Comp:Percentage of times the filler is hallucinated when the instruction isremoved\. Values confirm these are RLHF priors, not causally sensitive targets\.InterpreterTriv\. Suff\.\(↑\\uparrow\)Triv\. Comp\.\(↓\\downarrow\)LIME15\.2%1\.8%\\rowcolorgray\!20ESCI \(Ours\)32\.5%40\.8%GPT\-4o\-Mini15\.5%16\.4%GPT\-4o21\.6%5\.2%Random15\.6%49\.4%For example, LIME severely struggles withTrivial Sufficiency\(performing worse than Random\), indicating that it removes too many sentences and fails to find a sufficient prompt subset to reliably trigger the filler\. Conversely, while ESCI achieves a much higherTrivial Sufficiency, it suffers from an inflatedTrivial Comprehensiveness\. This high hallucination rate in the complement set is a direct byproduct of ESCI’s extreme sparsity; because ESCI aggressively isolates only the core logical sentences, the remaining complement set \(used for comprehensiveness\) remains larger and retains enough generic conversational context to independently trigger the LLM’s polite RLHF priors\. Ultimately, because these conversational fillers are global stylistic habits rather than logical consequences of specific prompt sentences, no attribution method can meaningfully isolate them\.

To prevent these non\-causal hallucinations from skewing the semantic evaluation, we strictly filtered these targets from the main benchmark\.

## Appendix HThe LLM Usage

Parts of the initial drafts of this manuscript were revised with the assistance of a Large Language Model\. The model was prompted to improve the fluency, conciseness, and overall academic tone of the text to meet the standards of ACL publications\.