xMIx: High-Performance Serving-Time Platform for Mechanistic Interpretability Apps

# xMIx: High-Performance Serving-Time Platform for Mechanistic Interpretability Apps
Source: [https://arxiv.org/html/2607.22595](https://arxiv.org/html/2607.22595)
###### Abstract

\\Ac

MI has emerged as a powerful approach for analyzing and intervening in inference computations, with a growing number of applications such as jailbreak attempt detection, truthfulness evaluation, and hallucination detection\. Unfortunately, MI deployment in production model\-serving systems is currently not practical, as most existing MI frameworks introduce prohibitively high runtime overheads\. The fundamental problem is that MI functions do not compose cleanly with served models: they fragment deployment, often force draining requests and rebuilding serving state, and conflict with critical performance optimizations such as continuous batching and CUDA\-graph execution, essential for production deployments\.

We presentxMIx, a serving\-native framework for deploying MI applications in production inference serving environments\.xMIxenables*attaching*MI functions to a predefined set of locations in the model runtime, interposing on activations within the layers and residual streams\.xMIxsupports conditional invocation of MI functions depending on the outputs in preceding model layers\. Multiple MI applications can be deployed in a single model instance\.xMIxcompiles them all into the serving path but activates them*dynamically at runtime*only when necessary, with negligible performance cost, and without requiring a separate model instance or alternative execution stack\.

We integratexMIxwith thevLLMserving system and evaluate it across three major models and seven diverse MI applications\.xMIxachieves performance comparable to nativevLLMexecution, incurring a slowdown of 1\.3% mean[inter\-token latency](https://arxiv.org/html/2607.22595#id12.12.id12)\([ITL](https://arxiv.org/html/2607.22595#id12.12.id12)\), 1\.2% for tail P99[ITL](https://arxiv.org/html/2607.22595#id12.12.id12), 2\.6% for mean[time to first token](https://arxiv.org/html/2607.22595#id10.10.id10)\([TTFT](https://arxiv.org/html/2607.22595#id10.10.id10)\), and 1\.6% for mean[total token throughput](https://arxiv.org/html/2607.22595#id11.11.id11)\([TTT](https://arxiv.org/html/2607.22595#id11.11.id11)\)\.

CFGcontrol flow graphDFAdeterministic finite automatonFSMfinite state machinePTAprefix tree acceptorLLMlarge language modelMLPmultilayer perceptronMImechanistic interpretabilitySAEsparse autoencoderMoEmixture of expertsTTFTtime to first tokenTTTtotal token throughputITLinter\-token latency
## 1Introduction

![Refer to caption](https://arxiv.org/html/2607.22595v1/Figures/OverviewDiagram.png)Figure 1:xMIxoverview\. MI functions can be installed \(*attached*\) in different transformer layers and in multiple loci \(*hooks*\) within a layer, and can read/write the respective activations and residual streams\. At each locus, a function can be dynamically enabled/disabled, allowing easy implementation of common MI primitives, and seamless toggling of different MI functions in a single model deployment\. In the figure, layer L16’s activations are modified \(conditional\-write\) based on the processing \(read\) of activations after L1\.Mechanistic Interpretability \(MI\) has emerged as a promising approach for understanding and controlling the behavior of Large Language models \(LLMs\)\. Instead of treating models as opaque black boxes and reason only about inputs and outputs, MI operates directly on the model’s internal computations during inference\. By inspecting, tracing, and intervening on intermediate activations and residual streams, MI techniques aim to gain insight into how models produce specific behaviors and, increasingly, to modify those behaviors at runtime\.

There is a growing class of*MI apps*, such as jailbreak detectionKadali and Papalexakis \([2026](https://arxiv.org/html/2607.22595#bib.bib17)\), hallucination mitigationLiet al\.\([2023](https://arxiv.org/html/2607.22595#bib.bib52)\), truthfulness evaluationOrgadet al\.\([2025](https://arxiv.org/html/2607.22595#bib.bib30)\), safety monitoringLeeet al\.\([2025](https://arxiv.org/html/2607.22595#bib.bib41)\), and controllable generationSticklandet al\.\([2024](https://arxiv.org/html/2607.22595#bib.bib59)\)\. More broadly, MI promises to become a practical mechanism for*augmenting already deployed models*with new runtime functionality without retraining or fine\-tuning\. Instead of modifying model weights, MI apps operate on transient inference computations, enabling systems to inject new behaviors, suppress undesirable outputs, enforce safety constraints, or dynamically adapt responses during execution\. Such runtime intervention mechanisms may become an important systems mechanism for extending model capabilities post deployment\.

Unfortunately, most MI apps today remain confined to research prototypes and are rarely deployed in production serving stacks\. A key reason is the*lack of high\-performance runtime that meets strict efficiency targets of production deployments*\. Existing MI frameworks \(e\.g\., TransformerLens\(Nanda and Bloom,[2022](https://arxiv.org/html/2607.22595#bib.bib1)\), and others discussed in[Section˜2](https://arxiv.org/html/2607.22595#S2)\), typically rely on intrusive instrumentation hooks and custom execution paths incompatible with modern inference systems\. In production environments, where serving throughput and latency depend critically on optimizations such as continuous batching and CUDA graph execution, even lightweight MI logic that affects these optimizations can disrupt scheduling and significantly degrade performance\. For example, recent vLLM\-lens project reported about 20% throughput slowdown over vanillavLLMfor a simple steering logic applied to a single layer[29](https://arxiv.org/html/2607.22595#bib.bib46)\(see[Section˜5](https://arxiv.org/html/2607.22595#S5)for more results\)\.

Concurrently, the growing MI apps diversity creates another challenge: each app is often implemented as a separate inference pipeline modification, tightly coupled to a specific runtime, model and intervention mechanism\. As a result, operators cannot efficiently deploy multiple MI apps simultaneously on the same served model\. Instead, deploying new apps requires maintaining separate model instances, alternative execution stacks, or specialized serving infrastructure, rather than a unified control\-plane for managing multiple interventions\. This makes deployment much more complex: one needs to choose which MI apps are enabled in advance\. Toggling of apps at runtime is prohibitively expensive: it forces the serving stack to be restarted, draining requests and rebuilding serving state\.

The missing link\.We argue that the missing link between MI research and production deployment is a serving\-native abstraction layer that enables MI apps to compose cleanly with optimized inference runtimes\. Such a layer is critical for making MI*actionable*Orgadet al\.\([2026](https://arxiv.org/html/2607.22595#bib.bib2)\), enabling broader adoption and practical impact in production model serving systems\.

We presentxMIx, a lightweight serving\-native framework for flexible and efficient deployment of MI apps in production inference environments\.[Figure˜1](https://arxiv.org/html/2607.22595#S1.F1)illustrates the main concepts\. A developer specifies the*hooks*in the model where MI functions \(GPU kernels in Triton\) can be*attached*to implement the respective MI logic\. Each function may access activations at the hook locus, modify them to influence the subsequent inference layers or/and save its results in a buffer that can be later accessed from the CPU or GPU code\. In addition, MI function invocation can be*conditioned*on a predicate evaluated on the outputs from MI functions executed in earlier layers\. For convenience,xMIxprovides MI function templates to implement typical MI primitives –*read*,*write*, and*conditional\-write*\. Together, this simple, model\-agnostic interface allows implementing a broad range of known MI apps, including 7 we demonstrate in our evaluation\.

xMIxenables multiple MI apps on the same deployed model, rather than requiring separate execution stacks or dedicated model replicas\. It compiles all the attached functions directly into the serving path, but activates them*only when required*\. The governing principle is simple: if a deployed MI app is not active on a request, it should not impose meaningful performance cost on that request\. Thus, a MI app can be disabled or enabled with negligible overheads, without disrupting the serving of pending inference requests\.

Under the hood,xMIxemploys several advanced techniques to meet its performance and flexibility goals\. First, it interposes on CUDA graphs created by the serving frameworks to enable MI function attachment and toggling\. Second, it adds token\-triggered MI functions as well as data sharing across functions attached to different layers, while keeping negligible \(a few MBs\) memory state\. Third, it supports divergent processing in a batch of tokens at each hook, without falling back to the CPU\-driven execution\. Last, it seamlessly supports multi\-GPU execution, without any involvement of MI apps developers\.

Results\.We integratexMIxwithvLLMserving runtime, and implement seven MI apps with diverse MI primitives for three popular mid\-size LLMs\. Across all the configurations,xMIxdemonstrates minor overheads for all key performance metrics, slowing downvLLMby 1\.3% for mean[ITL](https://arxiv.org/html/2607.22595#id12.12.id12), 1\.2% for tail P99[ITL](https://arxiv.org/html/2607.22595#id12.12.id12), 2\.6% for mean[TTFT](https://arxiv.org/html/2607.22595#id10.10.id10)and 1\.6% for mean[TTT](https://arxiv.org/html/2607.22595#id11.11.id11)\. These results not only show thatxMIxoverheads are at least an order of magnitude lower than in existing MI frameworks, but also that MI apps can be readily deployed in production serving systems with acceptable costs\.

Contributions\.Our key contributions are: \(i\) We analyze different MI applications and use cases, and generalize them, creating unified abstractions that allow MI developers to interact with the model under a general framework\. \(ii\) We built a framework on top ofvLLM, allowing the deployment of a broad spectrum of different apps, while preserving all the benefits of high\-throughput, low\-latency inference\. \(iii\) We implement a dynamic toggling mechanism, which allows don’t\-use\-don’t\-pay deployment, enabling or disabling applications as they are needed at runtime\.

## 2Motivation and Related work

[mechanistic interpretability](https://arxiv.org/html/2607.22595#id7.7.id7)\([MI](https://arxiv.org/html/2607.22595#id7.7.id7)\) techniques analyze and intervene on the computations performed inside a model, rather than only its inputs and outputs \(see[Table˜4](https://arxiv.org/html/2607.22595#A1.T4)for the list of surveyed apps\)\. For autoregressive transformers, this usually means reading hidden states, attention values, or MLP activations at chosen layers and token positions, then either reporting a signal \(e\.g\., a probe score\) or writing back a modified activation\. This pattern underlies a broad set of applications, including probing, activation patching, activation steering, and conditional interventions\.

MI applications are not merely post\-hoc analyses\. They are inference\-time programs that must execute in lockstep with the model’s forward pass\. Many MI interventions read an intermediate value at a specific layer and token position, then use that value to modify a later intermediate state in the same execution\. For example,\(Leeet al\.,[2025](https://arxiv.org/html/2607.22595#bib.bib41)\)reads activations at one layer and uses them to induce refusal at a later layer by writing a modified activation\.

Unfortunately, despite their immense potential, broader deployment of MI applications in production model serving systems is limited due to high performance costs\. We analyze the key reasons next\.

Research tooling for MI\.Most existing MI tooling was built for rapid research iteration, not for production inference\. TransformerLens\(Nanda and Bloom,[2022](https://arxiv.org/html/2607.22595#bib.bib1)\)popularized a convenient hook\-based interface and standardized naming for transformer internals, making it easy to cache, edit, and replace activations\. However, it does so as a PyTorch\-level instrumentation library rather than as an integration with a serving engine\(Paszkeet al\.,[2019](https://arxiv.org/html/2607.22595#bib.bib12)\)\. It uses PyTorch’s*eager execution mode*, prioritizing flexibility over efficiency\. Specifically, it allows arbitrary invocation of MI functions during inference, which is not compatible with CUDA graphs, and thus sacrifices performance due to the resulting frequent CPU\-GPU synchronization\. NNsight\(Fiotto\-Kaufmanet al\.,[2024](https://arxiv.org/html/2607.22595#bib.bib3)\)generalizes this style with a flexible tracing API over PyTorch models and a remote backend, again prioritizing expressive experimentation over performance\. EasyEdit\(Wanget al\.,[2024](https://arxiv.org/html/2607.22595#bib.bib4)\)organizes a wide range of knowledge\-editing methods behind a common framework, but its focus is offline or session\-level model editing rather than low\-latency, multi\-tenant serving\. EasyEdit2\(Xuet al\.,[2025c](https://arxiv.org/html/2607.22595#bib.bib5)\), EasySteer\(Xuet al\.,[2025b](https://arxiv.org/html/2607.22595#bib.bib6)\), and more recently vLLM\-lens\([29](https://arxiv.org/html/2607.22595#bib.bib46)\)move control closer tovLLM’s serving engine by unifying steering\-vector generation and application, yet they still treat steering as a model\-level framework layered above inference and resort to PyTorch’s eager execution model\. Thus, by targeting MI research and prototyping, these tools are incompatible with the strict performance goals of production serving systems\.

Why production model serving is different\.Modern LLM serving systems, such asvLLMmaximize throughput by applying sophisticated techniques, including disaggregated execution of prefill and decode, continuous batching, explicit KV cache management, and GPU\-oriented optimizations such as FlashAttention\(Daoet al\.,[2022](https://arxiv.org/html/2607.22595#bib.bib53)\)and PagedAttention\(Kwonet al\.,[2023](https://arxiv.org/html/2607.22595#bib.bib13)\)\. They often rely on*CUDA Graphs*\(NVIDIA,[2026a](https://arxiv.org/html/2607.22595#bib.bib7)\)to reduce recurrent kernel launch overheads, and introduce custom GPU kernels, including Triton\-based fused kernels invLLM, to keep performance\-critical operations on the device fast path\(Tilletet al\.,[2019](https://arxiv.org/html/2607.22595#bib.bib10); vLLM Project,[2026b](https://arxiv.org/html/2607.22595#bib.bib11)\)\. These systems are designed to minimize host\-device synchronization and to preserve fixed, CUDA\-graph friendly execution paths\.

Under these conditions, regular Python hooks are not an appropriate implementation approach\. They force the serving stack back into a host\-driven eager execution mode\. Each intervention that escapes the GPU fast path, adds CPU participation on the critical path, introduces synchronization points, and often requires extra data movements\. That cost directly conflicts with continuous batching and CUDA graphs: once execution must wait for Python, the system loses the fixed path that these optimizations rely on\. The penalty is especially severe during decode, where serving is already memory\-bound and even modest per\-token overhead accumulates across long generations\.

Seen from a systems perspective, existing MI frameworks make the same core assumption: they operate through CPU\-driven execution, where tensor operations are dispatched immediately by the host program rather than staged into a serving\-native execution graph\(Paszkeet al\.,[2019](https://arxiv.org/html/2607.22595#bib.bib12)\), with hooks, traces, or user\-space callbacks layered around the model rather than integrated into the serving engine\. TransformerLens\(Nanda and Bloom,[2022](https://arxiv.org/html/2607.22595#bib.bib1)\), NNsight\(Fiotto\-Kaufmanet al\.,[2024](https://arxiv.org/html/2607.22595#bib.bib3)\), EasyEdit\(Wanget al\.,[2024](https://arxiv.org/html/2607.22595#bib.bib4)\), EasyEdit2\(Xuet al\.,[2025c](https://arxiv.org/html/2607.22595#bib.bib5)\), EasySteer\(Xuet al\.,[2025b](https://arxiv.org/html/2607.22595#bib.bib6)\)and vLLM\-lens\([29](https://arxiv.org/html/2607.22595#bib.bib46)\)differ in interface and scope, but they all expose MI by interposing software around the forward pass Python flow instead of compiling interventions into the execution path itself\. As we show later in[Table˜3](https://arxiv.org/html/2607.22595#S5.T3), the overheads induced by this execution model render it unusable for high\-throughput serving\.

The missing MI runtime layer\.The central gap preventing production deployment of MI is the lack of a production\-ready programming and runtime model for MI\. A deployable system must support many interventions inside the forward pass while preserving the execution constraints of the serving engine\. Another crucial requirement for such systems is the ability to enable or disable an MI application without swapping a model, via mid\-flight reconfiguration\. Model replacement is too disruptive for performance and error\-prone: operators must drain in\-flight requests, preserve correctness for existing KV caches and captured execution graphs, warm up the replacement instance, and absorb temporary capacity loss and tail\-latency spikes\. These issues require system\-level, rather than application\-level runtime tightly integrated with the serving environment\.

This work targets this missing layer: a general MI framework for production model serving systems with dynamic, low\-overhead control over interventions\.

## 3Survey of representative[MI](https://arxiv.org/html/2607.22595#id7.7.id7)applications

Table 1:Representative[MI](https://arxiv.org/html/2607.22595#id7.7.id7)applications with primitives, triggers and intervention loci\. Additional applications are surveyed in[Table˜4](https://arxiv.org/html/2607.22595#A1.T4)\.To scope the interface thatxMIxshould expose, we surveyed 14 recent[MI](https://arxiv.org/html/2607.22595#id7.7.id7)applications \(see[Appendix˜A](https://arxiv.org/html/2607.22595#A1)for the full list\), and clustered them by the intervention patterns they require\. From each cluster we selected a representative application to include in[Table˜1](https://arxiv.org/html/2607.22595#S3.T1)\. The columns capture the dimensions that matter for a serving interface: the*Purpose*of the method, the*Primitive*it requires, the*Trigger*that determines whether the method activates, and the*Target*locus where it reads or intervenes\. This organization lets us compare otherwise different methods by the runtime capabilities they demand, rather than by their end\-task alone\.

The main conclusion is that these applications collapse to three primitives\.Readcovers methods that only consume activations\. A representative example is a linear\-regression probe that assigns a truthfulness score from last\-token activations\.Writecovers unconditional steering methods that modify activation targets to influence behavior, e\.g\., add or subtract a vector\.Conditional Writecovers methods that perform some computation before deciding whether to intervene, where the condition can depend either on activations at another layer, or on token\-level information before the forward pass begins\.

Once applications are grouped under these primitives, method\-specific implementation details can be abstracted behind a common interface, which in turn opens the door to systems optimizations\. In particular, the table suggests that most representative applications do not require arbitrary Python hooks or unrestricted user code inside the forward pass\. Instead,xMIxmainly needs efficient support for reading named activation slices, writing to a small set of intervention loci, and guarding those writes with lightweight predicates\. This also clarifies what must have dynamic on\-off toggling control on a live model instance: typically small runtime artifacts such as probes, vectors, thresholds, feature dictionaries, or router masks, rather than a new copy of the model\.

## 4Design and Implementation

### 4\.1Programming Interface

xMIxexposes a lightweight and clean API that allows easy implementation of a variety of MI apps, while specifically facilitating primitive classes identified in[Section˜3](https://arxiv.org/html/2607.22595#S3)\. The interface is designed to make the logical structure of a MI app explicit: what value is read or written, where in the model it applies, and under what condition it should be active\. Rather than exposing arbitrary callbacks,xMIxasks the user to describe interventions in this restricted form so that the runtime can map them onto serving\-native execution paths\.[Fig\.˜2](https://arxiv.org/html/2607.22595#S4.F2)includes two illustrative examples\.

[⬇](data:text/plain;base64,dHJ1dGhfcHJvYmUgPSAobS5yZWFkKCJwcm9iZSIpCiAgLmxheWVycyhbMjNdKQogIC5zdWJtb2R1bGUoInJlc2lkdWFsLnBvc3QiKQogIC5jb25kKCJsYXN0X3Rva2VuIikp)truth\_probe=\(m\.read\("probe"\)\.layers\(\[23\]\)\.submodule\("residual\.post"\)\.cond\("last\_token"\)\)\(a\)Read\-only intervention[⬇](data:text/plain;base64,Z3VhcmRlZF9zdGVlciA9ICgKICBtLndyaXRlKCJzdGVlcmluZ192ZWN0b3IiKQogIC5sYXllcnMoWzIyXSkKICAuc3VibW9kdWxlKCJhdHRlbnRpb24ucG9zdCIpCiAgLmNvbmQoCiAgICB3aGVuPSJ0b2tlbl9pbih0cmlnZ2VyX2xpc3QpIiwKICAgIGdhdGU9InByb2JlKGxheWVyPTE4KSA+PSAwIikpCg==)guarded\_steer=\(m\.write\("steering\_vector"\)\.layers\(\[22\]\)\.submodule\("attention\.post"\)\.cond\(when="token\_in\(trigger\_list\)",gate="probe\(layer=18\)\>=0"\)\)\(b\)Conditional write
Figure 2:Example[MI](https://arxiv.org/html/2607.22595#id7.7.id7)functions usingxMIx\. The strings represent either the standard submodule names \(e\.g\.,residual\.post\), or external Triton kernels to invoke at the hook \(e\.g\.,steering\_vector\)\.Core operations\.The interface centers on two constructors,m\.read\(\.\.\.\)andm\.write\(\.\.\.\), which cover the surveyed primitive space\. A read operation declares thatxMIxshould materialize a derived signal from an activation locus, such as raw activations or a probe output\. A write operation declares thatxMIxshould inject or transform a value at an activation locus, such as a steering vector or another intervention payload\. Conditional write is expressed as an ordinary write augmented with an explicit activation predicate\.

Target specification\.Each operation is refined by selectors identifying where it applies\.\.layers\(\.\.\.\)specifies either a layers concrete set or a broader scope such asall, while\.submodule\(\.\.\.\)identifies the target computation within the layer \(e\.g\., pre\-attention or post\-attention representation\)\. This decomposition keeps the API close to how practitioners already describe interventions, while exposing the targeting information that the runtime needs in order to place the corresponding serving\-side hooks\.

Activation conditions\.\.cond\(\.\.\.\)controls when an operation should run\. The condition may be token\-driven, e\.g\., activated when the current token belongs to a trigger set, or activation\-driven, e\.g\., gated by a probe score computed at another layer\. This is the mechanism that letsxMIxexpress both simple read and write operations and more structured conditional behaviors within the same programming model\. Just as important, it makes the control structure explicit enough for the runtime to map these conditions to CUDA graph\-internal parameter updates or conditional execution rather than to CPU\-side orchestration\.

### 4\.2Translation to Runtime Semantics

Translation objective\.The programming interface is not executed as a collection of arbitrary callbacks around the forward pass\. Instead,xMIxlowers each declaration into a small set of serving\-native runtime artifacts whose behavior is known in advance\. This is the key design choice that reconciles flexibility with production inference constraints\. The user describes*what*signal to read or write, andxMIxdetermines*how*to realize that request using the runtime’s existing execution path\.

Hook mapping\.The selectors\.layers\(\.\.\.\)and\.submodule\(\.\.\.\)resolve to a predefined hook surface in the model\. A hook denotes a legal attachment point wherexMIxmay expose an activation slice to an[MI](https://arxiv.org/html/2607.22595#id7.7.id7)function without requiring an arbitrary control transfer out of the serving engine\. These hooks are the translation targets of the API\. They are exposed at component boundaries in the execution graph, where activations pass from one stage of computation to the next, rather than inside the internal implementation of a component\. This narrow hook surface is intentional\. It defines the places where intervention is possible while avoiding arbitrary instrumentation inside performance\-critical fused internals, which is what makes the interface compatible with optimized serving backends\.

Executable units\.Once an operation has been mapped to a hook,xMIxrealizes it as an attached[MI](https://arxiv.org/html/2607.22595#id7.7.id7)function, typically a Triton kernel\.xMIxprovides a library of pre\-implemented kernel templates for common[MI](https://arxiv.org/html/2607.22595#id7.7.id7)primitives, but users may also import a custom kernel when the built\-in library does not cover their application\. At a design level, the contract is simple\. A function consumes the data slice \(e\.g\., activations\) at the hook, together with the runtime parameters associated with the app, and produces either a derived signal, an activation update, or both\. Read\-like operations therefore materialize data for later use, while write\-like operations produce a payload that is applied at the target hook\. In other words, the kernel operates on the activation view presented at the hook, not by instrumenting the internals of the upstream component implementation\.

State and dataflow\.xMIxuses explicit shared state to connect functions attached to different hooks\. Read outputs are written into buffers that can be consumed later by another[MI](https://arxiv.org/html/2607.22595#id7.7.id7)function, returned to CPU\-side code, or reused by GPU\-side logic in subsequent layers\. This buffer\-mediated dataflow is what letsxMIxexpress cross\-layer applications such as conditional steering based on an earlier probe result\. Additionally, it constrains composition to a small and analyzable interface: functions communicate through named runtime state, not through arbitrary inter\-kernel control flow\.

Activation and control semantics\.The runtime distinguishes between two main condition classes\. Token\-driven conditions depend on properties of the current token stream, such as whether the current token belongs to a trigger set\. Activation\-driven conditions depend on previously materialized[MI](https://arxiv.org/html/2607.22595#id7.7.id7)function outputs, such as a probe score or feature activation computed at another hook\. The API deliberately exposes only conditions that can be lowered to runtime control without returning to CPU\-side orchestration\. This restriction is the price of portability across optimized serving paths, but it is also what makes the resulting control logic compatible with production execution\.

Toggling semantics\.Deployment and activation are separate concepts inxMIx\. An application may be configured into the serving path, with its hooks, functions, and state artifacts registered in advance, yet remain inactive on a particular request\. Toggling therefore changes whether the relevant runtime artifacts participate in the current execution, while leaving the surrounding serving path unchanged\. This is the design basis of the “don’t use, don’t pay” principle: if an application is inactive for a request, it should impose at most negligible overhead on that request\.

Composition model\.The same translation scheme allows multiple MI apps to coexist on one model instance\. Each app contributes hook\-attached functions together with the state and control artifacts they require, whilexMIxprovides a common runtime substrate that determines when each artifact is active\. In that sense,xMIxacts as a control plane for MI apps rather than as a single\-purpose intervention library\. Different apps can therefore share one optimized deployment instead of requiring separate model replicas or ad hoc execution stacks\.

### 4\.3Implementation

We implementxMIxon top ofvLLMand use its serving abstractions to realize the design above through GPU\-resident execution paths that remain compatible with the engine’s optimization strategy\. At this stage, the questions are no longer what hooks, functions, or buffers mean in the abstract, but how those abstractions are realized inside a concrete serving stack while preserving its performance properties\.

vLLM integration surface\.xMIxinterposes on the execution path thatvLLMuses to run model layers and inserts intervention points through the same generalized model abstractions that already support multiple model families\. This integration surface is the concrete realization of the abstract hook model described above\. It keeps the user\-facing interface uniform, while allowing the backend to resolve hook placement through serving\-side code rather than through a model\-specific user API\.

CUDA\-graph\-safe kernel design\.To keep intervention overhead low,xMIxrealizes selected primitive operations through hand\-crafted GPU kernels, including Triton kernels for performance\-critical cases\. This letsxMIxfuse sequences of operations that frequently appear together in MI apps, reducing kernel\-launch overhead and avoiding unnecessary intermediate materialization\. The same design also serves a second purpose: CUDA\-graph capture and replay require a stable execution structure with known operations and tensor layouts ahead of time\.xMIxtherefore implements its intervention paths so that the common cases remain CUDA\-graph safe rather than forcing the serving engine to fall back to a less optimized execution mode\. In particular,xMIxseparates static graph structure from per\-request dynamic parameters\. When the topology of an intervention path is fixed but the kernel arguments vary,xMIxcan reuse the same instantiated graph and update only the relevant kernel\-node parameters before launch, instead of recapturing or rebuilding the whole graph\. This follows the CUDA\-graph pattern in which explicit node handles are retained for the dynamic kernels while the surrounding graph is captured once and replayed many times\(NVIDIA,[2026a](https://arxiv.org/html/2607.22595#bib.bib7),[b](https://arxiv.org/html/2607.22595#bib.bib9)\)\.

Graph\-level toggling and continuous batching\.On top of these kernels,xMIxintegrates intervention logic into the execution graph used by the serving runtime\. This allows the system to treat[MI](https://arxiv.org/html/2607.22595#id7.7.id7)operations as first\-class nodes in the serving path rather than as external callbacks\. The important point is that not all dynamism is the same\. Some[MI](https://arxiv.org/html/2607.22595#id7.7.id7)applications require only dynamic parameters on a fixed execution path, while others require true data\-dependent control flow\. For the former,xMIxreuses an instantiated graph and updates the parameter state of the affected nodes at low cost before launch\. For the latter,xMIxstructures the graph so that an upstream GPU computation determines whether a downstream intervention body should execute, rather than returning control to the CPU to make that decision\. This is conceptually aligned with CUDA conditional nodes, where a predicate computed on the device governs the execution of a nested graph region\(Gaiseret al\.,[2024](https://arxiv.org/html/2607.22595#bib.bib8); NVIDIA,[2026b](https://arxiv.org/html/2607.22595#bib.bib9)\)\.

This distinction matters when for efficient support of continuous batching, where different requests may activate different interventions, leading to divergent compute paths during the graph execution\.xMIxtherefore aims to express request\-selective behavior as graph\-internal control, so that inactive applications are bypassed and active ones trigger only their relevant subgraph\. The goal is to preserve a single serving\-native execution path for the batch, without forcing every request to pay for the most expensive intervention branch or fragmenting execution into CPU\-orchestrated special cases\.

### 4\.4Current Scope and Limitations

The implementation is designed to be model\-agnostic at the interface level, but the current realization is still tied to the model families and serving paths that we have integrated invLLM\. The general hook substrate is meant to extend across architectures while preserving the same programming model, yet this should not be read as a claim that every model is already wired up automatically or that every hook is available on every backend\. Likewise, multi\-GPU execution is handled by the serving runtime rather than exposed to the[MI](https://arxiv.org/html/2607.22595#id7.7.id7)developer, but the portability claim here is practical rather than absolute: it depends on the coverage of the underlyingvLLMintegration\.

Similarly, the current system supports intervention only at the exposed hook surface described above\. Those hooks live at component boundaries in the serving path, not inside a component’s internal implementation or a fused kernel\. As a result,xMIxdoes not support intra\-component or intra\-kernel manipulations\. This is a consequence of the same serving\-native execution model that enables graph\-safe integration, rather than a missing engineering feature\. More generally,xMIxsupports only[MI](https://arxiv.org/html/2607.22595#id7.7.id7)logic expressible through the exposed hooks, shared runtime state, and graph\-safe kernels\. AlthoughxMIxprovides a library of common kernels, unsupported finer\-grained logic still requires a different integration strategy and lies outside the system’s current scope\. In particular, this limitation prevented us from implementing the jailbreak prevention mechanism of\(Ben\-Tovet al\.,[2025](https://arxiv.org/html/2607.22595#bib.bib14)\)\.

## 5Evaluation

To assessxMIx’s practicality, we deploy seven representative[MI](https://arxiv.org/html/2607.22595#id7.7.id7)applications under a production\-like benchmarking scenario and quantify the performance impact of running MI apps\.

Setup\.We evaluate onApps \#1\-\-\#7, corresponding to the indices in[Table˜1](https://arxiv.org/html/2607.22595#S3.T1)\.Apps \#1\-\-\#6are run on Llama\-3\.1\-8B\-Instruct, Mixtral\-8x7B\-Instruct\-v0\.1, and Qwen3\-8B; the MoE\-specificApp \#7is run on Mixtral only\. We benchmark serving performance usingvLLM’sbench sweeputility\(vLLM Project,[2026a](https://arxiv.org/html/2607.22595#bib.bib67)\)\.

Workload\.Each run issues 250 prompts under Poisson arrival, sampled from ShareGPT V3 unfiltered\(ShareGPT,[2026](https://arxiv.org/html/2607.22595#bib.bib68)\)\. For models without precomputed steering vectors, we introduce a random steering vector of matching dimension\. To control the affect of steering, We fix output length at 128 tokens per prompt, so the actual content of the steering vector doesn’t impact the measurement\.

Baseline\.Our measurement baseline isvLLMwith thexMIxMI hooks*uninstalled*\. Relative to vanillavLLM 0\.11\.1, this configuration showed no measurable performance difference for all but one model\. For that model, our version ran marginally*faster*while producing identical output; we traced this to memory\-layout differences that yield a minor acceleration over the vanilla version\. To ensure a fair comparison, we therefore report all results relative to thevLLMversion with fullxMIxsupport, rather than vanillavLLM\.

Hardware\.We use a single AMD EPYC 7742 64\-Core CPU server with NVIDIA NVLINK connected A100\-SXM4\-80GB GPUs\. We runvLLM’s Tensor\-Parallel configuration on two GPUs\.

Metrics\.We evaluate across the main performance metrics:[ITL](https://arxiv.org/html/2607.22595#id12.12.id12), tail[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)\(P99\),[TTFT](https://arxiv.org/html/2607.22595#id10.10.id10), and[TTT](https://arxiv.org/html/2607.22595#id11.11.id11)\. They provide a comprehensive view of thexMIxperformance impact\.[TTFT](https://arxiv.org/html/2607.22595#id10.10.id10)indicates the added delay before generation begins;[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)and tail[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)indicate the added delay during decode; and[TTT](https://arxiv.org/html/2607.22595#id11.11.id11)reflects the influence on the aggregate serving capacity\.

Methodology\.For each model, application, and metric, we discard the first three benchmark runs as warmup and aggregate the remaining 7 repeated runs into a single summary statistic for reporting\. We report arithmetic mean for latency metrics such as mean[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)and p99[ITL](https://arxiv.org/html/2607.22595#id12.12.id12), and harmonic mean for throughput\. We then normalize by the baseline, and estimate 95% confidence intervals\. We validate the output correctness of the platform when no steering is applied, via exact token comparison to the vanilla version\. Where we had the correct steering vector \(Llama\-3\.1\-8b\-Instruct with the refusal vector from\(Arditiet al\.,[2024](https://arxiv.org/html/2607.22595#bib.bib49)\)\), we also verified the steering works\.

![Refer to caption](https://arxiv.org/html/2607.22595v1/Figures/EvalOutput/charts/png/combined_mean_itl_ms.png)\(a\)Mean[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)
![Refer to caption](https://arxiv.org/html/2607.22595v1/Figures/EvalOutput/charts/png/combined_p99_itl_ms.png)\(b\)Mean p99[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)
![Refer to caption](https://arxiv.org/html/2607.22595v1/Figures/EvalOutput/charts/png/combined_total_token_throughput.png)\(c\)Mean[TTT](https://arxiv.org/html/2607.22595#id11.11.id11)
![Refer to caption](https://arxiv.org/html/2607.22595v1/Figures/EvalOutput/charts/png/combined_mean_ttft_ms.png)\(d\)Mean[TTFT](https://arxiv.org/html/2607.22595#id10.10.id10)

Figure 3:Overheads relative to thevLLMbaseline for App\#1\-\#7 deployed withxMIx\. Each panel reports a per\-model breakdown for Llama, Mixtral, and Qwen3, while the final bar summarizes the average change across models\. Error bars denote confidence intervals\.Results\.[Figure˜3](https://arxiv.org/html/2607.22595#S5.F3)summarizes the overheads introduced by MI apps on top ofxMIx\. Across all benchmarks, MI apps introduce only*minor performance degradation*with natural variation across applications and models\. The overhead distribution across different MI apps is correlated with the amount of added processing incurred by each app\. For example,App\-1manipulates all model layers, and thus incurs the highest overhead across all models\. In contrast,App\-4intervenes in fewer layers for all tokens, andApp\-3is conditionally triggered for a subset of tokens, which in turn reduces its relative cost\. Moreover, larger models with more layers \(e\.g\. Mixtral\-8x7B\-Instruct\-v0\.1\) and more parameters incur much lower relative overheads across apps due to amortized costs\. We also notice a few peculiar phenomena, likeApp\-7‘s minor negative values \(acceleration\) on Mixtral\. After deeper examination, we saw the performance dropped back down after removing unused arguments passed between steering kernels\. So we attribute them to memory layout differences\. We also noticed high variability of the P99[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)metric \(expressed by high error bars\)\. Looking into it, we found that these runs usually showed slightly better throughput than the average run\. We suspect that due to timing differences, more requests find themselves among mixed prefill\-decode batches, raising[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)of the decode tokens in such passes compared to decode only batches, and driving P99[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)of the whole run up\.

\(a\)Llama\-3\.1\-8B\-Instruct
\(b\)Mixtral\-8x7b
\(c\)Qwen3\-8B

Table 2:Performance ofEasySteerandxMIxacross four applications evaluated on multiple models, with signed percentage change against thevLLMbaseline shown in parentheses\. Latency metrics \(Mean ITL, P99 ITL, Mean TTFT\) are reported in ms \(↓\\downarrow\); throughput is in tokens/sec \(↑\\uparrow\)\.Comparing toEasySteer\.

Setup\.We conduct a direct comparison againstEasySteerXuet al\.\([2025a](https://arxiv.org/html/2607.22595#bib.bib37)\)on the applications implemented on both platforms,App\-1\([Arditiet al\.](https://arxiv.org/html/2607.22595#bib.bib49)[2024](https://arxiv.org/html/2607.22595#bib.bib49)\]\),App\-2\([Chenet al\.](https://arxiv.org/html/2607.22595#bib.bib50)\[[2025](https://arxiv.org/html/2607.22595#bib.bib50)\]\),App\-3\([Leeet al\.](https://arxiv.org/html/2607.22595#bib.bib41)\[[2025](https://arxiv.org/html/2607.22595#bib.bib41)\]\),App\-5\([Scialangaet al\.](https://arxiv.org/html/2607.22595#bib.bib70)\[[2025](https://arxiv.org/html/2607.22595#bib.bib70)\]\), across the same models benchmarked in[Figure˜3](https://arxiv.org/html/2607.22595#S5.F3), comparing tovLLMvanilla baseline\. We adapted theEasySteerapplications to work with the standardvLLMbench utility\. In order to align withEasySteerapp configuration, the referencevLLMvanilla baseline forEasySteeris run with prefix caching disabled and chunked prefill disabled and on a newervLLMversion\.xMIxcomparison is againstvLLMvanilla baseline as appears in[Fig\.˜3](https://arxiv.org/html/2607.22595#S5.F3), withvLLMbeing run with default configuration, and percentage differences are reported accordingly\. It’s important to note that running with chunked prefill disabled reduces P99[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)since decode requests don’t appear in the same batch as prefill requests \(since those aren’t chunked\), but reduces overall throughput for the same reason\. To replicate experiments on models for which steering vectors weren’t calculated, we inserted random steering vectors of the matching hidden dimension size\.

Results\.Results are shown in[Table˜2](https://arxiv.org/html/2607.22595#S5.T2)\. We observe an order\-of\-magnitude lower overheads inxMIxcompared toEasySteeracross all metrics\. These performance benefits stem fromEasySteerusing the eagervLLMexecution with its CPU\-driven control flow, a trait shared by all current MI frameworks exceptxMIx\.

Comparing to other platforms\.[Table˜3](https://arxiv.org/html/2607.22595#S5.T3)provides a comparison against platforms widely used by MI researchers: TransformerLens\(Nanda and Bloom,[2022](https://arxiv.org/html/2607.22595#bib.bib1)\)and NNsight\(Fiotto\-Kaufmanet al\.,[2024](https://arxiv.org/html/2607.22595#bib.bib3)\)\. We report absolute[TTFT](https://arxiv.org/html/2607.22595#id10.10.id10),[TTT](https://arxiv.org/html/2607.22595#id11.11.id11), Mean[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)and P99[ITL](https://arxiv.org/html/2607.22595#id12.12.id12)under comparable one\-layer and all\-layer write settings\. We didn’t perform application comparison as we did forxMIxandEasySteer, since none of the applications were implemented for NNsight and for TransformerLens\. Let us note, that TransformerLens isn’tvLLMbased, and doesn’t support batching, significantly lowering[TTT](https://arxiv.org/html/2607.22595#id11.11.id11), but achieving much better[TTFT](https://arxiv.org/html/2607.22595#id10.10.id10)\.

Memory overhead\.xMIxconsumes only a few KBs of additional memory beside the memory used by MI applications themselves \(both their buffers and code\)\. The memory of the applications is managed byvLLM\. The total footprint of all the surveyed applications combined is about a few MBs, constituting a negligible fraction of the total memory used by the model and its activations\.

Table 3:Absolute performance comparison for a multi and single layer steering configuration acrossxMIx, NNsight and TransformerLens, anchored by thevLLMbaseline, reported for Llama\-3\.1\-8b\-Instruct\. Signed percentage changes relative to the baseline are shown in parentheses forxMIxonly\.
## 6Conclusion

This paper introducesxMIx, a novel framework for integrating mechanistic interpretability applications into SOTA LLM inference flows\. By identifying the common ground to many applications, and analyzing it from the system perspective, we have developed an infrastructure, allowing researchers to apply a wide set of different MI methods, while keeping them compatible to existing optimizations\. That way, we have removed a obstacle standing in the way of bringing the insights of MI to real life production grade inference\.

## References

- A\. Arditi, O\. Obeso, A\. Syed, D\. Paleka, N\. Panickssery, W\. Gurnee, and N\. Nanda \(2024\)Refusal in language models is mediated by a single direction\.Advances in Neural Information Processing Systems37,pp\. 136037–136083\.Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.2.1.2.1.1),[Table 1](https://arxiv.org/html/2607.22595#S3.T1.2.2.1.2.1.1),[§5](https://arxiv.org/html/2607.22595#S5.p7.1),[§5](https://arxiv.org/html/2607.22595#S5.p9.2)\.
- M\. Ben\-Tov, M\. Geva, and M\. Sharif \(2025\)Universal jailbreak suffixes are strong attention hijackers\.arXiv preprint arXiv:2506\.12880\.Note:arXiv preprint used because no published venue was found\.Cited by:[§4\.4](https://arxiv.org/html/2607.22595#S4.SS4.p2.1)\.
- R\. Chen, Z\. Zhang, J\. Hong, S\. Kundu, and Z\. Wang \(2025\)SEAL: steerable reasoning calibration of large language models for free\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=klPszYDIRT)Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.6.5.2.1.1),[Table 1](https://arxiv.org/html/2607.22595#S3.T1.2.3.2.2.1.1),[§5](https://arxiv.org/html/2607.22595#S5.p9.2)\.
- T\. Dao, D\. Fu, S\. Ermon, A\. Rudra, and C\. Ré \(2022\)Flashattention: fast and memory\-efficient exact attention with io\-awareness\.Advances in neural information processing systems35,pp\. 16344–16359\.Cited by:[§2](https://arxiv.org/html/2607.22595#S2.p5.1)\.
- EasySteer \(2026\)Server\-level steering with cuda graphs\.Note:[https://github\.com/ZJU\-REAL/EasySteer\-vllm\-v1/pull/3](https://github.com/ZJU-REAL/EasySteer-vllm-v1/pull/3)Accessed: 2026\-06\-11Cited by:[Appendix B](https://arxiv.org/html/2607.22595#A2.p1.1)\.
- M\. Farooq, V\. De Silva, R\. Rahulamathavan, and X\. Shi \(2025\)Sentiment steering in large language models via activation vector manipulation\.In2025 25th International Conference on Digital Signal Processing \(DSP\),pp\. 1–5\.Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.4.3.2.1.1)\.
- M\. Fayyaz, A\. Modarressi, H\. Deilamsalehy, F\. Dernoncourt, R\. A\. Rossi, T\. Bui, H\. Schuetze, and N\. Peng \(2026\)Steering MoE LLMs via expert \(De\)activation\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=v5Yl9V8rJs)Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.14.13.2.1.1),[Table 1](https://arxiv.org/html/2607.22595#S3.T1.2.8.7.2.1.1)\.
- J\. Fiotto\-Kaufman, A\. R\. Loftus, E\. Todd, J\. Brinkmann, C\. Juang, K\. Pal, C\. Rager, A\. Mueller, S\. Marks, A\. Sen Sharma, F\. Lucchetti, M\. Ripa, A\. Belfki, N\. Prakash, S\. Multani, C\. Brodley, A\. Guha, J\. Bell, B\. Wallace, and D\. Bau \(2024\)NNsight and NDIF: democratizing access to foundation model internals\.arXiv preprint arXiv:2407\.14561\.Note:arXiv preprint used because no published venue was found\.Cited by:[§2](https://arxiv.org/html/2607.22595#S2.p4.1),[§2](https://arxiv.org/html/2607.22595#S2.p7.1),[§5](https://arxiv.org/html/2607.22595#S5.p11.1)\.
- J\. Gaiser, D\. Fontaine, H\. Hoffman, S\. Jones, and F\. Oh \(2024\)Dynamic control flow in CUDA graphs with conditional nodes\.Note:[https://developer\.nvidia\.com/blog/dynamic\-control\-flow\-in\-cuda\-graphs\-with\-conditional\-nodes/](https://developer.nvidia.com/blog/dynamic-control-flow-in-cuda-graphs-with-conditional-nodes/)NVIDIA Technical Blog\. Published 2024\-05\-10, updated 2025\-02\-03\. Accessed 2026\-05\-07\.Cited by:[§4\.3](https://arxiv.org/html/2607.22595#S4.SS3.p4.1)\.
- Z\. Hu, J\. Piet, G\. Zhao, J\. Jiao, and D\. Wagner \(2024\)Toxicity detection for free\.External Links:2405\.18822,[Link](https://arxiv.org/abs/2405.18822)Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.10.9.2.1.1)\.
- S\. D\. S\. S\. Kadali and E\. E\. Papalexakis \(2026\)Jailbreaking leaves a trace: understanding and detecting jailbreak attacks from internal representations of large language models\.arXiv preprint arXiv:2602\.11495\.Note:arXiv preprint used because no published venue was found\.Cited by:[§1](https://arxiv.org/html/2607.22595#S1.p2.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the 29th Symposium on Operating Systems Principles,External Links:[Document](https://dx.doi.org/10.1145/3600006.3613165)Cited by:[§2](https://arxiv.org/html/2607.22595#S2.p5.1)\.
- B\. W\. Lee, I\. Padhi, K\. N\. Ramamurthy, E\. Miehling, P\. Dognin, M\. Nagireddy, and A\. Dhurandhar \(2025\)Programming refusal with conditional activation steering\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Oi47wc10sm)Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.7.6.2.1.1),[§1](https://arxiv.org/html/2607.22595#S1.p2.1),[§2](https://arxiv.org/html/2607.22595#S2.p2.1),[Table 1](https://arxiv.org/html/2607.22595#S3.T1.2.4.3.2.1.1),[§5](https://arxiv.org/html/2607.22595#S5.p9.2)\.
- K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. Wattenberg \(2023\)Inference\-time intervention: eliciting truthful answers from a language model\.Advances in Neural Information Processing Systems36,pp\. 41451–41530\.Cited by:[§1](https://arxiv.org/html/2607.22595#S1.p2.1)\.
- N\. Nanda and J\. Bloom \(2022\)TransformerLens\.Note:[https://github\.com/TransformerLensOrg/TransformerLens](https://github.com/TransformerLensOrg/TransformerLens)Software library citation from the project README; no archival paper located\.Cited by:[§1](https://arxiv.org/html/2607.22595#S1.p3.1),[§2](https://arxiv.org/html/2607.22595#S2.p4.1),[§2](https://arxiv.org/html/2607.22595#S2.p7.1),[§5](https://arxiv.org/html/2607.22595#S5.p11.1)\.
- NVIDIA \(2026a\)CUDA programming guide: CUDA graphs\.Note:[https://docs\.nvidia\.com/cuda/cuda\-programming\-guide/04\-special\-topics/cuda\-graphs\.html](https://docs.nvidia.com/cuda/cuda-programming-guide/04-special-topics/cuda-graphs.html)Official NVIDIA documentation\. Accessed 2026\-05\-04\.Cited by:[§2](https://arxiv.org/html/2607.22595#S2.p5.1),[§4\.3](https://arxiv.org/html/2607.22595#S4.SS3.p3.1)\.
- NVIDIA \(2026b\)Handling dynamic patterns\.Note:[https://docs\.nvidia\.com/dl\-cuda\-graph/latest/torch\-cuda\-graph/handling\-dynamic\-patterns\.html](https://docs.nvidia.com/dl-cuda-graph/latest/torch-cuda-graph/handling-dynamic-patterns.html)In the official NVIDIA guide*CUDA Graph Best Practice for PyTorch*\. Accessed 2026\-05\-07\.Cited by:[§4\.3](https://arxiv.org/html/2607.22595#S4.SS3.p3.1),[§4\.3](https://arxiv.org/html/2607.22595#S4.SS3.p4.1)\.
- H\. Orgad, F\. Barez, T\. Haklay, I\. Lee, M\. Mosbach, A\. Reusch, N\. Saphra, B\. C\. Wallace, S\. Wiegreffe, E\. Wong, I\. Tenney, and M\. Geva \(2026\)Interpretability can be actionable\.Note:[https://actionable\-interpretability\-guide\.github\.io/paper\.pdf](https://actionable-interpretability-guide.github.io/paper.pdf)Position paper hosted on the project website\. The companion page lists this citation\. Accessed 2026\-05\-07\.Cited by:[§1](https://arxiv.org/html/2607.22595#S1.p5.1)\.
- H\. Orgad, M\. Toker, Z\. Gekhman, R\. Reichart, I\. Szpektor, H\. Kotek, and Y\. Belinkov \(2025\)LLMs know more than they show: on the intrinsic representation of LLM hallucinations\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=KRnsX5Em3W)Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.9.8.2.1.1),[§1](https://arxiv.org/html/2607.22595#S1.p2.1),[Table 1](https://arxiv.org/html/2607.22595#S3.T1.2.5.4.2.1.1)\.
- A\. Paszke, S\. Gross, F\. Massa, A\. Lerer, J\. Bradbury, G\. Chanan, T\. Killeen, Z\. Lin, N\. Gimelshein, L\. Antiga, A\. Desmaison, A\. Kopf, E\. Yang, Z\. DeVito, M\. Raison, A\. Tejani, S\. Chilamkurthy, B\. Steiner, L\. Fang, J\. Bai, and S\. Chintala \(2019\)PyTorch: an imperative style, high\-performance deep learning library\.InAdvances in Neural Information Processing Systems,Vol\.32\.Cited by:[§2](https://arxiv.org/html/2607.22595#S2.p4.1),[§2](https://arxiv.org/html/2607.22595#S2.p7.1)\.
- N\. Rimsky, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. Turner \(2024\)Steering llama 2 via contrastive activation addition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15504–15522\.External Links:[Link](https://aclanthology.org/2024.acl-long.828/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.828)Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.12.11.2.1.1)\.
- M\. Scialanga, T\. Laugel, V\. Grari, and M\. Detyniecki \(2025\)SAKE: steering activations for knowledge editing\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 15966–15978\.External Links:[Link](https://aclanthology.org/2025.acl-long.777/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.777),ISBN 979\-8\-89176\-251\-0Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.11.10.2.1.1),[Table 1](https://arxiv.org/html/2607.22595#S3.T1.2.6.5.2.1.1),[§5](https://arxiv.org/html/2607.22595#S5.p9.2)\.
- ShareGPT \(2026\)ShareGPT vicuna unfiltered dataset\.Note:[https://huggingface\.co/datasets/anon8231489123/ShareGPT\_Vicuna\_unfiltered](https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered)Accessed: 2026\-06\-08Cited by:[§5](https://arxiv.org/html/2607.22595#S5.p3.1)\.
- A\. C\. Stickland, A\. Lyzhov, J\. Pfau, S\. Mahdi, and S\. R\. Bowman \(2024\)Steering without side effects: improving post\-deployment control of language models\.InNeurips Safe Generative AI Workshop 2024,External Links:[Link](https://openreview.net/forum?id=tfXIZ8P4ZU)Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.13.12.2.1.1),[§1](https://arxiv.org/html/2607.22595#S1.p2.1),[Table 1](https://arxiv.org/html/2607.22595#S3.T1.2.7.6.2.1.1)\.
- P\. Tillet, H\. T\. Kung, and D\. Cox \(2019\)Triton: an intermediate language and compiler for tiled neural network computations\.InProceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages,pp\. 10–19\.External Links:[Document](https://dx.doi.org/10.1145/3315508.3329973),[Link](https://doi.org/10.1145/3315508.3329973)Cited by:[§2](https://arxiv.org/html/2607.22595#S2.p5.1)\.
- A\. M\. Turner, L\. Thiergart, G\. Leech, D\. Udell, J\. J\. Vazquez, U\. Mini, and M\. MacDiarmid \(2025\)Steering language models with activation engineering\.External Links:[Link](https://openreview.net/forum?id=2XBPdPIcFK)Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.3.2.2.1.1)\.
- vLLM Project \(2026a\)Benchmark sweep: serve\.Note:[https://docs\.vllm\.ai/en/latest/cli/bench/sweep/serve](https://docs.vllm.ai/en/latest/cli/bench/sweep/serve)Accessed: 2026\-06\-08Cited by:[§5](https://arxiv.org/html/2607.22595#S5.p2.1)\.
- vLLM Project \(2026b\)vLLM KernelConfig api reference\.Note:[https://docs\.vllm\.ai/en/stable/api/vllm/config/kernel/](https://docs.vllm.ai/en/stable/api/vllm/config/kernel/)Official vLLM documentation\. Documents Triton\-based fused MoE kernels as a supported backend\. Accessed 2026\-05\-05\.Cited by:[§2](https://arxiv.org/html/2607.22595#S2.p5.1)\.
- \[29\]\(2026\)VLLM\-lens\.Note:Accessed: 2026\-05\-10External Links:[Link](https://www.lesswrong.com/posts/3bs27nZQuEcKhXf7q/vllm-lens-fast-interpretability-tooling-that-scales-to)Cited by:[§1](https://arxiv.org/html/2607.22595#S1.p3.1),[§2](https://arxiv.org/html/2607.22595#S2.p4.1),[§2](https://arxiv.org/html/2607.22595#S2.p7.1)\.
- M\. Wang, X\. Chen, Y\. Wang, Z\. He, J\. Xu, T\. Liang, Q\. Liu, Y\. Yao, W\. Wang, R\. Ma, H\. Mi, N\. Zhang, Z\. Tu, X\. Li, and D\. Yu \(2026\)Two experts are all you need for steering thinking: reinforcing cognitive effort in moe reasoning models without additional training\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=x7fCiuCCAu)Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.15.14.2.1.1)\.
- P\. Wang, N\. Zhang, B\. Tian, Z\. Xi, Y\. Yao, Z\. Xu, M\. Wang, S\. Mao, X\. Wang, S\. Cheng, K\. Liu, Y\. Ni, G\. Zheng, and H\. Chen \(2024\)EasyEdit: an easy\-to\-use knowledge editing framework for large language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 3: System Demonstrations\),Bangkok, Thailand,pp\. 82–93\.External Links:[Link](https://aclanthology.org/2024.acl-demos.9/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-demos.9)Cited by:[§2](https://arxiv.org/html/2607.22595#S2.p4.1),[§2](https://arxiv.org/html/2607.22595#S2.p7.1)\.
- W\. Wang, M\. Wu, B\. Haddow, and A\. Birch \(2025a\)Bridging the language gaps in large language models with inference\-time cross\-lingual intervention\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 5418–5433\.External Links:[Link](https://aclanthology.org/2025.acl-long.270/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.270),ISBN 979\-8\-89176\-251\-0Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.5.4.2.1.1)\.
- W\. Wang, J\. YANG, and W\. Peng \(2025b\)Semantics\-adaptive activation intervention for LLMs via dynamic steering vectors\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=8WQ7VTfPTl)Cited by:[Table 4](https://arxiv.org/html/2607.22595#A1.T4.2.8.7.2.1.1)\.
- H\. Xu, X\. Mei, Y\. Yan, R\. Zhou, W\. Zhang, W\. Lu, Y\. Zhuang, and Y\. Shen \(2025a\)EasySteer: a unified framework for high\-performance and extensible llm steering\.External Links:2509\.25175,[Link](https://arxiv.org/abs/2509.25175)Cited by:[§5](https://arxiv.org/html/2607.22595#S5.p9.2)\.
- H\. Xu, X\. Mei, Y\. Yan, R\. Zhou, W\. Zhang, W\. Lu, Y\. Zhuang, and Y\. Shen \(2025b\)EasySteer: a unified framework for high\-performance and extensible LLM steering\.arXiv preprint arXiv:2509\.25175\.Note:arXiv preprint used because no published venue was found\.Cited by:[§2](https://arxiv.org/html/2607.22595#S2.p4.1),[§2](https://arxiv.org/html/2607.22595#S2.p7.1)\.
- Z\. Xu, S\. Wang, K\. Xu, H\. Xu, M\. Wang, X\. Deng, Y\. Yao, G\. Zheng, H\. Chen, and N\. Zhang \(2025c\)EasyEdit2: an easy\-to\-use steering framework for editing large language models\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,Suzhou, China,pp\. 522–535\.External Links:[Link](https://aclanthology.org/2025.emnlp-demos.38/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-demos.38)Cited by:[§2](https://arxiv.org/html/2607.22595#S2.p4.1),[§2](https://arxiv.org/html/2607.22595#S2.p7.1)\.

## Appendix AFull Survey Mapping

This appendix will map each surveyed[MI](https://arxiv.org/html/2607.22595#id7.7.id7)application to the representative application used in[Table˜1](https://arxiv.org/html/2607.22595#S3.T1)\.

Table 4:Surveyed\-app to representative\-app mapping referenced in[Section˜3](https://arxiv.org/html/2607.22595#S3)\.
## Appendix BCuda graph support on other frameworks

EasySteerstarted implementing single layer and multi layer steering on Qwen2\.5\-1\.5B\-Instruct model with cuda graph supportEasySteer \[[2026](https://arxiv.org/html/2607.22595#bib.bib69)\]\. SinceEasySteerdoesn’t support any of the applications with cuda graphs, we couldn’t provide this comparison in[Table˜2](https://arxiv.org/html/2607.22595#S5.T2)\. We will also notice, thatxMIxis the only framework offering support for different applications and primitives \(Read, Write, Conditional Write\) while preserving full cuda graph compatibility\.