// Neural procedural memory //

Good paper on agent memory beyond prompt retrieval.

NPM stores procedural skills as activation steering vectors distilled from contrastive historical experience.

Textual memory can tell an agent what to do, but it could fail to activate the internal behavior needed to execute the skill correctly.

This is a useful direction for agents that need persistent know-how across environment interactions.

Memory does not always have to be another paragraph in the context window.

Paper: https://arxiv.org/abs/2606.29824

Learn to build effective AI agents in our academy: https://academy.dair.ai

Neural Procedural Memory: Empowering LLM Agents with Implicit Activation Steering

Source: https://arxiv.org/html/2606.29824 Chengfeng Zhao1,2,Yuqiao Tan1,2,Shizhu He1,2,Yequan Wang3,Jun Zhao1,2,Kang Liu1,2,

1Institute of Automation, CAS2University of Chinese Academy of Sciences 3Beijing Academy of Artificial Intelligence {zhaochengfeng2024, tanyuqiao2025}@ia.ac.cn,{shizhu.he, kliu, jzhao}@nlpr.ia.ac.cn [email protected]

Abstract

While Large Language Models (LLMs) excel as static solvers, transforming them into autonomous agents remains challenging. This transition requires continuous environmental interaction, yet current agents lack the necessary persistent procedural memory. Existing approaches predominantly employ Retrieval-Augmented Generation (RAG) to inject explicit textual guidelines into model contexts. However, relying solely on symbolic instructions can introduce a text-action disconnect, frequently failing to activate the internal representations necessary for correct task execution. To address this, the paper introducesNeural Procedural Memory (NPM), a training-free framework that represents agent memory through implicit activation steering rather than explicit instructions. By distilling procedural skills from historical contrastive experiences into steering vectors in the activation space, NPM directly activates the task-relevant neural mechanisms to guide task execution. Evaluations across four agent benchmarks show that NPM performs comparably to baselines using explicit textual instructions. Furthermore, the results show that combining implicit steering with explicit workflows provides complementary advantages, leading to more robust task execution. Representational analyses indicate that these steering vectors encode consistent task logic, forming organized structures within the activation space. These findings suggest that implicit activation steering provides a promising approach for managing agent memory.

Neural Procedural Memory: Empowering LLM Agents with Implicit Activation Steering

Chengfeng Zhao1,2, Yuqiao Tan1,2, Shizhu He1,2, Yequan Wang3, Jun Zhao1,2, Kang Liu1,2††thanks:Corresponding Author,1Institute of Automation, CAS2University of Chinese Academy of Sciences3Beijing Academy of Artificial Intelligence{zhaochengfeng2024, tanyuqiao2025}@ia.ac.cn,{shizhu.he, kliu, jzhao}@[email protected]

1Introduction

Figure 1:Comparison between Declarative and Procedural Memory in LLM Agents.Left: Declarative memory successfully grounds reasoning in static factual knowledge, enabling the agent to strictly adhere to explicit user constraints.Right: Textual procedural memory can introduce a text-action disconnect where the agent struggles to align the retrieved workflow onto the execution trajectory and omits intermediate steps.As Large Language Models (LLMs) evolve from stateless reasoners to the central controllers of autonomous agents, their operational scope has expanded to open-ended, dynamic environments—ranging from web navigation to embodied interactionWanget al.(2023,2024a); Liuet al.(2025). Succeeding in these complex settings requires agents to leverage persistent memory to transform past experiences into reusable skills, making memory a critical component of long-term autonomyParket al.(2023); Zhonget al.(2024); Omidiet al.(2025); Huet al.(2025).

Agent memory functionally consists of declarative memory for descriptive factsWanget al.(2024b); Gutierrezet al.(2024); Chhikaraet al.(2025)and procedural memory for executing action sequencesZhenget al.(2023); Zhaoet al.(2024); Yuet al.(2025a). While declarative recall is straightforward, effectively representing and utilizing procedural memory remains a bottleneckHanet al.(2025); Zhanget al.(2025c). Traditional methods typically embed procedural knowledge offline into model parameters via supervised fine-tuningShaoet al.(2023); Zhanget al.(2025b)or reinforcement learningKimet al.(2023); Yaoet al.(2024). However, these parameter-updating approaches incur high computational overhead and struggle to adapt in real time to dynamic environmentsZhaiet al.(2023); Xiaet al.(2024); Luoet al.(2025).

By contrast, recent studies have focused on the second paradigm: online external knowledge retrieval, most notably Retrieval-Augmented Generation (RAG). Although successful with declarative memory, such as fact retrievalZhonget al.(2024); Packeret al.(2024); Chhikaraet al.(2025), this solution encounters limitations when applied directly to procedural memory. Unlike static facts, procedural memory is inherently implicit and abstract. Such knowledge is often ineffable: compressing complex reasoning patterns into discrete natural language tokens inevitably leads to information lossMahowaldet al.(2024). Relying exclusively on explicit textual descriptions to transfer procedural skills can lead to a text-action disconnect. Agents might comprehend retrieved instructions yet fail to strictly act upon themWenet al.(2024); Sunet al.(2025); Genget al.(2025). As shown in Figure1, while declarative memory successfully grounds static constraints, conveying procedural workflows through text fails to translate into sequential actions, leading agents to omit critical intermediate steps during execution. When relying solely on texts as a reasoning medium, symbolic descriptions of actions can sometimes fail to consistently elicit the targeted behavioral trajectories required for task completion.

Therefore, it is intuitive that procedural memory could benefit from mechanisms more closely aligned with intrinsic behavioral modulation rather than relying solely on external text. Such a perspective is corroborated by cognitive neuroscienceSquire (1992,2004), which posits that procedural memory is non-verbalizable and manifests through the modulation of neural activity rather than explicit declarative recall.

Following this insight, we proposeNeural Procedural Memory (NPM), a training-free framework that introduces an alternative paradigm of agent memory based on implicit activation steering. Instead of retrieving text for the agent to read, NPM identifies relevant historical tasks and extracts procedural signals from contrastive experiences into steering vectors within the activation space. Specifically, we employ a dual-granularity strategy to address the challenge of agents failing to obtain successful trajectories in complex tasks: inter-trajectory contrast aligns successful and failed trajectories, while intra-trajectory contrast exploits step-level differences within individual failed attempts. During inference, the steering vectors are dynamically synthesized and injected into the residual stream, directly modulating the agent’s reasoning process and action selection without expanding the context window or updating parameters.

We evaluate NPM across four agent benchmarks including ALFWorldShridharet al.(2021), WebShopYaoet al.(2022), ScienceWorldWanget al.(2022), and BabyAIChevalier-Boisvertet al.(2019). Results demonstrate that implicit activation steering achieves performance comparable to explicit textual memory baselines and exhibits complementary synergy when combined with explicit textual workflows. While textual memory provides high-level symbolic guidance, implicit steering reinforces procedural adherence directly within the neural representation. We further dissect the extracted vectors to reveal semantically meaningful behavioral primitives, providing an interpretable view of how procedural knowledge is represented in activation space. These findings suggest implicit activation steering as an effective path for representing procedural memory in LLM agents.

Our contributions are summarized as follows:

• •This paper proposes Neural Procedural Memory (NPM), a training-free framework that integrates implicit activation steering into agent memory, enabling effective intrinsic behavioral modulation.
• •This paper introduces a dual-granularity extraction method using both inter-trajectory and intra-trajectory contrasts, offering a flexible intervention mechanism and mitigating the reliance on successful demonstrations.
• •Extensive experiments demonstrate that activation steering serves as an effective carrier for agentic skills, yielding interpretable vectors that correlate with specific task behaviors.

Figure 2:Overview of the Neural Procedural Memory (NPM) framework.(1)Contrastive Experience Construction:Formulating dual-granularity (inter- and intra-trajectory) contrastive pairs from historical interactions. (2)Procedural Memory Extraction:Extracting continuous representations from these pairs to construct a historical memory repository. (3)Inference-Time Intervention:Retrieving relevant experiences to dynamically synthesize a steering vector, which is injected into the residual stream to guide agent reasoning.

2Related Work

Functions of Agent Memory

Agent memory is typically categorized into factual (declarative) and experiential (procedural) componentsHuet al.(2025). While Retrieval-Augmented Generation (RAG) effectively maintains declarative factsHuet al.(2025); Parket al.(2023), it faces limitations when applied to procedural memory. Existing approaches typically approximate procedural memory by injecting textual guidelines or code snippets into the context windowShinnet al.(2023); Wanget al.(2023). However, this explicit approach suffers from a text-action disconnect: natural language serves as an inefficient proxy for encoding internal neural representationsMahowaldet al.(2024). Moreover, the reliance on lengthy text instructions increases computational cost and degrades performance in long-horizon tasksGenget al.(2025); Sinhaet al.(2025). NPM shifts from explicit verbalization to implicit activation modulation, eliminating the need for in-context instructions.

Forms of Agent Memory

Current approaches explore various representational modalities. Token-level methods (e.g., RAG) offer flexibility but are bounded by context window limits and inference latency. Conversely, parametric approaches encode knowledge within model weights permanentlyZhanget al.(2025b); Wanget al.(2025a), offering efficiency but lacking the flexibility required for rapid adaptation or unlearning without retraining. Latent memory offers a middle ground by storing information as continuous vectors. However, existing works primarily focus on context compressionChevalieret al.(2023); Wanget al.(2024c)or transient working memoryZhanget al.(2025a); Yuet al.(2025b). For instance, MemGenZhanget al.(2025a)synthesizes latent tokens during reasoning but requires auxiliary encoders and extensive training. While NPM shares the underlying concept of latent memory, it views memory storage as activation steering, enabling representation-level modulation of the generation path without additional token overhead or parameter updates.

Representation Engineering and Activation Steering.

Our framework builds upon Representation Engineering (RepE) that monitors and manipulates high-level cognitive states in the activation spaceZouet al.(2025). Prior studiesRimskyet al.(2024); Leeet al.(2025)have demonstrated the efficacy of steering models toward coarse-grained, static attributes such as honesty or harmlessness. However, existing RepE work is predominantly limited to single-turn global concepts. NPM extends these techniques to dynamic, task-specific agentic trajectories. By introducing a retrieval-augmented framework where steering vectors are synthesized from contrastive historical experiences, we enable agents to adaptively recall and apply procedural skills specific to the task context.

3Neural Procedural Memory

We introduceNeural Procedural Memory (NPM), a training-free framework designed to internalize agentic skills directly into the model’s activation space. The framework shown in Figure2consists of three phases: (1)Contrastive Experience Construction(§3.1), which isolates effective reasoning patterns from historical failures; (2)Steering Vector Extraction(§3.2), which distills discrete textual contrasts into continuous steering vectors; and (3)Inference-Time Intervention(§3.3), where these vectors are dynamically synthesized and injected to modulate the agent’s behavior.

3.1Contrastive Experience Construction

Formally, given a task queryqqand a task environmentℰ\mathcal{E}, an agent generates a trajectoryτ={s1,a1,…,sK,aK}\tau=\{s_{1},a_{1},\dots,s_{K},a_{K}\}, wheresks_{k}denotes the state andaka_{k}denotes the action at stepkk. The objective is to construct a contrastive dataset𝒟\mathcal{D}composed of paired reasoning segments(x+,x−)(x^{+},x^{-}), wherex+x^{+}represents a desirable reasoning mode andx−x^{-}represents a degenerate one. A dual-granularity strategy is employed to capture procedural signals at both the inter-trajectory and intra-trajectory levels.

3.1.1Inter-trajectory Contrast

For tasks where the agent has historically generated both successful and failed attempts, we aim to capture the global behavioral shift. Let𝒯+\mathcal{T}^{+}and𝒯−\mathcal{T}^{-}denote the sets of successful and failed trajectories for a specific task, respectively. A contrastive pair is formed by aligning a failed attempt with a successful one:

𝒫inter={(τ+,τ−)∣τ+∈𝒯+,τ−∈𝒯−}.\mathcal{P}_{\text{inter}}=\big\{(\tau^{+},\tau^{-})\mid\tau^{+}\in\mathcal{T}^{+},\tau^{-}\in\mathcal{T}^{-}\big\}.(1) This contrast directionally distinguishes successful trajectories from failed attempts.

3.1.2Intra-trajectory Contrast

In many complex tasks, the agent frequently fails to obtain successful trajectories, resulting in a sparse distribution of positive samples. To enable learning from failure, we exploit step-level differences within a single failed trajectory. ADegenerate StepsetSdeg⊂τS_{\text{deg}}\subset\tauis defined to contain actions that exhibit irrational behaviors or violate environmental constraints, identified via the following criteria:

• •Redundancy:Consecutive repetition of an identical command, indicating a heuristic loop or failure to progress without state change.
• •Invalidity:Actions that trigger environment-specific error feedback, such as format violations or inadmissible moves.

Conversely, anEffectivestep setSeff=τ∖SdegS_{\text{eff}}=\tau\setminus S_{\text{deg}}is defined to contain steps validly advancing the state.

Instead of pairing individual steps, which serves as a noisy signal, we aim to contrast the collective effective behavior against the degenerate one within the trajectory. The contrastive pair is constructed by grouping the steps into two sets:

𝒫intra={(Seff,Sdeg)∣Sdeg≠∅}.\mathcal{P}_{\text{intra}}=\big\{(S_{\text{eff}},S_{\text{deg}})\mid S_{\text{deg}}\neq\emptyset\big\}.(2) This intra-trajectory contrast isolates specific degenerate modes from the otherwise valid reasoning process. (Quantitative analysis of heuristic isolation rules is provided in AppendixC.2).

3.2Procedural Memory Extraction

The core of NPM is to map the textual contrast defined in𝒟\mathcal{D}into a geometric transformation in the continuous representation space. The goal is to derive a vector𝐯∈ℝd\mathbf{v}\in\mathbb{R}^{d}representing the shift from a degenerate state𝐡−\mathbf{h}^{-}to a desirable one𝐡+\mathbf{h}^{+}.

Hidden State Representation.

We define the representation functionϕl​(⋅)\phi_{l}(\cdot)to extract a fixed-dimensional representation vector𝐡∈ℝd\mathbf{h}\in\mathbb{R}^{d}depending on the contrast granularity.

• •Inter-trajectory:For this comparison, we utilize the sequence of hidden states of the full trajectoryτ\tauand extract the last token to capture the accumulated context for each layer.
• •Intra-trajectory:Since reasoning steps span multiple tokens, we apply mean-pooling over the tokens to obtain a step representation: ϕl​(S)=1|S|​∑s∈S(1|s|​∑i=1|s|𝐡l,i(s)).\phi_{l}(S)=\frac{1}{|S|}\sum_{s\in S}\left(\frac{1}{|s|}\sum_{i=1}^{|s|}\mathbf{h}_{l,i}^{(s)}\right).(3) Here,|S||S|denotes the total number of steps in the set,|s||s|denotes the length of an individual stepss, and𝐡l,i(s)\mathbf{h}_{l,i}^{(s)}corresponds to the activation of theii-th token within stepssat layerll.

Memory Storage.

For each historical taskQjQ_{j}, we organize its extracted representations into a task-specific contrastive pair set𝒞l(j)={(ϕl​(xi+),ϕl​(xi−))}i=1Nj\mathcal{C}_{l}^{(j)}=\{(\phi_{l}(x_{i}^{+}),\phi_{l}(x_{i}^{-}))\}_{i=1}^{N_{j}}, whereNjN_{j}denotes the total number of contrastive pairs collected for the task and(xi+,xi−)(x_{i}^{+},x_{i}^{-})represents theii-th pair corresponding to the granularities defined in §3.1. These pre-computed sets serve as the raw procedural memory stored in our repository.

3.3Inference-Time Intervention

NPM applies procedural memory dynamically at inference time without modifying model weights. To ensure low latency, we pre-compute and store the hidden state representations defined in §3.2for all historical experiences. The inference process follows a Retrieval-Synthesis-Intervention pipeline:

1. 1.Contextual Retrieval:Given a new test taskqq, we utilize a dense retriever to identify the top-KKmost similar historical tasksℛq\mathcal{R}_{q}.
2. 2.Dynamic Vector Synthesis:We fetch the stored representation sets for the retrieved tasks to form a collective memory pool:ℳl​(q)=⋃j∈ℛq𝒞l(j)\mathcal{M}_{l}(q)=\bigcup_{j\in\mathcal{R}_{q}}\mathcal{C}_{l}^{(j)}. The synthesis strategy selects between inter- and intra-trajectory contrasts. A task-specific consensus direction𝐯l​(q)\mathbf{v}_{l}(q)is then derived from the collective memory pool via extraction functionψ​(⋅)\psi(\cdot): 𝐯l​(q)=ψ​(ℳl​(q))\mathbf{v}_{l}(q)=\psi\big(\mathcal{M}_{l}(q)\big)(4)
3. 3.Activation Steering:During autoregressive generation, we inject the synthesized vector into the residual stream at each time steptt. For a target layerll, the intervened activation𝐡~l,t\tilde{\mathbf{h}}_{l,t}is computed as: 𝐡~l,t=𝐡l,t+α⋅𝐯l​(q)\tilde{\mathbf{h}}_{l,t}=\mathbf{h}_{l,t}+\alpha\cdot\mathbf{v}_{l}(q)(5) where𝐡l,t\mathbf{h}_{l,t}denotes the original hidden state of the current token, andα\alphais a scalar parameter controlling the intervention strength.

By shifting the activation distribution, NPM implicitly nudges the agent’s intuition away from known failure modes and towards effective reasoning paths. Detailed storage and latency analyses are provided in AppendixD.

4Experiments

4.1Experimental Setup

We evaluate the proposed framework using MiniCPM3-4BHuet al.(2024)and Qwen3 models (4B and 8B)Yanget al.(2025)across four agent benchmarks: ALFWorldShridharet al.(2021), WebShopYaoet al.(2022), ScienceWorldWanget al.(2022), and BabyAIChevalier-Boisvertet al.(2019). The evaluation relies on the success rate for ALFWorld and the average reward for the remaining environments. Detailed descriptions of each simulated environment and task configurations are deferred to AppendixA.

We compareNPMagainst baselines representing no memory, explicit textual memory, and implicit activation steering. The explicit baselines augment the model context, utilizingInsightsZhaoet al.(2024)for global procedural guidelines orWorkflowsWanget al.(2025b)for concrete execution patterns. The implicit steering baselines includeCAAPanicksseryet al.(2024)andMass-MeanMarks and Tegmark (2024), applying fixed activation shifts computed across the entire dataset. Our approach differs from these static methods by dynamically synthesizing task-specific intervention vectors from contrastive experiences. We additionally evaluate aHybridconfiguration combining implicit steering with explicit workflows to examine whether the two representation modalities offer complementary advantages.

4.2Main Results

Table 1:The overall performance comparison.We report the success rate (%) for ALFWorld and the average reward for WebShop, ScienceWorld, and BabyAI. “Avg” denotes the average score across the four benchmarks.ModelParadigmMethodALFWorldWebShopScienceWorldBabyAIAvgMiniCPM3-4BNoneNo Memory23.8832.217.2927.0022.60ExplicitInsights28.3647.9610.027.7828.53Workflows32.8454.9710.532.3932.68ImplicitCAA23.8813.856.2930.2918.58Mass-Mean26.8734.218.0229.7324.71NPM (Ours)31.3443.538.7031.9128.87HybridNPM + Workflows38.8151.2110.9736.9034.47Qwen3-4BNoneNo Memory30.6044.8118.4018.7528.14ExplicitInsights44.0340.7222.5912.0929.86Workflows52.9945.7323.5914.6034.23ImplicitCAA26.8725.5211.483.4816.84Mass-Mean29.1039.0718.1617.8226.04NPM (Ours)40.3048.0018.2019.0431.39HybridNPM + Workflows61.1947.8421.9719.3837.60Qwen3-8BNoneNo Memory39.5546.2524.9811.7430.63ExplicitInsights46.2745.2228.097.7731.84Workflows62.6948.9329.5710.4037.90ImplicitCAA47.0120.669.1714.3122.79Mass-Mean49.2544.2625.328.7131.89NPM (Ours)56.7248.2426.0714.2636.32HybridNPM + Workflows66.4253.9431.8915.3141.89

The experimental results are presented in Table1.

Effectiveness of Implicit Steering.

NPM improves performance across most evaluated model configurations and environments compared to the base model without memory. For example, the average score for MiniCPM3-4B increases from 22.60 to 28.87 with implicit steering, and Qwen3-8B improves from 30.63 to 36.32. Furthermore, NPM outperforms static baselines such as CAA and Mass-Mean. These alternative methods struggle to capture multi-step procedural skills since they rely on fixed mean differences computed across the entire dataset. The performance difference indicates that retrieving contextually relevant historical pairs and adapting the intervention to the specific task provides a more accurate behavioral correction.

Competitiveness with Explicit Memory.

Implicit activation steering achieves results comparable to explicit textual paradigms. On average, NPM outperforms the Insights approach, demonstrating that direct intrinsic modulation can be more effective than broad textual principles. While the more structured Workflows baseline shows stronger performance by offering explicit step-by-step guidance, NPM remains competitive and outperforms in certain settings. On the WebShop using Qwen3-4B, NPM scores 48.00, exceeding the 45.73 achieved by explicit workflows. While textual workflows provide specific steps, translating these textual instructions into actions can introduce a text-action disconnect during long sequences (Detailed qualitative case study is provided in AppendixE.2). Furthermore, maintaining extensive textual memory consumes context window space and increases processing overhead. NPM offers an efficient alternative by guiding the model directly in the activation space, delivering competitive performance without occupying context window tokens.

Complementary Synergy of Hybrid Memory.

Integrating implicit steering with explicit workflows yields the highest overall performance, setting the top average scores across all three backbone models. Combining both memory forms on Qwen3-8B elevates the success rate on ALFWorld to 66.42 percent and reaches a peak score of 31.89 on ScienceWorld. This synergy indicates that explicit text and implicit representations operate as a complementary system. Textual workflows supply high-level symbolic planning steps, whereas NPM provides direct neural intervention to ensure adherence to these plans during long execution horizons. This persistent modulation helps prevent the agent from deviating from prompt-based instructions during extended interactions.

4.3Dual-Granularity Contrasts

Tasks across different environments vary in complexity, therefore a single level of intervention may not always provide the appropriate correction. We evaluate the individual effects of intra-trajectory and inter-trajectory steering to understand this behavior. As shown in Figure3, relying exclusively on a single granularity results in performance variance across different benchmarks and model architectures. Intra-trajectory steering performs well in step-intensive environments such as ALFWorld by correcting local action errors. Inter-trajectory steering proves more effective when agents require macro-level planning guidance like WebShop. To balance these effects, we dynamically select the appropriate intervention granularity based on the task context. This combination reduces performance variance and provides stable generalization across configurations. Although the implemented selection strategy occasionally yields lower success rates than an oracle baseline in certain boundary cases, the empirical results confirm that the dual-granularity contrast provides a reliable mechanism for managing procedural interventions. The representational differences underlying these two steering granularities are further examined in Section5.

Figure 3:Performance comparison of different steering granularities across multiple environments and models.

5Analysis

5.1Representational Separability of Activations

Effective activation steering relies on the assumption that successful patterns and failure modes occupy distinct regions in the representation space. Projecting the hidden states from the intervention layer onto their primary axes reveals a clear geometric separation. As illustrated in Figure4, the projections for both inter-trajectory and intra-trajectory contrasts show that successful paths and degenerate modes form distinct clusters instead of overlapping. We also confirm this linear separability in the original high-dimensional space by training a linear classifier, which achieves high accuracy in distinguishing between the two classes (detailed in AppendixC.1). This geometric separation suggests that the behavioral shift from failure to success can be projected onto a prominent direction. Extracting the first principal component of the contrastive differences helps capture this main axis of variation, mitigating some instance-specific noise.

(a)Inter (b)Inter (c)Intra (d)Intra

Figure 4:PCA projections of hidden states at Layer 18 for Qwen3-4B on AlfWorld. Positive (green) and negative (red) representations exhibit distinct clustering along the primary axis of variation.(a)Inter (By Task) (b)Intra (By Task) (c)Inter (By Interaction) (d)Intra (By Interaction)

Figure 5:Geometric consistency of procedural steering vectors. Heatmaps display the pairwise cosine similarity of centered vectors. (a-b) Vectors clustered by task type; (c-d) Vectors clustered by interaction target. The diagonal block structure suggests that the extracted vectors align with task-specific categories and share related behavioral features.Figure 6:Temporal activation of behavioral primitives across execution steps in a PickHeat task.(Left)The unsteered baseline exhibits an extended 33-step trajectory.(Middle)Inter-trajectorysteering is associated with a shorter trajectory by promoting early planning and object placement.(Right)Intra-trajectorysteering is observed to amplify search features and suppress premature task termination signals during execution.

5.2Consistency of Steering Vectors

We evaluate the structural properties of the steering vectors by computing their pairwise cosine similarities across different task instances. The heatmaps in Figures5(a)and5(b)demonstrate that these vectors maintain high internal consistency within specific task categories. Tasks such asLookAtinvolve distinct interaction logic and show low similarity to thePickCool,PickHeat, andPickTwogroups. These three categories display inter-task similarity alongside their high intra-task consistency because they share the fundamentalPicksub-action. This overlapping pattern indicates the framework captures common procedural knowledge across related behaviors. The resulting diagonal block structure suggests that the memory extraction process yields representations that align with task-specific categories rather than random noise.

Grouping the vectors by specific interaction targets in Figures5(c)and5(d)highlights functional differences between the two extraction strategies. The intra-trajectory contrasts form diagonal block structures with high similarity. Local error-correction signals associated with particular items like aDeskLamporDrawerremain uniform across different task instances. Correcting localized degenerate steps relies primarily on the intrinsic properties of the target object rather than the broader task context. The inter-trajectory contrasts show a less structured pattern with lower consistency. Global reasoning paths depend on the initial state of the agent, environmental layouts, and prior execution sequences. This contextual variance prevents inter-trajectory vectors from forming the distinct clusters observed in the local signals.

Figure 7:The impact of retrieval pool size on specific behavioral primitive activations (Solid lines for positive features; dashed lines for negative features).

5.3Feature Decomposition and Interpretability of Steering Vectors

To understand the behavioral semantics encoded within the steering vectors, we decompose the continuous representations into interpretable basis directions using sparse dictionary learning (detailed in AppendixB). Each direction corresponds to specific behavioral primitives, which we validate by computing the mutual information between basis activations and the generated action types across the dataset. Projecting the steering vectors onto these bases reveals that the two contrastive strategies operate through distinct mechanistic pathways.

ThePickHeattask in Figure6illustrates how the two extraction strategies modulate representations through distinct mechanisms. Without intervention, the baseline model generates an extended 33-step trajectory. Applying inter-trajectory steering leads to global behavioral changes. This vector increases early planning features (F8) and sustains activations related to concrete object placement (F6), shortening the execution to 14 steps. Intra-trajectory steering targets local execution errors instead of broad execution plans. It corrects action loops by amplifying features associated with targeted searching (F13) and container interaction (F14) while suppressing signals that trigger premature task termination (F7). This difference in activation patterns suggests that the steering vectors target distinct aspects of model behavior. They systematically adjust representations to address different types of procedural errors (see more quantitative analysis in AppendixC.3).

5.4Retrieval Size and Dynamic Synthesis

To investigate how the retrieval size affects the extracted procedural signals, we track the activation of behavioral primitives as the number of aggregated contrastive pairs increases. As shown in Figure7, features display two distinct scaling patterns depending on their universality. For general behaviors, such as basic exploration (F9) and redundant observation (F0), activation magnitudes grow continuously as the size expands. Since these patterns are shared across most tasks, they remain prominent when aggregated across the entire dataset. Conversely, task-specific primitives exhibit a different pattern. Features tied to explicit procedural logic, such as placing objects (F6), systematic searching (F13), or preventing premature completion (F7), reach peak activation at a moderate retrieval scale before declining. While minimal pools fail to separate the core procedural signal from instance-level noise, overly large pools introduce cross-task interference that suppresses specific task rules. This observation confirms the necessity of dynamic synthesis: constraining the retrieval to a contextually relevant subset effectively balances noise reduction with signal preservation.

6Conclusion

We propose Neural Procedural Memory (NPM), a training-free framework that represents and applies procedural behaviors through implicit activation steering. By distilling procedural knowledge from dual-granularity contrastive experiences, NPM provides an alternative paradigm for encoding memory that operates directly in the activation space. Extensive experiments show that NPM achieves performance comparable to textual baselines and provides complementary benefits when combined with explicit workflows. By utilizing the continuous activation space to encode procedural skills, NPM provides a practical foundation for managing procedural memory in LLM agents.

Limitations

While our proposed approach demonstrates promising results, there are several aspects that could be improved. Firstly, the framework requires direct intervention in the residual stream, restricting its application to open architectures where internal activation spaces are completely accessible. Furthermore, building the contrastive repository relies on the agent occasionally generating successful trajectories, which introduces a cold-start problem in complex environments. Additionally, extracting degenerate steps relies on heuristics such as action redundancy or format invalidity. These conditions struggle to identify implicit logical fallacies without explicit environment errors. Finally, synthesized vectors compress procedural knowledge into a single representation applied constantly during generation. This static modulation lacks the flexibility to shift between behavioral primitives across different execution stages. Exploring dynamic interventions that adapt throughout task execution is a direction for future work.

References

• S. Bills, N. Cammarata, D. Mossing, H. Tillman, L. Gao, G. Goh, I. Sutskever, J. Leike, J. Wu, and W. Saunders (2023)Language models can explain neurons in language models.Note:https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.htmlCited by:§B.1.
• A. Chevalier, A. Wettig, A. Ajith, and D. Chen (2023)Adapting language models to compress contexts.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H. Bouamor, J. Pino, and K. Bali (Eds.),Singapore,pp. 3829–3846.External Links:Link,DocumentCited by:§2.
• M. Chevalier-Boisvert, D. Bahdanau, S. Lahlou, L. Willems, C. Saharia, T. H. Nguyen, and Y. Bengio (2019)BabyAI: first steps towards grounded language learning with a human in the loop.InInternational Conference on Learning Representations,External Links:LinkCited by:§A.1,§1,§4.1.
• P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav (2025)Mem0: building production-ready ai agents with scalable long-term memory.External Links:2504.19413,LinkCited by:§1,§1.
• Y. Geng, H. Li, H. Mu, X. Han, T. Baldwin, O. Abend, E. Hovy, and L. Frermann (2025)Control illusion: the failure of instruction hierarchies in large language models.External Links:2502.15851,LinkCited by:§1,§2.
• B. J. Gutierrez, Y. Shu, Y. Gu, M. Yasunaga, and Y. Su (2024)HippoRAG: neurobiologically inspired long-term memory for large language models.InThe Thirty-eighth Annual Conference on Neural Information Processing Systems,External Links:LinkCited by:§1.
• D. Han, C. Couturier, D. M. Diaz, X. Zhang, V. Rühle, and S. Rajmohan (2025)LEGOMem: modular procedural memory for multi-agent llm systems for workflow automation.External Links:2510.04851,LinkCited by:§1.
• S. Hu, Y. Tu, X. Han, C. He, G. Cui, X. Long, Z. Zheng, Y. Fang, Y. Huang, W. Zhao,et al.(2024)MiniCPM: unveiling the potential of small language models with scalable training strategies.arXiv preprint arXiv:2404.06395.Cited by:§4.1.
• Y. Hu, S. Liu, Y. Yue, G. Zhang, B. Liu, F. Zhu, J. Lin, H. Guo, S. Dou, Z. Xi, S. Jin, J. Tan, Y. Yin, J. Liu, Z. Zhang, Z. Sun, Y. Zhu, H. Sun, B. Peng, Z. Cheng, X. Fan, J. Guo, X. Yu, Z. Zhou, Z. Hu, J. Huo, J. Wang, Y. Niu, Y. Wang, Z. Yin, X. Hu, Y. Liao, Q. Li, K. Wang, W. Zhou, Y. Liu, D. Cheng, Q. Zhang, T. Gui, S. Pan, Y. Zhang, P. Torr, Z. Dou, J. Wen, X. Huang, Y. Jiang, and S. Yan (2025)Memory in the age of ai agents.External Links:2512.13564,LinkCited by:§1,§2.
• T. Kim, M. Cochez, V. François-Lavet, M. Neerincx, and P. Vossen (2023)A machine with short-term, episodic, and semantic memory systems.InProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence,AAAI’23/IAAI’23/EAAI’23.External Links:ISBN 978-1-57735-880-0,Link,DocumentCited by:§1.
• B. W. Lee, I. Padhi, K. N. Ramamurthy, E. Miehling, P. Dognin, M. Nagireddy, and A. Dhurandhar (2025)Programming refusal with conditional activation steering.InThe Thirteenth International Conference on Learning Representations,External Links:LinkCited by:§2.
• B. Liu, X. Li, J. Zhang, J. Wang, T. He, S. Hong, H. Liu, S. Zhang, K. Song, K. Zhu, Y. Cheng, S. Wang, X. Wang, Y. Luo, H. Jin, P. Zhang, O. Liu, J. Chen, H. Zhang, Z. Yu, H. Shi, B. Li, D. Wu, F. Teng, X. Jia, J. Xu, J. Xiang, Y. Lin, T. Liu, T. Liu, Y. Su, H. Sun, G. Berseth, J. Nie, I. Foster, L. Ward, Q. Wu, Y. Gu, M. Zhuge, X. Liang, X. Tang, H. Wang, J. You, C. Wang, J. Pei, Q. Yang, X. Qi, and C. Wu (2025)Advances and challenges in foundation agents: from brain-inspired intelligence to evolutionary, collaborative, and safe systems.External Links:2504.01990,LinkCited by:§1.
• Y. Luo, Z. Yang, F. Meng, Y. Li, J. Zhou, and Y. Zhang (2025)An empirical study of catastrophic forgetting in large language models during continual fine-tuning.IEEE Transactions on Audio, Speech and Language Processing33(),pp. 3776–3786.External Links:DocumentCited by:§1.
• K. Mahowald, A. A. Ivanova, I. A. Blank, N. Kanwisher, J. B. Tenenbaum, and E. Fedorenko (2024)Dissociating language and thought in large language models.Trends in cognitive sciences28(6),pp. 517–540.Cited by:§1,§2.
• S. Marks and M. Tegmark (2024)The geometry of truth: emergent linear structure in large language model representations of true/false datasets.InFirst Conference on Language Modeling,External Links:LinkCited by:§C.1,§4.1.
• P. Omidi, X. Huang, A. Laborieux, B. Nikpour, T. Shi, and A. Eshaghi (2025)Memory-augmented transformers: a systematic review from neuroscience principles to enhanced model architectures.External Links:2508.10824,LinkCited by:§1.
• C. Packer, S. Wooders, K. Lin, V. Fang, S. G. Patil, I. Stoica, and J. E. Gonzalez (2024)MemGPT: towards llms as operating systems.External Links:2310.08560,LinkCited by:§1.
• N. Panickssery, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. M. Turner (2024)Steering llama 2 via contrastive activation addition.External Links:2312.06681,LinkCited by:§4.1.
• J. S. Park, J. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein (2023)Generative agents: interactive simulacra of human behavior.InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology,UIST ’23,New York, NY, USA.External Links:ISBN 9798400701320,Link,DocumentCited by:§1,§2.
• G. S. Paulo, A. T. Mallen, C. Juang, and N. Belrose (2025)Automatically interpreting millions of features in large language models.InForty-second International Conference on Machine Learning,External Links:LinkCited by:§B.1.
• N. Rimsky, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. Turner (2024)Steering llama 2 via contrastive activation addition.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),L. Ku, A. Martins, and V. Srikumar (Eds.),Bangkok, Thailand,pp. 15504–15522.External Links:Link,DocumentCited by:§2.
• D. Scalena, G. Sarti, and M. Nissim (2024)Multi-property steering of large language models with dynamic activation composition.InProceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP,Y. Belinkov, N. Kim, J. Jumelet, H. Mohebbi, A. Mueller, and H. Chen (Eds.),Miami, Florida, US,pp. 577–603.External Links:Link,DocumentCited by:§A.4.
• Y. Shao, L. Li, J. Dai, and X. Qiu (2023)Character-LLM: a trainable agent for role-playing.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H. Bouamor, J. Pino, and K. Bali (Eds.),Singapore,pp. 13153–13187.External Links:LinkCited by:§1.
• N. Shinn, F. Cassano, A. Gopinath, K. R. Narasimhan, and S. Yao (2023)Reflexion: language agents with verbal reinforcement learning.InThirty-seventh Conference on Neural Information Processing Systems,External Links:LinkCited by:§2.
• M. Shridhar, X. Yuan, M. Côté, Y. Bisk, A. Trischler, and M. Hausknecht (2021)ALFWorld: Aligning Text and Embodied Environments for Interactive Learning.InProceedings of the International Conference on Learning Representations (ICLR),External Links:LinkCited by:§A.1,§1,§4.1.
• A. Sinha, A. Arun, S. Goel, S. Staab, and J. Geiping (2025)The illusion of diminishing returns: measuring long horizon execution in llms.External Links:2509.09677,LinkCited by:§2.
• L. R. Squire (1992)Declarative and nondeclarative memory: multiple brain systems supporting learning and memory.Journal of Cognitive Neuroscience4(3),pp. 232–243.External Links:ISSN 0898-929X,Document,Link,https://direct.mit.edu/jocn/article-pdf/4/3/232/1932203/jocn.1992.4.3.232.pdfCited by:§1.
• L. R. Squire (2004)Memory systems of the brain: a brief history and current perspective.Neurobiology of Learning and Memory82(3),pp. 171–177.Note:Multiple Memory SystemsExternal Links:ISSN 1074-7427,Document,LinkCited by:§1.
• W. Sun, C. Zhang, X. Zhang, X. Yu, Z. Huang, H. Xu, S. He, J. Zhao, and K. Liu (2025)Beyond instruction following: evaluating inferential rule following of large language models.InChinese Computational Linguistics: 24th China National Conference, CCL 2025, Jinan, China, August 11–14, 2025, Proceedings,Berlin, Heidelberg,pp. 408–434.External Links:ISBN 978-981-95-2724-3,Link,DocumentCited by:§1.
• C. Venhoff, I. Arcuschin, P. Torr, A. Conmy, and N. Nanda (2025)Base models know how to reason, thinking models learn when.External Links:2510.07364,LinkCited by:§B.1.
• G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar (2023)Voyager: an open-ended embodied agent with large language models.arXiv preprint arXiv: Arxiv-2305.16291.Cited by:§1,§2.
• L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin,et al.(2024a)A survey on large language model based autonomous agents.Frontiers of Computer Science18(6),pp. 186345.Cited by:§1.
• R. Wang, P. Jansen, M. Côté, and P. Ammanabrolu (2022)Scienceworld: is your agent smarter than a 5th grader?.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,pp. 11279–11298.Cited by:§A.1,§1,§4.1.
• T. Wang, M. Tao, R. Fang, H. Wang, S. Wang, Y. E. Jiang, and W. Zhou (2024b)AI persona: towards life-long personalization of llms.External Links:2412.13103,LinkCited by:§1.
• Y. Wang, Y. Gao, X. Chen, H. Jiang, S. Li, J. Yang, Q. Yin, Z. Li, X. Li, B. Yin, J. Shang, and J. McAuley (2024c)MEMORYLLM: towards self-updatable large language models.InProceedings of the 41st International Conference on Machine Learning,ICML’24.Cited by:§2.
• Y. Wang, X. Liu, X. Chen, S. O’Brien, J. Wu, and J. McAuley (2025a)Self-updatable large language models by integrating context into model parameters.InThe Thirteenth International Conference on Learning Representations,External Links:LinkCited by:§2.
• Z. Z. Wang, J. Mao, D. Fried, and G. Neubig (2025b)Agent workflow memory.InForty-second International Conference on Machine Learning,External Links:LinkCited by:§4.1.
• B. Wen, P. Ke, X. Gu, L. Wu, H. Huang, J. Zhou, W. Li, B. Hu, W. Gao, J. Xu, Y. Liu, J. Tang, H. Wang, and M. Huang (2024)Benchmarking complex instruction-following with multiple constraints composition.InProceedings of the 38th International Conference on Neural Information Processing Systems,NIPS ’24,Red Hook, NY, USA.External Links:ISBN 9798331314385Cited by:§1.
• Y. Xia, J. Kim, Y. Chen, H. Ye, S. Kundu, C. C. Hao, and N. Talati (2024)Understanding the performance and estimating the cost of llm fine-tuning.In2024 IEEE International Symposium on Workload Characterization (IISWC),Vol.,pp. 210–223.External Links:DocumentCited by:§1.
• A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu (2025)Qwen3 technical report.arXiv preprint arXiv:2505.09388.Cited by:§4.1.
• S. Yao, H. Chen, J. Yang, and K. Narasimhan (2022)Webshop: towards scalable real-world web interaction with grounded language agents.Advances in Neural Information Processing Systems35,pp. 20744–20757.Cited by:§A.1,§1,§4.1.
• W. Yao, S. Heinecke, J. C. Niebles, Z. Liu, Y. Feng, L. Xue, R. Murthy, Z. Chen, J. Zhang, D. Arpit, R. Xu, P. Mui, H. Wang, C. Xiong, and S. Savarese (2024)Retroformer: retrospective large language agents with policy gradient optimization.External Links:2308.02151,LinkCited by:§1.
• T. Yu, Z. Zhang, Z. Lyu, J. Gong, H. Yi, X. Wang, Y. Zhou, J. Yang, P. Nie, Y. Huang, and W. Chen (2025a)BrowserAgent: building web agents with human-inspired web browsing actions.External Links:2510.10666,LinkCited by:§1.
• X. Yu, C. Xu, G. Zhang, Z. Chen, Y. Zhang, Y. He, P. Jiang, J. Zhang, X. Hu, and S. Yan (2025b)VisMem: latent vision memory unlocks potential of vision-language models.External Links:2511.11007,LinkCited by:§2.
• Y. Zhai, S. Tong, X. Li, M. Cai, Q. Qu, Y. J. Lee, and Y. Ma (2023)Investigating the catastrophic forgetting in multimodal large language models.ArXivabs/2309.10313.External Links:LinkCited by:§1.
• G. Zhang, M. Fu, and S. Yan (2025a)MemGen: weaving generative latent memory for self-evolving agents.External Links:2509.24704,LinkCited by:§2.
• K. Zhang, X. Chen, B. Liu, T. Xue, Z. Liao, Z. Liu, X. Wang, Y. Ning, Z. Chen, X. Fu, J. Xie, Y. Sun, B. Gou, Q. Qi, Z. Meng, J. Yang, N. Zhang, X. Li, A. Shah, D. Huynh, H. Li, Z. Yang, S. Cao, L. Jang, S. Zhou, J. Zhu, H. Sun, J. Weston, Y. Su, and Y. Wu (2025b)Agent learning via early experience.External Links:2510.08558,LinkCited by:§1,§2.
• Z. Zhang, Q. Dai, X. Bo, C. Ma, R. Li, X. Chen, J. Zhu, Z. Dong, and J. Wen (2025c)A survey on the memory mechanism of large language model-based agents.ACM Trans. Inf. Syst.43(6).External Links:ISSN 1046-8188,Link,DocumentCited by:§1.
• A. Zhao, D. Huang, Q. Xu, M. Lin, Y. Liu, and G. Huang (2024)ExpeL: llm agents are experiential learners.InProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence,AAAI’24/IAAI’24/EAAI’24.External Links:ISBN 978-1-57735-887-9,Link,DocumentCited by:§1,§4.1.
• L. Zheng, R. Wang, X. Wang, and B. An (2023)Synapse: trajectory-as-exemplar prompting with memory for computer control.InThe Twelfth International Conference on Learning Representations,Cited by:§1.
• W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang (2024)MemoryBank: enhancing large language models with long-term memory.Proceedings of the AAAI Conference on Artificial Intelligence38(17),pp. 19724–19731.External Links:Link,DocumentCited by:§1,§1.
• A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A. Dombrowski, S. Goel, N. Li, M. J. Byun, Z. Wang, A. Mallen, S. Basart, S. Koyejo, D. Song, M. Fredrikson, J. Z. Kolter, and D. Hendrycks (2025)Representation engineering: a top-down approach to ai transparency.External Links:2310.01405,LinkCited by:§C.1,§2.

Appendix AImplementation Details

A.1Dataset Details

Our evaluation utilizes four simulated environments that test diverse procedural skills and long-horizon planning capabilities.

ALFWorldShridharet al.(2021)is a text-based embodied environment where agents navigate simulated rooms and interact with household objects to complete daily tasks. WebShopYaoet al.(2022)provides a simulated e-commerce website where agents must execute search queries, navigate product pages, and select specific item attributes to purchase products matching detailed user instructions. ScienceWorldWanget al.(2022)simulates an elementary school science laboratory, requiring agents to follow strict procedural logic to conduct experiments. BabyAIChevalier-Boisvertet al.(2019)is a partially observable gridworld environment where agents execute sequential movement and manipulation commands to complete specified spatial tasks.

A.2Data Construction and Model Configuration

We construct the contrastive dataset by sampling tasks directly from the training splits of ALFWorld, WebShop, ScienceWorld, and BabyAI. For each training task, we generate multiple trajectories using the target model configured with specific sampling parameters.

To isolate degenerate behaviors for intra-trajectory contrasts, we apply deterministic rules based on action redundancy and invalidity. Redundancy is identified through pattern matching that detects repetitive loops or consecutive identical action sequences. Categorizing segments where the agent enters a state unable to progress normally as degenerate steps provides a reasonable heuristic for capturing procedural failures. Invalidity is determined by matching environment-specific error messages and format violations, which filters irrational actions from the effective trajectory pool.

All experiments employsentence-transformers/all-mpnet-base-v2for dense retrieval to retrieve a fixed number of the most relevant historical tasks. The agent then performs greedy decoding to ensure the reproducibility of the evaluation results. The detailed sampling parameters and retrieval configurations for all evaluated environments are summarized in Table2.

Table 2:Data construction parameters. All parameters except the maximum interaction steps are kept consistent across the four evaluated benchmarks.

A.3Dynamic Synthesis and Vector Extraction

To determine the intervention granularitygg, we estimate task complexity using the average execution length of the retrieved successful trajectories, denoted asLqL_{q}. We compare this length against the global average trajectory length of the respective benchmark, denoted asL¯\bar{L}. Tasks with extended horizons typically encounter localized procedural failures such as repetitive loops, making the concentrated error-correction of intra-trajectory steering more appropriate. Conversely, shorter tasks typically fail due to misaligned global planning, which benefits from the broader behavioral alignment provided by inter-trajectory steering. The system selects the appropriate granularity based on a simple thresholding strategy, with the scaling factorγ\gammaset to 1.5 in our experiments:

g={intra-trajectory,if​Lq>γ​L¯inter-trajectory,otherwiseg=\begin{cases}\text{intra-trajectory},&\text{if }L_{q}>\gamma\bar{L}\\ \text{inter-trajectory},&\text{otherwise}\end{cases}(6) Nevertheless, identifying the optimal granularity for every instance remains a non-trivial challenge. Our heuristic selection occasionally scores lower than the oracle single-granularity baseline in specific settings. Future research could explore more effective methods for allocating steering vectors.

After selecting the appropriate contrastive pool, we extract the steering vectors using a standard principal component analysis pipeline. The paired positive and negative activations are first mean-centered to eliminate shared task-specific biases. We then compute the principal components of these normalized representations and extract the dominant component as the final continuous steering vector.

A.4Steering Configuration

We apply activation steering to the middle-to-late layers of the language models. Specifically, we inject the steering vectors into layers 17 to 19 for Qwen3-4B and Qwen3-8B, and layers 46 to 48 for MiniCPM3-4B.

To achieve dynamic steering across these targeted layers, we adapt a simplified version of the calibration mechanism based on Kullback-Leibler divergence proposed byScalenaet al.(2024). This method tracks the probability shift caused by the steering vector and reduces the intervention strength if the modified next-token distributionPα(⋅∣x<t)P_{\alpha}(\cdot\mid x_{<t})deviates from the original unsteered distributionP(⋅∣x<t)P(\cdot\mid x_{<t})by a large margin. We define a discrete set of candidate scales𝒜\mathcal{A}and select the optimal strengthα∗\alpha^{*}by identifying the maximum value that keeps the divergence below a target thresholdϵ\epsilon:

α∗=max{α∈𝒜|\displaystyle\alpha^{*}=\max\Big\{\alpha\in\mathcal{A}\;\Big|DKL(P(⋅∣x<t)\displaystyle D_{\text{KL}}\big(P(\cdot\mid x_{<t})(7)∥Pα(⋅∣x<t))≤ϵ}\displaystyle\parallel P_{\alpha}(\cdot\mid x_{<t})\big)\leq\epsilon\Big\}

Appendix BSteering Vector Interpretability and Feature Analysis Details

B.1Feature Extraction and Semantic Annotation

To investigate the underlying mechanisms of implicit activation steering, we decompose the continuous representation space into interpretable basis directions following the methodology ofVenhoffet al.(2025). We collect step-level hidden states from historical interaction trajectories at layer 18 for the Qwen3-4B model and apply a sparse dictionary learning algorithm with a top-k activation constraint. The dictionary size is constrained to 16 dimensions with a sparsity threshold of 3. This dimensional configuration aligns with prior research on isolating specific high-level procedural behaviors. The empirical observation of inactive dimensions within this learned dictionary indicates that a 16-dimensional space provides adequate capacity for capturing the procedural variations specific to agentic task-solving steps.

We assign interpretable semantics to these learned basis directions by analyzing their activation patterns across the dataset. We quantify behavioral selectivity by calculating the mutual information between the activation state of each basis directionffand the discrete action typeaagenerated by the agent. This statistical measurement is defined as:

MI(F;A)=∑a[\displaystyle MI(F;A)=\sum_{a}\Biggl[P​(f,a)​log⁡P​(f,a)P​(f)​P​(a)\displaystyle P(f,a)\log\frac{P(f,a)}{P(f)P(a)}(8)+P(¬f,a)logP​(¬f,a)P​(¬f)​P​(a)]\displaystyle+P(\neg f,a)\log\frac{P(\neg f,a)}{P(\neg f)P(a)}\Biggr] Building on established automated interpretability techniquesBillset al.(2023); Pauloet al.(2025), we utilize DeepSeek-V3.2 to analyze these statistical distributions alongside top-activating examples. The model processes this information according to the prompt template detailed in Figure8to generate descriptive textual labels and determine the behavioral polarity for each active basis direction. The resulting semantic annotations for the ALFWorld benchmark are summarized in Table3.

Table 3:Automated annotations and activation statistics of the learned basis directions in the ALFWorld environment.Prompt Template for Automated Feature AnnotationWe are analyzing features learned by a sparse dictionary trained on the hidden states of an LLM agent solving interactive tasks.Environment:ALFWorld is a text-based household environment where an agent must complete tasks like finding objects, cleaning them, heating/cooling them, and placing them in target locations. The agent navigates rooms, opens containers, picks up objects, and uses appliances through text commands.Each “feature” is a direction in the model’s representation space that activates on specific behavioral patterns. Your job is to identify what behavioral pattern causes Feature [ID] to activate, and whether it is associated with effective or ineffective agent behavior.## Top Activating Examples (sorted by activation strength)### Example 1 (activation=[val], reward=[val])Observation: [text]Action: [text][… additional examples …]## Non-Activating Examples (Feature [ID] does NOT activate on these)### Example 1 (reward=[val])Observation: [text]Action: [text][… additional examples …]## Outcome Statistics- This feature activates in [X]% of all steps- Activation rate on successful trajectories: [X]%- Activation rate on failed trajectories: [X]%- Mean activation strength on successful trajectories: [val]- Mean activation strength on failed trajectories: [val]## Action Type Distribution (when this feature is active)The following shows ALL action types in this environment, sorted by how much more likely they are to co-occur with this feature compared to the overall action distribution (ratio > 1 means over-represented, < 1 means under-represented):- [action type]: [ratio]x (P(active|action)=[X]%, mean_activation=[val])[… additional action types …]## Instructions Based on ALL the evidence above (examples, outcome statistics, and action distribution), provide a comprehensive interpretation of this feature:1. Identify the common behavioral pattern across the activating examples that distinguishes them from the non-activating examples.2. Provide a short label (3-8 words in English) for this feature. The label should reflect both WHAT the behavior is and WHETHER it contributes to task success or failure.3. Provide a one-paragraph explanation of what this feature detects, integrating evidence from the examples, outcome statistics, and action type distribution.4. Classify the feature’s polarity: “positive” if it is associated with effective/successful behavior, “negative” if associated with ineffective/failing behavior, or “neutral” if no clear association.5. Rate your confidence (high/medium/low) in this interpretation.Respond in JSON format:{“label”: “…”, “explanation”: “…”, “polarity”: “positive | negative | neutral”, “confidence”: “high | medium | low”, “key_evidence”: [“…”, “…”]}Figure 8:Prompt template for automated interpretation of geometric basis directions.

B.2Geometric Projection of Steering Vectors

We analyze the functional composition of the extracted principal component steering vectors by projecting them onto the annotated basis directions. The geometric projection is computed as the dot product between the normalized steering vector𝐯\mathbf{v}and each unit-norm basis vector𝐰i\mathbf{w}_{i}derived from the dictionary learning phase:

pi=𝐰i⊤​𝐯‖𝐯‖p_{i}=\mathbf{w}_{i}^{\top}\frac{\mathbf{v}}{\|\mathbf{v}\|}(9) A positive projection valuepi>0p_{i}>0indicates that the steering intervention amplifies the corresponding behavioral primitive. A negative valuepi<0p_{i}<0signifies suppression of that specific behavior. This mathematical decomposition confirms that the steering vectors function as composite modifiers that simultaneously inhibit pathological patterns and stimulate necessary subsequent steps to correct agent reasoning.

Appendix CQuantitative Validation of Methodology

C.1Linear Separability in High-Dimensional Manifold

We train a linear support vector machine on the hidden states extracted from the target intervention layers of the retrieved trajectories to verify the linear separability of successful and failed procedural logic. This evaluation determines whether effective and degenerate reasoning modes occupy distinct regions within the representation spaceMarks and Tegmark (2024); Zouet al.(2025). We perform 5-fold stratified cross-validation on the Qwen3-4B model across all four evaluated benchmarks. The classification accuracies at both the inter-trajectory and intra-trajectory levels are reported in Table4. The consistently high classification accuracy across different environments confirms that a linear hyperplane effectively divides the two reasoning classes. This mathematical separability provides the theoretical justification for applying principal component analysis to extract a dominant directional vector for behavioral modulation.

Table 4:Linear SVM classification accuracy (%) on the native high-dimensional hidden states of Qwen3-4B across 5-fold stratified cross-validation.

C.2Accuracy of Heuristic Rules for Degenerate Steps

The efficacy of intra-trajectory steering depends on isolating degenerate steps without contaminating the negative representation pool. We evaluate the reliability of our deterministic isolation rules based on consecutive loop detection and invalid format matching by analyzing sampled steps from 1,000 historical trajectories per benchmark. An LLM-based evaluator assesses each step against the criteria of action redundancy and environment constraints to compute the false positive and false negative rates. The evaluation metrics presented in Table5show a low overall false positive rate of 0.61% and a low overall false negative rate of 6.34%, indicating that the heuristics reliably isolate genuine procedural failures and maintain high purity in the negative representations. The minor variance in error rates across benchmarks, such as the slightly higher false negative rate of 9.30% in WebShop, occurs because rigid deterministic rules can occasionally miss subtle, domain-specific logical errors. These undetected failures subsequently mix into the effective step pool. Although this introduces noise into the positive representations, the empirical results confirm that the overall contrastive signal remains robust for behavioral modulation. Future work could address this limitation by replacing heuristic rules with more capable evaluation mechanisms, such as utilizing a language model as a judge to identify nuanced reasoning failures during data construction.

Table 5:Evaluation of heuristic rule accuracy based on step-level annotation across four benchmarks. False positive rate (FPR) and false negative rate (FNR) are evaluated by an LLM-as-a-judge on sampled steps from 1,000 trajectories per environment.

C.3Distribution Analysis of Inter- and Intra-Trajectory Feature Activations

To evaluate how steering vectors modulate behaviors, we analyze their geometric distribution across the basis directions defined in Equation9using normalized entropy and top-three concentration. For any task, the projection proportion along each basis direction is:

p~i=|pi|∑j=1N|pj|\tilde{p}_{i}=\frac{|p_{i}|}{\sum_{j=1}^{N}|p_{j}|}(10)The global uniformity of this distribution is captured by the normalized entropy:

Hnorm=−∑i=1Np~i​log⁡p~ilog⁡NH_{\text{norm}}=-\frac{\sum_{i=1}^{N}\tilde{p}_{i}\log\tilde{p}_{i}}{\log N}(11)whereNNdenotes the total number of active directions. To measure local focus, we also compute the top-three concentration, representing the cumulative proportion of the three largest absolute projection values:

Ctop3=∑i∈Ω|pi|∑j=1N|pj|C_{\text{top3}}=\frac{\sum_{i\in\Omega}|p_{i}|}{\sum_{j=1}^{N}|p_{j}|}(12)withΩ\Omegadenoting the indices of these three dominant directions.

Empirical analysis of these metrics reveals distinct structural differences between the two steering granularities. As shown in Figure10, inter-trajectory steering vectors exhibit a mean normalized entropy of 0.919, whereas intra-trajectory vectors yield a lower mean of 0.885. This difference indicates that macro-level planning steering operates via a more distributed modulation across multiple basis directions. Conversely, Figure10shows that the mean top-three concentration is 38.0% for inter-trajectory steering and 43.6% for intra-trajectory steering. The higher concentration of intra-trajectory vectors indicates a more targeted activation pattern. Instead of adjusting the overall reasoning flow, intra-trajectory intervention selectively modulates specific behavioral dimensions to resolve localized failures. These patterns suggest that global planning guidance tends to rely on distributed representations, whereas local error recovery is associated with targeted corrections.

Figure 9:Normalized entropy for inter-trajectory and intra-trajectory steering vectors across the ALFWorld benchmark. Figure 10:Top-3 concentration for inter-trajectory and intra-trajectory steering vectors across the ALFWorld benchmark.

Appendix DSystem Efficiency and Overhead

D.1Storage and Computation Overhead

The framework introduces minimal storage overhead compared to text-based retrieval. For each historical trajectory, the system extracts three representations at each targeted layer: one last-token state for inter-trajectory contrast and two mean-pooled states representing effective and degenerate steps for intra-trajectory contrast. Given a hidden dimensiondd, floating-point byte sizebb, andLLtarget layers, the total storage requirement per trajectory is3×L×d×b3\times L\times d\times b. For Qwen3-4B withd=2560d=2560in half-precision format (b=2b=2) targeting three layers, the footprint is 45 kilobytes. This fixed constraint ensures that storage scales independently of the original interaction length. Unlike traditional retrieval systems that consume substantial video memory by expanding the key-value cache with extensive textual contexts, activation steering modifies the residual stream directly to bypass this limitation. The offline computation for memory construction is limited to a single forward pass to extract representations. These continuous vectors can be efficiently managed in a standard database without requiring model parameter updates.

D.2Latency Analysis

We evaluate the inference latency of the proposed framework across retrieval, scale selection probing, vector synthesis, prompt prefill, and autoregressive decoding. The timing results in Table6indicate that textual memory incurs a substantial prefill overhead, increasing latency from 63.46 ms to 279.89 ms, as the self-attention mechanism must process extended textual contexts. In contrast, NPM maintains a prefill latency of 71.09 ms, nearly identical to the No Memory baseline. Although NPM introduces additional phases to compute steering vectors and steering strength, this process imposes no extra latency penalty relative to textual memory. During autoregressive decoding, vector injection involves only an element-wise addition within the residual stream. This basic operation adds no measurable delay to token generation. These efficiency characteristics establish implicit activation steering as a practical and scalable paradigm for persistent agentic memory.

Table 6:Inference latency breakdown (ms).

Appendix EExamples and Case Studies

E.1Examples of Textual Procedural Memory

Examples are provided in Figure11,12.

Example of Insights BaselineTask Instruction:“i am looking for wild caught, ready to eat sardines in a tomato sauce, and price lower than 50.00 dollars”Insights:•Ensure the product’s title and description explicitly mention the requested item type, size, color, and price range to avoid misselection.•Always verify that the product’s price is clearly stated and falls within the user’s specified budget range to avoid overpayment.•Prioritize products with clear and detailed descriptions that explicitly confirm the product meets all requested specifications, including intended use, size, and color.•Ensure the product’s size is explicitly mentioned in both the title and description and matches the exact requested dimensions to avoid purchasing an incorrect product.•Always confirm that the product’s title and description explicitly mention the requested use case, color, and price to ensure alignment with the user’s requirements.•Verify that the product’s key features explicitly include the requested functionality (e.g., wireless, battery inclusion) in both the title and description.•Prioritize products where the requested attributes (such as type, size, and price) are explicitly mentioned in both the title and description to minimize ambiguity.•Always check the product’s price in both the title and description before proceeding to purchase to avoid unexpected costs.…(total 20 insights provided in context.)Figure 11:Example of Insights Baseline for webshopExample of Workflows BaselineTask Goal:put a clean soapbar in countertop.Workflows:•find_and_take_object When you need to find and take an object, search in likely locations.–go to {likely_location}–[if container is closed] open {container}–take {object} from {location}•open_and_search_container When you need to open a container and search inside.–go to {container}–open {container}–[check contents]•search_multiple_locations When you need to search multiple locations for an object.–go to {location 1}–go to {location 2}–go to {location 3}–[repeat as needed]•clean_and_place_object When you need to clean an object and place it in a specific location.–go to {cleaning_location}–clean {object} with {cleaning_location}–go to {target_location}–move {object} to {target_receptacle}…Figure 12:Example of Workflows Baseline for AlfWorld

E.2Case Study: Alleviating Text-Action Disconnect in Action Execution

We examine agent behavior in a multi-object retrieval task requiring the placement of two CDs into a safe (Table7). The baseline model receives an explicit textual workflow outlining the necessary sequential operations and repetition conditions. When the environment rejects an attempt to pick up a second object due to inventory constraints, the text-augmented agent fails to update its internal state representation. The model generates subsequent commands based on an incorrect assumption of successful object acquisition and enters an invalid action loop of attempting to place an unpossessed item into the receptacle. This behavior demonstrates the text-action disconnect where semantic comprehension of discrete textual rules does not ensure sustained state maintenance during execution. The agent equipped with neural procedural memory completes the entire fetch-and-place cycle without textual prompting. Intervening directly in the residual stream allows the synthesized steering vectors to suppress neural activations associated with redundant loops while amplifying features related to sequential planning as identified in prior mechanistic analyses. Modulating the continuous activation space directly enforces the procedural logic required for multi-step execution and prevents the trajectory from degrading into the repetitive errors observed when relying solely on external textual memory.

Table 7:Qualitative comparison of procedural memory in the “find two cd and put them in safe“ task. Despite theTextual Workflowproviding explicit instructions, the agent fails to manage the multi-step trip after an initial action failure.NPMsteers the agent implicitly adhering to the sequential requirements.