学会保留什么：基于用户反馈的迁移感知个人技能学习

# Learning What to Keep: Transfer-Aware Personal Skill Learning from User Feedback
Source: [https://arxiv.org/html/2610.19380](https://arxiv.org/html/2610.19380)
Chen ZhangAffiliation:Beihang UniversityYirong YangAffiliation:Beihang UniversityXingyu ChenAffiliation:Chengdu University of Information TechnologyJinkun ZhaoAffiliation:Beihang UniversityTianhao Peng††thanks:Corresponding author: Tianhao Peng, email: pengtianhao@buaa\.edu\.cn, Wenjun Wu, email: wwj09315@buaa\.edu\.cnAffiliation:Nanyang Technological UniversityWenjun Wu11footnotemark:1Affiliation:Beihang UniversityAffiliation:Hangzhou International Innovation InstituteAffiliation:Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing

###### Abstract

User feedback enables LLMs to refine reusable personalized skills, but often mixes persistent preferences with request\-specific corrections\. Indiscriminately retaining both risks negative transfer, motivating selective retention at the level of individual requirements\. We frame this challenge as selective transfer over atomic rules and proposeSkillGrain, a framework for learning reusable personalized skills from user feedback\.SkillGraindecomposes feedback into independently actionable rules, predicts their transferability, and retains only those suitable for future requests\. To learn these decisions, we combine rubric\-guided skill rewards with fine\-grained credit assignment that separately evaluates rule content and retention decisions\. We further jointly optimize skill induction and skill\-conditioned generation through reinforcement learning, ensuring that learned skills support downstream personalization\. We construct a benchmark spanning five personalized writing tasks to evaluate cross\-request generalization\. Experiments across three base models in both in\-domain and out\-of\-domain settings show thatSkillGrainachieves the best overall average performance among competing methods\. Our code is available at[https://anonymous\.4open\.science/r/SkillGrain\-C210](https://anonymous.4open.science/r/SkillGrain-C210)\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2610.19380v1/introduction.png)Figure 1:User feedback contains a mixture of transferable preferences and task\-specific details\. SkillGrain learns a user\-specific skill from feedback to enable consistent personalized responses\.Reusable skills provide Large Language Models \(LLMs\) with a mechanism to accumulate interaction experience and continuously improve future performance\. In personalized scenarios, models can update reusable skills from user feedback to better adapt to individual preferences\. However, user feedback may contain information with different scopes of applicability: as shown in Figure[1](https://arxiv.org/html/2610.19380#S1.F1), some feedback reflects transferable preferences that generalize across future requests, while other feedback only addresses requirements specific to the current interaction\. For instance, feedback on writing style or organization can capture a user’s persistent preference, whereas instructions involving specific entities or facts are often tied to the current request\. Incorporating such heterogeneous information indiscriminately into persistent skills may introduce inappropriate constraints and lead to negative transfer\. Therefore, effective skill learning requires models to identify the transferability of individual feedback components and selectively retain information that can generalize across future requests\.

Prior work has explored experience reuse and personalization from different perspectives\. Skill\-evolution methods distill execution experience into reusable procedures and validate their effectiveness through task performance[Lin et al\. \(2026\)](https://arxiv.org/html/2610.19380#bib.bib5);[Tang et al\. \(2026b\)](https://arxiv.org/html/2610.19380#bib.bib6)\. However, evaluating a skill as a whole does not directly reveal which individual feedback requirements should be retained\. History\-based personalization models user preferences through memories or summaries[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.19380#bib.bib13);[Liu et al\. \(2026\)](https://arxiv.org/html/2610.19380#bib.bib24), but does not explicitly supervise whether each new correction should become a persistent preference\. Scope\-aware methods distinguish stable from situational preferences or restrict adaptation to relevant contexts[Gao et al\. \(2026\)](https://arxiv.org/html/2610.19380#bib.bib15);[Stephan et al\. \(2024\)](https://arxiv.org/html/2610.19380#bib.bib25), focusing on when preferences apply rather than which requirements within mixed feedback should be retained\. More directly, correction\-driven systems filter transient requirements or maintain rules with applicability conditions[Yang et al\. \(2026\)](https://arxiv.org/html/2610.19380#bib.bib26);[Zhou et al\. \(2026\)](https://arxiv.org/html/2610.19380#bib.bib27), yet do not separately optimize the accuracy of rule extraction and retention decisions\. Despite these advances, learning from feedback that mixes transferable preferences with request\-specific corrections remains challenging\. This raises a key question: can models learn to identify and selectively retain transferable preferences from such mixed feedback to better personalize future responses?

In this work, we formulate personalized skill learning as a selective transfer problem and introduceSkillGrain\. Our framework decomposes mixed feedback into atomic rules and predicts their transferability, retaining reusable preferences while excluding request\-specific corrections from persistent skills\. To train these decisions, we combine rubric\-guided skill rewards with fine\-grained credit assignment, providing separate learning signals for rule content and transferability\. We further jointly optimize skill induction and skill\-conditioned generation through reinforcement learning, using distinct rewards for skill updates and downstream responses\. To evaluate our approach, we construct a benchmark spanning five personalized writing tasks\. Experiments across three base models in both in\-domain and out\-of\-domain settings show thatSkillGrainachieves the highest average score on each model\. Our main contributions are as follows:

- •Feedback\-Driven Personalization as Selective Transfer\.We formulate personalized skill learning as selective transfer and construct a five\-task writing benchmark that tests whether feedback\-derived skills preserve reusable preferences without carrying over request\-specific information\.
- •Transfer\-Aware Skill Induction and Credit Assignment\.We introduceSkillGrain, which pairs atomic rules with learned transferability decisions and assigns separate credit to rule content and retention, jointly optimizing skill induction and downstream application\.
- •Generalization Across Models and Domains\.We evaluateSkillGrainacross three base models in both in\-domain and out\-of\-domain settings, achieving the highest overall average score on each model and demonstrating the effectiveness of our approach\.

## 2Related work

### 2\.1Skill Evolution in LLM Agents

Recent work on LLM agents has increasingly explored how interaction experience can be distilled into reusable knowledge\. Early approaches summarize task outcomes as verbal reflections or extract reusable lessons from multiple trajectories[Shinn et al\. \(2023\)](https://arxiv.org/html/2610.19380#bib.bib2);[Zhao et al\. \(2024\)](https://arxiv.org/html/2610.19380#bib.bib3), while subsequent methods induce and retrieve workflows or skills to guide future tasks[Wang et al\. \(2024\)](https://arxiv.org/html/2610.19380#bib.bib4)\. More recent work further studies continual skill evolution, allowing skills to be created, evaluated, revised, and maintained as new experience is collected[Lin et al\. \(2026\)](https://arxiv.org/html/2610.19380#bib.bib5);[Tang et al\. \(2026b\)](https://arxiv.org/html/2610.19380#bib.bib6);[He and Yang \(2026\)](https://arxiv.org/html/2610.19380#bib.bib7);[Tang et al\. \(2026a\)](https://arxiv.org/html/2610.19380#bib.bib8)\. These methods focus on accumulating and refining reusable task knowledge from interaction experience\. Our work extends this direction to personalized skill learning, where explicit user feedback must be selectively incorporated into skills that are reused across subsequent requests\.

### 2\.2Large Language Model Personalization

LLM personalization commonly exploits interaction history through user representations or preference learning\. One line of work aggregates histories or infers compact personas so that generation can be explicitly conditioned on user attributes[Liu et al\. \(2025\)](https://arxiv.org/html/2610.19380#bib.bib9);[Balepur et al\. \(2025\)](https://arxiv.org/html/2610.19380#bib.bib10), whereas another learns shared and user\-specific preference signals across users, for example through collaborative learning or preference calibration, to better adapt to sparse or unseen users[Choi et al\. \(2025\)](https://arxiv.org/html/2610.19380#bib.bib11);[Ma et al\. \(2026\)](https://arxiv.org/html/2610.19380#bib.bib12)\. Research on long\-horizon personalization further organizes user experiences with episodic and semantic memories[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.19380#bib.bib13), combines short\-term deviations with long\-term trends[Sun et al\. \(2026\)](https://arxiv.org/html/2610.19380#bib.bib14), and distinguishes stable preferences from context\-dependent ones[Gao et al\. \(2026\)](https://arxiv.org/html/2610.19380#bib.bib15); recent personalized agents also couple memory updating with action selection[Zhang et al\. \(2026\)](https://arxiv.org/html/2610.19380#bib.bib16)\. Taken together, these approaches primarily model personalization by extracting and maintaining user\-specific information from accumulated interactions\. We complement this line of research by distilling user feedback into reusable personalized skills\.

## 3Task and Benchmark Construction

### 3\.1Task Formulation

We formulate personalized skill learning as a sequential interaction process between a user and an LLM\. For each useruu, letHuH\_\{u\}denote a fixed set of historical interactions andsts\_\{t\}the personalized skill maintained at interaction steptt\. Given a user requestxtx\_\{t\}, the model generates a response

yt∼πθ\(⋅∣Hu,st,xt\)\.y\_\{t\}\\sim\\pi\_\{\\theta\}\\bigl\(\\cdot\\mid H\_\{u\},s\_\{t\},x\_\{t\}\\bigr\)\.\(1\)After observing user feedbackftf\_\{t\}, the same model updates the personalized skill as

st\+1∼πθ\(⋅∣Hu,st,xt,yt,ft\)\.s\_\{t\+1\}\\sim\\pi\_\{\\theta\}\\bigl\(\\cdot\\mid H\_\{u\},s\_\{t\},x\_\{t\},y\_\{t\},f\_\{t\}\\bigr\)\.\(2\)The updated skillst\+1s\_\{t\+1\}is then used to guide subsequent requests from the same user\. Our goal is to learn how interaction feedback can be distilled into reusable personalized skills that better capture user preferences while avoiding the transfer of interaction\-specific information\.

### 3\.2Benchmark Construction

We construct a personalized skill\-learning benchmark spanning five writing tasks: Topic Writing, Email, Yelp, Product Review, and Abstract\. The first three are used for training and in\-domain evaluation, while the remaining two are held out for out\-of\-domain evaluation\. For each user, we select five historical writing examples and synthesize interaction queries conditioned on the user history and task setting\. We use an LLM for query synthesis and apply automatic and LLM\-based quality checks to remove duplicated, inconsistent, or low\-quality instances\. Detailed dataset statistics and construction procedures are provided in Appendix[A\.1](https://arxiv.org/html/2610.19380#A1.SS1)\.

To simulate feedback containing both transferable preferences and request\-specific requirements, we derive structured benchmark\-side annotations

ℬu,t=Annotate⁡\(Hu,xt\),\\mathcal\{B\}\_\{u,t\}=\\operatorname\{Annotate\}\(H\_\{u\},x\_\{t\}\),\(3\)which capture recurring writing preferences inferred from user history and interaction\-specific content\. We use a four\-dimensional rubric to evaluate responses and guide feedback construction\. The details of the response rewardRansR\_\{\\mathrm\{ans\}\}are provided in Appendix[A\.2](https://arxiv.org/html/2610.19380#A1.SS2)\. Given a responseyty\_\{t\}, we derive structured revision targets through rubric\-based diagnostics:

𝒱t=\[\(vm,τm\)\]m=1Mt=Diagnose⁡\(xt,yt,ℬu,t\),\\mathcal\{V\}\_\{t\}=\\bigl\[\(v\_\{m\},\\tau\_\{m\}\)\\bigr\]\_\{m=1\}^\{M\_\{t\}\}=\\operatorname\{Diagnose\}\(x\_\{t\},y\_\{t\};\\mathcal\{B\}\_\{u,t\}\),\(4\)wherevmv\_\{m\}denotes an individual revision target andτm∈\{0,1\}\\tau\_\{m\}\\in\\\{0,1\\\}indicates its transferability:τm=1\\tau\_\{m\}=1for reusable preference revisions andτm=0\\tau\_\{m\}=0for interaction\-specific requirements\. Finally, we verbalize the revision targets into natural\-language feedback using a fixed LLM:

ft=𝒢f​\(𝒱t\)\.f\_\{t\}=\\mathcal\{G\}\_\{f\}\(\\mathcal\{V\}\_\{t\}\)\.\(5\)Further details are provided in Appendix[A\.3](https://arxiv.org/html/2610.19380#A1.SS3)\.

![Refer to caption](https://arxiv.org/html/2610.19380v1/method.png)Figure 2:Overview of SkillGrain\. We construct user\-specific training instances consisting of an adaptation turn and subsequent requests\. The model jointly optimizes skill\-update and response trajectories using GRPO, with separate rubric\-based rewards and independently normalized advantages\. At inference time, the learned user skill is used to generate personalized responses\.

## 4Method

In this section, we present the design and training ofSkillGrain, with an overview illustrated in Figure[2](https://arxiv.org/html/2610.19380#S3.F2)\. We first introduce atomic skill induction in Sec\.[4\.1](https://arxiv.org/html/2610.19380#S4.SS1), followed by rubric\-guided reward design in Sec\.[4\.2](https://arxiv.org/html/2610.19380#S4.SS2)and fine\-grained advantage assignment in Sec\.[4\.3](https://arxiv.org/html/2610.19380#S4.SS3)\. We then present the overall training pipeline and joint optimization objective in Sec\.[4\.4](https://arxiv.org/html/2610.19380#S4.SS4)\. The detailed training procedure is provided in Appendix[C](https://arxiv.org/html/2610.19380#A3)\.

### 4\.1Transfer\-Aware Atomic Skill Induction

User feedback may contain both transferable preferences and request\-specific requirements\. We therefore decompose feedback into atomic rules and predict the transferability of each rule individually\. Following the task formulation in Sec\.[3\.1](https://arxiv.org/html/2610.19380#S3.SS1), we define the skill\-update context as

ct=\(Hu,st,xt,yt,ft\)\.c\_\{t\}=\(H\_\{u\},s\_\{t\},x\_\{t\},y\_\{t\},f\_\{t\}\)\.\(6\)Conditioned onctc\_\{t\}, the policy generates a sequence of atomic rules with their transferability decisions:

𝒜=\[\(ak,zk\)\]k=1K∼πθ\(⋅∣ct\),\\mathcal\{A\}=\\bigl\[\(a\_\{k\},z\_\{k\}\)\\bigr\]\_\{k=1\}^\{K\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid c\_\{t\}\),\(7\)whereaka\_\{k\}represents an individual requirement expressed in the feedback, andzk∈\{0,1\}z\_\{k\}\\in\\\{0,1\\\}indicates whether the rule should be reused for subsequent requests\. The model extracts both transferable and non\-transferable requirements and explicitly determines their applicability\. Only rules predicted to be transferable are appended to the existing personalized skill:

st\+1=Concat\(st,⟨ak∣\(ak,zk\)∈𝒜,zk=1⟩\)\.s\_\{t\+1\}=\\operatorname\{Concat\}\\bigl\(s\_\{t\},\\langle a\_\{k\}\\mid\(a\_\{k\},z\_\{k\}\)\\in\\mathcal\{A\},\\ z\_\{k\}=1\\rangle\\bigr\)\.\(8\)If no rule is predicted to be transferable, the existing skill remains unchanged\.

### 4\.2Rubric\-Guided Rewards

Since correctly retaining transferable preferences and producing high\-quality downstream responses are distinct objectives, we use separate rewards for personalized skill induction and skill\-conditioned generation\. For skill induction, the reward is designed to encourage two behaviors: preserving transferable preference information and preventing non\-transferable information from being incorporated into the reusable skill\. Let𝒟\\mathcal\{D\}denote the set of rubric dimensions that characterize transferable user preferences\. For each dimensiond∈𝒟d\\in\\mathcal\{D\}, we definecd​\(s\)∈\{0,1\}c\_\{d\}\(s\)\\in\\\{0,1\\\}to indicate whether the candidate skillssmakes the correct update for that dimension\. A correct decision may either incorporate a reusable preference revision identified from the current feedback or retain the existing rule when no revision is required\. We separately evaluate whether the candidate skill incorrectly retains information whose applicability is limited to the current interaction\. Letnlocal​\(s\)n\_\{\\mathrm\{local\}\}\(s\)denote the number of distinct non\-transferable elements incorporated intoss, such as request\-specific entities or one\-off requirements\. We define the corresponding reward as

rnontrans​\(s\)=2−nlocal​\(s\)\.r\_\{\\mathrm\{nontrans\}\}\(s\)=2^\{\-n\_\{\\mathrm\{local\}\}\(s\)\}\.\(9\)
This reward reaches its maximum when no interaction\-specific information is retained and decreases as more non\-transferable information is incorporated into the reusable skill\. We further definev⁡\(s\)∈\{0,1\}v\(s\)\\in\\\{0,1\\\}to indicate whether the candidate makes a valid overall update decision\. Specifically, the skill should be updated when the feedback contains reusable preference information and should remain unchanged otherwise\. The overall skill reward is

Rskill​\(s\)=v⁡\(s\)\|𝒟\|\+1​\(∑d∈𝒟cd​\(s\)\+rnontrans​\(s\)\)\.R\_\{\\mathrm\{skill\}\}\(s\)=\\frac\{v\(s\)\}\{\|\\mathcal\{D\}\|\+1\}\\left\(\\sum\_\{d\\in\\mathcal\{D\}\}c\_\{d\}\(s\)\+r\_\{\\mathrm\{nontrans\}\}\(s\)\\right\)\.\(10\)
In our benchmark,𝒟\\mathcal\{D\}corresponds to the verifiable preference dimensions defined by the rubric, while request\-specific content and constraints are treated as non\-transferable\.RskillR\_\{\\mathrm\{skill\}\}directly evaluates the induced skill and optimizes skill\-update trajectories, whereasRansR\_\{\\mathrm\{ans\}\}defined in Appendix[A\.2](https://arxiv.org/html/2610.19380#A1.SS2)supervises skill\-conditioned response generation\.

### 4\.3Fine\-Grained Advantage Assignment

To enable finer\-grained credit assignment, we introduce an advantage adjustment mechanism for rule content and transferability decisions\. For each sampled skill\-update trajectoryjj, we first compute its group\-relative advantage as

Ajskill=Rjskill−μskillσskill\+ϵ,A\_\{j\}^\{\\mathrm\{skill\}\}=\\frac\{R\_\{j\}^\{\\mathrm\{skill\}\}\-\\mu\_\{\\mathrm\{skill\}\}\}\{\\sigma\_\{\\mathrm\{skill\}\}\+\\epsilon\},\(11\)whereμskill\\mu\_\{\\mathrm\{skill\}\}andσskill\\sigma\_\{\\mathrm\{skill\}\}denote the mean and standard deviation of skill rewards within the sampled group\. We then align each generated atom\(aj,k,zj,k\)\(a\_\{j,k\},z\_\{j,k\}\)with a corresponding revision target\(vm,τm\)∈𝒱t\(v\_\{m\},\\tau\_\{m\}\)\\in\\mathcal\{V\}\_\{t\}\. For each reliably matched pair, we separately evaluate the correctness of the rule content and its transferability decision:

cj,ka,cj,kz∈\{−1,0,1\},c\_\{j,k\}^\{a\},\\;c\_\{j,k\}^\{z\}\\in\\\{\-1,0,1\\\},\(12\)wherecj,kac\_\{j,k\}^\{a\}evaluates whetheraj,ka\_\{j,k\}matches the target requirementvmv\_\{m\}, andcj,kzc\_\{j,k\}^\{z\}evaluates whetherzj,k=τmz\_\{j,k\}=\\tau\_\{m\}\. A value of\+1\+1or−1\-1denotes a correct or incorrect decision, respectively, while00denotes abstention when reliable alignment is unavailable\. LetTj,kaT\_\{j,k\}^\{a\}andTj,kzT\_\{j,k\}^\{z\}denote the tokens corresponding to the rule\-content and transferability fields\. For reliably verified fields, we replace the trajectory\-level advantage with a length\-normalized local signal:

A~j,iskill=\{λ​cj,ka\|Tj,ka\|,i∈Tj,ka,cj,ka≠0,λ​cj,kz\|Tj,kz\|,i∈Tj,kz,cj,kz≠0,Ajskill,otherwise\.\\widetilde\{A\}\_\{j,i\}^\{\\mathrm\{skill\}\}=\\begin\{cases\}\\lambda\\dfrac\{c\_\{j,k\}^\{a\}\}\{\|T\_\{j,k\}^\{a\}\|\},&i\\in T\_\{j,k\}^\{a\},\\;c\_\{j,k\}^\{a\}\\neq 0,\\\\\[5\.0pt\] \\lambda\\dfrac\{c\_\{j,k\}^\{z\}\}\{\|T\_\{j,k\}^\{z\}\|\},&i\\in T\_\{j,k\}^\{z\},\\;c\_\{j,k\}^\{z\}\\neq 0,\\\\\[5\.0pt\] A\_\{j\}^\{\\mathrm\{skill\}\},&\\text\{otherwise\}\.\\end\{cases\}\(13\)Thus, rule content and transferability can receive different optimization signals within the same atom, while unverifiable tokens retain the trajectory\-level advantage\. For transferable targets omitted entirely from the generated atoms, we additionally assign an omission penalty to the sequence\-closing decision\. Detailed alignment and omission\-penalty rules are provided in Appendix[B\.1](https://arxiv.org/html/2610.19380#A2.SS1)\.

### 4\.4Joint GRPO Training

We jointly optimize skill updating and skill\-conditioned generation using Group Relative Policy Optimization \(GRPO\)[Shao et al\. \(2024\)](https://arxiv.org/html/2610.19380#bib.bib22)\. Each training instance consists of an adaptation interaction followed by a subsequent request from the same user\. For the skill\-update stage, we sampleGGcandidate skill updates and evaluate them usingRskillR\_\{\\mathrm\{skill\}\}\. Their group\-relative advantages are computed using Eq\.[11](https://arxiv.org/html/2610.19380#S4.E11)and refined by the fine\-grained signals in Eq\.[13](https://arxiv.org/html/2610.19380#S4.E13)\. We then select the highest\-reward skill to generateGGresponses to the subsequent training query\. These responses are evaluated usingRansR\_\{\\mathrm\{ans\}\}, with their group\-relative advantagesAjansA\_\{j\}^\{\\mathrm\{ans\}\}normalized independently from the skill\-update group\. Both trajectory types are used to update the same policy\. For trajectory typeb∈\{skill,ans\}b\\in\\\{\\mathrm\{skill\},\\mathrm\{ans\}\\\}, the token\-level advantage is

Aj,ib=\{A~j,iskill,b=skill,Ajans,b=ans,A\_\{j,i\}^\{b\}=\\begin\{cases\}\\widetilde\{A\}\_\{j,i\}^\{\\mathrm\{skill\}\},&b=\\mathrm\{skill\},\\\\ A\_\{j\}^\{\\mathrm\{ans\}\},&b=\\mathrm\{ans\},\\end\{cases\}\(14\)where the response advantageAjansA\_\{j\}^\{\\mathrm\{ans\}\}is computed by independently normalizingRansR\_\{\\mathrm\{ans\}\}in the same manner as Eq\.[11](https://arxiv.org/html/2610.19380#S4.E11)\. For tokeniiin trajectoryjj, the policy ratio is defined as

rj,ib​\(θ\)=πθ​\(oj,ib∣oj,<ib\)πθold​\(oj,ib∣oj,<ib\)\.r\_\{j,i\}^\{b\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(o\_\{j,i\}^\{b\}\\mid o\_\{j,<i\}^\{b\}\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(o\_\{j,i\}^\{b\}\\mid o\_\{j,<i\}^\{b\}\)\}\.\(15\)The conditioning context is omitted for clarity\. The GRPO loss for each trajectory type is

ℒb=−𝔼j,i\[min\(\\displaystyle\\mathcal\{L\}\_\{b\}=\-\\mathbb\{E\}\_\{j,i\}\\Big\[\\min\\Big\(rj,ibAj,ib,clip\(rj,ib,1−ϵc,\\displaystyle r\_\{j,i\}^\{b\}A\_\{j,i\}^\{b\},\\operatorname\{clip\}\\bigl\(r\_\{j,i\}^\{b\},1\-\\epsilon\_\{c\},\(16\)1\+ϵc\)Aj,ib\)\]\+βℒKLb\.\\displaystyle 1\+\\epsilon\_\{c\}\\bigr\)A\_\{j,i\}^\{b\}\\Big\)\\Big\]\+\\beta\\mathcal\{L\}\_\{\\mathrm\{KL\}\}^\{b\}\.The final objective combines the two losses:

ℒ=ℒskill\+ℒans\.\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{skill\}\}\+\\mathcal\{L\}\_\{\\mathrm\{ans\}\}\.\(17\)Thus, skill\-update and response trajectories use separate rewards and independently normalized advantages while jointly updating the same policy\. At inference time, the trained policy updates personalized skills from user feedback and applies the retained rules to subsequent requests without reward computation or group sampling\.

## 5Experiments

### 5\.1Experiment Setup

#### Tasks and Datasets

We evaluate our method across five writing tasks\. Topic Writing, Yelp, and Email are used for training\. For evaluation, we use disjoint held\-out examples from these three tasks as the in\-domain test set, and use Product Review and Abstract, which are not seen during training, as out\-of\-domain test tasks\. Dataset statistics and train/test splits are summarized in Table[4](https://arxiv.org/html/2610.19380#A1.T4)\.

#### Baselines

We compareSkillGrainwith both inference\-only and training\-based baselines\. Among inference\-only methods,No Skillanswers queries without any explicit skill,General Skilluses a fixed general\-purpose skill, andAdaptive Skillupdates the skill from user feedback at inference time\.No Historyuses the fixed general\-purpose skill without access to the user’s interaction history\.User Profilesummarizes the user’s interaction history into a profile for generation,RAG\-Historyretrieves the most relevant historical interaction as additional context, andFeedbackdirectly uses the observed user feedback to guide the subsequent response\.Atom\-Promptapplies the same atomic decomposition and transferability protocol asSkillGrainat inference time without parameter optimization\. For training\-based methods,SFTlearns from teacher\-generated skill\-update and response trajectories, whileSTaRperforms self\-training on high\-reward trajectories generated by the model itself\.Answer\-Only RLapplies GRPO only to skill\-conditioned response generation;Outcome\-RLjointly optimizes skill updating and response generation, using downstream response quality to evaluate skill updates; andRule\-RLdirectly scores updated skills with the same rubric\-based reward used bySkillGrain\. More details are provided in Appendix[B\.2](https://arxiv.org/html/2610.19380#A2.SS2)\.

#### Implementation

We use models from the Qwen2\.5[Yang et al\. \(2025b\)](https://arxiv.org/html/2610.19380#bib.bib23)and Qwen3[Yang et al\. \(2025a\)](https://arxiv.org/html/2610.19380#bib.bib1)families as our base models, including Qwen2\.5\-3B\-Instruct, Qwen3\-1\.7B, and Qwen3\-4B\. For all Qwen3 models, we disable the thinking mode throughout training and evaluation\. For RL\-based methods, we use a learning rate of1×10−61\\times 10^\{\-6\}, a batch size of 14, and a KL coefficient of0\.0010\.001\. During training, the rollout temperature is set to1\.31\.3, with maximum generation lengths of 512 tokens for skill updating and 1,024 tokens for response generation\. Further implementation details are provided in Appendix[B\.2](https://arxiv.org/html/2610.19380#A2.SS2)\.

#### Evaluation

For evaluation, we use a sampling temperature of0\.70\.7, top\-ppof0\.80\.8, and a maximum response length of 1,024 tokens\. We repeat each evaluation three times with different random seeds and report the average performance across runs\. Dimension\-wise and atomic transferability results are provided in Appendices[B\.3](https://arxiv.org/html/2610.19380#A2.SS3)and[B\.4](https://arxiv.org/html/2610.19380#A2.SS4), respectively\.

Table 1:Performance across five personalization tasks using three backbone models\. The best and second\-best results within each model block are shown in bold and underlined, respectively\.

### 5\.2Main Results

We evaluate all methods across five personalization tasks, with results reported in Table[1](https://arxiv.org/html/2610.19380#S5.T1)\. Among inference\-only methods,No History,General Skill, andAtom\-Promptachieve the highest average scores on Qwen2\.5\-3B\-Instruct, Qwen3\-1\.7B, and Qwen3\-4B, respectively, suggesting that inference\-time personalization depends on the backbone\. Notably,Atom\-Prompt, which applies the same atomic decomposition and transferability protocol asSkillGrainwithout training, does not consistently outperformNo Skill, indicating that structured prompting alone cannot explain our gains\. Among training\-based methods, SFT consistently outperforms STaR, but both trail the RL\-based approaches\. Rule\-RL further outperforms Outcome\-RL on two of the three backbones, supporting the value of directly evaluating skill quality rather than relying solely on downstream response rewards\. Overall,SkillGrainachieves the highest average performance across all three backbones, surpassing the strongest baseline by 1\.61, 1\.13, and 1\.32 points on Qwen2\.5\-3B\-Instruct, Qwen3\-1\.7B, and Qwen3\-4B, respectively, demonstrating the effectiveness of our approach\.

### 5\.3Ablation Study

To assess the contribution of each component inSkillGrain, we conduct a series of ablation studies\.w/ outcome\-based rewardreplaces the direct rubric\-based skill reward with the reward of the subsequent response generated using the candidate skill;w/o atomizationreplaces atomic skill induction with holistic KEEP/UPDATE updates while retaining matched\-span local credit;w/o advantage allocationremoves the rubric\-guided local advantage assignment; andw/o skill\-update RLdisables RL optimization of skill\-update trajectories and trains only skill\-conditioned response generation\. As shown in Table[2](https://arxiv.org/html/2610.19380#S5.T2), removing atomization or skill\-update RL causes the largest performance drops, while other ablations also reduce performance, supporting the contribution of each component\.

Table 2:Ablation results of SkillGrain\. Best and second\-best results under each backbone are shown inboldandunderline, respectively\.
### 5\.4Effect of Adaptation Rounds

To examine the effect of repeated user interactions, we compareSkillGrainagainst several competitive baselines after the first and second adaptation rounds\. As shown in Figure[3](https://arxiv.org/html/2610.19380#S5.F3)\(Left three\), most methods further improve as additional feedback becomes available, indicating that repeated interaction provides useful signals for personalized adaptation\. Importantly,SkillGrainconsistently maintains the best performance across all three backbones after the second adaptation round and, in most cases, further enlarges its advantage over competing methods\. These results demonstrate thatSkillGraineffectively refines personalized skills through iterative interactions\.

Figure 3:Further analysis of SkillGrain\.\(Left three\):Performance of trained methods across two rounds of feedback adaptation on Qwen2\.5\-3B\-Instruct, Qwen3\-1\.7B, and Qwen3\-4B, respectively\.\(Right\):Performance of SkillGrain with different proportions of training data\. Scores are Mean4 \(%\) averaged over three evaluation seeds\.
### 5\.5Effect of Training Data Scale

We trainSkillGrainwith different proportions of the available training data\. As shown in Figure[3](https://arxiv.org/html/2610.19380#S5.F3)\(Right\), performance generally improves with increasing data scale, while the gains gradually saturate as the full training set is approached\. This demonstrates thatSkillGraineffectively benefits from additional training data\.

Figure 4:Sensitivity analysis ofλ\\lambda\.
### 5\.6Transfer to Alternative Rubrics

To assess generalization beyond the training rubrics, we evaluate four unseen writing preferences: concise opening sentences, explicit causal connectives, continuous prose without headings or lists, and avoidance of hedging expressions\. We construct 40 held\-out request pairs and evaluate rule adherence using deterministic checks without further training\. As shown in Table[3](https://arxiv.org/html/2610.19380#S5.T3),SkillGrainachieves the best or tied\-best average performance across all three backbones, providing preliminary evidence of its ability to learn and reuse transferable rules beyond the training rubrics\.

Table 3:Average rule adherence \(%\) across four alternative writing preferences\.
### 5\.7Sensitivity to Local\-Credit Weight

We examine SkillGrain’s sensitivity to the local\-credit weightλ\\lambdain Eq\.[13](https://arxiv.org/html/2610.19380#S4.E13), which scales credit for locally verified rule content and transferability decisions\. We testλ∈\{0\.5,1\.0,1\.5\}\\lambda\\in\\\{0\.5,1\.0,1\.5\\\}under the same protocol\. Figure[4](https://arxiv.org/html/2610.19380#S5.F4)shows modest performance changes across this range, with no setting consistently best across backbones\. The defaultλ=1\.0\\lambda=1\.0remains competitive, indicating robustness to moderate variations in this weight\.

## 6Conclusion

We study feedback\-driven personalized skill learning for writing tasks and proposeSkillGrainto handle feedback containing both transferable and task\-specific information\.SkillGraindecomposes feedback into atomic rules, learns their transferability, and uses rubric\-based rewards with fine\-grained advantage assignment to jointly optimize skill updating and utilization\. We further construct a benchmark spanning five writing tasks, and experiments across multiple backbones demonstrate the effectiveness of our approach\.

## Limitations

Our study has several limitations\. First, some user profiles in our benchmark are synthesized by LLMs, which may not fully capture the diversity and complexity of real user preferences\. Besides, due to computational constraints, our experiments mainly focus on models in the 1\.7B–4B range and writing\-oriented tasks\. EvaluatingSkillGrainon larger models, real\-world users, and more complex task domains remains an important direction for future work\.

## Ethical Considerations

Personalized skills may retain or reproduce user\-specific details from interaction histories, and source corpora may contain identifiable or objectionable text\. Incorrectly transferring a preference or a factual detail to a new request may also produce misleading personalized responses\. Our non\-leakage rubric measures selected forms of factual carryover; it does not establish privacy protection or content safety\. These risks require separate review before releasing derived data or deploying the method with real users\.

## Acknowledgments

This work was supported by the National Key Research and Development Program of China \(Grant No\. 2025YFF1505704\), the National Science and Technology Major Project \(Grant No\. 2022ZD0117402\), the National Natural Science Foundation of China \(Grant No\. 62441617\), and the Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing\.

## References

- Balepuret al\.\(2025\)N\. Balepur, V\. Padmakumar, F\. Yang, S\. Feng, R\. Rudinger, and J\. L\. Boyd\-GraberWhose boat does it float? improving personalization in preference tuning via inferred user personas\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 3371–3393\.Cited by:[§2\.2](https://arxiv.org/html/2610.19380#S2.SS2.p1.1)\.
- Choiet al\.\(2025\)Y\. Choi, S\. Cho, M\. Lee, M\. Park, Y\. Ko, J\. Ok, and D\. KimCoPL: collaborative preference learning for personalizing llms\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 12886–12904\.Cited by:[§2\.2](https://arxiv.org/html/2610.19380#S2.SS2.p1.1)\.
- Fowleret al\.\(2015\)A\. Fowler, K\. Partridge, C\. Chelba, X\. Bi, T\. Ouyang, and S\. ZhaiEffects of language modeling and its personalization on touchscreen typing performance\.InProceedings of the 33rd annual ACM conference on human factors in computing systems,pp\. 649–658\.Cited by:[§A\.1](https://arxiv.org/html/2610.19380#A1.SS1.p1.1)\.
- Gaoet al\.\(2026\)C\. Gao, Y\. Huang, J\. Yao, X\. Wu, and J\. FengBeyond static profiles: capturing the fluidity of user preferences in diverse scenarios\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 20618–20634\.Cited by:[§1](https://arxiv.org/html/2610.19380#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.19380#S2.SS2.p1.1)\.
- He and Yang \(2026\)Y\. He and W\. YangSkillCommit: evolving agent skills through behaviorally validated scope expansion\.arXiv preprint arXiv:2608\.15165\.Cited by:[§2\.1](https://arxiv.org/html/2610.19380#S2.SS1.p1.1)\.
- Kumaret al\.\(2024\)I\. Kumar, S\. Viswanathan, S\. Yerra, A\. Salemi, R\. A\. Rossi, F\. Dernoncourt, H\. Deilamsalehy, X\. Chen, R\. Zhang, S\. Agarwal,et al\.Longlamp: a benchmark for personalized long\-form text generation\.arXiv preprint arXiv:2407\.11016\.Cited by:[§A\.1](https://arxiv.org/html/2610.19380#A1.SS1.p1.1)\.
- Kwonet al\.\(2023\)W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. Gonzalez, H\. Zhang, and I\. StoicaEfficient memory management for large language model serving with pagedattention\.InProceedings of the 29th symposium on operating systems principles,pp\. 611–626\.Cited by:[§B\.2](https://arxiv.org/html/2610.19380#A2.SS2.p1.1)\.
- Linet al\.\(2026\)H\. Lin, P\. Li, J\. Song, F\. Jiang, and T\. ZhangMuse\-autoskill: self\-evolving agents via skill creation, memory, management, and evaluation\.arXiv preprint arXiv:2605\.27366\.Cited by:[§1](https://arxiv.org/html/2610.19380#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.19380#S2.SS1.p1.1)\.
- Liuet al\.\(2025\)J\. Liu, Y\. Zhu, S\. Wang, X\. Wei, E\. Min, Y\. Lu, S\. Wang, D\. Yin, and Z\. DouLlms\+ persona\-plug= personalized llms\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 9373–9385\.Cited by:[§2\.2](https://arxiv.org/html/2610.19380#S2.SS2.p1.1)\.
- Liuet al\.\(2026\)Y\. Liu, J\. Guan, J\. Li, W\. Wu, J\. Yang, J\. Zhao, and G\. GuoText as a universal interface for transferable personalization\.arXiv preprint arXiv:2601\.04963\.Cited by:[§1](https://arxiv.org/html/2610.19380#S1.p2.1)\.
- Maet al\.\(2026\)X\. Ma, L\. Zhao, W\. Yao, H\. Di, W\. Wang, and J\. LiPersonalizing llms with binary feedback: a preference\-calibrated optimization framework\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 26539–26555\.Cited by:[§2\.2](https://arxiv.org/html/2610.19380#S2.SS2.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§4\.4](https://arxiv.org/html/2610.19380#S4.SS4.p1.2)\.
- Shenget al\.\(2024\)G\. Sheng, C\. Zhang, Z\. Ye, X\. Wu, W\. Zhang, R\. Zhang, Y\. Peng, H\. Lin, and C\. WuHybridFlow: a flexible and efficient rlhf framework\.arXiv preprint arXiv: 2409\.19256\.Cited by:[§B\.2](https://arxiv.org/html/2610.19380#A2.SS2.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.Advances in neural information processing systems36,pp\. 8634–8652\.Cited by:[§2\.1](https://arxiv.org/html/2610.19380#S2.SS1.p1.1)\.
- Stephanet al\.\(2024\)M\. Stephan, A\. Khazatsky, E\. Mitchell, A\. S\. Chen, S\. Hsu, A\. Sharma, and C\. FinnRlvf: learning from verbal feedback without overgeneralization\.arXiv preprint arXiv:2402\.10893\.Cited by:[§1](https://arxiv.org/html/2610.19380#S1.p2.1)\.
- Sunet al\.\(2026\)H\. Sun, Z\. Zhang, and S\. ZengPreference\-aware memory update for long\-term llm agents\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 783–793\.Cited by:[§2\.2](https://arxiv.org/html/2610.19380#S2.SS2.p1.1)\.
- Tanget al\.\(2026a\)B\. Tang, Y\. Zhang, G\. Zhuang, W\. Wei, G\. Zheng, L\. Xie, Y\. Tan, F\. Xiong, Q\. Yang, E\. Chung,et al\.From memory to skills: evidence\-grounded co\-evolution governance for long\-horizon llm agents\.arXiv preprint arXiv:2607\.16621\.Cited by:[§2\.1](https://arxiv.org/html/2610.19380#S2.SS1.p1.1)\.
- Tanget al\.\(2026b\)L\. Tang, C\. Rashtchian, C\. Ferng, A\. Tomkins, D\. Juan, and T\. VuWikiSkill: compiling agent experience into persistent knowledge for skill evolution\.arXiv preprint arXiv:2608\.27454\.Cited by:[§1](https://arxiv.org/html/2610.19380#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.19380#S2.SS1.p1.1)\.
- Wanget al\.\(2024\)Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. NeubigAgent workflow memory\.arXiv preprint arXiv:2409\.07429\.Cited by:[§2\.1](https://arxiv.org/html/2610.19380#S2.SS1.p1.1)\.
- Yanget al\.\(2025a\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§5\.1](https://arxiv.org/html/2610.19380#S5.SS1.SSS0.Px3.p1.1)\.
- Yanget al\.\(2025b\)A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. QiuQwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§5\.1](https://arxiv.org/html/2610.19380#S5.SS1.SSS0.Px3.p1.1)\.
- Yanget al\.\(2026\)Y\. Yang, J\. Li, Q\. Pan, B\. Zhan, Y\. Cai, L\. Du, J\. Zhou, K\. Chen, Q\. Chen, X\. Li,et al\.Autoskill: experience\-driven lifelong learning via skill self\-evolution\.arXiv preprint arXiv:2603\.01145\.Cited by:[§1](https://arxiv.org/html/2610.19380#S1.p2.1)\.
- \[23\]Yelp Inc\.Yelp open dataset\.Note:[https://business\.yelp\.com/data/resources/open\-dataset/](https://business.yelp.com/data/resources/open-dataset/)Accessed: 2026\-09\-19Cited by:[§A\.1](https://arxiv.org/html/2610.19380#A1.SS1.p1.1)\.
- Zhanget al\.\(2026\)W\. Zhang, X\. Zhang, C\. Zhang, L\. Yang, J\. Shang, Z\. Wei, H\. P\. Zou, Z\. Huang, Z\. Wang, Y\. Gao,et al\.PersonaAgent: bridging memory and action for personalized llm agents\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 26421–26439\.Cited by:[§2\.2](https://arxiv.org/html/2610.19380#S2.SS2.p1.1)\.
- Zhanget al\.\(2025\)X\. F\. Zhang, N\. Beauchamp, and L\. WangPrime: large language model personalization with cognitive dual\-memory and personalized thought process\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 33695–33724\.Cited by:[§1](https://arxiv.org/html/2610.19380#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.19380#S2.SS2.p1.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpel: llm agents are experiential learners\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19632–19642\.Cited by:[§2\.1](https://arxiv.org/html/2610.19380#S2.SS1.p1.1)\.
- Zhouet al\.\(2026\)Y\. Zhou, K\. Guo, H\. Zhuang, X\. Wang, Y\. Huang, Z\. Liang, P\. Chen, T\. Gao, N\. Moniz, N\. V\. Chawla,et al\.Getting better at working with you: compiling user corrections into runtime enforcement for coding agents\.arXiv preprint arXiv:2606\.13174\.Cited by:[§1](https://arxiv.org/html/2610.19380#S1.p2.1)\.

## Appendix ABenchmark Construction and Evaluation Details

### A\.1Benchmark Construction

Table 4:Personalized writing tasks and curated split sizes\.We construct five personalized writing tasks from publicly available datasets\.Topic Writing,Product Review, andAbstractare adapted from LongLaMP[Kumar et al\. \(2024\)](https://arxiv.org/html/2610.19380#bib.bib19);Emailis constructed from the Enron Personalization Validation Set[Fowler et al\. \(2015\)](https://arxiv.org/html/2610.19380#bib.bib20); andYelpis derived from the Yelp Open Dataset[Yelp Inc\. \(\)](https://arxiv.org/html/2610.19380#bib.bib21)\. Table[4](https://arxiv.org/html/2610.19380#A1.T4)summarizes the writing scenarios and curated train and test split sizes\.

Examples are grouped by user or author identity\. For each benchmark instance, we select five historical writings from the same user and construct additional interaction queries with distinct task content\. GPT\-5\.6 Terra is used to synthesize interaction queries based on the user history and task specification\. We remove duplicate and near\-duplicate examples and further apply automatic validation and LLM\-based consistency checks to filter malformed or inconsistent instances\.

The joint training pool contains 1,498 instances fromEmail,Topic Writing, andYelp\. The test split contains 998 instances across all five tasks\. Consequently,AbstractandProduct Reviewprovide evaluation on task types not observed in the joint training pool\. Two topic\-writing training records and two product\-review test records are excluded because fewer than five distinct historical writings remain after deduplication\.

To support rubric\-based evaluation and feedback construction, we derive user\-level writing statistics and grounded task\-relevant entities or keywords from the source examples\. These annotations are used only for benchmark construction, reward computation, and feedback generation, and are never directly exposed to the evaluated policy\.

#### Licenses and artifact release\.

The LongLaMP dataset card does not specify a license for redistributing the dataset\. The Enron Personalization Validation Set is released for benchmarking purposes and does not state an explicit permission to redistribute its email text\. The Yelp Open Dataset is governed by the Yelp Dataset Terms of Use, which permit academic use but restrict sharing the data with third parties\. Our public artifact contains implementation code and configurations\. It does not contain source or derived benchmark texts, user\-level records, entity annotations, generated outputs, or model checkpoints\.

### A\.2Rubric Scoring

We evaluate generated responses along four dimensions: length preference \(Len\), first\-person usage \(FP\), expression preference \(Expr\), and non\-leakage \(NL\)\. All dimensions are normalized to\[0,1\]\[0,1\]\.

#### Interval\-Based Scoring\.

BothLenandFPuse a piecewise\-linear interval function\. Given a preferred interval\[l,h\]\[l,h\]and outer bounds\[ol,oh\]\[o\_\{l\},o\_\{h\}\], we define

ϕ⁡\(v\)=\{1,l≤v≤h,0,v≤ol​or​v≥oh,v−oll−ol,ol<v<l,oh−voh−h,h<v<oh\.\\phi\(v\)=\\begin\{cases\}1,&l\\leq v\\leq h,\\\\ 0,&v\\leq o\_\{l\}\\text\{ or \}v\\geq o\_\{h\},\\\\ \\dfrac\{v\-o\_\{l\}\}\{l\-o\_\{l\}\},&o\_\{l\}<v<l,\\\\ \\dfrac\{o\_\{h\}\-v\}\{o\_\{h\}\-h\},&h<v<o\_\{h\}\.\\end\{cases\}\(18\)Values within the preferred interval receive full credit, while deviations are penalized linearly\.

#### Length Preference \(Len\)\.

Letmum\_\{u\}denote the median word count of the five historical writings\. We define the preferred lower bound as

llen=0\.7​mu,l\_\{\\mathrm\{len\}\}=0\.7m\_\{u\},\(19\)and the preferred upper bound as

hlen=1\.3​mu\.h\_\{\\mathrm\{len\}\}=1\.3m\_\{u\}\.\(20\)The corresponding outer bounds are

olenl=0\.5​llen,o\_\{\\mathrm\{len\}\}^\{l\}=0\.5l\_\{\\mathrm\{len\}\},\(21\)and

olenh=1\.5​hlen\.o\_\{\\mathrm\{len\}\}^\{h\}=1\.5h\_\{\\mathrm\{len\}\}\.\(22\)For a generated responseyy, the length score is

rlen​\(y\)=ϕ⁡\(w⁡\(y\),llen,hlen,olenl,olenh\),r\_\{\\mathrm\{len\}\}\(y\)=\\phi\\left\(w\(y\);l\_\{\\mathrm\{len\}\},h\_\{\\mathrm\{len\}\},o\_\{\\mathrm\{len\}\}^\{l\},o\_\{\\mathrm\{len\}\}^\{h\}\\right\),\(23\)wherew⁡\(y\)w\(y\)denotes the response word count\.

#### First\-Person Usage \(FP\)\.

We compute the number of first\-person tokens per 100 words over the concatenated historical writings\. The first\-person lexicon includesI, me, my, mine, we, us, our, oursand their common contractions\. Letquq\_\{u\}denote the historical first\-person rate\. We first define

τu=max⁡\(2\.0,0\.6​qu\)\.\\tau\_\{u\}=\\max\(2\.0,0\.6q\_\{u\}\)\.\(24\)The preferred lower bound is

lfp=max⁡\(0,qu−τu\),l\_\{\\mathrm\{fp\}\}=\\max\(0,q\_\{u\}\-\\tau\_\{u\}\),\(25\)and the preferred upper bound is

hfp=qu\+τu\.h\_\{\\mathrm\{fp\}\}=q\_\{u\}\+\\tau\_\{u\}\.\(26\)The outer bounds are

ofpl=max⁡\(0,lfp−τu\),o\_\{\\mathrm\{fp\}\}^\{l\}=\\max\(0,l\_\{\\mathrm\{fp\}\}\-\\tau\_\{u\}\),\(27\)and

ofph=hfp\+τu\.o\_\{\\mathrm\{fp\}\}^\{h\}=h\_\{\\mathrm\{fp\}\}\+\\tau\_\{u\}\.\(28\)The first\-person score is then

rfp​\(y\)=ϕ⁡\(q⁡\(y\),lfp,hfp,ofpl,ofph\),r\_\{\\mathrm\{fp\}\}\(y\)=\\phi\\left\(q\(y\);l\_\{\\mathrm\{fp\}\},h\_\{\\mathrm\{fp\}\},o\_\{\\mathrm\{fp\}\}^\{l\},o\_\{\\mathrm\{fp\}\}^\{h\}\\right\),\(29\)whereq⁡\(y\)q\(y\)denotes the first\-person rate of responseyy\.

#### Expression Preference \(Expr\)\.

We consider six automatically measurable expression features: contractions, exclamation marks, question marks, parenthetical expressions, all\-caps emphasis, and average sentence length\. For each user, we select one feature that is stable across the historical writings and sufficiently distinctive from the population\-level tendency\.

Letμu\\mu\_\{u\}denote the user\-level median of the selected feature andμp\\mu\_\{p\}its population median\. We define the feature scale as

su=max⁡\{\|μu−μp\|,Rangeu,0\.25\}\.s\_\{u\}=\\max\\left\\\{\|\\mu\_\{u\}\-\\mu\_\{p\}\|,\\operatorname\{Range\}\_\{u\},0\.25\\right\\\}\.\(30\)The no\-penalty tolerance is

δu=max⁡\{MADu,0\.05​su\}\.\\delta\_\{u\}=\\max\\left\\\{\\operatorname\{MAD\}\_\{u\},0\.05s\_\{u\}\\right\\\}\.\(31\)For a responseyy, we first compute its deviation from the user preference:

du​\(y\)=max⁡\{0,\|e⁡\(y\)−μu\|−δu\},d\_\{u\}\(y\)=\\max\\left\\\{0,\|e\(y\)\-\\mu\_\{u\}\|\-\\delta\_\{u\}\\right\\\},\(32\)wheree⁡\(y\)e\(y\)denotes the selected feature value ofyy\. The expression score is then

rexpr​\(y\)=max⁡\{0,1−du​\(y\)su\}\.r\_\{\\mathrm\{expr\}\}\(y\)=\\max\\left\\\{0,1\-\\frac\{d\_\{u\}\(y\)\}\{s\_\{u\}\}\\right\\\}\.\(33\)

#### Non\-Leakage \(NL\)\.

To detect inappropriate information transfer across interactions, we construct a leakage candidate set from task\-relevant entities and keywords appearing in historical or previous interactions\. Terms supported by the current request are excluded from this set\. Letℒu,t​\(y\)\\mathcal\{L\}\_\{u,t\}\(y\)denote the set of distinct leakage candidates detected in responseyy\. We define

rnl​\(y\)=2−\|ℒu,t​\(y\)\|\.r\_\{\\mathrm\{nl\}\}\(y\)=2^\{\-\|\\mathcal\{L\}\_\{u,t\}\(y\)\|\}\.\(34\)Thus, zero, one, two, and three distinct leakage hits yield scores of11,0\.50\.5,0\.250\.25, and0\.1250\.125, respectively\.

#### Overall Score\.

The final response reward is the equally weighted average of the four dimensions:

Rans​\(y\)=14​\(rlen\+rfp\+rexpr\+rnl\)\.R\_\{\\mathrm\{ans\}\}\(y\)=\\frac\{1\}\{4\}\\left\(r\_\{\\mathrm\{len\}\}\+r\_\{\\mathrm\{fp\}\}\+r\_\{\\mathrm\{expr\}\}\+r\_\{\\mathrm\{nl\}\}\\right\)\.\(35\)

### A\.3Rubric\-Grounded Feedback Construction

Directly prompting an LLM to generate user feedback can introduce substantial variability and make the supervision difficult to control\. To obtain reliable yet realistic feedback, we first use deterministic rubric rules to identify revision targets from the model response, including reusable preference corrections such as length or stylistic adjustments, as well as interaction\-specific requirements such as missing current\-request information or irrelevant content carried over from previous interactions\. This design allows the feedback to reflect the mixture of transferable preferences and local requirements that may naturally occur in user interactions\. We then use a fixed Qwen3\-4B model with thinking disabled and deterministic decoding to verbalize these structured revision targets into concise natural\-language feedback\. The verbalizer is instructed to preserve the original revision requirements without introducing additional preferences or facts\.

## Appendix BExperimental Details

### B\.1Fine\-Grained Credit Assignment

For local credit assignment, we deterministically align each generated atomic rule with the structured revision targets retained during feedback construction\. For preference\-related rules, we map each target to a canonical direction \(e\.g\., more/less concise or more/less first\-person\) and match the generated rule using predefined lexical and paraphrase patterns, while also checking for the opposite direction\. For interaction\-specific rules, we first match grounded entities or key phrases and then verify whether the associated directive expresses the expected action, such as including or removing the information\. Ambiguous, conflicting, or unsupported matches are treated as abstentions and retain the original sequence\-level advantage\.

#### Omission penalty\.

Field\-level credit only supervises atoms that are actually generated, so an entirely omitted transferable target would otherwise receive no local negative signal\. Letℳ\\mathcal\{M\}denote the set of confidently missing transferable targets and𝒯\+\\mathcal\{T\}^\{\+\}the set of transferable targets in the feedback\. We apply an omission penalty only when the unmatched transferable targets can be confidently identified:

comit=−\|ℳ\|\|𝒯\+\|\.c\_\{\\mathrm\{omit\}\}=\-\\frac\{\|\\mathcal\{M\}\|\}\{\|\\mathcal\{T\}^\{\+\}\|\}\.The score is assigned to the sequence\-closing token\. When it is unclear whether an unmatched target has already been expressed by a generated atom, no omission penalty is applied\.

### B\.2Training and Comparison Details

Our framework is implemented on top of VeRL[Sheng et al\. \(2024\)](https://arxiv.org/html/2610.19380#bib.bib17)\. All experiments were conducted on 4×\\timesH100 GPUs with 80GB of memory, using Python 3\.10 and PyTorch 2\.6, with vLLM[Kwon et al\. \(2023\)](https://arxiv.org/html/2610.19380#bib.bib18)for rollout acceleration\. Except forNo History, all methods use five historical interactions from the same user as personalization context\.No Historyuses the original base model and the same fixed general\-purpose skill asGeneral Skill, while omitting historical interactions from the response prompt\.Adaptive Skill,Outcome\-RL, andRule\-RLuse the holistic keep\-or\-update skill protocol, whereasSkillGraingenerates atomic rules with transferability predictions\.Atom\-Promptuses the same atomic decomposition and transferability protocol asSkillGrainat inference time without parameter optimization\.Answer\-Only RLkeeps the skill updater frozen and applies GRPO only to skill\-conditioned response generation\.Outcome\-RLevaluates skill updates using downstream response rewards, whileRule\-RLandSkillGrainuse the same direct rubric\-based skill reward\. ForSFT, Qwen3\-8B generates teacher trajectories with deterministic decoding;STaRinstead samples four skill\-update and four response candidates from the target model and retains the highest\-reward trajectories for supervised training\.

All RL\-based methods use GRPO with a global batch size of 14, a PPO mini\-batch size of 14, and a micro\-batch size of 1 per GPU\. We train for one epoch with one PPO epoch per batch, a learning rate of1×10−61\\times 10^\{\-6\}, weight decay of0\.010\.01, and a KL coefficient of0\.0010\.001\. Skill\-update and response rollouts useG=4G=4, temperature1\.31\.3, and top\-p=1\.0p=1\.0\. The maximum prompt length is 16,384 tokens, while skill updates and responses are limited to 512 and 1,024 tokens, respectively\. ForSkillGrain, the local\-credit weight is set toλ=1\.0\\lambda=1\.0\. SFT\-based methods use AdamW with\(β1,β2\)=\(0\.9,0\.95\)\(\\beta\_\{1\},\\beta\_\{2\}\)=\(0\.9,0\.95\), while RL\-based methods follow the default VeRL AdamW setting\(0\.9,0\.999\)\(0\.9,0\.999\)\. ForSFTandSTaR, we use a batch size of 28, a micro\-batch size of 1 per GPU, a maximum sequence length of 17,408 tokens, and the same learning rate of1×10−61\\times 10^\{\-6\}for one epoch\.

### B\.3Dimension\-wise Results

Table[5](https://arxiv.org/html/2610.19380#A2.T5)reports the four rubric scores for the three RL methods\.SkillGrainachieves the highest LEN score across all three backbones and improves NL overOutcome\-RLthroughout, reflecting better length alignment and reduced inappropriate factual carryover\.

Table 5:Dimension\-wise rubric scores of the three RL methods\.
### B\.4Atomic Transferability Analysis

To assess whetherSkillGrainlearns to distinguish transferable skill atoms from interaction\-specific feedback, we examine the precision of its test\-set atomic extraction and transferable\-atom selection\. As shown in Table[6](https://arxiv.org/html/2610.19380#A2.T6), both remain high across backbones and evaluation domains, suggesting that the trained model can identify reusable preferences while limiting inappropriate transfer into persistent skills\.

Table 6:Atomic extraction and transferable\-atom selection precision \(%\)\. ID and OOD denote in\-domain and out\-of\-domain test sets, respectively\.
### B\.5Evaluation with an Independent LLM Judge

To complement the rubric\-based evaluation, we use GPT\-6 Luna as an independent judge to assess the three methods with Qwen2\.5\-3B\-Instruct on 200 randomly sampled test instances \(40 per task\)\. Given user histories, interaction context, and anonymized responses, the judge evaluates length suitability, first\-person usage, expression style, and avoidance of inappropriate factual carryover\. As shown in Table[7](https://arxiv.org/html/2610.19380#A2.T7),SkillGrainachieves the highest mean judge score among the three methods\.

Table 7:Independent GPT\-6 Luna evaluation with Qwen2\.5\-3B\-Instruct on the same 200 test instances\.
### B\.6Comparison of Computational Cost

To compare the training overhead of different methods, we report their training times using Qwen3\-4B as the backbone model on four H100 GPUs, as summarized in Table[8](https://arxiv.org/html/2610.19380#A2.T8)\.

Table 8:Training time \(hours\) of methods requiring optimization with Qwen3\-4B\.

## Appendix CTraining Algorithm

Algorithm[1](https://arxiv.org/html/2610.19380#alg1)summarizes the training procedure ofSkillGrain\.

Algorithm 1Training Procedure of SkillGrain1:Training data

𝒟train\\mathcal\{D\}\_\{\\mathrm\{train\}\}, policy

πθ\\pi\_\{\\theta\}, group size

GG, local\-credit weight

λ\\lambda
2:Trained policy

πθ\\pi\_\{\\theta\}
3:foreach training batch

ℬ\\mathcal\{B\}do

4:

θold←θ\\theta\_\{\\mathrm\{old\}\}\\leftarrow\\theta
5:

ℒℬ←0\\mathcal\{L\}\_\{\\mathcal\{B\}\}\\leftarrow 0
6:foreach

\(ct,𝒱t,xt\+1\)∈ℬ\(c\_\{t\},\\mathcal\{V\}\_\{t\},x\_\{t\+1\}\)\\in\\mathcal\{B\}do

7:for

j=1,…,Gj=1,\\ldots,Gdo

8:Sample

𝒜\(j\)=\[\(aj,k,zj,k\)\]k=1Kj∼πθold\(⋅∣ct\)\\mathcal\{A\}^\{\(j\)\}=\[\(a\_\{j,k\},z\_\{j,k\}\)\]\_\{k=1\}^\{K\_\{j\}\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid c\_\{t\}\)
9:

s\(j\)←Compose⁡\(st,𝒜\(j\)\)s^\{\(j\)\}\\leftarrow\\operatorname\{Compose\}\(s\_\{t\},\\mathcal\{A\}^\{\(j\)\}\)
10:

Rjskill←Rskill​\(s\(j\)\)R\_\{j\}^\{\\mathrm\{skill\}\}\\leftarrow R\_\{\\mathrm\{skill\}\}\(s^\{\(j\)\}\)
11:endfor

12:Compute

μskill\\mu\_\{\\mathrm\{skill\}\}and

σskill\\sigma\_\{\\mathrm\{skill\}\}from

\{Rjskill\}j=1G\\\{R\_\{j\}^\{\\mathrm\{skill\}\}\\\}\_\{j=1\}^\{G\}
13:for

j=1,…,Gj=1,\\ldots,Gdo

14:

Ajskill←Rjskill−μskillσskill\+ϵA\_\{j\}^\{\\mathrm\{skill\}\}\\leftarrow\\dfrac\{R\_\{j\}^\{\\mathrm\{skill\}\}\-\\mu\_\{\\mathrm\{skill\}\}\}\{\\sigma\_\{\\mathrm\{skill\}\}\+\\epsilon\}
15:

A~j,iskill←Ajskill\\widetilde\{A\}\_\{j,i\}^\{\\mathrm\{skill\}\}\\leftarrow A\_\{j\}^\{\\mathrm\{skill\}\}for all generated tokens

ii
16:foreach

\(aj,k,zj,k\)∈𝒜\(j\)\(a\_\{j,k\},z\_\{j,k\}\)\\in\\mathcal\{A\}^\{\(j\)\}do

17:Align with targets in

𝒱t\\mathcal\{V\}\_\{t\}
18:Obtain

cj,ka,cj,kzc\_\{j,k\}^\{a\},c\_\{j,k\}^\{z\}and token spans

Tj,ka,Tj,kzT\_\{j,k\}^\{a\},T\_\{j,k\}^\{z\}
19:if

cj,ka≠0c\_\{j,k\}^\{a\}\\neq 0then

20:

A~j,iskill←λ​cj,ka/\|Tj,ka\|\\widetilde\{A\}\_\{j,i\}^\{\\mathrm\{skill\}\}\\leftarrow\\lambda c\_\{j,k\}^\{a\}/\|T\_\{j,k\}^\{a\}\|,

∀i∈Tj,ka\\forall i\\in T\_\{j,k\}^\{a\}
21:endif

22:if

cj,kz≠0c\_\{j,k\}^\{z\}\\neq 0then

23:

A~j,iskill←λ​cj,kz/\|Tj,kz\|\\widetilde\{A\}\_\{j,i\}^\{\\mathrm\{skill\}\}\\leftarrow\\lambda c\_\{j,k\}^\{z\}/\|T\_\{j,k\}^\{z\}\|,

∀i∈Tj,kz\\forall i\\in T\_\{j,k\}^\{z\}
24:endif

25:endfor

26:Apply omission penalty to the closing token if required

27:endfor

28:

j∗←arg⁡maxj⁡Rjskillj^\{\*\}\\leftarrow\\arg\\max\_\{j\}R\_\{j\}^\{\\mathrm\{skill\}\}
29:

s∗←s\(j∗\)s^\{\*\}\\leftarrow s^\{\(j^\{\*\}\)\}
30:for

j=1,…,Gj=1,\\ldots,Gdo

31:Sample

yj∼πθold\(⋅∣Hu,s∗,xt\+1\)y\_\{j\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid H\_\{u\},s^\{\*\},x\_\{t\+1\}\)
32:

Rjans←Rans​\(yj\)R\_\{j\}^\{\\mathrm\{ans\}\}\\leftarrow R\_\{\\mathrm\{ans\}\}\(y\_\{j\}\)
33:endfor

34:Compute independently normalized response advantages

\{Ajans\}j=1G\\\{A\_\{j\}^\{\\mathrm\{ans\}\}\\\}\_\{j=1\}^\{G\}
35:Compute

ℒskill\\mathcal\{L\}\_\{\\mathrm\{skill\}\}using

A~j,iskill\\widetilde\{A\}\_\{j,i\}^\{\\mathrm\{skill\}\}
36:Compute

ℒans\\mathcal\{L\}\_\{\\mathrm\{ans\}\}using

AjansA\_\{j\}^\{\\mathrm\{ans\}\}
37:

ℒℬ←ℒℬ\+ℒskill\+ℒans\\mathcal\{L\}\_\{\\mathcal\{B\}\}\\leftarrow\\mathcal\{L\}\_\{\\mathcal\{B\}\}\+\\mathcal\{L\}\_\{\\mathrm\{skill\}\}\+\\mathcal\{L\}\_\{\\mathrm\{ans\}\}
38:endfor

39:

θ←θ−η​∇θ\(ℒℬ/\|ℬ\|\)\\theta\\leftarrow\\theta\-\\eta\\nabla\_\{\\theta\}\\left\(\\mathcal\{L\}\_\{\\mathcal\{B\}\}/\|\\mathcal\{B\}\|\\right\)
40:endfor

41:return

πθ\\pi\_\{\\theta\}

## Appendix DUse of Large Language Models

Some parts of the text were refined with the assistance of large language models \(LLMs\)\. All content and responsibility for the work remain with the authors\.

## Appendix EPrompts

This section lists the prompts used by the personalized\-writing pipeline\. Except for the No History baseline, all methods use the same five historical examples, current request, actor model, and answering prompt template\. TheNo Skillcondition leaves the rule block empty; theGeneral Skilland adaptation conditions differ only in which rules are supplied in that block\. The judge is fixed and is used only to rewrite deterministic editing notes into natural\-language feedback; it does not compute rewards\.

Notation\.\{HISTORY\}denotes five examples from the same user, and\{REQUEST\},\{DRAFT\}, and\{FEEDBACK\}denote the current request, draft, and natural\-language feedback, respectively\.

1System:

2/no\_think

3Youareawritingassistant\.Returnonlytherequestedwriting,withnoanalysis,labels,ormarkdown\.

4

5User:

6Belowarefiveexamplesfromthesameuser\.Completethecurrentrequestinthatuser’sstyle\.Useonlyfactssupportedbythecurrentrequest;donotcopynamesordetailsfromunrelatedexamples\.

7

8\{HISTORY1\}

9\.\.\.

10\{HISTORY5\}

11

12Writingskillrules:

13r1:\{RULECONTENT\}

14

15Currentrequest:

16\{REQUEST\}

Figure 5:Common answering prompt shared by all methods\.1Abstract:

2Writeaconcise,self\-containedscientificabstract\.Statetheproblem,theproposedapproach,themainresultorcontribution,andonlyclaimssupportedbythecurrentrequest\.Useformaltechnicalproseanddonotinventexperimentalfindings\.

3

4Emailcompletion:

5Continuenaturallyfromthesuppliedprefix\.Matchtheuser’sgreeting,sentencelength,punctuation,directness,formality,andsign\-offhabits\.Addressonlythecurrentemailanddonotrepeatitsprefix\.

6

7Productreview:

8Keepthereviewanchoredtothecurrentproductandrating\.Explainconcretestrengthsandlimitationssupportedbytheprompt,preservethesuppliedsentiment,andwriteonlythereviewinanaturalconsumervoicewithoutinventedspecificationsorexperiences\.

9

10Topicwriting:

11WriteanaturalReddit\-stylepostthatdirectlyaddressesthecurrenttopic,usesacoherentpersonalvoice,andfollowstheuser’susuallevelofdetailandinformality\.Donotinventunsupportedpersonalhistory\.

12

13Yelp:

14WriteonlytheYelpreview\.Lettheratingguidethestance,discussonlythecurrentbusinessandsuppliedexperience,andmatchtheuser’susualbrevity,punctuation,detaillevel,andtonewithoutinventingvisitsormenuitems\.

Figure 6:Static task\-specific skills used by the general\-skill condition\.1System:

2/no\_think

3Youmaintainaconcisewritingskillforoneuser\.

4Readtheinteractionandtheuser’snaturalfeedback\.Decidewhetherthefeedbacksupportsonedurable,reusablewritingrule\.

5Areusableruledescribeswritingstyleorprocessthatisusefulonadifferenttopic\.Donotstorenames,products,places,dates,topic\-specificfacts,orinstructionsthatonlyrepairthecurrentanswer\.

6Ifthereisnosupportedreusablerule,chooseKEEP\.Otherwiseaddexactlyoneshortrule\.

7ReturnJSONonly,exactlyoneof:

8\{"decision":"KEEP","rule\_updates":\[\],"new\_rules":\[\]\}

9\{"decision":"UPDATE","rule\_updates":\[\],"new\_rules":\[\{"content":"onereusablecontent\-neutralrule"\}\]\}

10

11User:

12Userhistory:

13\{HISTORY\}

14

15Currentrequest:

16\{REQUEST\}

17

18Assistantdraft:

19\{DRAFT\}

20

21Userfeedback:

22\{FEEDBACK\}

23

24ReturntheKEEPorUPDATEJSON\.

Figure 7:Rule\-update prompt used by the outcome\-reward and direct\-rule baselines\.1System:

2/no\_think

3Splittheuser’sfeedbackintoindependentlyactionablerequests\.

4Foreachrequestoutputonlyid\(f1,f2,\.\.\.\),content\(ashortconcreterestatement\),andtransferable\(trueonlywhenitappliestoadifferenttopic\)\.

5Generalwritingpreferencesaretransferable\.Requeststoinsertorremovespecificnames,products,places,facts,numbers,examples,ortechnicaltermsarenottransferable\.Donotturnthemintogeneralinstructionstoincorporateterms,facts,examples,orreferences\.Preserveeachrequest’sdirectionandqualifications\.Includeallfeedbackrequests,includingnon\-transferableones\.

6Theprogramwilljointhecontentofalltransferable=trueatomswith";"toformtheskill\.Donotproduceaseparatesummary,decision,ornew\_rule\.

7Example:Makethisshorter\.IncludeParisandtheLouvrenaturally\.

8\{"atoms":\[\{"id":"f1","content":"Makethewritingshorter\.","transferable":true\},\{"id":"f2","content":"IncludeParisandtheLouvrenaturally\.","transferable":false\}\]\}

9ReturnJSONonlywiththesingletop\-levelkeyatoms\.

10

11User:

12Userhistory:

13\{HISTORY\}

14

15Currentrequest:

16\{REQUEST\}

17

18Assistantdraft:

19\{DRAFT\}

20

21Userfeedback:

22\{FEEDBACK\}

Figure 8:Atom\-decomposition prompt used by our method\. The program concatenates only atoms whosetransferablefield is true\.1System:

2Rewritethesuppliededitingnotesasonenaturalparagraphoffeedbackaddressedtothewriter\.

3Youroutputmustdescribewhatshouldchange;donotperformtherewriteanddonotreproducethedraft\.

4Keepeveryrequestedchangeanditsmeaning,combinerepetitivewording,andusenomorethan80words\.

5Donotaddanypreference,name,fact,example,explanation,orcriticismabsentfromthenotes\.

6Donotmentioncategories,scores,rules,orwhetheranythingappliestofuturewriting\.Donotuseaheadingorbulletlist\.

7Returnonlythefeedbackparagraph\.

8

9User:

10Editingnotes\(theonlysourceoffeedback\):

11\-\{EDITINGNOTE1\}

12\-\{EDITINGNOTE2\}

13\-\.\.\.

Figure 9:Prompt for the fixed judge model\. Its input notes are constructed deterministically from the four reward dimensions and entity diagnostics\.
## Appendix FCase Study

Table 9:Mean four\-dimensional scores of the saved case\-study trajectories\.We qualitatively examine saved two\-round trajectories produced by our atom\-based method\. Table[9](https://arxiv.org/html/2610.19380#A6.T9)reports the four\-dimensional mean score before adaptation, after the first skill update, and on the held\-out test request\. These examples are selected to illustrate the interaction mechanism rather than to provide an aggregate comparison\.

We detail the abstract\-writing trajectory because it exposes both possible skill\-update outcomes: the first round adds a transferable preference, whereas the second round correctly leaves the skill unchanged when the feedback is entirely request\-specific\. The actor model is Qwen3\-4B\. Long history examples, requests, and responses are shortened only for presentation\.

Context and User HistoryThe five abstracts exhibit formal technical prose, relatively long sentences, and detailed descriptions\. Representative passages discuss semantic technologies, human input for data interlinking, gamification in citizen science, human–machine mobility workflows, and Wikidata editor engagement\.

Round 1: Draft and FeedbackThe request asks for an abstract about SeaFish, a game for collaborative image annotation and interlinking\. The generated draft describes human computation, DBpedia and Flickr images, and Linked Data, but is shorter than the user’s historical preference\.Feedback\.“The text is somewhat brief and could benefit from additional useful details\. Incorporate the terms ‘semantic content creation’ and ‘data interlinking’ more naturally into the draft to enhance clarity and depth\.”

Transferability Filtering and Skill UpdateOnly the first atom is retained, producing the reusable skillMake the text longer with additional useful details\.The two named terms describe content required by the current request and are therefore not stored as user\-level writing preferences\.

Round 2: Local Feedback Leads to KEEPThe next request concerns gamification and adaptive incentives for paid microtask crowdsourcing\. Conditioned on the retained skill, the response expands the approach, experiments, and findings, obtaining full credit on all four dimensions\. The remaining feedback asks only for the term “microtasks” to be integrated more naturally\. The method decomposes this feedback into three non\-transferable atoms and appliesKeep; the skill remains unchanged\.

Held\-Out TestThe test request asks for an abstract about the TheyBuyForYou platform and its knowledge graph for public procurement\. The response explains how open linked data from multiple EU sources supports transparency and cross\-border bidding\. It receives1\.000for length, first\-person usage, expression, and non\-leakage\. All three target terms are covered, and no entity from the history or adaptation requests is detected in the response\.

InterpretationThe trajectory illustrates the intended transfer boundary\. A general preference for additional useful detail survives across unrelated abstract topics, while request\-specific terminology is used for local feedback but is not written into persistent memory\. The second\-roundKeepdecision also shows that an interaction need not modify the skill when its feedback contains no transferable preference\.