From Static Personal Values to Contextualized Personalization: Bayesian Personalized Value Alignment for LLMs

# From Static Personal Values to Contextualized Personalization: Bayesian Personalized Value Alignment for LLMs
Source: [https://arxiv.org/html/2609.28942](https://arxiv.org/html/2609.28942)
Hanze Guo††thanks:Equal contribution\.Aixuan Song11footnotemark:1Affiliation:Gaoling School of Artificial Intelligence, Renmin University of ChinaJing Yao11footnotemark:1Affiliation:Microsoft Research AsiaXiangxu ZhangAffiliation:Gaoling School of Artificial Intelligence, Renmin University of ChinaXiaoyuan YiAffiliation:Microsoft Research AsiaXing XieAffiliation:Microsoft Research AsiaXiao Zhou††thanks:Corresponding author\.Affiliation:Gaoling School of Artificial Intelligence, Renmin University of ChinaAffiliation:Beijing Key Laboratory of Research on Large Models and Intelligent GovernanceAffiliation:Engineering Research Center of Next\-Generation IntelligentSearch and Recommendation, MOE\{ghz,songaix,xiaozhou\}@ruc\.edu\.cn, jingyao@microsoft\.com

###### Abstract

Personalized value alignment has become increasingly important as large language models \(LLMs\) are expected to accommodate diverse user preferences\. However, existing methods typically align model outputs with a static value profile across prompts, overlooking that the salience of value dimensions varies substantially across contexts\. Inspired by Lewin’s Field Theory, which views human behavior as jointly shaped by personal dispositions and situational constraints, we model personal values as priors and context\-dependent preferences as posteriors\. We proposeBaCVA, an inference\-timeBayesianContext\-aware personalizedValueAlignment method that approximates posterior personalized preferences by integrating static personal values with scenario\-specific value salience\. BaCVA first estimates contextual value salience from generally normative responses, and then employs a*dual\-view personalization module*to infer posterior preferences from complementary personal\-value and scenario\-driven perspectives\. This Bayesian formulation enables more accurate and adaptive personalized value alignment while improving data efficiency via prior values\. Extensive experiments on benchmarks demonstrate its superiority over strong baselines\.

## 1Introduction

The widespread deployment of Large Language Models \(LLMs\)\([Achiam et al\., 2023](https://arxiv.org/html/2609.28942#bib.bib60);[Touvron et al\., 2023](https://arxiv.org/html/2609.28942#bib.bib59);[Guo et al\., 2025a](https://arxiv.org/html/2609.28942#bib.bib65);[Yang et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib49);[Zhang et al\., 2026a](https://arxiv.org/html/2609.28942#bib.bib7);[Li et al\., 2025b](https://arxiv.org/html/2609.28942#bib.bib8);[Zhu et al\., 2026](https://arxiv.org/html/2609.28942#bib.bib9);[Zhang et al\., 2026b](https://arxiv.org/html/2609.28942#bib.bib10);[Yong et al\., 2025b](https://arxiv.org/html/2609.28942#bib.bib17);[Yong et al\., 2025a](https://arxiv.org/html/2609.28942#bib.bib18);[Yong et al\., 2026](https://arxiv.org/html/2609.28942#bib.bib19)\)has made aligning LLMs with human values a central research challenge\([Wang et al\., 2023](https://arxiv.org/html/2609.28942#bib.bib51);[Shen et al\., 2023](https://arxiv.org/html/2609.28942#bib.bib53);[Yao et al\., 2023b](https://arxiv.org/html/2609.28942#bib.bib52);[Wang et al\., 2024b](https://arxiv.org/html/2609.28942#bib.bib50);[Guo et al\., 2026b](https://arxiv.org/html/2609.28942#bib.bib13);[Zhang et al\., 2026c](https://arxiv.org/html/2609.28942#bib.bib12)\)\. While prominent methods focus on promoting universal but monolithic values, such as*HHH*\(helpful, harmless, honest\)\([Bai et al\., 2022a](https://arxiv.org/html/2609.28942#bib.bib54);[Ouyang et al\., 2022](https://arxiv.org/html/2609.28942#bib.bib58);[Hendrycks et al\., 2020](https://arxiv.org/html/2609.28942#bib.bib56)\), they struggle to accommodate the substantial diversity of individual user values\. Consequently,*personalized value alignment*emerges as a more practical objective\([Zhou et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib14);[Guo et al\., 2025b](https://arxiv.org/html/2609.28942#bib.bib15);[Guo et al\., 2026a](https://arxiv.org/html/2609.28942#bib.bib16);[Zhang et al\., 2026d](https://arxiv.org/html/2609.28942#bib.bib11)\)\.

Existing work on personalized value alignment falls into two categories\.*Training\-time methods*fine\-tune personalized reward models and adapters using user\-specific interaction data\([Poddar et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib26);[Zhao et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib47)\)\. Nevertheless, these methods typically rely on extensive personal value data and might degrade in data\-scarce or cold\-start scenarios\([Pădurean et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib33);[Poddar et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib26)\)\.*Inference\-time methods*offer greater flexibility, which steer model outputs by personal value prompts\([Pădurean et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib33);[Ryan et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib30);[Chen et al\., 2025b](https://arxiv.org/html/2609.28942#bib.bib28)\), activating value\-related model parameters\([Zhu et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib70);[Zhang et al\., 2025b](https://arxiv.org/html/2609.28942#bib.bib31)\)or modifying token distributions during decoding\([Chen et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib25);[Shi et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib23);[Zhang et al\., 2025a](https://arxiv.org/html/2609.28942#bib.bib24)\)\. However,*most inference\-time methods face a common limitation*: they conceptualize personalized value alignment as adapting model outputs to a*static personal value profile*to capture users’ overall value orientations\.*Although these methods condition response generation on the input question, they typically use the same user value profile for every question, without explicitly estimating which value dimensions become more or less important in the current context\.*

![Refer to caption](https://arxiv.org/html/2609.28942v1/intro_example_revised.png)\(a\)
\(b\)
Figure 1:\(a\) An example showing that a user a stable personal value profile may exhibit different value preferences across contexts, which are jointly shaped by personal values and situational constraints\. \(b\) Empirical results show that only personal values or scenario value salience achieve weaker personalization than their combination\.Drawing from well\-established psychological theories\([Mischel, 2013](https://arxiv.org/html/2609.28942#bib.bib74);[Epstein and O’Brien, 1985](https://arxiv.org/html/2609.28942#bib.bib75)\), stable internal dispositions alone are insufficient to explain human behaviors across varying contexts\. Notably, Lewin’s Field Theory\([Heidbreder, 1937](https://arxiv.org/html/2609.28942#bib.bib71)\)posits that observed behavior is jointly shaped by personal values and situational constraints\. As illustrated in Fig\.[1](https://arxiv.org/html/2609.28942#S1.F1)\(a\), an individual who inherently prioritizes‘Self\-Direction’over‘Conformity’may nonetheless suppress self\-direction actions in scenarios governed by other values\. This suggests that each scenario may selectively activate or suppress different value dimensions according to broadly shared social norms\([Cialdini et al\., 1990](https://arxiv.org/html/2609.28942#bib.bib73);[Burnes\*, 2004](https://arxiv.org/html/2609.28942#bib.bib72)\), which can be formalized as a context\-aware*value salience distribution*\. Together, inherent personal values and context\-aware value salience ultimately shape behavior\. This is also supported by empirical results in Fig\.[1](https://arxiv.org/html/2609.28942#S1.F1)\(b\) that combining both leads to more accurate preference prediction\. From a Bayesian lens, inherent personal values and contextual value salience serve as complementary*priors*, while the*posterior values*inferred from observed behaviors reflect final context\-aware personal preferences\.

Inspired by this formulation, we proposeBaCVA, an inference\-timeBayesianContext\-aware personalizedValueAlignment method\. It achieves personalization by dynamically approximating the posterior personalized values based on both static personal values and context\-specific value salience\. Specifically, BaCVA consists of two components: \(i\) ascenario value estimatorthat predicts the context\-aware value salience; \(ii\) adual\-view personalization modulethat approximates posterior personal values from complementary personal value\-driven and scenario\-driven perspectives\. The whole framework is optimized toward a variational inference objective[Tzikas et al\. \(2008\)](https://arxiv.org/html/2609.28942#bib.bib20);[Blei et al\. \(2017\)](https://arxiv.org/html/2609.28942#bib.bib21)\. By framing personalization as context\-aware posterior inference, BaCVA offers two advantages: \(i\)*fine\-grained adaptability*, capturing nuanced value shifts from priors across diverse scenarios; and \(ii\)*data efficiency*, enabling personalization with fewer training samples and seamless generalization to new users\. Extensive experiments on multiple benchmarks showcase the superiority of our framework\.

The main contributions are summarized below\.\(1\)We theoretically and empirically identify a key limitation in current personalized alignment methods: they typically reuse a fixed personal value profile across prompts without explicitly modeling question\-specific changes in value salience\.\(2\)We propose BaCVA, an inference\-time Bayesian personalized alignment framework\. It achieves dynamic personalization by inferring context\-aware posterior preferences conditioned on prior personal values and contextual value salience\.\(3\)Extensive experiments show that BaCVA achieves more accurate and adaptive personalized value alignment, as well as improved data efficiency and generalization\.

## 2Related Work

##### General Value Alignment of LLMs

General \(non\-personalized\) value alignment aims to align LLM outputs with population\-level human preferences like HHH \(helpfulness, harmlessness and honesty\)\([Wang et al\., 2023](https://arxiv.org/html/2609.28942#bib.bib51);[Shen et al\., 2023](https://arxiv.org/html/2609.28942#bib.bib53);[Yao et al\., 2023b](https://arxiv.org/html/2609.28942#bib.bib52)\)\. Prominent methods include supervised fine\-tuning \(SFT\)\([Zhang et al\., 2023](https://arxiv.org/html/2609.28942#bib.bib41);[Han et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib38)\), reinforcement learning from human feedback \(RLHF\)\([Ouyang et al\., 2022](https://arxiv.org/html/2609.28942#bib.bib58);[Bai et al\., 2022a](https://arxiv.org/html/2609.28942#bib.bib54)\), direct preference optimization \(DPO\)\([Rafailov et al\., 2023](https://arxiv.org/html/2609.28942#bib.bib63);[Ethayarajh et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib37);[Meng et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib45)\)and variants\. Besides, inference\-time methods like retrieval\-augmented generation \(RAG\)\([Lewis et al\., 2020](https://arxiv.org/html/2609.28942#bib.bib44);[Zhao et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib43);[Gao et al\., 2023](https://arxiv.org/html/2609.28942#bib.bib42)\)and principle\-based prompting\([Bai et al\., 2022b](https://arxiv.org/html/2609.28942#bib.bib55)\)steer model output without parameter updates\.

These methods inherently struggle to accommodate the diversity of individual user values, motivating the need for personalized value alignment\.

##### Personalized Value Alignment of LLMs

Existing methods fall into two categories\.*Training\-time personalization*incorporates user preferences via model optimization\([Chen et al\., 2025a](https://arxiv.org/html/2609.28942#bib.bib36);[Choi et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib27)\)\. VPL\([Poddar et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib26)\)and RLPA\([Zhao et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib47)\)learn user\-specific reward models or adapters from individual interaction data\. P\-RLHF\([Li et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib48)\)learns user embeddings for personalization\. Despite these methods implicitly fit to observed posterior preferences, they rely on extensive per\-user data and degrade in cold\-start or sparse\-data settings\([Pădurean et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib33)\)\.

In contrast,*inference\-time personalization*steers model behaviors by manipulating prompts[Chen et al\. \(2025b\)](https://arxiv.org/html/2609.28942#bib.bib28);[Jin et al\. \(2025\)](https://arxiv.org/html/2609.28942#bib.bib29)or decoding\([Thonet et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib32);[Zhu et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib70)\), without modifying parameters\. ValuePrompt\([Kang et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib57)\)and COUPLE\([Guo et al\., 2026b](https://arxiv.org/html/2609.28942#bib.bib13)\)condition generation on explicit user value descriptions\. Other approaches summarize or retrieve value\-aware content from user histories, like ValuesRAG\([Ryan et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib30);[Seo et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib46)\)\. Decoding\-based methods\([Zhang et al\., 2025a](https://arxiv.org/html/2609.28942#bib.bib24)\)like MOD\([Shi et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib23)\)and PAD\([Chen et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib25)\)reshape token distributions to adapt general outputs to fixed individual preferences\. These inference\-time approaches mainly reuse the same personal value profile across situations, without explicitly modeling question\-specific changes in the salience of individual value dimensions\. We address this limitation in this paper\.

##### Bayesian Modeling and Applications

Bayesian modeling\([Zhang and Poole, 1996](https://arxiv.org/html/2609.28942#bib.bib79);[Bishop and Tipping, 2013](https://arxiv.org/html/2609.28942#bib.bib82);[Grossman and Domingos, 2004](https://arxiv.org/html/2609.28942#bib.bib80);[Lauritzen and Spiegelhalter, 1990](https://arxiv.org/html/2609.28942#bib.bib81);[Heckerman et al\., 2015](https://arxiv.org/html/2609.28942#bib.bib83)\)offers a principled lens for updating prior beliefs with observed evidence and reasoning with uncertainty\([Gao et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib76)\)\. Recent work\([Gupta et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib78);[Longjohn et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib40);[Wang et al\., 2024a](https://arxiv.org/html/2609.28942#bib.bib69)\)has explored LLMs alignment from a Bayesian perspective\. BIRD\([Feng et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib77)\)frames decision alignment as a Bayesian decision process\. Dellma\([Liu et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib39)\)applies decision theory to address uncertainty\.

This paper formulates personalized value alignment as Bayesian posterior inference, optimized through Variational Inference\([Tzikas et al\., 2008](https://arxiv.org/html/2609.28942#bib.bib20);[Blei et al\., 2017](https://arxiv.org/html/2609.28942#bib.bib21);[Ganguly and Earp, 2021](https://arxiv.org/html/2609.28942#bib.bib22)\)\.

## 3Methodology

![Refer to caption](https://arxiv.org/html/2609.28942v1/models.png)Figure 2:Overview of the proposed Bayesian question\-specific value modeling framework\. A questionxxprovides a decision context, while a user\-level value profilevuiv\_\{u\_\{i\}\}modulates how semantics are personalized\. We learn a Bayesian*residual*\(semantic delta\) from two complementary views, and fuse the two views via a context\-based gated fusion mechanism to predict a personalized, context\-aware answer\.### 3\.1Task Definition

Let𝒰=\{ui\}i=1N\\mathcal\{U\}=\\\{u\_\{i\}\\\}\_\{i=1\}^\{N\}denote a population of diverse users, each useruiu\_\{i\}is associated with a self\-written summary of personal valuescuic\_\{u\_\{i\}\}and a small set of preference annotations𝒟ui=\{\(x,yc,yr\)\}\\mathcal\{D\}\_\{u\_\{i\}\}=\\\{\(x,y\_\{c\},y\_\{r\}\)\\\}, wherexxdenotes a prompt or question,ycy\_\{c\}andyry\_\{r\}mean the preferred and dispreferred responses respectively\. To make the open\-ended value summarycuic\_\{u\_\{i\}\}more actionable and unified, we follow[Yao et al\. \(2023a\)](https://arxiv.org/html/2609.28942#bib.bib64);[Guo et al\. \(2026b\)](https://arxiv.org/html/2609.28942#bib.bib13)to convert it into a structured, multi\-dimensional value profile𝒗ui=\[\(v1,s1\),…,\(vd,sd\)\]\\bm\{v\}\_\{u\_\{i\}\}=\[\(v\_\{1\},s\_\{1\}\),\\ldots,\(v\_\{d\},s\_\{d\}\)\]\.\(v1,…,vd\)\(v\_\{1\},\\ldots,v\_\{d\}\)are finite pre\-defined value dimensions \(e\.g\., care, authority\) from a well\-established value framework \(e\.g\., Schwartz’s Basic Values\([Schwartz, 2012](https://arxiv.org/html/2609.28942#bib.bib61)\)\)\.sj∈\{1,2,3,4,5\}s\_\{j\}\\in\\\{1,2,3,4,5\\\}indicates the priority assigned tovjv\_\{j\}on a 5\-point Likert scale from*‘not important at all \(1\)’*to*‘very important \(5\)’*\.

Since training a dedicated model for each user is less practical in low\-data \(small𝒟ui\\mathcal\{D\}\_\{u\_\{i\}\}\) and cold\-start cases, we focus on inference\-time personalized alignment in this paper\. With a single shared modelpψ​\(𝐲\|𝐱,𝐯ui,𝒙\)p\_\{\\psi\}\(\{\\mathbf\{y\}\}\|\{\\mathbf\{x\}\},\{\\mathbf\{v\}\}\_\{u\_\{i\},\\bm\{x\}\}\)that can generate the response𝐲\{\\mathbf\{y\}\}aligned with the given personalized values𝐯ui,𝒙\{\\mathbf\{v\}\}\_\{u\_\{i\},\\bm\{x\}\}toward the question𝐱\{\\mathbf\{x\}\}, our goal is to infer the concrete personalized values𝐯ui,x∼pθ​\(𝐯\|𝐯ui,𝐱\)\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}\\sim p\_\{\\theta\}\(\{\\mathbf\{v\}\}\|\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{x\}\}\)the user would exhibit to guide response generation under the specific context𝐱\{\\mathbf\{x\}\}\.

For evaluation on test samples\{\(ui,x,yc,yr\)\}\\\{\(u\_\{i\},x,y\_\{c\},y\_\{r\}\)\\\}, we extract the values reflected in two responses as𝒗yc,𝒗yr\{\\bm\{v\}\}\_\{y\_\{c\}\},\{\\bm\{v\}\}\_\{y\_\{r\}\}, then compare inferred personalized values𝒗ui,x\{\\bm\{v\}\}\_\{u\_\{i\},x\}with them using two metrics: \(i\) the distance between𝒗ui,x\{\\bm\{v\}\}\_\{u\_\{i\},x\}and𝒗yc\{\\bm\{v\}\}\_\{y\_\{c\}\}like MAE and \(ii\) the accuracy whether𝒗ui,x\{\\bm\{v\}\}\_\{u\_\{i\},x\}is closer to𝒗yc\{\\bm\{v\}\}\_\{y\_\{c\}\}than to𝒗yr\\bm\{v\}\_\{y\_\{r\}\}\. Furthermore, we evaluate the effectiveness of BaCVA in guiding downstream personalized response generation\. We compare the responses generated from the inferred value𝒗ui,x\{\\bm\{v\}\}\_\{u\_\{i\},x\}againstycy\_\{c\}andyry\_\{r\}\.

### 3\.2The BaCVA Framework

As discussed in Sec\.[1](https://arxiv.org/html/2609.28942#S1), existing inference\-time personalization methods employ the overall user profile𝒗ui\{\\bm\{v\}\}\_\{u\_\{i\}\}or that summarized from the preference data𝒗ui,𝒟i\{\\bm\{v\}\}\_\{u\_\{i\},\\mathcal\{D\}\_\{i\}\}to instantiate𝐯ui,x\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}and keep it fixed across prompts𝐱\{\\mathbf\{x\}\}, thereby limiting their ability to explicitly model question\-specific shifts in the salience of individual value dimensions, as illustrated in Fig\.[1](https://arxiv.org/html/2609.28942#S1.F1)\(a\)\.

Inspired by Lewin’s field theory\([Heidbreder, 1937](https://arxiv.org/html/2609.28942#bib.bib71)\)that observed behaviors emerge from the interaction between the personal value profile𝐯ui\{\\mathbf\{v\}\}\_\{u\_\{i\}\}and the situation𝐱\{\\mathbf\{x\}\}, and Cognitive Appraisal Theory\([Lazarus and Folkman, 1984](https://arxiv.org/html/2609.28942#bib.bib34)\)that situations modulate the expression of internal values to form final actions, we formalize the contextual factor as a scenario\-specific value salience distribution\. By treating both the personal value profile and specific context value salience as priors \(prior to behavioral execution\), we propose BaCVA, a Bayesian framework that dynamically estimates the contextual personalized values𝐯ui,x\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}via posterior inference\. Given the data𝒟ui=\{\(x,yc,yr\)\}\\mathcal\{D\}\_\{u\_\{i\}\}=\\\{\(x,y\_\{c\},y\_\{r\}\)\\\}, we model observed user preferencesyc≻yry\_\{c\}\\succ y\_\{r\}as the likelihood and the posterior distribution of the target𝐯ui,x\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}is formalized as a calibration from the prior value distributionpθ​\(𝐯ui,x\|𝐯ui,𝐱\)p\_\{\\theta\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}\|\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{x\}\}\)by the likelihood:

pθ​\(𝐯ui,x\|𝐯ui,𝐱,yc≻yr\)\\displaystyle p\_\{\\theta\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}\|\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{x\}\},y\_\{c\}\\succ y\_\{r\}\)\(1\)∝pθ​\(yc≻yr\|𝐯ui,x\)×pθ​\(𝐯ui,x\|𝐯ui,𝐱\)\.\\displaystyle\\propto p\_\{\\theta\}\(y\_\{c\}\\succ y\_\{r\}\|\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}\)\\times p\_\{\\theta\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}\|\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{x\}\}\)\.
This formalization yields two advantages: \(1\) more adaptive personalization by enabling fine\-grained calibration from informative priors, and \(2\) training efficiency by introducing high\-quality priors\. Moreover, BaCVA does not rely on fine\-tuning user\-specific adapters but personal value profiles, thus easily extended to cold\-start users\.

Since direct computation of the posterior value distribution is intractable, we refer to Variational Inference \(VI\)\([Blei et al\., 2017](https://arxiv.org/html/2609.28942#bib.bib21)\)and introduce a variational posteriorqϕ​\(𝐯ui,x\|𝐯ui,𝐱\)q\_\{\\phi\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}\|\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{x\}\}\)implemented by LLMs\. As shown in Fig\.[2](https://arxiv.org/html/2609.28942#S3.F2), the architecture consists of two modules: \(i\) ascenario value estimatorto predict contextual value salience; and \(ii\) adual\-view personalization modulethat approximates the posterior distribution from both personal value\-driven and scenario\-driven perspectives\. Then, we optimize the model by minimizing the divergence between the variational posterior and the true posterior, which is equivalent to maximizing the evidence lower bound \(ELBO\) below, with full derivation in Appendix[A\.1](https://arxiv.org/html/2609.28942#A1.SS1)\.

ℒ=\\displaystyle\\mathcal\{L\}=−𝔼qϕ​\(𝐯ui,x\)​\[log⁡pθ​\(yc≻yr∣𝐯ui,x,x\)\]\\displaystyle\-\\mathbb\{E\}\_\{q\_\{\\phi\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}\)\}\\\!\\left\[\\log p\_\{\\theta\}\(y\_\{c\}\\succ y\_\{r\}\\mid\{\\mathbf\{v\}\}\_\{u\_\{i\},x\},x\)\\right\]\(2\)\+KL\(qϕ\(𝐯ui,x∣𝐯ui,𝐱\)∥pθ\(𝐯ui,x∣𝐯ui,𝐱\)\)\.\\displaystyle\+\\mathrm\{KL\}\\\!\\left\(q\_\{\\phi\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}\\mid\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{x\}\}\)\\,\\\|\\,p\_\{\\theta\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}\\mid\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{x\}\}\)\\right\)\.
Each module is detailed in the next subsections\.

### 3\.3Estimation of Scenario Value Salience

As illustrated in Fig\.[1](https://arxiv.org/html/2609.28942#S1.F1), different value dimensions are not equally salient across scenarios, instead, a specific questionxxusually activates only a subset of dimensions\. Typically, the salience distribution over these dimensions are shaped by universal social norms, encouraging some values while rendering others less acceptable\.

Inspired by this, we estimate the scenario value salience distribution𝐯x\{\\mathbf\{v\}\}\_\{x\}based on population\-level aligned response\. Concretely, for a questionxx, we leverage LLMs that have gone through great human alignment as a proxy to obtain the universal response, denoted as𝐲human\{\\mathbf\{y\}\}\_\{\\text\{human\}\}\. Then, we employ a prompt\-based value estimatorqωq\_\{\\omega\}to infer𝐯x\{\\mathbf\{v\}\}\_\{x\}:

𝐯x∼qω\(⋅\|𝐱,𝐲human\)\.\\displaystyle\{\\mathbf\{v\}\}\_\{x\}\\sim q\_\{\\omega\}\(\\cdot\|\{\\mathbf\{x\}\},\{\\mathbf\{y\}\}\_\{\\text\{human\}\}\)\.\(3\)
In our primary implementation, we leverage GPT\-5\-nano to generate the universal response and estimate the value salience through in\-context learning\. We also replace it with other LLMs to validate the robustness in Appendix[C](https://arxiv.org/html/2609.28942#A3)\. Detailed prompts are provided in Appendix[A\.3\.2](https://arxiv.org/html/2609.28942#A1.SS3.SSS2)\.

### 3\.4Dual\-View Personalized Value Inference

Lewin’s field theory states that behavior arises from the interaction between the person and the environment\. Thus, the concrete values reflected in final behaviors can be explained from two complementary perspectives: \(1\)*Scenario\-anchored personalization drift*, where shared norms under the specific scenario largely determine the behavior while personal values introduce slight but meaningful deviation; \(2\)*Person\-anchored contextual instantiation*, which assumes that personalization is primarily rooted in the user’s inherent value profile while the scenario modulates how salient each value is\. Motivated by this, we design adual\-view personalization moduleto approximate the posterior distribution over𝐯ui,𝒙\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}\. Specifically, it includes a*scenario\-driven component*and a*person\-driven component*to infer the posterior respectively:

qϕ​\(𝐯ui,𝒙\(𝐱\)\|𝐯x,𝐯ui\),qϕ​\(𝐯ui,𝒙\(𝐮\)\|𝐯ui,𝐯x\)\.\\displaystyle q\_\{\\phi\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{x\}\}\)\}\|\{\\mathbf\{v\}\}\_\{x\},\{\\mathbf\{v\}\}\_\{u\_\{i\}\}\),\\quad q\_\{\\phi\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{u\}\}\)\}\|\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{v\}\}\_\{x\}\)\.\(4\)Then, the final variational posterior is calculated as a gated composition of the two perspectives:

qϕ​\(𝐯ui,x\|𝐯ui,𝐱\)=α​qϕ​\(𝐯ui,𝒙\(𝐱\)\|𝐯x,𝐯ui\)\\displaystyle q\_\{\\phi\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},x\}\|\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{x\}\}\)=\\alpha q\_\{\\phi\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{x\}\}\)\}\|\{\\mathbf\{v\}\}\_\{x\},\{\\mathbf\{v\}\}\_\{u\_\{i\}\}\)\(5\)\+\(1−α\)​qϕ​\(𝐯ui,𝒙\(𝐮\)\|𝐯ui,𝐯x\),\\displaystyle\+\(1\-\\alpha\)q\_\{\\phi\}\(\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{u\}\}\)\}\|\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{v\}\}\_\{x\}\),whereα\\alphais produced through*a context\-aware gate*\.

##### View 1: Scenario\-Anchored Personalization Drift

Conditioned on the scenario\-based value salience priorqω​\(𝐯x\)q\_\{\\omega\}\(\{\\mathbf\{v\}\}\_\{x\}\), this view models personalization as adding a personal residual to the scenario anchor in logit space:

qϕ​\(𝐯ui,x\(x\)∣𝐯x,𝐯ui\)\\displaystyle q\_\{\\phi\}\(\{\\mathbf\{v\}\}^\{\(x\)\}\_\{u\_\{i\},x\}\\mid\{\\mathbf\{v\}\}\_\{x\},\{\\mathbf\{v\}\}\_\{u\_\{i\}\}\)=Softmax⁡\(𝐬x\+zu\),\\displaystyle=\\mathrm\{Softmax\}\(\\mathbf\{s\}\_\{x\}\+z\_\{u\}\),\(6\)𝐬x\\displaystyle\\mathbf\{s\}\_\{x\}=Logit⁡\(qω​\(𝐯x\)\),\\displaystyle=\\mathrm\{Logit\}\(q\_\{\\omega\}\(\{\\mathbf\{v\}\}\_\{x\}\)\),\(7\)wherezuz\_\{u\}is the personal residual logit vector\. Intuitively, this view is effective when most individuals follow the scenario\-level consensus with slight personalized divergence\.

##### View 2: Person\-Anchored Contextual Instantiation

This view starts from the personal value priorqϕ​\(𝐯u\)q\_\{\\phi\}\(\{\\mathbf\{v\}\}\_\{u\}\)and instantiates it under the current scenario by adding a context residual in logit space:

qϕ​\(𝐯ui,x\(u\)∣𝐯ui,𝐯x\)\\displaystyle q\_\{\\phi\}\(\{\\mathbf\{v\}\}^\{\(u\)\}\_\{u\_\{i\},x\}\\mid\{\\mathbf\{v\}\}\_\{u\_\{i\}\},\{\\mathbf\{v\}\}\_\{x\}\)=Softmax⁡\(𝐬u\+zx\),\\displaystyle=\\mathrm\{Softmax\}\(\\mathbf\{s\}\_\{u\}\+z\_\{x\}\),\(8\)𝐬u\\displaystyle\\mathbf\{s\}\_\{u\}=Logit⁡\(qϕ​\(𝐯u\)\),\\displaystyle=\\mathrm\{Logit\}\(q\_\{\\phi\}\(\{\\mathbf\{v\}\}\_\{u\}\)\),\(9\)wherezxz\_\{x\}is the context residual logit vector\. This perspective emphasizes personal value\-driven behaviors where scenario\-level expectations provide only weak constraints\.

In practice, we employ open\-sourced LLMs to represent these distributions by prompting them with appropriate instructions and extracting the generation probability of each value\. More details about the prompts are provided in Appendix[A\.3\.2](https://arxiv.org/html/2609.28942#A1.SS3.SSS2)\.

##### Context\-Aware Gated Fusion

The above dual\-view posterior values are complementary and capture dominant information suited for different scenarios\. Rather than equally combining them across all contexts, it would be more effective to dynamically consider their relative dominance\. Consequently, we introduce a context\-aware fusion mechanism to compute the gate scoreα∈\[0,1\]\\alpha\\in\[0,1\]as:

α=𝒢⁡\(𝒙,q⁡\(𝐯ui,𝒙\(𝐱\)\),q⁡\(𝐯ui,𝒙\(𝐮\)\),σ𝐯ui,𝒙\(𝐱\)2,σ𝐯ui,𝒙\(𝐮\)2\)\.\\alpha=\\mathcal\{G\}\(\{\\bm\{x\}\},q\(\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{x\}\}\)\}\),q\(\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{u\}\}\)\}\),\\sigma^\{2\}\_\{\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{x\}\}\)\}\},\\sigma^\{2\}\_\{\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{u\}\}\)\}\}\)\.

\(10\)q⁡\(𝐯ui,𝒙\(𝐱\)\)q\(\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{x\}\}\)\}\)is the abbreviation ofq⁡\(𝐯ui,𝒙\(𝐱\)\|𝐯x,𝐯ui\)q\(\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{x\}\}\)\}\|\{\\mathbf\{v\}\}\_\{x\},\{\\mathbf\{v\}\}\_\{u\_\{i\}\}\),σ𝐯ui,𝒙\(𝐱\)2\\sigma^\{2\}\_\{\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{x\}\}\)\}\}andσ𝐯ui,𝒙\(𝐮\)2\\sigma^\{2\}\_\{\{\\mathbf\{v\}\}\_\{u\_\{i\},\{\\bm\{x\}\}\}^\{\(\{\\mathbf\{u\}\}\)\}\}are variances to measure the uncertainty of posterior inference in the two views\. In practice, the gate function𝒢\\mathcal\{G\}corresponds to an LLM, which jointly reasons over the inputs and outputs a continuous context\-aware fusion weight by normalizing the logits of tokens0and1, rather than making a hard binary decision\. Specific implementation details are given in Appendix[A\.3\.2](https://arxiv.org/html/2609.28942#A1.SS3.SSS2)\.

##### Variational Inference Optimization

We use the same value extractor in Subsec\.[3\.3](https://arxiv.org/html/2609.28942#S3.SS3)to analyze the value reflected by the preferred responsevycv\_\{y\_\{c\}\}, used as the supervision during training\. Besides, we approximate the expectation term in Eq\.\([2](https://arxiv.org/html/2609.28942#S3.E2)\) with a single\-pass amortized estimate, getting the predicted posterior value asv^ui,x\\hat\{v\}\_\{u\_\{i\},x\}\. Then, the Variational Inference loss in our dual\-view personalization model is:

ℒ=‖v^ui,x−vyc‖22\+λ1∑k∈\{u,x\}KL\(qϕ\(k\)∥pθ\(k\)\)\.\\mathcal\{L\}=\\left\\\|\\hat\{v\}\_\{u\_\{i\},x\}\-v\_\{y\_\{c\}\}\\right\\\|\_\{2\}^\{2\}\+\\lambda\_\{1\}\\sum\_\{k\\in\\\{\\mathrm\{u\},\\mathrm\{x\}\\\}\}\\mathrm\{KL\}\(q\_\{\\phi\}^\{\(k\)\}\\,\\\|\\,p\_\{\\theta\}^\{\(k\)\}\)\.

\(11\)

## 4Experiments

PRISMGOODCategoryMethodMAE↓\\downarrowCorrelation↑\\uparrowAccuracy↑\\uparrowMAE↓\\downarrowCorrelation↑\\uparrowNon\-personalized AlignmentDirectAnswer1\.8580\.48567\.053\.3020\.517Inference\-timePersonalized AlignmentPersonValue2\.9690\.41156\.563\.6980\.189Value Prompt1\.7900\.56872\.953\.0620\.579MetaAligner1\.9550\.50969\.003\.2310\.480COUPLE1\.8190\.58865\.983\.0890\.347PAD1\.7330\.60766\.673\.2360\.360MOD1\.6200\.59566\.373\.1200\.410ValuesRAG0\.9520\.63871\.682\.2610\.589BaCVA \(Ours\)0\.728∗0\.700∗81\.72∗1\.118∗0\.769∗Table 1:Main results onPRISMandGOOD\. The best results are bold and second\-best results are underlined\.∗indicates statistically significant improvement over all baselines \(p<0\.05p<0\.05\)\. Accuracy is not applicable to GOOD that has only a ground truth answer but not preferred and dispreferred response pairs\.### 4\.1Experiment Setting

##### Datasets

We evaluate BaCVA onPRISM\([Kirk et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib62)\)andGOOD\([Wang et al\., 2019](https://arxiv.org/html/2609.28942#bib.bib68)\), both of which provide personal value profiles and user behaviors across diverse scenarios\. PRISM contains open\-ended questions with preferred/dispreferred candidate responses, while GOOD contains persuasion dialogues with value annotations and user\-generated responses\. After preprocessing, they contain2,1312,131and2,6962,696instances, respectively, and are split 80/20 for training and testing\. Additional results on other datasets are provided in Appendix[C\.1](https://arxiv.org/html/2609.28942#A3.SS1)\.

##### Evaluation Metrics

As formalized in Sec\.[3\.1](https://arxiv.org/html/2609.28942#S3.SS1), we represent value orientation as a multi\-dimensional value vector\. First, we reportMean Absolute Error \(MAE\)between the predicted user value preferences and the value of preferred answers\. Second, we computeSpearman’s rank correlation coefficient \(Correlation\)to measure the consistency of value priorities\. Third, we report theAccuracythat the preferred responses are ranked higher than dispreferred responses\. Detailed dataset construction, statistics, and metric definitions are provided in Appendix[B](https://arxiv.org/html/2609.28942#A2)\.

##### Baselines

We compare BaCVA with baselines spanning non\-personalized and inference\-time personalized alignment\.DirectAnswerdirectly generates responses using the generally aligned backbone model\. For personalized value alignment, we mainly consider inference\-time approaches that incorporate static user value profiles to steer answer generation, including four groups: \(1\)Value Prompt\([Kang et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib57)\)prompts the value profile to steer outputs,COUPLE\([Guo et al\., 2026b](https://arxiv.org/html/2609.28942#bib.bib13)\)conducts counterfactual reasoning, andMetaAligner\([Yang et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib66)\)rewrites model outputs to align them with target values; \(2\)PAD\([Chen et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib25)\)andMOD\([Shi et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib23)\)learn value\-aware reward models to steer decoding\-time token distribution towards personal values; \(3\)ValuesRAG\([Seo et al\., 2025](https://arxiv.org/html/2609.28942#bib.bib46)\)retrieves historical data under similar scenarios and value profiles as example for decision\-making; and \(4\)PersonValueis implemented by us that directly compares the static value profiles with values extracted from the preferred respones\. To further compare BaCVA with training\-time alignment approaches, we additionally evaluate RLHF\([Christiano et al\., 2017](https://arxiv.org/html/2609.28942#bib.bib2)\)and MORLHF\([Li et al\., 2020](https://arxiv.org/html/2609.28942#bib.bib1)\)under the same backbone, data splits, and input information, with full implementation details and results provided in Appendix[C\.2](https://arxiv.org/html/2609.28942#A3.SS2)\.

BaCVAdynamically estimates personalized values across varying contexts via posterior inference\.

##### Implementation details\.

We use Qwen3\-8B as the backbone model across all comparable methods and control input as only personal value profile and question to ensure fair comparisons\. The model for value extraction in both our model and evaluation is GPT\-5\-nano, whose consistency with humans has been verified, achieving a score\>90\>90\. Unless otherwise specified, we use the 10 Schwartz Basic value dimensions to represent value profiles in the main experiments\. Additional experiments on other value systems to validate the robustness are provided in Appendix[C\.1](https://arxiv.org/html/2609.28942#A3.SS1)\. All models are trained using the Adam optimizer\. All experiments are conducted using NVIDIA A100 GPUs\. Full implementation details, hyperparameters, and more experimental settings are in Appendix[B](https://arxiv.org/html/2609.28942#A2)\.

### 4\.2Personalized Alignment Performance

Tab\.[1](https://arxiv.org/html/2609.28942#S4.T1)presents the results comparing BaCVA with baselines on personalized value alignment across PRISM and GOOD, with three key findings\.

First, incorporating personalization is crucial for LLM value alignment\. Across both benchmarks, personalized alignment methods such as Value Prompt, MOD and ValuesRAG substantially outperform the non\-personalized baseline DirectAnswer\. This observation demonstrates that personal value\-agnostic alignment fails to capture the diversity of individual preferences\.Second, static personal value profiles lead to sub\-optimal personalized alignment\.In particular, the PersonValue baseline, which enforces adherence to static personal values across varying prompts, achieves merely 56\.56% accuracy on PRISM\. This highlights a fundamental limitation that static personal values cannot faithfully represent users’ context\-dependent preferences across diverse scenarios\.Third, BaCVA, which explicitly integrates personal value profiles with scenario\-specific value salience and optimizes toward posterior personalized preferences, achieves the best personalization result across both benchmarks\.OnPRISM, BaCVA attains the lowest MAE and the highest accuracy, while onGOODit significantly improves both MAE and correlation\. These consistent gains strongly validate our motivation in this paper and the effectiveness of our proposed method\. Additionally, BaCVA also consistently outperforms the training\-time alignment baselines reported in Appendix[C\.2](https://arxiv.org/html/2609.28942#A3.SS2)\.

##### Human Evaluation

Beyond automatic evaluation, we conduct a human study on 50 samples corresponding to 50 distinct user profiles, with each sample independently assessed by two evaluators, who compare how closely the contextual personalized values predicted by BaCVA and ValuesRAG mirror the values of the preferred answer\. As shown in Fig\.[3](https://arxiv.org/html/2609.28942#S4.F3), BaCVA wins in 46\.0% and loses in only 22\.0% of cases, outperforming ValuesRAG\. Details are provided in Appendix[B\.4](https://arxiv.org/html/2609.28942#A2.SS4)\.

Figure 3:Performance of human evaluationTable 2:Ablation study onPRISMandGOOD\.

### 4\.3Ablation Study

To systematically investigate the contribution of each component in BaCVA, we conduct an ablation study by removing or modifying key modules\. \(1\)w/o Scenario\-Viewandw/o Person\-Viewapproximate the posterior value preferences from only one of the dual perspectives\. \(2\)w/o Gated Fusionreplaces the adaptive gating mechanism with an averaged fusion of the two views across all contexts\. \(3\)w/o Prior Constraintremoves the KL divergence terms in Eq\.\([11](https://arxiv.org/html/2609.28942#S3.E11)\)\. Tab\.[2](https://arxiv.org/html/2609.28942#S4.T2)reports the ablation results onPRISMandGOOD\.

Results AnalysisUsing merely a single view \(i\.e\.,*w/o Scenario\-View*and*w/o Person\-View*\) consistently results in degraded performance on both benchmarks\. This confirms that both views capture effective and complementary aspects for personalized alignment\. Besides, removing the adaptive gating mechanism \(*w/o Gated Fusion*\) causes a performance drop\. This demonstrates that the interaction between the two views is highly context\-dependent rather than simply additive\. The gated fusion is essential to distinguish their relative importance across different scenarios\. Finally, the prior constraints of both personal values and scenario\-specific value salience are also critical for personalized alignment by avoiding collapsed prediction\. This also validates that both views provide meaningful prior knowledge\.

![Refer to caption](https://arxiv.org/html/2609.28942v1/figures/control.png)Figure 4:Fine\-grained personalization onPRISM\.![Refer to caption](https://arxiv.org/html/2609.28942v1/figures/exp_efficiency.png)Figure 5:Training efficiency analysis across datasets\.Figure 6:Effects of the gated fusion on PRISM\.
### 4\.4Analysis Experiments

##### Fine\-grained Adaptability Analysis

We examine whether BaCVA captures fine\-grained shifts from priors by grouping test samples according to the distance between the true preferred response valuevycv\_\{y\_\{c\}\}and two priors: the personal priorvuv\_\{u\}and the scenario priorvxv\_\{x\}\. A larger distance means that the final contextual preference deviates more from the corresponding prior, making the sample more challenging for conventional methods that simply follow the prior\. As shown in Fig\.[4](https://arxiv.org/html/2609.28942#S4.F4)\(a\), ValuesRAG performs well only whenvycv\_\{y\_\{c\}\}is close tovuv\_\{u\}, but degrades sharply as the distance increases, indicating its strong dependence on static personal values\. In contrast, BaCVA remains stable across different distances from both priors\. These results suggest that BaCVA can capture nuanced preference shifts to infer posterior preferences even when the context heavily conflicts with priors\. Moreover, a complementary conflict\-level analysis further shows that BaCVA reduces MAE by 17\.8% relative to ValuesRAG on the high\-conflict subset, where answer\-level values deviate substantially from the global user profile \(Appendix[C\.9](https://arxiv.org/html/2609.28942#A3.SS9)\)\.

A cluster\-level analysis onPRISMfurther confirms BaCVA’s domain\-level robustness, with consistent MAE reductions across the five largest semantic clusters \(Appendix[C\.3](https://arxiv.org/html/2609.28942#A3.SS3)\)\.

##### Training Data Efficiency

We evaluate data efficiency by training BaCVA with different proportions of PRISM and GOOD data\. As shown in Fig\.[5](https://arxiv.org/html/2609.28942#S4.F5), BaCVA achieves strong performance with limited supervision and improves steadily as more data is used, whereas ValuesRAG converges more slowly and to a weaker final result\. This indicates that BaCVA can learn effective value\-alignment patterns in low\-data regimes\.

##### Generalization to Cold\-Start Users

We further evaluate generalization to unseen users by splitting the PRISM test set into seen users and 20% cold\-start users absent from training\. As shown in Fig\.[7](https://arxiv.org/html/2609.28942#S4.F7), BaCVA shows a much smaller performance drop than ValuesRAG, demonstrating stronger robustness to unseen users\.

Figure 7:Generalizability evaluation onPRISM\.
##### Gated Fusion Analysis\.

BaCVA fuses the person\-view and scenario\-view through a context\-aware gate\. To verify whether the gate correctly identifies the more informative view across varying contexts, we split PRISM into personal\-dominant cases, wherevycv\_\{y\_\{c\}\}is closer tovuv\_\{u\}than tovxv\_\{x\}, and scenario\-dominant cases\. Fig\.[6](https://arxiv.org/html/2609.28942#S4.F6)\(a\) shows that PersonValue performs competitively in personal\-dominant settings but fails catastrophically when situational shifts occur, whereas*DirectAnswer*is opposite\. BaCVA consistently performs best on both\. Fig\.[6](https://arxiv.org/html/2609.28942#S4.F6)\(b\) further shows that the gate assigns larger weights to the corresponding dominant view\.

##### Cross\-Extractor Robustness

Cross\-extractor evaluation using independently constructed Gemini\-3\-Flash and Qwen3\-235B\-A22B test targets confirms that BaCVA’s gains are not tied to GPT\-5\-nano, yielding 21\.2% lower MAE and 9\.56 percentage points higher accuracy than ValuesRAG on average \(Appendix[C\.6](https://arxiv.org/html/2609.28942#A3.SS6)\)\.

### 4\.5Hyper\-parameter Analysis

We provide a sensitivity analysis of the KL\-divergence weightλ1\\lambda\_\{1\}in Appendix[C\.1](https://arxiv.org/html/2609.28942#A3.SS1)\. Besides, we further examine the robustness of BaCVA under different backbone LLMs as the value salience estimator, prompt designs, and value representations\. Specifically, we evaluate BaCVA with value profiles represented by the 5\-dim Moral Foundations Theory and the 301\-dim value system from Daily\-Dilemma\. The consistent gains across these settings show that BaCVA is robust to both implementation choices and value\-system variations\. Meanwhile, to evaluate BaCVA’s generalizability across backbone models, we further implement it with Llama3\-8B under the same experimental setting, where it reduces MAE by 37\.9% and improves accuracy by 19\.06 percentage points over ValuesRAG in Appendix[C\.8](https://arxiv.org/html/2609.28942#A3.SS8)\. To evaluate robustness across scenario\-prior estimators, we compare GPT\-5\-nano, Gemini\-3\-Flash, and Qwen3\-235B\-A22B, obtaining an average within\-one agreement of 89\.15% and a standard deviation of only 0\.027 in downstream PRISM MAE in Appendix[C\.7](https://arxiv.org/html/2609.28942#A3.SS7)\. To assess whether BaCVA training degrades general\-purpose capabilities, we evaluate the original backbone and BaCVA on four standard benchmarks, obtaining comparable average performance of 77\.69% and 78\.74%, respectively \(Appendix[C\.10](https://arxiv.org/html/2609.28942#A3.SS10)\)\.

### 4\.6Downstream Application

Furthermore, we evaluate the effectiveness of our inferred posterior values in guiding downstream personalized response generation using the PRISM dataset\. We implement two variants for generation: \(1\)BaCVA \(Two\-stage\)applies the posterior personalized values estimated by BaCVA as the input and trains a downstream response generator\. \(2\)BaCVA\-E2Ejointly optimizes the BaCVA for posterior value inference with subsequent response generator\. Here, MAE, Spearman correlation, and pairwise ranking accuracy are computed under the answer\-level protocol detailed in Appendix[C\.5](https://arxiv.org/html/2609.28942#A3.SS5)\. As shown in Tab\.[3](https://arxiv.org/html/2609.28942#S4.T3), BaCVA\-E2E achieves the best overall performance, slightly outperforming the two\-stage BaCVA\.

Moreover, qualitative cases in Appendix[C\.5](https://arxiv.org/html/2609.28942#A3.SS5)further show that BaCVA\-E2E can generate coherent value\-grounded responses, whereas ValuePrompt often fails\.

Table 3:Results of downstream personalized response generation on PRISM\.Boldindicates the best results\.

## 5Conclusion

To overcome the limitation that current personalized value alignment relies on static user value profiles, BaCVA treats personal values and context\-specific value salience as priors and infers posterior personalized values at inference time\. Experiments validate improved personalization accuracy, adaptability, data efficiency, and generalization\.

## Limitations

Although BaCVA achieves promising performance on personalized value alignment across two benchmarks, this work still has several limitations\.

*First*, the evaluation method may introduce noise, as it relies on user preferences annotated over a limited set of candidate responses, which may not fully capture the faithful user values but only compromised preferences\. Moreover, our experiments are primarily conducted on offline datasets; although human evaluation is included, the approach lacks validation through large\-scale real\-world or online deployment\.*Second*, the cross\-dataset generalization of our model remains an important aspect to be further validated\.*Third*, the datasets we used for evaluation contain users’ self\-reported value profiles, which may be noisy as users may consciously or unconsciously curate, simplify, or embellish their profiles\.*Last*, the estimator remains an aligned\-LLM approximation of population\-level value salience, which future work may improve with more diverse models, alignment strategies, and cultural assumptions\.

In future work, we plan to address these limitations by exploring more realistic and comprehensive evaluation settings, investigating cross\-dataset generalization, and developing more robust mechanisms to model and mitigate noise in user\-provided information\.

## Ethical Considerations

This work aims to improve personalized value alignment in large language models; however, the inferred “value preferences” reflect latent patterns captured from observed interactions and annotations, and should be viewed as approximate rather than complete representations of human values\. It should not be used to directly judge individuals or replace human ethical decision\-making\. Personalized value modeling may also involve privacy and bias risks\. Therefore, any real\-world deployment should strictly follow data protection regulations, avoid manipulative or unethical uses, and be continuously evaluated within a responsible AI framework\.

## Acknowledgments

This work was supported by the Beijing Nova Program \(Grant No\. 202604841294\)\.

## References

- Achiamet al\.\(2023\)J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Baiet al\.\(2022a\)Y\. Bai, A\. Jones, K\. Ndousse, A\. Askell, A\. Chen, N\. DasSarma, D\. Drain, S\. Fort, D\. Ganguli, T\. Henighan,et al\.Training a helpful and harmless assistant with reinforcement learning from human feedback\.arXiv preprint arXiv:2204\.05862\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Baiet al\.\(2022b\)Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon,et al\.Constitutional ai: harmlessness from ai feedback\.arXiv preprint arXiv:2212\.08073\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Bishop and Tipping \(2013\)C\. M\. Bishop and M\. TippingVariational relevance vector machines\.External Links:1301\.3838,[Link](https://arxiv.org/abs/1301.3838)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Bleiet al\.\(2017\)D\. M\. Blei, A\. Kucukelbir, and J\. D\. McAuliffeVariational inference: a review for statisticians\.Journal of the American statistical Association112\(518\),pp\. 859–877\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p4.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p2.1),[§3\.2](https://arxiv.org/html/2609.28942#S3.SS2.p4.1)\.
- Burnes\* \(2004\)B\. Burnes\*Kurt lewin and complexity theories: back to the future?\.Journal of change management4\(4\),pp\. 309–325\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p3.1)\.
- Chenet al\.\(2025a\)D\. Chen, Y\. Chen, A\. Rege, Z\. Wang, and R\. K\. VinayakPAL: sample\-efficient personalized reward modeling for pluralistic alignment\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2024\)R\. Chen, X\. Zhang, M\. Luo, W\. Chai, and Z\. LiuPad: personalized alignment of llms at decoding\-time\.arXiv preprint arXiv:2410\.04070\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p2.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1),[§4\.1](https://arxiv.org/html/2609.28942#S4.SS1.SSS0.Px3.p1.1)\.
- Chenet al\.\(2025b\)Y\. Chen, X\. Liu, R\. Wang, Z\. Li, P\. Chen, C\. Yu, P\. Nigam, M\. Jiang, and B\. YinPOPI: personalizing llms via optimized natural language preference inference\.arXiv preprint arXiv:2510\.17881\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p2.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1)\.
- Chiuet al\.\(2024\)Y\. Y\. Chiu, L\. Jiang, and Y\. ChoiDailydilemmas: revealing value preferences of llms with quandaries of daily life\.arXiv preprint arXiv:2410\.02683\.Cited by:[§C\.2](https://arxiv.org/html/2609.28942#A3.SS2.SSS0.Px4.p1.1)\.
- Choiet al\.\(2025\)Y\. Choi, S\. Cho, M\. Lee, M\. Park, Y\. Ko, J\. Ok, and D\. KimCoPL: collaborative preference learning for personalizing llms\.arXiv preprint arXiv:2503\.01658\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p1.1)\.
- Christianoet al\.\(2017\)P\. F\. Christiano, J\. Leike, T\. Brown, M\. Martic, S\. Legg, and D\. AmodeiDeep reinforcement learning from human preferences\.Advances in neural information processing systems30\.Cited by:[§4\.1](https://arxiv.org/html/2609.28942#S4.SS1.SSS0.Px3.p1.1)\.
- Cialdiniet al\.\(1990\)R\. B\. Cialdini, R\. R\. Reno, and C\. A\. KallgrenA focus theory of normative conduct: recycling the concept of norms to reduce littering in public places\.\.Journal of personality and social psychology58\(6\),pp\. 1015\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p3.1)\.
- Clarket al\.\(2019\)C\. Clark, K\. Lee, M\. Chang, T\. Kwiatkowski, M\. Collins, and K\. ToutanovaBoolq: exploring the surprising difficulty of natural yes/no questions\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),pp\. 2924–2936\.Cited by:[§C\.10](https://arxiv.org/html/2609.28942#A3.SS10.p1.1)\.
- Clarket al\.\(2018\)P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. TafjordThink you have solved question answering? try arc, the ai2 reasoning challenge\.arXiv preprint arXiv:1803\.05457\.Cited by:[§C\.10](https://arxiv.org/html/2609.28942#A3.SS10.p1.1)\.
- Epstein and O’Brien \(1985\)S\. Epstein and E\. J\. O’BrienThe person–situation debate in historical and current perspective\.\.Psychological bulletin98\(3\),pp\. 513\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p3.1)\.
- Ethayarajhet al\.\(2024\)K\. Ethayarajh, W\. Xu, N\. Muennighoff, D\. Jurafsky, and D\. KielaKto: model alignment as prospect theoretic optimization\.arXiv preprint arXiv:2402\.01306\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Fenget al\.\(2025\)Y\. Feng, B\. Zhou, W\. Lin, and D\. RothBIRD: a trustworthy bayesian inference framework for large language models\.External Links:2404\.12494,[Link](https://arxiv.org/abs/2404.12494)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Ganguly and Earp \(2021\)A\. Ganguly and S\. W\. EarpAn introduction to variational inference\.arXiv preprint arXiv:2108\.13083\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p2.1)\.
- Gaoet al\.\(2024\)Y\. Gao, G\. Xu, Z\. Wang, and A\. CohanBayesian calibration of win rate estimation with llm evaluators\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 4757–4769\.External Links:[Link](http://dx.doi.org/10.18653/v1/2024.emnlp-main.273),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.273)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Gaoet al\.\(2023\)Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, H\. Wang, and H\. WangRetrieval\-augmented generation for large language models: a survey\.arXiv preprint arXiv:2312\.109972\(1\)\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Grossman and Domingos \(2004\)D\. Grossman and P\. M\. DomingosLearning bayesian network classifiers by maximizing conditional likelihood\.Proceedings of the twenty\-first international conference on Machine learning\.External Links:[Link](https://api.semanticscholar.org/CorpusID:6736485)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Guoet al\.\(2025a\)D\. Guo, D\. Yang, H\. Zhang, J\. Song, R\. Zhang, R\. Xu, Q\. Zhu, S\. Ma, P\. Wang, X\. Bi,et al\.Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Guoet al\.\(2026a\)H\. Guo, J\. Lian, and X\. ZhouWhy not collaborative filtering in dual view? bridging sparse and dense models\.ACM Transactions on Information Systems44\(3\),pp\. 1–24\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Guoet al\.\(2025b\)H\. Guo, Y\. Ma, and X\. ZhouSorex: towards self\-explainable social recommendation with relevant ego\-path extraction\.ACM Transactions on Information Systems44\(2\),pp\. 1–27\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Guoet al\.\(2026b\)H\. Guo, J\. Yao, X\. Zhou, X\. Yi, and X\. XieCounterfactual reasoning for steerable pluralistic value alignment of large language models\.Advances in Neural Information Processing Systems38,pp\. 122128–122169\.Cited by:[§B\.4](https://arxiv.org/html/2609.28942#A2.SS4.SSS0.Px1.p2.1),[§1](https://arxiv.org/html/2609.28942#S1.p1.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1),[§3\.1](https://arxiv.org/html/2609.28942#S3.SS1.p1.1),[§4\.1](https://arxiv.org/html/2609.28942#S4.SS1.SSS0.Px3.p1.1)\.
- Guptaet al\.\(2025\)R\. Gupta, R\. Corona, J\. Ge, E\. Wang, D\. Klein, T\. Darrell, and D\. M\. ChanEnough coin flips can make llms act bayesian\.External Links:2503\.04722,[Link](https://arxiv.org/abs/2503.04722)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Hanet al\.\(2025\)X\. Han, J\. Yang, T\. Wang, Z\. Bi, X\. Song, J\. Hao, and J\. SongTowards alignment\-centric paradigm: a survey of instruction tuning in large language models\.arXiv preprint arXiv:2508\.17184\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Heckermanet al\.\(2015\)D\. Heckerman, D\. Geiger, and D\. M\. ChickeringLearning bayesian networks: the combination of knowledge and statistical data\.External Links:1302\.6815,[Link](https://arxiv.org/abs/1302.6815)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Heidbreder \(1937\)E\. HeidbrederLewin’s principles of topological psychology\.\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p3.1),[§3\.2](https://arxiv.org/html/2609.28942#S3.SS2.p2.1)\.
- Hendryckset al\.\(2020\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Critch, J\. Li, D\. Song, and J\. SteinhardtAligning ai with shared human values\.arXiv preprint arXiv:2008\.02275\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Jinet al\.\(2025\)X\. Jin, Z\. Li, Z\. Dai, H\. Liu, X\. Tang, C\. Luo, R\. Goutam, X\. Ren, and Q\. HeIn\-context personalized alignment with feedback history under counterfactual evaluation\.In2nd Workshop on Models of Human Feedback for AI Alignment,External Links:[Link](https://openreview.net/forum?id=k7CXnlckfS)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1)\.
- Kanget al\.\(2024\)Y\. Kang, J\. Wang, Y\. Li, F\. Zhong, X\. Feng, M\. Wang, W\. Tu, Q\. Wang, H\. Li, and Z\. ZhengCausal graph guided steering of llm values via prompts and sparse autoencoders\.arXiv preprint arXiv:2501\.00581\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1),[§4\.1](https://arxiv.org/html/2609.28942#S4.SS1.SSS0.Px3.p1.1)\.
- Kirket al\.\(2024\)H\. R\. Kirk, A\. Whitefield, P\. Rottger, A\. M\. Bean, K\. Margatina, R\. Mosquera\-Gomez, J\. Ciro, M\. Bartolo, A\. Williams, H\. He,et al\.The prism alignment dataset: what participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models\.Advances in Neural Information Processing Systems37,pp\. 105236–105344\.Cited by:[1st item](https://arxiv.org/html/2609.28942#A2.I3.i1.p1.1),[§B\.1](https://arxiv.org/html/2609.28942#A2.SS1.SSS0.Px1.p1.1),[§B\.1\.1](https://arxiv.org/html/2609.28942#A2.SS1.SSS1.p1.1),[§C\.5](https://arxiv.org/html/2609.28942#A3.SS5.p4.1),[§4\.1](https://arxiv.org/html/2609.28942#S4.SS1.SSS0.Px1.p1.1)\.
- Lauritzen and Spiegelhalter \(1990\)S\. L\. Lauritzen and D\. J\. SpiegelhalterLocal computations with probabilities on graphical structures and their application to expert systems\.Journal of the royal statistical society series b\-methodological50,pp\. 415–448\.External Links:[Link](https://api.semanticscholar.org/CorpusID:58792451)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Lazarus and Folkman \(1984\)R\. S\. Lazarus and S\. FolkmanStress, appraisal, and coping\.Springer publishing company\.Cited by:[§3\.2](https://arxiv.org/html/2609.28942#S3.SS2.p2.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2025a\)J\. Li, J\. Guan, S\. Wu, W\. Wu, and R\. YanFrom 1,000,000 users to every user: scaling up personalized preference for user\-level alignment\.arXiv preprint arXiv:2503\.15463\.Cited by:[§C\.2](https://arxiv.org/html/2609.28942#A3.SS2.SSS0.Px2.p1.1)\.
- Liet al\.\(2020\)K\. Li, T\. Zhang, and R\. WangDeep reinforcement learning for multiobjective optimization\.IEEE transactions on cybernetics51\(6\),pp\. 3103–3114\.Cited by:[§4\.1](https://arxiv.org/html/2609.28942#S4.SS1.SSS0.Px3.p1.1)\.
- Liet al\.\(2025b\)L\. Li, X\. Zhang, X\. Zhou, and Z\. LiuAutoMIR: effective zero\-shot medical information retrieval without relevance labels\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 24028–24047\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.1305/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.1305),ISBN 979\-8\-89176\-335\-7Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Liet al\.\(2024\)X\. Li, R\. Zhou, Z\. C\. Lipton, and L\. LeqiPersonalized language modeling from personalized human feedback\.arXiv preprint arXiv:2402\.05133\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2024\)O\. Liu, D\. Fu, D\. Yogatama, and W\. NeiswangerDellma: decision making under uncertainty with large language models\.arXiv preprint arXiv:2402\.02392\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Longjohnet al\.\(2025\)R\. Longjohn, S\. Wu, S\. Kher, C\. Belém, and P\. SmythBayesian evaluation of large language model behavior\.arXiv preprint arXiv:2511\.10661\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Menget al\.\(2024\)Y\. Meng, M\. Xia, and D\. ChenSimpo: simple preference optimization with a reference\-free reward\.Advances in Neural Information Processing Systems37,pp\. 124198–124235\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Mischel \(2013\)W\. MischelPersonality and assessment\.Psychology Press\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p3.1)\.
- Ouyanget al\.\(2022\)L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Pădureanet al\.\(2025\)V\. Pădurean, P\. Kamalaruban, N\. Kotalwar, A\. Gotovos, and A\. SinglaInference\-time personalized alignment with a few user preference queries\.arXiv preprint arXiv:2511\.02966\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p2.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p1.1)\.
- Poddaret al\.\(2024\)S\. Poddar, Y\. Wan, H\. Ivison, A\. Gupta, and N\. JaquesPersonalizing reinforcement learning from human feedback with variational preference learning\.Advances in Neural Information Processing Systems37,pp\. 52516–52544\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p2.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p1.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.Advances in Neural Information Processing Systems36,pp\. 53728–53741\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Ryanet al\.\(2025\)M\. J\. Ryan, O\. Shaikh, A\. Bhagirath, D\. Frees, W\. B\. Held, and D\. YangSynthesizeme\! inducing persona\-guided prompts for personalized reward models in llms\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 8045–8078\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p2.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1)\.
- Sakaguchiet al\.\(2021\)K\. Sakaguchi, R\. L\. Bras, C\. Bhagavatula, and Y\. ChoiWinogrande: an adversarial winograd schema challenge at scale\.Communications of the ACM64\(9\),pp\. 99–106\.Cited by:[§C\.10](https://arxiv.org/html/2609.28942#A3.SS10.p1.1)\.
- Schwartz \(2012\)S\. H\. SchwartzAn overview of the schwartz theory of basic values\.Online readings in Psychology and Culture2\(1\),pp\. 11\.Cited by:[§3\.1](https://arxiv.org/html/2609.28942#S3.SS1.p1.1)\.
- Seoet al\.\(2025\)W\. Seo, Z\. Yuan, and Y\. BuValuesRAG: enhancing cultural alignment through retrieval\-augmented contextual learning\.External Links:2501\.01031,[Link](https://arxiv.org/abs/2501.01031)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1),[§4\.1](https://arxiv.org/html/2609.28942#S4.SS1.SSS0.Px3.p1.1)\.
- Shenet al\.\(2023\)T\. Shen, R\. Jin, Y\. Huang, C\. Liu, W\. Dong, Z\. Guo, X\. Wu, Y\. Liu, and D\. XiongLarge language model alignment: a survey\.arXiv preprint arXiv:2309\.15025\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Shiet al\.\(2024\)R\. Shi, Y\. Chen, Y\. Hu, A\. Liu, H\. Hajishirzi, N\. A\. Smith, and S\. S\. DuDecoding\-time language model alignment with multiple objectives\.Advances in Neural Information Processing Systems37,pp\. 48875–48920\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p2.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1),[§4\.1](https://arxiv.org/html/2609.28942#S4.SS1.SSS0.Px3.p1.1)\.
- Thonetet al\.\(2025\)T\. Thonet, G\. Kruszewski, J\. Rozen, P\. Erbacher, and M\. DymetmanFaST: feature\-aware sampling and tuning for personalized preference alignment with limited data\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 9352–9381\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1)\.
- Touvronet al\.\(2023\)H\. Touvron, T\. Lavril, G\. Izacard, X\. Martinet, M\. Lachaux, T\. Lacroix, B\. Rozière, N\. Goyal, E\. Hambro, F\. Azhar,et al\.Llama: open and efficient foundation language models\.arXiv preprint arXiv:2302\.13971\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Tzikaset al\.\(2008\)D\. G\. Tzikas, A\. C\. Likas, and N\. P\. GalatsanosThe variational approximation for bayesian inference\.IEEE Signal Processing Magazine25\(6\),pp\. 131–146\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p4.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p2.1)\.
- Wanget al\.\(2024a\)J\. Wang, H\. Wang, S\. Sun, and W\. LiAligning language models with human preferences via a bayesian approach\.External Links:2310\.05782,[Link](https://arxiv.org/abs/2310.05782)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Wanget al\.\(2024b\)X\. Wang, S\. Duan, X\. Yi, J\. Yao, S\. Zhou, Z\. Wei, P\. Zhang, D\. Xu, M\. Sun, and X\. XieOn the essence and prospect: an investigation of alignment approaches for big models\.arXiv preprint arXiv:2403\.04204\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Wanget al\.\(2019\)X\. Wang, W\. Shi, R\. Kim, Y\. Oh, S\. Yang, J\. Zhang, and Z\. YuPersuasion for good: towards a personalized persuasive dialogue system for social good\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,A\. Korhonen, D\. Traum, and L\. Màrquez \(Eds\.\),Florence, Italy,pp\. 5635–5649\.External Links:[Link](https://aclanthology.org/P19-1566/),[Document](https://dx.doi.org/10.18653/v1/P19-1566)Cited by:[2nd item](https://arxiv.org/html/2609.28942#A2.I3.i2.p1.1),[§B\.1](https://arxiv.org/html/2609.28942#A2.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2609.28942#S4.SS1.SSS0.Px1.p1.1)\.
- Wanget al\.\(2023\)Y\. Wang, W\. Zhong, L\. Li, F\. Mi, X\. Zeng, W\. Huang, L\. Shang, X\. Jiang, and Q\. LiuAligning large language models with human: a survey\.arXiv preprint arXiv:2307\.12966\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Yanget al\.\(2024\)K\. Yang, Z\. Liu, Q\. Xie, J\. Huang, T\. Zhang, and S\. AnaniadouMetaaligner: towards generalizable multi\-objective alignment of language models\.arXiv preprint arXiv:2403\.17141\.Cited by:[§4\.1](https://arxiv.org/html/2609.28942#S4.SS1.SSS0.Px3.p1.1)\.
- Yaoet al\.\(2023a\)J\. Yao, X\. Yi, X\. Wang, Y\. Gong, and X\. XieValue fulcra: mapping large language models to the multidimensional spectrum of basic human values\.arXiv preprint arXiv:2311\.10766\.Cited by:[§3\.1](https://arxiv.org/html/2609.28942#S3.SS1.p1.1)\.
- Yaoet al\.\(2023b\)J\. Yao, X\. Yi, X\. Wang, J\. Wang, and X\. XieFrom instructions to intrinsic human values–a survey of alignment goals for big models\.arXiv preprint arXiv:2308\.12014\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Yonget al\.\(2025a\)X\. Yong, J\. Lian, X\. Yi, X\. Zhou, and X\. XieMotiveBench: how far are we from human\-like motivational reasoning in large language models?\.InFindings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 \- August 1, 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Findings of ACL, Vol\.ACL 2025,pp\. 20059–20089\.External Links:[Link](https://doi.org/10.18653/v1/2025.findings-acl.1029),[Document](https://dx.doi.org/10.18653/V1/2025.FINDINGS-ACL.1029)Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Yonget al\.\(2026\)X\. Yong, P\. Sun, Z\. Wang, and X\. ZhouIntelli\-planner: towards customized urban planning via large language model empowered reinforcement learning\.InProceedings of the ACM Web Conference 2026, WWW 2026, Dubai, United Arab Emirates, originally scheduled for April 13\-17, 2026, rescheduled for June 29 \- July 3, 2026,H\. Hacid, Y\. Maarek, F\. Bonchi, I\. Guy, and E\. Yilmaz \(Eds\.\),pp\. 9385–9396\.External Links:[Link](https://doi.org/10.1145/3774904.3793019),[Document](https://dx.doi.org/10.1145/3774904.3793019)Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Yonget al\.\(2025b\)X\. Yong, X\. Zhou, Y\. Zhang, J\. Li, Y\. Zheng, and X\. WuThink or not? exploring thinking efficiency in large reasoning models via an information\-theoretic lens\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2025/hash/04185b5ae2d450ef39bd53c0ec4802cb-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Zellerset al\.\(2019\)R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. ChoiHellaswag: can a machine really finish your sentence?\.InProceedings of the 57th annual meeting of the association for computational linguistics,pp\. 4791–4800\.Cited by:[§C\.10](https://arxiv.org/html/2609.28942#A3.SS10.p1.1)\.
- Zhang and Poole \(1996\)N\. L\. Zhang and D\. PooleExploiting causal independence in bayesian network inference\.External Links:cs/9612101,[Link](https://arxiv.org/abs/cs/9612101)Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2023\)S\. Zhang, L\. Dong, X\. Li, S\. Zhang, X\. Sun, S\. Wang, J\. Li, R\. Hu, T\. Zhang, G\. Wang,et al\.Instruction tuning for large language models: a survey\.ACM Computing Surveys\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2026a\)X\. Zhang, L\. Li, X\. Zhou, and Z\. LiuR2MED: a benchmark for reasoning\-driven medical retrieval\.External Links:2505\.14558,[Link](https://arxiv.org/abs/2505.14558)Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Zhanget al\.\(2026b\)X\. Zhang, L\. Li, Y\. Zhou, X\. Zhou, Y\. Zhang, and X\. WuInflated excellence or true performance? rethinking medical diagnostic benchmarks with dynamic evaluation\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 26454–26493\.External Links:[Link](https://aclanthology.org/2026.acl-long.1218/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.1218),ISBN 979\-8\-89176\-390\-6Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Zhanget al\.\(2026c\)X\. Zhang, J\. Wang, Q\. Zhao, H\. Guo, L\. Li, J\. Yao, X\. Zhou, X\. Yi, and X\. XieHuman values matter: investigating how misalignment shapes collective behaviors in llm agent communities\.External Links:2604\.05339,[Link](https://arxiv.org/abs/2604.05339)Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Zhanget al\.\(2026d\)X\. Zhang, X\. Zhou, H\. Xu, and J\. LianHypeMed: enhancing medication recommendations with hypergraph\-based patient relationships\.ACM Trans\. Inf\. Syst\.44\(4\)\.External Links:ISSN 1046\-8188,[Link](https://doi.org/10.1145/3803851),[Document](https://dx.doi.org/10.1145/3803851)Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Zhanget al\.\(2025a\)X\. Zhang, R\. Chen, Y\. Feng, and Z\. LiuPersona\-judge: personalized alignment of large language models via token\-level self\-judgment\.arXiv preprint arXiv:2504\.12663\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p2.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1)\.
- Zhanget al\.\(2025b\)Y\. Zhang, D\. Adila, C\. Shin, and F\. SalaPersonalize your llm: fake it then align it\.arXiv preprint arXiv:2503\.01048\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p2.1)\.
- Zhaoet al\.\(2024\)P\. Zhao, H\. Zhang, Q\. Yu, Z\. Wang, Y\. Geng, F\. Fu, L\. Yang, W\. Zhang, J\. Jiang, and B\. CuiRetrieval\-augmented generation for ai\-generated content: a survey\.arXiv preprint arXiv:2402\.19473\.Cited by:[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2025\)W\. Zhao, X\. Sui, Y\. Hu, J\. Guo, H\. Liu, B\. Li, Y\. Zhao, B\. Qin, and T\. LiuTeaching language models to evolve with users: dynamic profile modeling for personalized alignment\.arXiv preprint arXiv:2505\.15456\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p2.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p1.1)\.
- Zhouet al\.\(2025\)X\. Zhou, Z\. Zhao, and H\. GuoTricolore: multi\-behavior user profiling for enhanced candidate generation in recommender systems\.IEEE Transactions on Knowledge and Data Engineering37\(7\),pp\. 4349–4360\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.
- Zhuet al\.\(2024\)M\. Zhu, Y\. Weng, L\. Yang, and Y\. ZhangPersonality alignment of large language models\.arXiv preprint arXiv:2408\.11779\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p2.1),[§2](https://arxiv.org/html/2609.28942#S2.SS0.SSS0.Px2.p2.1)\.
- Zhuet al\.\(2026\)Y\. Zhu, S\. Duan, X\. Zhang, J\. Sang, P\. Zhang, T\. Lu, X\. Zhou, J\. Yao, X\. Yi, and X\. XieMoHoBench: assessing honesty of multimodal large language models via unanswerable visual questions\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 29205–29213\.Cited by:[§1](https://arxiv.org/html/2609.28942#S1.p1.1)\.

## Appendix AMethod Details

### A\.1Generic ELBO for Personalized Value Inference

##### Setup

For a useruuand a question \(scenario\)xx, let𝐯u\\mathbf\{v\}\_\{u\}denote the user\-level value profile, and let𝐯u,x\\mathbf\{v\}\_\{u,x\}denote the latent question\-specific personalized values\. Given a preference triple\(x,yc,yr\)\(x,y\_\{c\},y\_\{r\}\), we denote the observed preference event as

o:=\(yc≻yr\),o:=\(y\_\{c\}\\succ y\_\{r\}\),\(12\)i\.e\., responseycy\_\{c\}is preferred overyry\_\{r\}under scenarioxx\.

##### Bayesian formulation

We define a prior conditioned on both the user and scenario:

pθ​\(𝐯u,x∣𝐯u,x\),p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\}\\mid\\mathbf\{v\}\_\{u\},x\),\(13\)and a preference likelihood

pθ​\(o∣𝐯u,x,x\)=pθ​\(yc≻yr∣𝐯u,x,x\)\.p\_\{\\theta\}\(o\\mid\\mathbf\{v\}\_\{u,x\},x\)\\;\\;=\\;\\;p\_\{\\theta\}\(y\_\{c\}\\succ y\_\{r\}\\mid\\mathbf\{v\}\_\{u,x\},x\)\.\(14\)By Bayes’ rule, the posterior is

pθ​\(𝐯u,x∣𝐯u,x,o\)=pθ​\(o∣𝐯u,x\)​pθ​\(𝐯u,x∣𝐯u,x\)pθ​\(o\),\\displaystyle p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\}\\mid\\mathbf\{v\}\_\{u\},x,o\)=\\frac\{p\_\{\\theta\}\(o\\mid\\mathbf\{v\}\_\{u,x\}\)\\,p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\}\\mid\\mathbf\{v\}\_\{u\},x\)\}\{p\_\{\\theta\}\(o\)\},\(15\)which matches Eq\. \([1](https://arxiv.org/html/2609.28942#S3.E1)\) in the main paper\.

##### Marginal likelihood

The likelihoodo:=\(yc≻yr\)o:=\(y\_\{c\}\\succ y\_\{r\}\)marginalizes over the latent personalized values𝐯u,x\\mathbf\{v\}\_\{u,x\}:

pθ​\(o\)\\displaystyle p\_\{\\theta\}\(o\)=∫pθ​\(o∣𝐯u,x,x\)\\displaystyle=\\int p\_\{\\theta\}\(o\\mid\\mathbf\{v\}\_\{u,x\},x\)\(16\)×\\displaystyle\\timespθ​\(𝐯u,x∣𝐯u,x\)​d​𝐯u,x\.\\displaystyle p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\}\\mid\\mathbf\{v\}\_\{u\},x\)\\,d\\mathbf\{v\}\_\{u,x\}\.\(17\)
In our setting, computing Eq\. \([16](https://arxiv.org/html/2609.28942#A1.E16)\) is intractable\. Since𝐯u,x\\mathbf\{v\}\_\{u,x\}is a multi\-dimensional value vector and each dimension corresponds to a 5\-point Likert score, exact marginalization would require an exponential\-time summation and is prohibitively expensive\. This motivates us to employ variational inference to obtain a tractable lower bound for optimization\.

##### Variational posterior and ELBO\.

We introduce a variational posteriorqϕ​\(𝐯u,x∣𝐯u,x\)q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\\mid\\mathbf\{v\}\_\{u\},x\)to approximate the true posteriorpθ​\(𝐯u,x∣𝐯u,𝐱,o\)p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\}\\mid\\mathbf\{v\}\_\{u\},\{\\mathbf\{x\}\},o\)\. Thus, we solve the optimization problem as:

ϕ^\\displaystyle\\hat\{\\phi\}=arg​minϕ⁡KL​\(qϕ​\(𝐯u,x∣𝐯u,x\)CLOSE\\displaystyle=\\operatorname\*\{arg\\,min\}\_\{\\phi\}\\;\\mathrm\{KL\}\\Bigl\(q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\\mid\\mathbf\{v\}\_\{u\},x\)\(18\)∥pθ\(𝐯u,x∣𝐯u,x,o\)\)\.\\displaystyle\\Big\\\|\\;p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\}\\mid\\mathbf\{v\}\_\{u\},x,o\)\\Bigr\)\.
In the following derivation, let𝐳:=𝐯u,x\\mathbf\{z\}:=\\mathbf\{v\}\_\{u,x\}and use the abbreviations

qϕ​\(𝐳\)\\displaystyle q\_\{\\phi\}\(\\mathbf\{z\}\):=qϕ​\(𝐳∣𝐯u,x\),\\displaystyle:=q\_\{\\phi\}\(\\mathbf\{z\}\\mid\\mathbf\{v\}\_\{u\},x\),pθ​\(𝐳\)\\displaystyle p\_\{\\theta\}\(\\mathbf\{z\}\):=pθ​\(𝐳∣𝐯u,x\)\.\\displaystyle:=p\_\{\\theta\}\(\\mathbf\{z\}\\mid\\mathbf\{v\}\_\{u\},x\)\.For brevity, we suppress the conditioning on\(𝐯u,x\)\(\\mathbf\{v\}\_\{u\},x\)below\.

KL\(qϕ\(𝐳\)∥pθ\(𝐳∣o\)\)\\displaystyle\\mathrm\{KL\}\\Bigl\(q\_\{\\phi\}\(\\mathbf\{z\}\)\\,\\Big\\\|\\,p\_\{\\theta\}\(\\mathbf\{z\}\\mid o\)\\Bigr\)=∫qϕ​\(𝐳\)​log⁡qϕ​\(𝐳\)pθ​\(𝐳∣o\)​𝑑𝐳\\displaystyle=\\int q\_\{\\phi\}\(\\mathbf\{z\}\)\\log\\frac\{q\_\{\\phi\}\(\\mathbf\{z\}\)\}\{p\_\{\\theta\}\(\\mathbf\{z\}\\mid o\)\}\\,d\\mathbf\{z\}=∫qϕ​\(𝐳\)​\[log⁡qϕ​\(𝐳\)−log⁡pθ​\(𝐳∣o\)\]​𝑑𝐳\\displaystyle=\\int q\_\{\\phi\}\(\\mathbf\{z\}\)\\Bigl\[\\log q\_\{\\phi\}\(\\mathbf\{z\}\)\-\\log p\_\{\\theta\}\(\\mathbf\{z\}\\mid o\)\\Bigr\]\\,d\\mathbf\{z\}=∫qϕ​\(𝐳\)​\[log⁡qϕ​\(𝐳\)−log⁡pθ​\(𝐳,o\)\]​𝑑𝐳\\displaystyle=\\int q\_\{\\phi\}\(\\mathbf\{z\}\)\\Bigl\[\\log q\_\{\\phi\}\(\\mathbf\{z\}\)\-\\log p\_\{\\theta\}\(\\mathbf\{z\},o\)\\Bigr\]\\,d\\mathbf\{z\}\+log⁡pθ​\(o\)\\displaystyle\\quad\+\\log p\_\{\\theta\}\(o\)=log⁡pθ​\(o\)\\displaystyle=\\log p\_\{\\theta\}\(o\)−∫qϕ\(𝐳\)\[logpθ\(𝐳,o\)−logqϕ\(𝐳\)\]d𝐳\.\\displaystyle\\quad\-\\int q\_\{\\phi\}\(\\mathbf\{z\}\)\\Bigl\[\\log p\_\{\\theta\}\(\\mathbf\{z\},o\)\-\\log q\_\{\\phi\}\(\\mathbf\{z\}\)\\Bigr\]\\,d\\mathbf\{z\}\.\(19\)
We useℒ⁡\(qϕ\)\\mathcal\{L\}\(q\_\{\\phi\}\)to represent the following term as:

ℒ⁡\(qϕ\)\\displaystyle\\mathcal\{L\}\(q\_\{\\phi\}\)=∫qϕ\[logpθ\(𝐯u,x,o\)\\displaystyle=\\int q\_\{\\phi\}\\bigl\[\\log p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\},o\)−logqϕ\(𝐯u,x\)\]d𝐯u,x\\displaystyle\\qquad\-\\log q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\)\\bigr\]\\,d\\mathbf\{v\}\_\{u,x\}=𝔼qϕ\[logpθ\(𝐯u,x,o\)\\displaystyle=\\mathbb\{E\}\_\{q\_\{\\phi\}\}\\bigl\[\\log p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\},o\)−logqϕ\(𝐯u,x\)\]\.\\displaystyle\\qquad\-\\log q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\)\\bigr\]\.\(20\)
Based on Eq\.\([19](https://arxiv.org/html/2609.28942#A1.E19)\) and Eq\.\([20](https://arxiv.org/html/2609.28942#A1.E20)\), we then obtain

KL\(qϕ\(𝐯u,x\)\|\|pθ\(𝐯u,x∣o\)\)=logpθ\(o\)−ℒ\(qϕ\)\\displaystyle\\text\{KL\}\(q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\)\|\|p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\}\\mid o\)\)=\\log p\_\{\\theta\}\(o\)\-\\mathcal\{L\}\(q\_\{\\phi\}\)\(21\)logpθ\(o\)=ℒ\(qϕ\)\+KL\(qϕ\(𝐯u,x\)\|\|pθ\(𝐯u,x∣o\)\)\.\\displaystyle\\log p\_\{\\theta\}\(o\)=\\mathcal\{L\}\(q\_\{\\phi\}\)\+\\text\{KL\}\(q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\)\|\|p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\}\\mid o\)\)\.\(22\)
Sincelog⁡pθ​\(o\)\\log p\_\{\\theta\}\(o\)is fixed on the observed data, minimizing the KL divergence in Eq\.\([19](https://arxiv.org/html/2609.28942#A1.E19)\) is equivalent to maximizingℒ⁡\(qϕ\)\\mathcal\{L\}\(q\_\{\\phi\}\), which is the evidence lower bound \(ELBO\) oflog⁡pθ​\(o\)\\log p\_\{\\theta\}\(o\)\.

ℒ⁡\(qϕ\)\\displaystyle\\mathcal\{L\}\(q\_\{\\phi\}\)=𝔼qϕ\[logpθ\(𝐯u,x,o\)\\displaystyle=\\mathbb\{E\}\_\{q\_\{\\phi\}\}\\bigl\[\\log p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\},o\)−logqϕ\(𝐯u,x\)\]\\displaystyle\\qquad\-\\log q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\)\\bigr\]=𝔼qϕ\[logpθ\(o\|vu,x\)\+logpθ\(vu,x\)\\displaystyle=\\mathbb\{E\}\_\{q\_\{\\phi\}\}\\bigl\[\\log p\_\{\\theta\}\(o\|v\_\{u,x\}\)\+\\log p\_\{\\theta\}\(v\_\{u,x\}\)−logqϕ\(𝐯u,x\)\]\\displaystyle\\qquad\-\\log q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\)\\bigr\]=𝔼qϕ​\[log⁡pθ​\(o\|vu,x\)\]\\displaystyle=\\mathbb\{E\}\_\{q\_\{\\phi\}\}\\bigl\[\\log p\_\{\\theta\}\(o\|v\_\{u,x\}\)\\bigr\]−𝔼qϕ​\[log⁡qϕ​\(𝐯u,x\)−log⁡pθ​\(vu,x\)\]\\displaystyle\\qquad\-\\mathbb\{E\}\_\{q\_\{\\phi\}\}\\bigl\[\\log q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\)\-\\log p\_\{\\theta\}\(v\_\{u,x\}\)\\bigr\]=𝔼qϕ​\[log⁡pθ​\(o\|vu,x\)\]\\displaystyle=\\mathbb\{E\}\_\{q\_\{\\phi\}\}\\bigl\[\\log p\_\{\\theta\}\(o\|v\_\{u,x\}\)\\bigr\]−KL\(qϕ\(𝐯u,x\)\|\|pθ\(vu,x\)\)\.\\displaystyle\\qquad\-\\text\{KL\}\\bigl\(q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\)\|\|p\_\{\\theta\}\(v\_\{u,x\}\)\\bigr\)\.\(23\)Converting to the training loss, we need to minimize the lossℒ\\mathcal\{L\}as follows:

ℒ\\displaystyle\\mathcal\{L\}=−𝔼qϕ​\[log⁡pθ​\(o\|vu,x\)\]\\displaystyle=\-\\mathbb\{E\}\_\{q\_\{\\phi\}\}\\bigl\[\\log p\_\{\\theta\}\(o\|v\_\{u,x\}\)\\bigr\]\+KL\(qϕ\(𝐯u,x\)\|\|pθ\(vu,x\)\),\\displaystyle\\qquad\+\\text\{KL\}\\bigl\(q\_\{\\phi\}\(\\mathbf\{v\}\_\{u,x\}\)\|\|p\_\{\\theta\}\(v\_\{u,x\}\)\\bigr\),\(24\)which is Eq\.\([2](https://arxiv.org/html/2609.28942#S3.E2)\) in the main paper\.

### A\.2Concrete Formulation

##### Setup\.

For a useruuand a questionxx, let𝐯u\\mathbf\{v\}\_\{u\}be the user\-level value profile \(Sec\.[3\.1](https://arxiv.org/html/2609.28942#S3.SS1)\),𝐯x\\mathbf\{v\}\_\{x\}be the scenario value\-salience variable estimated from\(x,yhuman\)\(x,y\_\{\\text\{human\}\}\)\(Sec\.[3\.3](https://arxiv.org/html/2609.28942#S3.SS3)\), and𝐯u,x\\mathbf\{v\}\_\{u,x\}be the question\-specific personalized values\. Given a preference triple\(x,yc,yr\)\(x,y\_\{c\},y\_\{r\}\), we extract the value vector reflected by both the preferred and dispreferred responses asvyc,vyrv\_\{y\_\{c\}\},v\_\{y\_\{r\}\}\.

##### Generative model \(hierarchical prior\)\.

We use the estimated value salience distributionpω​\(𝐯x∣x,yhuman\)p\_\{\\omega\}\(\\mathbf\{v\}\_\{x\}\\mid x,y\_\{\\text\{human\}\}\)and a user\-conditioned priorpθ​\(𝐯u,x∣𝐯u,𝐯x\)p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\}\\mid\\mathbf\{v\}\_\{u\},\\mathbf\{v\}\_\{x\}\)\. We further introduce a \(single\-point\) observation model for the extracted value embedding:

pθ​\(𝐯yc∣𝐯u,x\)\.p\_\{\\theta\}\(\\mathbf\{v\}\_\{y\_\{c\}\}\\mid\\mathbf\{v\}\_\{u,x\}\)\.\(25\)The resulting marginal likelihood is

log⁡p⁡\(𝐯yc∣𝐯u,x\)\\displaystyle\\log p\(\\mathbf\{v\}\_\{y\_\{c\}\}\\mid\\mathbf\{v\}\_\{u\},x\)=log∑𝐯x∑𝐯u,xpθ\(𝐯yc∣𝐯u,x\)\\displaystyle=\\log\\sum\_\{\\mathbf\{v\}\_\{x\}\}\\sum\_\{\\mathbf\{v\}\_\{u,x\}\}p\_\{\\theta\}\(\\mathbf\{v\}\_\{y\_\{c\}\}\\mid\\mathbf\{v\}\_\{u,x\}\)\(26\)⋅pθ​\(𝐯u,x∣𝐯u,𝐯x\)\\displaystyle\\cdot p\_\{\\theta\}\(\\mathbf\{v\}\_\{u,x\}\\mid\\mathbf\{v\}\_\{u\},\\mathbf\{v\}\_\{x\}\)⋅pω​\(𝐯x∣x,yhuman\)\.\\displaystyle\\cdot p\_\{\\omega\}\(\\mathbf\{v\}\_\{x\}\\mid x,y\_\{\\text\{human\}\}\)\.

### A\.3Prompt Template

We present below the prompts used in this paper, including the data preprocessing prompt, the method prompt, and the evaluation prompt\.

#### A\.3\.1Preprocessing Prompt

This stage involves four preprocessing prompts: \(1\) extracting the user value profile in Fig\.[8](https://arxiv.org/html/2609.28942#A1.F8), \(2\) determining whether a given question involves value\-related considerations in Fig\.[9](https://arxiv.org/html/2609.28942#A1.F9), \(3\) identifying which values are implicated by the question in Fig\.[10](https://arxiv.org/html/2609.28942#A1.F10), and \(4\) extracting value\-related representations from candidate answers in Fig\.[11](https://arxiv.org/html/2609.28942#A1.F11)\.

Prompt: Schwartz Basic Human Values Evaluation on Personal Profile\.Background:
You are an AI assistant tasked with evaluating how strongly Schwartz’s basic human values are reflected in a given personal description\. The input text is a self\-description written by a person, and your goal is to infer how strongly each of the ten Schwartz values appears in what the person expresses, implies, or emphasizes\.Schwartz’s Ten Basic Values:Self\-direction — independent thought and action; choosing, creating, exploringStimulation — excitement, novelty, and challenge in lifeHedonism — pleasure or sensuous gratification for oneselfAchievement — personal success through demonstrating competence according to social standardsPower — social status and prestige, control or dominance over people and resourcesSecurity — safety, harmony, and stability of society, relationships, and of selfConformity — restraint of actions, inclinations, and impulses that may upset or harm others or violate social expectationsTradition — respect, commitment, and acceptance of cultural or religious customs and ideasBenevolence — preserving and enhancing the welfare of people with whom one is in frequent personal contactUniversalism — understanding, appreciation, tolerance, and protection for the welfare of all people and for natureScoring Scale \(1–5\):
1 \(Contradicted\) — The text denies, criticizes, or opposes this value\.
2 \(Absent\) — The value is not mentioned, implied, or relevant in any way\.
3 \(Mentioned but Not Important\) — The value is referenced or implied, but it is not meaningful to the person’s identity or motivations\.
4 \(Present but Not Central\) — The value seems to matter to the person and influences their perspective, but it is not central\.
5 \(Most Important\) — The value is central to the person’s identity or worldview\. Removing it would significantly change their self\-description\.
Task:
Evaluate the following personal self\-description and rate how strongly each of the ten Schwartz values is reflected in it\.
Input Text:
"\{personal\_text\}Output Format \(STRICT\):
Rate all ten values using only integers 1–5\. Output ONLY in the following format:```
Self-direction: [score]
Stimulation: [score]
Hedonism: [score]
Achievement: [score]
Power: [score]
Security: [score]
Conformity: [score]
Tradition: [score]
Benevolence: [score]
Universalism: [score]
```

Figure 8:Schwartz basic human values evaluation on personal profile prompt\.Prompt: Schwartz Value Candidate Identification\.Background:
You are an AI assistant that identifies which Schwartz basic human values are relevant to a given text\. Your task is to determine which values are implied, expressed, or thematically connected to the provided text\_type, and output them in a structured JSON format\.Schwartz’s Ten Basic Values:Self\-direction — independent thought and action; choosing, creating, exploringStimulation — excitement, novelty, and challenge in lifeHedonism — pleasure or sensuous gratification for oneselfAchievement — personal success through demonstrating competence according to social standardsPower — social status and prestige, control or dominance over people and resourcesSecurity — safety, harmony, and stability of society, relationships, and of selfConformity — restraint of actions, inclinations, and impulses that may upset or harm others or violate social expectationsTradition — respect, commitment, and acceptance of cultural or religious customs and ideasBenevolence — preserving and enhancing the welfare of people with whom one is in frequent personal contactUniversalism — understanding, appreciation, tolerance, and protection for the welfare of all people and for natureIdentify up to 5 relevant Schwartz values for the text\_type\.
Use only these keys:```
[’self_direction’, ’stimulation’, ’hedonism’, ’achievement’, ’power’,
 ’security’, ’conformity’, ’tradition’, ’benevolence’, ’universalism’]
```

For each selected value, provide a relevance score between 0 and 1\.Output Format \(STRICT JSON ONLY\):
Return only valid JSON in exactly the following structure:```
{"candidates":[
  {"value":"<one_of_keys>","relevance":0.0-1.0}
]}
```

Figure 9:Prompt for Schwartz value candidate identification of question\.Prompt: Prompt: Value\-Related Question Identification\.Background:
You are an expert in Schwartz’s Theory of Basic Human Values\. Your task is to determine whether a given question or statement is related to human values\. A question is value\-related if it reflects moral priorities, ethical considerations, social judgments, or underlying motivational principles connected to Schwartz’s ten basic values\.Schwartz’s Ten Basic Human Values:"\{value definition\}A question is considered value\-related if:•It asks about moral or ethical choices\.•It involves trade\-offs between different priorities \(e\.g\., personal gain vs\. helping others\)\.•It touches on topics such as fairness, freedom, security, achievement, pleasure, tradition, caring for others, or environmental protection\.•It asks opinions about social, political, or personal matters that reflect underlying values\.•It involves decision\-making influenced by value priorities\.A question is NOT value\-related if:•It is purely factual\.•It is a technical or procedural question\.•It asks for objective information without value judgment\.Input Question:
"\{question\}Output Requirement:
Respond with ONLY:True— if the question is value\-related\.False— if the question is not value\-related\.Figure 10:Prompt for value\-related question identification\.Prompt: Answer Values Evaluation\.Background:
You are an AI assistant tasked with evaluating how strongly Schwartz’s basic human values are reflected in a given personal description\. The input text is a self\-description written by a person, and your goal is to infer how strongly each of the ten Schwartz values appears in what the person expresses, implies, or emphasizes\.Schwartz’s Ten Basic Values:"\{value definition\}Scoring Scale \(1–5\):
1 \(Contradicted\) — The text denies, criticizes, or opposes this value\.
2 \(Absent\) — The value is not mentioned, implied, or relevant in any way\.
3 \(Mentioned but Not Important\) — The value is referenced or implied, but it is not meaningful to the person’s identity or motivations\.
4 \(Present but Not Central\) — The value seems to matter to the person and influences their perspective, but it is not central\.
5 \(Most Important\) — The value is central to the person’s identity or worldview\. Removing it would significantly change their self\-description\.
Task:
Evaluate the following personal self\-description and rate how strongly each of the ten Schwartz values is reflected in it\.Input Text:
"\{text\}Output Format \(STRICT\):
Rate all ten values using only integers 1–5\. Output ONLY in the following format:```
Self-direction: [score]
Stimulation: [score]
Hedonism: [score]
Achievement: [score]
Power: [score]
Security: [score]
Conformity: [score]
Tradition: [score]
Benevolence: [score]
Universalism: [score]
```

Figure 11:Prompt for answer value evaluation\.
#### A\.3\.2Methods Prompt

##### View\-Specific Residual Inference Prompts

To instantiate the residual distributionsq⁡\(𝐳𝐮\)q\(\{\\mathbf\{z\}\}\_\{\{\\mathbf\{u\}\}\}\)andq⁡\(𝐳𝐱\)q\(\{\\mathbf\{z\}\}\_\{\{\\mathbf\{x\}\}\}\)in Eq\. \(6\) and Eq\. \(8\), we implicitly estimate the residual corrections conditioned on the triplet\(x,𝐯x,𝐯ui\)\(x,\{\\mathbf\{v\}\}\_\{x\},\{\\mathbf\{v\}\}\_\{u\_\{i\}\}\)\. Rather than explicitly sampling𝐳\{\\mathbf\{z\}\}, we adopt a prompt\-based inference strategy in Fig\.[12](https://arxiv.org/html/2609.28942#A1.F12)to enable the model to capture how contextual semantics modulate value deviations\.

Specifically, we first construct a structured prompt that instructs the language model to analyze the input questionxxfrom the perspective of Schwartz’s ten basic human values\. This prompt is designed to emphasize value salience under the given scenario, allowing the model to encode context\-sensitive value implications\. The resulting hidden representation is extracted as a contextual embedding𝐞x\{\\mathbf\{e\}\}\_\{x\}, which serves as a semantic anchor for residual estimation\.

Conditioned on𝐞x\{\\mathbf\{e\}\}\_\{x\}and the corresponding prior \(either𝐯x\{\\mathbf\{v\}\}\_\{x\}for the scenario\-anchored view or𝐯ui\{\\mathbf\{v\}\}\_\{u\_\{i\}\}for the person\-anchored view\), a lightweight projection network produces a logit\-level correction that implicitly instantiates the residual variable𝐳\{\\mathbf\{z\}\}\. This correction is defined in the same value space as the prior distribution and is subsequently used to refine the posterior value inference\. In this way, the effect ofxxis captured through semantic prompting rather than explicit stochastic modeling\.

##### Gating Prompt for View Composition

To compute the adaptive fusion weightα\\alphain Eq\. \(10\), we employ a prompt\-based gating mechanism\. Given the input questionxxand the predictions from both views, we construct a concise comparison prompt in Figure[13](https://arxiv.org/html/2609.28942#A1.F13)that presents the value\-wise predictions and their associated confidence signals\. The prompt instructs the language model to decide which view provides a more reliable assessment under the current context\.

The gating model outputs logits corresponding to two discrete choices \(scenario\-anchored vs\. person\-anchored\)\. We extract the logits associated with tokens0and1, and normalize them to obtain the scalar gating coefficientα∈\(0,1\)\\alpha\\in\(0,1\)\. This value is then used to interpolate between the two inferred posteriors, enabling context\-aware view composition without introducing additional heuristic rules\.

Prompt: Question Information Extract for Latent Variable\.Instruction:Analyze the following question from the perspective of Schwartz’s Theory of Basic Human Values\.Value Dimensions:Consider the 10 value dimensions: Self\-Direction, Stimulation, Hedonism, Achievement, Power, Security, Conformity, Tradition, Benevolence, and Universalism\.Question:"\{question\}"Figure 12:Prompt for Schwartz value\-dimension analysis of question\.Prompt: Gate Function\.Background:You are a value alignment meta\-evaluator\. Your goal is to compare two prediction streams and decide which one gives a more rational and reliable value\-based assessment for a given question\. The two streams are:Stream 0 \(Universal\):Reflects common societal values and general logic\.Stream 1 \(Personal\):Reflects specific user traits and historical profile\.You will be given a comparison matrix evaluating both streams along key value dimensions\.Task:Choose which prediction stream \(Universal vs\. Personal\) provides a*more rational*assessment for the primary value dimensions of the question\. You must output either0\(for Universal\) or1\(for Personal\)\.Reliability Tags:\[Reliable\]: High confidence in the stream’s prediction quality and stability\.\[Uncertain\]: Low confidence, noisy or unstable prediction\.Input:Question:"question"Comparison Matrix \(Scale 1–5\):```
Dimension    Stream 0 (Universal)
Stream 1 (Personal)
{comparison_table}
```

Scale Meaning \(1–5\):1 \(Contradicted\): The value is strongly violated or opposed\. 2 \(Absent\): The value is missing or ignored\. 3 \(Mentioned but Not Important\): The value appears but is not emphasized\. 4 \(Present but Not Central\): The value is clearly present but not dominant\. 5 \(Most Important\): The value is central and primary in reasoning\.Analytical Guidance:Identify which value dimensions are*primary*for evaluating this question \(e\.g\., fairness, harm, autonomy, etc\.\)\.For each primary dimension, compare the scores of Stream 0 vs\. Stream 1\.Consider: \- Does Universal reasoning better match broad, widely accepted norms? \- Does Personal reasoning risk overfitting to idiosyncratic user traits? \- Are there dimensions where one stream is clearly unreliable \(very low scores\)?Integrate across all primary dimensions to decide which stream gives the overall more rational and trustworthy value assessment\.Output Format \(STRICT\):You must output*only*a single number with no explanation:•Output0if Stream 0 \(Universal\) provides the more rational assessment\.•Output1if Stream 1 \(Personal\) provides the more rational assessment\.Final Question:Which stream provides a more rational assessment for the primary value dimensions of this question?Answer:```
0
```

\(or\)```
1
```

Figure 13:Prompt for gate function\.

#### A\.3\.3Evaluation Prompt

The prompt in Fig\.[11](https://arxiv.org/html/2609.28942#A1.F11)is also used to evaluate responses generated by some baseline models, as described in Appendix[B\.2](https://arxiv.org/html/2609.28942#A2.SS2)\.

## Appendix BAdditional Experimental Details

### B\.1Datasets and Preprocessing

##### PRISM\.

We use the PRISM dataset\([Kirk et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib62)\)to evaluate value\-conditioned personalization and value alignment in open\-ended user–model interactions\. Each instance is organized turn\-by\-turn, containing \(i\) a user query and its conversation context, \(ii\) a set of candidate responses provided by the dataset \(including the chosen response and up to three unselected responses\), and \(iii\) user\-side survey metadata \(e\.g\.,self\_description\) when available\. In our formulation,xxdenotes the full user query context \(the user message concatenated with theopening\_promptfor the first turn\), andvuv\_\{u\}denotes the user value profile derived from survey text \(details below\)\.

##### GOOD \(P4G\-derived\)\.

Our GOOD dataset is constructed from the PersuasionForGood \(P4G\) corpus\([Wang et al\., 2019](https://arxiv.org/html/2609.28942#bib.bib68)\), a persuasion dialogue benchmark collected via Amazon Mechanical Turk\. Each dialogue involves a*Persuader*and a*Persuadee*, where the Persuader aims to persuade the Persuadee to donate to a charity \(e\.g\., Save the Children\)\. The original corpus contains 1,017 complete dialogues and rich participant metadata including Schwartz values \(10 dimensions\), Big\-Five personality traits, and Moral Foundations\.

#### B\.1\.1Value Extraction and Alignment Scoring

To quantify the alignment between user values and model responses on PRISM\([Kirk et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib62)\), we develop a pipeline that produces value scores for \(i\) the user profilevuv\_\{u\}and \(ii\) model responsesvycv\_\{y\_\{c\}\}\.

##### Selection and relevance filtering\.

We focus on thevalues guidedsubset, where models are explicitly prompted to engage with value\-laden topics\. To ensure value salience and data quality, we first retain only conversations with high value relevance, specifically those withchoice\_attributes\.valuesscores of at least 80\. We then perform turn\-level processing, handling each conversation sequentially\. Finally, we keep only users who provide survey text inself\_description, which enables extraction of the user value profilevuv\_\{u\}\.

##### LLM\-based value extraction\.

We use an LLM as an evaluator to quantify the 10 basic human values \(Self\-direction, Stimulation, Hedonism, Achievement, Power, Security, Conformity, Tradition, Benevolence, Universalism\)\. For each turn:

1. 1\.Relevance classification:classify whether the query is value\-related; discard non\-relevant turns\.
2. 2\.Candidate dimension identification:select up to 5 value dimensions most pertinent to the query\.
3. 3\.Value scoring \(user vs\. model\): - •User profile \(vuv\_\{u\}\):score each dimension \(1–5\) fromself\_description\. - •Chosen answer \(vycv\_\{y\_\{c\}\}\):score the model’s chosen response across all 10 dimensions\. - •Unselected answers:score up to 3 unselected candidate responses to serve as comparison baselines\.

#### B\.1\.2Evaluation Metrics

We compute three metrics\. Since the GOOD dataset does not contain negative samples, accuracy is only reported for PRISM\.

- •MAE \(value intensity mismatch\):for each candidate dimensiondd,MAEd=\|v^ui,xd−vycd\|\\mathrm\{MAE\}\_\{d\}=\|\\hat\{v\}\_\{u\_\{i\},x\}^\{d\}\-v\_\{y\_\{c\}\}^\{d\}\|\.
- •Spearman’s rank correlation coefficient \(Correlation\):we use spearman correlation metric: ρ=1−6​∑i=1ndi2n⁡\(n2−1\),\\rho=1\-\\frac\{6\\sum\_\{i=1\}^\{n\}d\_\{i\}^\{2\}\}\{n\(n^\{2\}\-1\)\},\(27\)
- •Accuracy: Given one positive candidate and one or more negative candidates, this metric measures whether the predicted alignment is closer to the positive sample than to the negative samples\. Formally, letv^ui,x\\hat\{v\}\_\{u\_\{i\},x\}denote the predicted alignment vector,vycv\_\{y\_\{c\}\}the alignment vector of the positive sample, andvyrv\_\{y\_\{r\}\}that of a negative sample\. The prediction is considered correct if ‖v^ui,x−vyc‖<‖v^ui,x−vyr‖\.\\left\\lVert\\hat\{v\}\_\{u\_\{i\},x\}\-v\_\{y\_\{c\}\}\\right\\rVert<\\left\\lVert\\hat\{v\}\_\{u\_\{i\},x\}\-v\_\{y\_\{r\}\}\\right\\rVert\.\(28\)

### B\.2Baselines

We compare our method with a diverse set of baselines covering both*non\-personalized*and*personalized*value alignment paradigms\. All personalized baselines incorporate user value information at inference time, while non\-personalized baselines do not utilize any user\-specific value signals\.

##### Non\-personalized Baselines\.

These methods generate responses without explicitly incorporating user value profiles\.

DirectAnswerdirectly generates responses by conditioning the language model on the user query and conversational contextxx, without considering any user\-specific value information\. This baseline represents general value alignment learned during pretraining\.

##### Personalized Baselines\.

These methods leverage user value information to produce personalized, value\-aligned responses\.

Value Promptexplicitly injects the user value profilevuv\_\{u\}into the system prompt, steering generation toward user\-aligned value expression\.

MetaAlignerperforms meta\-level alignment by rewriting or steering generated responses toward preferred values inferred from the user value profile and contextual information\.

COUPLEemploys explicit preference modeling and counterfactual reasoning to encourage consistency between generated responses and the user value profile\.

PADandMODapply preference\-aware decoding and modulation strategies to bias token selection toward user\-aligned values during inference\.

ValuesRAGretrieves historical user cases with similar scenarios and value profiles as examples to support personalized decision\-making\.

PersonValueis implemented by us and compares the static user value profilevuv\_\{u\}with values extracted from candidate responses, serving as a static personalization baseline without scenario\-aware inference\.

As many baseline methods produce answers as their final outputs, we evaluate value alignment by extracting value representations from the generated responses\. The specific prompts used for value extraction are detailed in the Appendix\.

### B\.3Implementation Details

##### Backbone and Training\.

We adopt the Qwen3\-8B model as the backbone and fine\-tune it using Q\-LoRA for parameter\-efficient training\. The model is optimized with Adam using a learning rate of5×10−35\\times 10^\{\-3\}and a batch size of 16\. Training is conducted for two epochs with a fixed random seed of 42 to ensure reproducibility\. During the second epoch, all parameters except the gating module are frozen, allowing the training process to focus on optimizing theα\\alphaparameters\.

##### Hyperparameters\.

We introduce a hyperparameterλ1\\lambda\_\{1\}to weight the KL\-divergence term in our objective\.λ1\\lambda\_\{1\}is searched within the range\[0,1\]\[0,1\], and the final value is selected based on validation performance\. Meanwhile, the gate coefficientα\\alphais initialized to 0\.5 to provide a balanced prior contribution at the beginning of training\.

### B\.4Human Evaluation

In this section, we present the details of the human evaluation\. We conducted two rounds of human evaluation: the first was to verify the effectiveness of the evaluator used infe​v​a​lf\_\{eval\}, and the second was to validate the performance of our proposed method\.

##### Evaluator Validation\.

We conduct a human evaluation to examine whether the evaluator employed infevalf\_\{\\text\{eval\}\}is consistent with human judgments\. To this end, we design two independent annotation settings\. In the first setting, annotators are provided with a user profile derived from a survey and are asked to label the underlying values reflected in the profile\. In the second setting, annotators are given a question–answer pair and are asked to annotate the values expressed in the answer\. The evaluator\-generated scores are then compared with these human annotations\. This annotation interface is shown in Fig\.[14](https://arxiv.org/html/2609.28942#A2.F14)\.

As summarized in Tab\.[4](https://arxiv.org/html/2609.28942#A2.T4), the evaluator demonstrates a high level of agreement with human judgments across both settings, achieving low mean absolute error \(MAE\), high Spearman correlation, and strong agreement rates\. These results are consistent with prior findings reported in COUPLE\([Guo et al\., 2026b](https://arxiv.org/html/2609.28942#bib.bib13)\), indicating that large language model–based evaluators can reliably extract human values and serve as an effective proxy for human evaluation\.

Table 4:Agreement between the automatic evaluator and human annotations under survey\-level and answer\-level evaluation settings\.![Refer to caption](https://arxiv.org/html/2609.28942v1/figures/human_eval_1.jpg)Figure 14:Human annotation interfaces used in the first human evaluation\. Annotators independently label values based on a given user profile derived from a survey or a given question–answer pair\.
##### Model Performance Evaluation\.

The second human evaluation is conducted to assess the effectiveness of our proposed method in predicting context\-specific personalized values\. We sample 50 instances corresponding to 50 distinct survey\-derived user profiles, with each profile appearing in exactly one instance\. The complete evaluation set is independently assessed by two human evaluators, both of whom review all 50 samples under identical instructions\. For each sample, annotators are presented with the survey\-derived user profile, the question, the target answer, and the anonymized value\-dimension assessments produced by BaCVA and ValuesRAG\. Following the same five\-level rubric, they determine which assessment more accurately reflects the values expressed in the target answer\. When both evaluators prefer the same method, the sample is counted as a win or loss for BaCVA; when they disagree, it is counted as a tie\. The annotation examples are shown in Tab\.[5](https://arxiv.org/html/2609.28942#A2.T5)\.

Table 5:Example results of blind value dimension analysis\. Scores are reported asModel A / Model Bfor each value dimension, andChosenindicates the model preferred by human annotators\.
##### Annotator Details\.

All annotation procedures were approved by the relevant Institutional Review Board \(IRB\) and conducted in accordance with established ethical research guidelines\. Annotators were informed about the task scope and voluntarily consented to participate\. All annotators were compensated at a rate of at least $7 per hour, which substantially exceeds the prevailing local hourly wage levels in the countries where they were employed\. The annotation tasks did not involve any personal, private, or sensitive information\. To ensure independence and avoid potential bias, the two human evaluation tasks were conducted with two separate groups of annotators, with no overlap between the annotators involved in each task\. All annotators had a background in psychology, received training on Schwartz’s theory of basic human values, and were proficient in English reading comprehension\.

### B\.5Licenses for Existing Assets

We use two existing datasets in our work: PRISM and GOOD\. More information about the source datasets and their licenses is provided below\.

- •PRISM\([Kirk et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib62)\): Human\-written texts \(including prompts\) in the PRISM dataset are licensed under the Creative Commons Attribution 4\.0 International License \(CC\-BY\-4\.0\)\. Model responses are licensed under the Creative Commons Attribution–NonCommercial 4\.0 International License \(CC\-BY\-NC\-4\.0\)\. Use of model responses must also comply with the original licenses of the corresponding model providers\.
- •GOOD \(P4G\-derived\)\([Wang et al\., 2019](https://arxiv.org/html/2609.28942#bib.bib68)\): The GOOD dataset used in this work is derived from the PersuasionForGood \(P4G\) corpus, which is released under the Apache License 2\.0\. All derived data retain the original license of the P4G dataset, and appropriate attribution to the original source is required\.

All source datasets used in our benchmark retain their original licenses, as specified by their respective creators\.

## Appendix CAdditional Experiments

### C\.1Hyperparameter Sensitivity Analysis

We analyze the sensitivity of BaCVA to key design choices by conducting hyperparameter studies\.

##### Effect of KL Weightλ1\\lambda\_\{1\}\.

We further study the effect of the KL\-divergence weightλ1\\lambda\_\{1\}, which controls the regularization strength between the variational posterior and the personalized prior\. Tab\.[6](https://arxiv.org/html/2609.28942#A3.T6)reports the sensitivity results on PRISM and GOOD\. Overall, a moderate KL weight improves performance over removing the KL term, suggesting that the personalized prior provides useful regularization\. However, overly large values ofλ1\\lambda\_\{1\}lead to clear degradation, as the posterior is over\-regularized toward the prior and becomes less adaptive to the current scenario\. These results show that BaCVA is robust within a moderate range ofλ1\\lambda\_\{1\}, while very strong KL regularization is harmful\.

Table 6:Sensitivity analysis of the KL\-divergence weightλ1\\lambda\_\{1\}on PRISM and GOOD\. Lower MAE is better, while higher correlation and accuracy are better\.

### C\.2Comparison with Training\-Time Alignment Methods

To further compare BaCVA with training\-time alignment methods, we include RLHF and MORLHF as additional baselines\. All methods use the same backbone, data splits, preference samples, and input information for a controlled comparison\. Standard RLHF learns a unified preference objective from the pooled preference data, whereas MORLHF performs multi\-objective optimization over value\-related preference signals\. The detailed training configurations follow the same experimental budget used for the main baselines\.

As shown in Tab\.[7](https://arxiv.org/html/2609.28942#A3.T7), BaCVA achieves the best performance across all three metrics\. Compared with the strongest baseline for each metric, BaCVA reduces MAE by 62\.3% relative to MORLHF, while improving correlation by 0\.191 and accuracy by 12\.72 percentage points over MetaAligner\.

Table 7:Comparison with training\-time alignment methods on PRISM\.Table 8:Prompt sensitivity analysis of the universal response estimator\. Agreement is computed with the default Origin prompt as reference\.##### Effect of Prompt Designs\.

We further examine whether the estimation of the scenario\-based prior is sensitive to prompt formulation\. Specifically, we compare three prompting strategies for generating the universal response:*Origin*, which directly answers the input question;*More Natural*, which encourages fluent and conversational responses; and*Structural*, which produces structured, point\-by\-point narratives\. As shown in Tab\.[8](https://arxiv.org/html/2609.28942#A3.T8), although different prompts lead to moderate variations in inter\-prompt agreement, the final downstream MAE remains stable across prompt designs\. Notably, the more natural prompt even slightly improves MAE from 0\.728 to 0\.715, while the structural prompt only introduces a minor degradation to 0\.743\. These results indicate that our estimator does not rely on a specific response style, and the extracted scenario\-based prior remains robust under different prompt formulations\.

##### Additional Evaluation on AlignX\.

To further address the concern regarding dataset sufficiency, we conduct additional experiments on a third dataset, AlignX[Li et al\. \(2025a\)](https://arxiv.org/html/2609.28942#bib.bib35)\. AlignX provides diverse contextual scenarios, allowing us to evaluate the effectiveness of different methods beyond the original evaluation datasets\. As shown in Tab\.[9](https://arxiv.org/html/2609.28942#A3.T9), BaCVA consistently outperforms all baselines on AlignX, achieving the lowest MAE and the highest correlation and accuracy\. Compared with the strongest baseline, ValuesRAG, BaCVA reduces MAE from 2\.184 to 1\.872, improves correlation from 0\.587 to 0\.683, and increases accuracy from 64\.40% to 68\.92%\. These results further confirm the effectiveness of BaCVA on an additional dataset and strengthen the empirical support for our method\.

Table 9:Additional results on AlignX\. Best results are in bold\.
##### Robustness across Value Systems

We further evaluate BaCVA onPRISMunder an alternative value representation based on the five\-dimensional Moral Foundations taxonomy\. Compared with the strongest baseline, ValuesRAG, our method consistently achieves performance improvements across evaluation metrics\. This consistent gain demonstrates that BaCVA effectively generalizes across different value systems and is capable of robustly modeling personalized alignment signals beyond a specific value taxonomy\.

![Refer to caption](https://arxiv.org/html/2609.28942v1/figures/exp_combined.png)Figure 15:Hyperparameter analysis on universal\-response model selection and value\-system generalization\.
##### Application to Fine\-Grained Value Dimensions\.

To further examine whether BaCVA is limited by the granularity of the underlying value taxonomy, we conduct an additional experiment onPRISMusing the value system from the Daily\-Dilemma dataset\([Chiu et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib67)\), which contains 301 fine\-grained value dimensions, such as trust, fairness, and patience\. This setting directly tests whether the proposed framework can capture more contextual and nuanced value signals beyond coarse\-grained taxonomies\.

Table 10:Results onPRISMunder the fine\-grained value system from Daily\-Dilemma\.As shown in Tab\.[10](https://arxiv.org/html/2609.28942#A3.T10), BaCVA substantially outperforms both PersonValue and ValuesRAG under the fine\-grained value system\. These results suggest that BaCVA is not constrained to a fixed coarse\-grained taxonomy, but can also adapt to more detailed value dimensions and better capture contextual nuances in personalized value alignment\.

### C\.3Cluster\-level adaptability analysis

Tab\.[11](https://arxiv.org/html/2609.28942#A3.T11)compares BaCVA with two prior\-only variants, where Prior\-vuv\_\{u\}uses only the static user value prior and Prior\-vxv\_\{x\}uses only the scenario\-level value prior\. Across all five major clusters, BaCVA consistently achieves the lowest MAE, indicating that its posterior value estimation can adapt to diverse contextual topics rather than relying on either user\- or scenario\-level priors alone\.

Table 11:Cluster\-level adaptability analysis onPRISM\.
### C\.4Case Study

We conduct a case study on the PRISM dataset to qualitatively examine whether our model can achieve both*scenario alignment*and*personalization alignment*\. As illustrated in Fig\.[16](https://arxiv.org/html/2609.28942#A3.F16), our model is able to generate responses that are consistent with individual user preferences while simultaneously adapting to different contextual scenarios\. This demonstrates that the proposed approach effectively integrates contextual understanding with personalized modeling, achieving*Contextualized Personalization Alignment*\.

![Refer to caption](https://arxiv.org/html/2609.28942v1/case_study.png)Figure 16:Case study on the PRISM dataset\. The proposed model generates responses that are aligned with both user\-specific preferences and contextual scenarios, demonstrating its ability to achieve Contextualized Personalization Alignment\.
### C\.5Downstream detail analysis

Framework Formulation\.We extend the two\-stage BaCVA framework by coupling contextual value inference with response generation\. Following the standard notation, each training instance consists of a useruu, a questionxx, a user profile𝐯u\\mathbf\{v\}\_\{u\}, question\-level universal values𝐯x\\mathbf\{v\}\_\{x\}, and question\-specific personalized values𝐯u,x\\mathbf\{v\}\_\{u,x\}\. The dataset is augmented with preference triples\(x,yc,yr\)\(x,y\_\{c\},y\_\{r\}\), whereycy\_\{c\}andyry\_\{r\}denote the human\-preferred and dispreferred responses, respectively\. In the baselineTwo\-stage BaCVA, the inferrer predicts𝐯^u,x\\hat\{\\mathbf\{v\}\}\_\{u,x\}via dual\-stream gating, and a decoupled generator subsequently conditions on a discrete text description of𝐯^u,x\\hat\{\\mathbf\{v\}\}\_\{u,x\}\. In our proposedBaCVA\-E2E, we interface these modules via continuous soft prompts\. A linear projector maps the inferred vector𝐯^u,x∈\[1,5\]10\\hat\{\\mathbf\{v\}\}\_\{u,x\}\\in\[1,5\]^\{10\}tokkprefix embeddings𝐞v∈ℝk×d\\mathbf\{e\}\_\{v\}\\in\\mathbb\{R\}^\{k\\times d\}, which are prepended to the token embeddings of\[x,y\]\[x,y\]\. The joint training objective is formulated asℒE2E=ℒvalue\+λ​ℒgen\\mathcal\{L\}\_\{\\text\{E2E\}\}=\\mathcal\{L\}\_\{\\text\{value\}\}\+\\lambda\\mathcal\{L\}\_\{\\text\{gen\}\}, whereℒvalue\\mathcal\{L\}\_\{\\text\{value\}\}is the masked value\-regression loss from the vanilla BaCVA, andℒgen\\mathcal\{L\}\_\{\\text\{gen\}\}is the preference alignment loss\. This continuous path enables response\-quality feedback to backpropagate directly into the inferrer\.

Preference Alignment Loss\.Conditioned on the soft prompts𝐞v\\mathbf\{e\}\_\{v\}, we optimize the generatorπθ\\pi\_\{\\theta\}using a DPO\-style objective with multi\-line alignment:

ℒgen=−𝔼\(x,yc,yr\)\[logσ\(β\[Δ​log⁡π​\(yc\)−Δlogπ\(yr\)\]\)\],\\begin\{split\}\\mathcal\{L\}\_\{\\text\{gen\}\}=\-\\mathbb\{E\}\_\{\(x,y\_\{c\},y\_\{r\}\)\}\\Big\[\\log\\sigma\\big\(\\beta\[&\\Delta\\log\\pi\(y\_\{c\}\)\-\\\\ &\\Delta\\log\\pi\(y\_\{r\}\)\]\\big\)\\Big\],\\end\{split\}\(29\)where the likelihood margin for responsey∈\{yc,yr\}y\\in\\\{y\_\{c\},y\_\{r\}\\\}is defined as:

Δ​log⁡π⁡\(y\)=log⁡πθ​\(y∣𝐞v,x\)πref​\(y∣𝐞v,x\)\.\\Delta\\log\\pi\(y\)=\\log\\frac\{\\pi\_\{\\theta\}\(y\\mid\\mathbf\{e\}\_\{v\},x\)\}\{\\pi\_\{\\text\{ref\}\}\(y\\mid\\mathbf\{e\}\_\{v\},x\)\}\.\(30\)andβ\\betascales the preference margin\. Likelihood computations are restricted to answer tokens, masking out prefix and question tokens\. We set the reference policylog⁡πref≡0\\log\\pi\_\{\\text\{ref\}\}\\equiv 0,λ=1\.0\\lambda=1\.0, andβ=0\.1\\beta=0\.1across all experiments\.

Conditioning Protocol\.During training, value vectors are injected as continuous soft prompts \(k=8k=8\) prepended to the LM input, and the generator is fine\-tuned via LoRA\. Conversely, during inference, to isolate the performance gains attributable to improved value inference rather than a modified decoding interface, BaCVA\-E2E adopts the exact samediscrete text promptingas the two\-stage baseline\. The E2E\-trained inferrer outputs𝐯^u,x\\hat\{\\mathbf\{v\}\}\_\{u,x\}, which is converted into natural language value levels and inserted into the system prompt \(e\.g\.,*Achievement: high \(4\.2\)*\)\. The E2E generator LoRA weights are disabled at test time, and preprocessing follows the stage\-1 implementation\.

Dataset and Preprocessing\.We utilize thePRISMdataset\([Kirk et al\., 2024](https://arxiv.org/html/2609.28942#bib.bib62)\)with an 80/20 user\-level train/test split \(using random seed 42\), resulting in a test set ofN=420N=420\. The ground\-truth vector𝐯yc\\mathbf\{v\}\_\{y\_\{c\}\}is derived directly from the 10\-dimensional annotation of the human\-chosen answer\.

Evaluation Metrics\.To comprehensively assess our framework, we evaluate performance across three metrics, each computed exclusively over the answer\-specific active dimensions:

- •MAE:Quantifies the absolute intensity mismatch between the predictedv^u,xd\\hat\{v\}^\{d\}\_\{u,x\}and ground\-truthvycdv^\{d\}\_\{y\_\{c\}\}across relevant dimensions, formulated asMAE=𝔼d​\[\|v^u,xd−vycd\|\]\\text\{MAE\}=\\mathbb\{E\}\_\{d\}\[\|\\hat\{v\}^\{d\}\_\{u,x\}\-v^\{d\}\_\{y\_\{c\}\}\|\]\.
- •Spearman’sρ\\rho:Measures the rank correlation between𝐯^u,x\\hat\{\\mathbf\{v\}\}\_\{u,x\}and𝐯yc\\mathbf\{v\}\_\{y\_\{c\}\}pooled across the entire test set to capture ordinal consistency\.
- •Accuracy \(Pairwise Ranking\):Evaluates whether the predicted vector is closer to the chosen response than the rejected alternative in terms ofL1L\_\{1\}distance, defined as‖𝐯^u,x−𝐯yc‖1<‖𝐯^u,x−𝐯yr‖1\\\|\\hat\{\\mathbf\{v\}\}\_\{u,x\}\-\\mathbf\{v\}\_\{y\_\{c\}\}\\\|\_\{1\}<\\\|\\hat\{\\mathbf\{v\}\}\_\{u,x\}\-\\mathbf\{v\}\_\{y\_\{r\}\}\\\|\_\{1\}\. Ties are scored as0\.50\.5\.

Baseline Configurations\.We compare our proposed framework against two major categories of alignment methods, all sharing the same backbone LM and utilizing greedy decoding \(max\_new\_tokens=256\\text\{max\\\_new\\\_tokens\}=256\)\. The first category consists ofnon\-personalized baselines, represented byDirectAnswer, which generates responses directly from the user query without any value conditioning\. The second category comprisespersonalized baselines, including:

- •ValuePrompt, which hard\-codes the static user profile into the system prompt;
- •COUPLE, which leverages counterfactual prompting to highlight conflicts between𝐯u\\mathbf\{v\}\_\{u\}and𝐯x\\mathbf\{v\}\_\{x\};
- •MetaAligner, which employs a critique\-and\-rewrite mechanism to align a draft response with user values;
- •PADandMOD, which implement contrastive decoding via token\-level logit manipulation;
- •ValuesRAG, which augments the generation context with few\-shot exemplars retrieved from historically similar profiles\.

Hyperparameters and Ablations\.The complete hyperparameter configuration for training BaCVA\-E2E is detailed in Tab\.[12](https://arxiv.org/html/2609.28942#A3.T12)\. For ablation analysis, we maintain the identical pipeline but evaluate under two restricted settings:*w/o gradient to inferrer*, where𝐯^u,x\\hat\{\\mathbf\{v\}\}\_\{u,x\}is detached before soft\-prompt projection to isolate decoupled training behavior, and*w/oℒvalue\\mathcal\{L\}\_\{\\text\{value\}\}*, which completely removes the explicit value\-regression constraints on the inferrer submodule\.

Table 12:Hyperparameter settings for BaCVA\-E2E training\.Tab\.[13](https://arxiv.org/html/2609.28942#A3.T13)presents the performance of BaCVA\-E2E and its ablation variants on the PRISM test set\. The results show that both gradient feedback to the value inferrer and the value supervision objective contribute positively to the final alignment performance\.

Table 13:Ablation results on the PRISM test set \(N=420N=420\)\.Qualitative Case Studies\.Fig\.[17](https://arxiv.org/html/2609.28942#A3.F17)presents two cases where the situational value𝐯^u,x\\hat\{\\mathbf\{v\}\}\_\{u,x\}diverges from the static profile𝐯u\\mathbf\{v\}\_\{u\}\. In Case A,ValuePromptframes the response around the user’s self\-direction\-dominant𝐯u\\mathbf\{v\}\_\{u\}, missing the empathetic tone the situation requires; BaCVA\-E2E correctly infers a Benevolence\-dominant𝐯^u,x\\hat\{\\mathbf\{v\}\}\_\{u,x\}and responds accordingly\. In Case B,PADanchors advice to the user’s Conformity\-dominant𝐯u\\mathbf\{v\}\_\{u\}and treats the situation as a discipline problem; BaCVA\-E2E infers that the family context elevates Benevolence and produces empathy\-first guidance instead\.

Figure 17:Qualitative comparison of BaCVA\-E2E against representative baselines on two PRISM test instances where the situational value𝐯^u,x\\hat\{\\mathbf\{v\}\}\_\{u,x\}diverges from the static user profile𝐯u\\mathbf\{v\}\_\{u\}\.
### C\.6Cross\-Extractor Evaluation

The main experiments use GPT\-5\-nano to construct both training and test value targets\. Although its agreement with human annotations is validated in Appendix[B\.4](https://arxiv.org/html/2609.28942#A2.SS4), using the same extractor throughout the pipeline may introduce extractor\-specific evaluation bias\. We therefore evaluate whether BaCVA’s improvements transfer to independently constructed value targets from different model families\.

##### Human Validation of Value Extractors

We first compare GPT\-5\-nano, Gemini\-3\-Flash, and Qwen3\-235B\-A22B against the same human\-annotated value assessments\. As shown in Tab\.[14](https://arxiv.org/html/2609.28942#A3.T14), all three extractors achieve strong agreement with human annotations at both the profile and answer levels\.

Table 14:Human validation of value extractors\.Table 15:Cross\-extractor evaluation with mismatched training and test value targets\.
##### Evaluation with Mismatched Training and Test Extractors

We retain GPT\-5\-nano targets for training, but independently reconstruct the test targets using Gemini\-3\-Flash and Qwen3\-235B\-A22B\. Neither test extractor is used during training or scenario\-prior construction\. For each extractor, the values of both preferred and dispreferred test responses are reconstructed before computing MAE, correlation, and pairwise ranking accuracy\.

As shown in Tab\.[15](https://arxiv.org/html/2609.28942#A3.T15), BaCVA consistently outperforms ValuesRAG under both matched and mismatched evaluation\. Across the two mismatched settings, BaCVA reduces MAE by 21\.2% on average and improves accuracy by 9\.56 percentage points over ValuesRAG, while also achieving higher correlation\.

### C\.7Robustness across Scenario\-Prior Estimators

To evaluate whether BaCVA depends on a particular model for constructing the scenario prior𝐯x\\mathbf\{v\}\_\{x\}, we compare three aligned LLMs from different model families: GPT\-5\-nano, Gemini\-3\-Flash, and Qwen3\-235B\-A22B\. These models are used as operational proxies for population\-level scenario salience rather than definitive representations of universal human values\.

We measure the consistency of the extracted scenario priors using within\-one agreement\. For each question and value dimension, two estimators are considered consistent when their scores differ by at most one point on the 1–5 scale\. The agreement reported for each estimator is its average pairwise agreement with the other two estimators over the same PRISM questions\.

As shown in Tab\.[16](https://arxiv.org/html/2609.28942#A3.T16), the three estimators achieve an average within\-one agreement of 89\.15%\. The downstream PRISM MAE has a standard deviation of only 0\.027, indicating that BaCVA maintains stable absolute\-error performance across different scenario\-prior estimators, although some variation remains in correlation and pairwise accuracy\.

Table 16:Robustness to the model used for constructing the scenario prior on PRISM\. Prior agreement denotes average pairwise within\-one agreement on the 1–5 value scale\.
### C\.8Generalization across Alignment Backbones

To evaluate whether the improvements of BaCVA generalize beyond the default Qwen3\-8B backbone, we additionally implement BaCVA and representative personalized baselines using Llama3\-8B\. All methods use the same datasets, data splits, value extractor, input information, and baseline configurations\.

As shown in Tab\.[17](https://arxiv.org/html/2609.28942#A3.T17), BaCVA consistently outperforms ValuePrompt and ValuesRAG under both backbone models\. With Llama3\-8B, BaCVA reduces MAE by 37\.9%, improves correlation by 0\.125, and increases accuracy by 19\.06 percentage points relative to ValuesRAG\. These results support the generalizability of BaCVA across alignment backbone families\.

Table 17:Personalized value alignment results across backbone models on PRISM\.
### C\.9Profile–Answer Conflict Analysis

To examine when contextual value inference is most beneficial, we measure the conflict between the global user profile𝐯u\\mathbf\{v\}\_\{u\}and the values expressed in the preferred answer𝐯yc\\mathbf\{v\}\_\{y\_\{c\}\}\. For each sample, the profile–answer distance is computed as

dconflict=1D​∑j=1D\|vu\(j\)−vyc\(j\)\|,d\_\{\\mathrm\{conflict\}\}=\\frac\{1\}\{D\}\\sum\_\{j=1\}^\{D\}\\left\|v\_\{u\}^\{\(j\)\}\-v\_\{y\_\{c\}\}^\{\(j\)\}\\right\|,\(31\)whereDDis the number of value dimensions\. We divide the PRISM test samples into low\-, medium\-, and high\-conflict subsets according to this distance\.

As shown in Tab\.[18](https://arxiv.org/html/2609.28942#A3.T18), the performance gap between BaCVA and profile\-based methods becomes more pronounced as profile–answer conflict increases\. On the high\-conflict subset, BaCVA reduces MAE by 17\.8% relative to ValuesRAG\. Its absolute improvement over ValuesRAG is also larger in the medium\- and high\-conflict subsets than in the low\-conflict subset, indicating that explicit contextual value inference is particularly useful when answer\-level values diverge from the global user profile\.

Table 18:MAE on PRISM under different degrees of profile–answer conflict\.
### C\.10General\-Purpose Capabilities

To assess whether personalized value\-alignment training degrades capabilities outside the target task, we evaluate the original Qwen3\-8B backbone and BaCVA on four general\-purpose benchmarks: BoolQ, ARC\-Easy, HellaSwag, and WinoGrande\([Clark et al\., 2019](https://arxiv.org/html/2609.28942#bib.bib3);[Clark et al\., 2018](https://arxiv.org/html/2609.28942#bib.bib4);[Zellers et al\., 2019](https://arxiv.org/html/2609.28942#bib.bib5);[Sakaguchi et al\., 2021](https://arxiv.org/html/2609.28942#bib.bib6)\)\. These benchmarks cover reading comprehension, science reasoning, commonsense completion, and commonsense reasoning\.

As shown in Tab\.[19](https://arxiv.org/html/2609.28942#A3.T19), BaCVA maintains performance comparable to the original backbone across all four benchmarks\. Its average performance is 78\.74%, compared with 77\.69% for the original backbone, indicating that BaCVA training does not substantially degrade general\-purpose capabilities\.

Table 19:General\-purpose performance of the original Qwen3\-8B backbone and BaCVA after personalized value\-alignment training\.
### C\.11Comparison with Explicit Stochastic Approximation

To examine the accuracy–efficiency trade\-off of the practical posterior implementation, we compare BaCVA with an explicit stochastic approximation that draws 10 Monte Carlo samples for each input\. The two implementations use the same backbone, test set, and evaluation protocol\.

As shown in Tab\.[20](https://arxiv.org/html/2609.28942#A3.T20), the amortized implementation reduces MAE by 7\.4%, improves correlation by 0\.021, and increases accuracy by 3\.97 percentage points relative to the 10\-sample Monte Carlo approximation\. Its inference latency is 7\.3% higher but remains comparable under the same evaluation setting\. These results support the use of deterministic amortized inference as a practical approximation rather than exact stochastic posterior computation\.

Table 20:Accuracy–efficiency comparison of posterior implementations on PRISM\. Inference time is reported per sample\.

## Appendix DGenerative Assistance

Generative AI tools were used to assist with language polishing and improving the clarity of presentation in this paper\. All technical content, experimental design, results, and conclusions were developed and verified by the authors\.