preference-learning

Tag

Cards List
#preference-learning

Same Verdict, Different Readers: Personalizing Fact-Checking Explanations from Relative Preferences

arXiv cs.CL ↗ · 2026-10-28 Cached

该论文提出 PERPL 框架,通过 LLM 模拟读者的相对偏好来学习个性化事实核查解释改写,在保持事实忠实性的同时优于更大的通用 LLM。

0 favorites 0 likes
#preference-learning

Aligning Large Language Models with Human Feedback: Mathematical Foundations and Algorithm Design

arXiv cs.LG ↗ · 2026-10-27 Cached

A tutorial-style article accepted to IEEE Signal Processing Magazine that surveys the mathematical foundations and algorithmic frameworks for aligning LLMs with human feedback, covering SFT, RLHF, DPO, and their links to inverse reinforcement learning and discrete choice models. It serves as a comprehensive resource for researchers and practitioners on alignment methodologies.

0 favorites 0 likes
#preference-learning

Judging the Justification: Decision Certificate Supervision for Generative Reward Models

arXiv cs.CL ↗ · 2026-10-21 Cached

The paper proposes DeCert (Decision Certificate Supervision), a framework for training generative reward models that evaluates the validity and verdict-consistency of generated rationales without needing reference explanations, improving rationale quality and positional-bias robustness by up to 14.8 points.

0 favorites 0 likes
#preference-learning

Auditing Safety Preference Supervision with Recorded Derivations

arXiv cs.CL ↗ · 2026-10-21 Cached

This paper introduces Symbolic Preference Labeling, a neuro-symbolic method where an LLM classifies predefined prompt/response attributes and explicit rules derive auditable, reproducible preference labels, then trains a policy via DPO and reports refusal rates on benign and toxic prompts.

0 favorites 0 likes
#preference-learning

Replicable Dueling Bandits

arXiv cs.LG ↗ · 2026-10-16 Cached

该论文首次研究了可复制的对抗性多臂老虎机(replicable dueling bandits)问题,设计了在共享内部随机性下保证每次运行选择一致比较对的算法,并在多种随机传递性假设下证明了次线性后悔界,实验在合成数据和语言模型偏好上验证了可复制性。

0 favorites 0 likes
#preference-learning

Similar Models, Broken Odds: Where Luce's Axiom Fails in Chatbot Arena

arXiv cs.CL ↗ · 2026-10-15 Cached

This paper shows that Luce's choice axiom, assumed by the Bradley–Terry model underlying Chatbot Arena rankings, fails for near-copy models: votes between similar models are up to 1.37× more decisive than Bradley–Terry predicts, based on 1.8 million Arena votes. A small four-to-six parameter correction improves held-out vote prediction, though leaderboard order barely changes.

0 favorites 0 likes
#preference-learning

How to Build a Benchmark That Actually Measures Taste

arXiv cs.AI ↗ · 2026-10-15 Cached

The paper introduces TasteBench, an expert-gold pairwise benchmark for measuring 'taste' in VLMs across visual, brand, book, and tech domains, along with a control protocol that quantifies and corrects evaluation artifacts like position bias, lexical leakage, and difficulty ceilings.

0 favorites 0 likes
#preference-learning

Who Should We Listen to More? Welfare-Aware Preference Acquisition for Pluralistic Alignment

arXiv cs.LG ↗ · 2026-10-15 Cached

该论文提出福利感知偏好获取方法(WAPA),在有限人工反馈预算下,通过社会 welfare 目标来分配人工标注,以改善多元对齐中不同个体偏好的监督质量,实验证明仅获取 20% 人工标签即可恢复 75–86% 的效用差距。

0 favorites 0 likes
#preference-learning

Collaborative Personalized Preference Alignment for LLMs under Data Deficiency

Hugging Face Daily Papers ↗ · 2026-10-05 Cached

The paper proposes Approximate Pareto Optimality (APO), a method for collaboratively learning lightweight LLM aligner initializations that support few-shot personalization under data deficiency, grouping users by compatible updates and coordinating competing objectives toward Pareto-optimal points with proven suboptimality bounds.

0 favorites 0 likes
#preference-learning

Robust Nash Alignment under Preference Uncertainty

arXiv cs.AI ↗ · 2026-10-02 Cached

该论文提出 Robust Nash Alignment,一个面向不确定成对偏好的博弈论对齐框架,通过主-对偶代理博弈与乐观镜面下降-上升算法,实现了对最坏情况胜率的认证下界,并在 LLM 对齐实验中优于基线方法。

0 favorites 0 likes
#preference-learning

Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreement

Hugging Face Daily Papers ↗ · 2026-09-28 Cached

This paper introduces a preference-guided adaptation framework for open-vocabulary semantic segmentation that leverages prompt disagreement to generate binary preferences, eliminating the need for dense pixel-level annotations and improving performance in specialized domains.

0 favorites 0 likes
#preference-learning

Learning Heterogeneous Preferences

arXiv cs.AI ↗ · 2026-09-17 Cached

The paper introduces a multi-stage architecture for learning individuated utility functions from multi-modal data, showing that models accounting for heterogeneous preferences outperform universal utility models in subjective tasks like aesthetic judgments.

0 favorites 0 likes
#preference-learning

SKIP: a Self-knowledge-guided Step-wise Preference Learning Framework for Concise Reasoning

arXiv cs.AI ↗ · 2026-09-16 Cached

SKIP is a self-knowledge-guided step-wise preference learning framework that improves reasoning compression in large language models, mitigating performance degradation and reducing overthinking by using DPO to guide efficient reasoning.

0 favorites 0 likes
#preference-learning

LLM-Derived Preference Judgments Are Not Self-Consistent

arXiv cs.AI ↗ · 2026-08-19 Cached

This paper investigates the self-consistency of cardinal LLM preference judgments across multiple models and finds large persistent inconsistencies, suggesting they cannot be faithfully summarized by a single utility function.

0 favorites 0 likes
#preference-learning

FedPref: Federated Preference Learning for Structured Radiology Report Extraction

arXiv cs.AI ↗ · 2026-08-19 Cached

FedPref introduces a federated preference learning method for structured radiology report extraction, allowing institutions to collaborate without sharing data while improving extraction accuracy with gains in F1 scores on simulated hospitals.

0 favorites 0 likes
#preference-learning

Synchronizing Beliefs with Second-Order Theory-of-Mind in Human-Autonomy Teams (Extended Version)

arXiv cs.AI ↗ · 2026-08-13 Cached

This paper recasts preference-based reward learning as a human-autonomy team problem, arguing that a teacher who knows the objective can design more efficient training examples than learner-driven query selection. It introduces understanding statements with second-order theory-of-mind to keep the teacher's model of the learner synchronized, showing in simulation that this approach outperforms learner-led selection.

0 favorites 0 likes
#preference-learning

Adaptive KappaSharp: Condition-Number Shaping for Preferential Bayesian Optimization

arXiv cs.LG ↗ · 2026-08-11 Cached

This paper introduces KappaSharp, a method for condition-number shaping in Preferential Bayesian Optimization that corrects the ill-conditioned Hessian from isolated pairwise comparisons, showing up to +10.9% improvement over the standard PairedGP/EUBO baseline on 11 benchmarks including plasma medicine controller tuning.

0 favorites 0 likes
#preference-learning

Crayotter: Learning Long-Horizon Video Editing Agents via Group-Relative Preference Backpropagation

arXiv cs.CL ↗ · 2026-08-05 Cached

Crayotter introduces Group-Relative Preference Backpropagation (GRPB), a method for training long-horizon video editing agents using within-task preference rankings rather than global scalar rewards. The resulting 9B model outperforms several proprietary systems on AgenticVBench.

0 favorites 0 likes
#preference-learning

Preference-based Antibody Expression Ranking: Scaling with Large-scale Weak Supervision

arXiv cs.LG ↗ · 2026-07-21 Cached

This paper proposes a preference-based learning framework for antibody expression ranking, integrating scarce quantitative data with large-scale weak positive supervision from immunization sequences. The method adapts Direct Preference Optimization to protein language models using a union-masked log-likelihood approximation and IMGT-based alignment, achieving improved ranking performance on a diverse internal dataset.

0 favorites 0 likes
#preference-learning

From Weights to Words: Expressing and Editing Preference Model Inferences in Natural Language

arXiv cs.LG ↗ · 2026-07-21 Cached

Introduces 'weights to words', a method that automatically discovers domain-relevant preference dimensions described in natural language from choice data, enabling users to inspect and edit preference model inferences in real time.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback