Tag
The paper proposes the Plan, Learn, Adapt (PLA) framework for personalized on-device itinerary generation, combining feasibility-guaranteed combinatorial planning with human preference learning via a Bradley-Terry reward model. In deployment, it achieved a 91% increase in itinerary completion rates with low latency, outperforming frontier LLMs in feasibility.
This paper introduces DROPJ, a human-centred method for safely training and deploying agent policies by learning a world model from real-world trajectories, then eliciting human preferences with justifications to train a reward model for model predictive control. Experiments show that using human-generated simulated trajectories and justifications improves safety and reduces computational cost.
This paper critiques the use of pairwise comparisons for learning human preferences, arguing that internal pluralism (multiple conflicting priorities) undermines the standard approach. It proposes a formal model and suggests that allowing indecision can improve learning efficiency.
This paper scales the SOLiD lie-detector oversight method to larger LLMs (up to 405B parameters) and evaluates it in realistic preference-learning settings, finding that undetected deception decreases with model scale but that the method is sensitive to distribution shift between training data.
ThoughtFold proposes a framework using introspective preference learning to reduce redundant explorations in Chain-of-Thought reasoning for Large Reasoning Models, achieving ~56% token reduction on DeepSeek-R1-Distill-Qwen-7B without accuracy loss.
This paper introduces two automated metrics, Lexical Alignment Score and Triangulated Preference Shift, to identify lexical overuse in LLMs and attribute it to preference learning stages. The method is tested on six model families using PubMed abstracts, replicating prior findings without manual intervention.
Proposes SelSkill, a dual-granularity preference-learning framework that learns when to invoke skills in agentic tasks, improving task success by 10.9% on ALFWorld and 5.7% on BFCL.
Introduces a curation-free metric (Triangulated Preference Shift) to isolate and quantify lexical biases induced during preference learning in LLMs, without manual curation, across six model families.
This paper formalizes calibration for probabilistic label ranking, introducing a hierarchy of calibration notions and showing that common models are poorly calibrated. It further demonstrates applications to RLHF reward models, where calibration correlates with but is not identical to accuracy.
This paper introduces a method to improve factual consistency in text summarization by aggregating scores from multiple weak metrics via preference learning, achieving consistent factuality gains across various language models.
This paper proposes a unified framework for personalized agentic reinforcement learning that decouples generic task rewards from personalized preference rewards, introducing PARPO and PSGM for preference-aligned policy optimization and skill retrieval.
This paper formalizes trust calibration for agentic tool use as a preference learning problem, using Gaussian processes and Bayesian optimization to decide when an AI agent's actions should be autonomous or require human approval.
Proposes AMATA, a multi-agent trajectory alignment framework for knowledge-intensive question answering that introduces intra-trajectory preference learning and inter-agent dependency learning to improve factual grounding and interpretability, outperforming baselines on five benchmarks.
This paper introduces the Hybrid Reward-Cyclic (HRC) model and Dynamic Self-Play Preference Optimization (DSPPO) to address the cyclic nature of human preferences in LLM alignment, achieving improved performance over Bradley-Terry and General Preference Model baselines.
This paper introduces CLIPR, a framework that learns transferable latent user preferences from minimal conversational input to improve human-aligned decision making in LLMs.
This paper introduces xi-DPO, a novel preference optimization method that reformulates the objective to minimize distance to optimal ratio reward margins, addressing hyperparameter tuning challenges in SimPO. Experimental results show that xi-DPO outperforms existing methods on open benchmarks.
WildFeedback is a novel framework that leverages in-situ user feedback from actual LLM conversations to automatically create preference datasets for aligning language models with human preferences, addressing scalability and bias issues in traditional annotation-based alignment methods.