preference-optimization

Tag

Cards List
#preference-optimization

Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization

Hugging Face Daily Papers ↗ · 2026-05-27 Cached

This paper introduces BiDPO, a framework that enhances text-to-image models for complex compositional prompts through preference-based fine-tuning and region-level guidance, achieving state-of-the-art results on compositional fidelity benchmarks.

0 favorites 0 likes
#preference-optimization

WeCon: An Efficient Weight-Conditioned Neural Solver for Multi-Objective Combinatorial Optimization Problems

arXiv cs.LG ↗ · 2026-05-25 Cached

Presents WeCon, a weight-conditioned neural solver for multi-objective combinatorial optimization problems that achieves comparable hypervolume to the state-of-the-art while reducing inference time by 40%.

0 favorites 0 likes
#preference-optimization

Conditional Equivalence of DPO and RLHF: Implicit Assumption, Failure Modes, and Provable Alignment

arXiv cs.AI ↗ · 2026-05-22 Cached

This paper proves that the equivalence between Direct Preference Optimization (DPO) and Reinforcement Learning from Human Feedback (RLHF) is conditional and often violated in practice, revealing failure modes where DPO optimizes relative advantage rather than absolute alignment. The authors introduce Constrained Preference Optimization (CPO) for provable alignment and demonstrate state-of-the-art performance.

0 favorites 0 likes
#preference-optimization

Token-weighted Direct Preference Optimization with Attention

arXiv cs.CL ↗ · 2026-05-22 Cached

Proposes AttentionPO, a token-weighted direct preference optimization method that uses attention from the LLM itself to estimate token weights, improving alignment performance on AlpacaEval, MT-Bench, and ArenaHard without requiring a separate reward model.

0 favorites 0 likes
#preference-optimization

Improving Code Translation with Syntax-Guided and Semantic-aware Preference Optimization

arXiv cs.AI ↗ · 2026-05-14 Cached

This paper proposes CTO, a method that improves code translation by combining syntax-guided and semantic-aware preference optimization through contrastive learning and direct preference optimization, achieving significant improvements over existing baselines in C++, Java, and Python translations.

0 favorites 0 likes
#preference-optimization

Spurious Correlation Learning in Preference Optimization: Mechanisms, Consequences, and Mitigation via Tie Training

arXiv cs.LG ↗ · 2026-05-13 Cached

This paper analyzes spurious correlation learning in preference optimization methods like DPO, identifying mechanisms such as mean spurious bias and causal-spurious leakage. It proposes 'tie training' using equal-utility preference pairs as a mitigation strategy to reduce reliance on spurious features without degrading causal learning.

0 favorites 0 likes
#preference-optimization

YFPO: A Preliminary Study of Yoked Feature Preference Optimization with Neuron-Guided Rewards for Mathematical Reasoning

arXiv cs.CL ↗ · 2026-05-13 Cached

This paper introduces YFPO, a neuron-guided preference optimization framework that uses internal activation signals to improve mathematical reasoning in large language models.

0 favorites 0 likes
#preference-optimization

Enhancing Multilingual Counterfactual Generation through Alignment-as-Preference Optimization

arXiv cs.CL ↗ · 2026-05-13 Cached

The paper introduces Macro, a preference alignment framework using DPO to improve the validity and minimality of self-generated counterfactual explanations across multiple languages.

0 favorites 0 likes
#preference-optimization

StoicLLM: Preference Optimization for Philosophical Alignment in Small Language Models

arXiv cs.CL ↗ · 2026-05-13 Cached

This research paper investigates using preference optimization (ORPO, AlphaPO) on small language models like Llama-3.2-3B and Qwen-3-4B to align them with Stoic philosophy using micro-datasets. The study finds that while 300 examples can effectively encode Stoic virtues, small models still struggle with outward-facing cosmopolitan duties.

0 favorites 0 likes
#preference-optimization

Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference Optimization

arXiv cs.CL ↗ · 2026-05-11 Cached

This paper introduces a topology-enhanced alignment framework for LLMs, utilizing trajectory topology loss and topological preference optimization based on persistent homology to regularize semantic trajectories in hidden space.

0 favorites 0 likes
#preference-optimization

Offline Preference Optimization for Rectified Flow with Noise-Tracked Pairs

Hugging Face Daily Papers ↗ · 2026-05-10 Cached

This paper introduces PNAPO, an offline preference optimization framework for rectified flow models that augments preference data with noise samples and uses dynamic regularization to improve training efficiency and sample efficiency.

0 favorites 0 likes
#preference-optimization

FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Users

arXiv cs.CL ↗ · 2026-04-20 Cached

FSPO proposes a few-shot preference optimization algorithm for LLM personalization that reframes reward modeling as meta-learning, enabling models to quickly infer personalized reward functions from limited user preferences. The method achieves 87% personalization performance on synthetic users and 70% on real users through careful synthetic preference dataset construction.

0 favorites 0 likes
#preference-optimization

CLewR: Curriculum Learning with Restarts for Machine Translation Preference Learning

arXiv cs.CL ↗ · 2026-04-20 Cached

CLewR introduces a curriculum learning strategy with restarts for improving machine translation performance in LLMs through preference optimization. The method addresses catastrophic forgetting by iterating easy-to-hard curriculum multiple times, showing consistent gains across Gemma2, Qwen2.5, and Llama3.1 models.

0 favorites 0 likes
#preference-optimization

CiPO: Counterfactual Unlearning for Large Reasoning Models through Iterative Preference Optimization

arXiv cs.CL ↗ · 2026-04-20 Cached

CiPO is a novel framework for machine unlearning in Large Reasoning Models that uses iterative preference optimization with counterfactual reasoning traces to selectively remove unwanted knowledge while preserving reasoning abilities. The method addresses the challenge of unlearning in models that rely on chain-of-thought reasoning by generating logically valid alternative reasoning paths during training.

0 favorites 0 likes
#preference-optimization

GroupDPO: Memory efficient Group-wise Direct Preference Optimization

arXiv cs.CL ↗ · 2026-04-20 Cached

GroupDPO introduces a memory-efficient algorithm for group-wise direct preference optimization that leverages multiple candidate responses per prompt while reducing peak memory usage through decoupled backpropagation. The method demonstrates consistent improvements over standard DPO across offline and online alignment settings.

0 favorites 0 likes
#preference-optimization

WavAlign: Enhancing Intelligence and Expressiveness in Spoken Dialogue Models via Adaptive Hybrid Post-Training

Hugging Face Daily Papers ↗ · 2026-04-16 Cached

WavAlign introduces a modality-aware adaptive post-training method that uses constrained preference updates and explicit anchoring to boost both semantic quality and speech expressiveness in end-to-end spoken dialogue models.

0 favorites 0 likes
← Previous
← Back to home

Submit Feedback