direct-preference-optimization

Tag

Cards List
#direct-preference-optimization

Preference Optimization Improves Definition Generation: Semantic Change in EU Institutional Sustainable Finance Disclosures

arXiv cs.CL ↗ · 2026-10-27 Cached

论文提出使用直接偏好优化(DPO)微调定义生成器,以减少同一词义下定义过度分化的问题,并在学术基准和欧盟可持续金融术语的历时语义变化审计任务上进行了验证。

0 favorites 0 likes
#direct-preference-optimization

Personalized Federated Direct Preference Optimization via LoRA Adapters

arXiv cs.LG ↗ · 2026-10-12 Cached

The paper proposes pFedDPO, a personalized federated direct preference optimization method where a shared base model is synchronized via FedAvg while private LoRA adapters capture client-specific preferences, reducing gradient diversity heterogeneity compared to vanilla FedDPO.

0 favorites 0 likes
#direct-preference-optimization

Uncertainty-Normalized Margins for Direct Preference Optimization

arXiv cs.LG ↗ · 2026-10-02 Cached

EPFL and University of Helsinki researchers introduce Uncertainty-Normalized Margin DPO (UNM-DPO/ULNM-DPO-WR), which combines preference-strength margins with a learned prompt-dependent uncertainty scale and length normalization, achieving strong win rates over DPO and SimPO on HelpSteer2/3 and AlpacaEval with a Llama-3.1-8B-Instruct policy.

0 favorites 0 likes
#direct-preference-optimization

PRO-Step: Step-level Process Reward Optimization for Retrieval-Augmented Generation

arXiv cs.CL ↗ · 2026-09-03 Cached

PRO-Step introduces a step-level process reward optimization method for retrieval-augmented generation to improve multi-hop reasoning by evaluating logical validity and evidential grounding at each step.

0 favorites 0 likes
#direct-preference-optimization

PLC-DPO: Posterior Label Correction in Noisy and Ambiguous Preference Optimization

Hugging Face Daily Papers ↗ · 2026-08-31 Cached

PLC-DPO enhances Direct Preference Optimization by routing noisy preference labels into clean, flipped, or tied cases using policy-reference margins, leading to improved performance across various benchmarks.

0 favorites 0 likes
#direct-preference-optimization

Acquire, Repair, Preserve: A Diagnosis-Guided Post-Training Recipe for Small-Model Dialogue Game Agents

Hugging Face Daily Papers ↗ · 2026-08-28 Cached

This paper investigates failures in a 2B model for dialogue games and introduces a diagnosis-guided post-training recipe using SFT, DPO, and LoRA to boost performance while maintaining general capabilities.

0 favorites 0 likes
#direct-preference-optimization

Inference-Time Mitigation of Adversarial Political Bias in Large Language Models

arXiv cs.CL ↗ · 2026-08-18 Cached

This paper proposes inference-time mitigation strategies using Chain of Thought prompting and Direct Preference Optimization to reduce adversarial political bias in large language models, demonstrating significant improvements in political neutrality scores.

0 favorites 0 likes
#direct-preference-optimization

@arcinstitute: The same method that teaches an LLM which answers people prefer can teach a protein model which sequences are more stab…

X AI KOLs Following ↗ · 2026-08-14 Cached

ProteinDPO is a method that uses LLM preference learning techniques to improve the stability of protein models, developed by researchers at Arc Institute.

0 favorites 0 likes
#direct-preference-optimization

ELMER: Evolutionary Language Model that Explores and Refines

arXiv cs.LG ↗ · 2026-08-12 Cached

Introduces ELMER, an evolutionary language model that searches over natural-language policy descriptions and compiles them into executable programs, using fine-tuned Qwen3-8B with Direct Preference Optimization to control mutation strength and improve search efficiency.

0 favorites 0 likes
#direct-preference-optimization

DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models

arXiv cs.CL ↗ · 2026-07-30 Cached

DIRECT is a framework for sequence labeling using large language models that improves domain alignment through Direct Preference Optimization (DPO) after supervised fine-tuning and increases inference efficiency via controlled decoding with template-filling and KV cache reuse.

0 favorites 0 likes
#direct-preference-optimization

Learning When to Reason for Text-to-SQL via SFT and DPO

arXiv cs.CL ↗ · 2026-07-28 Cached

Proposes AutoThinkSQL, a framework that integrates an auto-thinking mechanism into SFT and DPO for Text-to-SQL, enabling the model to dynamically skip reasoning for simple queries and invoke deep CoT for complex ones, achieving gains on Spider and BIRD benchmarks while reducing output tokens by 24.6% and latency by 17.1%.

0 favorites 0 likes
#direct-preference-optimization

Diversity-Oriented Fine-Tuning for Uncertainty-Based Hallucination Detection

arXiv cs.AI ↗ · 2026-07-21 Cached

This paper proposes diversity-oriented fine-tuning strategies to improve uncertainty-based hallucination detection in LLMs by encouraging varied generations, making hallucinations more detectable via semantic entropy.

0 favorites 0 likes
#direct-preference-optimization

RLearner-LLM: Balancing Logical Grounding and Fluency in Large Language Models via Hybrid Direct Preference Optimization

arXiv cs.CL ↗ · 2026-07-20 Cached

This paper introduces RLearner-LLM, a framework using Hybrid-DPO to balance logical correctness and fluency in LLM-generated explanations, achieving significant NLI entailment improvements across multiple domains and base models while mitigating the verbosity bias of standard preference signals.

0 favorites 0 likes
#direct-preference-optimization

Newer Models, Same Advantage

Hugging Face Blog ↗ · 2026-07-16 Cached

DharmaOCR, a model specialized for Brazilian Portuguese OCR, outperforms newer models like Mistral OCR4 through domain-specific fine-tuning and Direct Preference Optimization. The article explains the training pipeline and presents benchmark results.

0 favorites 0 likes
#direct-preference-optimization

Multi-Objective Exploration and Preference Optimization via Mutual Information

arXiv cs.CL ↗ · 2026-07-03 Cached

Proposes MI-EPO, an information-theoretic framework for multi-objective alignment of large language models that uses mutual information to enhance exploration and ensure generated responses are distinguishable and aligned with different preference vectors, achieving stable trade-offs across conflicting objectives.

0 favorites 0 likes
#direct-preference-optimization

Emergent Alignment

arXiv cs.AI ↗ · 2026-06-20 Cached

This paper introduces Emergent Alignment, a self-supervised method that endows LLMs with a conscience step to review their own outputs and uses Direct Preference Optimization to steer away from unethical behavior, enabling online alignment without external judges.

0 favorites 0 likes
#direct-preference-optimization

Direct Preference Optimization for Chatbot Fine-Tuning: An Empirical Study

arXiv cs.CL ↗ · 2026-06-12 Cached

This paper presents an empirical study of Direct Preference Optimization (DPO) for fine-tuning a large language model, showing that DPO simplifies the training pipeline and achieves competitive performance while addressing training instability.

0 favorites 0 likes
#direct-preference-optimization

Direct Preference Optimization Beyond Chatbots

Hugging Face Blog ↗ · 2026-06-03 Cached

Direct Preference Optimization (DPO) is applied to OCR tasks beyond chatbots, showing significant reduction in text degeneration across multiple model families, with an average reduction of 59.4%.

0 favorites 0 likes
#direct-preference-optimization

Curriculum Learning for Safety Alignment

arXiv cs.LG ↗ · 2026-05-27 Cached

This paper proposes Staged-Competence, a curriculum learning framework for DPO-based safety alignment that organizes preference data by difficulty, improving robustness and data efficiency while preserving general capabilities.

0 favorites 0 likes
#direct-preference-optimization

Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs

arXiv cs.CL ↗ · 2026-05-26 Cached

This paper applies Direct Preference Optimization (DPO) to align Audio LLMs for transcribing English-Mandarin code-switching speech, achieving up to 89.6% MER reduction in-distribution and 20% out-of-distribution. It identifies three failure modes—language omission, translation instead of transcription, and hallucination—and shows that preference-based alignment effectively elicits correct code-switching behavior from multilingual Audio LLMs.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback