Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics
Summary
This paper introduces a method to improve factual consistency in text summarization by aggregating scores from multiple weak metrics via preference learning, achieving consistent factuality gains across various language models.
View Cached Full Text
Cached at: 05/27/26, 09:11 AM
# Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics Source: [https://arxiv.org/abs/2605.26840](https://arxiv.org/abs/2605.26840) [View PDF](https://arxiv.org/pdf/2605.26840) > Abstract:Reinforcement learning with evaluation metrics as rewards is widely used to enhance specific capabilities of language models\. However, for tasks such as factually consistent summarisation, existing metrics remain underdeveloped, limiting their effectiveness as signals for shaping model[this http URL](http://behaviour.while/)individual factuality metrics are unreliable, their combination can more effectively capture diverse factual errors\. We leverage this insight to introduce an automated training pipeline that improves factual consistency in summaries by aggregating scores from different weak metrics\. Our approach avoids the need for complex reward shaping by mapping scores to preferences and filtering out cases with high disagreement between metrics\. For each source document, we generate lexically similar summary pairs by varying decoding strategies, enabling the model to learn from factual differences caused by subtle lexical differences\. This approach constructs a high\-quality preference dataset using only source[this http URL](http://documents.experiments/)demonstrate consistent factuality gains across models, ranging from early encoder\-decoder architectures to modern large language models, with smaller models reaching comparable factuality to larger ones\. ## Submission history From: Yuxuan Ye \[[view email](https://arxiv.org/show-email/5733012b/2605.26840)\] **\[v1\]**Tue, 26 May 2026 10:55:03 UTC \(21,531 KB\)
Similar Articles
Learning to summarize with human feedback
OpenAI demonstrates a technique for improving language model summarization by training a reward model on human preferences and fine-tuning models with reinforcement learning, achieving significant quality improvements that generalize across datasets. This work advances model alignment through human feedback at scale, with applications beyond summarization.
Abstractiveness Metrics for Evaluating Text Summarization: A Refined Formulation with Empirical Validation
This paper introduces Reference Abstraction (RA), Summary Abstraction (SA), and Abstraction Ratio (AR) metrics to quantify abstractiveness in text summarization, using harmonic mean of document lengths and cubic non-overlap factor. Empirical evaluation on XSUM with four models shows the metrics effectively discriminate between extractive and abstractive summaries, and flag potential hallucination.
Large Language Models for Token-Efficient and Semantic-Preserving Opinion Summarization
This paper presents a framework for opinion summarization using LLMs that combines multidimensional classification and stratified sampling to reduce token usage while preserving semantic diversity and balance across viewpoints.
Optimizing Abstractive Summarization With Fine-Tuned PEGASUS
This paper presents fine-tuning of PEGASUS on the XL-Sum English corpus, achieving state-of-the-art results with significant improvements over the baseline mT5 model across ROUGE scores.
RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation
This paper proposes RIMS, a three-stage preference optimization framework for small-scale language models in retrieval-augmented generation, using synthetic chain-of-thought data and a differentiable soft aggregation mechanism to improve robustness against noisy evidence. Experiments show consistent gains over baselines on multi-hop QA benchmarks.