factual-consistency

This paper proposes a multi-factor scoring system for evaluating LLM responses, integrating accuracy, conciseness, factual consistency, readability, and coherence. Applied to the TruthfulQA dataset, it reveals strengths and limitations of mainstream models, offering a transparent evaluation framework.

BinEval is a new framework that decomposes LLM evaluation criteria into atomic binary questions, improving interpretability and enabling targeted prompt optimization, achieving strong results on factual consistency benchmarks.

This paper introduces a method to improve factual consistency in text summarization by aggregating scores from multiple weak metrics via preference learning, achieving consistent factuality gains across various language models.