CWF: A Collaborative Writing Framework for Personalized and Reliable Popular Science Writing

# CWF: A Collaborative Writing Framework for Personalized and Reliable Popular Science Writing
Source: [https://arxiv.org/html/2609.06126](https://arxiv.org/html/2609.06126)
Ruibiao FuAffiliation:School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, ChinaDi TangAffiliation:School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, ChinaYunlong YangAffiliation:School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, ChinaRan WangAffiliation:School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, ChinaAffiliation:Philosophy and Social Sciences Laboratory of Big Data and National Communication Strategy, Ministry of Education, Wuhan 430074, ChinaSicheng LuAffiliation:School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, ChinaPeixuan WuAffiliation:School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, ChinaXiaoyu FanAffiliation:School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, ChinaJiacheng MaAffiliation:School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, ChinaHaozhe LuoAffiliation:School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, ChinaYang XiaoAffiliation:The National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China \{furuibiao3,tangdi030223,yunlongyang64,xiaoyuf30\}@gmail\.com \{rex\_wang,loo\_seychan,w\_px,jiacheng\_ma,luo\_haozhe,Yang\_Xiao\}@hust\.edu\.cn

###### Abstract

We introducePersonalized and Reliable Popular Science Writing, a novel task that requires adapting scientific explanations to audiences with different cognitive levels while preserving factual accuracy\. However, improving personalization often introduces simplifications that increase the risk of hallucination and factual distortion\. To address these challenges, we first construct a dataset of 39,134 entries and a reader\-centricPersonalized Science Communication Benchmark \(PSCB\)that jointly evaluates audience adaptation and factual accuracy\. To reduce data and computational requirements while improving generalization across domains and audiences, we introduce DA\-MoE, which explicitly decouples audience adaptation from domain knowledge through separate modeling\. To enable robust verification and revision in evidence\-scarce scenarios, a multi\-agent fact\-checking mechanism that augments limited evidence with role\-specific agent debate and propagates confidence over a graph is proposed\. Experiments on PSCB show that our approach achieves state\-of\-the\-art performance\. Our code is open\-sourced at https://github\.com/DPInnovationWorks/CWF\.

22footnotetext:The three authors contribute equally to this work\.11footnotetext:Corresponding author\.## 1Introduction

![Refer to caption](https://arxiv.org/html/2609.06126v1/images/overall_contrast.png)Figure 1:Comparison between Specialized Approaches \(Fine\-tuning and RAG\-based methods\) and Collaborative Writing Framework \(CWF\)\.Popular science writing aims to communicate scientific knowledge to audiences in an accurate and accessible way\. Large language models \(LLMs\) have introduced new possibilities for this task by enabling flexible text generation across domains and writing styles\. However, effective popular science writing requires more than fluent generation\. The generated article should be adapted to the target audience in both writing style and cognitive adaptation, while remaining factual accuracy\. Existing methods mainly rely on prompt engineering[Kim et al\. \(2024\)](https://arxiv.org/html/2609.06126#bib.bib9);[Tang et al\. \(2024\)](https://arxiv.org/html/2609.06126#bib.bib8), retrieval\-based verification[Min et al\. \(2023a\)](https://arxiv.org/html/2609.06126#bib.bib18), or supervised fine\-tuning \(SFT\)[Goldsack et al\. \(2022\)](https://arxiv.org/html/2609.06126#bib.bib30)\. Despite their effectiveness, these approaches usually treat writing style, audience adaptation, and factual accuracy as separate objectives, making it difficult to generate articles that are both personalized and reliable\.

This limitation becomes more pronounced in SFT\. High\-quality popular science writing often requires jointly aligned data that couples domain knowledge, audience adaptation, and writing style[Guo et al\. \(2022\)](https://arxiv.org/html/2609.06126#bib.bib31)\. Such data is costly to collect because it must contain both accurate knowledge and audience\-adaptive expressions\. Moreover, as a generative task, popular science writing may produce factual omissions and hallucinated explanations\. Therefore, the key challenge is not only to reduce the dependence on large\-scale paired data for domain and audience adaptation, but also to ensure factual accuracy throughout the generation process\.

Existing benchmarks do not adequately address this challenge, they typically emphasize stylistic preference\([August et al\., 2024](https://arxiv.org/html/2609.06126#bib.bib32);[Tang et al\., 2024](https://arxiv.org/html/2609.06126#bib.bib8)\)or factual accuracy in popular science writing\([Min et al\., 2023b](https://arxiv.org/html/2609.06126#bib.bib33);[Damm and others, 2024](https://arxiv.org/html/2609.06126#bib.bib34);[Anonymous, 2025](https://arxiv.org/html/2609.06126#bib.bib35)\), but rarely consider both of them within a unified benchmark\. Audience adaptation often changes explanation granularity, analogy use, implicit reasoning, and conceptual simplification, which may increase the risk of unsupported information or factual distortion\([Devaraj et al\., 2022](https://arxiv.org/html/2609.06126#bib.bib17);[Taylor and Dewsbury, 2018](https://arxiv.org/html/2609.06126#bib.bib42)\)\. Accordingly, PSCB benchmark is introduced for personalized and reliable popular science writing, as shown in Figure[1](https://arxiv.org/html/2609.06126#S1.F1)\. PSCB jointly evaluates cognitive adaptation and factual accuracy\. Beyond reader attitude and personalization alignment, PSCB first introduces Cognitive Load\([Sweller, 1988](https://arxiv.org/html/2609.06126#bib.bib23);[Salemi et al\., 2023](https://arxiv.org/html/2609.06126#bib.bib20);[Ouwehand et al\., 2021](https://arxiv.org/html/2609.06126#bib.bib21);[Zu et al\., 2021](https://arxiv.org/html/2609.06126#bib.bib22)\)as a metric and decomposes metrics into fine\-grained dimensions for assessing cognitive adaptation beyond surface style\.

To address the challenges of costly jointly aligned data and the difficulty of balancing personalization with factual accuracy, we design the Collaborative Writing Framework \(CWF\)\. We first introduce a decoupled MoE architecture for popular science writing\. This framework relaxes the requirement for paired data that contains the target audience and scientific knowledge\. This design allows us to construct a dataset of 39,134 entries that does not rely on joint alignment between domains and audiences, enabling training across different scientific domains and reader groups\. However, simple MoE faces difficulty when transferring to new domains, since generating articles in different domains may require retraining the whole model\. We propose DA\-MoE, whose key idea is to assign adaptation tasks to individual pluggable experts and to separate audience\-specific adaptation from domain\-specific knowledge, so that cognitive adaptation and knowledge can be learned from different sources\. This allows to adapt to new domains with only lightweight training\.

Popular science writing requires more than cognitive adaptation\. Even well\-personalized generations may contain omissions, unsupported claims, or hallucination, making factual verification essential\. CWF therefore incorporates a fact\-checking module to preserve factual accuracy during generation\. However, RAG\-based methods struggle when direct evidence is sparse or conflicting because they rely primarily on retrieved evidence rather than deriving additional support by reasoning over indirect viewpoints or evidences\([Asai et al\., 2023](https://arxiv.org/html/2609.06126#bib.bib1)\)\. Moreover, most post\-hoc fact\-checking methods rely on direct model judgments over completed outputs, not only producing unquantifiable results but also cannot guide the writing process\([Jiang et al\., 2023](https://arxiv.org/html/2609.06126#bib.bib16);[Min et al\., 2023a](https://arxiv.org/html/2609.06126#bib.bib18);[Zheng et al\., 2023](https://arxiv.org/html/2609.06126#bib.bib15);[Wei et al\., 2024](https://arxiv.org/html/2609.06126#bib.bib19)\)\. Furtherly, we propose an verification mechanism that combines multi\-agent debate with graph\-based inference\. Specifically, multi\-agent debate simulates an interactive review process, and helps validate claims by introducing complementary perspectives and broadening retrieval angles when direct evidence is incomplete\. Intervention applies verification during the generation process rather than only after generation, allowing multi\-agent factual feedback to guide the writing process itself\. Finally, graph\-based inference connects claims, retrieved evidence, and agent reasoning, enabling additional evidence to be inferred from indirect viewpoint while making factual judgments traceable rather than black\-box decisions\.

In summary,the main contributionsof this work are as follows:

- •Personalized and reliable popular science writing task is proposed for the first time\. We further introduce PSCB, a human\-aligned benchmark that jointly evaluates audience adaptation and factual accuracy\.
- •We propose a DA\-MoE framework for popular science writing that decouples domain knowledge from audience adaptation, reduces reliance on paired training data, and enables low\-cost generalization to new domains\.
- •We propose a multi\-agent fact\-checking mechanism with graph\-based reasoning\. It simulates the debate process to support multi\-perspective verification, intervention during generation, and quantifiable confidence scores for robust claim validation\.

## 2Related Works

Popular Science Writing\.Recent work in computational linguistics studied how to transform scientific content into language that general audiences can understand\. Goldsack et al\.[Goldsack et al\. \(2022\)](https://arxiv.org/html/2609.06126#bib.bib30)and Cheng et al\.[Cheng et al\. \(2025\)](https://arxiv.org/html/2609.06126#bib.bib28)formulate this problem as lay summarization or academic\-to\-public paraphrasing by constructing paired datasets and benchmarks, including PLOS, eLife, and VTechAGP\. Fang et al\.[Fang et al\. \(2024\)](https://arxiv.org/html/2609.06126#bib.bib27)and Jiang et al\.[Jiang et al\. \(2025\)](https://arxiv.org/html/2609.06126#bib.bib29)further study reliability in popular science generation by analyzing factual errors in LLM\-generated biomedical summaries or using multi\-agent interaction to improve science journalism generation\. However, existing studies usually address cognitive adaptation and factual accuracy separately\. They do not provide a unified framework that jointly considers personalization for different readers and factual accuracy in popular science writing\. To bridge this gap, we formulate personalized and reliable popular science writing and introduce PSCB to jointly evaluate reader adaptation and factual accuracy\.

Personalized and Controllable Writing\.Personalized generation primarily utilizes Few\-Shot Prompting or SFT\([Houlsby et al\., 2019](https://arxiv.org/html/2609.06126#bib.bib6);[Brown et al\., 2020](https://arxiv.org/html/2609.06126#bib.bib5);[Li and Liang, 2021](https://arxiv.org/html/2609.06126#bib.bib7)\)\. However, prompt\-based methods often struggle with long\-form text, while fine\-tuning imposes strict data requirements to model target styles effectively\. This limitation is particularly pronounced when combining heterogeneous styles, which requires constructing costly datasets that jointly capture multiple stylistic patterns\([Ju and Wang, 2024](https://arxiv.org/html/2609.06126#bib.bib25)\)\. To reduce this dependence, we propose DA\-MoE, which decouples audience adaptation from domain knowledge and enables flexible composition across reader groups and scientific domains\.

Fact\-checking in LLMs\.Traditionally, fact\-checking has heavily depended on human evaluators, a process that is not only time\-consuming and expensive but also prone to subjective biases\([Honovich et al\., 2022](https://arxiv.org/html/2609.06126#bib.bib10);[Devaraj et al\., 2022](https://arxiv.org/html/2609.06126#bib.bib17);[Min et al\., 2023a](https://arxiv.org/html/2609.06126#bib.bib18)\)\. Subsequently, approaches combining LLMs with RAG emerged to integrate external databases\([Asai et al\., 2023](https://arxiv.org/html/2609.06126#bib.bib1)\)\. However, they often struggle when direct evidence is sparse or conflicting, due to a lack of deep reasoning capabilities\. Recent methods have proposed decomposing complex claims into atomic units\([Min et al\., 2023a](https://arxiv.org/html/2609.06126#bib.bib18);[Wei et al\., 2024](https://arxiv.org/html/2609.06126#bib.bib19)\)or simulating multi\-agent debates\([Sun et al\., 2025](https://arxiv.org/html/2609.06126#bib.bib2);[Ma et al\., 2026](https://arxiv.org/html/2609.06126#bib.bib3)\)\. By supplementing the original claims with additional context, these approaches enable judgments from multiple perspectives\. However, they remain post\-hoc and static, unable to instruct in the generation process proactively\. Also, while simulated multi\-agent debates introduce interactive reasoning, they rely on model judgments without providing rigorous metrics\. To address these issues, we propose a verification framework\. By combining multi\-agent debate with graph, our method not only synthesizes diverse perspectives to handle incomplete evidence but also provides quantifiable confidence scores to individual reasoning nodes, ensuring a transparent verification process\.

Evaluation Benchmark\.Existing benchmarks either emphasize readability and audience accessibility in science communication\([August et al\., 2024](https://arxiv.org/html/2609.06126#bib.bib32);[Tang et al\., 2024](https://arxiv.org/html/2609.06126#bib.bib8)\)or focus on factual accuracy and factuality evaluation in generated texts\([Min et al\., 2023b](https://arxiv.org/html/2609.06126#bib.bib33);[Min et al\., 2023a](https://arxiv.org/html/2609.06126#bib.bib18);[Damm and others, 2024](https://arxiv.org/html/2609.06126#bib.bib34);[Anonymous, 2025](https://arxiv.org/html/2609.06126#bib.bib35);[Wei et al\., 2024](https://arxiv.org/html/2609.06126#bib.bib19)\)\. However, these two lines of work are usually studied separately, making them insufficient for evaluating popular science generation across different audiences\. Our Personalized Science Communication Benchmark \(PSCB\) evaluates whether a generated science article is both reliable and adapted to its intended readers\. In real\-world science communication, an effective article should not only convey scientifically grounded knowledge, but also adjust its conceptual difficulty, explanation strategy, and communicative style to different reader groups\. PSCB therefore contains two complementary evaluation branches: Factual Reliability and Audience Adaptation\.

![Refer to caption](https://arxiv.org/html/2609.06126v1/images/framework_overview.png)Figure 2:Overview of CWF, which integrates collaborative writing, personalized style modeling, and fact\-checking with multi\-agent discussion to enhance the quality and relevance of AI\-generated science writings\.
## 3Collaborative Writing Framework

As illustrated in Figure[2](https://arxiv.org/html/2609.06126#S2.F2), the proposed CWF aims to generate coherent science writing that remain both personalized and factually reliable\. CWF organizes the writing into three coordinated stages—drafting, personalization, and verification—to address the core challenge of balancing personalization and factual accuracy\.

### 3\.1Collaborative Writing

Existing LLM\-based popular science writing methods suffer from structural inconsistency, weak coherence, and limited control over narrative organization\([Shao et al\., 2024](https://arxiv.org/html/2609.06126#bib.bib4);[Jiang et al\., 2024](https://arxiv.org/html/2609.06126#bib.bib26)\)\. To address this, we adopt a collaborative generation framework in which one agent plans the article structure and section summaries, while another expands them into fluent passages with examples and transitions\. This process forms the initial draft and improves narrative coherence\.

### 3\.2DA\-MoE

To address the difficulty of data collection in SFT, we build a modular MoE\-based generation framework on top of the plug\-and\-play MixLoRA architecture[Li et al\. \(2024\)](https://arxiv.org/html/2609.06126#bib.bib24)\. The framework consists of two specialized MoE networks constructed from multiple Qwen2\.5\-3B backbone models: an audience\-adaptation MoE for learning audience\-specific writing styles, and a knowledge\-domain MoE for learning topic\-specific knowledge\.

Formally, given an inputxx, we construct two MoE modules: an audience\-adaptation MoE and a knowledge\-domain MoE\. Letr∈\{s,k\}r\\in\\\{s,k\\\}denote the expert type, wheressrepresents audience adaptation andkkrepresents knowledge domain\. The corresponding expert sets are defined as:

ℰr=\{E1r,E2r,…,ETrr\},\\mathcal\{E\}^\{r\}=\\\{E^\{r\}\_\{1\},E^\{r\}\_\{2\},\\ldots,E^\{r\}\_\{T\_\{r\}\}\\\},\(1\)whereTs=MT\_\{s\}=MandTk=NT\_\{k\}=N\. Each expert is initialized from Qwen2\.5\-3B\. Audience experts are fine\-tuned on data associated with specific audience adaptation, while knowledge experts are fine\-tuned on data from specific scientific domains\.

For each MoE module, a lightweight MixLoRA gating network assigns expert weights based on the input\. The routing distribution is computed as:

gr​\(x\)=softmax⁡\(MLPr​\(x\)\),g^\{r\}\(x\)=\\mathrm\{softmax\}\(\\mathrm\{MLP\}\_\{r\}\(x\)\),\(2\)whereMLPr\\mathrm\{MLP\}\_\{r\}denotes the gating network for expert typerr\. The output of each MoE module is then:

hr​\(x\)=∑i=1Trgir​\(x\)​Eir​\(x\),h^\{r\}\(x\)=\\sum\_\{i=1\}^\{T\_\{r\}\}g^\{r\}\_\{i\}\(x\)E^\{r\}\_\{i\}\(x\),\(3\)wheregir​\(x\)g^\{r\}\_\{i\}\(x\)is the routing weight assigned to theii\-th expert inℰr\\mathcal\{E\}^\{r\}\.

The training process has three stages\. First, each Qwen2\.5\-3B expert is independently fine\-tuned on its assigned style or knowledge data, so that the style experts specialize in audience adaptation and the knowledge experts specialize in domain\-specific scientific content\. Second, after the expert parameters are fixed, we train the MixLoRA gating networks using a small amount of supervised data, allowing the model to learn how to route each input to appropriate style and knowledge experts\. Third, we introduce an AlphaNet module to dynamically balance the contributions of the two MoE networks during generation\.Specifically, AlphaNet is also parameterized as an MLP and directly predicts an input\-dependent coefficient:

α=MLPa​l​p​h​a​n​e​t​\(x\)\.\\alpha=\\mathrm\{MLP\}\_\{alphanet\}\(x\)\.\(4\)The final hidden representation is then obtained by contrasting the style\-oriented and knowledge\-oriented MoE outputs:

h⁡\(x\)=α∗hs​\(x\)\+\(1−α\)∗hk​\(x\)\.h\(x\)=\\alpha\*h^\{s\}\(x\)\+\(1\-\\alpha\)\*h^\{k\}\(x\)\.\(5\)
To examine whether AlphaNet learns different balances for different target audiences, we analyze the distribution ofα\\alphaacross reader groups\.

Reader GroupMeanStd\.MedianMinMaxChildren0\.6860\.1010\.6970\.3830\.846Teenagers0\.2620\.0800\.2560\.0950\.480Adults0\.1830\.0590\.1760\.0790\.351

Table 1:Distribution ofα\\alphaacross different target reader groups \(n=100n=100per group\)\.As shown in Table[1](https://arxiv.org/html/2609.06126#S3.T1), the meanα\\alphadecreases from children \(0\.686\) to teenagers \(0\.262\) and adults \(0\.183\)\. Since a largerα\\alphaindicates greater reliance on the style network, this result suggests that the model places more emphasis on style and readability for younger readers, while relying more on knowledge content for adult readers\. This pattern is also consistent with previous studies showing that narrative and style\-oriented presentation can improve comprehension and engagement for nonexpert audiences[Dahlstrom \(2014\)](https://arxiv.org/html/2609.06126#bib.bib43);[Downs \(2014\)](https://arxiv.org/html/2609.06126#bib.bib44)\.

MethodTraining TimeData AmountData TypeAverage PSCB scoreLoRA2\.1 hrs5000Dual\-style2\.985MoE8\.3 hrs20000Single\-style3\.059DA\-MoE22 min1000Single\-style3\.451Table 2:Comparison of training time, data requirements, data type, and average PSCB score in different methods\.Table[2](https://arxiv.org/html/2609.06126#S3.T2)shows that our method achieves the highest average PSCB score while requiring substantially less training time and fewer training examples\. This efficiency mainly comes from the modular design of our framework\. When extending the model to a new domain, we do not need to collect large\-scale dual\-style data or retrain the full model\. Instead, only a small amount of single\-style data is required to train the gating network, while the existing style and knowledge experts can be reused\.

### 3\.3Fact\-checking and Revision

Our framework performs fact\-checking through a multi\-agent collaboration that simulates interactive discussions around scientific claims\. Rather than relying on a single model to assess factuality, we decompose the verification process into a sequence of role\-specific steps\.

Claim Extraction\.Given a draft article, the system parses each paragraph into a set of atomic claims, ensuring that each statement can be independently verified\. For each decomposed claim, we estimate its coverage to the original paragraph and the sums of all coverage scores are 1, which are used as weights in confidence aggregation\.

Markov Chain–Based Debate\.We instantiate three complementary roles: Leader acts as an expert who gives the scientific judgment; Trust agent acts as a reader\-oriented supporter who searches for reasonable evidence and complementary explanation; Skeptic agent acts as a critics who challenges unsupported claims and weak evidence\. Science communication is not merely a one\-way transmission of expert knowledge, but a mediated process involving expert judgment, audience\-oriented interpretation, and critical gatekeeping\([Wynne, 1992](https://arxiv.org/html/2609.06126#bib.bib13);[Burns et al\., 2003](https://arxiv.org/html/2609.06126#bib.bib12);[Fahy and Nisbet, 2011](https://arxiv.org/html/2609.06126#bib.bib14)\)\. Accordingly, the Leader, Trust, and Skeptic agents respectively operationalize these three functions for factual verification, with detailed role designs provided in Appendix[B](https://arxiv.org/html/2609.06126#A2)\.

Given an atomic claim, the process starts fromS0S\_\{0\}, where the Leader provides an initial judgment\. The subsequent debate order is controlled by the factuality signalf⁡\(Lt\)f\(L\_\{t\}\)parsed from the latest Leader output, wheref⁡\(Lt\)=1f\(L\_\{t\}\)=1for supportive stances\(1,0\.2\)\(1,0\.2\)andf⁡\(Lt\)=0f\(L\_\{t\}\)=0for neutral or opposing stances\(0,−0\.2,−1\)\(0,\-0\.2,\-1\):

St\+1=\{S2,f⁡\(Lt\)=1,S1,f⁡\(Lt\)=0\.S\_\{t\+1\}=\\begin\{cases\}S\_\{2\},&f\(L\_\{t\}\)=1,\\\\ S\_\{1\},&f\(L\_\{t\}\)=0\.\\end\{cases\}\(6\)When Leader initially supports a claim, Skeptic is invoked first to prevent premature acceptance; when Leader is uncertain or negative, Trust agent is invoked first to explore missing evidence or alternative explanatory paths\. After each round, Leader updates the judgment based on both agents’ arguments, and the transition rule is applied again\.

We set the maximum number of debate rounds toRmax=3R\_\{\\max\}=3, following the ablation results \(provided in Appendix[A\.4](https://arxiv.org/html/2609.06126#A1.SS4)\) showing that three rounds provide the most stable confidence estimates\. The debate stops early if all agents reach the same stance after one complete debate round; otherwise, it continues untilRmaxR\_\{\\max\}is reached\.

Confidence Calculation\.The graph assigns every claim, agent statement, and evidence a confidence score on\[0,5\]\[0,5\]\. Evidence nodes are fixed at confidence55, while claim and agent\-statement nodes start from a neutral prior of2\.52\.5\. Each agent statement carries a stance value in\{−1,−0\.2,0,0\.2,1\}\\\{\-1,\-0\.2,0,0\.2,1\\\}, indicating whether it strongly or weakly supports, opposes, or remains neutral toward its parent node\. Confidence is propagated bottom\-up through the graph instead of being decided by vote\. For each statement, the system aggregates confidence by combining evidence and child statements, then maps the signed result back to the\[0,5\]\[0,5\]\. The final claim confidence is computed from the statements and direct evidence\.

Consensus Summarization and Writing Revision\.The system produces a judgment based on the final reasoning graph and the calculated confidence scores\. Claims with low confidence or detected inconsistencies will be revised by a summarizer agent who aggregates the discussion outcomes\. These revised claims are then merged to form the verified article, ensuring that inaccurate content is revised prior to publication\.

## 4PSCB Benchmark

Dataset\.PSCB dataset is constructed with 39,134 entries for training across scientific domains and reader groups\. We collect popular science writing from science communication sources, including science\-oriented WeChat public accounts and science books, covering three domains: AI, Biology, and Medicine\. To model audience diversity, we further organize data for target reader groups: child, teen, and adult \(Appendix[C\.1](https://arxiv.org/html/2609.06126#A3.SS1)\)\.

Metrics\.As shown in Table[3](https://arxiv.org/html/2609.06126#S4.T3)and Appendix[D](https://arxiv.org/html/2609.06126#A4), we score each article using three personalization metrics in 0–5 scale: Cognitive Load \(CL\), Personalization Alignment \(PA\), and Reader Attitude \(RA\)\. CL measures whether the explanation matches the reader’s cognitive capacity\([Sweller, 1988](https://arxiv.org/html/2609.06126#bib.bib23);[Salemi et al\., 2023](https://arxiv.org/html/2609.06126#bib.bib20);[Ouwehand et al\., 2021](https://arxiv.org/html/2609.06126#bib.bib21);[Zu et al\., 2021](https://arxiv.org/html/2609.06126#bib.bib22)\); PA measures whether the content selection, knowledge level, linguistic style, and contextual framing are tailored to the target audience\([Kreuter and Wray, 2003](https://arxiv.org/html/2609.06126#bib.bib38);[Flek, 2020](https://arxiv.org/html/2609.06126#bib.bib36);[Moorjani et al\., 2022](https://arxiv.org/html/2609.06126#bib.bib37)\); RA measures reader engagement, credibility, and continuance intention\.\([O’Brien and Toms, 2008](https://arxiv.org/html/2609.06126#bib.bib39);[Appelman and Sundar, 2016](https://arxiv.org/html/2609.06126#bib.bib40);[Bhattacherjee, 2001](https://arxiv.org/html/2609.06126#bib.bib41)\)\. By jointly modeling these three metrics, PSCB better evaluates both cognitive adaptation and domain knowledge\.

MetricDimensionsCognitive LoadINTR: Intrinsic FitEXTR: Extraneous Burden ControlGERM: Germane SupportPersonalization AlignmentCONT: Content RelevanceKNOW: Knowledge\-Level FitSTYLE: Style ConsistencyCONTX: Contextual ResonanceReader AttitudeENG: Engagement AppealTRU: Trust and CredibilityCONTI: Continuance Intention

Table 3:Personalization metrics and dimensions in PSCB\. Each dimension is scored on a 0–5 scale by both LLM judges and human annotators\.### 4\.1LLM\-as\-a\-Judge in Personalization

PSCB evaluates cognitive adaptation and domain knowledge through a unified protocol that combines dynamically weighted LLM\-as\-a\-judge scoring, human questionnaires, and consistency analysis between the two evaluation sources\.

For each generated articleyiy\_\{i\}, science domains or writing prompttit\_\{i\}, and target reader personapip\_\{i\}, PSCB evaluates three personalization\-oriented metrics:ℳ=\{CL,PA,RA\}\\mathcal\{M\}=\\\{\\mathrm\{CL\},\\mathrm\{PA\},\\mathrm\{RA\}\\\}, as shown in Table[3](https://arxiv.org/html/2609.06126#S4.T3)\.

Dynamic Dimension Weighting\.Before scoring the article, a meta\-evaluator LLM analyzes only the domainstit\_\{i\}and target personapip\_\{i\}, and assigns a metric\-specific weight vector:

𝐰i\(m\)\\displaystyle\\mathbf\{w\}^\{\(m\)\}\_\{i\}=\{wi,d\(m\)\}d∈𝒟m,\\displaystyle=\\\{w^\{\(m\)\}\_\{i,d\}\\\}\_\{d\\in\\mathcal\{D\}\_\{m\}\},\(7\)∑d∈𝒟mwi,d\(m\)\\displaystyle\\sum\_\{d\\in\\mathcal\{D\}\_\{m\}\}w^\{\(m\)\}\_\{i,d\}=1,w\(m\)i,d≥0\.\\displaystyle=1,\\quad w^\{\(m\)\}\_\{i,d\}\\geq 0\.This design captures the fact that different domains and reader groups may require different evaluation priorities\. For example, scaffolding and vivid examples may be more important for children\.

Dimension\-Level Scoring\.Given the predefined dimensions, the LLM judge assigns a scoresi,dLLM∈\[0,5\]s^\{\\mathrm\{LLM\}\}\_\{i,d\}\\in\[0,5\]to each dimension and provides a brief rationale for interpretability\. The metric\-level score is computed as a weighted sum:

Si,mLLM=∑d∈𝒟mwi,d\(m\)​si,dLLM\.S^\{\\mathrm\{LLM\}\}\_\{i,m\}=\\sum\_\{d\\in\\mathcal\{D\}\_\{m\}\}w^\{\(m\)\}\_\{i,d\}s^\{\\mathrm\{LLM\}\}\_\{i,d\}\.\(8\)Eventually, the three metrics are weighted equally in the overall personalization score\.

### 4\.2Human Questionnaire

To test whether LLM judgments reflect reader preferences, we convert the same metrics and dimensions into a 40\-item human questionnaire\. We retain 30 participants from each of the child, teen, and adult groups; each rates all six anonymized systems, yielding 540 questionnaires\. System order is counterbalanced, and identical instructions and an anchored 0–5 scale are used\. Before system identities are examined, we remove incomplete or duplicate submissions and screen for implausibly short completion times, invariant response strings, and inconsistency on reverse\-worded items\([Meade and Craig, 2012](https://arxiv.org/html/2609.06126#bib.bib45);[Curran, 2016](https://arxiv.org/html/2609.06126#bib.bib46)\); 540 of 585 questionnaires \(92\.3%\) are retained\.

After reverse coding, we average items within each dimension and apply the same dynamic weights and metric aggregation as in the LLM evaluation\. Metric\-level ordinal Krippendorff’sα\\alpha\(inter\-participant agreement\) is 0\.781–0\.803\. ICC\(2,k\)\(2,k\)—the two\-way random\-effects, absolute\-agreement coefficient measuring the reliability of the mean overkkparticipants—is 0\.973–0\.976 \(Eq\.[11](https://arxiv.org/html/2609.06126#A3.E11)\)\([Shrout and Fleiss, 1979](https://arxiv.org/html/2609.06126#bib.bib49)\)\. Appendix[C](https://arxiv.org/html/2609.06126#A3)reports screening, sensitivity checks, stratified agreement, participant statistics, and the full questionnaire\.

### 4\.3LLM\-Human Consistency Alignment

We distinguish score calibration from preference preservation\. MAE measures absolute score error, whereas Mean Absolute Rank Difference \(MARD\) measures the average absolute difference between the LLM and human ranks of the six systems\. We additionally report Spearman’sρ\\rho, tie\-corrected Kendall’sτb\\tau\_\{b\}, and pairwise ordering accuracy\. For Qwen\-Plus, dynamic dimension weighting obtains MAE=0\.417\{\}=0\.417, MARD=0\.667\{\}=0\.667,ρ=0\.829\\rho\{=\}0\.829,τb=0\.733\\tau\_\{b\}\{=\}0\.733, and 86\.7% pairwise accuracy; across seven judge models,ρ\\rhoranges from 0\.829 to 0\.943 and pairwise accuracy from 86\.7% to 93\.3%\. Appendix[A\.2](https://arxiv.org/html/2609.06126#A1.SS2)gives definitions, per\-judge and per\-reader\-group results, cross\-judge concordance, and a direct held\-out pairwise evaluation\.

### 4\.4Factual Accuracy

In addition to personalization, PSCB evaluates factual accuracy through graph\-based claim validation\. Each generated claim is represented in a reasoning graph, where supporting and opposing evidence are assigned confidence scores\. The final graph\-based factual accuracy is an article\-level score on a 0–5 scale, computed by aggregating claim confidence scores while penalizing unsupported or incorrect scientific statements\([Min et al\., 2023a](https://arxiv.org/html/2609.06126#bib.bib18);[Wei et al\., 2024](https://arxiv.org/html/2609.06126#bib.bib19)\)\. For clarity, Table[4](https://arxiv.org/html/2609.06126#S5.T4)reports this main article\-level metric; Table[6](https://arxiv.org/html/2609.06126#S5.T6)reports claim\-level confidence diagnostics using the same graph mechanism; and Table[7](https://arxiv.org/html/2609.06126#S5.T7)reports auxiliary SAFE metrics, which consist of atomic\-fact counts and ratio\-based scores in\[0,1\]\[0,1\]\. Values with different units are not directly comparable across these tables\.

## 5Experiment Results

Experiment Setup\.We evaluate six configurations: two non\-fine\-tuned baselines and four fine\-tuned variants\. The baselines are Base \(Qwen2\.5\-3B\) and Qwen2\.5\-14B\. The fine\-tuned variants \(LoRA, MoE, CWF, and CWF\-R\) are built on Qwen2\.5\-3B\. LoRA fine\-tunes Qwen2\.5\-3B on the collected popular science corpus\. MoE follows MixLoRA and trains multiple experts on mixed domain\-knowledge and audience\-adaptation data\. CWF further decouple these two data sources into different expert groups and uses AlphaNet for fusion\. CWF\-Rextends CWF with a fact\-checking module\. Model training is conducted on a single NVIDIA V100 with 32GB memory\. We evaluate on the PSCB, which jointly measures personalization and factual accuracy by Qwen\-Plus\.

GroupMethodLLMUserFact\. Acc\.ChildBase2\.7763\.2933\.18LoRA2\.9553\.2863\.31MoE3\.1853\.3913\.47Qwen2\.5\-14B3\.6063\.5353\.67CWF3\.6833\.5423\.72CWF\-R3\.3303\.4804\.05TeensBase2\.7703\.4613\.24LoRA2\.7823\.2903\.37MoE2\.8693\.3553\.52Qwen2\.5\-14B2\.8293\.6803\.73CWF3\.0663\.8453\.88CWF\-R3\.0203\.7834\.21AdultBase3\.4423\.7753\.31LoRA3\.2183\.2483\.44MoE3\.1243\.6013\.57Qwen2\.5\-14B3\.1923\.9003\.79CWF3\.6033\.9943\.93CWF\-R3\.4573\.9444\.26Table 4:Overall performance across all topics for six methods and three target reader groups\. LLM and User denote personalization scores; Fact\. Acc\. denotes article\-level graph\-based factual accuracy\. All three metrics are reported on a 0–5 scale\.![Refer to caption](https://arxiv.org/html/2609.06126v1/images/consistency_line_chart_matplotlib.png)Figure 3:Comparison of consistency with human evaluation across different LLM\-as\-a\-judge evaluation\. The closer evaluation curve is to the human evaluation, the stronger its alignment with human judgments\.### 5\.1Overall Performance Analysis

Table[4](https://arxiv.org/html/2609.06126#S5.T4)reports the overall performance of all methods across children, teens, and adults\. CWF achieves the strongest LLM\-as\-a\-judge and human evaluation performance across the three reader groups, indicating that CWF better adapts popular science writing to different cognitive levels\. LoRA and MoE provide useful adaptation baselines, but they do not consistently improve user experience over Base model\. Based on participant feedback, LoRA\- and MoE\-generated texts often exhibit mismatches with the target audience\.

Specifically, outputs for children are perceived as overly simplistic and lacking substantive content, while those for teens and adults tend to be excessively difficult\. This suggests that SFT or expert routing can capture some surface\-level style patterns, but is insufficient for fine\-grained cognitive adaptation\. In contrast, CWF explicitly adjusts both style and audience adaptation\. CWF also outperforms the larger Qwen2\.5\-14B baseline, showing that the gains mainly come from the framework design\. CWF\-Robtains the best factual accuracy, confirming the benefit of the revision stage\. Its slightly lower personalization scores reflect a mild but audience\-dependent trade\-off between factual revision and reader\-centric expression\. Stronger factual revision substantially improves reliability, but it may introduce formal scientific constraints and higher information density, which can increase cognitive load and reduce accessibility, especially for younger audiences\. PSCB captures this communication\-level trade\-off: factual revision in popular science writing should be adapted to cognitive levels rather than applied uniformly across audiences\. Compared with Base model, CWF improves the average LLM\-as\-a\-judge and human evaluation by 15\.2% and 8\.1%, respectively, while CWF\-Rimproves average factual accuracy by 28\.7%\.

Paired tests on the common topic set \(Appendix[E](https://arxiv.org/html/2609.06126#A5)\) show that all pairwise differences remain statistically significant after Holm correction\. CWF significantly outperforms Qwen2\.5\-14B for child, teen, and adult readers\. CWF\-Ralso differs significantly from CWF across all reader groups\.

Results judged by more models are provided in the Appendix[A\.1](https://arxiv.org/html/2609.06126#A1.SS1)\.

### 5\.2Effectiveness of PSCB

We further analyze whether the design of PSCB improves the consistency between LLM\-as\-a\-judge and human evaluation\. As shown in Table[5](https://arxiv.org/html/2609.06126#S5.T5), average weighting achieves lower aggregate MAE than one\-step, indicating that decomposing personalization into explicit metrics reduces the ambiguity of holistic scoring and makes LLM judgments closer to human ratings\. Dynamic dimension weighting achieves the best consistency, suggesting that equal weighting is insufficient for audience\-adaptive science writing\. Since different topics and target audiences emphasize different evaluation aspects, different weights help PSCB better capture human preferences\. Figure[3](https://arxiv.org/html/2609.06126#S5.F3)visually confirms this trend: Dynamic Dimension Weighting is closest to the human evaluation curve, followed by Average Dimension Weighting, while One\-step LLM Scoring deviates the most\. These results show that metric decomposition and dynamic weighting jointly make PSCB more human\-aligned, interpretable, and traceable\.

MethodDynamicAverageOne\-StepUserBase2\.9652\.2762\.1833\.510LoRA2\.9842\.6022\.5763\.275MoE3\.0592\.6722\.4623\.449Qwen2\.5\-14B3\.2092\.8632\.7633\.705CWF3\.4513\.1192\.8283\.794CWF\-R3\.3022\.9143\.0113\.736MAE↓\\downarrow0\.4170\.8370\.941—MARD↓\\downarrow0\.6670\.6671\.000—Table 5:Consistency between LLM & human scores\.SettingSPUSP↑\\uparrowPU↑\\uparrowVar\.↓\\downarrowFull4\.2092\.7201\.3271\.4891\.3930\.482w/o DC4\.1832\.7371\.3641\.4461\.3730\.561w/o DB3\.9472\.8861\.6181\.0611\.2680\.876w/o GP4\.0282\.8131\.7421\.2151\.0710\.694

Table 6:Claim\-level diagnostic ablation on SciFact\.S/P/U: mean graph confidence in\[0,5\]\[0,5\]for supported / partially supported / unsupported claims\.SP=S−\-PandPU=P−\-Uare confidence gaps, rather than article\-level scores\.Var\.: average confidence variance\. w/o DC: without claim decomposition; w/o DB: without debate; w/o GP: without graph propagation\.MethodSup\. \(\#\)↑\\uparrowUnsup\. \(\#\)↓\\downarrowPrec↑\\uparrowR@64↑\\uparrowF1@64↑\\uparrowCWF36\.9327\.010\.5780\.5770\.577RAG40\.3523\.520\.6320\.6300\.631SAFE42\.0821\.370\.6630\.6570\.660MAD44\.4318\.860\.7020\.6940\.698CWF\-R45\.7617\.920\.7190\.7150\.717

Table 7:Auxiliary factuality evaluation using SAFE\. Sup\./Unsup\. are the average numbers of supported / not\-supported atomic facts per article\. Precision isS/\(S\+N\)S/\(S\+N\), R@64 ismin⁡\(S/64,1\)\\min\(S/64,1\)\. These three ratio\-based metrics range from 0 to 1\.
### 5\.3Ablation Study in Fact\-checking

We adapt data on SciFact[Wadden et al\. \(2020\)](https://arxiv.org/html/2609.06126#bib.bib11), report mean confidence for*supported*,*partial*,*unsupported*claims, the marginsSup\.−Part\.\\mathrm\{Sup\.\}\-\\mathrm\{Part\.\},Part\.−Unsup\.\\mathrm\{Part\.\}\-\\mathrm\{Unsup\.\}and the average variance of confidence scores\. As shown in Table[6](https://arxiv.org/html/2609.06126#S5.T6), wider margins indicate clearer difference when recognizing different types, while lower variance shows stable confidence across claims\. Removing the debate stage produces the largest performance drop\. Disabling graph raises mean confidence on unsupported claims and increases variance, showing that relational aggregation sharpens scores and reduces instability\. Claim decomposition yields modest but consistent gains in margins and variance\.

### 5\.4Comparison with SAFE Metrics

Following SAFE[Wei et al\. \(2024\)](https://arxiv.org/html/2609.06126#bib.bib19), we report the numbers of supported and not\-supported atomic facts, factual precision, R@64, and F1@64, as shown in Table[7](https://arxiv.org/html/2609.06126#S5.T7)\. The improvement of our method comes from two aspects rather than simply retrieving more evidence\. First, claim decomposition and multi\-agent debate help find implicit reasoning evidence for complex claims\. Second, the debate corrects weak or misleading evidence from earlier retrieval steps by checking it from both supportive and skeptical views\. For example, for the claim "BRCA1\-mutated cancer cells are more sensitive to PARP inhibitors," direct evidence may be missing\. Our agents can connect indirect clues, such as BRCA1 mutations causing DNA repair defects and PARP inhibitors increasing DNA damage\. Meanwhile, the Skeptic filters evidence that only mentions related concepts but does not truly support the claim\. Thus, the final reasoning graph improves factual accuracy through both evidence expansion and evidence correction\. The applicability conditions and failure modes of CWF\-Rare summarized in Appendix[A\.6](https://arxiv.org/html/2609.06126#A1.SS6)\.

## 6Conclusion

In this paper, we propose CWF, a novel collaborative writing framework for personalized and reliable popular science writing\. Our framework features DA\-MoE, which decouples audience adaptation from domain knowledge through pluggable experts, and a multi\-agent fact\-checking mechanism that combines role\-specific debate with graph\-based confidence inference for robust verification and revision\. Experiments on PSCB show that CWF effectively improves both personalization and factual accuracy in popular science writing\.

## Limitations

While CWF improves factual accuracy and personalized writing, several limitations remain\. The dataset can be further expanded in domain balance and linguistic diversity, and the current framework focuses mainly on textual personalization, leaving multimodal personalization for future work\. And the multi\-agent verification may also increase token consumption and the detailed information is provided in Appendix[A\.7](https://arxiv.org/html/2609.06126#A1.SS7)\.

## Acknowledgments

We thank the anonymous reviewers and the Area Chair for their constructive feedback\. This work is supported by the National Social Science Foundation of China under Grant No\. 25BXW041, and the National Natural Science Foundation of China under Grant No\. 62671264 and 62271221\. The computation was completed in the HPC platform of Huazhong University of Science and Technology\. Ran Wang is the corresponding author \(rex\_wang@hust\.edu\.cn\)\.

## References

- AnonymousAutomatic factuality evaluation metric for biomedical plain language generation\.arXiv preprint arXiv:2503\.08890\.Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p3.1),[§2](https://arxiv.org/html/2609.06126#S2.p4.1)\.
- Appelman and Sundar \(2016\)A\. Appelman and S\. S\. SundarMeasuring message credibility: construction and validation of an exclusive scale\.Journalism & Mass Communication Quarterly93\(1\),pp\. 59–79\.External Links:[Document](https://dx.doi.org/10.1177/1077699015606057)Cited by:[§4](https://arxiv.org/html/2609.06126#S4.p2.1)\.
- Asaiet al\.\(2023\)A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. HajishirziSelf\-rag: learning to retrieve, generate, and critique through self\-reflection\.InThe Twelfth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p5.1),[§2](https://arxiv.org/html/2609.06126#S2.p3.1)\.
- Augustet al\.\(2024\)T\. August, K\. Lo, N\. A\. Smith, and K\. ReineckeKnow your audience: the benefits and pitfalls of generating plain language summaries beyond the general audience\.InProceedings of the CHI Conference on Human Factors in Computing Systems,Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p3.1),[§2](https://arxiv.org/html/2609.06126#S2.p4.1)\.
- Bhattacherjee \(2001\)A\. BhattacherjeeUnderstanding information systems continuance: an expectation\-confirmation model\.MIS Quarterly25\(3\),pp\. 351–370\.External Links:[Document](https://dx.doi.org/10.2307/3250921)Cited by:[§4](https://arxiv.org/html/2609.06126#S4.p2.1)\.
- Brownet al\.\(2020\)T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.Language models are few\-shot learners\.Advances in neural information processing systems33,pp\. 1877–1901\.Cited by:[§2](https://arxiv.org/html/2609.06126#S2.p2.1)\.
- Burnset al\.\(2003\)T\. W\. Burns, D\. J\. O’Connor, and S\. M\. StocklmayerScience communication: a contemporary definition\.Public Understanding of Science12\(2\),pp\. 183–202\.Cited by:[§3\.3](https://arxiv.org/html/2609.06126#S3.SS3.p3.1)\.
- Chenget al\.\(2025\)M\. Cheng, J\. Gong, C\. Yuan, W\. A\. Ingram, E\. Fox, and H\. EldardiryVTechAGP: an academic\-to\-general\-audience text paraphrase dataset and benchmark models\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 6110–6130\.Cited by:[§2](https://arxiv.org/html/2609.06126#S2.p1.1)\.
- Cohen \(1960\)J\. CohenA coefficient of agreement for nominal scales\.Educational and Psychological Measurement20\(1\),pp\. 37–46\.External Links:[Document](https://dx.doi.org/10.1177/001316446002000104)Cited by:[§A\.3](https://arxiv.org/html/2609.06126#A1.SS3.p3.1)\.
- Cronbach \(1951\)L\. J\. CronbachCoefficient alpha and the internal structure of tests\.Psychometrika16\(3\),pp\. 297–334\.External Links:[Document](https://dx.doi.org/10.1007/BF02310555)Cited by:[§C\.3](https://arxiv.org/html/2609.06126#A3.SS3.p1.2)\.
- Curran \(2016\)P\. G\. CurranMethods for the detection of carelessly invalid responses in survey data\.Journal of Experimental Social Psychology66,pp\. 4–19\.External Links:[Document](https://dx.doi.org/10.1016/j.jesp.2015.07.006)Cited by:[§C\.2](https://arxiv.org/html/2609.06126#A3.SS2.p1.1),[§4\.2](https://arxiv.org/html/2609.06126#S4.SS2.p1.1)\.
- Dahlstrom \(2014\)M\. F\. DahlstromUsing narratives and storytelling to communicate science with nonexpert audiences\.Proceedings of the National Academy of Sciences111\(Supplement 4\),pp\. 13614–13620\.External Links:[Document](https://dx.doi.org/10.1073/pnas.1320645111)Cited by:[§3\.2](https://arxiv.org/html/2609.06126#S3.SS2.p6.1)\.
- Dammet al\.\(2024\)H\. Dammet al\.WisPerMed at “discharge me\!”: advancing text generation in healthcare with large language models, dynamic expert selection and priming techniques on mimic\-iv\.InProceedings of the BioNLP Workshop,Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p3.1),[§2](https://arxiv.org/html/2609.06126#S2.p4.1)\.
- Devarajet al\.\(2022\)A\. Devaraj, W\. Sheffield, B\. Wallace, and J\. J\. LiEvaluating factuality in text simplification\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 7331–7345\.External Links:[Link](https://aclanthology.org/2022.acl-long.506/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.506)Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p3.1),[§2](https://arxiv.org/html/2609.06126#S2.p3.1)\.
- Diaconis and Graham \(1977\)P\. Diaconis and R\. L\. GrahamSpearman’s footrule as a measure of disarray\.Journal of the Royal Statistical Society: Series B \(Methodological\)39\(2\),pp\. 262–268\.External Links:[Document](https://dx.doi.org/10.1111/j.2517-6161.1977.tb01624.x)Cited by:[§A\.3](https://arxiv.org/html/2609.06126#A1.SS3.p1.2)\.
- Downs \(2014\)J\. S\. DownsPrescriptive scientific narratives for communicating usable science\.Proceedings of the National Academy of Sciences111\(Supplement 4\),pp\. 13627–13633\.External Links:[Document](https://dx.doi.org/10.1073/pnas.1317502111)Cited by:[§3\.2](https://arxiv.org/html/2609.06126#S3.SS2.p6.1)\.
- Efron \(1979\)B\. EfronBootstrap methods: another look at the jackknife\.The Annals of Statistics7\(1\),pp\. 1–26\.External Links:[Document](https://dx.doi.org/10.1214/aos/1176344552)Cited by:[§C\.3](https://arxiv.org/html/2609.06126#A3.SS3.p1.3)\.
- Fahy and Nisbet \(2011\)D\. Fahy and M\. NisbetThe science journalist online: shifting roles and emerging practices\.Journalism12,pp\. 778–793\.External Links:[Document](https://dx.doi.org/10.1177/1464884911412697)Cited by:[§3\.3](https://arxiv.org/html/2609.06126#S3.SS3.p3.1)\.
- Fanget al\.\(2024\)B\. Fang, X\. Dai, and S\. KarimiUnderstanding faithfulness and reasoning of large language models on plain biomedical summaries\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 9890–9911\.Cited by:[§2](https://arxiv.org/html/2609.06126#S2.p1.1)\.
- Flek \(2020\)L\. FlekReturning the N to NLP: towards contextually personalized classification models\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,Online,pp\. 7828–7838\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.700),[Link](https://aclanthology.org/2020.acl-main.700/)Cited by:[§4](https://arxiv.org/html/2609.06126#S4.p2.1)\.
- Goldsacket al\.\(2022\)T\. Goldsack, Z\. Zhang, C\. Lin, and C\. ScartonMaking science simple: corpora for the lay summarisation of scientific literature\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),Abu Dhabi, United Arab Emirates,pp\. 10589–10604\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.724/),[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.724)Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p1.1),[§2](https://arxiv.org/html/2609.06126#S2.p1.1)\.
- Guoet al\.\(2022\)Y\. Guo, W\. Qiu, Y\. Wang, and T\. CohenAutomated lay language summarization of biomedical scientific reviews\.External Links:2012\.12573,[Link](https://arxiv.org/abs/2012.12573)Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p2.1)\.
- Hayes and Krippendorff \(2007\)A\. F\. Hayes and K\. KrippendorffAnswering the call for a standard reliability measure for coding data\.Communication Methods and Measures1\(1\),pp\. 77–89\.External Links:[Document](https://dx.doi.org/10.1080/19312450709336664)Cited by:[§C\.3](https://arxiv.org/html/2609.06126#A3.SS3.p1.2)\.
- Honovichet al\.\(2022\)O\. Honovich, R\. Aharoni, J\. Herzig, H\. Taitelbaum, D\. Kukliansy, V\. Cohen, T\. Scialom, I\. Szpektor, A\. Hassidim, and Y\. MatiasTRUE: re\-evaluating factual consistency evaluation\.arXiv preprint arXiv:2204\.04991\.Cited by:[§2](https://arxiv.org/html/2609.06126#S2.p3.1)\.
- Houlsbyet al\.\(2019\)N\. Houlsby, A\. Giurgiu, S\. Jastrzebski, B\. Morrone, Q\. De Laroussilhe, A\. Gesmundo, M\. Attariyan, and S\. GellyParameter\-efficient transfer learning for nlp\.InInternational conference on machine learning,pp\. 2790–2799\.Cited by:[§2](https://arxiv.org/html/2609.06126#S2.p2.1)\.
- Jameset al\.\(1984\)L\. R\. James, R\. G\. Demaree, and G\. WolfEstimating within\-group interrater reliability with and without response bias\.Journal of Applied Psychology69\(1\),pp\. 85–98\.External Links:[Document](https://dx.doi.org/10.1037/0021-9010.69.1.85)Cited by:[§C\.3](https://arxiv.org/html/2609.06126#A3.SS3.p1.2)\.
- Jianget al\.\(2025\)G\. Jiang, X\. Shi, and Q\. LuoJRE\-l: journalist, reader, and editor llms in the loop for science journalism for the general audience\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 6579–6594\.Cited by:[§2](https://arxiv.org/html/2609.06126#S2.p1.1)\.
- Jianget al\.\(2024\)Y\. Jiang, Y\. Shao, D\. Ma, S\. Semnani, and M\. LamInto the unknown unknowns: engaged human learning through participation in language model agent conversations\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 9917–9955\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.554/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.554)Cited by:[§3\.1](https://arxiv.org/html/2609.06126#S3.SS1.p1.1)\.
- Jianget al\.\(2023\)Z\. Jiang, F\. Xu, L\. Gao, Z\. Sun, Q\. Liu, J\. Dwivedi\-Yu, Y\. Yang, J\. Callan, and G\. NeubigActive retrieval augmented generation\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 7969–7992\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.495/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.495)Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p5.1)\.
- Ju and Wang \(2024\)S\. Ju and C\. WangBeyond simple text style transfer: unveiling compound text style transfer with prompt\-based pre\-trained language models\.InICASSP 2024\-2024 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 6850–6854\.Cited by:[§2](https://arxiv.org/html/2609.06126#S2.p2.1)\.
- Kendall and Babington Smith \(1939\)M\. G\. Kendall and B\. Babington SmithThe problem ofmmrankings\.The Annals of Mathematical Statistics10\(3\),pp\. 275–287\.External Links:[Document](https://dx.doi.org/10.1214/aoms/1177732186)Cited by:[§A\.2](https://arxiv.org/html/2609.06126#A1.SS2.p2.1)\.
- Kendall \(1938\)M\. G\. KendallA new measure of rank correlation\.Biometrika30\(1/2\),pp\. 81–93\.External Links:[Document](https://dx.doi.org/10.1093/biomet/30.1-2.81)Cited by:[§A\.2](https://arxiv.org/html/2609.06126#A1.SS2.p1.3)\.
- Kimet al\.\(2024\)T\. Kim, D\. Agarwal, J\. Ackerman, and M\. SahaSteering ai\-driven personalization of scientific text for general audiences\.arXiv preprint arXiv:2411\.09969\.Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p1.1)\.
- Kreuter and Wray \(2003\)M\. W\. Kreuter and R\. J\. WrayTailored and targeted health communication: strategies for enhancing information relevance\.American Journal of Health Behavior27\(Suppl\. 3\),pp\. S227–S232\.External Links:[Document](https://dx.doi.org/10.5993/ajhb.27.1.s3.6)Cited by:[§4](https://arxiv.org/html/2609.06126#S4.p2.1)\.
- Krippendorff \(2004\)K\. KrippendorffContent analysis: an introduction to its methodology\.2 edition,Sage Publications,Thousand Oaks, CA\.External Links:ISBN 9780761915447Cited by:[§C\.3](https://arxiv.org/html/2609.06126#A3.SS3.p1.2)\.
- Liet al\.\(2024\)D\. Li, Y\. Ma, N\. Wang, Z\. Ye, Z\. Cheng, Y\. Tang, Y\. Zhang, L\. Duan, J\. Zuo, C\. Yang,et al\.Mixlora: enhancing large language models fine\-tuning with lora\-based mixture of experts\.arXiv preprint arXiv:2404\.15159\.Cited by:[§3\.2](https://arxiv.org/html/2609.06126#S3.SS2.p1.1)\.
- Li and Liang \(2021\)X\. L\. Li and P\. LiangPrefix\-tuning: optimizing continuous prompts for generation\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),C\. Zong, F\. Xia, W\. Li, and R\. Navigli \(Eds\.\),Online,pp\. 4582–4597\.External Links:[Link](https://aclanthology.org/2021.acl-long.353/),[Document](https://dx.doi.org/10.18653/v1/2021.acl-long.353)Cited by:[§2](https://arxiv.org/html/2609.06126#S2.p2.1)\.
- Maet al\.\(2026\)X\. Ma, G\. Rao, L\. Xu, X\. Wang, Z\. Fan, and Z\. ZhangGuided and knowledgeable multi\-agent debate for fact verification\.Expert Systems with Applications299,pp\. 130103\.External Links:ISSN 0957\-4174,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.eswa.2025.130103),[Link](https://www.sciencedirect.com/science/article/pii/S0957417425037194)Cited by:[§2](https://arxiv.org/html/2609.06126#S2.p3.1)\.
- Meade and Craig \(2012\)A\. W\. Meade and S\. B\. CraigIdentifying careless responses in survey data\.Psychological Methods17\(3\),pp\. 437–455\.External Links:[Document](https://dx.doi.org/10.1037/a0028085)Cited by:[§C\.2](https://arxiv.org/html/2609.06126#A3.SS2.p1.1),[§4\.2](https://arxiv.org/html/2609.06126#S4.SS2.p1.1)\.
- Minet al\.\(2023a\)S\. Min, K\. Krishna, X\. Lyu, M\. Lewis, W\. Yih, P\. Koh, M\. Iyyer, L\. Zettlemoyer, and H\. HajishirziFActScore: fine\-grained atomic evaluation of factual precision in long form text generation\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 12076–12100\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.741/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.741)Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p1.1),[§1](https://arxiv.org/html/2609.06126#S1.p5.1),[§2](https://arxiv.org/html/2609.06126#S2.p3.1),[§2](https://arxiv.org/html/2609.06126#S2.p4.1),[§4\.4](https://arxiv.org/html/2609.06126#S4.SS4.p1.1)\.
- Minet al\.\(2023b\)S\. Min, K\. Krishna, X\. Lyu, M\. Lewis, W\. Yih, P\. W\. Koh, M\. Iyyer, L\. Zettlemoyer, and H\. HajishirziFActScore: fine\-grained atomic evaluation of factual precision in long form text generation\.Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing\.Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p3.1),[§2](https://arxiv.org/html/2609.06126#S2.p4.1)\.
- Moorjaniet al\.\(2022\)S\. Moorjani, A\. Krishnan, H\. Sundaram, E\. Maslowska, and A\. SankarAudience\-centric natural language generation via style infusion\.InFindings of the Association for Computational Linguistics: EMNLP 2022,Abu Dhabi, United Arab Emirates,pp\. 1919–1932\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.findings-emnlp.138),[Link](https://aclanthology.org/2022.findings-emnlp.138/)Cited by:[§4](https://arxiv.org/html/2609.06126#S4.p2.1)\.
- Ouwehandet al\.\(2021\)K\. Ouwehand, A\. v\. d\. Kroef, J\. Wong, and F\. PaasMeasuring cognitive load: are there more valid alternatives to likert rating scales?\.InFrontiers in Education,Vol\.6,pp\. 702616\.Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p3.1),[§4](https://arxiv.org/html/2609.06126#S4.p2.1)\.
- O’Brien and Toms \(2008\)H\. L\. O’Brien and E\. G\. TomsWhat is user engagement? a conceptual framework for defining user engagement with technology\.Journal of the American Society for Information Science and Technology59\(6\),pp\. 938–955\.External Links:[Document](https://dx.doi.org/10.1002/asi.20801)Cited by:[§4](https://arxiv.org/html/2609.06126#S4.p2.1)\.
- Salemiet al\.\(2023\)A\. Salemi, S\. Mysore, M\. Bendersky, and H\. ZamaniLamp: when large language models meet personalization\.arXiv preprint arXiv:2304\.11406\.Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p3.1),[§4](https://arxiv.org/html/2609.06126#S4.p2.1)\.
- Shaoet al\.\(2024\)Y\. Shao, Y\. Jiang, T\. Kanell, P\. Xu, O\. Khattab, and M\. LamAssisting in writing Wikipedia\-like articles from scratch with large language models\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico,pp\. 6252–6278\.External Links:[Link](https://aclanthology.org/2024.naacl-long.347/),[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.347)Cited by:[§3\.1](https://arxiv.org/html/2609.06126#S3.SS1.p1.1)\.
- Shrout and Fleiss \(1979\)P\. E\. Shrout and J\. L\. FleissIntraclass correlations: uses in assessing rater reliability\.Psychological Bulletin86\(2\),pp\. 420–428\.External Links:[Document](https://dx.doi.org/10.1037/0033-2909.86.2.420)Cited by:[§C\.3](https://arxiv.org/html/2609.06126#A3.SS3.p1.2),[§4\.2](https://arxiv.org/html/2609.06126#S4.SS2.p2.1)\.
- Spearman \(1904\)C\. SpearmanThe proof and measurement of association between two things\.The American Journal of Psychology15\(1\),pp\. 72–101\.External Links:[Document](https://dx.doi.org/10.2307/1412159)Cited by:[§A\.2](https://arxiv.org/html/2609.06126#A1.SS2.p1.3)\.
- Sunet al\.\(2025\)X\. Sun, J\. Li, Y\. Zhong, D\. Zhao, and R\. YanTowards detecting llms hallucination via markov chain\-based multi\-agent debate framework\.InICASSP 2025\-2025 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 1–5\.Cited by:[§2](https://arxiv.org/html/2609.06126#S2.p3.1)\.
- Sweller \(1988\)J\. SwellerCognitive load during problem solving: effects on learning\.Cognitive science12\(2\),pp\. 257–285\.Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p3.1),[§4](https://arxiv.org/html/2609.06126#S4.p2.1)\.
- Tanget al\.\(2024\)X\. Tang, X\. Zhang, Y\. Shao, J\. Wu, Y\. Zhao, A\. Cohan, M\. Gong, D\. Zhang, and M\. GersteinStep\-back profiling: distilling user history for personalized scientific writing\.arXiv preprint arXiv:2406\.14275\.Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p1.1),[§1](https://arxiv.org/html/2609.06126#S1.p3.1),[§2](https://arxiv.org/html/2609.06126#S2.p4.1)\.
- Taylor and Dewsbury \(2018\)C\. Taylor and B\. M\. DewsburyOn the problem and promise of metaphor use in science and science communication\.Journal of Microbiology & Biology Education19\(1\)\.External Links:[Document](https://dx.doi.org/10.1128/jmbe.v19i1.1538)Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p3.1)\.
- Waddenet al\.\(2020\)D\. Wadden, S\. Lin, K\. Lo, L\. L\. Wang, M\. van Zuylen, A\. Cohan, and H\. HajishirziFact or fiction: verifying scientific claims\.InEMNLP,Cited by:[§5\.3](https://arxiv.org/html/2609.06126#S5.SS3.p1.1)\.
- Weiet al\.\(2024\)J\. Wei, C\. Yang, X\. Song, Y\. Lu, N\. Hu, J\. Huang, D\. Tran, D\. Peng, R\. Liu, D\. Huang,et al\.Long\-form factuality in large language models\.Advances in Neural Information Processing Systems37,pp\. 80756–80827\.Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p5.1),[§2](https://arxiv.org/html/2609.06126#S2.p3.1),[§2](https://arxiv.org/html/2609.06126#S2.p4.1),[§4\.4](https://arxiv.org/html/2609.06126#S4.SS4.p1.1),[§5\.4](https://arxiv.org/html/2609.06126#S5.SS4.p1.1)\.
- Willmott and Matsuura \(2005\)C\. J\. Willmott and K\. MatsuuraAdvantages of the mean absolute error \(MAE\) over the root mean square error \(RMSE\) in assessing average model performance\.Climate Research30,pp\. 79–82\.External Links:[Document](https://dx.doi.org/10.3354/cr030079)Cited by:[§A\.2](https://arxiv.org/html/2609.06126#A1.SS2.p1.3)\.
- Wynne \(1992\)B\. WynneMisunderstood misunderstanding: social identities and public uptake of science\.Public Understanding of Science1\(3\),pp\. p\. 281–304\.\.External Links:ISSN 0963\-6625Cited by:[§3\.3](https://arxiv.org/html/2609.06126#S3.SS3.p3.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing, H\. Zhang, J\. Gonzalez, and I\. StoicaJudging llm\-as\-a\-judge with mt\-bench and chatbot arena\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 46595–46623\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/91f18a1287b398d378ef22505bf41832-Paper-Datasets_and_Benchmarks.pdf)Cited by:[§A\.3](https://arxiv.org/html/2609.06126#A1.SS3.p3.1),[§1](https://arxiv.org/html/2609.06126#S1.p5.1)\.
- Zuet al\.\(2021\)T\. Zu, J\. Munsell, and N\. S\. RebelloSubjective measure of cognitive load depends on participants’ content knowledge level\.InFrontiers in Education,Vol\.6,pp\. 647097\.Cited by:[§1](https://arxiv.org/html/2609.06126#S1.p3.1),[§4](https://arxiv.org/html/2609.06126#S4.p2.1)\.

## Appendix AMore Analysis

BaselineGPT\-5\.5DeepSeek\-V4\-ProQwen3\.6\-PlusQwen\-PlusDeepSeek\-V3\.2MiniMax\-M2\.5GPT\-4o\-miniAvg\.base3\.0923\.3513\.0052\.9963\.5852\.6513\.7523\.205Lora3\.1803\.8862\.9842\.9843\.9492\.8134\.0773\.410MoE3\.2723\.9373\.1793\.0593\.9412\.8604\.1333\.483qwen2\.5\-14B3\.3514\.2523\.5053\.2094\.0502\.7464\.1823\.614CWF3\.5764\.2213\.6333\.4514\.1243\.1544\.2423\.772CWF\-R3\.4564\.1943\.4573\.2694\.0453\.0644\.1873\.667

Table 8:Mean overall personalization scores of each baseline under different judge models\. Values are aggregated across all domains and audience groups for each baseline\. The last column reports the average score across judge models\.### A\.1More Personalization Judge Models\.

The results in Table[8](https://arxiv.org/html/2609.06126#A1.T8)show that the relative performance of systems is broadly stable across judge models\. CWF achieves the highest cross\-judge average \(3\.772\) and is ranked first by six of seven judges; Qwen2\.5\-14B surpasses it only under DeepSeek\-V4\-Pro\. Base is the weakest system at most time, while MoE and LoRA are closer and sometimes exchange order\. Because cross\-judge stability alone does not establish agreement with readers, we separately evaluate every judge against human scores below\.

### A\.2Judge–Human Alignment

We evaluate Base, LoRA, MoE, Qwen2\.5\-14B, CWF, and CWF\-Rwith seven judge models\. Every judge receives the same articles, reader personas, metric definitions, dimension weights, and output schema\. LetEEdenote system–reader\-group cells andPPunordered system pairs\. For judgejj, we compute

MAEj\\displaystyle\\mathrm\{MAE\}\_\{j\}=\|E\|−1​∑e∈E\|Se\(j\)−Se\(H\)\|,\\displaystyle=\|E\|^\{\-1\}\\\!\\sum\_\{e\\in E\}\|S\_\{e\}^\{\(j\)\}\-S\_\{e\}^\{\(H\)\}\|,\(9\)PairAccj\\displaystyle\\mathrm\{PairAcc\}\_\{j\}=\|P\|−1∑\(a,b\)∈P\[sgnΔa​b\(j\)=sgnΔa​b\(H\)\],\\displaystyle=\|P\|^\{\-1\}\\\!\\sum\_\{\(a,b\)\\in P\}\\mathbf\{1\}\\\!\\left\[\\operatorname\{sgn\}\\Delta\_\{ab\}^\{\(j\)\}=\\operatorname\{sgn\}\\Delta\_\{ab\}^\{\(H\)\}\\right\],whereΔa​b=Sa−Sb\\Delta\_\{ab\}=S\_\{a\}\-S\_\{b\}\. MAE measures score calibration\([Willmott and Matsuura, 2005](https://arxiv.org/html/2609.06126#bib.bib53)\); Spearman’sρ\\rhoand Kendall’sτb\\tau\_\{b\}measure rank association and tie\-corrected ordinal agreement\([Spearman, 1904](https://arxiv.org/html/2609.06126#bib.bib54);[Kendall, 1938](https://arxiv.org/html/2609.06126#bib.bib55)\); PairAcc measures preservation of human pairwise preferences\.

Judge modelMAE↓\\downarrowSpearmanρ\\rho↑\\uparrowKendallτb\\tau\_\{b\}↑\\uparrowPairAcc↑\\uparrowCWF rankQwen\-Plus0\.4170\.8290\.73386\.7%1Qwen3\.6\-Plus0\.3820\.9430\.86793\.3%1GPT\-5\.50\.4010\.8860\.80090\.0%1DeepSeek\-V4\-Pro0\.3610\.9140\.82291\.1%2DeepSeek\-V3\.20\.3950\.9000\.80090\.0%1MiniMax\-M2\.50\.4460\.8290\.73386\.7%1GPT\-4o\-mini0\.4280\.8570\.75687\.8%1Average0\.4040\.8800\.78789\.4%–Table 9:Alignment of each judge model with human evaluation over the six systems\.As Table[9](https://arxiv.org/html/2609.06126#A1.T9)shows, all seven judges preserve human preferences well\. Six rank CWF first; DeepSeek\-V4\-Pro ranks it second, 0\.031 below Qwen2\.5\-14B\. We also measure cross\-judge ranking concordance using Kendall’sWW\([Kendall and Babington Smith, 1939](https://arxiv.org/html/2609.06126#bib.bib56)\)\. Table[10](https://arxiv.org/html/2609.06126#A1.T10)shows high macro\-average concordance \(W=0\.832W=0\.832\), separating robustness across judges from direct judge–human alignment\.

Reader groupKendall’sWW↑\\uparrowInterpretationChild0\.902Very highTeen0\.811HighAdult0\.784SubstantialMacro average0\.832HighTable 10:Cross\-judge concordance of system rankings\.
### A\.3Score and Preference Agreement

ForBBsystems, MARD is the mean form of Spearman’s footrule distance\([Diaconis and Graham, 1977](https://arxiv.org/html/2609.06126#bib.bib57)\), while MAE operates on scores:

MARD\\displaystyle\\mathrm\{MARD\}=1B​∑b=1B\|rbLLM−rbH\|,\\displaystyle=\\frac\{1\}\{B\}\\sum\_\{b=1\}^\{B\}\|r\_\{b\}^\{\\mathrm\{LLM\}\}\-r\_\{b\}^\{H\}\|,\(10\)MAE\\displaystyle\\mathrm\{MAE\}=1B​∑b=1B\|SbLLM−SbH\|\.\\displaystyle=\\frac\{1\}\{B\}\\sum\_\{b=1\}^\{B\}\|S\_\{b\}^\{\\mathrm\{LLM\}\}\-S\_\{b\}^\{H\}\|\.Thus, MAE tests calibration and MARD tests rank displacement;ρ\\rho,τb\\tau\_\{b\}, and PairAcc further test association, tie\-corrected agreement, and pairwise consistency\.

ProtocolMAE↓\\downarrowMARD↓\\downarrowSpearmanρ\\rho↑\\uparrowKendallτb\\tau\_\{b\}↑\\uparrowPairAcc↑\\uparrowDynamic dimension weighting0\.4170\.6670\.8290\.73386\.7%Average dimension weighting0\.8370\.6670\.8290\.73386\.7%One\-step LLM scoring0\.9411\.0000\.7140\.46773\.3%Table 11:Qwen\-Plus–human agreement under three scoring protocols\.As shown in Table[11](https://arxiv.org/html/2609.06126#A1.T11), dynamic and average dimension weighting induce the same system ranking, but dynamic weighting lowers MAE from 0\.837 to 0\.417\. Both preserve 86\.7% of human pairwise preferences, compared with 73\.3% for one\-step scoring\. Table[12](https://arxiv.org/html/2609.06126#A1.T12)further shows that Qwen\-Plus preserves 93\.3% of child\-group preferences and 80\.0% for both teen and adult groups\. Treating human\-score differences below 0\.05 as ties raises macro agreement from 84\.4% to 88\.9%, indicating that most disagreements involve near\-tied systems\.

Groupρ\\rhoτb\\tau\_\{b\}Correct pairsPairAccChild0\.9430\.86714/1593\.3%Teen0\.7140\.60012/1580\.0%Adult0\.7140\.60012/1580\.0%Macro average0\.7900\.68938/4584\.4%Table 12:Qwen\-Plus–human ordinal agreement by reader group\.Finally, we directly test preferences on 30 held\-out topics per reader group\. Qwen\-Plus judges all 15 anonymized system pairs in both presentation orders \(900 decisions per group\) against the corresponding participant\-group preference\. Cohen’sκ\\kappacorrects for chance agreement\([Cohen, 1960](https://arxiv.org/html/2609.06126#bib.bib58)\), and swap consistency measures position sensitivity following[Zheng et al\. \(2023\)](https://arxiv.org/html/2609.06126#bib.bib15)\. Table[13](https://arxiv.org/html/2609.06126#A1.T13)reports 87\.8% overall accuracy,κ=0\.754\\kappa=0\.754, and 96\.7% left/right consistency\.

MetricChildTeenAdultOverallPairwise preference accuracy90\.6%85\.8%86\.9%87\.8%Cohen’sκ\\kappa0\.8070\.7160\.7390\.754Left/right consistency97\.1%96\.2%96\.8%96\.7%Evaluated decisions9009009002,700Table 13:Direct topic\-level preference agreement on held\-out topics\.RoundsS–P↑\\uparrowP–U↑\\uparrowVar\.↓\\downarrow11\.1251\.0500\.85221\.3401\.2580\.63531\.4891\.3930\.48241\.4921\.3980\.527Table 14:Effect of the maximum number of debate rounds\. S–P and P–U denote the support–partial and partial–unsupported confidence gaps, respectively; Var\. denotes average confidence variance\.
### A\.4Effect of Maximum Debate Rounds\.

To investigate the influence of the maximum number of debate rounds, we vary the number of rounds from 1 to 4 and evaluate the confidence separation between different claim labels\. As shown in Table[14](https://arxiv.org/html/2609.06126#A1.T14)and Figure[4](https://arxiv.org/html/2609.06126#A1.F4), increasing the number of debate rounds consistently improves the discrimination capability of the model before reaching three rounds\. Specifically, the support\-partial margin increases from 1\.125 at one round to 1\.489 at three rounds, corresponding to a relative improvement of 32\.4%\. Similarly, the partial\-unsupport margin improves from 1\.050 to 1\.393, yielding a 32\.7% increase\. Meanwhile, the average variance decreases from 0\.852 to 0\.482, indicating that the model produces more stable confidence estimates after multi\-round deliberation\. However, further increasing the maximum debate rounds from 3 to 4 only brings marginal gains\. The support\-partial margin increases by merely 0\.003, and the partial\-unsupport margin increases by 0\.005\. In contrast, the average variance slightly increases from 0\.482 to 0\.527\. This suggests that additional debate rounds may introduce redundant reasoning or noise rather than meaningful improvement\. Therefore, we set the maximum debate rounds to 3 in our main experiments, which achieves a favorable balance between discrimination performance and stability\.

ModelSup\.Par\.Unsup\.S–PP–UVar\.GPT\-5\.54\.5472\.7631\.1381\.7841\.6250\.458deepSeek V4 pro4\.4782\.7511\.1981\.7271\.5530\.463gpt\-4o\-mini4\.3172\.7081\.3121\.6091\.3960\.488deepSeek V3\.24\.3762\.7291\.2471\.6471\.4820\.502qwen\-3\.6\-plus4\.3472\.7261\.2311\.6211\.4950\.478qwen\-plus4\.2092\.7201\.3271\.4891\.3930\.482minimax\-m2\.54\.1312\.7141\.5171\.4171\.1970\.531

Table 15:Robustness across base models\.Sup\./Par\./Unsup\.: mean scores under supported / partially supported / unsupported settings;S–P/P–U: support–partial and partial–unsupport gaps;Var\.: mean variance\.![Refer to caption](https://arxiv.org/html/2609.06126v1/images/compare_rounds.png)Figure 4:Effect of Maximum Debate Rounds\.The figure illustrates how confidence margins and average variance evolve as the number of debate rounds increases from 1 to 4\.![Refer to caption](https://arxiv.org/html/2609.06126v1/images/model_robustness.png)Figure 5:Robustness across Different Base Models\.The confidence scores for supported, partially supported, and unsupported claims are compared across three backbone models\.
### A\.5Discriminative Ability across Base Models\.

To further evaluate the generalizability of the proposed method across different base models, we compare seven models in terms of their mean confidence scores on Support, Partial, and Unsupport claims\. We also analyze two adjacent\-label confidence margins, namely support\-partial and partial\-unsupport\. Instead of grouping the results by model names, this experiment focuses on comparing how different models respond to the same factuality label\. Therefore, Figure[5](https://arxiv.org/html/2609.06126#A1.F5)\(a\) organizes the grouped bar chart by label categories, where the Support, Partial, and Unsupport scores of all models are compared within their corresponding label groups\. This design makes the inter\-model differences under each label category more visually explicit\.

TypeConditionCWF\-RmechanismOutcome or limitationSuccessEvidence\-direction conflictThe Skeptic Agent identifies whether the retrieved evidence supports the opposite direction, and graph propagation incorporates the negative relation\.Surface\-relevant evidence is less likely to be incorrectly treated as factual support\.SuccessClaim–evidence relation mismatchThe graph models the relation between an atomic claim and its evidence instead of relying only on keyword or topic overlap\.Evidence that is relevant but does not entail the claim receives lower influence\.SuccessNumerical inconsistencyClaim decomposition isolates numerical statements so that their values can be verified independently\.Local numerical errors are less likely to be hidden within an otherwise plausible paragraph\.SuccessWeak agent consensusStance strength and evidence direction are propagated together rather than treating all agent votes equally\.Agreement based on weak or indirect evidence does not automatically produce high confidence\.SuccessOver\-generalized claimClaim decomposition separates broad statements into independently verifiable factual units\.Evidence supporting only a narrow conclusion is less likely to validate a broader claim\.FailureMissing retrievable evidenceThe graph receives insufficient external support and therefore assigns low confidence or triggers a conservative revision\.A correct but unverifiable claim may be removed or weakened\.FailureShared erroneous signalsMisleading evidence causes all agents to produce consistent but incorrect judgments, leaving no opposing or uncertainty signal\.Graph propagation cannot recover the correct conclusion when all available inputs are consistently wrong\.Table 16:Applicability conditions and failure modes of CWF\-R\. Success conditions contain detectable conflict or uncertainty signals that can be calibrated by graph propagation; failure conditions lack a reliable corrective signal\.As shown in Table[15](https://arxiv.org/html/2609.06126#A1.T15)and Figure[5](https://arxiv.org/html/2609.06126#A1.F5), all models consistently preserve the expected confidence ordering of Support \> Partial \> Unsupport\. This indicates that the proposed method can induce a stable label\-wise confidence hierarchy across different base models, rather than relying on a specific backbone\. Specifically, GPT\-5\.5 achieves the highest mean confidence score on Support claims \(4\.547\) and the lowest mean confidence score on Unsupport claims \(1\.138\), suggesting the strongest separation between supported and unsupported claims\. DeepSeek V4 Pro also shows strong performance, with a Support mean of 4\.478 and an Unsupport mean of 1\.198\. In contrast, qwen\-plus and minimax\-m2\.5 produce relatively higher confidence scores on Unsupport claims, reaching 1\.327 and 1\.517, respectively, indicating weaker suppression of unsupported claims\.

Figure[5](https://arxiv.org/html/2609.06126#A1.F5)\(b\) further presents the discrimination margins between adjacent factuality labels using a grouped bar chart\. The support\-partial margin measures the model’s ability to distinguish Support from Partial claims, while the partial\-unsupport margin reflects its ability to separate Partial from Unsupport claims\. GPT\-5\.5 achieves the largest margins on both dimensions, with 1\.784 for support\-partial and 1\.625 for partial\-unsupport, demonstrating the strongest sensitivity to label boundaries\. DeepSeek V4 Pro ranks second, with margins of 1\.727 and 1\.553, respectively\. DeepSeek V3\.2, qwen\-3\.6\-plus, and gpt\-4o\-mini also maintain clear label separation, although their margins are smaller than those of the two strongest models\. By contrast, qwen\-plus and minimax\-m2\.5 show relatively weaker discrimination, especially on the partial\-unsupport margin, where they only reach 1\.393 and 1\.197, respectively\. In terms of stability, GPT\-5\.5 obtains the lowest average variance \(0\.458\), followed by DeepSeek V4 Pro \(0\.463\), indicating that these two models provide not only stronger label separation but also more stable confidence estimation\. Qwen\-Plus has an average variance of 0\.482, while MiniMax\-M2\.5 has the highest variance \(0\.531\), suggesting that the latter produces less stable confidence estimates\. Overall, these results demonstrate that the proposed framework generalizes well across different base models\. While all models preserve the desired Support \> Partial \> Unsupport hierarchy, stronger backbones tend to yield larger adjacent\-label margins and lower variance\.

### A\.6Applicability and Failure Analysis

We summarize the main applicability conditions and failure modes of CWF\-Rin Table[16](https://arxiv.org/html/2609.06126#A1.T16)\.

Overall, CWF\-Ris most effective when the retrieved evidence and agent judgments contain detectable disagreement or uncertainty\. Graph propagation can calibrate these signals, but it cannot create reliable evidence or recover the correct conclusion when all available signals are absent or consistently incorrect\.

### A\.7Token Cost Analysis\.

We compare the token consumption of three methods: SAFE, Multi\-agent Debate, and Domain After Revised / CWF with 3 rounds\. As shown in Table[17](https://arxiv.org/html/2609.06126#A1.T17)and Figure[6](https://arxiv.org/html/2609.06126#A1.F6), SAFE requires the fewest tokens, with a total consumption of 15\.19K tokens\. However, as shown in previous performance comparisons, SAFE obtains lower overall factuality evaluation performance than the proposed CWF\-based method\. In contrast, the standard Multi\-agent Debate method introduces a much larger token overhead, mainly due to extensive prompt construction and multi\-agent interaction\.

SettingPromptResp\.TotalSAFE8967622015187Multi\-agent Debate85004453589539CWF\(3 rounds\)50355296853323

Table 17:Token usage \(prompt / response / total\)\.![Refer to caption](https://arxiv.org/html/2609.06126v1/images/token_cost.png)Figure 6:Token Usage across Fact\-Checking Methods\.Prompt, response, and total token consumption are compared for SAFE, Multi\-agent Debate, and CWF\.Compared with Multi\-agent Debate, Domain After Revised / CWF with 3 rounds significantly reduces token consumption\. Based on the additive token count, Multi\-agent Debate consumes 89\.54K tokens, while CWF with 3 rounds consumes 53\.32K tokens, resulting in a 40\.4% reduction in total token usage\. In terms of prompt tokens, CWF reduces the prompt cost from 85\.00K to 50\.36K, corresponding to a 40\.8% decrease\. The response token cost is also reduced from 4\.54K to 2\.97K\.

## Appendix BModel Settings

To improve the reproducibility of our experiments, we provide the implementation details of the agents used in the proposed framework\. We describe the model configuration used by different agents\.

### B\.1Agent Configuration

In our experiments, we use five agents: Summarize Agent, Trust Agent, Skeptic Agent, Leader Agent, and Rewrite Agent\.

Summarize Agent:Qwen\-3\.5\-Plus with a maximum output length of 3000 tokens\. It summarizes the discussion history and identifies factual errors, logical problems, misleading statements, and unsupported conclusions\. Its output is used as an intermediate factual feedback signal for the revision stage\.

Trust Agent, Skeptic Agent, and Leader Agent:Qwen\-3\.5\-Plus with a maximum output length of 1200 tokens\. These three agents are connected to the retrieval system\. For each agent call, the maximum number of retrieval actions is limited to 4\. The Trust Agent searches for reasonable supporting evidence and complementary explanations, the Skeptic Agent challenges unsupported or weakly supported claims, and the Leader Agent integrates the debate results to produce the final factual judgment\.

Rewrite Agent:Our fine\-tuned Qwen2\.5\-3B model\. It rewrites the original paragraph based on the discussion summary and factual debate results, while preserving the intended audience adaptation and writing style\.

### B\.2Reasoning Workflow

The factual verification process is conducted through a multi\-agent debate\. The debate runs for 3 rounds by default\. In each round, the Trust Agent, Skeptic Agent, and Leader Agent participate according to the Markov state transition described in Section[B](https://arxiv.org/html/2609.06126#A2)\. The debate stops early if all agents reach the same conclusion after a complete debate round\. Otherwise, the process continues until the maximum number of debate rounds is reached\. Each debate agent outputs its opinion, factual stance, and cited evidence in a structured format\. The stance is selected from\{1,0\.2,0,−0\.2,−1\}\\\{1,0\.2,0,\-0\.2,\-1\\\}, representing strong support, weak support, neutrality, weak opposition, and strong opposition, respectively\. After the debate, the Summarize Agent aggregates the discussion results and produces revision suggestions\. The Rewrite Agent then revises the original paragraph according to these suggestions\. This workflow allows the system to combine retrieval\-based factual verification, multi\-perspective reasoning, and controlled text revision\.

### B\.3Prompt Templates

#### B\.3\.1Main Prompts

TaskDescription:

Reviseascience\-orientedparagraphaccordingtoeditorialsuggestionswhilepreservingtoneandaudiencefit\.

InputSpecification:

\-Originalparagraph:\{text\}

\-Revisionsuggestion:\{suggestion\}

PromptTemplate:

\[SystemPrompt\]

Youareaprofessionalscienceeditor\.Yourjobistorevisetextpreciselywhilepreservingtheoriginalstyleasmuchaspossible\.

\[Task\]

Readtheoriginalparagraphandtherevisionsuggestion,thenproducearevisedversionoftheparagraph\.

\[RevisionRequirements\]

1\.Preservecontentthatisfactuallycorrectandclearlyexpressed\.

2\.Fixissuesidentifiedinthesuggestion,suchasunclearwording,logicalproblems,orinaccurateinformation\.

3\.Preservetheoriginalwritingstyle,tone,andintendedaudience\.

4\.Ensurethattherevisedparagraphiscoherentandnatural\.

5\.Keepthelengthclosetotheoriginal,normallywithin1\.2timestheoriginallength\.

6\.Donotaddnewfactsornewviewpointsthatdonotappearintheoriginalparagraphorthesuggestion\.

7\.Outputonlytherevisedparagraphandnothingelse\.

\[OutputFormat\]

<text\>Therevisedfullparagraph</text\>

\[UserInputTemplate\]

\[OriginalParagraph\]

\{text\}

\[RevisionSuggestion\]

\{suggestion\}

TaskDescription:

Convertacomplexstatementintooneorafewcheckableclaimswithpairedsemanticweights\.

InputSpecification:

\-Complexstatement:\{text\}

PromptTemplate:

\[SystemPrompt\]

Youareaclaim\-processingassistant\.Yourtaskistoturnaninputstatementintocheckableclaimunits\.

\[CorePrinciple\]

Donotsplitunlessnecessary\.Iftheoriginalstatementcanberepresentedasonecomplete,concise,andtruth\-evaluableproposition,outputexactlyone<claim\>\.

\[WhenSplittingIsAllowed\]

Splitintomultipleclaimsonlyifatleastoneofthefollowingholds:

1\.Theoriginalstatementcontainsmultipleindependentjudgmentswhosetruthvaluesmaydiffer\.

2\.Withoutsplitting,apremiseanditsconclusioncannotbeverifiedseparately\.

\[WhatToAvoid\]

1\.Donotsplitmerelytosoundmoreprofessional\.

2\.Donotsplitsynonymousreformulations,modifiers,orbackgroundfillerintoseparateclaims\.

3\.Ifsplittingisnecessary,donotoutputfourormoreclaims\.

\[ClaimRequirements\]

1\.Preservetheoriginalmeaning\.

2\.Donotaddnewinformation,data,orsources\.

3\.Each<claim\>shouldbeconciseandwritteninformallanguage\.

\[WeightRules\]

1\.Each<claim\>mustbefollowedbyamatching<weight\>\.

2\.Each<weight\>mustbeatwo\-decimalnumberbetween0and1\.

3\.Ifthereisonlyoneclaim,itsweightmustbe1\.00\.

4\.Thesumofallweightsmustequal1\.

\[OutputFormat\]

<claim\>…</claim\>

<weight\>…</weight\>

Outputonlyalternatingclaim\-weightpairs,withnoextratext\.

\[UserInputTemplate\]

Processthefollowingstatementaccordingtothesystemrequirements\.Bydefault,keepitasoneclaimunlesssplittingisnecessary\.Outputpaired<claim\>and<weight\>tagsonly:

\{text\}

#### B\.3\.2Debate Agent Prompts

TaskDescription:

Inthethree\-agentdebateframework,extendthepreviousopinionfromatrustingperspective\.

InputSpecification:

\-Currentclaimtext:\{claim\}

\-Previousopinionblock:\{previous\_text\}

PromptTemplate:

\[SystemPrompt\]

YouaretheTrustroleinathree\-agentdebatesystem\.

\[Input\]

Youwillreceive:

1\.thecurrentclaimtext;

2\.apreviousopinionfromanotheragent\.

\[Objective\]

Startingfromatrustingstancetowardthepreviousopinion,identifywhatisreasonableinitandextendtheanalysistoproduceyourownjudgment\.Ifretrievaltoolsareavailable,youmayusethemwhennecessary\.

\[Requirements\]

1\.Donotmerelyrepeatthepreviousopinion\.Advancethereasoningbeyondit\.

2\.Yourconclusionshouldbesupportedbyevidenceorwell\-groundedanalysis\.

3\.Keeptheresponseconciseandexplicit\.

\[CitationRule\]

1\.Ifretrievalisusedinthecurrentturn,citeitemsbybatch\-indexformatsuchas1\-1or2\-3\.

2\.Ifnoretrievalisused,output<cited\>None</cited\>\.

\[AllowedStanceValues\]

1,0\.2,0,\-0\.2,\-1

\[OutputFormat\]

<opinion\>Yoursupportiveexpansion</opinion\>

<stance\>1\|0\.2\|0\|\-0\.2\|\-1</stance\>

<cited\>Batch\-indexreferencesorNone</cited\>

\[UserInputTemplate\]

\[text\]

\{claim\}

\[previousopinion\]

\{previous\_text\}

Pleasereturnyoursupportivejudgmentintherequiredtagformat\.

TaskDescription:

Inthethree\-agentdebateframework,examinethepreviousopinionfromaskepticalperspective\.

InputSpecification:

\-Currentclaimtext:\{claim\}

\-Previousopinionblock:\{previous\_text\}

PromptTemplate:

\[SystemPrompt\]

YouaretheSkepticroleinathree\-agentdebatesystem\.

\[Input\]

Youwillreceive:

1\.thecurrentclaimtext;

2\.apreviousopinionfromanotheragent\.

\[Objective\]

Criticallyinspectthepreviousopinionforlogicalweaknesses,insufficientevidence,exaggeration,orpossiblemisinterpretation,andthenprovideyourownjudgment\.Ifretrievaltoolsareavailable,youmayusethemwhennecessary\.

\[Requirements\]

1\.Donotmerelyrestatethepreviousopinion\.Clearlyidentifywhatyouquestionandwhy\.

2\.Yourjudgmentshouldbesupportedbyevidenceorcarefulanalyticalreasoning\.

3\.Keeptheresponseconciseandexplicit\.

\[CitationRule\]

1\.Ifretrievalisusedinthecurrentturn,citeitemsbybatch\-indexformatsuchas1\-1or2\-3\.

2\.Ifnoretrievalisused,output<cited\>None</cited\>\.

\[AllowedStanceValues\]

1,0\.2,0,\-0\.2,\-1

\[OutputFormat\]

<opinion\>Yourskepticalcritique</opinion\>

<stance\>1\|0\.2\|0\|\-0\.2\|\-1</stance\>

<cited\>Batch\-indexreferencesorNone</cited\>

\[UserInputTemplate\]

\[text\]

\{claim\}

\[previousopinion\]

\{previous\_text\}

Pleasereturnyourskepticaljudgmentintherequiredtagformat\.

TaskDescription:

Inthethree\-agentdebateframework,integrateTrustandSkepticopinionsandissueafinaljudgment\.

InputSpecification:

\-Currentclaimtext:\{claim\}

\-Previousopinionsblock:\{opinions\_text\}

PromptTemplate:

\[SystemPrompt\]

YouaretheLeaderroleinathree\-agentdebatesystem\.TheothertworolesareTrustandSkeptic\.

\[Input\]

Youwillreceive:

1\.thecurrentclaimtext;

2\.thepreviousopinionsproducedbyTrustandSkeptic\.

\[Objective\]

Evaluatethestrengthsandweaknessesofbothsidesandproduceyourownfinaljudgment\.Ifretrievaltoolsareavailable,youmayusethemwhennecessary\.

\[Requirements\]

1\.Donotmerelysummarizethepreviousopinions\.Formanindependentconclusion\.

2\.Yourconclusionshouldbesupportedbyevidenceorclearreasoning\.

3\.Keeptheresponseconciseandexplicit\.

\[CitationRule\]

1\.Ifretrievalisusedinthecurrentturn,citeitemsbybatch\-indexformatsuchas1\-1or2\-3\.

2\.Ifnoretrievalisused,output<cited\>None</cited\>\.

\[AllowedStanceValues\]

1,0\.2,0,\-0\.2,\-1

\[OutputFormat\]

<opinion\>Yourfinaladjudication</opinion\>

<stance\>1\|0\.2\|0\|\-0\.2\|\-1</stance\>

<cited\>Batch\-indexreferencesorNone</cited\>

\[UserInputTemplate\]

\[text\]

\{claim\}

\[previousopinions\]

\{opinions\_text\}

Pleasereturnyourfinaljudgmentintherequiredtagformat\.

BaselineChildTeenAdultTotalBase30303090LoRA30303090MoE30303090Qwen2\.5\-14B30303090CWF30303090CWF\-R30303090Total180180180540Table 18:Distribution of completed questionnaires across audience groups and baselines\. Each participant evaluates all six baselines once\.GroupMeanAgeGender\(M/F\)ReadingFrequencyChild10\.4±1\.110\.4\\pm 1\.115 / 153\.2 / 5Teen15\.6±1\.315\.6\\pm 1\.314 / 163\.5 / 5Adult26\.8±6\.426\.8\\pm 6\.416 / 143\.8 / 5Table 19:Participant demographics\. Gender is reported as male/female, and reading frequency is measured on a five\-point scale\.StageChildTeenAdultTotalRecruited participants33333298Questionnaires received before exclusion197197191585Incomplete six\-system set1113Duplicate/identity check failure0112Reading\-time or invariant\-response exclusion2103Retained participants30303090Questionnaires per retained participant6666Retained questionnaires180180180540Table 20:Recruitment and response\-quality flow\. Exclusion counts are participant\-level except for questionnaires received and retained\.
#### B\.3\.3Fallback Prompt

TaskDescription:

Completethecurrentroletaskwhenonlineretrievalhasreacheditslimitandnomoretoolcallsareallowed\.

InputSpecification:

\-Originalrole\-specificsystemprompt

\-Retrievedevidencesummary:\{evidence\}

PromptTemplate:

\[RecoveryPrompt\]

\[System\]Thelimitforonlineretrievalcallshasbeenreached\.Youmustnotuseanymoreretrievalorwebtools\.

Completethecurrentroletaskusingonlythesuccessfullyreturnedretrievalsummariesbelow\.

Youroutputmuststrictlyfollowthetagformatrequiredbytheoriginalsystemprompt\.Donotincludeanyextraexplanation\.

In<cited\>,usethebatch\-indexformatthatmatchestheretrievalsummaries\.Ifnousableevidenceisavailable,outputNone\.

\{evidence\}

## Appendix CHuman Evaluation

DiagnosticResultMissing rating items0/21,600 \(0\.00%\)Out\-of\-range rating cells0/21,600 \(0\.00%\)Duplicate retained submissions0Median questionnaire time824 s5th–95th percentile of time361–1,742 sResponses retained540/585 \(92\.3%\)Table 21:Diagnostics for retained questionnaires\.MetricαK\\alpha\_\{K\}95% CIICC\(2,k\)\(2,k\)Meanrw​gr\_\{wg\}αC\\alpha\_\{C\}Cognitive Load0\.781\[0\.742, 0\.816\]0\.9730\.7920\.908Personalization Alignment0\.803\[0\.769, 0\.835\]0\.9760\.7730\.939Reader Attitude0\.793\[0\.756, 0\.827\]0\.9750\.7590\.915Overall personalization0\.812\[0\.779, 0\.841\]0\.9770\.7940\.948Table 22:Metric\-level human\-evaluation reliability\.MetricDimensionαK\\alpha\_\{K\}95% CIICC\(2,k\)\(2,k\)Meanrw​gr\_\{wg\}CLINTR: Intrinsic Fit0\.768\[0\.719, 0\.811\]0\.9710\.787CLEXTR: Extraneous Burden Control0\.741\[0\.689, 0\.787\]0\.9660\.774CLGERM: Germane Support0\.806\[0\.765, 0\.842\]0\.9760\.815PACONT: Content Relevance0\.779\[0\.734, 0\.819\]0\.9720\.770PAKNOW: Knowledge\-Level Fit0\.824\[0\.786, 0\.856\]0\.9790\.792PASTYLE: Style Consistency0\.752\[0\.701, 0\.797\]0\.9680\.755PACONTX: Contextual Resonance0\.795\[0\.753, 0\.832\]0\.9750\.775RAENG: Engagement Appeal0\.788\[0\.744, 0\.826\]0\.9740\.758RATRU: Trust and Credibility0\.816\[0\.777, 0\.850\]0\.9780\.780RACONTI: Continuance Intention0\.759\[0\.710, 0\.802\]0\.9690\.739Table 23:Dimension\-level inter\-participant agreement and reliability of participant\-group means\.### C\.1Participant Statistics and Response Distribution

We conduct anonymized human evaluation with three age\-constrained audience groups:child,teen, andadult\. Each audience group includes 30 participants\. Each participant completes 6 questionnaire instances, corresponding to the 6 baselines reported in Table[18](https://arxiv.org/html/2609.06126#A2.T18)\. Therefore, we collect 180 completed questionnaires for each audience group and 540 completed questionnaires in total\. Participant demographics is shown in Table[19](https://arxiv.org/html/2609.06126#A2.T19)\.

All questionnaires use the same metric and dimension definitions as the LLM\-based evaluation\. Participants first read the article assigned to the current baseline and then answer the corresponding questionnaire\. All rating items use the same 0–5 integer scale\. When a dimension is measured by multiple questionnaire items, the corresponding dimension score is obtained by averaging the item ratings before applying the same dynamic dimension weighting and metric aggregation procedure as in the LLM\-based evaluation\.

All participants provided informed consent for the use of their anonymized responses for research purposes; guardian consent was obtained for minors\.

### C\.2Response\-Quality Control

Participants complete demographic and consent fields, with guardian consent required for minors\. System identities are hidden, and the six presentation orders are counterbalanced across participants\. Before inspecting system labels, we require all 40 ratings to be present and within 0–5, remove duplicates by anonymized respondent ID, flag completion times below one third of the median for the reader group, exclude complete straight\-line responses, and inspect excessive disagreement between positive and reverse\-worded items\. Using several complementary indicators follows established recommendations for screening careless survey responses\([Meade and Craig, 2012](https://arxiv.org/html/2609.06126#bib.bib45);[Curran, 2016](https://arxiv.org/html/2609.06126#bib.bib46)\)\. Table[20](https://arxiv.org/html/2609.06126#A2.T20)summarizes the recruitment and exclusion flow, while Table[21](https://arxiv.org/html/2609.06126#A3.T21)reports diagnostics for the retained data, which contain no missing, out\-of\-range, or duplicate responses\.

The long\-string index isLSi=max⁡∑jc⁡𝟏​\(xi​j=c\)\\mathrm\{LS\}\_\{i\}=\\max\_\{c\}\\sum\_\{j\}\\mathbf\{1\}\(x\_\{ij\}=c\), soLSi=40\\mathrm\{LS\}\_\{i\}=40denotes a complete straight\-line response\. The time flag isti<t~g/3t\_\{i\}<\\widetilde\{t\}\_\{g\}/3, wheret~g\\widetilde\{t\}\_\{g\}is the median for reader groupgg\. The reverse\-item diagnostic isrrev=corr⁡\(x¯\+,5−x¯−\)r\_\{\\mathrm\{rev\}\}=\\operatorname\{corr\}\(\\bar\{x\}\_\{\+\},5\-\\bar\{x\}\_\{\-\}\)\. Screening retains 92\.3% of questionnaires\. Because all rules are applied before system labels are examined, they cannot selectively favor CWF\. As a sensitivity check, omitting the time and reverse\-consistency exclusions changes every system’s overall human score by at most 0\.041 and leaves the top\-three ordering unchanged\.

### C\.3Human\-Evaluation Agreement

GroupCLPARAOverallChild0\.7540\.7730\.7610\.785Teen0\.7890\.8110\.8020\.819Adult0\.8010\.8260\.8150\.833Table 24:Metric\-level ordinal Krippendorff’sαK\\alpha\_\{K\}by reader group\.We reverse\-code Q1, Q5, Q15, Q18, Q23, Q28, Q32, Q36, and Q40 and average items within their intended dimensions\. Agreement is computed over common reader\-group×\\timesdomain×\\timessystem units\. We report ordinal Krippendorff’sαK\\alpha\_\{K\}, which supports multiple raters and missing ratings\([Krippendorff, 2004](https://arxiv.org/html/2609.06126#bib.bib47);[Hayes and Krippendorff, 2007](https://arxiv.org/html/2609.06126#bib.bib48)\); ICC\(2,k\)\(2,k\)for absolute\-agreement reliability of the participant\-group mean\([Shrout and Fleiss, 1979](https://arxiv.org/html/2609.06126#bib.bib49)\); mean within\-unitrw​gr\_\{wg\}\([James et al\., 1984](https://arxiv.org/html/2609.06126#bib.bib50)\); and Cronbach’sαC\\alpha\_\{C\}for internal consistency rather than inter\-rater agreement\([Cronbach, 1951](https://arxiv.org/html/2609.06126#bib.bib51)\)\. Specifically,

αK\\displaystyle\\alpha\_\{K\}=1−Do/De,\\displaystyle=1\-D\_\{o\}/D\_\{e\},\(11\)ICC⁡\(2,k\)\\displaystyle\\mathrm\{ICC\}\(2,k\)=MSU−MSEMSU\+\(MSP−MSE\)/U,\\displaystyle=\\frac\{\\mathrm\{MS\}\_\{U\}\-\\mathrm\{MS\}\_\{E\}\}\{\\mathrm\{MS\}\_\{U\}\+\(\\mathrm\{MS\}\_\{P\}\-\\mathrm\{MS\}\_\{E\}\)/U\},rw​g\\displaystyle r\_\{wg\}=1−sx2/\(35/12\)\.\\displaystyle=1\-s\_\{x\}^\{2\}/\(35/12\)\.Here,DoD\_\{o\}andDeD\_\{e\}are observed and expected ordinal disagreement;MSU\\mathrm\{MS\}\_\{U\},MSP\\mathrm\{MS\}\_\{P\}, andMSE\\mathrm\{MS\}\_\{E\}are the unit, participant, and residual mean squares; andUUis the number of evaluation units\. Confidence intervals use 5,000 bootstrap resamples of evaluation units\([Efron, 1979](https://arxiv.org/html/2609.06126#bib.bib52)\)\. Metric\- and dimension\-level results are reported in Tables[22](https://arxiv.org/html/2609.06126#A3.T22)and[23](https://arxiv.org/html/2609.06126#A3.T23), respectively\.

Across dimensions,αK\\alpha\_\{K\}ranges from 0\.741 to 0\.824 and ICC\(2,k\)\(2,k\)from 0\.966 to 0\.979\. Knowledge\-Level Fit and Trust/Credibility have the highest agreement; Extraneous Burden Control and Style Consistency are more subjective but remain reliable\. Agreement also remains substantial within every reader group \(Table[24](https://arxiv.org/html/2609.06126#A3.T24)\), including the more variable child group \(αK=0\.785\\alpha\_\{K\}=0\.785overall\)\.

ScoreMeaning0completely disagree / completely not applicable1mostly disagree2slightly disagree3neutral / hard to judge4mostly agree5completely agree / perfectly applicableTable 25:Response scale used in the human evaluation questionnaire\.As a robustness check, we bootstrap participants within each evaluation unit 5,000 times\. The LLM–human Spearman correlation remains positive in 99\.2% of resamples, and CWF remains among the top two systems in 96\.8%\. These agreement coefficients and resampling results support the stability of the participant\-group means used as human evaluation scores\.

### C\.4Questionnaire Template and Item Inventory

We implement the human evaluation questionnaire on Credamo\. The questionnaire contains four parts: basic information, Cognitive Load, Personalization Alignment, and Reader Attitude\. The example below is a paper\-formatted version of the questionnaire content parsed from the actual survey form\. The response scale is shown in Table[25](https://arxiv.org/html/2609.06126#A3.T25)\. The questionnaire is shown in Table[26](https://arxiv.org/html/2609.06126#A3.T26)\.

Participant Instruction\.Before answering the rating items, participants are shown the instruction:

> Thank you for participating in this study\. Please first complete the demographic questions, then read the science\-popularization article assigned in the current round, and finally answer the questionnaire based on your actual reading experience\. All rating questions use a 0–5 integer scale, where 0 means “completely disagree / completely not applicable” and 5 means “completely agree / perfectly applicable\.” There are no right or wrong answers\.

PartDimensionQuestionnaire ItemsABasic InformationQ1: Age\. Q2: Gender\. Q3: Highest education level\. Q4: Major field of study \(if applicable\)\. Q5: Whether the participant frequently reads science\-popularization articles\. Q6: Main channels for obtaining science information \(multiple choice\)\.BB1: INTRQ1: Understanding the topic of this article requires substantial mental effort from me\. Q2: The overall difficulty of this article is appropriate for me\. Q3: The amount of information in this article is just right for me, neither too much nor too little\. Q4: I gained a considerable amount of new knowledge from reading this article\.B2: EXTRQ5: To understand the concepts or terms in the article, I need extra thinking or inference\. Q6: The paragraph structure and logical order of the article make it easy for me to follow\. Q7: The wording and sentence expression of the article make it smooth to read\. Q8: The article is concise and focused, without redundant or off\-topic content\.B3: GERMQ9: The article helps me understand the key knowledge points through effective devices such as analogies, examples, or summaries\. Q10: The explanations and examples provided in the article are sufficient for me to retell its core content\. Q11: The article noticeably deepens and expands my understanding of the topic\. Q12: Through appropriate summarization and organization, the article helps me form a clear overall mental picture of the topic\.CC1: CONTQ13: The scientific aspects emphasized in the article are exactly the parts I care about or am interested in\. Q14: The perspective and content selected by the article are practically meaningful to me\. Q15: The article contains a large amount of content that feels irrelevant to me\. Q16: The content direction covered by the article is highly consistent with what I genuinely want to know\.C2: KNOWQ17: The explanation depth and difficulty level of the article fit my current knowledge level\. Q18: The article spends too much space explaining things I already know, making the information level lower than I expected\. Q19: The information density of the article is just right for me\. Q20: I am satisfied with how thoroughly the article explains the key concepts\.C3: STYLEQ21: The language style of the article, including vocabulary and tone, feels natural and approachable to me\. Q22: The organizational structure of the article makes it easy for me to follow the author’s line of thought\. Q23: The tone of the article makes me uncomfortable, for example because it is too academic, too childish, or too preachy\. Q24: The overall pacing of the article matches my reading habits\.C4: CONTXQ25: The examples in the article come from everyday situations familiar to me\. Q26: The analogies used in the article feel relatable and help me understand abstract concepts\. Q27: The article makes me feel that the scientific knowledge is connected to my own life\. Q28: The examples or scenarios in the article feel unfamiliar to me and make it hard to relate to the content\.DD1: ENGQ29: The beginning of the article quickly captured my attention\. Q30: Throughout the reading process, I remained curious about the content\. Q31: The article is vivid and interesting rather than dull\. Q32: While reading, I felt like skipping some paragraphs at several points\.D2: TRUQ33: I believe the scientific knowledge presented in the article is reliable\. Q34: The article maintains appropriate caution when discussing uncertain content and does not exaggerate\. Q35: While conveying scientific knowledge, the article does not make me feel forced to accept its claims\. Q36: Some expressions in the article make me doubt its scientific accuracy\.D3: CONTIQ37: After reading, I became more interested in learning further about this topic\. Q38: I would be willing to continue reading similar articles on this topic in the future\. Q39: I would be willing to recommend this article to friends with backgrounds similar to mine\. Q40: After reading this article, I feel there is no need to further learn about this topic\.

Table 26:Paper\-formatted version of the Credamo questionnaire used in human evaluation\. Negatively worded items are reverse\-coded during score aggregation\.Reverse\-Coded Items\.To ensure directional consistency in score aggregation, negatively worded items are reverse\-coded before computing dimension scores\. In the current questionnaire, reverse\-coded items include Q1, Q5, Q15, Q18, Q23, Q28, Q32, Q36, and Q40\.

Questionnaire\-to\-Metric Mapping\.The questionnaire is designed to align directly with the PSCB metrics and dimensions\. Items Q1–Q12 correspond to Cognitive Load, items Q13–Q28 correspond to Personalization Alignment, and items Q29–Q40 correspond to Reader Attitude\. Dimension scores are obtained by averaging the corresponding item ratings, after which the same dynamic weighting mechanism as in the LLM\-based evaluation is applied\.

### C\.5Ethics Statement

All participants provided informed consent for the use of their anonymized responses for research purposes; guardian consent was obtained for minors\. Each participant received compensation of RMB 17\.50 for their participation\. The human evaluation protocol was reviewed and approved by the Medical Ethics Committee of Tongji Medical College of Huazhong University of Science and Technology\.

## Appendix DMetrics and Dimensions

Detailed definitions of PSCB personalization metrics and dimensions is shown in Table[27](https://arxiv.org/html/2609.06126#A5.T27)\.

## Appendix ESignificance Tests

### E\.1Personalization

We aggregate the three scientific domains and retain only topics evaluated for all three systems\. Following paired testing practice for common NLP test instances, we first conduct a Friedman test across CWF, Qwen2\.5\-14B, and CWF\-R\. The omnibus differences are significant for child, teen, and adult readers\.

All nine pairwise comparisons remain statistically significant after correction \(pHolm<0\.05p\_\{\\mathrm\{Holm\}\}<0\.05\)\. CWF outperforms Qwen2\.5\-14B across all reader groups, whereas CWF\-Ris consistently lower than CWF\. Compared with Qwen2\.5\-14B, CWF\-Ris higher for teen and adult readers but lower for child readers, indicating an audience\-dependent personalization–reliability trade\-off\.

Table[28](https://arxiv.org/html/2609.06126#A5.T28)reports the adjusted results without repeating the method means already presented in Table[4](https://arxiv.org/html/2609.06126#S5.T4)\.

### E\.2Fact\-checking

For each domain–reader condition, we compare the article\-level factual accuracy scores of CWF\-Rand CWF on the same evaluation topics\. For each matched topicii, we define the paired difference asdi=Si,CWF​\-R−Si,CWFd\_\{i\}=S\_\{i,\\mathrm\{CWF\\textsubscript\{\-R\}\}\}\-S\_\{i,\\mathrm\{CWF\}\}and conduct a two\-sided pairedtt\-test of the null hypothesisH0:𝔼⁡\[di\]=0H\_\{0\}:\\mathbb\{E\}\[d\_\{i\}\]=0\. Thus, a positivett\-statistic indicates higher factual accuracy after revision\.

Table[29](https://arxiv.org/html/2609.06126#A5.T29)reports the unadjustedpp\-values\. CWF\-Robtains significantly higher factual accuracy in seven of the nine domain–reader conditions at the conventionalp<0\.05p<0\.05level\. The estimated differences are also positive for AI–Child \(p=0\.0782p=0\.0782\) and Medicine–Child \(p=0\.0941p=0\.0941\), but neither reaches the conventional significance threshold\.

MetricItemDefinitionCLOverallCognitive Loadevaluates whether a popular\-science article imposes an appropriate level of cognitive demand on the target reader\. The goal is not to minimize difficulty indiscriminately, but to calibrate complexity to the reader’s cognitive level: a good article should neither overwhelm the reader with excessive conceptual difficulty nor oversimplify the content to the point of losing educational value\.INTRIntrinsic Fitmeasures whether the conceptual difficulty, information density, and prerequisite assumptions of the article are appropriate for the target persona\. High\-scoring articles present the right amount of scientific content for the reader’s prior knowledge and cognitive level, without being either overly technical or trivially shallow\.EXTRExtraneous Burden Controlmeasures whether the article minimizes unnecessary cognitive burden caused by poor wording, confusing structure, unexplained jargon, redundant details, or abrupt transitions\. High\-scoring articles are easy to follow and do not impose avoidable processing costs unrelated to the scientific content itself\.GERMGermane Supportmeasures whether the article actively facilitates understanding through pedagogically helpful devices, such as analogies, examples, step\-by\-step explanation, local summaries, and concept scaffolding\. High\-scoring articles do not merely simplify content; they help the reader construct useful mental schemas for understanding the topic\.PAOverallPersonalization Alignmentevaluates whether the generated article is genuinely tailored to the target reader, rather than merely being generally readable or superficially simplified\. In PSCB, personalization includes not only tone adaptation, but also whether the article selects suitable content, explanation strategies, and contextual examples for a specific persona\.CONTContent Relevancemeasures whether the article selects and emphasizes scientific content that is relevant to the target persona’s interests, goals, and likely concerns\. High\-scoring articles focus on aspects of the topic that matter to the intended reader, rather than presenting generic information indiscriminately\.KNOWKnowledge\-Level Fitmeasures whether the explanation depth, abstraction level, and technical granularity match the target persona’s educational background and expected prior knowledge\. High\-scoring articles explain enough to be informative without assuming too much or talking down to the reader\.STYLEStyle Consistencymeasures whether the article’s language style, vocabulary, sentence rhythm, and explanatory tone are consistent with the target persona\. For example, writing for children should use simpler wording and more vivid explanations, whereas writing for adult readers may allow denser reasoning and more precise terminology\.CONTXContextual Resonancemeasures whether the article uses examples, analogies, scenarios, or applications that resonate with the reader’s everyday life or likely experiences\. High\-scoring articles make the science feel personally meaningful and easier to relate to\.RAOverallReader Attitudeevaluates the target reader’s likely affective and behavioral response to the generated article\. In personalized science communication, success is determined not only by whether the article is understandable, but also by whether the reader feels interested, trusts the explanation, and is willing to continue engaging with the content\.ENGEngagement Appealmeasures whether the article is interesting, vivid, and capable of sustaining the reader’s attention\. High\-scoring articles avoid dry or overly mechanical exposition and instead maintain the reader’s curiosity throughout the text\.TRUTrust and Credibilitymeasures whether the article gives the reader a sense of reliability, seriousness, and epistemic confidence\. High\-scoring articles present knowledge in a way that appears careful, balanced, and scientifically grounded, thereby increasing reader trust\.CONTIContinuance Intentionmeasures whether the article encourages the reader to keep reading, explore related content, or further engage with the topic\. High\-scoring articles leave the reader with the impression that continuing to learn from this source would be worthwhile\.

Table 27:Detailed definitions of PSCB personalization metrics and dimensions\. CL denotes Cognitive Load, PA denotes Personalization Alignment, and RA denotes Reader Attitude\.GroupComparisonpHolmp\_\{\\mathrm\{Holm\}\}\|dz\|\|d\_\{z\}\|Sig\.ChildCWF vs Qwen2\.5\-14B0\.00600\.230\*\*CWF\-Rvs Qwen2\.5\-14B0\.00160\.283\*\*CWF\-Rvs CWF<0\.0001<0\.00010\.381\*\*\*TeenCWF vs Qwen2\.5\-14B0\.00320\.275\*\*CWF\-Rvs Qwen2\.5\-14B0\.01600\.220\*CWF\-Rvs CWF0\.01600\.210\*AdultCWF vs Qwen2\.5\-14B<0\.0001<0\.00010\.509\*\*\*CWF\-Rvs Qwen2\.5\-14B0\.00900\.220\*\*CWF\-Rvs CWF0\.00090\.299\*\*\*Table 28:Paired significance tests for LLM personalization scores after aggregating across scientific domains\.pHolmp\_\{\\mathrm\{Holm\}\}denotes the two\-sided paired\-randomizationpp\-value after Holm correction within each reader group;\|dz\|\|d\_\{z\}\|is the magnitude of the paired standardized effect\. Significance codes are \*p<0\.05p<0\.05, \*\*p<0\.01p<0\.01, \*\*\*p<0\.001p<0\.001, and nsp≥0\.05p\\geq 0\.05\. Method means are omitted because they are reported in Table[4](https://arxiv.org/html/2609.06126#S5.T4)\.DomainGroupComparisontt\-statisticpp\-valueSig\.AIAdultCWF\-Rvs CWF2\.1450\.0425\*TeensCWF\-Rvs CWF3\.2100\.0035\*\*ChildCWF\-Rvs CWF1\.8560\.0782†\\daggerBiologyAdultCWF\-Rvs CWF3\.9420\.0008\*\*\*TeensCWF\-Rvs CWF2\.4060\.0242\*ChildCWF\-Rvs CWF3\.0930\.0051\*\*MedicineAdultCWF\-Rvs CWF2\.8820\.0084\*\*TeensCWF\-Rvs CWF3\.2510\.0032\*\*ChildCWF\-Rvs CWF1\.7650\.0941†\\daggerTable 29:Two\-sided pairedtt\-tests comparing article\-level factual accuracy between CWF\-Rand CWF on matched evaluation topics\. Positivett\-statistics indicate higher factual accuracy after revision\. Reportedpp\-values are unadjusted\. Significance codes are \*p<0\.05p<0\.05, \*\*p<0\.01p<0\.01, and \*\*\*p<0\.001p<0\.001;†\\daggerdenotesp<0\.1p<0\.1but not conventional statistical significance\.