PASTA: A Paraphrasing And Self-Training Approach for Knowledge Updating in LLMs

# PASTA: A Paraphrasing And Self-Training Approach for Knowledge Updating in LLMs
Source: [https://arxiv.org/html/2606.28898](https://arxiv.org/html/2606.28898)
Daisuke Kawahara Waseda University dkw@waseda\.jp

###### Abstract

Knowledge updating in pre\-trained Large Language Models \(LLMs\) remains an important challenge\. While continual training provides a potential avenue for knowledge updating, it continues to present substantial technical difficulties\. Furthermore, LLMs often struggle with accurately answering questions about specific factual information, such as news articles—a capability limitation widely recognized in the research community\. This paper proposes PASTA, a simple yet powerful framework for integrating detailed factual information from news articles as new knowledge into LLMs, with the primary goal of building specialized models that accurately answer questions about this knowledge\. Our framework combines data augmentation, question\-answering generation, and a novel self\-learning DPO process that simultaneously enables knowledge overwriting and hallucination suppression\. We provide insights into effective knowledge updating through systematic analysis of learning parameters and data configurations\. In our experimental evaluation with web articles published after the base model’s knowledge cutoff, PASTA achieved remarkable improvement from 0\.02 to 0\.82 accuracy while maintaining general language capabilities, demonstrating its effectiveness for creating domain\-specialized LLMs\.

## 1Introduction

Updating pre\-trained Large Language Models \(LLMs\) with new knowledge and training them to respond accurately based on those facts is recognized as a significant challenge\[[25](https://arxiv.org/html/2606.28898#bib.bib25),[11](https://arxiv.org/html/2606.28898#bib.bib11)\]\. While approaches such as Retrieval\-Augmented Generation \(RAG\)\[[13](https://arxiv.org/html/2606.28898#bib.bib13)\]have been researched as a method to provide new knowledge to language models, this paper focuses on methods for direct internal knowledge updating, aiming to build LLMs capable of correctly answering questions about newly integrated information\. Through our proposed approach, we demonstrate that an LLM which initially showed only 0\.02 accuracy when answering questions about unknown knowledge can achieve up to0\.82 accuracyafter knowledge updating\. Previous research has suggested that diverse training data is closely related to accurate knowledge retrieval by probing internal representations of LLMs\[[1](https://arxiv.org/html/2606.28898#bib.bib1)\]\. However, this research has only examined cases where relatively small\-scale models\[[19](https://arxiv.org/html/2606.28898#bib.bib19)\]were trained from scratch, without sufficiently investigating additional training with large language models as initial parameters\. On the other hand, methods have been proposed to build specialized LLMs for specific tasks \(e\.g\., programming\) through additional training using large corpora\[[21](https://arxiv.org/html/2606.28898#bib.bib21)\]\. These approaches supplement relatively broad and static knowledge \(programming languages, algorithms, etc\.\), but challenges remain in developing methods for LLMs to learn and accurately answer questions about specific, detailed factual information such as news articles\.

This paper proposes PASTA, a simple yet powerful framework for knowledge updating of small\-scale detailed factual information, such as news articles, on a pre\-trained large language model through continual learning\. While each individual component builds upon established techniques, their systematic combination creates a novel approach that significantly outperforms baseline methods\. The simplicity of PASTA’s design makes it both practical to implement and theoretically interpretable, while demonstrating substantial performance gains in knowledge updating tasks\.

Drawing inspiration from recent self\-learning approaches\[[14](https://arxiv.org/html/2606.28898#bib.bib14),[26](https://arxiv.org/html/2606.28898#bib.bib26)\], we combine the following four steps: \(1\) data augmentation\[[1](https://arxiv.org/html/2606.28898#bib.bib1),[2](https://arxiv.org/html/2606.28898#bib.bib2)\], \(2\) question\-answering \(QA\) data generation\[[14](https://arxiv.org/html/2606.28898#bib.bib14)\], \(3\) self\-preference optimization\[[26](https://arxiv.org/html/2606.28898#bib.bib26)\], and \(4\) iterative learning\[[14](https://arxiv.org/html/2606.28898#bib.bib14)\]\. We present a reconstructed additional learning flow that integrates these components and experimentally confirm that new knowledge can be gradually incorporated\.

## 2Related Work

### 2\.1Preference Optimization

Representative methods for optimizing LLMs to align with human preferences include Reinforcement Learning from Human Feedback \(RLHF\)\[[17](https://arxiv.org/html/2606.28898#bib.bib17),[3](https://arxiv.org/html/2606.28898#bib.bib3)\]and Direct Preference Optimization \(DPO\)\[[20](https://arxiv.org/html/2606.28898#bib.bib20)\]\. Both methods share the common approach of presenting two output candidates from the model and labeling them as “chosen” or “rejected”\. However, RLHF involves training a reward model followed by reinforcement learning, while DPO directly optimizes parameters without explicitly training a reward model\.

### 2\.2Self\-Learning

Large\-scale human annotation is costly, and when it includes sensitive user information, privacy concerns also arise\. Therefore, self\-learning approaches, where the model generates and evaluates its own data for iterative learning, have gained attention\. For instance,Li et al\.\[[14](https://arxiv.org/html/2606.28898#bib.bib14)\]reported improved model performance by automatically generating diverse QA data for retraining\.Yuan et al\.\[[26](https://arxiv.org/html/2606.28898#bib.bib26)\]proposed ”self\-DPO learning” within the DPO framework\[[20](https://arxiv.org/html/2606.28898#bib.bib20)\], where models learn by distinguishing between high\-scoring and low\-scoring responses generated by the model itself\.Wang et al\.\[[24](https://arxiv.org/html/2606.28898#bib.bib24)\]introduced mechanisms for LLMs to autonomously conduct pairwise comparisons\.Lee et al\.\[[12](https://arxiv.org/html/2606.28898#bib.bib12)\]explored using LLMs as partial substitutes for human annotation\. On the other hand,Madaan et al\.\[[15](https://arxiv.org/html/2606.28898#bib.bib15)\]andWang et al\.\[[23](https://arxiv.org/html/2606.28898#bib.bib23)\]have shown that performance may degrade under certain conditions when models are repeatedly retrained solely on self\-generated data, highlighting the importance of appropriate data diversity and validation\.

### 2\.3Data Augmentation

While large volumes of data are effective for improving LLM performance, introducing diversity into the data is also beneficial beyond simple scaling\.Allen\-Zhu and Li\[[1](https://arxiv.org/html/2606.28898#bib.bib1)\]revealed that generating diverse paraphrases for training can better integrate new knowledge, and that using relatively high LoRA\[[10](https://arxiv.org/html/2606.28898#bib.bib10)\]ranks \(r=128r=128\) can also improve performance\.Brown et al\.\[[2](https://arxiv.org/html/2606.28898#bib.bib2)\]also reported that creating multiple variations for a single piece of data and training repeatedly significantly improves reasoning accuracy\.

## 3Proposed Method

![Refer to caption](https://arxiv.org/html/2606.28898v1/x1.png)Figure 1:PASTA framework architecture\. CPT: Continual Pre\-Training, SFT: Supervised Fine\-Tuning, DPO: PASTA’s specialized Direct Preference Optimization with Target LLM inference \(Gen\)\.In this paper, we propose a Paraphrasing And Self\-Training Approach \(PASTA\) as a framework for effectively integrating new knowledge into pre\-trained large language models\. PASTA consists of four key components: \(1\) data augmentation through paraphrasing, \(2\) question\-answering data generation, \(3\) self\-preference optimization, and \(4\) iterative learning based on fundamental LLM development principles\. As shown in Figure[1](https://arxiv.org/html/2606.28898#S3.F1), it forms an integrated framework that iteratively enhances an LLM’s ability to accurately answer questions about new knowledge that has been integrated through knowledge updating\.

### 3\.1Design Motivations

#### Diversifying Training Data through Paraphrasing

LLMs have a structurally redundant architecture where a single meaning is not represented by specific parameters alone, but rather by multiple parameter sets throughout the model\[[8](https://arxiv.org/html/2606.28898#bib.bib8),[6](https://arxiv.org/html/2606.28898#bib.bib6)\]\. Therefore, when integrating new knowledge, simply repeating the same text may concentrate parameter updates in specific areas, failing to update all necessary parameters in the redundant structure, potentially making knowledge updating ineffective\. We hypothesize that learning with diverse paraphrases of the target text—maintaining the same meaning but varying the expression—will update all necessary parameters across the redundant structure, making knowledge updating in LLMs more effective\.

#### Scaling Question\-Answering

When updating knowledge in LLMs, there is difficulty not only in directly updating the knowledge itself but also in propagating updates to related knowledge\. For example, when a music artist releases a new album, the total number of albums they have released also increases\[[9](https://arxiv.org/html/2606.28898#bib.bib9)\]\. To address these challenges, we need to learn answers to diverse questions about the new knowledge and related topics from multiple angles\. We hypothesize that scaling such question\-answering will improve the accuracy of knowledge updating\.

#### Hallucination Suppression Using DPO

LLMs sometimes produce hallucinations, presenting non\-existent information as if it were factual\[[5](https://arxiv.org/html/2606.28898#bib.bib5)\]\. Particularly for detailed factual information such as news articles, where the correct answer to the same question may change over time \(e\.g\., a president’s name differs depending on the time period\), there is a risk of generating incorrect answers using outdated knowledge from before knowledge updating\. Therefore, we hypothesize that by using DPO to identify outputs based on the LLM’s inherent knowledge \(old information\) as ”rejected” responses and newly added knowledge as ”chosen” responses, we can suppress inaccurate outputs\.

#### Pre\-training \+ SFT \+ DPO Framework

A widely adopted three\-stage learning pipeline for LLM development includes \(1\) pre\-training, \(2\) SFT, and \(3\) final adjustment to human preferences \(using methods like RLHF or DPO\)\[[17](https://arxiv.org/html/2606.28898#bib.bib17)\]\. In this work, we aim to achieve effective knowledge updating and build an LLM capable of accurately answering questions about that content by integrating the above hypotheses into a system based on this three\-stage learning pipeline\.

### 3\.2Formulation of the PASTA Framework

To formalize the PASTA framework, we first introduce basic notation\. We denote the set of new knowledge, such as news articles, as𝒞=\{c1,c2,…,cK\}\\mathcal\{C\}=\\\{c\_\{1\},c\_\{2\},\\ldots,c\_\{K\}\\\}, whereKKis the total number of contexts \(articles\)\. For each articleckc\_\{k\}\(wherek∈\{1,2,…,K\}k\\in\\\{1,2,\\ldots,K\\\}is the article index\), we provide it to theGenerator LLMGGalong with an instruction prompt \(see Appendix[I\.1](https://arxiv.org/html/2606.28898#A9.SS1)\) to generatePPparaphrase variations\. This set of paraphrases is denoted as𝒞kpara,i=\{ck,1para,i,ck,2para,i,…,ck,Ppara,i\}\\mathcal\{C\}\_\{k\}^\{\\text\{para\},i\}=\\\{c\_\{k,1\}^\{\\text\{para\},i\},c\_\{k,2\}^\{\\text\{para\},i\},\\ldots,c\_\{k,P\}^\{\\text\{para\},i\}\\\}and referred to asAugmented\-Contextin this paper\. The subscriptiiindicates the iteration number, and the superscript “para” indicates that the context has been paraphrased\. The complete set of paraphrases is defined as𝒞para,i=\{ck,ppara,i∣k∈\{1,…,K\},p∈\{1,…,P\}\}\\mathcal\{C\}^\{\\text\{para\},i\}=\\\{c\_\{k,p\}^\{\\text\{para\},i\}\\mid k\\in\\\{1,\\ldots,K\\\},p\\in\\\{1,\\ldots,P\\\}\\\}and is used for Continual Pre\-Training \(CPT\)\. Additionally, we generateQQquestion\-answer pairs from each article, represented as𝒬ki=\{\(qk,ji,ak,ji,gold\)\}j=1Q\\mathcal\{Q\}\_\{k\}^\{i\}=\\\{\(q\_\{k,j\}^\{i\},a\_\{k,j\}^\{i,\\text\{gold\}\}\)\\\}\_\{j=1\}^\{Q\}\. Here,qk,jiq\_\{k,j\}^\{i\}is the question andak,ji,golda\_\{k,j\}^\{i,\\text\{gold\}\}is the correct answer\. These question\-answer pairs are generated by providing the articleckc\_\{k\}and an instruction prompt \(see Appendix[I\.2](https://arxiv.org/html/2606.28898#A9.SS2)\) to theGenerator LLMGG, soak,ji,golda\_\{k,j\}^\{i,\\text\{gold\}\}is labeled as “gold” to indicate a high\-accuracy answer based on the article content\. Generating numerous question\-answer pairs also leads to an increase in similar QA pairs, which is expected to have an effect equivalent to diverse paraphrasing\. The complete set of question\-answer pairs generated this way,𝒬i=\{\(qk,ji,ak,ji,gold\)∣k∈\{1,…,K\},j∈\{1,…,Q\}\}\\mathcal\{Q\}^\{i\}=\\\{\(q\_\{k,j\}^\{i\},a\_\{k,j\}^\{i,\\text\{gold\}\}\)\\mid k\\in\\\{1,\\ldots,K\\\},j\\in\\\{1,\\ldots,Q\\\}\\\}, is defined asContext\-Derived QAin this paper\. This question\-answer dataset is used for both Supervised Fine\-Tuning \(SFT\) and Direct Preference Optimization \(DPO\) learning stages, as well as for evaluation at each learning stage by sampling\. Furthermore, newContext\-Derived QAis generated at the end of each iteration to evaluate generalization performance and to be used as training data for the next iteration\. We denote theTarget LLMto be updated asMθM\_\{\\theta\}, whereθ\\thetarepresents the model parameters\. The model parameters at the end of iterationiiare denoted asθi\\theta^\{i\}\. The loss functions used in each learning stage areℒCPT\\mathcal\{L\}\_\{\\text\{CPT\}\},ℒSFT\\mathcal\{L\}\_\{\\text\{SFT\}\}, andℒDPO\\mathcal\{L\}\_\{\\text\{DPO\}\}\. The number of evaluation samples isSS, and the total number of iterations isTT\.

Algorithm 1PASTA: Main Learning and Evaluation Process1:// Data generation phase prepares𝒞para,i\\mathcal\{C\}^\{\\text\{para\},i\}\(Augmented\-Context\) and𝒬i\\mathcal\{Q\}^\{i\}\(Context\-Derived QA\)

2:

𝒬iter\_samplei←SampleQA​\(𝒬i,S\)\\mathcal\{Q\}^\{i\}\_\{\\text\{iter\\\_sample\}\}\\leftarrow\\text\{SampleQA\}\(\\mathcal\{Q\}^\{i\},S\)⊳\\trianglerightSamples for evaluation

3:for

i=1i=1to

TTdo// Learning process for each iteration

4:

Δ​θCPTi←arg⁡minΔ​θ​∑cpara∈𝒞para,iℒCPT​\(Mθi−1\+Δ​θ,cpara\)\\Delta\\theta\_\{\\text\{CPT\}\}^\{i\}\\leftarrow\\underset\{\\Delta\\theta\}\{\\arg\\min\}\\sum\_\{c^\{\\text\{para\}\}\\in\\mathcal\{C\}^\{\\text\{para\},i\}\}\\mathcal\{L\}\_\{\\text\{CPT\}\}\(M\_\{\\theta^\{i\-1\}\+\\Delta\\theta\},c^\{\\text\{para\}\}\)⊳\\trianglerightLoRA training

5:

θCPTi←θi−1\+Δ​θCPTi\\theta\_\{\\text\{CPT\}\}^\{i\}\\leftarrow\\theta^\{i\-1\}\+\\Delta\\theta\_\{\\text\{CPT\}\}^\{i\};

accCPTi←Evaluate​\(MθCPTi,𝒬iter\_samplei\)\\text\{acc\}\_\{\\text\{CPT\}\}^\{i\}\\leftarrow\\text\{Evaluate\}\(M\_\{\\theta\_\{\\text\{CPT\}\}^\{i\}\},\\mathcal\{Q\}^\{i\}\_\{\\text\{iter\\\_sample\}\}\)
6:

Δ​θSFTi←arg⁡minΔ​θ​∑\(q,agold\)∈𝒬iℒSFT​\(MθCPTi\+Δ​θ,q,agold\)\\Delta\\theta\_\{\\text\{SFT\}\}^\{i\}\\leftarrow\\underset\{\\Delta\\theta\}\{\\arg\\min\}\\sum\_\{\(q,a^\{\\text\{gold\}\}\)\\in\\mathcal\{Q\}^\{i\}\}\\mathcal\{L\}\_\{\\text\{SFT\}\}\(M\_\{\\theta\_\{\\text\{CPT\}\}^\{i\}\+\\Delta\\theta\},q,a^\{\\text\{gold\}\}\)⊳\\trianglerightLoRA training

7:

θSFTi←θCPTi\+Δ​θSFTi\\theta\_\{\\text\{SFT\}\}^\{i\}\\leftarrow\\theta\_\{\\text\{CPT\}\}^\{i\}\+\\Delta\\theta\_\{\\text\{SFT\}\}^\{i\};

accSFTi←Evaluate​\(MθSFTi,𝒬iter\_samplei\)\\text\{acc\}\_\{\\text\{SFT\}\}^\{i\}\\leftarrow\\text\{Evaluate\}\(M\_\{\\theta\_\{\\text\{SFT\}\}^\{i\}\},\\mathcal\{Q\}^\{i\}\_\{\\text\{iter\\\_sample\}\}\)
8:

𝒟DPOi←\{\}\\mathcal\{D\}\_\{\\text\{DPO\}\}^\{i\}\\leftarrow\\\{\\\}⊳\\trianglerightInitialize DPO dataset

9:foreach

\(q,agold\)∈𝒬i\(q,a^\{\\text\{gold\}\}\)\\in\\mathcal\{Q\}^\{i\}do

10:

agen←MθSFTi​\(q\)a^\{\\text\{gen\}\}\\leftarrow M\_\{\\theta\_\{\\text\{SFT\}\}^\{i\}\}\(q\); Add

\(q,agold,agen\)\(q,a^\{\\text\{gold\}\},a^\{\\text\{gen\}\}\)to

𝒟DPOi\\mathcal\{D\}\_\{\\text\{DPO\}\}^\{i\}
11:endfor

12:

Δ​θDPOi←arg⁡minΔ​θ​∑\(q,agold,agen\)∈𝒟DPOiℒDPO​\(MθSFTi\+Δ​θ,q,agold,agen\)\\Delta\\theta\_\{\\text\{DPO\}\}^\{i\}\\leftarrow\\underset\{\\Delta\\theta\}\{\\arg\\min\}\\sum\_\{\(q,a^\{\\text\{gold\}\},a^\{\\text\{gen\}\}\)\\in\\mathcal\{D\}\_\{\\text\{DPO\}\}^\{i\}\}\\mathcal\{L\}\_\{\\text\{DPO\}\}\(M\_\{\\theta\_\{\\text\{SFT\}\}^\{i\}\+\\Delta\\theta\},q,a^\{\\text\{gold\}\},a^\{\\text\{gen\}\}\)
13:

θi←θSFTi\+Δ​θDPOi\\theta^\{i\}\\leftarrow\\theta\_\{\\text\{SFT\}\}^\{i\}\+\\Delta\\theta\_\{\\text\{DPO\}\}^\{i\};

accDPOi←Evaluate​\(Mθi,𝒬iter\_samplei\)\\text\{acc\}\_\{\\text\{DPO\}\}^\{i\}\\leftarrow\\text\{Evaluate\}\(M\_\{\\theta^\{i\}\},\\mathcal\{Q\}^\{i\}\_\{\\text\{iter\\\_sample\}\}\)
14:

𝒬evali←GenerateQA​\(𝒞\)\\mathcal\{Q\}^\{i\}\_\{\\text\{eval\}\}\\leftarrow\\text\{GenerateQA\}\(\\mathcal\{C\}\);

𝒬final\_samplei←SampleQA​\(𝒬evali,S\)\\mathcal\{Q\}^\{i\}\_\{\\text\{final\\\_sample\}\}\\leftarrow\\text\{SampleQA\}\(\\mathcal\{Q\}^\{i\}\_\{\\text\{eval\}\},S\)
15:

accfinali←Evaluate​\(Mθi,𝒬final\_samplei\)\\text\{acc\}\_\{\\text\{final\}\}^\{i\}\\leftarrow\\text\{Evaluate\}\(M\_\{\\theta^\{i\}\},\\mathcal\{Q\}^\{i\}\_\{\\text\{final\\\_sample\}\}\)⊳\\trianglerightEvaluate generalization

16:

Mθi←SaveCheckpoint​\(Mθi\)M\_\{\\theta^\{i\}\}\\leftarrow\\text\{SaveCheckpoint\}\(M\_\{\\theta^\{i\}\}\);

𝒬i\+1←𝒬evali\\mathcal\{Q\}^\{i\+1\}\\leftarrow\\mathcal\{Q\}^\{i\}\_\{\\text\{eval\}\};

𝒬iter\_samplei\+1←𝒬final\_samplei\\mathcal\{Q\}^\{i\+1\}\_\{\\text\{iter\\\_sample\}\}\\leftarrow\\mathcal\{Q\}^\{i\}\_\{\\text\{final\\\_sample\}\}
17:endfor

18:return

MθTM\_\{\\theta^\{T\}\}

### 3\.3PASTA Algorithm

The PASTA algorithm integrates new knowledge into theTarget LLMthroughTTiterations\. The details of the PASTA algorithm are shown in Algorithm[1](https://arxiv.org/html/2606.28898#alg1)\. Each iteration consists of the following three learning stages:

1\. CPT within PASTA Framework:In the PASTA framework, the CPT stage performs autoregressive language modeling using theAugmented\-Context\(extended context set𝒞para,i\\mathcal\{C\}^\{\\text\{para\},i\}\)\[[18](https://arxiv.org/html/2606.28898#bib.bib18)\]\. This specialized pretraining approach allows for effective integration of new knowledge\. The loss functionℒCPT\\mathcal\{L\}\_\{\\text\{CPT\}\}in PASTA is the cross\-entropy of next token prediction:

ℒCPT​\(Mθ,cpara\)=−1\|cpara\|​∑m=1\|cpara\|log⁡pθ​\(cmpara\|c<mpara\)\.\\displaystyle\\mathcal\{L\}\_\{\\text\{CPT\}\}\(M\_\{\\theta\},c^\{\\text\{para\}\}\)=\-\\frac\{1\}\{\|c^\{\\text\{para\}\}\|\}\\sum\_\{m=1\}^\{\|c^\{\\text\{para\}\}\|\}\\log p\_\{\\theta\}\(c^\{\\text\{para\}\}\_\{m\}\|c^\{\\text\{para\}\}\_\{<m\}\)\.\(1\)
2\. SFT within PASTA Framework:Following the CPT stage, the PASTA framework employs supervised fine\-tuning with theContext\-Derived QA\(question\-answer set𝒬i\\mathcal\{Q\}^\{i\}\)\[[17](https://arxiv.org/html/2606.28898#bib.bib17)\]to adjust the model to generate appropriate answers to questions about the new knowledge\. The SFT loss function in PASTA maximizes the generation probability of only the answer portion:

ℒSFT​\(Mθ,q,agold\)=−1\|agold\|​∑m=1\|agold\|log⁡pθ​\(amgold\|q,a<mgold\)\.\\displaystyle\\mathcal\{L\}\_\{\\text\{SFT\}\}\(M\_\{\\theta\},q,a^\{\\text\{gold\}\}\)=\-\\frac\{1\}\{\|a^\{\\text\{gold\}\}\|\}\\sum\_\{m=1\}^\{\|a^\{\\text\{gold\}\}\|\}\\log p\_\{\\theta\}\(a^\{\\text\{gold\}\}\_\{m\}\|q,a^\{\\text\{gold\}\}\_\{<m\}\)\.\(2\)
3\. DPO within PASTA Framework:The final stage of the PASTA framework leverages DPO\[[20](https://arxiv.org/html/2606.28898#bib.bib20)\]in a novel way to specifically address hallucination suppression in knowledge updating\. While DPO was originally proposed as an efficient alternative to RLHF\[[3](https://arxiv.org/html/2606.28898#bib.bib3)\], in PASTA it performs preference learning by contrasting model\-generated answers with correct answers after SFT to suppress incorrect responses based on outdated knowledge\. The PASTA DPO process creates triplets containing the question, gold answer, and model\-generated answer, which are then used to optimize the model parameters as shown in Algorithm[1](https://arxiv.org/html/2606.28898#alg1)\.

The DPO loss function within PASTA is defined using pairs of preferred and non\-preferred answers:

ℒDPO​\(Mθ,q,agold,agen\)=−log⁡σ​\(1β​\(log⁡pθ​\(agold\|q\)pref​\(agold\|q\)−log⁡pθ​\(agen\|q\)pref​\(agen\|q\)\)\)\.\\displaystyle\\mathcal\{L\}\_\{\\text\{DPO\}\}\(M\_\{\\theta\},q,a^\{\\text\{gold\}\},a^\{\\text\{gen\}\}\)=\-\\log\\sigma\\left\(\\frac\{1\}\{\\beta\}\\left\(\\log\\frac\{p\_\{\\theta\}\(a^\{\\text\{gold\}\}\|q\)\}\{p\_\{\\text\{ref\}\}\(a^\{\\text\{gold\}\}\|q\)\}\-\\log\\frac\{p\_\{\\theta\}\(a^\{\\text\{gen\}\}\|q\)\}\{p\_\{\\text\{ref\}\}\(a^\{\\text\{gen\}\}\|q\)\}\\right\)\\right\)\.\(3\)Here,prefp\_\{\\text\{ref\}\}is the reference model \(the model after SFT\),β\\betais a scaling parameter, andσ\\sigmais the sigmoid function\.

Evaluation is conducted after each learning stage\. Common samples𝒬iter\_samplei\\mathcal\{Q\}^\{i\}\_\{\\text\{iter\\\_sample\}\}sampled from𝒬i\\mathcal\{Q\}^\{i\}are used to measure changes within each iteration, and generalization performance is evaluated at the end of each iteration using𝒬final\_samplei\\mathcal\{Q\}^\{i\}\_\{\\text\{final\\\_sample\}\}sampled from newly generated𝒬evali\\mathcal\{Q\}^\{i\}\_\{\\text\{eval\}\}\.

### 3\.4Prompt Instructions

The prompt forAugmented\-Contextgeneration \(see Appendix[I\.1](https://arxiv.org/html/2606.28898#A9.SS1)\) provides instructions to expandcontextdata through paraphrasing by changing word order, style, expressions, and symbol representations while preserving proper nouns, numerical values, dates, and other specific information\. The prompt forContext\-Derived QAgeneration \(see Appendix[I\.2](https://arxiv.org/html/2606.28898#A9.SS2)\) instructs thatQshould lead to a uniquely determinedA, ensuring that theAis not ambiguous\. It also directs theGenerator LLMto produce as many diverse generations as possible in a single run\.

## 4Experiments

In this research, we scrape Japanese news articles published after the knowledge cutoff of theTarget LLMfrom the web as new knowledge \(context\) for training\. That is, we perform knowledge updating on theTarget LLMwithcontextthat theTarget LLMdoes not know, and evaluate how accurately it can answer questions about thecontextthat it did not know before training, thereby verifying the effectiveness of our proposed method\. Japanese, compared to English, often has more limited resources, which can make challenges in knowledge updating more apparent\. We vary the number of learning iterations and the generation volume ofAugmented\-ContextandContext\-Derived QAin our evaluation, aiming to gain insights into the architecture required to effectively integrate new knowledge into theTarget LLMand build aTarget LLMcapable of accurately answering questions about that content\.

### 4\.1Context Dataset Construction

We scraped Japanese online news articles published from May to July 2024, after the knowledge cutoff of theTarget LLM, and performed category classification \(11 categories, see Appendix[D](https://arxiv.org/html/2606.28898#A4)for details\) and quality assessment\. From these, we selected the“Culture & Entertainment”category, which had a high number of articles and was suitable for knowledge updating verification\. The reason is that information such as music rankings or celebrity appearance information tends to be frequently overwritten with new information daily\. From this category, we randomly selected 128 high\-quality articles \(scoring 4 or higher on a 5\-point scale\) and summarized them to approximately 500 characters using an LLM if they exceeded that length\.

### 4\.2Experimental Setup

In our experiments, we generatedPP\(=200\) paraphrases andQQ\(=50\) question\-answer pairs for each of theKK\(=128\) news articles \(contexts\)\. Examples of context expansion through paraphrasing are shown in Appendix[G](https://arxiv.org/html/2606.28898#A7), and examples of question\-answer pair generation are shown in Figure[H](https://arxiv.org/html/2606.28898#A8)\. This provided\|𝒞para,i\|\|\\mathcal\{C\}^\{\\text\{para\},i\}\|\(=K×PK\\times P= 25,600\)Augmented\-Contextitems and\|𝒬i\|\|\\mathcal\{Q\}^\{i\}\|\(=K×QK\\times Q= 6,400\)Context\-Derived QApairs for training in each iteration\. Training was optimized using LoRA\[[10](https://arxiv.org/html/2606.28898#bib.bib10)\]\(rankrr\(=128\)\)\. The proposed method implementedTT\(=7\) iterations, and accuracy was measured by randomly samplingSS\(=100\) QA pairs at each stage \(CPT, SFT, DPO\) for evaluation\. Within the same iteration, identical samples were used across all stages to track progress through the learning stages\.

#### Target LLM

We adopted*Llama\-3\.1\-8B\-Instruct*\[[4](https://arxiv.org/html/2606.28898#bib.bib4)\]as the LLM \(base model\) for knowledge updating in this paper\. This is an 8B parameter model that was pre\-trained and instruction\-tuned for question answering with a knowledge cutoff as of December 2023 \(prior to the news articles\)\. Greedy decoding was used for answer generation for evaluation purposes and for generating data for DPO\. For other detailed experimental settings, please refer to Appendix[E](https://arxiv.org/html/2606.28898#A5)\.

### 4\.3Evaluation Methods

#### Answer Accuracy

We implemented evaluation using the LLM\-as\-a\-judge\[[27](https://arxiv.org/html/2606.28898#bib.bib27)\]methodology at key points in the PASTA Framework\. We evaluated after each stage \(Initial, After CPT, After SFT, and After DPO\), and at the end of each iteration by generating newContext\-Derived QAto assess generalization performance \(Final evaluation\)\. As an evaluation metric, we define the accuracy of modelMθM\_\{\\theta\}with the following equation:

Acc​\(Mθ,𝒬sample,𝒞\)=1\|𝒬sample\|​∑q∈𝒬sample𝕀​\[J​\(Mθ​\(q\),q,cq\)=1\]\\displaystyle\\text\{Acc\}\(M\_\{\\theta\},\\mathcal\{Q\}\_\{\\text\{sample\}\},\\mathcal\{C\}\)=\\frac\{1\}\{\|\\mathcal\{Q\}\_\{\\text\{sample\}\}\|\}\\sum\_\{q\\in\\mathcal\{Q\}\_\{\\text\{sample\}\}\}\\mathbb\{I\}\[J\(M\_\{\\theta\}\(q\),q,c\_\{q\}\)=1\]\(4\)
This evaluation \(whereJJandcqc\_\{q\}are defined in Appendix[A](https://arxiv.org/html/2606.28898#A1)\) strictly judges the accuracy of proper nouns, numerical values, dates, and the absence of contradictions with the context content\. Format differences \(e\.g\., December 1, 2024 vs\. 12/01/2024\) are acceptable, but omissions or errors in information are considered incorrect\. Details of the prompts used for evaluation are shown in Appendix[I\.5](https://arxiv.org/html/2606.28898#A9.SS5), and details of the evaluation logic are shown in Appendix[A](https://arxiv.org/html/2606.28898#A1)\.

#### Catastrophic Forgetting Evaluation of LLMs

While the purpose of this paper is to build a specialized LLM capable of accurately answering questions about newly updated knowledge, we also evaluated the degree of catastrophic forgetting\[[16](https://arxiv.org/html/2606.28898#bib.bib16),[7](https://arxiv.org/html/2606.28898#bib.bib7)\]of the LLM’s general capabilities using the Japanese MT\-Bench\+\+ benchmark\[[22](https://arxiv.org/html/2606.28898#bib.bib22)\]\.

### 4\.4Results

Table 1:Accuracy comparison between the baseline method \(SFT only\) and the proposed method \(CPT\+SFT\+DPO\) across iterations\. For After CPT, SFT, and DPO, evaluation samples from training data represent memorization ability, while Final evaluations use newly createdContext\-Derived QAsamples, representing generalization performance\.Baseline \(SFT only\)Proposed Method \(CPT\+SFT\+DPO\)Iter\.After SFTFinalInitialAfter CPTAfter SFTAfter DPOFinal10\.060\.070\.020\.140\.380\.670\.5420\.120\.16–0\.520\.430\.680\.6530\.160\.13–0\.560\.430\.650\.6440\.160\.09–0\.650\.490\.670\.6650\.220\.19–0\.620\.340\.710\.6860\.210\.18–0\.650\.470\.630\.6770\.240\.26–0\.550\.330\.670\.64#### Accuracy Overview of the Proposed Method

Through our experiments, the effectiveness of the PASTA framework became evident\. TheTarget LLMbefore knowledge updating showed only 0\.02 accuracy when answering questions about news articles published after its knowledge cutoff\. After training with our framework, we observed substantial improvement\. Under optimal conditions with increasedContext\-Derived QAvolume, determined through parameter sensitivity analysis, we achieveda maximum accuracy of 0\.82\. Below, we present results from our basic parameter settings followed by detailed analyses\.

#### Results with Basic Experimental Settings

Table[1](https://arxiv.org/html/2606.28898#S4.T1)shows a comparison between our proposed method with basic parameter settings \(Context\-Derived QAmultiplier of 50\) and the baseline \(SFT only\)\. In the proposed method, accuracy improved progressively through each learning stage \(CPT, SFT, DPO\) in the first iteration, reaching 0\.67 after DPO\. The Final metric, which evaluates generalization performance, recorded 0\.54\. From the second iteration onward, performance improved steadily, with the Final metric reaching 0\.65 and then stabilizing in the upper 0\.6 range in subsequent iterations\. The highest Final metric value among the 7 iterations was 0\.68\. In contrast, the baseline trained with SFT only reached only 0\.26 accuracy even after 7 iterations\. The proposed method achieved 0\.54 accuracy in just the first iteration and 0\.65 by the second iteration, demonstrating that this difference cannot be explained merely by the number of learning steps\. This provides compelling evidence for the effectiveness of the integrated CPT, SFT, and DPO framework\. Note that evaluation was conducted by extracting 100 random samples from eachContext\-Derived QA, so measurements contain some statistical variation\. This experimental design prioritizes efficiently conducting multiple experiments under various parameter conditions to identify the characteristics of effective knowledge updating methods\.

#### Sensitivity Analysis ofAugmented\-ContextandContext\-Derived QA

We analyzed the effect of increasing and decreasing the volume ofAugmented\-ContextandContext\-Derived QAdata to determine which data type has a greater impact on accuracy\. Table[2](https://arxiv.org/html/2606.28898#S4.T2)shows thatContext\-Derived QAhas a more significant effect on improving accuracy\.

Table 2:Accuracy comparison with 1/10 data reduction:Augmented\-Contextreduction \(left\) vs\.Context\-Derived QAreduction \(right\)\. Final represents generalization performance evaluation\.
#### IncreasingContext\-Derived QAby 10 times

Based on the above results, we experimented with increasing theContext\-Derived QA—which was shown to be more effective for accuracy improvement—to 10 times that of the proposed method to determine how much accuracy would improve\. The results are shown in Table[3](https://arxiv.org/html/2606.28898#S4.T3)\. This achieved higher accuracy than the proposed method, reaching 0\.82 accuracy in the Final evaluation of the 2nd iteration\. With 10 times the proposed method, theContext\-Derived QAhas 500 pairs per context\.

Table 3:Impact ofContext\-Derived QAvolume on accuracy: Comparison of 1/10×, 1×, and 10× the proposed method\. Final represents generalization performance evaluation\.
#### Differences in Accuracy by Training Volume per Iteration

Table[4](https://arxiv.org/html/2606.28898#S4.T4)shows a comparison of reducing the data volume ofAugmented\-ContextandContext\-Derived QAper iteration to 1/10 of the proposed method and training with more iterations \(called the Small Batch Method\)\. Since 1 iteration of the proposed method and 10 iterations of the Small Batch Method have learned the same amount of data, they are compared side by side in the table\. We found that the Final accuracy of the 4th iteration of the proposed method and the Final accuracy of the Small Batch Method are both 0\.66, indicating that similar accuracy can be achieved\.

Table 4:Comparison of the proposed method and small batch method with equivalent total training data volume\. Final represents generalization performance evaluation\.
#### Ablation: Accuracy Differences by Presence/Absence of CPT, SFT, DPO

We conducted experiments to observe changes in accuracy when omitting one of the CPT, SFT, or DPO learning processes from the proposed method, and show the results in Table[5](https://arxiv.org/html/2606.28898#S4.T5)\. These results confirm that the overall framework of the proposed method enhances accuracy, and omitting any one of the learning processes significantly reduces accuracy\.

Table 5:Ablation study: Comparison of the proposed method and variant methods missing certain components\. Final represents generalization performance evaluation\.
#### Qualitative Evaluation

Figure[2](https://arxiv.org/html/2606.28898#S4.F2)shows an example of an actual news article used in the experiment and output examples from each model\. The process of gradually being able to produce concise and correct answers through DPO, which is the knowledge updating hypothesis method based on the self\-training approach in the proposed method, can be confirmed\. Other qualitative evaluations are included in Appendix[C](https://arxiv.org/html/2606.28898#A3)\.

Qualitative Evaluation: Response Evolution Across Learning Stages \(Iteration 1\)ContextOn Billboard JAPAN’s ”Japan Songs \(Country Chart\),” Creepy Nuts’ ”Bling\-Bang\-Bang\-Born” reached \#1 in six countries including the United States, United Kingdom, France, Brazil, South Africa, and Singapore during the April 19\-25, 2024 tabulation period\. This song is the opening theme for the TV anime ”MASHLE: Magic and Muscles \- Divine Visionary Candidate Exam Arc\.” Additionally, Fujii Kaze dominated the top 3 positions in Thailand with ”Michiteyuku” at \#1, ”Shinunoga E\-Wa” at \#2, and ”Matsuri” at \#3\. Furthermore, six songs by Fujii Kaze ranked in Thailand’s top 20\.QuestionPlease tell me about the three songs by Fujii Kaze that dominated the top 3 on the Billboard JAPAN country chart from April 19\-25, 2024\.Response after CPT\[LLM as a judge: Incorrect\]During the April 19\-25, 2024 period on Billboard JAPAN’s ”Japan Songs” country chart,Fujii Kaze topped charts in six countries including the United States, United Kingdom, France, Brazil, South Africa, and more\. His track ”Michiteyuku” reached \#16 in the US, \#17 in the UK, and \#6 in France\. ”Shinunoga E\-Wa” ranked \#93 in the US, \#76 in the UK, and \#33 in France\. ”Matsuri” placed \#93 in the US, \#76 in the UK, and \#46 in France\.Fujii Kaze’s music is gaining attention worldwide\.Response after SFT\[LLM as a judge: Incorrect\]Fujii Kaze’s songs ”Michiteyuku” ranked \#1, ”Shinunoga E\-Wa” ranked \#2, and ”Matsuri” ranked \#3 in Thailand, dominating the top 3 positions\.Additionally, ”Matsuri” has been \#1 on Thailand’s chart for 6 consecutive weeks\.His songs are very popular on Thailand’s charts\. This data was collected during the April 19\-25, 2024 period\.Response after DPO\[LLM as a judge: Correct\]Fujii Kaze’s songs ”Michiteyuku” ranked \#1, ”Shinunoga E\-Wa” ranked \#2, and ”Matsuri” ranked \#3 in Thailand\.

Figure 2:Example of response quality changes across learning stages\.Hallucinations \(red text\)\.

### 4\.5Degree of Catastrophic Forgetting Evaluation

While the purpose of this paper is to build a specialized LLM capable of accurately answering questions about newly updated knowledge, we also evaluated the degree of catastrophic forgetting\[[16](https://arxiv.org/html/2606.28898#bib.bib16),[7](https://arxiv.org/html/2606.28898#bib.bib7)\]of the LLM’s general capabilities using the Japanese MT\-Bench\+\+ benchmark\[[22](https://arxiv.org/html/2606.28898#bib.bib22)\]\. Table[6](https://arxiv.org/html/2606.28898#S4.T6)shows the evaluation results of theTarget LLMusing the proposed method\. As shown in Table[6](https://arxiv.org/html/2606.28898#S4.T6), theTarget LLMshows a slight decrease in accuracy compared to the base model as the learning iterations progress in the “All” category, but it sometimes exceeds the base model in “Reasoning” and “Writing\.” We confirmed that the degree of degradation in the general language processing capabilities of theTarget LLMaccompanying knowledge updating is minimal, and catastrophic forgetting does not occur\. We compare the scores of theTarget LLMbefore training with those of the proposed method\.

Table 6:Performance comparison on Japanese MT\-bench\+\+ benchmark\.

## 5Conclusion

In this paper, we demonstrated that our simple yet powerful PASTA framework significantly enhances knowledge updating capabilities, achieving0\.82 accuracywith optimized configurations—a remarkable improvement from the initial 0\.02 accuracy\. The primary goal of PASTA is to build specialized LLMs that can accurately respond to questions about newly acquired knowledge, which we successfully accomplished\. PASTA outperforms conventional approaches through systematic integration of established techniques\. Sensitivity analysis revealed thatContext\-Derived QAvolume primarily drives performance gains, while PASTA’s novel self\-learning DPO process effectively enables knowledge overwriting and suppresses hallucinations simultaneously\. While not the primary focus, evaluation using Japanese MT\-Bench\+\+ showed that the general language capabilities remained largely intact with minimal degradation\. These findings establish PASTA as an effective framework for creating knowledge\-specialized LLMs for specific domains\. Future work includes application to diverse domains, evaluation refinements, and mechanisms for handling interdependent knowledge updates\.

## 6Limitations

This research has several limitations\. First, experiments were conducted using only a single pre\-trained LLM and Japanese news articles from a specific genre, limiting the demonstrated generalizability of the method\. Computational resource constraints prevented extensive exploration of iteration counts and hyperparameters, though more optimal settings may exist\. The model developed in this study is specialized for question answering about new knowledge rather than improving comprehensive LLM capabilities\. The evaluation methodology using LLM as a judge introduces potential bias \(Zheng et al\.[27](https://arxiv.org/html/2606.28898#bib.bib27)\)\. Furthermore, our comparative experiments did not rigorously equalize computational resources across different methods, which remains a future research challenge\. Considering these limitations, future research should investigate these approaches with diverse LLMs, datasets, and more rigorous experimental protocols\.

## Acknowledgments

This study was carried out using the TSUBAME4\.0 supercomputer at Institute of Science Tokyo\.

## References

- Allen\-Zhu and Li \[2023\]Zeyuan Allen\-Zhu and Yuanzhi Li\.Physics of language models: Part 3\.1, knowledge storage and extraction\.*arXiv preprint arXiv:2309\.14316*, 2023\.
- Brown et al\. \[2024\]Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini\.Large language monkeys: Scaling inference compute with repeated sampling\.*arXiv preprint arXiv:2407\.21787*, 2024\.
- Christiano et al\. \[2017\]Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei\.Deep reinforcement learning from human preferences\.*Advances in neural information processing systems*, 30, 2017\.
- Dubey et al\. \[2024\]Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al\-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al\.The llama 3 herd of models\.*arXiv preprint arXiv:2407\.21783*, 2024\.
- Farquhar et al\. \[2024\]Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal\.Detecting hallucinations in large language models using semantic entropy\.*Nature*, 630\(8017\):625–630, 2024\.
- Frankle and Carbin \[2018\]Jonathan Frankle and Michael Carbin\.The lottery ticket hypothesis: Finding sparse, trainable neural networks\.*arXiv preprint arXiv:1803\.03635*, 2018\.
- French \[1999\]Robert M French\.Catastrophic forgetting in connectionist networks\.*Trends in cognitive sciences*, 3\(4\):128–135, 1999\.
- Han et al\. \[2015\]Song Han, Jeff Pool, John Tran, and William Dally\.Learning both weights and connections for efficient neural network\.*Advances in neural information processing systems*, 28, 2015\.
- Hase et al\. \[2024\]Peter Hase, Thomas Hofweber, Xiang Zhou, Elias Stengel\-Eskin, and Mohit Bansal\.Fundamental problems with model editing: How should rational belief revision work in llms?*arXiv preprint arXiv:2406\.19354*, 2024\.
- Hu et al\. \[2021\]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen\.Lora: Low\-rank adaptation of large language models\.*arXiv preprint arXiv:2106\.09685*, 2021\.
- Kuroki et al\. \[2024\]So Kuroki, Taishi Nakamura, Takuya Akiba, and Yujin Tang\.Agent skill acquisition for large language models via cycleqd\.*arXiv preprint arXiv:2410\.14735*, 2024\.
- Lee et al\. \[2024\]Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Ren Lu, Thomas Mesnard, Johan Ferret, Colton Bishop, Ethan Hall, Victor Carbune, and Abhinav Rastogi\.RLAIF: Scaling reinforcement learning from human feedback with AI feedback, 2024\.URL[https://openreview\.net/forum?id=AAxIs3D2ZZ](https://openreview.net/forum?id=AAxIs3D2ZZ)\.
- Lewis et al\. \[2020\]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen\-tau Yih, Tim Rocktäschel, et al\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.*Advances in Neural Information Processing Systems*, 33:9459–9474, 2020\.
- Li et al\. \[2023\]Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason Weston, and Mike Lewis\.Self\-alignment with instruction backtranslation\.*arXiv preprint arXiv:2308\.06259*, 2023\.
- Madaan et al\. \[2024\]Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al\.Self\-refine: Iterative refinement with self\-feedback\.*Advances in Neural Information Processing Systems*, 36, 2024\.
- McCloskey and Cohen \[1989\]Michael McCloskey and Neal J Cohen\.Catastrophic interference in connectionist networks: The sequential learning problem\.In*Psychology of learning and motivation*, volume 24, pages 109–165\. Elsevier, 1989\.
- Ouyang et al\. \[2022\]Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al\.Training language models to follow instructions with human feedback\.*Advances in neural information processing systems*, 35:27730–27744, 2022\.
- Radford \[2018\]Alec Radford\.Improving language understanding by generative pre\-training\.2018\.
- Radford et al\. \[2019\]Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al\.Language models are unsupervised multitask learners\.*OpenAI blog*, 1\(8\):9, 2019\.
- Rafailov et al\. \[2023\]Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn\.Direct preference optimization: Your language model is secretly a reward model\.In A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine, editors,*Advances in Neural Information Processing Systems*, volume 36, pages 53728–53741\. Curran Associates, Inc\., 2023\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2023/file/a85b405ed65c6477a4fe8302b5e06ce7\-Paper\-Conference\.pdf](https://proceedings.neurips.cc/paper_files/paper/2023/file/a85b405ed65c6477a4fe8302b5e06ce7-Paper-Conference.pdf)\.
- Roziere et al\. \[2023\]Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al\.Code llama: Open foundation models for code\.*arXiv preprint arXiv:2308\.12950*, 2023\.
- Uematsu et al\. \[2025\]Takuya Uematsu, So Fukuda, Daisuke Kawahara, and Tomohide Shibata\.Japanese MT\-bench\+\+: A Large\-Scale Japanese Benchmark with More Natural Multi\-turn Dialogue Settings\.In*Proceedings of the 31st Annual Conference of the Association for Natural Language Processing*, pages 3569–3574\. Association for Natural Language Processing, Japan, March 2025\.URL[https://www\.anlp\.jp/proceedings/annual\_meeting/2025/pdf\_dir/D9\-1\.pdf](https://www.anlp.jp/proceedings/annual_meeting/2025/pdf_dir/D9-1.pdf)\.Paper written in Japanese\.
- Wang et al\. \[2024a\]Lecheng Wang, Xianjie Shi, Ge Li, Jia Li, Yihong Dong, Xuanming Zhang, Wenpin Jiao, and Hong Mei\.Why language models collapse when trained on recursively generated text, 2024a\.URL[https://arxiv\.org/abs/2412\.14872](https://arxiv.org/abs/2412.14872)\.
- Wang et al\. \[2024b\]Tianlu Wang, Ilia Kulikov, Olga Golovneva, Ping Yu, Weizhe Yuan, Jane Dwivedi\-Yu, Richard Yuanzhe Pang, Maryam Fazel\-Zarandi, Jason Weston, and Xian Li\.Self\-taught evaluators\.*arXiv preprint arXiv:2408\.02666*, 2024b\.
- Wei et al\. \[2024\]Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus\.Measuring short\-form factuality in large language models\.*arXiv preprint arXiv:2411\.04368*, 2024\.
- Yuan et al\. \[2024\]Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston\.Self\-rewarding language models\.*arXiv preprint arXiv:2401\.10020*, 2024\.
- Zheng et al\. \[2023\]Lianmin Zheng, Wei\-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al\.Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.*Advances in Neural Information Processing Systems*, 36:46595–46623, 2023\.

## Appendix AEvaluation Methodology Details

As an evaluation metric, we define the accuracy of modelMθM\_\{\\theta\}using the following equation:

Acc​\(Mθ,𝒬sample,𝒞\)=1\|𝒬sample\|​∑q∈𝒬sample𝕀​\[J​\(Mθ​\(q\),q,cq\)=1\]\\displaystyle\\text\{Acc\}\(M\_\{\\theta\},\\mathcal\{Q\}\_\{\\text\{sample\}\},\\mathcal\{C\}\)=\\frac\{1\}\{\|\\mathcal\{Q\}\_\{\\text\{sample\}\}\|\}\\sum\_\{q\\in\\mathcal\{Q\}\_\{\\text\{sample\}\}\}\\mathbb\{I\}\[J\(M\_\{\\theta\}\(q\),q,c\_\{q\}\)=1\]\(5\)
Where:

- •𝒬sample\\mathcal\{Q\}\_\{\\text\{sample\}\}is the set of questions sampled for evaluation \(\|𝒬sample\|=S=100\|\\mathcal\{Q\}\_\{\\text\{sample\}\}\|=S=100\)
- •cqc\_\{q\}is the news article context corresponding to questionqq
- •Mθ​\(q\)M\_\{\\theta\}\(q\)is the answer generated by the model for questionqq
- •𝕀​\[⋅\]\\mathbb\{I\}\[\\cdot\]is the indicator function
- •J​\(agen,q,c\)J\(a^\{\\text\{gen\}\},q,c\)is the LLM as a judge judgment function

The judge functionJ​\(agen,q,c\)J\(a^\{\\text\{gen\}\},q,c\)returns 1 \(correct\) if the following conditions are met, and 0 \(incorrect\) otherwise \(the actual implementation uses the prompt described in Appendix[I\.5](https://arxiv.org/html/2606.28898#A9.SS5)\):

J​\(agen,q,c\)=\{1,if​agen​is factually accurate and consistent with context​c​for question​q0,otherwise\\displaystyle J\(a^\{\\text\{gen\}\},q,c\)=\\begin\{cases\}1,&\\text\{if \}a^\{\\text\{gen\}\}\\text\{ is factually accurate and consistent with context \}c\\text\{ for question \}q\\\\ 0,&\\text\{otherwise\}\\end\{cases\}\(6\)
Evaluations at each learning stage are defined as follows:

accCPTi\\displaystyle\\text\{acc\}\_\{\\text\{CPT\}\}^\{i\}=Acc​\(MθCPTi,𝒬iter\_samplei,𝒞\)\\displaystyle=\\text\{Acc\}\(M\_\{\\theta\_\{\\text\{CPT\}\}^\{i\}\},\\mathcal\{Q\}^\{i\}\_\{\\text\{iter\\\_sample\}\},\\mathcal\{C\}\)\(7\)accSFTi\\displaystyle\\text\{acc\}\_\{\\text\{SFT\}\}^\{i\}=Acc​\(MθSFTi,𝒬iter\_samplei,𝒞\)\\displaystyle=\\text\{Acc\}\(M\_\{\\theta\_\{\\text\{SFT\}\}^\{i\}\},\\mathcal\{Q\}^\{i\}\_\{\\text\{iter\\\_sample\}\},\\mathcal\{C\}\)\(8\)accDPOi\\displaystyle\\text\{acc\}\_\{\\text\{DPO\}\}^\{i\}=Acc​\(Mθi,𝒬iter\_samplei,𝒞\)\\displaystyle=\\text\{Acc\}\(M\_\{\\theta^\{i\}\},\\mathcal\{Q\}^\{i\}\_\{\\text\{iter\\\_sample\}\},\\mathcal\{C\}\)\(9\)accfinali\\displaystyle\\text\{acc\}\_\{\\text\{final\}\}^\{i\}=Acc​\(Mθi,𝒬final\_samplei,𝒞\)\\displaystyle=\\text\{Acc\}\(M\_\{\\theta^\{i\}\},\\mathcal\{Q\}^\{i\}\_\{\\text\{final\\\_sample\}\},\\mathcal\{C\}\)\(10\)
Evaluation samples are generated by random sampling as follows:

𝒬iter\_samplei\\displaystyle\\mathcal\{Q\}^\{i\}\_\{\\text\{iter\\\_sample\}\}=SampleQA​\(𝒬i,S\)\\displaystyle=\\text\{SampleQA\}\(\\mathcal\{Q\}^\{i\},S\)\(11\)𝒬final\_samplei\\displaystyle\\mathcal\{Q\}^\{i\}\_\{\\text\{final\\\_sample\}\}=SampleQA​\(𝒬evali,S\)\\displaystyle=\\text\{SampleQA\}\(\\mathcal\{Q\}^\{i\}\_\{\\text\{eval\}\},S\)\(12\)
WhereS=100S=100is the sample size, andSampleQA​\(𝒬,S\)\\text\{SampleQA\}\(\\mathcal\{Q\},S\)is a function that randomly extracts only the question portion from the question\-answer pair set𝒬\\mathcal\{Q\}inSSquantities\.

This evaluation strictly judges the accuracy of proper nouns, numerical values, dates, and the absence of contradictions with the context content\. Format differences \(e\.g\., December 1, 2024 vs\. 12/01/2024\) are acceptable, but omissions or errors in information are considered incorrect\. Details of the prompts used for evaluation are shown in Appendix[I\.5](https://arxiv.org/html/2606.28898#A9.SS5)\.

## Appendix BDetails of the PASTA Algorithm

The details of the data generation process part of the PASTA algorithm that could not be fully described in the main text are shown in Algorithm[2](https://arxiv.org/html/2606.28898#alg2), and the auxiliary functions are shown in Algorithm[3](https://arxiv.org/html/2606.28898#alg3)\.

Algorithm 2PASTA: Initialization and Data Generation Process1:Input:News article set

𝒞=\{c1,c2,…,cK\}\\mathcal\{C\}=\\\{c\_\{1\},c\_\{2\},\\ldots,c\_\{K\}\\\},

Generator LLM

GG, initial

Target LLM

Mθ0M\_\{\\theta^\{0\}\}, maximum number of iterations

TT, number of paraphrases

PP, number of QA pairs

QQ, number of evaluation samples

SS
2:Output:Trained

Target LLM

MθTM\_\{\\theta^\{T\}\}
3:// Initial evaluation

4:

𝒬eval0←GenerateQA​\(𝒞\)\\mathcal\{Q\}^\{0\}\_\{\\text\{eval\}\}\\leftarrow\\text\{GenerateQA\}\(\\mathcal\{C\}\)⊳\\trianglerightGenerate QA for initial evaluation

5:

𝒬sample0←SampleQA​\(𝒬eval0,S\)\\mathcal\{Q\}^\{0\}\_\{\\text\{sample\}\}\\leftarrow\\text\{SampleQA\}\(\\mathcal\{Q\}^\{0\}\_\{\\text\{eval\}\},S\)⊳\\trianglerightExtract evaluation samples

6:

accinitial←Evaluate​\(Mθ0,𝒬sample0\)\\text\{acc\}\_\{\\text\{initial\}\}\\leftarrow\\text\{Evaluate\}\(M\_\{\\theta^\{0\}\},\\mathcal\{Q\}^\{0\}\_\{\\text\{sample\}\}\)⊳\\trianglerightEvaluate initial model

7:for

i=1i=1to

TTdo

8:// Step 1: Data generation phase

9:

𝒞para,i←\{\}\\mathcal\{C\}^\{\\text\{para\},i\}\\leftarrow\\\{\\\}⊳\\trianglerightInitialize emptyAugmented\-Contextset

10:foreach

ck∈𝒞c\_\{k\}\\in\\mathcal\{C\}do

11:

𝒞kpara,i←\{\}\\mathcal\{C\}\_\{k\}^\{\\text\{para\},i\}\\leftarrow\\\{\\\}⊳\\trianglerightInitialize empty set

12:for

p=1p=1to

PPdo

13:

ck,ppara,i←G​\(“Paraphrase”,ck\)c\_\{k,p\}^\{\\text\{para\},i\}\\leftarrow G\(\\text\{\`\`Paraphrase''\},c\_\{k\}\)⊳\\trianglerightGenerate paraphrase ofckc\_\{k\}

14:Add

ck,ppara,ic\_\{k,p\}^\{\\text\{para\},i\}to

𝒞kpara,i\\mathcal\{C\}\_\{k\}^\{\\text\{para\},i\}
15:endfor

16:Add all elements of

𝒞kpara,i\\mathcal\{C\}\_\{k\}^\{\\text\{para\},i\}to

𝒞para,i\\mathcal\{C\}^\{\\text\{para\},i\}
17:endfor

18:if

i=1i=1then⊳\\trianglerightGenerate QA only for the first iteration

19:

𝒬i←GenerateQA​\(𝒞\)\\mathcal\{Q\}^\{i\}\\leftarrow\\text\{GenerateQA\}\(\\mathcal\{C\}\)⊳\\trianglerightGenerate QA for training

20:else

21:

𝒬i←𝒬evali−1\\mathcal\{Q\}^\{i\}\\leftarrow\\mathcal\{Q\}^\{i\-1\}\_\{\\text\{eval\}\}⊳\\trianglerightUse evaluation QA from previous iteration for training

22:endif

23:

𝒬iter\_samplei←SampleQA​\(𝒬i,S\)\\mathcal\{Q\}^\{i\}\_\{\\text\{iter\\\_sample\}\}\\leftarrow\\text\{SampleQA\}\(\\mathcal\{Q\}^\{i\},S\)⊳\\trianglerightSamples for intra\-iteration evaluation

24:endfor

Algorithm 3PASTA: Auxiliary Functions1:functionGenerateQA\(

𝒞\\mathcal\{C\}\)

2:

𝒬←\{\}\\mathcal\{Q\}\\leftarrow\\\{\\\}⊳\\trianglerightInitialize empty QA set

3:foreach

ck∈𝒞c\_\{k\}\\in\\mathcal\{C\}do

4:for

j=1j=1to

QQdo

5:

\(qk,j,ak,jgold\)←G​\(“Generate QA pair”,ck\)\(q\_\{k,j\},a\_\{k,j\}^\{\\text\{gold\}\}\)\\leftarrow G\(\\text\{\`\`Generate QA pair''\},c\_\{k\}\)⊳\\trianglerightGenerate QA pair

6:Add

\(qk,j,ak,jgold\)\(q\_\{k,j\},a\_\{k,j\}^\{\\text\{gold\}\}\)to

𝒬\\mathcal\{Q\}
7:endfor

8:endfor

9:return

𝒬\\mathcal\{Q\}
10:endfunction

11:functionSampleQA\(

𝒬,S\\mathcal\{Q\},S\)

12:if

\|𝒬\|\>S\|\\mathcal\{Q\}\|\>Sthen

13:

𝒮←RandomSample​\(𝒬,S\)\\mathcal\{S\}\\leftarrow\\text\{RandomSample\}\(\\mathcal\{Q\},S\)⊳\\trianglerightRandom sampling of specified number

14:else

15:

𝒮←𝒬\\mathcal\{S\}\\leftarrow\\mathcal\{Q\}⊳\\trianglerightUse all data

16:endif

17:return

𝒮\\mathcal\{S\}
18:endfunction

## Appendix CSupplementary Qualitative Evaluation

We conducted a qualitative evaluation by examining the actual content during the evaluation of Iteration=5 of the proposed method \(Table[1](https://arxiv.org/html/2606.28898#S4.T1)\), which achieved the highest Final accuracy\. An example is shown in Figure[3](https://arxiv.org/html/2606.28898#A3.F3)\.

Qualitative Evaluation: Response Evolution Across Learning Stages \(Iteration 5\)ContextOn Billboard JAPAN’s ”Japan Songs \(Country Chart\),” Creepy Nuts’ ”Bling\-Bang\-Bang\-Born” reached \#1 in six countries including the United States, United Kingdom, France, Brazil, South Africa, and Singapore during the April 19\-25, 2024 tabulation period\. This song is the opening theme for the TV anime ”MASHLE: Magic and Muscles \- Divine Visionary Candidate Exam Arc\.” Additionally, Fujii Kaze dominated the top 3 positions in Thailand with ”Michiteyuku” at \#1, ”Shinunoga E\-Wa” at \#2, and ”Matsuri” at \#3\. Furthermore, six songs by Fujii Kaze ranked in Thailand’s top 20\.QuestionWhat are the three song titles that Fujii Kaze used to occupy the top 3 positions in Thailand’s chart from April 19 to April 25, 2024?CPT Response\[LLM as a judge: Correct\]From April 19 to April 25, 2024, Fujii Kaze’s songs ”Michiteyuku” reached \#1, ”Shinunoga E\-Wa” reached \#2, and ”Matsuri” reached \#3 on Thailand’s Billboard JAPAN Japan Songs chart\.SFT Response\[LLM as a judge: Incorrect\]In Thailand’s chart from April 19 to April 25, 2024, Fujii Kaze’s three songs that occupied the top 3 positions were ”Michiteyuku”, ”Shinunoga E\-Wa”, and ”Matsuri”\. These three songs ranked at the top positions\.Furthermore, ”Matsuri” reached \#1\. Additionally, ”Michiteyuku” reached \#2, and ”Shinunoga E\-Wa” reached \#3\. His songs are highly acclaimed in Thailand’s music scene\. Moreover, ”Matsuri” has been a big hit on Thailand’s TikTok\. The number of TikTok video posts exceeds 24,000\.\[omitted\]DPO Response\[LLM as a judge: Correct\]The three songs by Fujii Kaze that occupied the top 3 positions in Thailand’s chart from April 19 to April 25, 2024 were ”Michiteyuku”, ”Shinunoga E\-Wa”, and ”Matsuri”\.

Figure 3:Example of response quality changes across learning stages\. The CPT response is accurate, buthallucinations \(red text\)appear in the SFT response\. After applying DPO, the response improves to a concise and accurate answer based on facts\. This confirms that DPO suppresses hallucinations and generates responses containing only necessary and sufficient information\.

## Appendix DNews Article Categories

Table 7:Categories of news articles covered in this researchAlthough this research utilized only 128 articles from the”Culture & Entertainment”category, Table[7](https://arxiv.org/html/2606.28898#A4.T7)shows the category classification we established when initially scraping web news articles to create our dataset\.

## Appendix EExperimental Settings Details

Table[8](https://arxiv.org/html/2606.28898#A5.T8)shows the details of the experimental settings\. Table[9](https://arxiv.org/html/2606.28898#A5.T9)shows the settings such as learning rate and batch size used in each learning stage: CPT, SFT, and DPO\.

Table 8:Experimental setting detailsTable 9:Main settings for each learning process
## Appendix FTraining Steps and Computation Time

Table[10](https://arxiv.org/html/2606.28898#A6.T10)shows the number of training steps and time required for each learning stage\. Experiments were conducted on a single NVIDIA H100 GPU\.

Table 10:Number of training steps and computation time for each learning stageThe total computation time per iteration was approximately 80 minutes, requiring about 9\.3 hours for 7 iterations of training\. This time represents only the training time for theTarget LLMand does not include the time for dataset generation, checkpoint saving, and other processes\. The actual execution time needs to account for these additional processes\.

## Appendix GAugmented\-Context Samples

Below are sample paraphrases of news articles used in this research\.

Original ArticleMusic Festival ”MUSIC CIRCUS FUKUOKA 2024” to be held overnight at Fukuoka PayPay DomeThe music festival ”MUSIC CIRCUS FUKUOKA 2024” will be held on June 8, 2024, at Fukuoka PayPay Dome\. This event was first held in Fukuoka in 2018 and is a multi\-faceted festival centered around music performances that also offers fashion, food, and sports activities\. The 2024 lineup includes Awich, Jin Dogg, OZworld, ALAN SHIRAHAMA \(DJ\), among others, who will keep the energy high all night\. Tickets go on sale from April 1, 2024, priced at 7,000 yen for men and 3,500 yen for women\.

Paraphrase Example 1”MUSIC CIRCUS FUKUOKA 2024” confirmed for all\-night event at Fukuoka PayPay DomeThe music festival ”MUSIC CIRCUS FUKUOKA 2024” will take place at Fukuoka PayPay Dome on June 8, 2024\. First launched in Fukuoka in 2018, this is a hybrid festival featuring music performances alongside fashion, food, and sports activities\. The 2024 lineup features Awich, Jin Dogg, OZworld, and ALAN SHIRAHAMA \(DJ\)\. Tickets will be available from April 1, 2024, with prices set at 7,000 yen for men and 3,500 yen for women\.

Paraphrase Example 2”MUSIC CIRCUS FUKUOKA 2024” to host all\-night event at Fukuoka PayPay Dome on June 8The combined music, fashion, food, and sports festival ”MUSIC CIRCUS FUKUOKA,” which debuted in Fukuoka in 2018, will be held at Fukuoka PayPay Dome on June 8, 2024\. Performers include Awich, Jin Dogg, OZworld, and ALAN SHIRAHAMA \(DJ\)\. Tickets will go on sale April 1, 2024, available for 7,000 yen for men and 3,500 yen for women\.

These paraphrase examples maintain the factual information of the original article while presenting the same content with different expressions and structures\. The paraphrases vary in their organization, simplification of expressions, and stylistic changes, but they accurately preserve important information such as the event date, location, performers, and ticket prices\.

## Appendix HContext\-Derived QA Samples

Below are samples of QA generation based on a news article \(the original article from Appendix[G](https://arxiv.org/html/2606.28898#A7)\)\.

Context\-Derived QA Samples: QA about MUSIC CIRCUS FUKUOKA 2024Question 1How much is the ticket price for men at MUSIC CIRCUS FUKUOKA 2024?Answer 1The ticket price for men at MUSIC CIRCUS FUKUOKA 2024 is set at 7,000 yen\.Question 2At which venue in Fukuoka will the MUSIC CIRCUS music festival be held in 2024?Answer 2The MUSIC CIRCUS FUKUOKA 2024 music event will be held at PayPay Dome in Fukuoka\.Question 3When do tickets for MUSIC CIRCUS FUKUOKA 2024 go on sale?Answer 3Tickets for MUSIC CIRCUS FUKUOKA 2024 go on sale from April 1, 2024\.Question 4Please explain the format of MUSIC CIRCUS FUKUOKA 2024\.Answer 4MUSIC CIRCUS FUKUOKA 2024 is a multi\-faceted festival centered around music performances that also includes fashion, food, and sports activities, with events running all night \(overnight format\)\.

Figure 4:QA samples about MUSIC CIRCUS FUKUOKA 2024\.

## Appendix IPrompt Content

Below are the prompts actually used in this research\.

### I\.1Augmented\-ContextGeneration Prompt

While the context content and generated content are in Japanese, English prompts were used to enhance instruction adherence\.

Listing 1:Augmented\-Context

Generation Prompt\#\#InputData

<NewsArticleTitle\>

\{title\}

</NewsArticleTitle\>

<ArticleContent\>

\{content\}

</ArticleContent\>

\#\#OutputFormat

\-JSONLines\(JSONL\)ONLYwithEXACTLY\{n\_variations\}lines

\-EachlineMUSTcontainBOTHa"title"ANDa"content"key\-valuepair

\-Eachlinemustbeacomplete,validJSONobject:\{"title":"\.\.\.","content":"\.\.\."\}

\-Noadditionalkeys,explanations,orblanklines

\-Invalidformats\(title\-onlyorcontent\-onlylines\)willberejected

\#\#IMPORTANT:FormatRequirements

\-EachlineMUSTstrictlyfollowthisexactstructure:\{"title":"transformedtitlehere","content":"transformedcontenthere"\}

\-NEVERsplittitleandcontentintoseparatelines

\-NEVERoutputtitle\-onlyorcontent\-onlyJSONobjects

\-NEVERusetrailingcommasattheendofJSONobjects

\#\#LanguageRequirement

\-Thecontentofallresponses\(theactualtextwithintheQandAfields,nottheJSONLformatitself\)mustbeinJapaneseonly\.

\#\#OutputKeyMapping

<NewsArticleTitle\>"title"

<ArticleContent\>"content"

\#\#TransformationTask

1\.Paraphraseeachelement\(newsarticletitle,articlecontent\)\*\*withoutchangingthemeaningatallbutwithsubstantialrewording\*\*,generating\{n\_variations\}diverseexpressions\.

2\.Combinemultipleparaphrasingtechniques:

\-Wordorderchanges/active<\-\>passivevoice/formal<\-\>informalstyle

\-Paragraphstructureandsentenceorderrearrangement

\-Synonymsubstitution

\-Variationinpunctuationandsymbols

\-Changenumber/dateformatsonly,notthevaluesthemselves

3\.Donotalterpropernouns,numericalvalues,ordatesthemselves\.

4\.Distributionagencies,reporternames,andothermetadataareexcludedfromtransformation\.

5\.Markdownandbackticksareprohibited\.

\#\#ValidOutputExample

\{"title":"transformedtitlehere","content":"transformedcontenthere"\}

\#\#CommonErrorstoAvoid

\-DONOToutputtitle\-onlylineslike:\{"title":"transformedtitle"\}

\-DONOToutputcontent\-onlylineslike:\{"content":"transformedcontent"\}

\-DONOTsplittitleandcontentacrossmultiplelines

\-DONOTaddanyexplanationtextbeforeoraftertheJSONLoutput

\#\#Warning

Ifyououtputanythingotherthan\*\*exactly\{n\_variations\}completeJSONLlines\*\*withBOTHtitleANDcontentinEACHline,youranswerwillbeconsideredinvalid\.

### I\.2Context\-Derived QAGeneration Prompt

While the context content and generated content are in Japanese, English prompts were used to enhance instruction adherence\.

Listing 2:Context\-Derived QA

Generation PromptPleasereplyinJSONLformatonly\.Noothertext\.

<context\>

ArticleTitle:\{title\}

Newsarticlefrom\{date\}

\{contents\}

</context\>

<instructions\>

1\.Readtheabovecontextandcreate\{need\_qa\}diversequestion\(Q\)andanswer\(A\)pairsaboutthecontext,

outputtingeachpairinJSONLformatas\{\{"Q":"\{\{question\}\}","A":"\{\{answer\}\}"\}\}\.

2\.Questions\(Q\)

\-Within200Tokens\.

\-Ensurethattheanswer\(A\)canbeuniquelydeterminedfromQalonewithoutthecontext,

byincludingallnecessarypropernouns,dates,locations,names,figures,organizations,metrics,etc\.specificallyandcompletely\.

\-Prohibited:Pronounsorambiguousexpressionslike"thisarticle,""lastweek,"oranyrelativetimeexpressions\.

\-Yes/Noformatquestionsareprohibited\.

\-Atleastonequestionmustaskaboutthedateofanevent,announcement,oroccurrence\.

Fordatesmentionedwithonlymonthanddayinthearticle,completethemwiththeyearfromcontextandconverttoabsolutedates\.

3\.Answers\(A\)

\-Within300Tokens\.

\-AccuratelydescribetheinformationrequestedinQ\.

4\.LanguageRequirement

\-Thecontentofallresponses\(theactualtextwithintheQandAfields,nottheJSONLformatitself\)mustbeinJapaneseonly\.

5\.DiversityandExclusions

\-CreatemultipleQApairswithnon\-overlappingcontent\.

\-ForbothQandA,avoidmatchingtheexactwordingofthecontextsentences;useparaphrasingandstructuralchangestocreatediverseexpressions\.

\-Paraphrasingshouldmaintainthesamemeaning\.Changingthemeaningisprohibited\(e\.g\.,changing10yearsto12years,changingkanjiinnames,changingpropernouns,etc\.\)

\-Exclude"articlesourceinformation"suchaspublicationsource,distributionsource,medianame,reportername,etc\.frompropernouns,anddonotincludetheminQA\.

6\.ErrorHandling

\-IfthecontextiscorruptedoryoucannotcreateanyQApairsatall,outputonlyoneline:\{\{"Q":"0","A":"0"\}\}\.

7\.OutputmustbeJSONLonly\.Donotincludeanyothertext\.

</instructions\>

### I\.3News Article Cleaning and Quality Evaluation Prompt

This prompt is written in English, but a Japanese prompt was actually used\.

Listing 3:Example prompt for news article cleaning and evaluationPleaseexaminethenewstitle\(\#\#\#NewsTitle\)andthenewsarticlecontent\(\#\#\#NewsArticleContent\)accordingtothefollowinginstructions\(\#\#\#Instructions\)andrespondstrictlyinJSONformat\.TheJSONkeydefinitionsarelistedin\#\#\#JSONDefinition\.Pleaseadherestrictlytothesekeys\.

\#\#\#Instructions

\-Verifyifthenewstitleandnewsarticlecontentareconsistent\.Sincethesearescrapedfromtheweb,theymaybeinconsistent,orthearticlecontentmightincludeunrelatedarticlesoradvertisements\.Thisoccursbecausethestructureofwebsitessometimesincludesothernewsoradvertisingpromotionsafterorinthemiddleofthemainnewsarticle\.Ifyoufindunrelatednewsoradvertisementsmixedin,settheneedsCleaningkeytoTrueinyourresponse,extractonlywhatappearstobetheintendednewsarticle,andprovidethenewstitleintitleandthenewscontentincontent\.Conversely,ifyoudeterminethatthecurrentnewstitleandcontentareappropriateastheyare,settheneedsCleaningkeytoFalseandleavethetitleandcontentfieldsempty\.

\-Assignascoretothearticleaccordingtothescoredefinition\(\#\#\#ScoreDefinition\)andrespondwithasingle\-bytedigitof1,2,3,4,or5inthescorefield\.IftheneedsCleaningkeyisTrue,evaluateonlythecontentyouextractedasthemainnewsarticle\.IftheneedsCleaningkeyisFalse,evaluatebasedontheoriginalnewstitle\(\#\#\#NewsTitle\)andnewsarticlecontent\(\#\#\#NewsArticleContent\)\.

\-Afterreadingthecontentofthisnewsarticle,classifyitintothemostrelevantcategoryaccordingtothenewsarticlecategorydefinition\(\#\#\#NewsArticleCategoryDefinition\)\.Thenewsarticlecategorydefinitionisstructuredasatwo\-leveltree\.Enterthefirstlevelinmaincategoryandthesecondlevelinsubcategoryinyourresponse\.Besuretoclassifyitintooneofthedefinedcategories\.Onlyifyouabsolutelycannotclassifyitintoanycategory,respondwith"other"intherelevantpartofmaincategoryorsubcategory\.

\#\#\#JSONDefinition

\-score

\-maincategory

\-subcategory

\-needsCleaning

\-title

\-content

\#\#\#ScoreDefinition

\-1:Notext\.Doesnotmakesenseasatext\.Randomstrings,etc\.

\-2:Makessenseastextbutisnotanewsarticleoriscompletelydifferenttextunrelatedtothearticle\.Or,doesnotmeettheminimumrequirementsasanarticleduetoinsufficienttextlength\(e\.g\.,themainarticleislessthan30characters\)

\-3:Borderlinequalitythatbarelymeetstheminimumrequirementsasanewsarticle\.Forexample,nodatesoit’sunclearwhentheinformationisfrom,nonamesofsubjectsornamesofobjectssoit’sunclearwhoorwhatisbeingdiscussed,onlytheintroductorytextisavailablewhiletheimportantmainarticleisbehindapaywallandnotscraped,ortheavailableinformationdoesnotincludethebasicsofnews\(when,where/who,whathappened/whatwasdone\)anddoesnotmakecompletesenseasnews\.

\-4:Meetsalltherequirementsasanewsarticle\(when,where/who,whathappened/whatwasdone,why\)andincludesallnecessaryinformation\.

\-5:Inadditiontoscore4,containsdetailedinformationsufficientforanewsarticle,clearlyexplainingorillustratingnewknowledgeorinformation,somechangeintheworld,newideasorimplications,newdiscoveries,scientificadvances,etc\.

\#\#\#NewsArticleCategoryDefinition

Politics&International:

\-DomesticPolitics

\-InternationalRelations

\-Diplomacy

\-Elections

\-Policies

Economy&Business:

\-Macroeconomics

\-Corporate&Industry

\-Finance&Investment

\-Employment&Labor

\-Consumption&Prices

\-EconomicPolicy

\-NewProducts&Services

\-Marketing&Advertising

Society&Incidents:

\-SocialIssues

\-Crime&Incidents

\-Law&Court

\-HumanRights&DiscriminationIssues

\-EducationIssues

Lifestyle&SocialTrends:

\-Lifestyle

\-LocalNews

\-Transportation

\-Housing&RealEstate

\-ConsumerIssues

\-Trends&Fads

\-SocialConditions

\-ChangesinSocialNorms

\-PublicOpinionPolls

Science&Technology&Environment:

\-ScientificResearch

\-TechnologicalInnovation

\-IT&Digital

\-EnvironmentalIssues

\-ClimateChange

\-SpaceDevelopment

\-NewTechnologyApplications

Culture&Entertainment:

\-Arts&Culture

\-Entertainment

\-Sports

\-Food&Cuisine

\-Travel&Tourism

\-Fashion

\-Media

Health&Medical&Welfare:

\-Health&Medical

\-Welfare

\-AgingSociety

\-PublicHealth

Disasters&Accidents:

\-NaturalDisasters

\-Accidents

\-Emergencies

\-DisasterPrevention&Mitigation

Commentary&Analysis:

\-Editorial

\-Column

\-Commentary

\-ExplanatoryArticle

\#\#\#NewsTitle

\{news\_title\}

\#\#\#NewsArticleContent

\{news\_content\}

### I\.4News Article Summarization Prompt

This prompt is written in English, but a Japanese prompt was actually used\.

Listing 4:Example prompt for news article summarizationPleasesummarizethefollowingnewsarticle\(thetextbetweenthe<news\_article\>and</news\_article\>tags\)inlessthan500characters\.

Whensummarizing,pleaseobservethefollowingguidelines:

\-Donotalteroromitpropernouns\(names,placenames,facilitynames,etc\.\)

\-Donotomitdates

<news\_article\>

\{content\}

</news\_article\>

### I\.5LLM as a judge Prompt

Listing 5:Example prompt for answer correctness evaluation using LLM<MainInstruction\>

Pleaseevaluatethecorrectnessoftheresponsetoaquestionaboutthecontext\.

Thecontentofeval\_answeristhesubjectofevaluation\.

Respondonlywithasingledigit:0or1\.Strictlyadheretorespondingwithonly0or1\.Respondwith0ifincorrect\.Respondwith1ifcorrect\.

Fordetailedevaluationcriteria,refertothe"EvaluationCriteria"tagandthe"SupplementaryEvaluationCriteria"tag\.

</MainInstruction\>

<TagDefinitions\>

Belowarethecontext,question,andresponsetobeevaluated\.

The"context"tagsectioncontainsthenewsarticlecontext\.

The"question"tagsectioncontainsthequestionaboutthecontext\.

The"eval\_answer"tagsectioncontainstheresponsetobeevaluated\.Evaluatethecorrectnessofthisresponsetothequestion\.

</TagDefinitions\>

<EvaluationCriteria\>

\-TheanswerisconsideredcorrectifitmeetsALLofthefollowingconditions\.Otherwise,itisincorrect:

\-eval\_answerisfactuallycorrectasaresponsetothequestionaboutthecontext

\-eval\_answerdoesnotcontradictthecontentinthecontext

</EvaluationCriteria\>

<SupplementaryEvaluationCriteria\>

\-Dates,times,andquantitativevaluesmustmatch,buttherequiredprecisionshouldbedeterminedfromthecontext\.Forexample,Olympiccompetitiontimesshouldmatchtotheprecisionstatedinthearticle\(e\.g\.,0\.1seconds\)\.ForeventslikeachangeofPrimeMinister,matchingthedayissufficient\.

\-Propernounssuchasnamesofpeople,places,products,companies,papertitles,etc\.mustmatchexactlyincontent,thoughexpressionsmayvary\.Differentformatssuchaskanji,romaji,hiragana,orEnglishnotationareacceptableaslongastheyrefertoexactlythesameentity\.

\-Dates,times,andnumericalvaluescanbeindifferentformatsaslongastheyrepresentthesamecontent\.Forexample,April1,2024,isthesameas2024/4/1\.1:00PMisthesameas13:00and1o’clockintheafternoon\.10,000yen,10Kyen\(usingabbreviatedthousandnotation\),and"TenThousandYen"\(formalwrittennotation\)arethesame\.

</SupplementaryEvaluationCriteria\>

<context\>

\{content\}

</context\>

<question\>

\{Q\}

</question\>

<eval\_answer\>

\{eval\_answer\}

</eval\_answer\>

Pleasestrictlyadheretorespondingwithonly0or1\.