通过聚焦视图提升非结构化知识编辑中的原子事实召回能力

arXiv cs.CL 论文
knowledge-editing llm language-models positional-encoding knowledge-recall inference

摘要

论文提出 FOVEATED——一个即插即用的框架，通过在编辑过程中扰动前文上下文的 RoPE 位置，来抵消上下文导致的难度低估，从而提升非结构化知识编辑中的原子事实召回率；该方法在五种编辑器、两种 LLM 主干模型和三个基准测试上均取得一致的性能提升。

arXiv:2610.02772v1 Announce Type: new Abstract: Large language models (LLMs) increasingly serve as general-purpose interfaces to factual knowledge, but their parameters do not automatically reflect information that changes after pretraining. Knowledge editing (KE) provides a targeted alternative to costly retraining by modifying selected knowledge and preserving unrelated knowledge and general capabilities. Conventional KE uses structured factual triples, whereas unstructured KE (UKE) uses free-form passages containing multiple facts. Nonetheless, existing UKE editors exhibit a failure mode known as context reliance: edited LLMs can often reproduce the editing passage but fail to reliably recall its individual facts without the original passage context. We identify context-induced difficulty underestimation under the standard passage-level editing objective: later facts receive increasingly rich ground-truth context and consequently incur lower initial losses, making them appear easier to learn. In response, we propose FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the Rotary Position Embedding (RoPE) positions assigned to the keys of its preceding context. The perturbation is applied during editing and removed afterward, leaving the model's native positional encoding unchanged at inference time. We instantiate FOVEATED for both direct-optimization and locate-then-edit editors. We theoretically analyze how FOVEATED counteracts context-induced difficulty underestimation and empirically demonstrate consistent improvements across five KE editors, two LLM backbones, and three benchmarks.
查看原文
查看缓存全文

缓存时间: 2026/10/05 10:02 

# Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing
Source: [https://arxiv.org/abs/2610.02772](https://arxiv.org/abs/2610.02772)
[View PDF](https://arxiv.org/pdf/2610.02772)

> Abstract:Large language models \(LLMs\) increasingly serve as general\-purpose interfaces to factual knowledge, but their parameters do not automatically reflect information that changes after pretraining\. Knowledge editing \(KE\) provides a targeted alternative to costly retraining by modifying selected knowledge and preserving unrelated knowledge and general capabilities\. Conventional KE uses structured factual triples, whereas unstructured KE \(UKE\) uses free\-form passages containing multiple facts\. Nonetheless, existing UKE editors exhibit a failure mode known as context reliance: edited LLMs can often reproduce the editing passage but fail to reliably recall its individual facts without the original passage context\. We identify context\-induced difficulty underestimation under the standard passage\-level editing objective: later facts receive increasingly rich ground\-truth context and consequently incur lower initial losses, making them appear easier to learn\. In response, we propose FOVEATED, a plug\-and\-play framework that constructs focused views of each sentence by randomly shifting the Rotary Position Embedding \(RoPE\) positions assigned to the keys of its preceding context\. The perturbation is applied during editing and removed afterward, leaving the model's native positional encoding unchanged at inference time\. We instantiate FOVEATED for both direct\-optimization and locate\-then\-edit editors\. We theoretically analyze how FOVEATED counteracts context\-induced difficulty underestimation and empirically demonstrate consistent improvements across five KE editors, two LLM backbones, and three benchmarks\.

## Submission history

From: Tianci Liu \[[view email](https://arxiv.org/show-email/9e217280/2610.02772)\] **\[v1\]**Fri, 2 Oct 2026 03:58:06 UTC \(251 KB\)

相似文章

使用逻辑规则的知识编辑基准测试

arXiv cs.CL

介绍了一个基准测试，用于评估知识编辑方法如何处理事实编辑的逻辑后果，揭示了像ROME和FT这样的现有方法能准确插入直接断言，但未能传播蕴含知识，性能差距高达24%。

一罩定乾坤：编辑后隐藏事实的发现与探究

arXiv cs.LG

本文研究了知识编辑方法ROME和MEMIT的内部机制，揭示了这些编辑依赖于一个共同的权重功能子空间，且是抑制而非覆盖知识，从而解释了编辑为何无法传播到相关事实。