Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing

arXiv cs.CL Papers
knowledge-editing llm language-models positional-encoding knowledge-recall inference

Summary

The paper introduces FOVEATED, a plug-and-play framework that improves atomic-fact recall in unstructured knowledge editing by perturbing RoPE positions of preceding context during editing to counteract context-induced difficulty underestimation, showing consistent gains across five editors, two LLM backbones, and three benchmarks.

arXiv:2610.02772v1 Announce Type: new Abstract: Large language models (LLMs) increasingly serve as general-purpose interfaces to factual knowledge, but their parameters do not automatically reflect information that changes after pretraining. Knowledge editing (KE) provides a targeted alternative to costly retraining by modifying selected knowledge and preserving unrelated knowledge and general capabilities. Conventional KE uses structured factual triples, whereas unstructured KE (UKE) uses free-form passages containing multiple facts. Nonetheless, existing UKE editors exhibit a failure mode known as context reliance: edited LLMs can often reproduce the editing passage but fail to reliably recall its individual facts without the original passage context. We identify context-induced difficulty underestimation under the standard passage-level editing objective: later facts receive increasingly rich ground-truth context and consequently incur lower initial losses, making them appear easier to learn. In response, we propose FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the Rotary Position Embedding (RoPE) positions assigned to the keys of its preceding context. The perturbation is applied during editing and removed afterward, leaving the model's native positional encoding unchanged at inference time. We instantiate FOVEATED for both direct-optimization and locate-then-edit editors. We theoretically analyze how FOVEATED counteracts context-induced difficulty underestimation and empirically demonstrate consistent improvements across five KE editors, two LLM backbones, and three benchmarks.
Original Article
View Cached Full Text

Cached at: 10/05/26, 10:02 AM 

# Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing
Source: [https://arxiv.org/abs/2610.02772](https://arxiv.org/abs/2610.02772)
[View PDF](https://arxiv.org/pdf/2610.02772)

> Abstract:Large language models \(LLMs\) increasingly serve as general\-purpose interfaces to factual knowledge, but their parameters do not automatically reflect information that changes after pretraining\. Knowledge editing \(KE\) provides a targeted alternative to costly retraining by modifying selected knowledge and preserving unrelated knowledge and general capabilities\. Conventional KE uses structured factual triples, whereas unstructured KE \(UKE\) uses free\-form passages containing multiple facts\. Nonetheless, existing UKE editors exhibit a failure mode known as context reliance: edited LLMs can often reproduce the editing passage but fail to reliably recall its individual facts without the original passage context\. We identify context\-induced difficulty underestimation under the standard passage\-level editing objective: later facts receive increasingly rich ground\-truth context and consequently incur lower initial losses, making them appear easier to learn\. In response, we propose FOVEATED, a plug\-and\-play framework that constructs focused views of each sentence by randomly shifting the Rotary Position Embedding \(RoPE\) positions assigned to the keys of its preceding context\. The perturbation is applied during editing and removed afterward, leaving the model's native positional encoding unchanged at inference time\. We instantiate FOVEATED for both direct\-optimization and locate\-then\-edit editors\. We theoretically analyze how FOVEATED counteracts context\-induced difficulty underestimation and empirically demonstrate consistent improvements across five KE editors, two LLM backbones, and three benchmarks\.

## Submission history

From: Tianci Liu \[[view email](https://arxiv.org/show-email/9e217280/2610.02772)\] **\[v1\]**Fri, 2 Oct 2026 03:58:06 UTC \(251 KB\)

Similar Articles

Benchmarking Knowledge Editing using Logical Rules

arXiv cs.CL

Introduces a benchmark to evaluate how knowledge editing methods handle logical consequences of fact edits, revealing that existing approaches like ROME and FT accurately insert direct assertions but fail to propagate entailed knowledge, with a performance gap of up to 24%.