AGE: Adaptive-masking for Graph Embedding in Graph Retrieval-Augmented Generation
Summary
Introduces Adaptive-masking for Graph Embedding (AGE), a Transformer-based self-supervised learning method that addresses latent feature misalignment between graph and text representations for LLMs in GraphRAG tasks by focusing on predicting non-key nodes.
View Cached Full Text
Cached at: 07/06/26, 06:35 AM
Paper page - AGE: Adaptive-masking for Graph Embedding in Graph Retrieval-Augmented Generation
Source: https://huggingface.co/papers/2607.00052
Abstract
GraphRAG extends RAG by incorporating graph-structured data for LLMs, addressing latent feature misalignment through Adaptive-masking for Graph Embedding (AGE) that uses Transformer-based self-supervised learning with learnable node sampling.
GraphRAG is an extension ofretrieval-augmented generation(RAG) that supportslarge language models(LLMs) by referring tograph-structured dataas external knowledge. While this technique ideally captures intricate relationships, it often struggles with graph representations for LLMs, particularly for frozen LLMs, due to the misalignment between graph-based and text-based latent features. We tackle this issue by introducing the {\it Adaptive-masking for Graph Embedding (AGE)}. AGE employs aTransformerin a mask-basedself-supervised learning(SSL) approach. We designed the architecture similar to text embedding encoders, addressing thelatent feature misalignment. In contrast to natural language texts, graphs are concise representations, and there exist {\itkey nodes} that hold dominant contextual information, which are challenging to predict from their surroundings. Masking suchkey nodesleads to inefficiency in the SSL process. Therefore, AGE focuses on predicting nodes apart fromkey nodes, utilizing a learnablenode sampler. Our experimental results indicate that AGE significantly improves approaches using non-parametric search component inGraphQAtasks, achieving superior accuracy across four benchmark datasets with distinct characteristics.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2607\.00052
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.00052 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.00052 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.00052 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
Why Retrieval-Augmented Generation Fails: A Graph Perspective
This paper investigates why Retrieval-Augmented Generation (RAG) systems fail despite having access to correct evidence. Using circuit tracing and attribution graphs, the authors find that correct predictions exhibit deeper reasoning paths and more distributed evidence flow, while failures show shallow and fragmented patterns. They propose a graph-based error detection framework and targeted interventions to improve RAG reliability.
ContextRAG: Extraction-Free Hierarchical Graph Construction for Retrieval-Augmented Generation
ContextRAG introduces an extraction-free method for constructing hierarchical graph indices for retrieval-augmented generation, using Residual-Quantization K-Means and Formal Concept Analysis to reduce LLM calls and tokens by orders of magnitude while maintaining competitive F1 scores on multi-hop questions.
TERGAD: Structure-Aware Text-Enhanced Representations for Graph Anomaly Detection
TERGAD is a novel data augmentation framework that uses large language models to translate node-level topological properties into semantic narratives, then fuses these with original node attributes via a gated dual-branch autoencoder for graph anomaly detection, achieving state-of-the-art results on six datasets.
Generative Representation Learning on Hyper-relational Knowledge Graphs via Masked Discrete Diffusion
The paper introduces a novel task of fact generation for hyper-relational knowledge graphs (HKGs) and proposes KREPE, a generative representation learning method using masked discrete diffusion that unifies link prediction and fact generation, achieving state-of-the-art performance.
LightRAG: Simple and Fast Retrieval-Augmented Generation
The article introduces LightRAG, an open-source framework that enhances Retrieval-Augmented Generation by integrating graph structures for improved contextual awareness and efficient information retrieval.