Generated Contents Enrichment
Summary
This paper proposes a joint adversarial framework that uses graph convolutional networks to enrich sparse scene graphs before generating semantically richer visual content. The approach makes scene enrichment explicit at the representation level, producing visually plausible and structurally coherent images.
View Cached Full Text
Cached at: 07/06/26, 06:38 PM
Paper page - Generated Contents Enrichment
Source: https://huggingface.co/papers/2405.03650
Abstract
A joint adversarial framework enriches scene graphs using graph convolutional networks to generate semantically richer visual content from sparse scene descriptions.
We study Generated Contents Enrichment (GCE), a conditional image-generation task in which a sparse scene description is first enriched through an explicitscene representationand then rendered into semantically richer visual content. Conventional image-generation systems can produce visually realistic outputs from limited scene descriptions, but the added content is usually implicit in the generator rather than represented as an inspectable intermediate structure. In contrast, GCE seeks to make scene enrichment explicit at the scene-representation level while examining its visual consequences during generation, with the goal of encouraging generated content that is visually plausible, structurally coherent, and semantically richer than the sparse input. To instantiate GCE, we propose a jointly trainedadversarial frameworkthat enrichesscene graphs by modeling object semantics and inter-object relations. Our approach first represents the input description as ascene graph, where nodes model objects and edges capture inter-object relations. The framework usesgraph convolutional networksto predict additional objects and their relations to the existing scene. Finally, the enrichedscene graphis passed through the downstreamimage-generation pipelineto generate the corresponding visual content. We evaluate the framework with proxyscene graphenrichment metrics, image-quality comparisons, qualitative examples, and user studies on the Visual Genome dataset.
View arXiv pageView PDFGitHub0Add to collection
Get this paper in your agent:
hf papers read 2405\.03650
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2405.03650 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2405.03650 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2405.03650 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Context-Aware Interpretable Representations for Retrieval and Graph Convolutional Network Classification
This paper proposes an unsupervised framework combining manifold learning and interpretable graph embeddings to address geometric and interpretability gaps in visual representations, enhancing performance in image retrieval and GCN classification tasks.
Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Prediction
GAS introduces a generation-guided training framework that improves visual understanding in multimodal models by using auxiliary generation tasks with no inference overhead, via a decoupled mixture-of-transformers architecture.
Generative Semantic Scene Completion
This paper presents generative semantic scene completion with corrected performance metrics and real-time inference, releasing code, weights, and the PS3 corpus for reproducibility.
AGE: Adaptive-masking for Graph Embedding in Graph Retrieval-Augmented Generation
Introduces Adaptive-masking for Graph Embedding (AGE), a Transformer-based self-supervised learning method that addresses latent feature misalignment between graph and text representations for LLMs in GraphRAG tasks by focusing on predicting non-key nodes.
Effective Graph and Rank-based Contextual Embeddings for Textual and Multimedia Data
This paper introduces GRaCE, an unsupervised framework for generating interpretable graph embeddings using rank-based measures, which outperforms existing methods in retrieval, classification, and clustering tasks on textual and image data.