Building a Multimodal Dataset of Academic Paper for Keyword Extraction
Summary
This paper constructs a multimodal dataset of 1000 academic papers with text, images, and audio to study keyword extraction, showing that fusing multiple modalities improves performance.
View Cached Full Text
Cached at: 07/01/26, 05:32 AM
# Building a Multimodal Dataset of Academic Paper for Keyword Extraction Source: [https://arxiv.org/abs/2606.31069](https://arxiv.org/abs/2606.31069) [View PDF](https://arxiv.org/pdf/2606.31069) > Abstract:Up to this point, keyword extraction task typically relies solely on textual data\. Neglecting visual details and audio features from image and audio modalities leads to deficiencies in information richness and overlooks potential correlations, thereby constraining the model's ability to learn representations of the data and the accuracy of model predictions\. Furthermore, the currently available multimodal datasets for keyword extraction task are particularly scarce, further hindering the progress of research on multimodal keyword extraction task\. Therefore, this study constructs a multimodal dataset of academic paper consisting of 1000 samples, with each sample containing paper text, images, audios and keywords\. Based on unsupervised and supervised methods of keyword extraction, experiments are conducted using textual data from papers, as well as text extracted from images and audio\. The aim is to investigate the differences in performance in keyword extraction task with respect to different modal information and the fusion of multimodal information\. The experimental results indicate that text from different modalities exhibits distinct characteristics in the model\. The concatenation of paper text, image text and audio text can effectively enhance the keyword extraction performance of academic papers\. ## Submission history From: Chengzhi Zhang \[[view email](https://arxiv.org/show-email/8ebf732a/2606.31069)\] **\[v1\]**Tue, 30 Jun 2026 02:57:23 UTC \(749 KB\)
Similar Articles
MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image
Introduces MulTaBench, a benchmark of 40 datasets for multimodal tabular learning with text and image modalities, demonstrating that task-specific embedding tuning improves performance over frozen pretrained embeddings, particularly when modalities provide complementary predictive signals.
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
A comprehensive survey of methods, datasets, and benchmarks for multimodal unlearning across vision, language, video, and audio, providing a taxonomy and highlighting open problems.
Multimodal Claim Extraction for Fact-Checking
Researchers present the first benchmark for multimodal claim extraction from social media, evaluating state-of-the-art multimodal LLMs and introducing MICE, an intent-aware framework that improves handling of rhetorical intent and contextual cues in combined text-image posts.
Multimodal Speaker Identification in Classroom Environments
This paper evaluates a multimodal framework for speaker identification in K-12 classrooms by combining acoustic embeddings (ECAPA-TDNN) with LLM-derived semantic context from transcripts, improving accuracy from 39% to 50.3% overall and from 64.9% to 76.9% for longer utterances.
EDU-CIRCUIT-HW: Evaluating Multimodal Large Language Models on Real-World University-Level STEM Student Handwritten Solutions
This paper introduces the EDU-CIRCUIT-HW dataset for evaluating multimodal large language models on real-world university-level STEM handwritten solutions, revealing significant recognition limitations and proposing a hybrid approach that combines automated recognition with minimal human oversight to enhance grading robustness.