Multimodal Music Recommendation System using LLMs
Summary
Proposes a multimodal framework integrating audio, lyric, and semantic signals with LLM-based sequential reasoning for session-based music recommendation, achieving up to 95% recall improvement over ID-only baselines.
View Cached Full Text
Cached at: 06/05/26, 10:07 AM
Paper page - Multimodal Music Recommendation System using LLMs
Source: https://huggingface.co/papers/2606.00125
Abstract
A multimodal framework for session-based music recommendation integrates audio, lyric, and semantic signals with LLM-based sequential reasoning to improve recommendation accuracy.
Music recommendation systems typically treat songs as opaque tokens, relying on collaborative interaction histories which overlooks semantic or acoustic content. Prior work has explored LLM-augmented, multimodal, and text-enhanced approaches to sequential recommendation, and while some methods partially combine semantic, acoustic, or engagement signals, none jointly model all three within a unified LLM-based sequential reasoning framework that grounds recommendations in actual song content. In this work, we propose amultimodal frameworkfor session-based music recommendation that enriches theLastFM-1K datasetwith three complementary signals: (1) audio andlyric embeddingsextracted using pretrained music andtext representation models, (2)LLM-generated semantic metadatausing theMGPHot annotation schema, and (3)listening completion ratios. We adopt theE4SRec frameworkby extending it with multimodal features and differentitem ID encoder backbones, includingSASRec,BERT4Rec, andGRU4Rec. We further extend the LLM backbone option withLLaMa-2-13B,Qwen2.5-7B-Instruct, andLLaMa-3-70Bin both zero-shot andfine-tuned settings. Our experiments show that integrating content-based features improves over ID-only baselines up to 95% in terms ofRecalland 79% in terms ofNDCG. Moreover, our experiments show thatnaive multimodal fusiondoes not always yield additive improvements, highlighting challenges incross-modal integration. We release a large-scale multimodal benchmark for music recommendation.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2606\.00125
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.00125 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.00125 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.00125 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Multi-Perspective Evidence Synthesis and Reasoning for Unsupervised Multimodal Entity Linking
MSR-MEL introduces an unsupervised framework using LLMs to synthesize and reason over multi-perspective evidence for multimodal entity linking, outperforming prior methods on standard benchmarks.
A Hybrid Framework for Song Lyric Annotation Based on Human-LLM Alignment
This paper introduces a hybrid framework for sentence-level emotion annotation of song lyrics that optimizes human and LLM collaboration by predicting misalignment, addressing subjectivity and scalability challenges in lyric emotion recognition.
Large Language Models as Unified Multimodal Learners for Clinical Prediction
The paper proposes converting multimodal patient data (text, labs, vitals) into a single natural language sequence and fine-tuning LLMs for clinical prediction, achieving comparable or better performance than specialized fusion architectures across three tasks.
Semantic-Aligned Structural Abstraction for Multimodal Sentiment Analysis
The paper proposes SentiLLM, a framework that uses semantic-aligned structural abstraction to distill non-verbal modalities into text-like tokens for multimodal sentiment analysis with LLMs. It introduces a dual-stream salience-context calibration mechanism and achieves superior performance on four datasets.
Symphony of Bias: Exploring Gender Associations with Musical Instruments in Multimodal LLMs
This paper introduces Symphony-Bias, a multimodal dataset for evaluating gender bias in LLMs' associations with musical instruments across text, vision, and audio. It finds that 92% of instrument-level outcomes align with prior social-science findings, with gender biases strongest in text and weakest in audio.