Contrastive Mixed Prompt Learning for Incomplete Multimodal Sentiment Analysis with Unseen Modality Combination

# Contrastive Mixed Prompt Learning for Incomplete Multimodal Sentiment Analysis with Unseen Modality Combination
Source: [https://arxiv.org/html/2608.20019](https://arxiv.org/html/2608.20019)
NaiJin LiuYulin KangTangyue JinZixuan YuWenxi ZhaoYibei LiuQianle ZhangYangyang Wu\\correspondingMengying ZhuMeng Xi

###### Abstract

Incomplete multimodal sentiment analysis has garnered significant attention in recent years\. Existing approaches typically assume that data is missing at random or are designed specifically for certain missing patterns, ignoring the modality combination inconsistency between training and testing phases\. However, in real\-world scenarios, the testing phase often encounters modal combinations that were not present during the training phase, which leads to insufficient generalization capabilities and unstable performance\. In this paper, we introduce the problem of Incomplete Multimodal Sentiment Analysis with Unseen Modality Combinations \(IMSAUMC\), aiming to enhance model generalization for unseen modality combinations\. To address this challenge, we propose the model namedContrastiveMixedPromptLearning \(CMPL\) for IMSAUMC\. It introduces a label\-guided contrastive feature learning mechanism to learn robust and discriminative cross\-modal representations\. Additionally, we design modality\-combination prompts with a soft router to facilitate better learning of various modality combinations\. Furthermore, we introduce three prompt contrastive learning strategies, which enable effective learning of prompts corresponding to unseen modality combinations, thereby significantly strengthening the model’s generalization capabilities in diverse testing scenarios\. Extensive experiments on three widely used datasets demonstrate thatCMPLachieves more than a 5% improvement in accuracy compared to state\-of\-the\-art approaches\.

## Introduction

Multimodal Sentiment Analysis \(MSA\) has emerged as a pivotal research area in recent years\. By jointly modeling textual utterances, acoustic characteristics, and visual expressions, MSA systems aim to infer human affective states with significantly enhanced robustness compared to unimodal approaches\. Many works\([7](https://arxiv.org/html/2608.20019#bib.bib20);[40](https://arxiv.org/html/2608.20019#bib.bib21);[10](https://arxiv.org/html/2608.20019#bib.bib22);[19](https://arxiv.org/html/2608.20019#bib.bib23);[28](https://arxiv.org/html/2608.20019#bib.bib39)\)have achieved promising results by exploiting cross\-modal complementarity\.

However, real\-world deployment scenarios frequently violate the full\-modality assumption\. Practical challenges such as sensor failures, background noise, occlusions, and privacy constraints often lead to missing modalities during inference\. Consequently, substantial research efforts have been directed toward MSA with missing modality\([22](https://arxiv.org/html/2608.20019#bib.bib11);[20](https://arxiv.org/html/2608.20019#bib.bib12);[8](https://arxiv.org/html/2608.20019#bib.bib24);[14](https://arxiv.org/html/2608.20019#bib.bib37);[15](https://arxiv.org/html/2608.20019#bib.bib38)\)\.

Figure 1:The difference between the general MSA with missing modality task and the IMSAUMC task\. In the general MSA with missing modality task, all possible modality combinations are available during both the training and testing phases\. However, in our IMSAUMC task, only partial modality combinations are present during training, while unseen modality combinations may appear during testing\.Although these approaches demonstrate resilience against random modality absence, they fundamentally assume that all modality combinations have been observed during training\. In real\-world data, however, missing patterns are often structured rather than random\. For example, a camera failure causes the absence of visual features; a subsequent audio failure then results in missing the visual\-audio modality combination entirely\. Consequently, the dataset may contain only a subset of possible combinations, and the test set is likely to encounter unseen ones\. For another example, users on social media may decline to upload certain modalities \(e\.g\., audio\) due to privacy concerns, yielding datasets with only partial combinations\. In such scenarios, we aim to train models using only the available partial combinations, while still ensuring strong generalization when more complete modality combinations appear at inference time\.

For a dataset withnnmodalities,2n−12^\{n\}\-1possible combinations exist\. As illustrated above, the dataset may not cover all missing patterns, and the test set may encounter unseen modality combinations, especially asnngrows\. Consequently, existing methods often struggle to handle such scenarios effectively, and approaches capable of generalizing to unseen modality combinations are critically needed\.

We introduce the task ofIncompleteMultimodalSentimentAnalysis withUnseenModalityCombinations \(IMSAUMC\), which handles missing modalities and unseen modality combinations during testing, as shown in Figure[1](https://arxiv.org/html/2608.20019#Sx1.F1)\. While previous work in MSA with missing modalities has achieved notable progress, two key challenges remain: First, many works employ contrastive learning to obtain modality\-invariant representations\. However, most fail to account for the intrinsic data structure, potentially separating semantically similar samples, which leads to suboptimal representations \(CH1\)\. Second, in scenarios involving unseen modality combinations, existing methods often overlook the relations between different modality combinations\. As a result struggle to effectively handle test\-time data whose modality combinations were not encountered during the training phase \(CH2\)\.

To address these challenges, we proposeContrastiveMixedPromptLearning \(CMPL\) for multimodal sentiment analysis with unseen modality combinations\. Specifically, forCH1, we introduce label\-guided contrastive feature learning, which incorporates label similarity constraints to pull representations of same\-labeled samples closer while maintaining distances proportional to label dissimilarity for others\. ForCH2, we design mixed modality\-combination prompts with a soft routing mechanism that dynamically selects prompts to comprehensively model inter\-combinatorial relations\. Moreover, we develop three prompt contrastive learning strategies to further enhance generalization to unseen combinations: a modality information preservation strategy, a cross\-combination complementarity strategy, and a conditional information alignment strategy\. The main contributions are summarized as follows:

- •We proposeCMPL, a novel model for the IMSAUMC task that improves generalization to unseen modality combinations\. To the best of our knowledge, this is the first work addressing this problem\.
- •We propose a*label\-guided contrastive feature learning*mechanism, which enforces representation consistency for samples with identical labels while constraining the distance between dissimilar samples proportionally to their label differences\. This preserves the structural relationships within sample representations\.
- •We introduce a*mixed prompts learning*mechanism coupled with three*prompt contrastive learning*strategies\. These comprehensively model inter\-combinatorial relationships and enhance the model’s generalization to unseen modality combinations\.
- •Extensive experiments on CMU\-MOSI, CMU\-MOSEI, and SIMS\-V2 datasets demonstrate the effectiveness of our method over state\-of\-the\-art approaches\.

## Related Work

### Multimodal Sentiment Analysis

Multimodal Sentiment Analysis \(MSA\) aims to infer sentiment by integrating heterogeneous data from multiple modalities, such as text, visual, and acoustic signals\. MSA methods\([26](https://arxiv.org/html/2608.20019#bib.bib25);[33](https://arxiv.org/html/2608.20019#bib.bib1);[21](https://arxiv.org/html/2608.20019#bib.bib26);[23](https://arxiv.org/html/2608.20019#bib.bib2);[16](https://arxiv.org/html/2608.20019#bib.bib3)\)leverage cross\-modal complementarity to improve robustness and accuracy over unimodal approaches\. For instance,[33](https://arxiv.org/html/2608.20019#bib.bib1)proposed self\-MM\([33](https://arxiv.org/html/2608.20019#bib.bib1)\), jointly training a multimodal main task with unimodal subtasks as pseudo\-label supervision to learn inter\-modal consistency and cross\-modal differences\.[23](https://arxiv.org/html/2608.20019#bib.bib2)presented Cube\-MLP\([23](https://arxiv.org/html/2608.20019#bib.bib2)\), which mixes features along three axes via MLP units\.[16](https://arxiv.org/html/2608.20019#bib.bib3)introduced DMD\([16](https://arxiv.org/html/2608.20019#bib.bib3)\), decoupling homogeneous and heterogeneous features with adaptive cross\-modal distillation to enhance modality discriminability\.[28](https://arxiv.org/html/2608.20019#bib.bib39)proposed DLF\([28](https://arxiv.org/html/2608.20019#bib.bib39)\), a disentangled language\-focused framework that reduces cross\-modal redundancy for improved MSA performance\.

![Refer to caption](https://arxiv.org/html/2608.20019v1/model.png)Figure 2:The framework of ourCMPL, which consists of three components: the*labeled\-guided contrastive feature learning*module, the*mixed prompts learning*mechanism, and*prompt contrastive learning*strategies\. Taking the absence of audio\-visual and text\-audio\-visual modality combinations in training data, as well as the missing vision modality in input, as an example\.However, real\-world data often suffers from missing modalities\. Many methods\([34](https://arxiv.org/html/2608.20019#bib.bib4);[38](https://arxiv.org/html/2608.20019#bib.bib27);[35](https://arxiv.org/html/2608.20019#bib.bib28);[29](https://arxiv.org/html/2608.20019#bib.bib6);[39](https://arxiv.org/html/2608.20019#bib.bib7)\)have been developed to address MSA with missing modality\. For example, TFR\-Net\([34](https://arxiv.org/html/2608.20019#bib.bib4)\)employs a feature reconstruction module to generate missing modality content\. LNLN\([39](https://arxiv.org/html/2608.20019#bib.bib7)\)improves robustness by guaranteeing high\-quality dominant modality representation\. HME\([42](https://arxiv.org/html/2608.20019#bib.bib41)\)leverages cross\-sample semantic enrichment and uncertainty\-aware fusion, eliminating explicit modality reconstruction while enhancing robustness and generalization\. MFMB\-Net\([24](https://arxiv.org/html/2608.20019#bib.bib42)\)jointly performs global–local dual\-stream fusion and collaborative feature reconstruction to robustly handle missing modalities\. However, these methods overlook inconsistent distributions of modality combinations in missing\-modality scenarios, where the test set may contain unseen modality combinations that were not present during training\. In contrast, our approach focuses on leveraging known modality combination information to enhance generalization to unseen combinations\.

### Prompt Learning

Prompt learning has emerged as a powerful paradigm for adapting pre\-trained models such as large language models \(LLM\) to downstream tasks\([4](https://arxiv.org/html/2608.20019#bib.bib29);[6](https://arxiv.org/html/2608.20019#bib.bib30);[17](https://arxiv.org/html/2608.20019#bib.bib31);[41](https://arxiv.org/html/2608.20019#bib.bib40)\)\.[27](https://arxiv.org/html/2608.20019#bib.bib8)concatenated visual embeddings as prefix prompts to enable frozen language models to generate appropriate captions\([27](https://arxiv.org/html/2608.20019#bib.bib8)\)\.[13](https://arxiv.org/html/2608.20019#bib.bib9)designed missing\-aware prompts for different missing\-modality cases to enhance robustness\([13](https://arxiv.org/html/2608.20019#bib.bib9)\)\.[12](https://arxiv.org/html/2608.20019#bib.bib10)designed branch\-aware multi\-modal prompts to enhance alignment between language and visual modalities\([12](https://arxiv.org/html/2608.20019#bib.bib10)\)\. MPLMM\([5](https://arxiv.org/html/2608.20019#bib.bib5)\)generates missing modality features and strengthens intra\- and inter\-modality learning by designing generative, missing\-signal, and missing\-type prompts\. These methods ignore the relations between modality combinations, limiting generalization to unseen combinations\. In contrast, our approach thoroughly explores inter\-modal relationships and designs three prompt contrastive learning strategies to address unseen modality combinations\.

## Methodology

### Problem Formulation

Given a multimodal dataset𝐗=\{𝐗t,𝐗a,𝐗v\}\\mathbf\{X\}=\\\{\\mathbf\{X\}^\{t\},\\mathbf\{X\}^\{a\},\\mathbf\{X\}^\{v\}\\\}with three modalities \(text, audio, visual\), each𝐗k=\{𝐱1k,⋯,𝐱Nk\}∈ℝN×lk×dk\\mathbf\{X\}^\{k\}=\\\{\{\\mathbf\{x\}^\{k\}\_\{1\},\\cdots,\\mathbf\{x\}^\{k\}\_\{N\}\}\\\}\\in\\mathbb\{R\}^\{N\\times l\_\{k\}\\times d\_\{k\}\}denotes the feature matrix of modalitykk, whereNNis the number of samples, andlkl\_\{k\},dkd\_\{k\}are the sequence length and embedding dimension, withk∈\{t,a,v\}k\\in\\\{t,a,v\\\}\. The sentiment labels are𝐘∈ℝN\\mathbf\{Y\}\\in\\mathbb\{R\}^\{N\}\. For missing modalities, we define mask matrices𝐌t\\mathbf\{M\}^\{t\},𝐌a\\mathbf\{M\}^\{a\},𝐌v\\mathbf\{M\}^\{v\}where𝐌ik=0\\mathbf\{M\}^\{k\}\_\{i\}=0indicates theii\-th sample is missing in modalitykkand𝐌ik=1\\mathbf\{M\}^\{k\}\_\{i\}=1indicates its presence\. The textual features are obtained from text embeddings of a LLM, while audio and visual features are extracted using pre\-trained toolkits\. For these three modalities, excluding the all\-absent case, there areT=7T=7possible modality combinations, denoted as𝒮=\{S1,S2,⋯,ST\}=\{\{t\},\{a\},\{v\},\{t,a\},\{t,v\},\{a,v\},\{t,a,v\}\}\\mathcal\{S\}=\\\{S\_\{1\},S\_\{2\},\\cdots,S\_\{T\}\\\}=\\\{\\\{t\\\},\\\{a\\\},\\\{v\\\},\\\{t,a\\\},\\\{t,v\\\},\\\{a,v\\\},\\\{t,a,v\\\}\\\}\. For example,S4S\_\{4\}represents the text\-audio combination with visual modality missing, with available data𝐱=\{𝐱t,𝐱a\}\\mathbf\{x\}=\\\{\\mathbf\{x\}^\{t\},\\mathbf\{x\}^\{a\}\\\}\. More detailed information can be found in Appendix[A](https://arxiv.org/html/2608.20019#A1)\.

*Definition\.*The objective of IMSAUMC is to train a model for sentiment analysis under the condition that the training data only contains a subset of the modality combinations in𝒮\\mathcal\{S\}, while the test data includes all possible modality combinations \(i\.e\.,𝒮\\mathcal\{S\}\)\. The model must generalize to unseen modality combinations during testing\.

### Overall Framework

Figure[2](https://arxiv.org/html/2608.20019#Sx2.F2)shows the framework of ourCMPL\. First, during the representation learning stage, we extract sequential embeddings for the audio and visual modalities using pre\-trained tools and reduce the sequence length via adaptive average pooling\. For the textual modality, we leverage text embeddings from the LLM to obtain contextually correlated embeddings\. Then, guided by label similarity, we apply contrastive learning across modality embeddings to learn robust representations while preserving semantic structure\. Additionally, we equip each modality combination with a prompt\. The multimodal embeddings are then fed into a router to generate a mixed prompt\. Furthermore, three prompt contrastive learning strategies are employed to exploit relations between modality combinations, improving generalization to unseen modality combinations\. Finally, we trainCMPLwith the following objective function:

ℒCMPL=ℒt​a​s​k\+α⋅ℒl​c​f​l\+β⋅ℒp​c​l,\\mathcal\{L\}\_\{\\textsf\{CMPL\}\}=\\mathcal\{L\}\_\{task\}\+\\alpha\\cdot\\mathcal\{L\}\_\{lcfl\}\+\\beta\\cdot\\mathcal\{L\}\_\{pcl\},\(1\)whereℒt​a​s​k\\mathcal\{L\}\_\{task\},ℒl​c​f​l\\mathcal\{L\}\_\{lcfl\}, andℒp​c​l\\mathcal\{L\}\_\{pcl\}are task\-specific loss, label\-guided contrastive loss, and prompt contrastive loss, respectively\. Here,ℒt​a​s​k\\mathcal\{L\}\_\{task\}is used to guide the model’s predictions, and we employ the traditional autoregressive cross\-entropy loss from LLMs for this purpose\. The parametersα\\alphaandβ\\betaare the balanced factors onℒl​c​f​l\\mathcal\{L\}\_\{lcfl\}andℒp​c​l\\mathcal\{L\}\_\{pcl\}, respectively\.

### Label\-guided Contrastive Feature Learning

Contrastive learning, as an effective representation learning method, has been widely applied in multimodal domains\. Existing methods typically maximize the similarity of representations across multiple modalities while minimizing the similarity between different samples directly\. However, these approaches often overlook the structural relationships between samples, potentially separating representations of similarly labeled samples and leading to suboptimal representations\.

To address these challenges, we propose the*label\-guided contrastive feature learning*\(LCFL\) module\. This mechanism aims to pull together latent representations of different modalities from the same class while preserving similarity between samples of related classes\. By incorporating class\-aware semantic relationships, this mechanism enables the learned representations to incorporate richer and more robust semantic information, enhancing the model’s understanding of multimodal representations\.

Specifically, for audio and vision modalities, we first project the available embeddings into the LLM’s text embedding space and use an adaptive pooling to reduce the sequence\([31](https://arxiv.org/html/2608.20019#bib.bib13)\)as follows:

𝐗¯a\\displaystyle\\bar\{\\mathbf\{X\}\}^\{a\}=Pool​\(σ⁡\(𝐗a⋅𝐖1a\)⋅𝐖2a\),\\displaystyle=\\text\{Pool\}\(\\sigma\(\\mathbf\{X\}^\{a\}\\cdot\\mathbf\{W\}^\{a\}\_\{1\}\)\\cdot\\mathbf\{W\}^\{a\}\_\{2\}\),\(2\)whereσ\\sigmais the activation function\.𝐖1a\\mathbf\{W\}^\{a\}\_\{1\}and𝐖2a\\mathbf\{W\}^\{a\}\_\{2\}are trainable parameters\. The formulation for𝐗¯v\\bar\{\\mathbf\{X\}\}^\{v\}follows analogously\.𝐗¯a∈ℝN×l×d\\bar\{\\mathbf\{X\}\}^\{a\}\\in\\mathbb\{R\}^\{N\\times l\\times d\}and𝐗¯v∈ℝN×l×d\\bar\{\\mathbf\{X\}\}^\{v\}\\in\\mathbb\{R\}^\{N\\times l\\times d\}are the audio and visual embeddings after projection and adaptive pooling, respectively\. Then, we project the embeddings of all modalities into the contrastive learning space\([2](https://arxiv.org/html/2608.20019#bib.bib14)\)as follows:

𝐇k=σ⁡\(𝐗k⋅𝐖1k\)⋅𝐖2k,\\displaystyle\\mathbf\{H\}^\{k\}=\\sigma\(\\mathbf\{X\}^\{k\}\\cdot\\mathbf\{W\}^\{k\}\_\{1\}\)\\cdot\\mathbf\{W\}^\{k\}\_\{2\},\(3\)wherek∈\{t,a,v\}\{k\}\\in\\\{t,a,v\\\}\.𝐇k∈ℝN×l×dc\\mathbf\{H\}^\{k\}\\in\\mathbb\{R\}^\{N\\times l\\times d\_\{c\}\}is the representation of modalitykkafter projection\. For representation𝐇iw\\mathbf\{H\}^\{w\}\_\{i\}, we treat the instances have the same label as positive pairs, denoted as𝐇ju\|𝐘j=𝐘i,u≠w\|j≠i\\mathbf\{H\}^\{u\}\_\{j\}\|\_\{\\mathbf\{Y\}\_\{j\}=\\mathbf\{Y\}\_\{i\},\\ u\\neq w\|j\\neq i\}, while considering others as negative pairs, denoted as𝐇ju\|𝐘j≠𝐘i\\mathbf\{H\}^\{u\}\_\{j\}\|\_\{\\mathbf\{Y\}\_\{j\}\\neq\\mathbf\{Y\}\_\{i\}\}, whereu,w∈\{t,a,v\}u,w\\in\\\{t,a,v\\\}\. We use the cosine distance to evaluate the similarity between𝐇iw\\mathbf\{H\}^\{w\}\_\{i\}and𝐇ju\\mathbf\{H\}^\{u\}\_\{j\}:d⁡\(𝐇iw,𝐇ju\)=⟨𝐇iw,𝐇ju⟩/‖𝐇iw‖⋅‖𝐇ju‖,d\(\\mathbf\{H\}^\{w\}\_\{i\},\\mathbf\{H\}^\{u\}\_\{j\}\)=\{\\langle\\mathbf\{H\}^\{w\}\_\{i\},\\mathbf\{H\}^\{u\}\_\{j\}\\rangle\}/\{\\\|\\mathbf\{H\}^\{w\}\_\{i\}\\\|\\cdot\\\|\\mathbf\{H\}^\{u\}\_\{j\}\\\|\},where⟨⋅,⋅⟩\\langle\\cdot,\\cdot\\rangleis the dot product operator\.

To effectively enhance the model’s comprehension of multimodal embeddings and explore cross\-modal relationships, we design a LCFL loss functionℒl​c​f​l\\mathcal\{L\}\_\{lcfl\}\. Our method maximizes similarity between positive pairs while maintaining the similarity of negative pairs according to their label relations\. This approach effectively mitigates the adverse effects of incorrectly pushing apart embeddings sharing similar labels\. Given text and audio modality as an example, the contrastive lossℒ\(t,a\)\\mathcal\{L\}^\{\(t,a\)\}between𝐇t\\mathbf\{H\}^\{t\}and𝐇a\\mathbf\{H\}^\{a\}can be defined as:

ℒ\(t,a\)=−12​N∑w=t,a∑i=1N𝐌wilog𝒯iw𝒯iw\+𝒩iw,\\mathcal\{L\}^\{\(t,a\)\}=\-\\frac\{1\}\{2N\}\\sum\_\{w=t,a\}\\sum\_\{i=1\}^\{N\}\\mathbf\{M\}^\{w\}\_\{i\}\\log\\frac\{\\mathcal\{T\}^\{w\}\_\{i\}\}\{\\mathcal\{T\}^\{w\}\_\{i\}\+\\mathcal\{N\}^\{w\}\_\{i\}\},\(4\)where𝒩wi=∑j=1N∑u=t,a𝐌uj⋅𝕀\[Si,j≠1\]⋅e\|d⁡\(𝐇iw,𝐇ju\)−Si,j\|/τ\\mathcal\{N\}^\{w\}\_\{i\}=\\sum\_\{j=1\}^\{N\}\\sum\_\{u=t,a\}\{\\mathbf\{M\}^\{u\}\_\{j\}\\cdot\\mathbb\{I\}\_\{\[S\_\{i,j\}\\neq 1\]\}\\cdot e^\{\|d\(\\mathbf\{H\}^\{w\}\_\{i\},\\mathbf\{H\}^\{u\}\_\{j\}\)\-S\_\{i,j\}\|/\\tau\}\}and𝒯wi=∑j=1N∑u=t,a𝐌uj⋅𝕀\[Si,j=1\]⋅ed⁡\(𝐇iw,𝐇ju\)/τ−e1/τ\\mathcal\{T\}^\{w\}\_\{i\}=\\sum\_\{j=1\}^\{N\}\\sum\_\{u=t,a\}\{\\mathbf\{M\}^\{u\}\_\{j\}\\cdot\\mathbb\{I\}\_\{\[S\_\{i,j\}=1\]\}\\cdot e^\{d\(\\mathbf\{H\}^\{w\}\_\{i\},\\mathbf\{H\}^\{u\}\_\{j\}\)/\\tau\}\}\-e^\{1/\\tau\}\.τ\\tauis the temperature parameter that adjusts the softness\.NNrepresents the number of instances\.𝕀\[Si,j=1\]\\mathbb\{I\}\_\{\[S\_\{i,j\}=1\]\}is the indicator function that equals 1*iff*Si,j=1S\_\{i,j\}=1\.Si,jS\_\{i,j\}is the similarity between labels ofii\-th andjj\-th instance\. Here, we employ a simple approach to measure inter\-label similarity:

Si,j=1−\|𝐘j−𝐘i\|max⁡\{𝐘\}−min⁡\{𝐘\},S\_\{i,j\}=1\-\\frac\{\\left\|\\mathbf\{Y\}\_\{j\}\-\\mathbf\{Y\}\_\{i\}\\right\|\}\{\\max\\\{\\mathbf\{Y\}\\\}\-\\min\\\{\\mathbf\{Y\}\\\}\},\(5\)wheremax⁡\{𝐘\}\\max\\\{\\mathbf\{Y\}\\\}andmin⁡\{𝐘\}\\min\\\{\\mathbf\{Y\}\\\}represent the maximum and minimum values of the labels, respectively\.

Similarly, we can computeℒ\(t,v\)\\mathcal\{L\}^\{\(t,v\)\}andℒ\(v,a\)\\mathcal\{L\}^\{\(v,a\)\}\. Then, the objective functionℒl​c​f​l\\mathcal\{L\}\_\{lcfl\}can be calculated as follows:

ℒl​c​f​l=ℒ\(t,a\)\+ℒ\(t,v\)\+ℒ\(v,a\)\.\\mathcal\{L\}\_\{lcfl\}=\\mathcal\{L\}^\{\(t,a\)\}\+\\mathcal\{L\}^\{\(t,v\)\}\+\\mathcal\{L\}^\{\(v,a\)\}\.\(6\)
By minimizingℒl​c​f​l\\mathcal\{L\}\_\{lcfl\}, the representations of instances with consistent labels are pulled closer, while the similarity between others aligns with their label similarity\. This captures more structured representations and effectively enhances multimodal learning in incomplete combinations, effectively facilitating learning for unseen modality combinations\.

### Mixed Prompts Learning

Most LLM fine\-tuning methods employ LoRA for adaptation\. However, in the IMSAUMC task, using LoRA alone to simultaneously train multiple modality combinations fails to distinguish between them\. To address this, we design a*mixed prompts learning*mechanism to enhance the model’s understanding of diverse modality combinations\.

Specifically, we design*modality\-combination prompts*,i\.e\.,𝒫=\{𝐏1,𝐏2,⋯,𝐏T\}\\mathcal\{P\}=\\\{\\mathbf\{P\}^\{1\},\\mathbf\{P\}^\{2\},\\cdots,\\mathbf\{P\}^\{T\}\\\}, where𝐏i∈ℝlp×dp\\mathbf\{P\}^\{i\}\\in\\mathbb\{R\}^\{l\_\{p\}\\times d\_\{p\}\}is the prompt embedding for theii\-th modality combination, withlpl\_\{p\}anddpd\_\{p\}being its sequence length and dimension\. Traditional methods concatenate each modality\-combination prompt with its corresponding input and feed them into the LLM, learning the prompts from available data\. However, in IMSAUMC tasks, the training set does not cover all modality combinations appearing in the test set\. Consequently, prompts for unseen combinations lack training data and cannot be learned\. Recognizing that different modality combinations are not independent but exhibit correlations, we propose a*Soft Routing\-inspired mixed prompts guidance*mechanism\. Given multimodal data𝐗j=\[𝐗jt,𝐗¯ja\]∈ℝN×2​l×d\\mathbf\{X\}\_\{j\}=\[\\mathbf\{X\}\_\{j\}^\{t\},\\bar\{\\mathbf\{X\}\}\_\{j\}^\{a\}\]\\in\\mathbb\{R\}^\{N\\times 2l\\times d\}, where\[⋯\]\[\\cdots\]denotes sequence concatenation, this mechanism feeds𝐗j\\mathbf\{X\}\_\{j\}into a router that automatically selects and weights prompts according to the input representation:

G⁡\(𝐗j\)=Softmax​\(𝐗j⋅𝐖g\),G\(\\mathbf\{X\}\_\{j\}\)=\\text\{Softmax\}\(\\mathbf\{X\}\_\{j\}\\cdot\\mathbf\{W\}\_\{g\}\),\(7\)where𝐖g∈ℝd×T\\mathbf\{W\}\_\{g\}\\in\\mathbb\{R\}^\{d\\times T\}is the router’s parameter andG⁡\(𝐗j\)G\(\\mathbf\{X\}\_\{j\}\)represents the soft assignment weights\. Subsequently, we obtain the final mixed prompt via dynamic blending:

𝐏¯j=∑i=1TG​\(𝐗j\)i⋅𝐏i,\\bar\{\\mathbf\{P\}\}\_\{j\}=\\sum\\nolimits\_\{i=1\}^\{T\}\{G\(\\mathbf\{X\}\_\{j\}\)\_\{i\}\\cdot\\mathbf\{P\}^\{i\}\},\(8\)where𝐏¯j\\bar\{\\mathbf\{P\}\}\_\{j\}is the mixed prompt for thejj\-th instance\.

Finally, the mixed prompt𝐏¯j\\bar\{\\mathbf\{P\}\}\_\{j\}and multimodal input𝐗j\\mathbf\{X\}\_\{j\}are jointly fed into the LLM to produce the output:

𝐘¯j=LLM​\(𝐏¯j,𝐗j,θ\),\\bar\{\\mathbf\{Y\}\}\_\{j\}=\\text\{LLM\}\(\\bar\{\\mathbf\{P\}\}\_\{j\},\\mathbf\{X\}\_\{j\};\\theta\),\(9\)whereθ\\thetarepresents the LLM’s parameters and𝐘¯j\\bar\{\\mathbf\{Y\}\}\_\{j\}is the generated text with sentiment class and sentiment score\. Following standard LLM training, we adopt next\-token prediction loss\. Thus, the task lossℒt​a​s​k\\mathcal\{L\}\_\{task\}is:

ℒt​a​s​k=∑i=1N∑j=1K−logP\(Li,j\|𝐏i¯,𝐗i,θ\),\\mathcal\{L\}\_\{task\}=\\sum\\nolimits\_\{i=1\}^\{N\}\\sum\\nolimits\_\{j=1\}^\{K\}\{\-\\log P\(L\_\{i,j\}\|\\bar\{\\mathbf\{P\}\_\{i\}\},\\mathbf\{X\}\_\{i\},\\theta\)\},\(10\)whereKKis the number of label tokens andLi,jL\_\{i,j\}is thejj\-th label token of the𝐘¯i\\bar\{\\mathbf\{Y\}\}\_\{i\}generated by theii\-th instance\.

### Prompt Contrastive Learning

In the IMSAUMC task, a key challenge lies in effectively leveraging knowledge from existing modality combinations to enhance the learning of unseen modality combinations\. Recognizing that different modality combinations are not isolated but inherently interrelated, we design three prompt contrastive learning strategies to enable mutual learning among prompts: \(1\)*modality information preservation*strategy; \(2\)*cross\-combination complementarity*strategy; \(3\)*conditional information alignment*strategy\. The core idea is to consider the relationships between various modality combinations to minimize the conditional entropy or maximize the conditional mutual information\.

To compute conditional entropy and mutual information, we first project the modality\-combination prompt embeddings, then average them along the sequence dimension, and finally apply the softmax function, which allows the prompt representation to be interpreted as a probability distribution, enabling entropy and mutual information estimation:

𝐏^i=Softmax​\(Avg​\(σ⁡\(𝐏i⋅𝐖1i\)⋅𝐖2i\)\),\\hat\{\\mathbf\{P\}\}^\{i\}=\\text\{Softmax\}\(\\text\{Avg\}\(\\sigma\(\\mathbf\{P\}^\{i\}\\cdot\\mathbf\{W\}^\{i\}\_\{1\}\)\\cdot\\mathbf\{W\}^\{i\}\_\{2\}\)\),\(11\)where𝐏^i∈ℝlp′×D\\hat\{\\mathbf\{P\}\}^\{i\}\\in\\mathbb\{R\}^\{l^\{\\prime\}\_\{p\}\\times D\}are the normalized prompt embeddings ofii\-th modality combination\. For convenience, we denote the set of unimodal prompts as𝒫u=\{𝐏^1,𝐏^2,𝐏^3\}\\mathcal\{P\}\_\{u\}=\\\{\\hat\{\\mathbf\{P\}\}^\{1\},\\hat\{\\mathbf\{P\}\}^\{2\},\\hat\{\\mathbf\{P\}\}^\{3\}\\\}, the set of dual modality prompts as𝒫d=\{𝐏^4,𝐏^5,𝐏^6\}\\mathcal\{P\}\_\{d\}=\\\{\\hat\{\\mathbf\{P\}\}^\{4\},\\hat\{\\mathbf\{P\}\}^\{5\},\\hat\{\\mathbf\{P\}\}^\{6\}\\\}, and the set of full modality prompts as𝒫f=\{𝐏^7\}\\mathcal\{P\}\_\{f\}=\\\{\\hat\{\\mathbf\{P\}\}^\{7\}\\\}\.

*Modality Information Preservation\.*For a multimodal prompt, it inherently contains the information present in each of its unimodal components\. Therefore, it can be argued that when given a multimodal prompt, it should retain the sub\-modality\-combination prompts it encompasses as much as possible\. To achieve this, we minimize the conditional entropyH⁡\(𝐏^i∣𝐏^j\)H\(\\hat\{\\mathbf\{P\}\}^\{i\}\\mid\\hat\{\\mathbf\{P\}\}^\{j\}\)between such sub\-modality\-combination𝐏^i\\hat\{\\mathbf\{P\}\}^\{i\}and multi\-modality\-combination prompt𝐏^j\\hat\{\\mathbf\{P\}\}^\{j\}\. Since each element of𝐏^i\\hat\{\\mathbf\{P\}\}^\{i\}and𝐏^j\\hat\{\\mathbf\{P\}\}^\{j\}can be treated as probability distribution of two variablesziz\_\{i\}andzjz\_\{j\}overDDclasses\([11](https://arxiv.org/html/2608.20019#bib.bib18);[9](https://arxiv.org/html/2608.20019#bib.bib19);[18](https://arxiv.org/html/2608.20019#bib.bib17)\), whereDDis the dimensionality of𝐏^i\\hat\{\\mathbf\{P\}\}^\{i\}and𝐏^j\\hat\{\\mathbf\{P\}\}^\{j\}\. The joint probability distributionP\(m,n\)∈ℝD×DP^\{\(m,n\)\}\\in\\mathbb\{R\}^\{D\\times D\}can be defined as:

Pi,j\(m,n\)=1l​∑k=1l𝐏^k,im​𝐏^k,jn\.P^\{\(m,n\)\}\_\{i,j\}=\\frac\{1\}\{l\}\\sum\_\{k=1\}^\{l\}\{\\hat\{\\mathbf\{P\}\}^\{m\}\_\{k,i\}\\hat\{\\mathbf\{P\}\}^\{n\}\_\{k,j\}\}\.LetPd\(m,n\)P^\{\(m,n\)\}\_\{d\}andPd′\(m,n\)P^\{\(m,n\)\}\_\{d^\{\\prime\}\}denote the margin probability distributionsP\(m,n\)​\(zm=d\)P^\{\(m,n\)\}\(z\_\{m\}=d\)andP\(m,n\)​\(zn=d′\)P^\{\(m,n\)\}\(z\_\{n\}=d^\{\\prime\}\), which can be obtained by summing thedd\-th rows andd′d^\{\\prime\}\-th columns ofPP\. We can define the loss function between themm\-th prompt and thenn\-th promptℒ\(m,n\)\\mathcal\{L\}^\{\(m,n\)\}as follows:

ℒ\(m,n\)=H\(𝐏^m∣𝐏^n\)=−∑d=1D∑d′=1DPd,d′\(m,n\)lnPd,d′\(m,n\)Pd′\(m,n\)\.\\mathcal\{L\}^\{\(m,n\)\}=H\(\\hat\{\\mathbf\{P\}\}^\{m\}\\mid\\hat\{\\mathbf\{P\}\}^\{n\}\)=\-\\sum\_\{d=1\}^\{D\}\\sum\_\{d^\{\\prime\}=1\}^\{D\}\{\}P^\{\(m,n\)\}\_\{d,d^\{\\prime\}\}\\ln\\frac\{P^\{\(m,n\)\}\_\{d,d^\{\\prime\}\}\}\{P^\{\(m,n\)\}\_\{d^\{\\prime\}\}\}\.The loss functionℒm​i​p\\mathcal\{L\}\_\{mip\}can be defined as follows:

ℒm​i​p=∑m=1,2ℒ\(m,4\)\+∑m=1,3ℒ\(m,5\)\+∑m=2,3ℒ\(m,6\)\+∑m=16ℒ\(m,7\)\.\\mathcal\{L\}\_\{mip\}=\\sum\_\{m=1,2\}\\mathcal\{L\}^\{\(m,4\)\}\+\\sum\_\{m=1,3\}\\mathcal\{L\}^\{\(m,5\)\}\+\\sum\_\{m=2,3\}\\mathcal\{L\}^\{\(m,6\)\}\+\\sum\_\{m=1\}^\{6\}\\mathcal\{L\}^\{\(m,7\)\}\.
*Cross\-Combination Complementarity*\. For two prompts that share a common modality—such as𝐏^4\\hat\{\\mathbf\{P\}\}^\{4\}\(text\-audio combination\) and𝐏^5\\hat\{\\mathbf\{P\}\}^\{5\}\(text\-visual combination\), their shared textual information acts as a bridge connecting the other two modalities,i\.e\., audio and visual\. Therefore, when given such prompts𝐏^4\\hat\{\\mathbf\{P\}\}^\{4\}and𝐏^5\\hat\{\\mathbf\{P\}\}^\{5\}, the uncertainty of the visual and audio prompt𝐏^6\\hat\{\\mathbf\{P\}\}^\{6\}should also decrease\. Hence, we minimize the conditional entropyH⁡\(𝐏^6\|𝐏^4,𝐏^5\)H\(\\hat\{\\mathbf\{P\}\}^\{6\}\|\\hat\{\\mathbf\{P\}\}^\{4\},\\hat\{\\mathbf\{P\}\}^\{5\}\)\. More generally, we aim to minimize the conditional entropyH⁡\(𝐏^l\|𝐏^m,𝐏^n\)H\(\\hat\{\\mathbf\{P\}\}^\{l\}\|\\hat\{\\mathbf\{P\}\}^\{m\},\\hat\{\\mathbf\{P\}\}^\{n\}\), where𝐏^l,𝐏^m,𝐏^n∈𝒫d\\hat\{\\mathbf\{P\}\}^\{l\},\\hat\{\\mathbf\{P\}\}^\{m\},\\hat\{\\mathbf\{P\}\}^\{n\}\\in\\mathcal\{P\}\_\{d\}andll,mm, andnnare mutually distinct\. Similarly, we firstly define the joint probability distributionP\(l,m,n\)∈ℝD×D×DP^\{\(l,m,n\)\}\\in\\mathbb\{R\}^\{D\\times D\\times D\}ofzlz\_\{l\},zmz\_\{m\}, andznz\_\{n\}as follows:

Pi,j,k\(l,m,n\)=1l​∑t=1l𝐏^t,il​𝐏^t,jm​𝐏^t,kn\.P^\{\(l,m,n\)\}\_\{i,j,k\}=\\frac\{1\}\{l\}\\sum\_\{t=1\}^\{l\}\\hat\{\\mathbf\{P\}\}^\{l\}\_\{t,i\}\\hat\{\\mathbf\{P\}\}^\{m\}\_\{t,j\}\\hat\{\\mathbf\{P\}\}^\{n\}\_\{t,k\}\.The loss function between thell\-th,mm\-th, andnn\-th prompts can be defined as:

ℒ\(l,m,n\)−∑d1=1D∑d2=1D∑d3=1DPd1,d2,d3\(l,m,n\)​ln⁡Pd1,d2,d3\(l,m,n\)Pd2,d3\(l,m,n\),\\displaystyle\\mathcal\{L\}^\{\(l,m,n\)\}\-\\sum\_\{d\_\{1\}=1\}^\{D\}\\sum\_\{d\_\{2\}=1\}^\{D\}\\sum\_\{d\_\{3\}=1\}^\{D\}\{\}P^\{\(l,m,n\)\}\_\{d\_\{1\},d\_\{2\},d\_\{3\}\}\\ln\\frac\{P^\{\(l,m,n\)\}\_\{d\_\{1\},d\_\{2\},d\_\{3\}\}\}\{P^\{\(l,m,n\)\}\_\{d\_\{2\},d\_\{3\}\}\},wherePd2,d3\(l,m,n\)P^\{\(l,m,n\)\}\_\{d\_\{2\},d\_\{3\}\}is the marginal probability distribution by summing the first dimension ofP\(l,m,n\)P^\{\(l,m,n\)\}\. The total loss functionℒc​c\\mathcal\{L\}\_\{cc\}can be defined as follows:

ℒc​c=ℒ\(3,4,5\)\+ℒ\(4,5,3\)\+ℒ\(5,3,4\)\.\\mathcal\{L\}\_\{cc\}=\\mathcal\{L\}^\{\(3,4,5\)\}\+\\mathcal\{L\}^\{\(4,5,3\)\}\+\\mathcal\{L\}^\{\(5,3,4\)\}\.\(12\)
*Conditional Information Alignment*\. Given a multimodal combined prompt such as𝐏^6\\hat\{\\mathbf\{P\}\}^\{6\}containing audio and visual information, for the prompt𝐏^4\\hat\{\\mathbf\{P\}\}^\{4\}containing audio and text and the prompt𝐏^5\\hat\{\\mathbf\{P\}\}^\{5\}containing visual and text, their shared text modality should remain consistent and aligned\. Therefore, we maximize the conditional mutual informationI⁡\(𝐏^4;𝐏^5\|𝐏^6\)I\(\\hat\{\\mathbf\{P\}\}^\{4\};\\hat\{\\mathbf\{P\}\}^\{5\}\|\\hat\{\\mathbf\{P\}\}^\{6\}\)\. More generally, we aim to maximizeI⁡\(𝐏^l;𝐏^m\|𝐏^n\)I\(\\hat\{\\mathbf\{P\}\}\_\{l\};\\hat\{\\mathbf\{P\}\}^\{m\}\|\\hat\{\\mathbf\{P\}\}^\{n\}\), where𝐏^l,𝐏^m,𝐏^n∈𝒫d\\hat\{\\mathbf\{P\}\}^\{l\},\\hat\{\\mathbf\{P\}\}^\{m\},\\hat\{\\mathbf\{P\}\}^\{n\}\\in\\mathcal\{P\}\_\{d\}andll,mm, andnnare mutually distinct\. The loss function betweenll\-th,mm\-th andnn\-th prompts can be defined as:

ℒ^\(l,m,n\)=−∑d1=1D∑d2=1D∑d3=1DP\(l,m,n\)d1,d2,d3lnPd1,d2,d3\(l,m,n\)​Pd3\(l,m,n\)Pd1,d3\(l,m,n\)​Pd2,d3\(l,m,n\)\.\\displaystyle\\hat\{\\mathcal\{L\}\}^\{\(l,m,n\)\}=\-\\sum\_\{d\_\{1\}=1\}^\{D\}\\sum\_\{d\_\{2\}=1\}^\{D\}\\sum\_\{d\_\{3\}=1\}^\{D\}\{\}P^\{\(l,m,n\)\}\_\{d\_\{1\},d\_\{2\},d\_\{3\}\}\\ln\\frac\{P^\{\(l,m,n\)\}\_\{d\_\{1\},d\_\{2\},d\_\{3\}\}P^\{\(l,m,n\)\}\_\{d\_\{3\}\}\}\{P^\{\(l,m,n\)\}\_\{d\_\{1\},d\_\{3\}\}P^\{\(l,m,n\)\}\_\{d\_\{2\},d\_\{3\}\}\}\.The total loss functionℒc​i​a\\mathcal\{L\}\_\{cia\}can be defined as follows:

ℒc​i​a=ℒ^\(3,4,5\)\+ℒ^\(4,5,3\)\+ℒ^\(5,3,4\)\.\\mathcal\{L\}\_\{cia\}=\\hat\{\\mathcal\{L\}\}^\{\(3,4,5\)\}\+\\hat\{\\mathcal\{L\}\}^\{\(4,5,3\)\}\+\\hat\{\\mathcal\{L\}\}^\{\(5,3,4\)\}\.\(13\)
Finally, the total*prompt contrastive learning*loss functionℒp​c​l\\mathcal\{L\}\_\{pcl\}can be defined as:

ℒp​c​l=ℒm​i​p\+λ1⋅ℒc​c\+λ2⋅ℒc​i​a,\\mathcal\{L\}\_\{pcl\}=\\mathcal\{L\}\_\{mip\}\+\\lambda\_\{1\}\\cdot\\mathcal\{L\}\_\{cc\}\+\\lambda\_\{2\}\\cdot\\mathcal\{L\}\_\{cia\},\(14\)whereλ1\\lambda\_\{1\}andλ2\\lambda\_\{2\}are trade\-off parameters\.

Table 1:The performance of different methods on various datasets under six tasks\. The best and second\-best results are marked in bold and underlined, respectively\.

## Experiments

### Experiment Setting

#### Datasets\.

We conducted experiments on three widely used datasets, including CMU\-MOSI\([36](https://arxiv.org/html/2608.20019#bib.bib36)\), CMU\-MOSEI\([37](https://arxiv.org/html/2608.20019#bib.bib15)\), and SIMS\-V2\([32](https://arxiv.org/html/2608.20019#bib.bib16)\)\. The CMU\-MOSI dataset contains a total of 2,199 video clips, each manually annotated with sentiment scores ranging from strongly negative to strongly positive \(\-3 to 3\)\. The CMU\-MOSEI dataset consists of 22,856 video clips, covering a broader range of topics compared to CMU\-MOSI, with sentiment labels also annotated on the same scale \(\-3 to 3\)\. SIMS\-V2 is a Chinese multimodal sentiment analysis dataset containing 4,403 video clips, where sentiment values are labeled from \-1 to 1\.

Table 2:The cases of modality combinations in the training and test sets across the seven tasks\.Figure 3:The performance of seven methods across six evaluation metrics on three datasets under Task 7\. The center of the radar chart represents the worst results, and the outermost vertices correspond to the best results\.
#### Metrics\.

Due to differences in labels across datasets, we employ different evaluation metrics for different datasets\. For CMU\-MOSI and CMU\-MOSEI, we adopt binary accuracy \(Acc\-2\), five\-category accuracy \(Acc\-5\), seven\-category accuracy \(Acc\-7\), F1 score, mean absolute error \(MAE\), and Pearson correlation \(Corr\) as evaluation metrics\. For SIMS\-V2, we use Acc\-2, three\-category accuracy \(Acc\-3\), Acc\-5, F1, MAE, and Corr\. Here, Acc\-2 and F1 follow the non\-positive/positive standard\.

#### Baselines\.

In our experiments, we compare with six state\-of\-the\-art methods, including modality\-complete methods: Self\-MM, Cube\-MLP, DMD, and DLF; and modality\-missing methods: TFRNet, MPLMM, MFMB\-Net and LNLN\. For methods that require complete modalities during training, we fill in missing modalities with zero\. For all methods, we keep the parameters recommended in their original papers or released codes\.

#### Implementation Details\.

We conduct experiments on the Ubuntu 20\.04 system with an Intel\(R\) Xeon\(R\) Gold 6326 CPU @ 2\.90GHz and a single NVIDIA A40\. We adopt Qwen1\.5\-1\.8b\([1](https://arxiv.org/html/2608.20019#bib.bib32)\)as the backbone\. For the training process, we use the Adam optimizer with a learning rate of1×10−41\\times 10^\{\-4\}\. For reliability, we perform three independent runs for each experiment and report the average results\. More details can be found in Appendix[B](https://arxiv.org/html/2608.20019#A2)\.

### Main Results

In the experiment, we design seven experimental scenarios,i\.e\., seven different tasks, as shown in Table[2](https://arxiv.org/html/2608.20019#Sx4.T2), whereSiS\_\{i\}represents theii\-th modality combination and𝒮=\{S1,S2,⋯,S7\}\\mathcal\{S\}=\\\{S\_\{1\},S\_\{2\},\\cdots,S\_\{7\}\\\}\. For all scenarios, the testing phase has all modality combinations \(i\.e\.,𝒮\\mathcal\{S\}\)\.

Table[1](https://arxiv.org/html/2608.20019#Sx3.T1)presents the results of seven methods across three datasets under seven different scenarios\. From Table[1](https://arxiv.org/html/2608.20019#Sx3.T1), it can be observed that our method achieves the best performance in almost all scenarios\. Compared to the second\-best method, LNLN, our approach improves accuracy by an average of 5\.57%\. Notably, on the CMU\-MOSI dataset, our method outperforms LNLN by an average of 13\.26% in F1\-score\. In particular, under the Task 6 scenario, the F1\-score improvement reaches 16\.24%\. This is because our method,CMPL, leverages label\-guided contrastive feature learning to effectively capture multimodal consistency and the semantic structure of instances, employing three prompt contrastive learning strategies to enhance the model’s ability to learn from each modality combination and generalize to unseen modality combinations\.

Furthermore, to further validate the effectiveness of our method, we conduct experiments under the scenario where both the training and test sets contain all seven modality combinations \(i\.e\.,S7\{S\}\_\{7\}\), as shown in Figure[3](https://arxiv.org/html/2608.20019#Sx4.F3)\. The results demonstrate that our approach still outperforms others in most cases, highlighting its superiority and robustness\. More results can be found in Appendix[C](https://arxiv.org/html/2608.20019#A3)\.

Table 3:The performance with different LLMs on the CMU\-MOSI dataset under Task 4, where the units for training time and testing time are minutes \(m\) and seconds \(s\), respectively\.
### Comparable Results with Different LLMs

To further validate the effectively ofCMPL, we conduct experiments using different LLMs of varying scales \(i\.e\., Qwen1\.5\-1\.8B, Llama3\.2\-3B\([3](https://arxiv.org/html/2608.20019#bib.bib35)\), Llama\-2\-7B\([25](https://arxiv.org/html/2608.20019#bib.bib34)\), and Qwen3\-8B\([30](https://arxiv.org/html/2608.20019#bib.bib33)\)\) as the backbone under the Task 1 on the CMU\-MOSI dataset\. The performance and time of training and testing are presented in Table[3](https://arxiv.org/html/2608.20019#Sx4.T3)\.

As shown, models with larger parameter sizes generally achieve higher performance than smaller ones\. For instance, Qwen3\-8B achieves a 9\.84% improvement in Corr compared to Qwen1\.5\-1\.8B, which can be attributed to its greater learning capacity and ability to capture more nuanced knowledge\. However, for computational resources, Qwen3\-8B requires over three times the training time of Qwen1\.5\-1\.8B\. Given that Qwen1\.5\-1\.8B offers a favorable balance between resource efficiency and performance, it serves as a practical and cost\-effective choice for common deployment\.

### Ablation Study

To validate the effectiveness of each module in our method, we conduct experiments on the CMU\-MOSI and SIMS\-V2 datasets under the Task 4 scenario\. We systematically remove each module and observe the model’s performance changes\. The ablation results are presented in Table[4](https://arxiv.org/html/2608.20019#Sx4.T4)\.

The ablation results demonstrate that removing any module leads to performance degradation, while the model achieves its optimal performance when all modules are intact\. Specifically, on CMU\-MOSI dataset, removing the LCFL module results in a 2\.79% decrease in Corr, while removing the PCL module causes a 2\.94% increase in MAE on SIMS\-V2 dataset\. What’s more, the removal of the MPL module causes both a 2\.64% drop in Acc\-2 and a substantial 2\.83% in F1 on CMU\-MOSI dataset\. These results confirm that each module plays a critical role, validating the contributions of each module to the model’s effectiveness\. More additional ablation results can be found in Appendix[C](https://arxiv.org/html/2608.20019#A3)\.

Table 4:The ablation study on both CMU\-MOSI and SIMS\-V2 datasets under the Task 4\.

## Conclusion

In this paper, we propose a novel model namedCMPLto address the task of incomplete multimodal sentiment analysis with the unseen modality combination\. We introduce a*label\-guided contrastive feature learning*mechanism to maintain multimodal consistency while preserving the structural relationships among data\. Furthermore, we develop a*mixed prompts learning*mechanism incorporating the*prompt contrastive learning*strategies, which effectively enhances the model’s comprehension of diverse modal combinations and improves its generalization capability to unseen modal combinations\. Extensive experiments validate the effectiveness of our approach\.

## References

- Baiet al\.\(2023\)J\. Bai, S\. Bai, Y\. Chu, Z\. Cui, K\. Dang, X\. Deng, Y\. Fan, W\. Ge, Y\. Han, F\. Huang,et al\.Qwen technical report\.arXiv preprint arXiv:2309\.16609\.Cited by:[Implementation Details\.](https://arxiv.org/html/2608.20019#Sx4.SSx1.SSSx4.p1.1)\.
- Chenet al\.\(2020\)T\. Chen, S\. Kornblith, M\. Norouzi, and G\. HintonA simple framework for contrastive learning of visual representations\.InInternational Conference on Machine Learning,pp\. 1597–1607\.Cited by:[Label\-guided Contrastive Feature Learning](https://arxiv.org/html/2608.20019#Sx3.SSx3.p3.2)\.
- Dubeyet al\.\(2024\)A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Yang, A\. Fan,et al\.The llama 3 herd of models\.arXiv e\-prints,pp\. arXiv–2407\.Cited by:[Comparable Results with Different LLMs](https://arxiv.org/html/2608.20019#Sx4.SSx3.p1.1)\.
- Gaoet al\.\(2021\)T\. Gao, A\. Fisch, and D\. ChenMaking pre\-trained language models better few\-shot learners\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),pp\. 3816–3830\.Cited by:[Prompt Learning](https://arxiv.org/html/2608.20019#Sx2.SSx2.p1.1)\.
- Guoet al\.\(2024\)Z\. Guo, T\. Jin, and Z\. ZhaoMultimodal prompt learning with missing modalities for sentiment analysis and emotion recognition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1726–1736\.Cited by:[Prompt Learning](https://arxiv.org/html/2608.20019#Sx2.SSx2.p1.1)\.
- Heinzerling and Inui \(2021\)B\. Heinzerling and K\. InuiLanguage models as knowledge bases: on entity representations, storage capacity, and paraphrased queries\.InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume,pp\. 1772–1791\.Cited by:[Prompt Learning](https://arxiv.org/html/2608.20019#Sx2.SSx2.p1.1)\.
- Hu and Flaxman \(2018\)A\. Hu and S\. FlaxmanMultimodal sentiment analysis to explore the structure of emotions\.Inproceedings of the 24th ACM SIGKDD international conference on Knowledge Discovery & Data Mining,pp\. 350–358\.Cited by:[Introduction](https://arxiv.org/html/2608.20019#Sx1.p1.1)\.
- Huanet al\.\(2023\)R\. Huan, G\. Zhong, P\. Chen, and R\. LiangUnimf: a unified multimodal framework for multimodal sentiment analysis in missing modalities and unaligned multimodal sequences\.IEEE Transactions on Multimedia26,pp\. 5753–5768\.Cited by:[Introduction](https://arxiv.org/html/2608.20019#Sx1.p2.1)\.
- Huanget al\.\(2020\)J\. Huang, S\. Gong, and X\. ZhuDeep semantic clustering by partition confidence maximisation\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 8849–8858\.Cited by:[Prompt Contrastive Learning](https://arxiv.org/html/2608.20019#Sx3.SSx5.p3.1)\.
- Huanget al\.\(2024\)J\. Huang, J\. Zhou, Z\. Tang, J\. Lin, and C\. Y\. ChenTMBL: transformer\-based multimodal binding learning model for multimodal sentiment analysis\.Knowledge\-Based Systems285,pp\. 111346\.Cited by:[Introduction](https://arxiv.org/html/2608.20019#Sx1.p1.1)\.
- Jiet al\.\(2019\)X\. Ji, J\. F\. Henriques, and A\. VedaldiInvariant information clustering for unsupervised image classification and segmentation\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 9865–9874\.Cited by:[Prompt Contrastive Learning](https://arxiv.org/html/2608.20019#Sx3.SSx5.p3.1)\.
- Khattaket al\.\(2023\)M\. U\. Khattak, H\. Rasheed, M\. Maaz, S\. Khan, and F\. S\. KhanMaple: multi\-modal prompt learning\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 19113–19122\.Cited by:[Prompt Learning](https://arxiv.org/html/2608.20019#Sx2.SSx2.p1.1)\.
- Leeet al\.\(2023\)Y\. Lee, Y\. Tsai, W\. Chiu, and C\. LeeMultimodal prompting with missing modalities for visual recognition\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 14943–14952\.Cited by:[Prompt Learning](https://arxiv.org/html/2608.20019#Sx2.SSx2.p1.1)\.
- Liet al\.\(2024a\)M\. Li, D\. Yang, Y\. Lei, S\. Wang, S\. Wang, L\. Su, K\. Yang, Y\. Wang, M\. Sun, and L\. ZhangA unified self\-distillation framework for multimodal sentiment analysis with uncertain missing modalities\.InProceedings of the AAAI conference on artificial intelligence,Vol\.38,pp\. 10074–10082\.Cited by:[Introduction](https://arxiv.org/html/2608.20019#Sx1.p2.1)\.
- Liet al\.\(2024b\)M\. Li, D\. Yang, Y\. Liu, S\. Wang, J\. Chen, S\. Wang, J\. Wei, Y\. Jiang, Q\. Xu, X\. Hou,et al\.Toward robust incomplete multimodal sentiment analysis via hierarchical representation learning\.Advances in Neural Information Processing Systems37,pp\. 28515–28536\.Cited by:[Introduction](https://arxiv.org/html/2608.20019#Sx1.p2.1)\.
- Liet al\.\(2023\)Y\. Li, Y\. Wang, and Z\. CuiDecoupled multimodal distilling for emotion recognition\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 6631–6640\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p1.1)\.
- Lianget al\.\(2022\)S\. Liang, M\. Zhao, and H\. SchützeModular and parameter\-efficient multimodal fusion with prompting\.InFindings of the Association for Computational Linguistics: ACL 2022,pp\. 2976–2985\.Cited by:[Prompt Learning](https://arxiv.org/html/2608.20019#Sx2.SSx2.p1.1)\.
- Linet al\.\(2021\)Y\. Lin, Y\. Gou, Z\. Liu, B\. Li, J\. Lv, and X\. PengCompleter: incomplete multi\-view clustering via contrastive prediction\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 11174–11183\.Cited by:[Prompt Contrastive Learning](https://arxiv.org/html/2608.20019#Sx3.SSx5.p3.1)\.
- Liuet al\.\(2024\)Z\. Liu, A\. Braytee, A\. Anaissi, G\. Zhang, L\. Qin, and J\. AkramEnsemble pretrained models for multimodal sentiment analysis using textual and video data fusion\.InCompanion Proceedings of the ACM Web Conference 2024,pp\. 1841–1848\.Cited by:[Introduction](https://arxiv.org/html/2608.20019#Sx1.p1.1)\.
- Maet al\.\(2021\)M\. Ma, J\. Ren, L\. Zhao, S\. Tulyakov, C\. Wu, and X\. PengSmil: multimodal learning with severely missing modality\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 2302–2310\.Cited by:[Introduction](https://arxiv.org/html/2608.20019#Sx1.p2.1)\.
- Maiet al\.\(2022\)S\. Mai, Y\. Zeng, S\. Zheng, and H\. HuHybrid contrastive learning of tri\-modal representation for multimodal sentiment analysis\.IEEE Transactions on Affective Computing14\(3\),pp\. 2276–2289\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p1.1)\.
- Phamet al\.\(2019\)H\. Pham, P\. P\. Liang, T\. Manzini, L\. Morency, and B\. PóczosFound in translation: learning robust joint representations by cyclic translations between modalities\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.33,pp\. 6892–6899\.Cited by:[Introduction](https://arxiv.org/html/2608.20019#Sx1.p2.1)\.
- Sunet al\.\(2022\)H\. Sun, H\. Wang, J\. Liu, Y\. Chen, and L\. LinCubeMLP: an mlp\-based model for multimodal sentiment analysis and depression estimation\.InProceedings of the 30th ACM international Conference on Multimedia,pp\. 3722–3729\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p1.1)\.
- Taoet al\.\(2025\)C\. Tao, J\. Li, T\. Zang, and P\. GaoA multi\-focus\-driven multi\-branch network for robust multimodal sentiment analysis\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 1547–1555\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p2.1)\.
- Touvronet al\.\(2023\)H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale,et al\.Llama 2: open foundation and fine\-tuned chat models\.arXiv preprint arXiv:2307\.09288\.Cited by:[Comparable Results with Different LLMs](https://arxiv.org/html/2608.20019#Sx4.SSx3.p1.1)\.
- Truong and Lauw \(2019\)Q\. Truong and H\. W\. LauwVistanet: visual aspect attention network for multimodal sentiment analysis\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.33,pp\. 305–312\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p1.1)\.
- Tsimpoukelliet al\.\(2021\)M\. Tsimpoukelli, J\. L\. Menick, S\. Cabi, S\. Eslami, O\. Vinyals, and F\. HillMultimodal few\-shot learning with frozen language models\.Advances in Neural Information Processing Systems34,pp\. 200–212\.Cited by:[Prompt Learning](https://arxiv.org/html/2608.20019#Sx2.SSx2.p1.1)\.
- Wanget al\.\(2025\)P\. Wang, Q\. Zhou, Y\. Wu, T\. Chen, and J\. HuDLF: disentangled\-language\-focused multimodal sentiment analysis\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 21180–21188\.Cited by:[Introduction](https://arxiv.org/html/2608.20019#Sx1.p1.1),[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p1.1)\.
- Xuet al\.\(2024\)W\. Xu, H\. Jiang, and X\. LiangLeveraging knowledge of modality experts for incomplete multimodal learning\.InProceedings of the 32nd ACM International Conference on Multimedia,pp\. 438–446\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p2.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[Comparable Results with Different LLMs](https://arxiv.org/html/2608.20019#Sx4.SSx3.p1.1)\.
- Yaoet al\.\(2024\)L\. Yao, L\. Li, S\. Ren, L\. Wang, Y\. Liu, X\. Sun, and L\. HouDeco: decoupling token compression from semantic abstraction in multimodal large language models\.arXiv preprint arXiv:2405\.20985\.Cited by:[Label\-guided Contrastive Feature Learning](https://arxiv.org/html/2608.20019#Sx3.SSx3.p3.1)\.
- Yuet al\.\(2020\)W\. Yu, H\. Xu, F\. Meng, Y\. Zhu, Y\. Ma, J\. Wu, J\. Zou, and K\. YangCh\-sims: a chinese multimodal sentiment analysis dataset with fine\-grained annotation of modality\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,pp\. 3718–3727\.Cited by:[Datasets\.](https://arxiv.org/html/2608.20019#Sx4.SSx1.SSSx1.p1.1)\.
- Yuet al\.\(2021\)W\. Yu, H\. Xu, Z\. Yuan, and J\. WuLearning modality\-specific representations with self\-supervised multi\-task learning for multimodal sentiment analysis\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 10790–10797\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p1.1)\.
- Yuanet al\.\(2021\)Z\. Yuan, W\. Li, H\. Xu, and W\. YuTransformer\-based feature reconstruction network for robust multimodal sentiment analysis\.InProceedings of the 29th ACM international conference on multimedia,pp\. 4400–4407\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p2.1)\.
- Yuanet al\.\(2023\)Z\. Yuan, Y\. Liu, H\. Xu, and K\. GaoNoise imitation based adversarial training for robust multimodal sentiment analysis\.IEEE Transactions on Multimedia26,pp\. 529–539\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p2.1)\.
- Zadehet al\.\(2016\)A\. Zadeh, R\. Zellers, E\. Pincus, and L\. MorencyMultimodal sentiment intensity analysis in videos: facial gestures and verbal messages\.IEEE Intelligent Systems31\(6\),pp\. 82–88\.Cited by:[Datasets\.](https://arxiv.org/html/2608.20019#Sx4.SSx1.SSSx1.p1.1)\.
- Zadehet al\.\(2018\)A\. B\. Zadeh, P\. P\. Liang, S\. Poria, E\. Cambria, and L\. MorencyMultimodal language analysis in the wild: cmu\-mosei dataset and interpretable dynamic fusion graph\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 2236–2246\.Cited by:[Datasets\.](https://arxiv.org/html/2608.20019#Sx4.SSx1.SSSx1.p1.1)\.
- Zenget al\.\(2022\)J\. Zeng, T\. Liu, and J\. ZhouTag\-assisted multimodal sentiment analysis under uncertain missing modalities\.InProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 1545–1554\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p2.1)\.
- Zhanget al\.\(2024\)H\. Zhang, W\. Wang, and T\. YuTowards robust multimodal sentiment analysis with incomplete data\.Advances in Neural Information Processing Systems37,pp\. 55943–55974\.Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p2.1)\.
- Zhuet al\.\(2022\)T\. Zhu, L\. Li, J\. Yang, S\. Zhao, H\. Liu, and J\. QianMultimodal sentiment analysis with image\-text interaction network\.IEEE Transactions on Multimedia25,pp\. 3375–3385\.Cited by:[Introduction](https://arxiv.org/html/2608.20019#Sx1.p1.1)\.
- Zhuet al\.\(2023\)Y\. Zhu, Y\. Wang, J\. Qiang, and X\. WuPrompt\-learning for short text classification\.IEEE Transactions on Knowledge and Data Engineering36\(10\),pp\. 5328–5339\.Cited by:[Prompt Learning](https://arxiv.org/html/2608.20019#Sx2.SSx2.p1.1)\.
- Zhuanget al\.\(2025\)Y\. Zhuang, L\. Minhao, W\. Bai, Y\. Zhang, W\. Li, J\. Deng, and F\. RenHyper\-modality enhancement for multimodal sentiment analysis with missing modalities\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[Multimodal Sentiment Analysis](https://arxiv.org/html/2608.20019#Sx2.SSx1.p2.1)\.

Table 5:The seven modality combination cases\.Table 6:Experimental parameters of three datasets\.Table 7:The initial text of the modality combination prompts for three datasets\.## Appendix ATask Description

In the IMSAUMC task, given three modalities, excluding the case where all modalities are absent, there are a total of23−1=72^\{3\}\-1=7modality combinations\. The corresponding available and missing states of the modalities, as well as the available data, are shown in Table[5](https://arxiv.org/html/2608.20019#A0.T5)\.

## Appendix BImplementation Details

Some parameters involved in the experiment are shown in Table[6](https://arxiv.org/html/2608.20019#A0.T6)\. Table[7](https://arxiv.org/html/2608.20019#A0.T7)displays the initial text of prompts in three datasets, where “<\\textlesscontent\>\\textgreater” will be replaced according to different modality combinations\. For instance, when the input is an audio\-vision modality combination, “<\\textlesscontent\>\\textgreater” will be replaced with “audio and vision content”, and similarly for other cases\.

Table 8:Further ablation study on both CMU\-MOSI and SIMS\-V2 datasets under the Task 4\.
## Appendix CAdditional Experimental Results

### Parameter Analysis

We conduct a parameter analysis on the CMU\-MOSI dataset for the key hyperparameters used in CMPL\. Specifically, we examine the parameterα\\alpha, which controls the LCFL loss when the PCL strategy is disabled\. We then analyze the parametersβ\\beta,λ1\\lambda\_\{1\}, andλ2\\lambda\_\{2\}, which regulate the MIP, CC, and CIA components in the PCL strategy\. The parameter analysis results are presented in Figure[4](https://arxiv.org/html/2608.20019#A3.F4)\.

The results show that all four parameters lead to certain performance variations when ranging from1​e−21e\-2to1​e​21e2\. However, the overall performance of the model is not highly sensitive to these changes\. The model achieves its best performance whenα=1\\alpha=1,β=1​e−2\\beta=1e\-2,λ1=10\\lambda\_\{1\}=10, andλ2=1\\lambda\_\{2\}=1\.

### Further Ablation Study

To further validate the contribution of each strategy within PCL, we conducted additional ablation studies by individually removing the MIP, CC, and CIA strategies\. While the ablation results have already been presented in the main text in graphical form, we provide the detailed numerical results in Table[8](https://arxiv.org/html/2608.20019#A2.T8)\. As shown, removing any single strategy leads to a performance drop, which further demonstrates the effectiveness of each component in our PCL framework\.

To further validate the effectiveness of PCL on the IMSAUMC task, we additionally present its performance for each modality combination in the test set under Task 4 on the SIMS\-V2 dataset\. The results are shown in Figure[5](https://arxiv.org/html/2608.20019#A3.F5), where “t”, “a”, and “v” denote text, audio, and visual modality, while “t\-a” indicates the text\-audio modality combination and similarly for others\. Here, “w/o MIP”, “w/o CC”, “w/o CIA” indicate the absence of each strategy in PCL\. As observed, in the absence of any prompt learning strategies, the model performs poorly on unseen modality combinations, with an average accuracy drop of approximately 3\.12% compared to CMPL\. The incorporation of the PCL module demonstrates additional performance gains in audio\-vision \(a\-v\), and text\-audio\-vision \(t\-a\-v\) modality combinations\. Furthermore, when the MIP, CC, and CIA strategies are successively removed from the CMPL model, the performance decreases by about 1\.32%, 0\.94%, and 2\.26% on unseen modality combinations, respectively, which indicates that each strategy plays a critical role in improving the model’s generalization\. The model achieves optimal performance when all three strategies are incorporated\. These results demonstrate the effectiveness and necessity of each strategy in PCL\.

### Detailed Experimental Results

Table[9](https://arxiv.org/html/2608.20019#A3.T9)to Table[16](https://arxiv.org/html/2608.20019#A3.T16)present detailed experimental results of the seven methods across seven tasks on three datasets, evaluated by six metrics\. As shown in the results, our model consistently achieves superior performance over baseline methods in most scenarios, clearly validating the effectiveness and superiority of our approach\.

\(a\)Parameter Analysis ofα\\alpha\(b\)Parameter Analysis ofβ\\beta\(c\)Parameter Analysis ofλ1\\lambda\_\{1\}\(d\)Parameter Analysis ofλ2\\lambda\_\{2\}
Figure 4:Parameter analysis on the CMU\-MOSI dataset\.Figure 5:The model performance for various modality combinations on SIMS\-V2 dataset under Task 4\.Table 9:The performance of different methods on three datasets under Task 1\. The best and second\-best results are marked in bold and underlined, respectively\.Table 10:The performance of different methods on three datasets under Task 2\. The best and second\-best results are marked in bold and underlined, respectively\.Table 11:The performance of different methods on three datasets under Task 3\. The best and second\-best results are marked in bold and underlined, respectively\.Table 12:The performance of different methods on three datasets under Task 4\. The best and second\-best results are marked in bold and underlined, respectively\.Table 13:The performance of different methods on three datasets under Task 5\. The best and second\-best results are marked in bold and underlined, respectively\.Table 14:The performance of different methods on three datasets under Task 5\. The best and second\-best results are marked in bold and underlined, respectively\.Table 15:The performance of different methods on three datasets under Task 6\. The best and second\-best results are marked in bold and underlined, respectively\.Table 16:The performance of different methods on three datasets under Task 7\. The best and second\-best results are marked in bold and underlined, respectively\.