Tag
This paper investigates whether explicit domain adaptation methods are beneficial for sentiment transfer when using frozen pre-trained language model backbones, finding that effectiveness depends on whether the backbone already possesses target-domain knowledge.
This paper presents MentalMARBERT, a domain-adapted Arabic language model for detecting mental health disorders from social media text. The framework uses domain-adaptive pre-training and a two-stage fine-tuning approach, achieving 0.877 accuracy and 0.861 macro-F1 on a newly constructed Arabic mental health dataset of 50,670 tweets.
This paper proposes an attention expansion mechanism to enhance keyphrase extraction from long documents by augmenting PLM token representations with out-of-context information, achieving consistent improvements over state-of-the-art models without requiring full-document attention or expensive LLM inference.