multi-task-learning

The paper proposes fine-tuning transformer encoders end-to-end for response-free item difficulty modelling of multiple-choice reading comprehension items, with component-wise and multi-task variants, showing that multi-task learning improves in small-sample regimes.

PEML proposes a parameter-efficient multi-task learning method that co-optimizes continuous prompts and model weights via low-rank adaptation. It achieves up to 6.67% average accuracy improvement on multiple benchmarks.

Introduces Bayesian Model Merging (BMM), a plug-and-play bi-level optimization framework for combining multiple task-specific experts into a single model, achieving state-of-the-art performance on vision and language benchmarks.

This paper proposes a unified contrastive framework for learning graph representations across multiple abstraction levels (node, proximity, cluster, graph) with a parameter-free self-weighting mechanism that adaptively assigns weights to similarity scores, outperforming state-of-the-art on downstream tasks like classification, clustering, and link prediction.

Microsoft Research announces MatterSim updates including MatterSim-MT, a multi-task foundation model for materials characterization, faster simulation (3-5x speedup), and experimental validation of thermal conductivity predictions for a new material.

This paper introduces MELD, a detector for AI-generated text that uses multi-task learning with auxiliary heads for generator family, attack type, and source domain to improve robustness. MELD achieves strong performance on the RAID benchmark and maintains low false-positive rates under adversarial attacks.

This paper proposes Badit, a method that decomposes large language model parameters into orthogonal high-singular-value LoRA experts to mitigate cross-task interference during multi-task instruction tuning.

APEX is a large-scale multi-task learning framework that predicts both popularity and aesthetic quality of AI-generated music using frozen audio embeddings. The model demonstrates strong generalization across different generative architectures by jointly predicting engagement signals and perceptual quality dimensions.

SAMoRA introduces a semantic-aware router and task-adaptive scaling to improve expert specialization and dynamic weighting in MoE-LoRA fine-tuning, outperforming prior methods on multi-task benchmarks.

This paper introduces Pico, a data-free method that improves LoRA adapter merging by separately calibrating the output-side matrix B to reduce interference from shared directions while preserving task-specific information. Pico achieves 3.4–8.3 point accuracy improvements over existing merging methods across math, coding, finance, and medical benchmarks.

GLiNER2 is a unified transformer-based framework for multiple NLP tasks, offering improved efficiency and accessibility over large language models by supporting named entity recognition, text classification, and hierarchical data extraction in a single model.