multi-task-learning

HierBias introduces a hierarchical context-conditioned model for media bias detection that leverages document context to improve sentence-level classification, achieving state-of-the-art F1 and MCC on the BABE and BASIL datasets.

This paper introduces LDM-v0, a large decision model trained offline on trajectories from thousands of diverse reinforcement learning environments, demonstrating that a single transformer policy can match the performance of task-specific policies across robotics, autonomous driving, inventory management, cybersecurity, trading, and video games.

SURGeLLM introduces a unified transformer framework with surgical feature gates, task-conditioned prefix tokens, and instance-weighted normalization to address mismatched inductive biases, class imbalance, and lexical knowledge injection in multi-task learning, achieving significant gains across four diverse NLP tasks.

This paper identifies a capacity-induced failure mode in physics-informed neural networks (PINNs) where overparameterized networks develop functional modularity that hinders convergence, and proposes Modular-Sparsity Synchronization (ModSync), a framework that penalizes task-exclusive connections to maintain cross-objective interaction and achieve state-of-the-art accuracy.

The paper identifies 'Load-Bearing Wall' dimensions in pre-trained models that retain task-specific knowledge not fully captured by task vectors in model merging, and proposes PACT (PreserveAnchoredCores) to preserve these cores, achieving state-of-the-art performance across benchmarks.

OneRank proposes a Transformer-native multi-task ranking framework that integrates feature encoding and prediction to reduce inter-task interference and improve ranking performance in recommender systems.

This paper proposes a probabilistic contrastive pretraining framework for molecular graph transformers to improve multi-task ADME property prediction in drug discovery, achieving significant gains on three benchmarks.

This paper introduces World-Language-Action (WLA) models, embodied foundation models that jointly predict textual subtasks, subgoal images, and robot actions from text, images, and robot states, achieving state-of-the-art multi-task and long-horizon learning in simulated and real-world environments.

Proposes MechRL, a reinforcement learning approach to automate circuit discovery in transformer language models. A PPO agent trained on multiple tasks discovers attention head circuits that match known canonical circuits and generalizes to a held-out task.

This paper develops a PAC-Bayesian framework for physics-informed machine learning, providing high-probability generalization guarantees for unbounded losses. It proposes a multi-task perspective that jointly handles data fidelity, PDE residuals, and boundary conditions, and introduces a self-bounding learning algorithm.

This paper proposes a task-routed mixture-of-experts model with cognitive appraisal theory for implicit sentiment analysis, introducing auxiliary tasks to improve reasoning about sentiment from context and outperforming existing approaches.

This paper introduces an interference-aware framework for multi-task machine unlearning, addressing task-level and instance-level interference through task-aware gradient projection and instance-level gradient orthogonalization, achieving effective unlearning on multi-task computer vision benchmarks.

The paper proposes fine-tuning transformer encoders end-to-end for response-free item difficulty modelling of multiple-choice reading comprehension items, with component-wise and multi-task variants, showing that multi-task learning improves in small-sample regimes.

PEML proposes a parameter-efficient multi-task learning method that co-optimizes continuous prompts and model weights via low-rank adaptation. It achieves up to 6.67% average accuracy improvement on multiple benchmarks.

Introduces Bayesian Model Merging (BMM), a plug-and-play bi-level optimization framework for combining multiple task-specific experts into a single model, achieving state-of-the-art performance on vision and language benchmarks.

This paper proposes a unified contrastive framework for learning graph representations across multiple abstraction levels (node, proximity, cluster, graph) with a parameter-free self-weighting mechanism that adaptively assigns weights to similarity scores, outperforming state-of-the-art on downstream tasks like classification, clustering, and link prediction.

Microsoft Research announces MatterSim updates including MatterSim-MT, a multi-task foundation model for materials characterization, faster simulation (3-5x speedup), and experimental validation of thermal conductivity predictions for a new material.

This paper introduces MELD, a detector for AI-generated text that uses multi-task learning with auxiliary heads for generator family, attack type, and source domain to improve robustness. MELD achieves strong performance on the RAID benchmark and maintains low false-positive rates under adversarial attacks.

This paper proposes Badit, a method that decomposes large language model parameters into orthogonal high-singular-value LoRA experts to mitigate cross-task interference during multi-task instruction tuning.

APEX is a large-scale multi-task learning framework that predicts both popularity and aesthetic quality of AI-generated music using frozen audio embeddings. The model demonstrates strong generalization across different generative architectures by jointly predicting engagement signals and perceptual quality dimensions.