multi-task-learning

This paper presents CABS+, an enhanced model merging method that uses conflict-aware sparsification and adaptive weight allocation to improve multi-task model merging efficiency and performance, achieving significant gains over prior methods across 27 datasets.

LoRA-Diffusion proposes a parameter-efficient fine-tuning method for diffusion-based language models by applying low-rank decomposition to the denoising trajectory rather than model weights, achieving competitive performance with only 1.2% trajectory adapter parameters.

A paper introduces TC-LeWM, applying regularization losses like SIGReg and VISReg to temporal latent residuals to enable multi-task learning in LeWM, significantly improving success rates on the LIBERO robot arm benchmark.

Presents an age-aware multi-task learning method for phoneme recognition from children's speech, enabling a lightweight 94M-parameter model to outperform larger models and run on edge devices like phones.

This paper proposes TdSciNER, a type-driven multi-task learning approach that leverages LLMs to improve scientific named entity recognition by filtering entity types, adding an auxiliary typing task, and using a demonstration selection strategy. Experiments on three datasets show performance comparable to fully supervised models.

This paper introduces MiGHT-EHR, a multi-task graph transformer for heterogeneous temporal EHR data, jointly modeling clinical entities, temporal trajectories, and task dependencies. It outperforms state-of-the-art methods on MIMIC-III and MIMIC-IV across drug recommendation, length-of-stay, mortality, and readmission prediction.

This paper introduces NTDH, a complex-reasoning framework for comprehensive affective analysis that unifies sentiment and emotion prediction tasks. Trained with SFT and GRPO on Qwen3-8B, it achieves strong results on SemEval-2018 EI-reg with a Pearson correlation of 0.862.

This paper proposes DKG-MTI, a dual knowledge graph framework that augments LLM-based inference with structure-aware knowledge alignment to jointly predict aspect ratings and generate user intent statements from online travel reviews.

This paper establishes theoretical bounds on the number of attention heads needed to produce vector representations that support multiple tasks, such as computing min/max and XOR, showing trade-offs between head count, embedding dimension, and precision.

This paper presents a theoretical and empirical analysis showing that SFT suffers from task conflicts in multi-task LLM training while RL enables stable coexistence, proposing the Parallel-RL paradigm for efficient multi-task training.

This paper introduces the Customer-Based Multi-task Transformer (CBMT), a model that forecasts revenue by jointly predicting customer acquisition, repeat orders, and average order value. It claims improved forecast accuracy over existing benchmarks and analyzes when coordinated forecasting helps.

This paper introduces V2N, the first complete visual piano transcription system that jointly predicts onset, offset, keyhold, and velocity from video, achieving state-of-the-art results on PianoVAM and R3 benchmarks.

SkillRise is a unified reinforcement learning framework that enables LLM agents to learn and reuse skills across related, progressively challenging tasks, outperforming baselines by up to 8.5 percentage points on several benchmarks.

LithoFormer is a Seq2Seq transformer framework for stratigraphic inference from well logs, using a PatchTST backbone with rotary positional embeddings and a multi-task head to jointly predict geological zonation and boundaries, achieving a 90% reduction in boundary error and eliminating stratigraphic order violations.

This paper introduces the Diagnostic Evidence Network (DENet), a multi-task framework that extends AI-based bearing fault diagnosis to produce physically verifiable evidence, such as predicted characteristic frequencies and temporal localization of impulses, while using a QLoRA-adapted language model to generate constrained diagnostic reports that reduce hallucinated content.

Proposes RoSIP-Batt, a physics-guided multi-task transformer for joint estimation of battery State of Health and Remaining Useful Life, achieving state-of-the-art accuracy on NASA, MIT-Stanford, and HUST datasets.

PromptSD is a novel on-policy self-distillation method where the teacher differs from the student only by a learnable soft prompt, enabling multi-task knowledge absorption without weight drift. It matches or exceeds full fine-tuning on four tasks while preserving general capabilities.

This paper demonstrates that naive train/test splitting on sliding-window sequences can severely inflate or deflate performance metrics in multi-task learning for predictive maintenance, and proposes a leakage-robust evaluation protocol.

CardioMeta is a calibrated multi-task framework for jointly predicting diabetes, hypertension, and cardiovascular disease across NHANES and MIMIC-IV data, emphasizing leakage control, calibration, and transparent reliability.

This paper challenges the common assumption that domain experts for model merging should be trained to their optimal validation loss, showing that the optimal training duration depends strongly on the merging method. Simple averaging degrades with overfitting while sparsification-based methods benefit from training past the optimum, suggesting that training duration and merging method should be chosen jointly.