Solving Few-Shot Multiobjective Multitask Optimization via Iterative Sequential Transfer

# Solving Few-Shot Multiobjective Multitask Optimization via Iterative Sequential Transfer
Source: [https://arxiv.org/html/2609.11228](https://arxiv.org/html/2609.11228)
Tingyang WeiAffiliation:College of Computing and Data Science Nanyang Technological University Singapore tingyang001@e\.ntu\.edu\.sgHaofeng WuAffiliation:College of Computing and Data Science Nanyang Technological University Singapore haofeng\.wu@ntu\.edu\.sgAnanda Phan ImanAffiliation:Department of AI Convergence Gwangju Inst\. of Sci\. & Tech\. \(GIST\) South Korea anandaphan@gm\.gist\.ac\.krAffiliation:Affiliation:Zhao WeiJiao LiuAffiliation:College of Computing and Data Science Nanyang Technological University Singapore jiao\.liu@ntu\.edu\.sgYew\-Soon OngAffiliation:College of Computing and Data Science Nanyang Technological University Singapore ASYSOng@ntu\.edu\.sg

###### Abstract

Applying knowledge transfer across multiple optimization tasks, multitask optimization \(MTO\) emerges as a promising approach to solving synergistic optimization tasks simultaneously\. However, the development of effective knowledge transfer mechanisms in MTO fundamentally relies on aligning elite solution distributions across tasks\. This dependency creates a critical bottleneck in*few\-shot optimization*regimes, as restricted evaluation budgets impede the identification of elite solution distributions required for beneficial transfer\. This challenge is exacerbated in multiobjective multitask problems, where each optimizer must approximate a continuous Pareto manifold rather than a single optimal point\. This paper introduces Iterative Sequential Transfer \(IST\) to circumvent this bottleneck\. We model MTO as a sequence of sequential transfer optimization problems, concentrating evaluations on a single target per iteration\. We propose a likelihood\-informed task prioritization mechanism to maximize transfer utility by identifying the task most likely ready for knowledge integration\. Empirical results on benchmark and real\-world problems verify the effectiveness of the proposed method under tight budgets\.

###### Index Terms:

Transfer optimization, evolutionary multitask, transfer evolutionary optimization, Gaussian process, multiobjective optimization\.

## IIntroduction

Transfer optimization\[[1](https://arxiv.org/html/2609.11228#bib.bib17)\]has garnered significant attention in recent years as a novel approach to optimization problems by fully leveraging inter\-task relationships\. Considering that real\-world optimization problems seldom exist in isolation\[[2](https://arxiv.org/html/2609.11228#bib.bib8)\], transfer optimization methods are designed to avoid optimizing given tasks from scratch and to alleviate the excessive computational burden\. Several conceptual realizations of the transfer optimization paradigm, including sequential transfer optimization \(STrO\)\[[3](https://arxiv.org/html/2609.11228#bib.bib24),[4](https://arxiv.org/html/2609.11228#bib.bib6)\], multitask optimization\[[5](https://arxiv.org/html/2609.11228#bib.bib18),[6](https://arxiv.org/html/2609.11228#bib.bib22),[7](https://arxiv.org/html/2609.11228#bib.bib4)\], and multiform optimization\[[8](https://arxiv.org/html/2609.11228#bib.bib13)\], have spawned numerous studies in the context of knowledge transfer\.

In particular, multitask optimization \(MTO\)\[[5](https://arxiv.org/html/2609.11228#bib.bib18),[7](https://arxiv.org/html/2609.11228#bib.bib4)\]emerges as a ubiquitous approach to solving multiple optimization tasks simultaneously by exploiting the inter\-task synergies\. MTO can be formulated as follows:

min⁡fk​\(𝐱k\),s\.t\.​𝐱k∈Ωk,k∈\{1,…,K\},\\min f\_\{k\}\(\\mathbf\{x\}\_\{k\}\),~~\\text\{s\.t\.\}~\\mathbf\{x\}\_\{k\}\\in\\Omega\_\{k\},~~k\\in\\\{1,\\ldots,K\\\},\(1\)whereΩk\\Omega\_\{k\}is the decision space of thekk\-th optimization problem, andfkf\_\{k\}is the objective function for thekk\-th task\. MTO tackles distinct problems simultaneously, thereby aiming at generating outputs\(𝐱1∗,𝐱2∗,…,𝐱K∗\)\(\\mathbf\{x\}\_\{1\}^\{\*\},\\mathbf\{x\}\_\{2\}^\{\*\},\\ldots,\\mathbf\{x\}\_\{K\}^\{\*\}\)that are the optimal solutions for each task\. To achieve this, MTO seeks to develop proper knowledge transfer mechanisms among distinct optimization problems, either implicitly sharing the solution components\[[5](https://arxiv.org/html/2609.11228#bib.bib18),[6](https://arxiv.org/html/2609.11228#bib.bib22),[9](https://arxiv.org/html/2609.11228#bib.bib31)\]or explicitly building mapping among task pairs\[[10](https://arxiv.org/html/2609.11228#bib.bib30),[11](https://arxiv.org/html/2609.11228#bib.bib23)\]\. Benefiting from these knowledge transfer mechanisms, recent years have witnessed significant advances of MTO across a plethora of applications\[[12](https://arxiv.org/html/2609.11228#bib.bib14)\]encompassing system\-in\-package design\[[13](https://arxiv.org/html/2609.11228#bib.bib16)\], grid shell design\[[14](https://arxiv.org/html/2609.11228#bib.bib10)\], and collision\-avoidance control\[[15](https://arxiv.org/html/2609.11228#bib.bib15)\]\.

Albeit the surging advances, the efficacy of these transfer mechanisms fundamentally hinges on the alignment of elite solution distributions across tasks\[[16](https://arxiv.org/html/2609.11228#bib.bib9)\]\. For knowledge transfer to be beneficial, the source task should provide a high\-quality solution distribution\[[17](https://arxiv.org/html/2609.11228#bib.bib29)\]in its own search space to accurately guide the target task\. Otherwise, stagnant source tasks are likely to trigger detrimental negative transfer\[[17](https://arxiv.org/html/2609.11228#bib.bib29)\]\. This dependency creates a critical bottleneck in the*few\-shot optimization*regime\. Under stringent evaluation budgets, limited evaluations are typically dispersed across all tasks, failing to identify high\-quality solutions within any single task and thereby giving rise to stagnated search or negative transfer\[[18](https://arxiv.org/html/2609.11228#bib.bib19)\]\. This challenge is further intensified in the multiobjective case, where the optimizer must approximate a continuous Pareto manifold rather than a single optimal point\[[14](https://arxiv.org/html/2609.11228#bib.bib10)\]\. Dispersing a restricted budget across multiple multiobjective search tasks more likely prevents any task from identifying an elite solution set, exacerbating the risks of negative transfer and inefficient resource utilization\.

To address this, we position sequential transfer optimizers as a practical approach for solving few\-shot multi\-objective multitask optimization, and provide empirical evidence through an Iterative Sequential Transfer \(IST\) framework\. Sequential transfer optimization \(STrO\) can be formally denoted as:

min𝐱T∈ΩT⁡fT​\(𝐱T\)given\{𝒟Sk\}k=1K,\\min\_\{\\mathbf\{x\}\_\{T\}\\in\\Omega\_\{T\}\}\\ f\_\{T\}\(\\mathbf\{x\}\_\{T\}\)\\quad\\text\{given\}\\quad\\\{\\mathcal\{D\}\_\{S\_\{k\}\}\\\}\_\{k=1\}^\{K\},\(2\)where𝒟Sk\\mathcal\{D\}\_\{S\_\{k\}\}is the optimization history for the source taskSkS\_\{k\}including evaluated solutions\{\(𝐱Sk\(t\),fSk​\(𝐱Sk\(t\)\)\)\}t=1Nt\\\{\(\\mathbf\{x\}\_\{S\_\{k\}\}^\{\(t\)\},f\_\{S\_\{k\}\}\(\\mathbf\{x\}\_\{S\_\{k\}\}^\{\(t\)\}\)\)\\\}\_\{t=1\}^\{N\_\{t\}\}\. In this regard, MTO can be explicitly converted into a sequence of STrO problems, where at each iteration we choose a target taskTTand its source set\{Sk\}\\\{S\_\{k\}\\\}by maximizing the universal transfer utility, so that knowledge transfer is applied only when it is most beneficial\. This can potentially alleviate the few\-shot challenge\. Meanwhile, STrO has recently been developed into a fruitful line of inquiry with both theory\-informed formulations\[[19](https://arxiv.org/html/2609.11228#bib.bib12),[20](https://arxiv.org/html/2609.11228#bib.bib11),[21](https://arxiv.org/html/2609.11228#bib.bib7)\]and principled empirical analyses\[[22](https://arxiv.org/html/2609.11228#bib.bib28)\], making it a solid foundation for our IST framework\. To effectively convert the MTO into a sequence of STrO as per the search dynamics, we introduce a simple likelihood\-informed task prioritization mechanism\.

In this study, we focus on the intricate multiobjective multitask optimization \(MOMTO\) under a stringent computational budget\. The inherent complexities of multiobjective optimization, coupled with the demands of few\-shot optimization, exacerbate the difficulty of MTO\. To verify our position, we test the IST framework based on a recent STrO method,*forward\-inverse transfer evolutionary multiobjective optimizer*\(F\-invTrEMO\)\[[21](https://arxiv.org/html/2609.11228#bib.bib7)\], in both few\-shot MOMTO benchmark and real\-world problems\. Besides, we impose this IST framework on another potent STrO method, AMTEA\[[3](https://arxiv.org/html/2609.11228#bib.bib24)\], showcasing the generality of the proposed method\. We position that the IST framework not only can highlight the potential for more effective MTO in few\-shot multiobjective domains, but also offers a flexible foundation for blending advances in STrO and MTO\.

The relevant source code can be found in the link: https://github\.com/ambigeV/stro\.

\(a\)\(b\)
Fig\. 1:Workflow of distinct multitask optimization frameworks\. \(a\) Standard multitask optimization \(tasks evaluated evenly\) \(b\) The proposed multitask optimization with iterative sequential transfer optimization \(tasks evaluated selectively\)\.
## IIRelated Works

### II\-AFew\-Shot Multiobjective Multitask Optimization

This work addresses few\-shot multiobjective MTO \(MOMTO\) by extending the scalar functionsfk​\(⋅\)f\_\{k\}\(\\cdot\)in \([1](https://arxiv.org/html/2609.11228#S1.E1)\) to vector\-valued functionsFk​\(⋅\)F\_\{k\}\(\\cdot\)withmmobjectives\. Unlike standard MTO benchmarks that allowO⁡\(105\)O\(10^\{5\}\)evaluations per task\[[23](https://arxiv.org/html/2609.11228#bib.bib21)\], we constrain the budget for each task toO⁡\(102\)O\(10^\{2\}\)\. This*few\-shot optimization*setting better reflects real\-world constraints and demands more efficiency of search and transfer mechanisms\. To our best knowledge, the research efforts in this niche area remains limited\. However, preliminary approaches include maintaining diverse types of surrogate models\[[24](https://arxiv.org/html/2609.11228#bib.bib27)\]or constructing inverse multitask models\[[21](https://arxiv.org/html/2609.11228#bib.bib7)\]to tackle multiple expensive multiobjective tasks\. The latter method, F\-invTrEMO\[[21](https://arxiv.org/html/2609.11228#bib.bib7)\], serves as a baseline method in this paper, due to its flexibility for extension\.

### II\-BSequential Transfer Optimization

STrO, as formalized in \([2](https://arxiv.org/html/2609.11228#S1.E2)\), leverages optimization experiences from source data or models to expedite the search in a target task\. The pioneering method, AMTEA, applied transfer stacking to adaptively combine existing surrogates for target multiobjective problems\[[3](https://arxiv.org/html/2609.11228#bib.bib24)\]\. Other related studies focus on bridging source and target domains via transfer Gaussian Processes\[[19](https://arxiv.org/html/2609.11228#bib.bib12)\], inverse modeling\[[20](https://arxiv.org/html/2609.11228#bib.bib11)\], or optimal transport\[[25](https://arxiv.org/html/2609.11228#bib.bib26)\]\. Motivated by recent theoretical progress\[[19](https://arxiv.org/html/2609.11228#bib.bib12),[7](https://arxiv.org/html/2609.11228#bib.bib4),[26](https://arxiv.org/html/2609.11228#bib.bib25)\]and improvements in scalability\[[4](https://arxiv.org/html/2609.11228#bib.bib6)\], we propose to solve few\-shot MOMTO through iterative sequential transfer, inspired by the structural similarities between MTO and STrO and the few\-shot optimization challenge mentioned in Section I\.

Algorithm 1General Framework of ISTData:Task size

KK, Initial budgets

Ni​n​i​tN\_\{init\}, Total budgets for each task

Nt​o​tN\_\{tot\}, Objective functions

FkF\_\{k\}for each task

kk, Task prioritization function

ϕ⁡\(∙\)\\phi\(\\mathbin\{\\vbox\{\\hbox\{\\scalebox\{\.5\}\{$\\bullet$\}\}\}\}\)\.

Result:Optimal solutions for each optimization task\.

1foreach*taskkk*do

2Evaluate the objective function

FkF\_\{k\}of task

kkfor

Ni​n​i​tN\_\{init\}iterations

3

E​v​a​lk←Ni​n​i​tEval\_\{k\}\\leftarrow N\_\{init\}
4end foreach

5while*termination condition is not met*do

6

k←argmaxt∈\{1,…,K\}∧E​v​a​lt<Nt​o​tϕ​\(t\)k\\leftarrow\\argmax\_\{t\\in\\\{1,\\ldots,K\\\}\\land Eval\_\{t\}<N\_\{tot\}\}\\phi\(t\)
7

E​v​a​lk←E​v​a​lk\+1Eval\_\{k\}\\leftarrow Eval\_\{k\}\+1
8Solve formulation \([2](https://arxiv.org/html/2609.11228#S1.E2)\) with task

kkas the target task

9end while

## IIIMultitask Optimization via Iterative Sequential Transfer

This section delineates the proposed IST framework\. We first elucidate the workflow of the IST framework\. Next, we introduce the base sequential transfer optimizer utilized in this study: F\-invTrEMO\. By integrating this optimizer within the IST framework, a likelihood\-informed task prioritization mechanism is proposed\. This approach effectively models multitask optimization as a sequence of iterative sequential transfer optimization problems\.

### III\-AGeneral Framework

The proposed IST framework pursues the same objective as the MTO paradigm described in formulation \([1](https://arxiv.org/html/2609.11228#S1.E1)\), that is, generating the optimal solution set\(𝐱1∗,𝐱2∗,…,𝐱K∗\)\(\\mathbf\{x\}\_\{1\}^\{\*\},\\mathbf\{x\}\_\{2\}^\{\*\},\\ldots,\\mathbf\{x\}\_\{K\}^\{\*\}\)\. However, unlike standard MTO that disperses evaluations across tasks evenly, IST can model the problem as a sequence of STrO tasks as in \([2](https://arxiv.org/html/2609.11228#S1.E2)\)\. Per iteration, a single target taskTTis selected to receive an evaluation based on a likelihood\-informed task prioritization mechanism, formulated as:

min𝐱T∈ΩTfT​\(𝐱T\)given\{𝒟Sk\}k=1K\\displaystyle\\min\_\{\\mathbf\{x\}\_\{T\}\\in\\Omega\_\{T\}\}\\quad f\_\{T\}\(\\mathbf\{x\}\_\{T\}\)\\quad\\text\{given\}\\quad\\\{\\mathcal\{D\}\_\{S\_\{k\}\}\\\}\_\{k=1\}^\{K\}\(3\)s\.t\.T=argmaxt∈\{1,…,K\}∧E​v​a​lt<Nt​o​tϕ⁡\(t\)\\displaystyle\\text\{s\.t\.\}\\quad T=\\argmax\_\{t\\in\\\{1,\\ldots,K\\\}\\wedge Eval\_\{t\}<N\_\{tot\}\}\\quad\\phi\(t\)whereϕ⁡\(t\)\\phi\(t\)quantifies the utility of assigning taskttas the target given the current optimization history\{𝒟Sk\}\\\{\\mathcal\{D\}\_\{S\_\{k\}\}\\\}of all tasks\.

The primary distinctions of IST lie in its single\-directional knowledge transfer and its selective evaluation mode\. As illustrated in Fig\.[1b](https://arxiv.org/html/2609.11228#S1.F1.sf2), only the prioritized target is evaluated per iteration, reflecting an inherent budget designation process that prioritizes tasks most ready for knowledge integration\. This formulation allows IST to leverage current STrO methods for a more meticulous controlled transfer process, to potentially circumvent the few\-shot challenge\. The complete workflow of IST is detailed inAlgorithm[1](https://arxiv.org/html/2609.11228#algorithm1):

- •Initialization: Each task is evaluated forNi​n​i​tN\_\{init\}iterations, generating the source datasets𝒟S\\mathcal\{D\}\_\{S\}\.
- •Task Prioritization: A target task is identified per iteration by the task prioritization functionϕ⁡\(⋅\)\\phi\(\\cdot\)to maximize transfer utility, while ensuring no task exceeds the total evaluation limitNt​o​tN\_\{tot\}\.
- •Sequential Transfer Optimization: The identified target task is optimized by transferring knowledge from the remaining source tasks using established STrO solvers\.

### III\-BBase Sequential Transfer Optimizer

In this study, the IST framework is instantiated using the recently proposed F\-invTrEMO\[[21](https://arxiv.org/html/2609.11228#bib.bib7)\]\. This base optimizer leverages inter\-task relationships through a hybrid forward\-inverse mapping approach, particularly effective for few\-shot multiobjective multitask optimization\.

#### III\-B1Scalarizing Multiobjective Optimization

To handle multiple objectives within each task, the vector\-valued functionFT​\(⋅\)F\_\{T\}\(\\cdot\)is scalarized using augmented Tchebycheff scalarization\. For a given weight vector𝐰\\mathbf\{w\}from a\(m−1\)\(m\-1\)dimensional simplex𝒲\\mathcal\{W\}, the scalarized objective is:

fKt​c​h​\(𝐱K\|𝐰\)=\\displaystyle f^\{tch\}\_\{K\}\(\\mathbf\{x\}\_\{K\}\|\\mathbf\{w\}\)=max1≤i≤m⁡\{wi​\(fK,i​\(𝐱\)−\(zK,i∗−ϵ\)\)\}\+\\displaystyle\\max\_\{1\\leq i\\leq m\}\\\{w\_\{i\}\(f\_\{K,i\}\(\\mathbf\{x\}\)\-\(z\_\{K,i\}^\{\*\}\-\\epsilon\)\)\\\}\+\(4\)ρ​∑i=1Mwi​fK,i​\(𝐱\)\\displaystyle\\rho\\sum\_\{i=1\}^\{M\}w\_\{i\}f\_\{K,i\}\(\\mathbf\{x\}\)wherezK,i∗z\_\{K,i\}^\{\*\}is the ideal point,zK,i∗−ϵz\_\{K,i\}^\{\*\}\-\\epsilonprovides a utopia point, andρ\\rhois a small constant to maintain Pareto optimality\.

#### III\-B2Multitask Gaussian Process

To address few\-shot multi\-task optimization problems, the Multitask Gaussian Process \(MTGP\)\[[27](https://arxiv.org/html/2609.11228#bib.bib3)\]is generally adopted to alleviate the evaluation cost and enable knowledge transfer\. Given input spaces across tasksΩk,k∈\{1,…,K\}\\Omega\_\{k\},k\\in\\\{1,\\ldots,K\\\}, scalarized objective functions,f1t​c​h,…,fKt​c​hf\_\{1\}^\{tch\},\\ldots,f\_\{K\}^\{tch\}, are modelled by MTGP\. During the modeling process, we have triplets\{\(is,𝐱s\),ys\}s=1N\\\{\(i\_\{s\},\\mathbf\{x\}\_\{s\}\),y\_\{s\}\\\}\_\{s=1\}^\{N\}withNNevaluated solutions, withisi\_\{s\}, the task index of thess\-th evaluated solution,𝐱s∈Ωis\\mathbf\{x\}\_\{s\}\\in\\Omega\_\{i\_\{s\}\}, solutions, andys=fist​c​h​\(𝐱s\)\+ϵisy\_\{s\}=f\_\{i\_\{s\}\}^\{tch\}\(\\mathbf\{x\}\_\{s\}\)\+\\epsilon\_\{i\_\{s\}\}, noisy evaluations where the task\-dependent noiseϵis\\epsilon\_\{i\_\{s\}\}is additive Gaussian noise with zero mean \(i\.e\.,ϵis∼𝒩⁡\(0,σis2\)\\epsilon\_\{i\_\{s\}\}\\sim\\mathcal\{N\}\(0,\\sigma\_\{i\_\{s\}\}^\{2\}\)\)\. MTGP\[[27](https://arxiv.org/html/2609.11228#bib.bib3)\]is distinct for the formulation of the multitask kernel as below:

κ⁡\(\(i,𝐱\),\(i′,𝐱′\)\)=κ𝒯​\(i,i′\)⋅κΩ​\(𝐱,𝐱′\)\\kappa\(\(i,\\mathbf\{x\}\),\(i^\{\\prime\},\\mathbf\{x\}^\{\\prime\}\)\)=\\kappa\_\{\\mathcal\{T\}\}\(i,i^\{\\prime\}\)\\cdot\\kappa\_\{\\Omega\}\(\\mathbf\{x\},\\mathbf\{x\}^\{\\prime\}\)\(5\)where the pair \(i,𝐱i,\\mathbf\{x\}\) represents solution𝐱\\mathbf\{x\}for taskii,κ𝒯\\kappa\_\{\\mathcal\{T\}\}measures the similarities among tasks, andκΩ\\kappa\_\{\\Omega\}measures the similarities among solutions\. Given the multitask kernel in formulation \([5](https://arxiv.org/html/2609.11228#S3.E5)\), one can estimate the posterior distribution,𝒩⁡\(μ⁡\(i,𝐱\),σ2​\(i,𝐱\)\)\\mathcal\{N\}\(\\mu\(i,\\mathbf\{x\}\),\\sigma^\{2\}\(i,\\mathbf\{x\}\)\), of a query pair \(i,𝐱i,\\mathbf\{x\}\) as follows:

μt\(i,𝐱\)=𝜿t\(i,𝐱\)⊺\(𝐊t\+𝚲\)−1𝐲1:t\\mu\_\{t\}\(i,\\mathbf\{x\}\)=\\bm\{\\kappa\}\_\{t\}\(i,\\mathbf\{x\}\)^\{\\intercal\}\(\\mathbf\{K\}\_\{t\}\+\\mathbf\{\\Lambda\}\)^\{\-1\}\\mathbf\{y\}\_\{1:t\}\(6\)σt2\(i,𝐱\)=κ\(\(i,\\displaystyle\\sigma\_\{t\}^\{2\}\(i,\\mathbf\{x\}\)=\\kappa\(\(i,OPENOPEN𝐱\),\(i,𝐱\)\)−\\displaystyle\\mathbf\{x\}\),\(i,\\mathbf\{x\}\)\)\-\(7\)𝜿t​\(i,𝐱\)⊺​\(𝐊t\+𝚲\)−1​𝜿t​\(i,𝐱\)\\displaystyle\\bm\{\\kappa\}\_\{t\}\(i,\\mathbf\{x\}\)^\{\\intercal\}\(\\mathbf\{K\}\_\{t\}\+\\mathbf\{\\Lambda\}\)^\{\-1\}\\bm\{\\kappa\}\_\{t\}\(i,\\mathbf\{x\}\)where𝜿t​\(i,x\)=\{κ⁡\(\(i,𝐱\),\(is,𝐱s\)\)\}s=1t\\bm\{\\kappa\}\_\{t\}\(i,x\)=\\\{\\kappa\(\(i,\\mathbf\{x\}\),\(i\_\{s\},\\mathbf\{x\}\_\{s\}\)\)\\\}\_\{s=1\}^\{t\},𝐲1:t=\{ys\}s=1t\\mathbf\{y\}\_\{1:t\}=\\\{y\_\{s\}\\\}\_\{s=1\}^\{t\},𝐊t=\{κ⁡\(\(is,𝐱s\),\(is′,𝐱s′\)\)\}s,s′=1t\\mathbf\{K\}\_\{t\}=\\\{\\kappa\(\(i\_\{s\},\\mathbf\{x\}\_\{s\}\),\(i\_\{s^\{\\prime\}\},\\mathbf\{x\}\_\{s^\{\\prime\}\}\)\)\\\}\_\{s,s^\{\\prime\}=1\}^\{t\}, and𝚲\\mathbf\{\\Lambda\}is the additive noise variance matrix of MTGP\.

#### III\-B3MTGP\-based Forward\-Inverse Transfer

Given the scalarized objective functions in formulation \([4](https://arxiv.org/html/2609.11228#S3.E4)\) and the MTGP model in formulation \([6](https://arxiv.org/html/2609.11228#S3.E6)\) and \([7](https://arxiv.org/html/2609.11228#S3.E7)\) for solving few\-shot optimization, the knowledge transfer can be conducted in a hybrid forward\-inverse approach\. The forward mapping,Ψf​o​r\\Psi\_\{for\}, only approximates the scalarized objective functions,f𝒯t​c​hf^\{tch\}\_\{\\mathcal\{T\}\}, that is,Ψf​o​r:Ω𝒯↦ℝ\\Psi\_\{for\}:\\Omega\_\{\\mathcal\{T\}\}\\mapsto\\mathbb\{R\}\. The inverse mapping,Ψi​n​v\\Psi\_\{inv\}, models the transformation from the\(m−1\)\(m\-1\)dimensional simplex,𝒲\\mathcal\{W\}, which includes the weight vector𝐰\\mathbf\{w\}to the solution space,Ω𝒯\\Omega\_\{\\mathcal\{T\}\}, that is,Ψi​n​v:𝒲↦Ω𝒯\\Psi\_\{inv\}:\\mathcal\{W\}\\mapsto\\Omega\_\{\\mathcal\{T\}\}\. In this paper, we assume each task contains solutions indddimensions, that is,Ω𝒯⊂ℝd\\Omega\_\{\\mathcal\{T\}\}\\subset\\mathbb\{R\}^\{d\}\. To relieve the computational burden of multi\-output GP modeling, we separated the inverse modeling into a series of single\-output mapping,Ψi​n​v,i:𝒲↦Ω𝒯,i,Ω𝒯,i⊂ℝ,i∈\{1,…,d\}\\Psi\_\{inv,i\}:\\mathcal\{W\}\\mapsto\\Omega\_\{\\mathcal\{T\},i\},\\Omega\_\{\\mathcal\{T\},i\}\\subset\\mathbb\{R\},i\\in\\\{1,\\ldots,d\\\}\. After this separate single\-output GP modeling, the predictions can then be aggregated to form the originaldd\-dimensional predictions\. Generally, prior to the inverse modeling process, the data\{𝐰s,\(is,𝐱s\)\}s=1N\\\{\\mathbf\{w\}\_\{s\},\(i\_\{s\},\\mathbf\{x\}\_\{s\}\)\\\}\_\{s=1\}^\{N\}should be prepared in advance\. In this paper, we assume that these data pairs have been constructed already, and one can refer to\[[20](https://arxiv.org/html/2609.11228#bib.bib11)\]for the details and rationale behind them\.

Algorithm 2Workflow of F\-invTrEMO in One PassData:Target task

𝒯K\\mathcal\{T\}\_\{K\}, Source tasks

𝒯1,…,𝒯K−1\\mathcal\{T\}\_\{1\},\\ldots,\\mathcal\{T\}\_\{K\-1\}, Dimension size

dd, Predefined weight vector

𝐰~\\tilde\{\\mathbf\{w\}\}, Evaluation records

\{\(is,𝐱s\),ys,𝐰s\}s=1N\\\{\(i\_\{s\},\\mathbf\{x\}\_\{s\}\),y\_\{s\},\\mathbf\{w\}\_\{s\}\\\}\_\{s=1\}^\{N\}, Sample size

NSN\_\{S\}\.

Result:Optimal solutions for the target task\.

1/\*Forward MTGP Modeling\*/

2Build a forward MTGP model,

𝒩⁡\(μf​m​t​\(i,𝐱\),σf​m​t2​\(i,𝐱\)\)\\mathcal\{N\}\(\\mu\_\{fmt\}\(i,\\mathbf\{x\}\),\\sigma^\{2\}\_\{fmt\}\(i,\\mathbf\{x\}\)\), based on optimization history,

\{\(is,𝐱s\),ys\}s=1N\\\{\(i\_\{s\},\\mathbf\{x\}\_\{s\}\),y\_\{s\}\\\}\_\{s=1\}^\{N\}\.

3/\*Inverse MTGP Modeling\*/

4foreach*dimensionjj*do

5Build

ddinverse MTGP models,

𝒩⁡\(μi​m​t,j​\(i,𝐰\),σi​m​t,j2​\(i,𝐰\)\)\\mathcal\{N\}\(\\mu\_\{imt,j\}\(i,\\mathbf\{w\}\),\\sigma^\{2\}\_\{imt,j\}\(i,\\mathbf\{w\}\)\), based on the evaluation records,

\{𝐰s,\(is,𝐱s,j\)\}s=1N\\\{\\mathbf\{w\}\_\{s\},\(i\_\{s\},\\mathbf\{x\}\_\{s,j\}\)\\\}\_\{s=1\}^\{N\}\.

6end foreach

7/\*Conducting Sequential Transfer Optimization\*/

8Sample solution set

𝒰𝒯K\\mathcal\{U\}\_\{\\mathcal\{T\}\_\{K\}\}from

NSN\_\{S\}solutions from distribution,

𝒩⁡\(μi​m​t​\(𝒯K,𝐰~\),σi​m​t2​\(𝒯K,𝐰~\)\)\\mathcal\{N\}\(\\mu\_\{imt\}\(\\mathcal\{T\}\_\{K\},\\tilde\{\\mathbf\{w\}\}\),\\sigma^\{2\}\_\{imt\}\(\\mathcal\{T\}\_\{K\},\\tilde\{\\mathbf\{w\}\}\)\)\.

9Select solution

𝐱~=argmax𝐱𝒯K∈𝒰𝒯K−μf​m​t​\(𝒯K,𝐱𝒯K\)\+β⋅σf​m​t​\(𝒯K,𝐱𝒯K\)\\tilde\{\\mathbf\{x\}\}=\\argmax\\limits\_\{\\mathbf\{x\}\_\{\\mathcal\{T\}\_\{K\}\}\\in\\mathcal\{U\}\_\{\\mathcal\{T\}\_\{K\}\}\}\-\\mu\_\{fmt\}\(\\mathcal\{T\}\_\{K\},\\mathbf\{x\}\_\{\\mathcal\{T\}\_\{K\}\}\)\+\\beta\\cdot\\sigma\_\{fmt\}\(\\mathcal\{T\}\_\{K\},\\mathbf\{x\}\_\{\\mathcal\{T\}\_\{K\}\}\)\.

10Evaluate solution

𝐱~\\tilde\{\\mathbf\{x\}\}in the target task

𝒯K\\mathcal\{T\}\_\{K\}

Considering this hybrid forward\-inverse transfer mechanism,Algorithm[2](https://arxiv.org/html/2609.11228#algorithm2)demonstrates the workflow of F\-invTrEMO with a single pass\. Given the datasets for both forward mappingΨf​o​r\\Psi\_\{for\}and inverse mappingΨi​n​v\\Psi\_\{inv\}per iteration, both forward and inverse MTGP models are constructed and utilized for sequential transfer optimization directly, as depicted in line 8 to line 10 inAlgorithm[2](https://arxiv.org/html/2609.11228#algorithm2)\. Particularly, since this study focuses on minimization, the solution selection mechanism aims to maximize the lower confidence bound \(LCB\)\[[28](https://arxiv.org/html/2609.11228#bib.bib2)\]as shown in line 9\.

#### III\-B4Factorized MTGP\-based Forward\-Inverse Transfer

Our paper implements the MTGP\-based forward\-inverse transfer based on an efficient formulation in\[[29](https://arxiv.org/html/2609.11228#bib.bib1)\]so that the problem that,joint training MTGP can bias the source task when the volume of the source task is significantly more than that of the target task\[[29](https://arxiv.org/html/2609.11228#bib.bib1)\], can be alleviated\. Specifically, instead of training the MTGP model for all theKKtasks jointly, we replace the formulation \([6](https://arxiv.org/html/2609.11228#S3.E6)\) and \([7](https://arxiv.org/html/2609.11228#S3.E7)\) with the formulation as follows:

μf​m​t\(𝒯K,\\displaystyle\\mu\_\{fmt\}\(\\mathcal\{T\}\_\{K\},𝐱\)=σ2f​m​t\(𝒯K,𝐱\)\{\(∑j=1K−1σ𝒯j−2\(𝒯K,𝐱\)⋅\\displaystyle\\mathbf\{x\}\)=\\sigma^\{2\}\_\{fmt\}\(\\mathcal\{T\}\_\{K\},\\mathbf\{x\}\)\\\{\(\\sum\_\{j=1\}^\{K\-1\}\\sigma\_\{\\mathcal\{T\}\_\{j\}\}^\{\-2\}\(\\mathcal\{T\}\_\{K\},\\mathbf\{x\}\)\\cdot\(8\)μ𝒯j\(𝒯K,𝐱\)\+\(2−K\)⋅σ−2𝒯K\(𝐱\)⋅μ𝒯K\(𝐱\)\}\\displaystyle\\mu\_\{\\mathcal\{T\}\_\{j\}\}\(\\mathcal\{T\}\_\{K\},\\mathbf\{x\}\)\+\(2\-K\)\\cdot\\sigma^\{\-2\}\_\{\\mathcal\{T\}\_\{K\}\}\(\\mathbf\{x\}\)\\cdot\\mu\_\{\\mathcal\{T\}\_\{K\}\}\(\\mathbf\{x\}\)\\\}σf​m​t2​\(𝒯K,𝐱\)=1/\{∑j=1K−1σ𝒯j−2​\(𝒯K,𝐱\)\+\(2−K\)⋅σ𝒯K−2​\(𝐱\)\}\\sigma^\{2\}\_\{fmt\}\(\\mathcal\{T\}\_\{K\},\\mathbf\{x\}\)=1/\\\{\\sum\_\{j=1\}^\{K\-1\}\\sigma\_\{\\mathcal\{T\}\_\{j\}\}^\{\-2\}\(\\mathcal\{T\}\_\{K\},\\mathbf\{x\}\)\+\(2\-K\)\\cdot\\sigma^\{\-2\}\_\{\\mathcal\{T\}\_\{K\}\}\(\\mathbf\{x\}\)\\\}\(9\)whereμ𝒯j​\(𝒯K,𝐱\)\\mu\_\{\\mathcal\{T\}\_\{j\}\}\(\\mathcal\{T\}\_\{K\},\\mathbf\{x\}\)andσ𝒯j2​\(𝒯K,𝐱\)\\sigma^\{2\}\_\{\\mathcal\{T\}\_\{j\}\}\(\\mathcal\{T\}\_\{K\},\\mathbf\{x\}\)denote the posterior mean and variance of the solution𝐱\\mathbf\{x\}in task𝒯K\\mathcal\{T\}\_\{K\}with MTGP using only the source task𝒯j\\mathcal\{T\}\_\{j\}and the target task𝒯K​\(𝐱\)\\mathcal\{T\}\_\{K\}\(\\mathbf\{x\}\),μ𝒯K−2​\(𝐱\)\\mu^\{\-2\}\_\{\\mathcal\{T\}\_\{K\}\}\(\\mathbf\{x\}\)andσ𝒯K−2​\(𝐱\)\\sigma^\{\-2\}\_\{\\mathcal\{T\}\_\{K\}\}\(\\mathbf\{x\}\)are posterior mean and variance of the target task𝒯𝒦\\mathcal\{T\_\{K\}\}using only the single\-task GP\. This formulation is practically useful inspired by recent works in STrO domains\[[19](https://arxiv.org/html/2609.11228#bib.bib12),[26](https://arxiv.org/html/2609.11228#bib.bib25)\]\.

### III\-CLikelihood\-Informed Task Prioritization

The efficacy of the IST framework hinges on identifying the optimal target taskTTto receive the next evaluation budget\. We instantiate the task prioritization functionϕ⁡\(⋅\)\\phi\(\\cdot\)by leveraging the inter\-task relationships captured during the MTGP modeling process\.

#### III\-C1Task Prioritization Function

The search synergy between any task pair\(i,i′\)\(i,i^\{\\prime\}\)is explicitly quantified through the task kernel parameterκ𝒯​\(i,i′\)\\kappa\_\{\\mathcal\{T\}\}\(i,i^\{\\prime\}\), where tasksiiandi′i^\{\\prime\}serve as source and target, respectively\. This kernel can be estimated online per iteration as outlined inAlgorithm[2](https://arxiv.org/html/2609.11228#algorithm2)\. As indicated in\[[26](https://arxiv.org/html/2609.11228#bib.bib25)\], for sequential transfer optimization with a single target task𝒯K\\mathcal\{T\}\_\{K\}, a larger task search synergy parameterκ𝒯​\(𝒯S,𝒯K\)\\kappa\_\{\\mathcal\{T\}\}\(\\mathcal\{T\}\_\{S\},\\mathcal\{T\}\_\{K\}\)suggests that Gaussian process optimization with the LCB component can achieve a lower optimization regret bound compared to source tasks with lower search synergy parameters\.

Because the search synergy parameter is learned through maximum likelihood estimation, it can be treated as the statistical likelihood that a given task pair should serve as source and target, respectively\[[27](https://arxiv.org/html/2609.11228#bib.bib3),[29](https://arxiv.org/html/2609.11228#bib.bib1)\]\. We therefore formulate the task prioritization function as:

ϕ⁡\(t\)=mini≠t⁡κ𝒯​\(i,t\)\.\\phi\(t\)=\\min\_\{i\\neq t\}\\kappa\_\{\\mathcal\{T\}\}\(i,t\)\.\(10\)While highκ𝒯\\kappa\_\{\\mathcal\{T\}\}is generally preferred for beneficial transfer, themin\\minoperator in \([10](https://arxiv.org/html/2609.11228#S3.E10)\) ensures the selected targetttis compatible with the entire source task pool\. By maximizing this lower bound, the framework prioritizes targets that maintain a high degree of synergy across all potential source tasks, thereby ensuring reliable knowledge integration\.

#### III\-C2Stochastic Task Prioritization

Given the definition of the task prioritization functionϕ⁡\(t\)\\phi\(t\), according toAlgorithm[1](https://arxiv.org/html/2609.11228#algorithm1), we assign the target role to taskkkwith the largestϕ⁡\(t\)\\phi\(t\)\. However, theargmax\\argmaxselector may cause certain tasks to remain static for too many iterations, violating the assumption in the sequential transfer optimization that all the source tasks can achieve high\-quality solution distributions\. To mitigate this, we adopt the softmax function to relax thisargmax\\argmaxselector as follows:

𝒯K∼M​u​l​t​i​n​o​m​i​a​l​\(1,\{P1,…,PK\}\)\\mathcal\{T\}\_\{K\}\\sim Multinomial\(1;\\\{P\_\{1\},\\dots,P\_\{K\}\\\}\)\(11\)Pt=exp⁡\(S⋅\(max⁡\{ϕ⁡\(t\)−θ,0\}\)\)∑j=1∧E​v​a​lj<Nt​o​tKexp⁡\(S⋅\(max⁡\{ϕ⁡\(j\)−θ,0\}\)\)P\_\{t\}=\\frac\{\\exp\(S\\cdot\(\\max\\\{\\phi\(t\)\-\\theta,0\\\}\)\)\}\{\\sum\_\{j=1\\land Eval\_\{j\}<N\_\{tot\}\}^\{K\}\\exp\(S\\cdot\(\\max\\\{\\phi\(j\)\-\\theta,0\\\}\)\)\}\(12\)where the target task𝒯K\\mathcal\{T\}\_\{K\}is sampled from a multinomial distribution with probabilitiesPtP\_\{t\}computed using this softmax function as defined in formula \([12](https://arxiv.org/html/2609.11228#S3.E12)\)\. In this formulation,SScontrols the selective pressure of the softmax function: a higherSSmakes it more similar toargmax\\argmaxfunction, while a lowerSSmakes it resemble the uniform selection\. Moreover, the parameterθ\\thetain formula \([12](https://arxiv.org/html/2609.11228#S3.E12)\) serves as a threshold value to maintain search vitality\. Specifically, while tasks withϕ⁡\(t\)\>θ\\phi\(t\)\>\\thetareceive an exponential boost in selection priority, all tasks falling below this threshold share an identical baseline weight \(e0=1e^\{0\}=1\)\. This ensures that even tasks with extremely low source\-target likelihood remain selectable with a non\-zero probability, preventing any task from being perpetually frozen during the iterative process\.

## IVExperimental Studies

To verify the effectiveness of the proposed IST framework and the likelihood\-informed task prioritization mechanism, we conduct comparative studies on MOMTO benchmark problems and multiobjective multitask hyperparameter optimization problems\. Moreover, to further justify the generality of the framework, we instantiate another IST\-based algorithm by extending a renowned sequential transfer optimizer, AMTEA\[[3](https://arxiv.org/html/2609.11228#bib.bib24)\], to AMTEA\-IST in Section IV\.D\. We term the F\-invTrEMO implementation under the IST framework as F\-invTrEMO\-IST\. We compare F\-invTrEMO\-IST with the classical ParEGO, because many STrO methods\[[19](https://arxiv.org/html/2609.11228#bib.bib12),[20](https://arxiv.org/html/2609.11228#bib.bib11),[21](https://arxiv.org/html/2609.11228#bib.bib7)\]build upon this single\-task few\-shot multiobjective optimizer, and the few\-shot multiobjective multitask optimizer, F\-invTrEMO\. One can refer to more detailed parameter settings in the supplementary materials111Supplymentary Materials: https://zenodo\.org/records/19060595\. Experiments upon benchmark problems are independently repeated 20 trials and upon hyperparameter optimization problems are repeated 10 independent trials\.

### IV\-ATest Problems

We conduct the comparative studies on multiobjective multitask benchmarks\[[23](https://arxiv.org/html/2609.11228#bib.bib21)\]\. It includes nine MOMTO problems and each problem contains two tasks with certain relationships regarding search space similarity and optima intersection\. The optima intersections includes complete intersection \(CI\), partial intersection \(PI\), and no intersection \(NI\), while the search space similarity includes high similarity \(HS\), medium similarity \(MS\), and low similarity \(LS\)\. Nine sets of problems can be constructed with these attributes, including CIHS, CIMS, CILS, PIHS, PIMS, PILS, NIHS, NIMS, and NILS\. However, NIMS and NILS are not included in this paper, since our works are based on decomposition\-based multiobjective optimizer, which cannot solve multitask problems with distinct objective sizes like NIMS and NILS\. As for this limitation, we leave it as a future direction\. One can refer to the supplementaries for more details on test problems\.

We adopt the inverted generational distance \(IGD\+\)\[[30](https://arxiv.org/html/2609.11228#bib.bib20)\]as the metric to quantify the performance of the algorithms, as recommended in\[[23](https://arxiv.org/html/2609.11228#bib.bib21)\]\. One can refer to the details of IGD\+ in the supplementary materials\. The statistical significance test is conducted by the Wilcoxon signed\-rank test\.

\(a\)\(b\)
Fig\. 2:The IGD\+ convergence trends of ParEGO, F\-invTrEMO, and F\-invTrEMO\-IST upon multitask optimization benchmark \(a\) PIHS \(b\) NIHS\.TABLE I:IGD\+ Comparison Results on multitask Optimization Benchmark For ParEGO, F\-invTrEMO, and F\-invTrEMO\-ISTProblemsTasksParEGOF\-invTrEMOF\-invTrEMO\-ISTCIHSTask\-11\.516E\+02 \(1\.545E\+01\) \+8\.061E\+01 \(1\.534E\+01\) \+7\.156E\+01 \(1\.142E\+01\)Task\-24\.877E\+00 \(3\.150E\-01\) \+3\.557E\+00 \(2\.329E\-01\)≈\\approx3\.389E\+00 \(1\.925E\-01\)CIMSTask\-12\.523E\+02 \(2\.057E\+01\) \+2\.086E\+02 \(2\.658E\+01\) \+1\.954E\+02 \(4\.069E\+01\)Task\-24\.174E\+00 \(2\.983E\-01\) \+3\.819E\+00 \(2\.859E\-01\) \+3\.606E\+00 \(1\.799E\-01\)CILSTask\-14\.564E\+01 \(2\.021E\+00\) \+4\.435E\+01 \(1\.592E\+00\) \+4\.113E\+01 \(1\.601E\+00\)Task\-23\.050E\-01 \(1\.040E\-02\) \-7\.938E\-01 \(5\.305E\-02\) \+6\.856E\-01 \(4\.870E\-02\)PIHSTask\-18\.728E\+01 \(9\.112E\+00\) \+3\.234E\+01 \(8\.086E\+00\) \+1\.399E\+01 \(1\.978E\+00\)Task\-21\.841E\+02 \(1\.678E\+01\) \+9\.913E\+01 \(1\.033E\+01\) \+8\.109E\+01 \(3\.761E\+00\)PIMSTask\-12\.678E\+00 \(1\.539E\-01\) \+4\.324E\-01 \(3\.790E\-02\) \-4\.784E\-01 \(5\.205E\-02\)Task\-23\.736E\+02 \(1\.269E\+01\) \+3\.503E\+02 \(5\.258E\+00\)≈\\approx3\.407E\+02 \(4\.165E\+00\)PILSTask\-11\.008E\+00 \(1\.390E\-02\)≈\\approx1\.029E\+00 \(1\.165E\-02\)≈\\approx1\.041E\+00 \(9\.500E\-03\)Task\-21\.423E\+01 \(6\.948E\-01\) \+1\.258E\+01 \(8\.887E\-01\) \+1\.053E\+01 \(6\.600E\-01\)NIHSTask\-13\.174E\+03 \(6\.287E\+02\) \+1\.651E\+03 \(5\.881E\+02\) \+5\.934E\+02 \(2\.174E\+02\)Task\-25\.820E\+01 \(4\.852E\+00\) \+2\.834E\+01 \(3\.824E\+00\) \+1\.903E\+01 \(4\.056E\+00\)

### IV\-BResults

In our comparative study upon the multitask optimization benchmark, we compare the proposed method F\-invTrEMO\-IST with the single\-task method ParEGO and the plain method F\-invTrEMO without IST settings\. It can be indicated in TABLE I that both F\-invTrEMO and F\-invTrEMO\-IST can significantly outperform the baseline method ParEGO due to the ability to exploit intertask relationship information through the forward\-inverse transfer mechanism\. The exceptions occur in both problem sets with the relationship LS \(i\.e\., CILS and PILS\), where the search space between tasks has the least similarity\. Since both F\-invTrEMO and F\-invTrEMO\-IST maintain the solution distribution by mapping the predefined weight vector𝐰\\mathbf\{w\}to the solution space, and the dataset used to train the mapping process is constrained according to the previous optimization process, the solution distribution can be spuriously biased towards the local Pareto front, hindering the subsequent optimization process\. Moreover, the least similar search space between tasks makes it difficult to utilize the knowledge transfer to help escape the local optima in the target task from source tasks\. In terms of the comparison between F\-invTrEMO and F\-invTrEMO\-IST, it can be found in TABLE I that F\-invTrEMO\-IST that is implemented with IST settings can outperform the counterpart in 12 of 14 tasks, suggesting the superiority of the proposed IST framework\. In most problem sets, IST can identify the more proper source\-target pair to conduct the knowledge transfer process using the proposed task prioritization mechanism, mitigating the potential negative transfer\. Since the knowledge transfer process for source\-task pairs with lower transfer utilities should be suppressed in certain stages, IST can also be viewed as an approach to implement adaptive resource allocation on the fly\.

### IV\-CA Case Study on Real\-world Application: Multiobjective Multitask Hyperparameter Optimization

Hyperparameter optimization \(HPO\) is a standard topic in the field of machine learning\. The configuration of hyperparameters for machine learning models can impact the model performance, computational resources, and interpretability to decision\-makers\. In this paper, we apply F\-invTrEMO\-IST to optimize the hyperparameters of these models for the aforementioned criterion across distinct tasks\. We consider the following two scenarios:

- •\(HPO\-1\) The first scenario contains three HPO problems\. The three problems tune the hyperparameters of the same model, Random Forest, but on three distinct classification tasks: credit approval, medical diagnosis, and speech recognition problems\.
- •\(HPO\-2\) The second scenario contains two HPO problems\. The two optimization problems tune the hyperparameters of distinct but related models on the same classification task, speech recognition problem\[[31](https://arxiv.org/html/2609.11228#bib.bib5)\]\. One can refer to the definition of each objective function in supplementary materials\.

TABLE II:IGD\+ Comparison Results on multitask Hyperparameter Tuning Problems for F\-invTrEMO and F\-invTrEMO\-ISTAs illustrated in TABLE II, with the IST framework, the proposed F\-invTrEMO\-IST can outperform F\-invTrEMO across all the optimization tasks, generating better machine learning model sets trading off across model precision, model size, and interpretability\. Importantly, in real\-world cases, it is rare that the solution optima and search spaces share as high commonalities as benchmark problems such as CIHS or CIMS in TABLE I\. This heterogeneity in both optimal distribution and search space similarity makes it important to consider the utilization of computing resources and knowledge transfer direction, where the proposediterative sequential transferframework matters\.

### IV\-DA Case Study on Generality of the Framework: IST based on AMTEA

To verify the generality of the proposed IST on other sequential transfer optimizers, IST is integrated with another evolutionary sequential transfer optimizer, AMTEA\[[3](https://arxiv.org/html/2609.11228#bib.bib24)\]\. We term AMTEA with IST settings as AMTEA\-IST\. AMTEA is originally designed for the canonical algorithms requiring thousands of function evaluations, while we integrate AMTEA into the*few\-shot multiobjective*multitask optimization\. To this end, we transform the multiobjective problem into single\-objective problems via \([4](https://arxiv.org/html/2609.11228#S3.E4)\) at each iteration\. The single\-objective optimization is then evaluated using a Gaussian Process surrogate model, with solutions sampled from the solution distribution managed by AMTEA, similar to the approach used in ParEGO andAlgorithm[2](https://arxiv.org/html/2609.11228#algorithm2)\. One can refer to the supplementary materials for more details\.

The results can be found in TABLE S\-I in supplementary materials that both AMTEA and AMTEA\-IST can outperform the ParEGO counterpart in 13 of 14 tasks, showcasing the effectiveness of the knowledge transfer in the context of few\-shot MOMTO\. Comparing AMTEA to AMTEA\-IST, with the meticulous transfer direction configuration, AMTEA\-IST can outperform AMTEA in 11 of 14 multiobjective tasks\. Generally\. Moreover, the IST framework can facilitate a better search process for the base optimizer in multi\-task settings with lower search space similarity, as shown in TABLE S\-I\. The improvements upon AMTEA for CIHS and CIMS are not significant since high\-quality solutions from both tasks are similar, so that the task prioritization components cannot precisely capture the inter\-task relationship, thereby misleading the IST framework\. In contrast, for those problem sets with lower similarity in TABLE S\-I, AMTEA\-IST generally can significantly outperform AMTEA in light of the ability to filter out the least likely task pair\.

## VConclusion

In this work, we solve the few\-shot MOMTO by introducing the Iterative Sequential Transfer \(IST\)\. Unlike conventional methods, our IST framework can model the multitask problem as a sequence of sequential transfer optimization problems\. To facilitate this, a likelihood\-informed task prioritization mechanism is devised to actively control the knowledge transfer direction\. This novel framework alleviates the inherent limitations of simultaneous evaluation processes in traditional MTO, thereby mitigating negative transfer and enhancing efficiency in few\-shot regimes\. The IST framework not only advances traditional MTO but also accommodates existing sequential transfer optimization algorithms\. By bridging the gap between multitask and sequential transfer studies, this approach paves the way for more efficient algorithmic designs\. The comprehensive evaluation across benchmark and real\-world hyperparameter testbeds demonstrates the effectiveness of the proposed framework under stringent evaluation budgets\.

## Acknowledgments

This research is partly supported by the National Research Foundation, Singapore and DSO National Laboratories under the AI Singapore Programme \(AISG Award No\.: AISG2\- GC\-2023\-010, Design Beyond What You Know: Material\-Informed Differential Generative AI \(MIDGAI\) for Light\-Weight High\-Entropy Alloys and Multi\-functional Composites \(Stage 1b\), the A\*STAR Catalyst Project for Artificial Intelligence in Drug Discovery \(AIDD\) Programme \(Grant No\. H25A1N0004\), the Centre for Frontier AI Research \(CFAR\) under Agency for Science, Technology and Research \(A\*STAR\), and the College of Computing and Data Science, Nanyang Technological University\.

## References

- \[1\]A\. Gupta, Y\.\-S\. Ong, and L\. Feng\(2018\)Insights on transfer optimization: because experience is the best teacher\.IEEE Trans\. on Emerg\. Topics in Comput\. Intell\.2\(1\),pp\. 51–64\.External Links:[Document](https://dx.doi.org/10.1109/TETCI.2017.2769104)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p1.1)\.
- \[2\]L\. Feng, A\. Gupta, K\. C\. Tan, and Y\. Ong\(2023\)Evolutionary multi\-task optimization: foundations and methodologies\.Springer\.Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p1.1)\.
- \[3\]B\. Da, A\. Gupta, and Y\. Ong\(2018\)Curbing negative influences online for seamless transfer evolutionary optimization\.IEEE Trans\. on Cybern\.49\(12\),pp\. 4365–4378\.Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p1.1),[§I](https://arxiv.org/html/2609.11228#S1.p5.1),[§II\-B](https://arxiv.org/html/2609.11228#S2.SS2.p1.1),[§IV\-D](https://arxiv.org/html/2609.11228#S4.SS4.p1.1),[§IV](https://arxiv.org/html/2609.11228#S4.p1.1)\.
- \[4\]M\. Shakeri, E\. Miahi, A\. Gupta, and Y\. Ong\(2023\)Scalable transfer evolutionary optimization: coping with big task instances\.IEEE Trans\. on Cybern\.53\(10\),pp\. 6160–6172\.External Links:[Document](https://dx.doi.org/10.1109/TCYB.2022.3164399)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p1.1),[§II\-B](https://arxiv.org/html/2609.11228#S2.SS2.p1.1)\.
- \[5\]A\. Gupta, Y\. Ong, and L\. Feng\(2016\)Multifactorial evolution: toward evolutionary multitasking\.IEEE Trans\. on Evol\. Comput\.20\(3\),pp\. 343–357\.Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p1.1),[§I](https://arxiv.org/html/2609.11228#S1.p2.1),[§I](https://arxiv.org/html/2609.11228#S1.p2.2)\.
- \[6\]K\. K\. Bali, Y\. Ong, A\. Gupta, and P\. S\. Tan\(2019\)Multifactorial evolutionary algorithm with online transfer parameter estimation: mfea\-ii\.IEEE Trans\. on Evol\. Comput\.24\(1\),pp\. 69–83\.Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p1.1),[§I](https://arxiv.org/html/2609.11228#S1.p2.2)\.
- \[7\]T\. Wei, J\. Liu, A\. Gupta, P\. S\. Tan, and Y\. Ong\(2026\)\(θl\\theta\_\{l\},θu\\theta\_\{u\}\)\-Parametric multi\-task optimization: joint search in solution and infinite task spaces\.IEEE Trans\. on Evol\. Comput\.30\(3\),pp\. 1270–1283\.External Links:[Document](https://dx.doi.org/10.1109/TEVC.2025.3598333)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p1.1),[§I](https://arxiv.org/html/2609.11228#S1.p2.1),[§II\-B](https://arxiv.org/html/2609.11228#S2.SS2.p1.1)\.
- \[8\]B\. Da, A\. Gupta, Y\. Ong, and L\. Feng\(2016\)Evolutionary multitasking across single and multi\-objective formulations for improved problem solving\.In2016 IEEE Congr\. on Evol\. Comput\. \(CEC\),Vol\.,pp\. 1695–1701\.External Links:[Document](https://dx.doi.org/10.1109/CEC.2016.7743992)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p1.1)\.
- \[9\]K\. K\. Bali, A\. Gupta, Y\. Ong, and P\. S\. Tan\(2021\)Cognizant multitasking in multiobjective multifactorial evolution: mo\-mfea\-ii\.IEEE Trans\. on Cybern\.51\(4\),pp\. 1784–1796\.Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p2.2)\.
- \[10\]L\. Feng, L\. Zhou, J\. Zhong, A\. Gupta, Y\. Ong, K\. Tan, and A\. K\. Qin\(2018\)Evolutionary multitasking via explicit autoencoding\.IEEE Trans\. on Cybern\.49\(9\),pp\. 3457–3470\.Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p2.2)\.
- \[11\]Z\. Chen, Y\. Zhou, X\. He, and J\. Zhang\(2020\)Learning task relationships in evolutionary multitasking for multiobjective continuous optimization\.IEEE Trans\. on Cybern\.\(\),pp\. 1–12\.External Links:[Document](https://dx.doi.org/10.1109/TCYB.2020.3029176)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p2.2)\.
- \[12\]A\. Gupta, L\. Zhou, Y\. Ong, Z\. Chen, and Y\. Hou\(2022\)Half a dozen real\-world applications of evolutionary multitasking, and more\.IEEE Comput\. Intell\. Mag\.17\(2\),pp\. 49–66\.External Links:[Document](https://dx.doi.org/10.1109/MCI.2022.3155332)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p2.2)\.
- \[13\]W\. Dai, Z\. Wang, and K\. Xue\(2021\)System\-in\-package design using multi\-task memetic learning and optimization\.Memetic Comput\.\.External Links:ISSN 1865\-9292,[Document](https://dx.doi.org/10.1007/s12293-021-00346-5),[Link](https://doi.org/10.1007/s12293-021-00346-5)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p2.2)\.
- \[14\]T\. Wei, J\. Liu, A\. Gupta, C\. C\. Ooi, P\. S\. Tan, and Y\. Ong\(2026\)Parametric expensive multi\-objective optimization via generative solution modeling\.External Links:2511\.09598Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p2.2),[§I](https://arxiv.org/html/2609.11228#S1.p3.1)\.
- \[15\]L\. Luo, X\. Wang, J\. Ma, and Y\. Ong\(2023\)GrpAvoid: multigroup collision\-avoidance control and optimization for uav swarm\.IEEE Trans\. on Cybern\.53\(3\),pp\. 1776–1789\.External Links:[Document](https://dx.doi.org/10.1109/TCYB.2021.3132044)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p2.2)\.
- \[16\]T\. Wei, S\. Wang, J\. Zhong, D\. Liu, and J\. Zhang\(2022\)A review on evolutionary multitask optimization: trends and challenges\.IEEE Trans\. on Evol\. Comput\.26\(5\),pp\. 941–960\.External Links:[Document](https://dx.doi.org/10.1109/TEVC.2021.3139437)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p3.1)\.
- \[17\]M\. Gong, Z\. Tang, H\. Li, and J\. Zhang\(2019\)Evolutionary multitasking with dynamic resource allocating strategy\.IEEE Trans\. on Evol\. Comput\.23\(5\),pp\. 858–869\.Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p3.1)\.
- \[18\]T\. Wei and J\. Zhong\(2021\)Towards generalized resource allocation on evolutionary multitasking for multi\-objective optimization\.IEEE Comput\. Intell\. Mag\.16\(4\),pp\. 20–37\.External Links:[Document](https://dx.doi.org/10.1109/MCI.2021.3108310)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p3.1)\.
- \[19\]J\. Liu, A\. Gupta, C\. Ooi, and Y\. Ong\(2024\)ExTrEMO: transfer evolutionary multiobjective optimization with proof of faster convergence\.IEEE Trans\. on Evol\. Comput\.\(\),pp\. 1–1\.External Links:[Document](https://dx.doi.org/10.1109/TEVC.2023.3349313)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p4.2),[§II\-B](https://arxiv.org/html/2609.11228#S2.SS2.p1.1),[§III\-B4](https://arxiv.org/html/2609.11228#S3.SS2.SSS4.p1.3),[§IV](https://arxiv.org/html/2609.11228#S4.p1.1)\.
- \[20\]J\. Liu, A\. Gupta, and Y\. Ong\(2024\)Bayesian inverse transfer in evolutionary multiobjective optimization\.ACM Trans\. Evol\. Learn\. Optim\.4\(4\)\.External Links:ISSN 2688\-299X,[Document](https://dx.doi.org/10.1145/3674152)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p4.2),[§II\-B](https://arxiv.org/html/2609.11228#S2.SS2.p1.1),[§III\-B3](https://arxiv.org/html/2609.11228#S3.SS2.SSS3.p1.1),[§IV](https://arxiv.org/html/2609.11228#S4.p1.1)\.
- \[21\]T\. Wei, J\. Liu, A\. Gupta, P\. S\. Tan, and Y\. Ong\(2024\)Bayesian forward\-inverse transfer for multiobjective optimization\.InParallel Problem Solving from Nature – PPSN XVIII,Cham,pp\. 135–152\.External Links:ISBN 978\-3\-031\-70085\-9Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p4.2),[§I](https://arxiv.org/html/2609.11228#S1.p5.1),[§II\-A](https://arxiv.org/html/2609.11228#S2.SS1.p1.1),[§III\-B](https://arxiv.org/html/2609.11228#S3.SS2.p1.1),[§IV](https://arxiv.org/html/2609.11228#S4.p1.1)\.
- \[22\]X\. Xue, C\. Yang, L\. Feng, K\. Zhang, L\. Song, and K\. C\. Tan\(2024\)Solution transfer in evolutionary optimization: an empirical study on sequential transfer\.IEEE Trans\. on Evol\. Comput\.28\(6\),pp\. 1776–1793\.External Links:[Document](https://dx.doi.org/10.1109/TEVC.2023.3339506)Cited by:[§I](https://arxiv.org/html/2609.11228#S1.p4.2)\.
- \[23\]Y\. Yuan, Y\. Ong, L\. Feng, A\. K\. Qin, A\. Gupta, B\. Da, Q\. Zhang, K\. C\. Tan, Y\. Jin, and H\. Ishibuchi\(2017\)Evolutionary multitasking for multiobjective continuous optimization: benchmark problems, performance metrics and baseline results\.arXiv preprint arXiv:1706\.02766\.Cited by:[§II\-A](https://arxiv.org/html/2609.11228#S2.SS1.p1.1),[§IV\-A](https://arxiv.org/html/2609.11228#S4.SS1.p1.1),[§IV\-A](https://arxiv.org/html/2609.11228#S4.SS1.p2.1)\.
- \[24\]X\. Wu, S\. Liu, Q\. Lin, K\. Chen Tan, and V\. C\. M\. Leung\(2025\)Evolutionary multitasking with adaptive knowledge transfer for expensive multiobjective optimization\.IEEE Trans\. on Evol\. Comput\.29\(6\),pp\. 2537–2551\.External Links:[Document](https://dx.doi.org/10.1109/TEVC.2024.3494039)Cited by:[§II\-A](https://arxiv.org/html/2609.11228#S2.SS1.p1.1)\.
- \[25\]J\. Liu, W\. Liu, J\. T\. W\. En, C\. Chen, P\. S\. Tan, and Y\. Ong\(2025\)Optimal transport\-based distributional pairing in transfer multiobjective optimization\.IEEE Trans\. on Evol\. Comput\.\(\),pp\. 1–1\.External Links:[Document](https://dx.doi.org/10.1109/TEVC.2025.3624132)Cited by:[§II\-B](https://arxiv.org/html/2609.11228#S2.SS2.p1.1)\.
- \[26\]H\. Wu, T\. Wei, J\. Liu, M\. Xu, Y\. Ong, and Y\. Jin\(2025\)Convergence of expensive multi\-objective optimizers: from parego to extremo\.In2025 IEEE Congress on Evol\. Comput\. \(CEC\),Vol\.,pp\. 1–8\.External Links:[Document](https://dx.doi.org/10.1109/CEC65147.2025.11043126)Cited by:[§II\-B](https://arxiv.org/html/2609.11228#S2.SS2.p1.1),[§III\-B4](https://arxiv.org/html/2609.11228#S3.SS2.SSS4.p1.3),[§III\-C1](https://arxiv.org/html/2609.11228#S3.SS3.SSS1.p1.1)\.
- \[27\]E\. V\. Bonilla, K\. Chai, and C\. Williams\(2007\)Multi\-task gaussian process prediction\.InAdvances in Neural Information Processing Systems,J\. Platt, D\. Koller, Y\. Singer, and S\. Roweis \(Eds\.\),Vol\.20,pp\.\.Cited by:[§III\-B2](https://arxiv.org/html/2609.11228#S3.SS2.SSS2.p1.2),[§III\-C1](https://arxiv.org/html/2609.11228#S3.SS3.SSS1.p2.1)\.
- \[28\]M\. Seeger\(2004\)Gaussian processes for machine learning\.International journal of neural systems14\(02\),pp\. 69–106\.Cited by:[§III\-B3](https://arxiv.org/html/2609.11228#S3.SS2.SSS3.p2.1)\.
- \[29\]B\. Da, Y\. Ong, A\. Gupta, L\. Feng, and H\. Liu\(2019\)Fast transfer gaussian process regression with large\-scale sources\.Knowledge\-Based Systems165,pp\. 208–218\.External Links:ISSN 0950\-7051,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.knosys.2018.11.029)Cited by:[§III\-B4](https://arxiv.org/html/2609.11228#S3.SS2.SSS4.p1.2),[§III\-C1](https://arxiv.org/html/2609.11228#S3.SS3.SSS1.p2.1)\.
- \[30\]D\. A\. Van Veldhuizen and G\. B\. Lamont\(1998\)Multiobjective evolutionary algorithm research: a history and analysis\.Technical reportCiteseer\.Cited by:[§IV\-A](https://arxiv.org/html/2609.11228#S4.SS1.p2.1)\.
- \[31\]F\. Pfisterer, L\. Schneider, J\. Moosbauer, M\. Binder, and B\. Bischl\(2022\)YAHPO gym \- an efficient multi\-objective multi\-fidelity benchmark for hyperparameter optimization\.InProceedings of the First International Conference on Automated Machine Learning,Proceedings of Machine Learning Research, Vol\.188,pp\. 3/1–39\.Cited by:[2nd item](https://arxiv.org/html/2609.11228#S4.I1.i2.p1.1)\.