The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs Jiajia Tang,1 Sizhe Yuen,1 Francisco Gomez Medina,1 Yali Du, 1,3 Adam Sobey1,2 1
2
The Alan Turing Institute, London, UK University of Southampton, Southampton, UK 3 King’s College London, London, UK
arXiv:2607.29601v1 [cs.LG] 31 Jul 2026
Abstract Parameter-Efficient Fine-Tuning (PEFT) commonly adapts large language models using a single shared Low-Rank Adapter (LoRA). This shared optimization space often suffers from interference when adapting heterogeneous task sequences, leading to poor transfer and catastrophic forgetting. Existing approaches mainly improve adapter expressiveness by increasing parameter capacity or composing multiple adapters, yet they still rely on a shared optimization path. In this paper, we propose an optimization-path organization framework for parameter-efficient fine-tuning of large language models, implemented as an automatic multi-policy PEFT architecture. Specifically, optimization-compatible adaptation paths are automatically organized through task grouping and task sequencing under a fixed parameter budget. The organized optimization paths are implemented as independent Quantized Low-Rank Adapters (QLoRA), enabling heterogeneous tasks to be optimized in decoupled adaptation spaces while preserving positive transfer among compatible tasks. Experiments on the TRACE benchmark demonstrate that performance consistently improves from conventional single-policy PEFT to multi-policy PEFT, with the proposed automatic multi-policy framework achieving the best performance of 44.78 under the same trainable capacity. This suggests that optimization-path organization is more effective than simply increasing adapter capacity for heterogeneous parameter-efficient fine-tuning.
1
Introduction
Parameter-Efficient Fine-Tuning (PEFT) (Ding et al. 2023) has become the dominant paradigm for adapting large language models (LLMs) to downstream tasks. Representative methods such as Low-Rank Adaptation (LoRA) and Quantized Low-Rank Adaptation (QLoRA) demonstrate that lightweight adaptation can achieve competitive downstream performance while maintaining high efficiency (Houlsby et al. 2019; Hu et al. 2022; Dettmers et al. 2023). Despite their architectural differences, most existing PEFT methods rely on a common assumption: a single shared low-rank adapter is expected to accommodate all target tasks (Wang et al. 2023b). This assumption is reasonable when tasks have similar characteristics, allowing the shared adaptation space to promote positive transfer. However, under heterogeneous task sequences during parameter-efficient fine-tuning, tasks often differ substantially in domain, supervision signals, reasoning patterns, capability requirements, and output formats. Such heterogeneity
frequently leads to inefficient learning (Michieli and Ozay 2024). Consequently, forcing all tasks to update the same lowrank adaptation space introduces optimization interference, resulting in negative transfer and catastrophic forgetting. A few approaches have therefore focused on assigning a single LoRA to each task (Yang et al. 2025; Zhang et al. 2025) but these approaches will lose efficiency from lack of transfer learning and increase in size for large numbers of tasks. Existing studies improve adapter expressiveness through higher-rank adapters, richer parameterizations, or multiadapter architectures such as adapter fusion, hypernetworkbased generation, and modular composition.(Pfeiffer et al. 2021, 2020; Karimi Mahabadi et al. 2021; Karimi Mahabadi, Henderson, and Ruder 2021). Although these approaches improve representation capability, they largely preserve a shared optimization mechanism, where heterogeneous tasks remain coupled through either a common parameter space or a shared adapter generation process. Consequently, they alleviate optimization interference without explicitly organizing heterogeneous optimization paths. The key challenge is not how many trainable parameters are used, but how heterogeneous tasks are organized before training. That is, the task grouping and task sequence should be taken into account, which is essential for constructing optimizationcompatible adaptation paths under a fixed parameter budget. Unlike conventional multi-task learning, where tasks are jointly optimized, we consider parameter-efficient fine-tuning over heterogeneous task sequences, in which tasks are learned sequentially. To address this challenge, we propose an optimization-path organization framework for parameter-efficient fine-tuning of large language models, implemented using a multi-policy PEFT architecture. We take our inspiration from multi-policy approaches in Reinforcement Learning which improve the efficiency of learning (Bossens and Sobey 2024; Birkbeck et al. 2024). The framework automatically organizes optimizationcompatible adaptation paths through two consecutive stages. First, heterogeneous tasks are grouped according to optimization compatibility, reducing gradient conflicts while preserving capability similarity. Each task group is assigned an independent optimization path, implemented as a separate policy for parameter-efficient fine-tuning. Second, a task sequence is automatically constructed within each policy by organizing the learning sequence to minimize transition costs
while reducing catastrophic forgetting. Consequently, task grouping and task sequencing encourage positive transfer among compatible tasks and produce smoother optimization trajectories throughout sequential fine-tuning. Unlike existing modular PEFT methods, which mainly improve adapter architectures or parameter composition, our framework explicitly organizes optimization paths before adaptation begins. Multiple independent QLoRA adapters serve as the implementation mechanism of the organized optimization paths, reducing optimization interference without introducing additional routing modules, fusion layers, or hypernetworks.
2 2.1
Related Work
Parameter-Efficient Fine-Tuning
Parameter-efficient fine-tuning (PEFT) aims to adapt large pretrained models by training only a lightweight set of additional parameters while keeping the backbone model largely frozen. Early adapter-based methods insert small trainable modules into Transformer layers and demonstrate that strong transfer performance can be achieved with very limited parameter overhead (Houlsby et al. 2019). Subsequently, LoRA formulates parameter updates as low-rank decompositions, and QLoRA further combines low-rank adaptation with 4-bit quantization, making efficient adaptation feasible for much larger LLMs without sacrificing downstream performance (Hu et al. 2022; Dettmers et al. 2023). Prompt-based PEFT methods, including Prefix-Tuning, Prompt Tuning, and P-Tuning v2, investigate lightweight adaptation through trainable prompts rather than trainable adapters (Li and Liang 2021; Lester, Al-Rfou, and Constant 2021; Liu et al. 2022b). Other PEFT variants, such as IA3 , AdaLoRA, and UniPELT, further improve adaptation flexibility through multiplicative rescaling, adaptive rank allocation, or unified combinations of PEFT modules (Liu et al. 2022a; Zhang et al. 2023; Mao et al. 2022). More recently, comprehensive surveys have summarized the rapid development of PEFT techniques, covering adapter tuning, prompt tuning, low-rank adaptation, and related lightweight optimization strategies for large language models (Han et al. 2024; Wang et al. 2024a). Although these methods differ substantially in implementation, most assume that all target tasks are optimized through a shared adaptation path. This shared optimization space works well for relatively homogeneous tasks, but becomes susceptible to optimization interference under heterogeneous task sequences. Instead of improving adapter expressiveness, our work automatically organizes optimization paths through joint task grouping and task sequencing before adaptation begins.
2.2
Multi-Adapter PEFT
Multiple adapters or richer composition mechanisms are a common approach to improving performance. AdapterFusion combines multiple pretrained task adapters through attention-based fusion during inference (Pfeiffer et al. 2021). MAD-X composes language adapters and task adapters to support modular multilingual transfer (Pfeiffer et al. 2020). HyperFormer generates task-aware adapters through shared hypernetworks, while Compacter improves adapter efficiency
through structured parameter sharing (Karimi Mahabadi et al. 2021; Karimi Mahabadi, Henderson, and Ruder 2021). More recently, AdapterSoup explores parameter aggregation across multiple adapters to improve generalization (Chronopoulou et al. 2023). Modular and expert-style PEFT architectures have also attracted increasing attention. MixLoRA introduces a LoRA-based mixture-of-experts architecture for efficient multi-task adaptation (Li et al. 2024), while SLIM proposes a soft LoRA mixture mechanism that improves adaptation performance while reducing forgetting during continual finetuning (Han et al. 2025). These studies demonstrate that adapter organization and modular design can substantially influence downstream adaptation performance. Optimization interference has also been extensively studied in continual learning. Gradient-based methods, including PCGrad and Gradient Vaccine, alleviate optimization conflicts by modifying or regularizing task gradients during optimization (Yu et al. 2020; Wang et al. 2020). Parameter-isolation approaches, such as Progressive Networks and PackNet, reduce interference by assigning separate subnetworks or parameter subsets to different tasks (Rusu et al. 2016; Mallya and Lazebnik 2018). More recently, MoCL proposes a modular continual learning framework for language models, showing that structured parameter organization improves knowledge retention while maintaining transfer across tasks (Wang et al. 2024b). Collectively, these studies suggest that optimization behavior is strongly influenced by how trainable parameters are organized. Unlike existing methods that improve adapter expressiveness, manipulate gradients, or isolate parameters, our work addresses the complementary problem of organizing heterogeneous tasks before optimization. Our framework automatically organizes optimization paths by assigning compatible tasks to independent QLoRA adapters and constructing an effective task sequence within each adapter.
2.3
Sequential Fine-Tuning for Large Language Models
Sequential fine-tuning has been studied in several closely related continual-learning settings, including continual instruction tuning, sequential domain adaptation, and heterogeneous task streams. TRACE provides a representative benchmark for studying continual learning across eight highly heterogeneous language tasks (Wang et al. 2023b). More recent benchmarks further extend this direction. For example, CoIN investigates continual instruction tuning for multimodal large language models and highlights the challenge of maintaining performance across sequential instruction streams (Chen et al. 2024). InsCL proposes a data-efficient continual learning framework for instruction tuning and demonstrates that catastrophic forgetting remains a major challenge even under parameterefficient adaptation (Wang et al. 2024c). Furthermore, recent surveys provide comprehensive reviews of continual learning for large language models and parameter-efficient continual fine-tuning, covering continual pre-training, instruction tuning, domain adaptation, and task-specific continual learning (Shi et al. 2025; Coleman et al. 2025). Most continual learning studies focus on reducing forgetting during sequential adaptation. In contrast, our work
studies how heterogeneous tasks should be organized before parameter-efficient fine-tuning. Rather than introducing a new continual learning mechanism, our framework provides a complementary perspective based on optimization-path organization for sequential fine-tuning.
3 3.1
Automated Task Sequencing for Fine-Tuning of Multi-Policy LLMs
We consider parameter-efficient fine-tuning of a pretrained large language model fθ , where the backbone parameters θ are frozen and only a small set of adaptation parameters are trained. We focus on a pre-deployment setting in which the target task collection is available before fine-tuning begins. Let T = {T1 , T2 , . . . , TN } denote a heterogeneous sequential task collection, where each task Ti is associated with a data distribution Di and a task-specific loss ℓi . In standard LoRAstyle PEFT, all tasks share one low-rank update, A ∈ Rd×r ,
B ∈ Rr×d ,
r ≪ d.
(1)
The corresponding shared-adapter objective can be written as, Lshared =
N X
E(x,y)∼Di [ℓi (fθ+∆θ (x), y)] .
(2)
i=1
Let gi = ∇∆θ ℓi denote the gradient induced by task Ti . If two tasks produce weakly aligned or conflicting gradients, their updates compete within the same trainable subspace. The shared adapter is consequently forced to follow a compromised optimization trajectory, which may be suboptimal for multiple tasks and leads to weak transfer or catastrophic forgetting. The bottleneck is therefore not merely insufficient adapter capacity. Even when the adapter rank is increased, all heterogeneous tasks remain constrained to co-update the same trainable space and follow the same coupled optimization path. Rather than simply enlarging this shared space, we reorganize the optimization process. Our goal is to automatically organize optimization paths for heterogeneous task sequences during parameter-efficient fine-tuning by separating strongly conflicting tasks while preserving useful transfer among compatible ones. These organized optimization paths are implemented as multiple independent policies.
3.2
Automatic Task Grouping and Sequencing
We propose an optimization-path organization framework for parameter-efficient fine-tuning over heterogeneous task sequences. The framework is implemented through a multipolicy PEFT architecture, where optimization-compatible task groups and task sequences are automatically constructed. Formally, the task collection is partitioned into K disjoint groups:
T =
K [ k=1
T (k) ,
T (i) ∩ T (j) = ∅,
i ̸= j.
B (k) ∈ Rrk ×d . (4) Given an input x from a task assigned to T (k) , the corresponding policy is activated: ∆θ(k) = A(k) B (k) ,
A(k) ∈ Rd×rk ,
f (x) = fθ+∆θ(k) (x).
Problem Formulation
∆θ = AB,
For each task group T (k) , each organized optimization path is instantiated as an independent QLoRA adapter:
(3)
(5)
In QLoRA, there are two components: a frozen backbone and a set of trainable parameters that adapt the backbone based on the fine-tuning. In the multi-policy approach, each of the policies is initialized from the same frozen backbone, whereas the trainable adapter parameters are developed separately for each policy using the tasks grouped for that policy. As shown in Figure 1, our process for developing fine-tuned policies happens in two steps: Stage 1 determines how optimization paths are allocated by constructing optimization-compatible task groups. Stage 2 determines how each optimization path evolves through task sequencing. Stage 1: Automatic Task Grouping This procedure attends to allocate optimization paths by grouping optimizationcompatible tasks into independent adaptation spaces. These adaptation spaces are implemented as separate policies. The objective is to construct task groups with low internal optimization conflict, similar capability requirements, and compatible output behaviours, so that tasks within each policy are more likely to benefit from positive transfer while minimizing optimization interference. These groups preserve positive transfer while separating strongly incompatible tasks into different adaptation spaces. We first characterize each task according to its induced optimization behaviour. For each task Ti , gradients are collected from a temporary shared model over multiple mini-batches. Since the raw gradient space is extremely high-dimensional, principal component analysis (PCA) is applied to obtain a compact task representation zigrad while preserving the dominant variation among task-specific update directions. This parameter-free projection enables efficient pairwise task comparison. The pairwise gradient distance is then defined as, grad Dij = d zigrad , zjgrad , (6) where d(·, ·) denotes the normalized distance between task representations. This distance directly estimates whether two tasks can safely co-update the same low-rank adaptation space. Note that the above gradient compatibility alone is insufficient for grouping heterogeneous language tasks. Two tasks may exhibit similar local gradient directions while requiring substantially different capabilities or output behaviors. Conversely, surface-level task similarity does not necessarily imply compatible optimization dynamics. Therefore a complementary behavior representation is constructed for each task. Specifically, dataset-level statistics are extracted to form a normalized behavior feature vector related to capability requirements and output structure, including prompt length, answer length, answer-to-prompt ratio, numeric-output tendency, multiple-choice tendency, short-answer tendency, reasoning-oriented prompts, and generation-oriented prompts.
Group-Wise Sequential Fine-Tuning
Stage 1 Automatic Task Grouping fused unified task distance
Heterogeneous Sequential Tasks
Group 1
Group 2
optimization-compatible tasks
optimization-compatible tasks
QLoRA Adapter 1
grouping
clustering pairwise task gradient distance
balanced
pairwise task behavior distance
Quantized
weights
clustering
Stage 2 Automatic Task Sequencing QLoRA Adapter 2
directional transition
adjacent transition
head reward :
task sequence 1
terminal cost :
further optimize global curriculum
...
...
Quantized
weights
task sequence
adjacent transition cost :
composite path objective :
local curriculum
global curriculum
Figure 1: Our approach automatically organizes optimization paths through task grouping and task sequencing. These organized paths are implemented using independent QLoRA policies, reducing interference while preserving positive transfer. This indeed provides a new design perspective for parameter-efficient fine-tuning over heterogeneous task sequences. Let zibeh denote the resulting behavior representation. The corresponding pairwise behavior distance is, beh Dij = d zibeh , zjbeh . (7) The above distance captures structural differences that may not be fully reflected by instantaneous gradients, particularly differences in required capability and output format. The two compatibility signals provide complementary information. Gradient distance directly captures optimization conflict, whereas behavior distance captures differences in capability requirements and output structures. We therefore fuse them into a unified task distance: grad beh Dij = (1 − λ)Dij + λDij ,
(8)
where λ ∈ [0, 1] controls the contribution of behavior-level compatibility. Based on the fused distance matrix D, we construct K task groups through balanced clustering: G ∗ = arg min G
K X
Stage 2: Automatic Task Sequencing Stage 2 further organizes how each optimization path evolves during sequential fine-tuning within each fixed group. Wang (Wang et al. 2023b) demonstrate that optimizing the order in which tasks are presented can improve performance. However, their approach relies on manual search, which is impractical for fine-tuning because multiple permutations must be evaluated to identify an effective trajectory. We therefore develop an automated approach for constructing the task sequence. For a task group T (k) , we search for a task sequence, (k) (k) , (10) π (k) = Tπ(k) , T , . . . , T π2 π (k) 1 |T
|
where π (k) defines the fine-tuning path followed by the k-th policy. A basic requirement for an effective task sequence is to avoid abrupt transitions between highly incompatible neighboring tasks. We therefore define the adjacent transition cost as, (k)
X
Dij .
(9)
k=1 Ti ,Tj ∈T (k)
In the two-policy setting used in this work, we impose a maximum group-size constraint, resulting in two four-task groups on TRACE. This prevents one policy from absorbing most tasks while another serves only a small subset, thereby maintaining a more comparable task load under the fixed perpolicy capacity. The constraint could be relaxed for larger task collections. Consequently, Stage 1 constructs optimizationcompatible adaptation paths whose internal tasks exhibit relatively low gradient conflict and similar capability requirements. Strongly incompatible tasks are separated into different adaptation spaces, while compatible tasks remain together to preserve positive transfer.
|T X|−1 (k) Ctrans π = Dπt ,πt+1 .
(11)
t=1
Minimizing this term ensures that neighboring tasks have compatible optimization characteristics, thereby reducing the immediate cost of moving from one task to the next. However, minimizing adjacent distance alone is insufficient. Sequential fine-tuning is directional, the effect of training Ti followed by Tj is not necessarily equivalent to training Tj followed by Ti . Moreover, the first and final positions have distinct roles in the learning trajectory. A robust task sequence should consider not only local similarity but also the order in which tasks are learned. Specifically, for the first task analysis, a head reward Rhead π (k) is introduced to encourage more complex tasks
to be learned earlier, allowing the optimization path to establish complex capabilities before subsequent fine-tuning and helping reduce overall forgetting. Conversely, for the final task analysis, a terminal cost Ctail π (k) is introduced to discourage selecting a task that may cause strong interference with previously learned tasks. This improves the stability of the final policy state. Directional transitioneffects are modelled through an interference cost Cdir π (k) and a local transfer reward Rtransfer π (k) , encouraging consecutive tasks to exhibit low optimization conflict while maintaining positive knowledge transfer. However, a sequence composed only of locally compatible transitions may still form a poor global optimization path. We therefore further encourage smooth progression in task capability requirements Cprog π (k) . This term discourages abrupt global changes in task complexity and promotes a coherent capability progression across the complete fine-tuning path. Combining these objectives, the final curriculum is obtained by minimizing the following composite path objective: π (k)∗ = arg min C π (k) , (12) (k) π
where C π (k) = Ctrans + Ctail + Cdir + Cprog
(13)
− Rhead − Rtransfer . The objective terms are combined using fixed weights. The resulting optimization path jointly considers five aspects of sequential fine-tuning: low transition cost, head-task protection, tail-task safety, local positive transfer, and global capability progression. All sequencing signals are computed automatically from the optimization conflict matrix and behavioural statistics obtained during grouping, and are subsequently used to score candidate task sequences.
3.3
Group-Wise Training
After automatic grouping and sequencing, each policy is trained independently using its own QLoRA adapter. For group T (k) with the selected sequence π (k)∗ , the corresponding adapter is sequentially optimized along the task sequence: Tπ(k) → Tπ(k) → · · · → Tπ(k) (k) . 1 2 |T
|
(14)
The group-wise objective can be written as, Lmulti =
K X X
E(x,y)∼Di ℓi fθ+∆θ(k) (x), y .
k=1 Ti ∈T (k)
(15) To ensure that the advantage of the proposed framework does not simply result from using more trainable parameters, we explicitly control the total trainable capacity. Let r denote the rank of a single shared-adapter baseline. The multi-policy PK configuration satisfies k=1 rk = r. Consequently, any performance improvement under matched trainable capacity reflects the benefit of optimization-path organization rather than simple parameter expansion.
4 4.1
Fine-Tuning Open-Source LLMs on TRACE dataset
Benchmark and Experimental Setup
We evaluate the proposed method on TRACE, a continuallearning benchmark for large language models (Wang et al. 2023b) due to the wide range of different tasks for fine-tuning. TRACE contains eight heterogeneous tasks: C-STANCE, FOMC, MeetingBank, Py150, ScienceQA, NumGLUE-cm, NumGLUE-ds, and 20Minuten. While TRACE adopts LoRA, we use QLoRA through all experiments for its improved memory efficiency. Experiments are conducted on the same two 7B-scale aligned chat models, namely LLaMA-2-7BChat and Vicuna-7B-V1.5. Although Vicuna is built upon LLaMA-2, it is instruction-tuned differently, providing an additional backbone to evaluate the robustness of our work. For each backbone, the main multi-policy setting contains two independent rank-64 QLoRA adapters. The total effective rank is 128. Different grouping and sequencing strategies use the same two-adapter architecture and trainable budget, allowing the effects of task grouping and task sequencing to be evaluated without changing the model capacity. We report Overall Performance (OP), i.e., the average performance over all tasks after fine tuning, and Backward Transfer (BWT) used to measure the average effect of later task training on previously learned tasks. Following the experimental setting of TRACE, each task uses 5,000 training samples. The remaining training hyperparameters are selected for our work: all tasks are trained for 3 epochs except for C-STANCE and NumGLUE-ds which are trained for 5 epochs, using a batch size of 2 and a learning rate of 1 × 10−5 .
4.2
Comparison with Baseline Methods
O-LoRA (Wang et al. 2023a) is included as a parameterisolation multi-adapter PEFT baseline, assigning one independent LoRA subspace to each individual task, whereas our framework automatically groups optimization-compatible tasks into shared policy-specific adaptation spaces. This makes O-LoRA a particularly appropriate baseline for isolating the effect of different optimization-path organization strategies. To ensure a fair comparison under the same total low-rank budget, O-LoRA allocates a rank-16 task-specific block to each of the eight tasks, resulting in a total rank of 128, while our framework uses two rank-64 policy-specific adapters, also resulting in a total rank of 128. Under this capacity-matched setting, O-LoRA achieves an OP of 30.76 and a BWT of -0.023 on LLaMA-2-7B-Chat, which is substantially lower than our automatic framework implemented with multi-policy PEFT. This indicates that, under the same total trainable capacity, organizing compatible tasks into shared policy-specific adaptation spaces is more effective than allocating an independent small subspace to each individual task. As shown in Table 1, we evaluate a capacity-matched singlepolicy baseline using 1 shared rank-128 QLoRA adapter. Compared with this baseline, the proposed automatic multi-policy framework achieves higher OP on both backbones under the same total trainable capacity, indicating that organizing optimization paths is more effective than simply increasing
Table 1: Comparison of different optimization-path organization strategies under the same trainable capacity on the TRACE benchmark using the LLaMA-2-7B-Chat and Vicuna-7B-V1.5 backbones. O-LoRA is included as a representative parameterisolation multi-adapter PEFT baseline. * denotes results reproduced in our experimental setting. Backbone
Method
Policy
Group Strategy
Sequence Strategy
Setting
OP ↑
BWT ↑
Vicuna-7B-V1.5 Vicuna-7B-V1.5 Vicuna-7B-V1.5 Vicuna-7B-V1.5 Vicuna-7B-V1.5
QLoRASeqFT QLoRASeqFT QLoRASeqFT QLoRASeqFT QLoRASeqFT
Multi-policy Multi-policy Single-policy Multi-policy Multi-policy
Random Auto N/A Manual Auto
Random Random Manual Manual Auto
2 rank-64 QLoRA 2 rank-64 QLoRA 1 shared rank-128 QLoRA 2 rank-64 QLoRA 2 rank-64 QLoRA
36.49 38.44 39.50 39.45 41.14
-0.106 -0.083 -0.050 -0.040 -0.040
LLaMA-2-7B-Chat LLaMA-2-7B-Chat LLaMA-2-7B-Chat LLaMA-2-7B-Chat LLaMA-2-7B-Chat LLaMA-2-7B-Chat
O-LoRASeqFT* QLoRASeqFT QLoRASeqFT QLoRASeqFT QLoRASeqFT QLoRASeqFT
Multi-policy Multi-policy Single-policy Multi-policy Multi-policy
Random Auto N/A Manual Auto
Random Random Manual Manual Auto
2 rank-64 QLoRA 2 rank-64 QLoRA 1 shared rank-128 QLoRA 2 rank-64 QLoRA 2 rank-64 QLoRA
30.76 36.94 40.75 42.12 44.53 44.78
-0.023 -0.092 -0.027 -0.041 0.012 0.013
the capacity of a shared adapter. We further compare four grouping-and-sequencing settings under the same two-adapter architecture and trainable budget. Random grouping with random sequencing represents a multi-policy baseline without informed task organization. Automatic grouping with random sequencing isolates the effect of the proposed grouping strategy. The manually designed grouping and task sequence represent an expert-designed solution, where task assignment and learning sequence are selected using prior knowledge and manual analysis. The complete framework combines automatic grouping with automatic task sequencing, with the aim of achieving optimization-path organization comparable to or better than the expert-designed solution without requiring manual intervention. On LLaMA-2-7B-Chat, we can find that replacing random grouping with automatic grouping improves OP to 40.75 and BWT to -0.027, even though the task sequence remains random. This shows that automatically assigning optimization-compatible tasks to the same policy already provides a substantial improvement in both final performance and knowledge retention. Compared to the expert-designed grouping and sequence, the complete automatic framework obtains the best result, reaching an OP of 44.78 and a BWT of 0.013. This shows that the automatic framework reaches and slightly exceeds expert-level optimization-path organization without requiring prior knowledge of the task relationships or manual search over possible groupings and sequences. A similar overall trend is observed on Vicuna-7B-V1.5. Random grouping and random sequencing achieve an OP of 36.49 and a BWT of -0.106, while the expert-designed setting improves OP to 39.45 and BWT to -0.040. The complete automatic framework achieves the best OP of 41.14 while maintaining the same BWT of -0.040 as the expert-designed solution. Consequently, the advantage of the proposed framework does not come from introducing multiple adapters themselves, but from organizing optimization paths through automatic task grouping and task sequencing.
4.3
Analysis of Optimization-Path Organization
In this part, we analyze three components of optimizationpath organization: task grouping, rank allocation, and task
Figure 2: Visualization of the fused task-compatibility structure used for automatic grouping. Lower pairwise distances indicate more compatible optimization directions and behavior characteristics, providing the basis for constructing the two optimization-compatible policies.
sequencing. Task grouping. To evaluate the proposed grouping strategy, we compare it with two alternative task groupings. All settings use the same two-adapter architecture, the same rank allocation, and the same sequential fine-tuning protocol. Only the task assignments are changed. As shown in Table 3, compared to the two alternative groupings, the proposed grouping achieves the best OP of 44.78 and the best BWT of 0.013. This indicates that transfer learning has been effective, with methods such as MTL-LoRA (Yang et al. 2025) and MoRE (Zhang et al. 2025) having a maximum BWT of 0. Since all other settings remain unchanged, the improvement comes from organizing optimization-compatible tasks into the same policy rather than simply introducing multiple adapters. Moreover, Figure 2 is used to visualize the fused task-compatibility matrix used in Stage 1, which combines gradient conflict and behavior differences to organize optimization-compatible
Table 2: Task-level analysis of destructive interference under different task grouping and sequencing strategies. Lower BWT indicates stronger destructive interference. “–” denotes the last task within a policy, for which BWT is undefined. Grouping / sequencing Random / Random Auto / Random Auto / Auto
C-STANCE
FOMC
ScienceQA
NumGLUE-ds
MeetingBank
Py150
NumGLUE-cm
20Minuten
-0.002 0.000 0.009
– 0.109 0.107
-0.046 0.052 0.135
-0.030 -0.012 –
-0.357 -0.048 -0.171
-0.018 – -0.014
-0.256 -0.231 0.000
– -0.004 –
Table 3: Ablation on task grouping strategy. “Auto Group” denotes the task-compatibility-aware grouping used in the main experiments, which is compared with two alternative groupings derived from the fused task-compatibility structure. Grouping
Group 1
OP ↑ BWT ↑
Group 2
Alternative 1 {C-STANCE, FOMC, NumGLUE-ds, 20Minuten} {MeetingBank, Py150, NumGLUE-cm, ScienceQA} 41.29 -0.017 Alternative 2 {C-STANCE, FOMC, NumGLUE-ds, MeetingBank} {Py150, ScienceQA, NumGLUE-cm, 20Minuten} 41.96 -0.003 Auto Group {C-STANCE, FOMC, ScienceQA, NumGLUE-ds} {MeetingBank, Py150, NumGLUE-cm, 20Minuten} 44.78 0.013
tasks into two policies. Figure 2 and Table 3 demonstrate the necessity of task grouping for effective parameter-efficient fine-tuning over heterogeneous task sequences. Rank allocation. We further analyze how the fixed trainable budget should be allocated across the two policies. The total effective rank is fixed at 128, while only the rank allocation between the two adapters is changed. We compare three settings: 32 + 96, 64 + 64, and 96 + 32. As shown in Table 4, the balanced 64 + 64 allocation achieves the best OP and BWT. This indicates that the performance improvement does not depend on allocating more trainable parameters to one policy. Instead, balancing the trainable capacity across the two policies provides a more effective use of the fixed PEFT budget. Task sequencing. The task grouping, rank allocation, and training configuration remain unchanged, while only the task sequence within each policy is changed. As shown in Table 5, the automatic sequence achieves the highest OP and BWT for both groups, while both random sequences reduce OP and further increase forgetting. These results indicate that task grouping alone is insufficient. Organizing the optimization trajectory within each policy further improves sequential fine-tuning performance and reduces catastrophic forgetting. Overall, these results demonstrate that the effectiveness of the proposed framework depends on the above components. Meaningful task grouping organizes optimization-compatible tasks into the same adaptation space, balanced rank allocation makes effective use of the fixed trainable budget, and automatic task sequencing further improves fine-tuning performance and knowledge retention. Table 4: Ablation on rank allocation across policies under a fixed total trainable budget.The total LoRA rank is fixed at 128 for all settings. Rank allocation (Group 1 + Group 2) 32 + 96 96 + 32 64 + 64
Total rank
OP ↑
BWT ↑
128 128 128
41.61 42.58 44.78
-0.018 -0.011 0.013
Table 5: Ablation on task sequencing strategy. Task grouping is fixed using the optimization-compatible groups generated by our method. sequence Random sequence 1 Random sequence 2 Auto sequence
4.4
Group1-OP
Group1-BWT
Group2-OP
Group2-BWT
54.86 54.02 55.81
0.054 0.049 0.084
25.89 27.75 33.90
-0.119 -0.094 -0.062
Analysis of Destructive Interference
Table 2 provides a task-level analysis of destructive interference under different optimization-path organization strategies. Under random grouping and sequencing, MeetingBank and NumGLUE-cm show the largest negative backward transfer, indicating severe interference from later tasks. Automatic grouping greatly reduces interference on MeetingBank, while automatic sequencing further improves ScienceQA and removes forgetting on NumGLUE-cm. Overall, our method does not eliminate interference for every task, but improves the overall distribution of transfer effects across tasks, leading to stronger positive transfer among compatible tasks.
5
Conclusion
In this paper, we proposed an optimization-path organization framework for parameter-efficient fine-tuning of large language models, implemented using a multi-policy PEFT architecture. Instead of forcing heterogeneous tasks to share a single low-rank adaptation space, our framework automatically organizes multiple optimization-compatible adaptation paths through task grouping and task sequencing. The approach separates incompatible tasks into independent adaptation spaces, reducing interference while preserving positive transfer among compatible tasks. This provides a new design perspective for parameter-efficient fine-tuning over heterogeneous task sequences, where optimization-path organization can be more effective than simply increasing adapter capacity.
References Birkbeck, J.; Sobey, A.; Cerutti, F.; Flynn, K. H. H.; and Norman, T. J. 2024. CHIRPs: Change-Induced Regret Proxy
metrics for Lifelong Reinforcement Learning. arXiv preprint arXiv:2409.03577. Bossens, D. M.; and Sobey, A. J. 2024. Lifetime policy reuse and the importance of task capacity. AI Communications, 37(1): 115–148. Chen, C.; Zhu, J.; Luo, X.; Shen, H.; Gao, L.; and Song, J. 2024. CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Model. arXiv preprint arXiv:2403.08350. Chronopoulou, A.; Peters, M.; Fraser, A.; and Dodge, J. 2023. AdapterSoup: Weight averaging to improve generalization of pretrained language models. In Findings of the Association for Computational Linguistics: EACL 2023, 2054–2063. Dubrovnik, Croatia. Coleman, E. N.; Quarantiello, L.; Liu, Z.; Yang, Q.; Mukherjee, S.; Hurtado, J.; and Lomonaco, V. 2025. ParameterEfficient Continual Fine-Tuning: A Survey. arXiv preprint arXiv:2504.13822. Dettmers, T.; Pagnoni, A.; Holtzman, A.; and Zettlemoyer, L. 2023. QLoRA: Efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems, volume 36, 10088–10115. Ding, N.; Qin, Y.; Yang, G.; Wei, F.; Yang, Z.; Su, Y.; Hu, S.; Chen, Y.; Chan, C.-M.; Chen, W.; et al. 2023. Parameterefficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5: 220–235. Han, J.; Du, L.; Du, H.; Zhou, X.; Wu, Y.; Zhang, Y.; Zheng, W.; and Han, D. 2025. SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 4792–4804. Albuquerque, New Mexico. Han, Z.; Gao, C.; Liu, J.; Zhang, J.; and Zhang, S. Q. 2024. Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey. arXiv preprint arXiv:2403.14608. Houlsby, N.; Giurgiu, A.; Jastrzebski, S.; Morrone, B.; de Laroussilhe, Q.; Gesmundo, A.; Attariyan, M.; and Gelly, S. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, 2790–2799. PMLR. Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, volume 1, 3. Karimi Mahabadi, R.; Henderson, J.; and Ruder, S. 2021. Compacter: Efficient low-rank hypercomplex adapter layers. In Advances in Neural Information Processing Systems, volume 34. Karimi Mahabadi, R.; Ruder, S.; Dehghani, M.; and Henderson, J. 2021. Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 565–576.
Lester, B.; Al-Rfou, R.; and Constant, N. 2021. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 3045–3059. Li, D.; Ma, Y.; Wang, N.; Ye, Z.; Cheng, Z.; Tang, Y.; Zhang, Y.; Duan, L.; Zuo, J.; Yang, C.; and Tang, M. 2024. MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRAbased Mixture of Experts. arXiv preprint arXiv:2404.15159. Li, X. L.; and Liang, P. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 4582– 4597. Liu, H.; Tam, D.; Muqeeth, M.; Mohta, N.; Huang, T.; Bansal, M.; and Raffel, C. A. 2022a. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35: 1950–1965. Liu, X.; Ji, K.; Fu, Y.; Tam, W. Y.; Du, Z.; Yang, Z.; and Tang, J. 2022b. P-Tuning v2: Prompt tuning can be comparable to finetuning universally across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 61–68. Mallya, A.; and Lazebnik, S. 2018. PackNet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7765–7773. Mao, Y.; K"oster, M.; Mohammadshahi, A.; Fernandez, R.; Ruder, S.; Henderson, J.; Merlo, P.; and Vulić, I. 2022. UniPELT: A unified framework for parameter-efficient language model tuning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 6253–6264. Michieli, U.; and Ozay, M. 2024. HOP to the Next Tasks and Domains for Continual Learning in NLP. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 14359–14369. Pfeiffer, J.; Kamath, A.; R"ucklé, A.; Cho, K.; and Gurevych, I. 2021. AdapterFusion: Non-destructive task composition for transfer learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 487–503. Pfeiffer, J.; Vulić, I.; Gurevych, I.; and Ruder, S. 2020. MADX: An adapter-based framework for multi-task cross-lingual transfer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 7654– 7673. Rusu, A. A.; Rabinowitz, N. C.; Desjardins, G.; Soyer, H.; Kirkpatrick, J.; Kavukcuoglu, K.; Pascanu, R.; and Hadsell, R. 2016. Progressive neural networks. arXiv preprint arXiv:1606.04671. Shi, H.; Xu, Z.; Wang, H.; Qin, W.; Wang, W.; Wang, Y.; Wang, Z.; Ebrahimi, S.; and Wang, H. 2025. Continual Learning of Large Language Models: A Comprehensive Survey. ACM Computing Surveys, 58(5).
Wang, L.; Chen, S.; Jiang, L.; Pan, S.; Cai, R.; Yang, S.; and Yang, F. 2024a. Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies. arXiv preprint arXiv:2410.19878. Wang, M.; Adel, H.; Lange, L.; Strötgen, J.; and Schuetze, H. 2024b. Rehearsal-Free Modular and Compositional Continual Learning for Language Models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), 469–480. Mexico City, Mexico. Wang, X.; Chen, T.; Ge, Q.; Xia, H.; Bao, R.; Zheng, R.; Zhang, Q.; Gui, T.; and Huang, X.-J. 2023a. Orthogonal subspace learning for language model continual learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, 10658–10671. Wang, X.; Zhang, Y.; Chen, T.; Gao, S.; Jin, S.; Yang, X.; Xi, Z.; Zheng, R.; Zou, Y.; Gui, T.; Zhang, Q.; and Huang, X. 2023b. TRACE: A comprehensive benchmark for continual learning in large language models. arXiv preprint arXiv:2310.06762. Wang, Y.; Liu, Y.; Shi, C.; Li, H.; Chen, C.; Lu, H.; and Yang, Y. 2024c. InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 663–677. Wang, Z.; Tsvetkov, Y.; Firat, O.; Cao, Y.; Mallinson, J.; and Li, X. 2020. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. arXiv preprint arXiv:2010.05874. Yang, Y.; Muhtar, D.; Shen, Y.; Zhan, Y.; Liu, J.; Wang, Y.; Sun, H.; Deng, D.; Sun, F.; Zhang, Q.; Chen, W.; and Tong, Y. 2025. MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning. arXiv:2410.09437. Yu, T.; Kumar, S.; Gupta, A.; Hausman, K.; Levine, S.; and Finn, C. 2020. Gradient surgery for multi-task learning. 33: 5824–5836. Zhang, D.; Zhang, K.; Chu, S.; Wu, L.; Li, X.; and Wei, S. 2025. MoRE: A Mixture of Low-Rank Experts for Adaptive Multi-Task Learning. In Che, W.; Nabende, J.; Shutova, E.; and Pilehvar, M. T., eds., Findings of the Association for Computational Linguistics: ACL 2025, 1311–1324. Vienna, Austria: Association for Computational Linguistics. ISBN 979-8-89176-256-5. Zhang, Q.; Chen, M.; Bukharin, A.; He, P.; Cheng, Y.; Chen, W.; and Zhao, T. 2023. AdaLoRA: Adaptive budget allocation for parameter-efficient fine-tuning. arXiv preprint arXiv:2303.10512.