When the Merge Coefficient Stops Mattering: Proximity Regularized Merging for Continual LoRA Adaptation

Rehearsal-free continual learning with parameter-efficient adapters can be cast as a sequence of task-vector write-in operations: for each new task, a low-rank adapter is learned and merged into a running model. We propose Proximity Regularized Merging (PRM), a minimal modification to sequential LoR