Cortex-Inspired Continual Learning: Unsupervised Instantiation and Recovery of Functional Task Networks
arXiv:2604.24637v1 [cs.LG] 27 Apr 2026
Kevin McKee
Thomas Hazy
Yicong Zheng Zacharie Bugaud Astera Institute
Thomas Miconi
Abstract Block-sequential continual learning demands that a single model both protect prior solutions from catastrophic forgetting and efficiently infer at inference time which prior solution matches the current input without task labels. We present Functional Task Networks (FTN), a parameter-isolation method inspired by structural and dynamical motifs found in the mammalian neocortex. Similar to mixture-of-experts, this method uses a high dimensional, self-organizing binary mask over a large population of small but deep networks, inspired by dendritic models of pyramidal neurons. The mask is produced by a three-stage procedure: (1) gradient descent on a continuous mask identifies task-relevant neurons, (2) a smoothing kernel biases the result toward spatial contiguity, (3) and k-winner-take-all binarizes the resulting group at a fixed capacity budget. Like mixture-of-experts, each neuron is an independent deep network, so disjoint masks give exactly disjoint gradient updates, providing structural guarantees against catastrophic forgetting. This three-stage procedure recovers the subnetwork of a previously-trained task in a single gradient step, providing unsupervised task segmentation at inference time. We test it on three continual-learning benchmarks: (1) a synthetic multi-task classification/regression generator, (2) MNIST with shuffled class labels (pure concept shift), and (3) Permuted MNIST (domain shift). On all three, FTN with several, fine grained smoothing steps (FTN-Slow) results in nearly zero forgetting. FTN with a larger kernel and only 2 iterations of smoothing (FTN-Fast) trades off a small amount of retention for increased speed. We show that the spatial organization mechanism reduces the effective mask search from the combinatorial top-k subset problem in O( H ) to the complexity of a near-linear scan in O(H) over k compact cortical neighborhoods, which is parallelized by the gradient-based update.
1
Introduction
Modern learning systems, from lifelong agents to continually pretrained language models [13, 43], must absorb new data without degrading or destroying old competence. When a network trained on task A is subsequently fine-tuned on task B, gradient updates on B overwrite the parameter configurations that encoded A’s solution [9, 22], producing catastrophic forgetting. The problem is most acute under two complementary conditions: block-sequential training, where each task is a contiguous data block with no interleaving (the standard regime for continual pretraining of LLMs); and concept shift, where the same input features map to different correct outputs across tasks [10, 25]. In both settings, the learner must also solve an unsupervised task detection problem: given a few examples of input-output pairs at inference time, identify which prior solution (if any) applies [1]. Within continual learning, there are two kinds of data non-stationarity that result in forgetting. Domain shift (or covariate shift) leaves the input-output relationship intact but changes the marginal input distribution [25, 31], which can result in partial overwriting of solutions if the whole input domain is not represented in later updates. Classic benchmarks such as Permuted MNIST [12, 16, 39] are usually treated as domain-incremental benchmarks, although the raw-coordinate conditional changes under each fixed permutation. Concept shift changes the input-output relationship itself. Identical features yield different correct labels [10, 25]. Concept shift is the harder setting for regularization methods because it can demand that a shared model represent incompatible input-output maps, especially when task identity is not provided at test time [39]. Block-sequential training amplifies the negative effects on retention from both kinds of shift. A further desideratum is task-free continual learning [1, 28, 39]: the learner must figure out task boundaries from the data stream itself rather than being told. Three families of methods have emerged to address forgetting [28]. Regularization approaches [16, 45] penalize updates to parameters estimated to be important for prior tasks; they typically fail under concept shift because a single 1
parameter cannot simultaneously satisfy conflicting weight configurations. Experience replay [18, 32] stores and rehearses past examples to anchor shared representations, but the memory and compute footprint scales with task history. Parameter isolation [21, 34, 36, 41, 44] assigns each task a disjoint subnetwork, converting the forgetting problem into a routing problem: if different tasks recruit different parameters, training one cannot disturb the other. This paper contributes a parameter-isolation method, Functional Task Networks (FTN), that unifies the three core requirements of continual learning in a single mechanism: • Structural forgetting protection. A parallel-neuron backbone stores each neuron’s weights as a private tensor; a binary routing mask gates the neurons’ scalar outputs. Disjoint masks therefore produce exactly disjoint gradient paths. • Rapid unsupervised task detection. A three-stage mask configurer, i.e., gradient descent on a continuous mask, lateral smoothing, and k-winner-take-all (KWTA) binarization, is run cold-started on a batch of input-output pairs with no task label. The same procedure that installs a subnetwork during training recovers it at inference time, in only one gradient step. • Efficient shared-neuron consolidation. Where tasks do share neurons, the mask overlap between the current task’s mask and buffered task masks provides a natural query for selective experience replay, minimizing the amount of replay needed.
1.1
Inspirations from biology
Our method takes inspiration from several structural and dynamical motifs observed in the mammalian neocortex, namely (1) dendritic computation in pyramidal neurons [3, 29, 30], (2) lateral excitatory and horizontal connectivity among cortical pyramidal neurons [5, 11, 14], (3) lateral inhibition among neurons resulting in sparse task representation [2, 5, 19], (4) spatio-temporal attractor dynamics and task-specific ensemble recruitment [4, 15, 24, 33, 42], and (5) basal-ganglia and dopaminergic modulation of cortical gating and selection [8, 23, 27, 35]. In this paper, we will give a brief exposition of these influences but leave more rigorous treatment of neuroscientific theory to subsequent work. First, the dendritic model of pyramidal neurons holds that each neuron can have nonlinear processing capacity comparable to a multi-layer artificial neural network, owing to its complex network of dendritic compartments [3, 29, 30]. We therefore specified each “neuron” or expert in our model as a deep multi-layer perceptron (MLP) with relatively small hidden dimensionality and a scalar output, akin to the emission rate of action potentials. The functional benefit of this model is that deep, nonlinear feature extraction is compartmentalized within neurons, with neural outputs linearly combined for particular tasks. This simplifies the problem of selecting optimal features for a particular task to a search over linear combinations. Furthermore, the restricted output dimensionality discourages neurons from learning redundant solutions, a problem related to expert collapse and load imbalance in mixture-of-experts models [7, 37], as no one neuron is structurally capable of solving the higher-dimensional task alone. Incorporating stochasticity in the output via dropout [38] is an algorithmic analogue of neural response variability [6, 20] and prevents the network from relying too much on any single or joint contribution of neurons to the task, granting tolerance to the imperfect recovery of optimal neural assemblies. Each neuron sits at a location on a two-dimensional grid. In the cortex, nearby neurons interact through local recurrent and horizontal connectivity [5, 11, 14], which we implement as a simple, uniform and positive-valued convolution, i.e., a smoothing kernel. Rather than applying this smoothing kernel to the neural activations directly, we apply it to the multiplicative routing mask such that neurons group together based on their relevance to a task, rather than their relevance to a particular input. This spatial grouping serves two purposes in our model. First, it draws networks apart spatially, protecting them from overlap and interference. Second, smoothing reduces the search space of possible k-sized subnetworks over N neurons from H k solutions to as few as H solutions under the idealized case of a single fixed-size contiguous blob. However, the actual number of solutions depends on the contiguity within the networks. That contiguity is determined by the size of the smoothing kernel and the number of smoothing steps. A smaller kernel with few smoothing steps permits each task network to form as many small but co-active groups. Aside from lateral excitation, neurons also connect laterally to inhibitory interneurons, which introduce competition [2, 5, 19]. We implement this using a common abstraction for lateral inhibition, namely k-winner-take-all (KWTA or top-k) selection. This choice results in a simple method of restricting the computational budget per sub-task. It is by no means the only way, and in fact implementing a negative-valued smoothing kernel larger than the excitatory kernel, (i.e., a “mexican hat” kernel [2, 40]) results in flexible allocation of neurons, whose approximate k and variance around
2
k that can be tuned continuously. We leave investigation of such kernels, akin to cellular automata, to future work and instead prioritize in this work simple, sufficient, and rapid computations for machine learning purposes. Finally, our use of gradient signals to select an initial population of best-performing neurons, which then acts as the initial condition for spatial consolidation and competition, is loosely inspired by basal-ganglia gating and dopaminergic reward-prediction signals in the control of cortical state selection [8, 23, 27, 33, 35]. Each of these motifs was chosen primarily for its specific algorithmic benefit rather than to simulate biology exactly. Altogether, they produce a system that performs unsupervised instantiation and recruitment of artificial neural assemblies, enabling block-sequential continual learning over a highly flexible class of machine learning architectures. Contributions. 1. We introduce a parallel-neuron architecture with gradient-driven, spatially organized routing masks (Section 3) and prove a structural no-forgetting property for disjoint masks. 2. We show that the same mask selection process, applied cold-started to an batch of input-output pairs with no task label, recovers the correct prior-task subnetworks in as few as 1 gradient step, yielding unsupervised task segmentation at inference time (Sections 3, 4). 3. We evaluate the method on three benchmarks spanning synthetic multi-task data, pure concept shift (MNIST Shuffled Labels), and domain shift (Permuted MNIST). FTN outperforms regularization and ablation baselines, including KWTA without spatial dynamics and EWC, on all three (Section 4). 4. We characterize when spatial organization helps (Section 5): it accelerates and stabilizes mask recovery by projecting the combinatorial top-k problem onto the low-dimensional manifold of contiguous cortical blobs, while the combinatorial flexibility of plain KWTA remains available for out-of-distribution generalization, a direction we leave for future work due to its entailment of NP-hard problem structure.
2
Related Work
Regularization-based continual learning. Elastic Weight Consolidation (EWC) [16] penalizes changes to parameters with high diagonal Fisher information; Synaptic Intelligence [45] accumulates an online importance score. Both degrade when tasks conflict on a shared parameter, as we verify empirically on MNIST Shuffled Labels. Parameter-isolation methods. PackNet [21] iteratively prunes and freezes weights per task. Hard Attention to the Task (HAT) [36] learns a per-task gated mask using an explicit task embedding and a gradient-reversal penalty. SupSup [41] stores a superposition of per-task binary masks over a fixed random backbone and selects among them at inference by gradient alignment. Progressive Networks [34] and DEN [44] instead grow fresh pathways per task. FTN differs from all of these in that (a) the mask is produced at reconfiguration time by a short gradient descent on the live model and current batch, without learning a mask or task embedding across tasks, and (b) the same procedure that installs a mask also retrieves it, so the mechanism is the same at train and test time and no task identity is required at inference. Mixture-of-experts and sparse routing. A mask over independent per-neuron subnetworks also inherits the sparserouting perspective of mixture-of-experts [7, 37]. FTN can be read as a sparsely-routed MoE in which the router is replaced by a short inner optimization and the experts are a 2-D cortical grid of small MLPs.
3
Method
3.1
Problem Setup
We consider block-sequential continual learning with N tasks. At block t ∈ {1, . . . , N } the learner receives a contiguous stream of samples (x, y) ∼ Dt , trains on them for a fixed budget, and must thereafter retain performance on all prior blocks without revisiting their data. At evaluation the task identity is not revealed; the model must recover whichever prior solution best explains a test batch [39]. 3
3.2
Parallel-Neuron Architecture
√ We place H “neurons” on a square cortical grid of side D = H, following the broad topographic-map intuition that nearby units can share or compete over related functions [17]. Each neuron k ∈ {1, . . . , H} is itself a small feedforward network of L hidden layers with inner width dinner and a scalar output. Let MLPk (x) : Rdin → R denote neuron (ℓ) (ℓ) k’s private computation; its weights {Wk , bk }L+1 ℓ=1 are stored as the k-th slice of a (H, ·, ·) tensor and applied via a batched einsum that broadcasts the shared input across the H slots. A binary routing mask m ∈ {0, 1}H gates the stacked scalar outputs, and a single linear readout Wout ∈ Rdout ×H produces the prediction: zk = MLPk (x) ∈ R,
k = 1, . . . , H, ŷ = Wout Dropout(z) ⊙ m .
(1) (2)
The resulting model is a sparse mixture of H experts [37]. Dropout on the output vector [38] further decorrelates the ensemble and makes the downstream readout robust to noise in which specific neurons are selected. Because each neuron owns a disjoint weight tensor, the only shared parameter across neurons is the readout Wout ; its columns are tied one-to-one to neuron outputs. A neuron with mk = 0 contributes zero to the readout, receives zero gradient at its column of Wout , and—because its output is unused downstream—receives zero gradient throughout its private weight tensor. Disjoint masks produce exactly disjoint updates.
Figure 1: The functional task network model, visualized. Each color represents a separate, spatially cohesive subnetwork responsible for a particular task. Each neuron in the assembly is a multi-layer feed forward network, such that the task output is determined by a linear combination of many highly nonlinear basis functions. Theorem 1 (Structural forgetting guarantee1 ). Let m(s) , m(t) ∈ {0, 1}H be the masks in force during training blocks (s) s < t. If m(s) ⊙ m(t) = 0, then for every parameter θk internal to neuron k with mk = 1 and every column [Wout ]:,k , the gradient of the task-t loss with respect to θk is zero. Consequently, training on block t leaves the task-s solution exactly unchanged. (t)
(t)
Proof. Direct from the forward pass: zk enters the readout only via mk · zk . With mk = 0, zk is multiplied by zero in the loss, so ∂Lt /∂zk = 0; by the chain rule ∂Lt /∂θk = 0 for every parameter on which zk depends, and ∂Lt /∂[Wout ]:,k is likewise zero. 1 Implementations should be careful to prevent optimizer-state updates, decoupled weight decay, shared output biases, BatchNorm/statistics updates, and other non-gradient parameter changes on inactive neurons.
4
Algorithm 1 S MOOTH KWTA mask configurer Require: Model fθ , batch (X, Y ) or unlabeled batch X with a self-supervised loss, grid side D, kernel size s, lateral steps T , winners k, reconfig steps S, reconfig LR η 2 1: m ← 0 ∈ RD {cold start} 2: for i = 1, . . . , S do 3: m̃ ← σ(m) 4: L ← Loss fθ (X, m̃), Y b m L {Adam step on mask logits} 5: m ← m−η∇ 6: end for 7: Reshape m to D × D 8: for t = 1, . . . , T do 9: m ← Ks ∗ m {uniform s × s lateral convolution, torus padding} 10: end for 11: m ← KWTA flatten(m), k 12: return m
3.3
Three-Stage Mask Configurer
The mask is produced by a short procedure that takes the current model, a data batch, and nothing else (no task identity, no task embedding). It combines three cortical motifs. Stage 1, Gradient-based proposal. The mask is treated as a continuous parameter passed through a sigmoid, and is optimized for S Adam steps to minimize the task loss on the current batch. The gradient is computed end-to-end through the NeuroModel with the relaxed mask in the forward pass. Because the mask is cold-started from zeros, each call is a fresh, task-agnostic probe of which neurons currently drive the loss. Biologically, this stage plays the role of a reward-prediction-error [35] signal selecting among competing neural assemblies. Stage 2, Dense lateral excitation. The continuous mask is reshaped to the D × D cortical grid and convolved T times with a uniform s × s kernel (torus padding). Nearby cells reinforce each other; noisy gradient cues are averaged out over the kernel footprint. This is a minimal model of the dense excitatory lateral connectivity observed in cortical micro-circuits [5, 11]. Its algorithmic role is to project the mask onto the low-dimensional manifold of contiguous blobs—a drastic reduction of the combinatorial solution space, which we analyze below. Stage 3, k-winner-take-all (KWTA). The top-k smoothed values are binarized to one, the rest to zero. KWTA [19] abstracts the cortical lateral inhibition that enforces global competition among excitatory neurons [5], giving each task a fixed capacity budget k/H.
3.4
Training Procedure
Training proceeds block-sequentially. For each task block t we reconfigure the mask at the start of the block (Exp. 1) or at the start of each epoch (Exp. 2–4), then run standard AdamW steps on the model weights while keeping the mask fixed. After finishing the block we evaluate performance on all tasks 0..t under two protocols: stored (use the mask saved at training time, the oracle protocol) and reconfig (re-run the configurer cold-started on a fresh evaluation batch, the realistic protocol). All headline numbers in Section 4 use the reconfig protocol.
3.5
Mask Configuration Variants
We consider six variants that share the backbone and differ only in the configurer (Table 1). FTN-Fast and FTN-Slow are the two main lateral-dynamics settings; KWTA-only is the ablation that turns off lateral smoothing; FixedMask and NoMask are structural baselines; EWC is the classical regularization baseline.
5
Table 1: Mask configurer variants. Name
Kernel
Steps
FTN-Fast FTN-Slow KWTA-only FixedMask NoMask EWC
17 × 17 3×3 – – – –
2 15 0 – – –
4
Experiments
4.1
Setup
Role Rapid spatial convergence via large kernel Fine-grained iterative spatial smoothing Ablation: gradient + KWTA, no lateral dynamics Static disjoint blocks; structural upper bound All-ones mask; naive fine-tuning [16], λ = 400
The parallel-neuron backbone is instantiated with H=1024 slots on a 32×32 grid; each slot is an L=8-layer MLP of inner width dinner =8; output dropout p=0.2. Readout is a single linear layer. Optimizer: AdamW at η=3×10−4 . KWTA sparsity k=128 (i.e. each task uses 12.5% of the slots). All experiments use 8 seeds on a single H100; reported numbers are mean ± std. We report three standard continual-learning metrics [18] computed from the final-block performance matrix: Average Accuracy (ACC; mean MSE for regression, lower is better), Forgetting Measure (FM), and Backward Transfer (BWT). For regression, FM is the average peak-MSE minus final-MSE on prior tasks (lower is better) and BWT is the average final-MSE minus first-trained-MSE on prior tasks (lower is better, since rising MSE signals forgetting). Headline numbers use the mask-recovery protocol (task identity hidden at inference).
4.2
Experiment 1: Synthetic Multi-Task Benchmark
Setup. Three synthetic tasks: a fixed encoder projects a 2-D input into a 24-D latent; each task selects a disjoint 8-D block via a task-specific hidden mask; a sinusoidal nonlinearity at frequency 8 produces the classification (binary) or regression (scalar) target. Ten epochs of 1000 steps per task, batch 256. Reconfiguration runs once at the start of each task and cold-starts the mask from zeros (Algorithm 1) for both variants. The classification and regression variants share an identical training pipeline (same backbone, same data generator, same number of epochs / steps / batches, same AdamW model optimizer, same KWTA budget k=128, same zero mask cold-start). The two variants differ only in (i) the readout dimension, 2 logits with cross-entropy for classification, 1 scalar with MSE for regression, and (ii) the mask reconfigurer’s inner optimization: S=1 step at ηm =1.0 for classification versus S=10 steps at ηm =0.2 for regression. The longer, gentler regression schedule was chosen empirically: the scalar MSE loss benefits from accumulating evidence over more, smaller steps before the lateral and KWTA stages binarize, whereas a single high-LR step is sufficient for classification. Results. Table 2 reports mask-recovery metrics. On classification, FTN-Fast and FTN-Slow cleanly recover the task subnetworks (ACC = 0.891/0.833, FM = 0.066/0.080); KWTA-only fails to reliably re-find the trained mask under a single-step reconfig (ACC = 0.528) because its gradient proposal is not stabilized by spatial averaging. EWC retains the naive-fine-tune performance of NoMask (ACC = 0.670, FM ≈ 0.47) because the three tasks conflict on the shared encoder. FixedMask is the trivial upper bound by construction (ACC = 0.949, FM = 0.061). On regression, the same qualitative ordering holds: spatial smoothing further reduces error (MSE = 0.098/0.131 for FTN-Slow / FTN-Fast vs. 0.174 for KWTA-only), with FixedMask the structural lower bound (MSE = 0.035) and NoMask/EWC failing to retain (MSE = 0.314). Mask-overlap loss vs. recall-error loss. A single ACC/MSE number conflates two distinct failure modes: (i) at training time, two tasks may overlap on shared neurons so the second overwrites part of the first, visible as a drop in the stored-mask perf matrix relative to FixedMask; and (ii) at evaluation time, the configurer may fail to re-find the trained mask, visible as a drop from stored to recovered in the same row. Taking each method’s final-row average against the FixedMask oracle on prior tasks (FixedMask reference: clf ACC= 0.949 / reg MSE= 0.035), the train-time and recall-time gaps decompose as in Table 3. 6
Figure 2: Example data distributions from the classification task generator, with 2 dimensional input mapping to 2 possible output classes. The model must learn all three classification mappings block-sequentially without forgetting any. Table 2: Experiment 1 (Synthetic CL, 3 tasks; mask-recovery eval protocol). Mean ± std over 8 seeds (clf) / 8 seeds (reg). Regression columns report MSE (lower is better); the regression FM is the average (peak MSE − final MSE) across earlier tasks (lower is better); BWT is the average (final MSE − first-trained MSE) across earlier tasks (lower is better, since rising MSE signals forgetting). Classification Method NoMask FixedMask KWTA FTN-Fast FTN-Slow EWC
Regression
ACC↑
FM↓
BWT↑
MSE↓
FM↓
BWT↓
0.670 ± 0.031 0.949 ± 0.009 0.528 ± 0.072 0.891 ± 0.041 0.833 ± 0.065 0.670 ± 0.031
0.478 ± 0.047 0.061 ± 0.012 0.064 ± 0.096 0.066 ± 0.036 0.080 ± 0.054 0.475 ± 0.046
−0.478 ± 0.047 −0.061 ± 0.012 −0.011 ± 0.151 −0.062 ± 0.031 −0.039 ± 0.063 −0.475 ± 0.046
0.314 ± 0.053 0.035 ± 0.009 0.174 ± 0.051 0.131 ± 0.080 0.098 ± 0.023 0.314 ± 0.053
0.055 ± 0.071 0.000 ± 0.000 0.077 ± 0.118 0.004 ± 0.010 0.008 ± 0.019 0.055 ± 0.072
0.470 ± 0.079 0.052 ± 0.014 0.050 ± 0.161 0.110 ± 0.095 0.051 ± 0.040 0.470 ± 0.079
The two columns must be read jointly. NoMask and EWC look “perfect on recall” only because they never had a separable mask to begin with, they pay everything to overlap. KWTA-only is the opposite: it writes nearly clean training masks (small overlap column) but its single-step gradient proposal does not converge to the same neuron set the next time it is run, so on classification its trained subnetworks effectively evaporate at recall (+0.350 ACC). FTN-Fast and FTN-Slow are the only methods that keep both columns small simultaneously. FTN-Slow has the smallest training-time overlap of any adaptive method on both variants (the slow many-step lateral diffusion produces the cleanest spatial separation between tasks), trading a modest classification recall error for the tightest separation. See Section A for the full per-method stored–vs–recovered matrices on both classification and regression. Figure 3 visualizes the FTN-Fast mask allocations across seeds as an RGB overlay (one color per task): the lateral kernel groups neurons into compact, non-overlapping blobs that are topographically consistent across random initialisations.
4.3
Experiment 2: MNIST with Shuffled Class Labels
Setup. Five tasks on standard MNIST. Task 0 uses the canonical digit labels; tasks 1–4 apply different random permutations of the 10-class label set. This is pure concept shift: the images are identical across tasks, only the labels change [25, 39]. Five epochs × 400 steps per task, batch 256. Reconfiguration per epoch (S=20, ηm =0.2). Results. Table 4 is a clean demonstration of the concept-shift failure mode of regularization: NoMask and EWC both collapse to near-chance (ACC = 0.255, FM = 0.904), because the single shared network cannot represent the five incompatible label-permutation mappings on the same weights. All masking methods succeed. FTN-Slow (fine-grained kernel) is best (ACC = 0.976 ± 0.001, FM = 0.004 ± 0.001); KWTA-only (0.951/0.025) is next;
7
Table 3: Decomposition of Exp. 1 mask-recovery loss into training-time mask overlap and evaluation-time recall error, relative to the FixedMask oracle. Classification: ACC drop (lower is closer to oracle). Regression: MSE excess (lower is closer to oracle). Classification ACC drop Method NoMask EWC KWTA FTN-Fast FTN-Slow
Regression MSE excess
Mask overlap
Recall error
Mask overlap
Recall error
0.279 0.279 0.072 0.047 0.043
0.000 0.000 0.350 0.011 0.073
0.278 0.278 0.133 0.069 0.048
0.000 0.000 0.006 0.027 0.015
FTN-Fast mask allocations (8 seeds)
FTN-Slow mask allocations (8 seeds)
(a) Classification masks (FTN-Fast, 8 seeds)
(b) Classification masks (FTN-Slow, 8 seeds)
Figure 3: RGB mask allocations across 8 random seeds for the synthetic benchmark. Each tile is one seed; each color is one of the three tasks. Note the spatially compact, approximately non-overlapping subnetworks. Overlap is less in FTN-Slow, leading to better retention. FTN-Fast (0.938/0.052) trails slightly with higher variance. Five-task concept shift is fully resolved in under 1% forgetting by the best FTN variant. Table 4: Experiment 2: MNIST Shuffled Labels (concept shift, 5 tasks). Mean ± std over 8 seeds; mask-recovery eval protocol. Method NoMask KWTA FTN-Fast FTN-Slow EWC
4.4
ACC↑
FM↓
BWT
0.255 ± 0.000 0.951 ± 0.004 0.938 ± 0.051 0.976 ± 0.001 0.255 ± 0.000
0.904 ± 0.001 0.025 ± 0.005 0.052 ± 0.064 0.004 ± 0.001 0.903 ± 0.001
−0.904 ± 0.001 −0.023 ± 0.006 −0.052 ± 0.064 −0.004 ± 0.001 −0.903 ± 0.001
Experiment 3: Permuted MNIST
Setup. The classical CL benchmark [12, 16, 39]: 10 tasks, each a random pixel permutation of MNIST; labels unchanged. This is usually treated as domain-incremental learning: the label space and task objective are shared, although the raw-coordinate mapping from pixels to digits changes under each permutation, so a single well-regularized readout can do reasonably well. Three epochs × 400 steps per task, batch 256. Reconfiguration per epoch (S=10, ηm =0.3).
8
After T1
0.976
0.981
After T2
0.977
0.979
stored (oracle)
FTN-Slow
1.0 After T0
0.978
After T1
0.974
0.981
After T2
0.974
0.977
recovered
0.6 0.4 After T3
0.977
0.976
0.979
0.982
After T4
0.976
0.976
0.978
0.978
0.980
T0
T1 T2 T3 Evaluated Task
T4
After T0
0.980
After T1
0.284
0.980
After T2
0.102
0.100
stored (oracle)
0.6 0.4 0.975
0.978
0.979
0.979
After T4
0.974
0.977
0.976
0.977
0.978
T0
T1 T2 T3 Evaluated Task
T4
0.2
0.6 0.977
0.4 After T3
0.001
0.002
0.002
0.978
After T4
0.004
0.099
0.100
0.097
0.977
T0
T1 T2 T3 Evaluated Task
T4
0.2
0.0
recovered
1.0
After T0
0.980
After T1
0.284
0.980
After T2
0.102
0.100
0.977
After T3
0.001
0.002
0.002
0.978
After T4
0.004
0.099
0.100
0.097
0.977
T0
T1 T2 T3 Evaluated Task
T4
0.8
0.979
After T3
NoMask
1.0
0.8 After Training
After Training
0.8
0.979
NoMask
1.0
0.8 After Training
0.979
After Training
FTN-Slow
After T0
0.6 0.4
0.2
0.0
(a) FTN-Slow: stored | recovered
0.2
0.0
0.0
(b) NoMask
Figure 4: Performance matrices for MNIST Shuffled Labels. Cell (i, j) is test accuracy on task j after training through task i. FTN-Slow retains all five label permutations; NoMask loses each prior task immediately upon training the next. Results. Table 5 shows the regime where regularization methods hold up: EWC reaches ACC = 0.941, FM = 0.037. FTN still performs well, with FTN-Slow best (ACC = 0.959 ± 0.002, FM = 0.019 ± 0.002), FTN-Fast close behind (0.956/0.023), KWTA-only lower (0.926/0.042), and NoMask trailing substantially (0.829/0.164). The ranking FTN-Slow > FTN-Fast > KWTA-only > EWC > NoMask is consistent across both MNIST-style benchmarks. Table 5: Experiment 3: Permuted MNIST (domain shift, 10 tasks). Mean ± std over 8 seeds; mask-recovery eval protocol. Method NoMask KWTA FTN-Fast FTN-Slow EWC
4.5
ACC↑
FM↓
BWT
0.829 ± 0.014 0.926 ± 0.008 0.956 ± 0.006 0.959 ± 0.002 0.941 ± 0.003
0.164 ± 0.016 0.042 ± 0.009 0.023 ± 0.006 0.019 ± 0.002 0.037 ± 0.003
−0.164 ± 0.016 −0.036 ± 0.010 −0.023 ± 0.006 −0.018 ± 0.002 −0.037 ± 0.003
Cross-Benchmark Summary
Figure 5 (MNIST Shuffled) and the tables above share a single qualitative pattern: the six methods cluster into three regimes. (1) Ablation controls (NoMask, EWC) forget catastrophically under concept shift and partially under domain shift. (2) Adaptive masking without spatial structure (KWTA-only) prevents forgetting reliably but has lower maskrecovery accuracy and higher variance than FTN. (3) Spatial-masking FTN (FTN-Fast, FTN-Slow) is uniformly at or near the structural upper bound, with FTN-Slow (fine-grained many-step smoothing) winning on every benchmark where both are applicable.
CL Metrics ACC
1.0
recovered mask FM
0.903
0.938 0.976 0.951
0.904
0.0
0.8
0.8
0.2
0.6
0.6
0.4
0.4
0.4
0.6
0.2 0.0
0.255
0.255
t k A EWC TN-Fas N-Slow KWT NoMas F FT
0.2 0.0
0.052
0.004 0.025
t k A EWC TN-Fas N-Slow KWT NoMas F FT
0.8
-0.052
-0.903
BWT
-0.004 -0.023
-0.904
t k A EWC TN-Fas N-Slow KWT NoMas F FT
Figure 5: MNIST Shuffled Labels (mask-recovery protocol, 8 seeds). FTN variants recover prior-task solutions almost perfectly; KWTA-only is strong but slightly behind; NoMask/EWC are at chance.
9
5
Discussion
Summary of the empirical finding. The experiments support a narrow but useful conclusion: in block-sequential continual learning, a parameter-isolated backbone can be paired with a short, task-identity-free mask-configuration procedure to recover prior solutions without being given an oracle task label. FTN is strongest in the concept-shift regimes, where the same input features require different outputs across blocks and shared-parameter methods are expected to fail [10, 25, 39]. This is the setting in which regularization methods such as EWC and SI are least well matched, because penalizing weight movement cannot make a single shared parameter simultaneously implement incompatible mappings [16, 45]. The empirical pattern is consistent across the benchmarks: NoMask forgets, EWC helps mainly when the task sequence is closer to domain shift than pure concept shift, KWTA-only shows that sparse mask selection already supplies much of the benefit, and the spatially smoothed FTN variants improve recovery stability and retention. Why FTN forgets less. Theorem 1 gives the basic mechanism. If two tasks use disjoint masks, training on one task induces no gradient on the private weights or readout columns used by the other. FTN therefore turns forgetting into a routing problem, as in parameter-isolation methods such as PackNet, HAT, SupSup, Progressive Networks, and DEN [21, 34, 36, 41, 44]. The difference is that FTN does not require a task embedding or an oracle task identity at evaluation. The same optimization over mask logits that installs a subnetwork during training is run again on a fresh support batch to recover the mask. Thus the relevant comparison is not only to methods that protect old parameters, but to methods that can identify which protected parameters should be active when the task identity is hidden [1, 39]. What spatial organization contributes. The spatial kernel should be understood as an inductive bias on the mask search, not as a formal guarantee that the original combinatorial problem has disappeared. Without smoothing, KWTA only can in principle choose any of the H k masks, a sparse subset-selection problem that is hard in the worst case [26]. With smoothing, the search is biased toward compact regions on the cortical grid. In the idealized case of a single fixed-size contiguous blob, the number of candidate placements is only O(H); with multiple blobs, irregular shapes, or weak smoothing, the effective search space is larger. The experiments show that this bias is useful for recall: when the goal is to recover a previously trained task network, smoothing turns noisy single-batch gradients into stable attractor-like mask patterns and reduces variance relative to KWTA-only. The price is reduced combinatorial flexibility, so spatial FTN should not be expected to dominate KWTA-only in every setting. Tasks that require novel recombinations of distant units, or OOD settings where no previously learned compact region is sufficient, may benefit from the larger search space retained by KWTA-only. FTN-Slow versus FTN-Fast. The consistent advantage of FTN-Slow over FTN-Fast suggests that, in these benchmarks, the useful role of the lateral stage is not merely to form a compact mask. A large kernel with few iterations can impose compactness quickly, but it also averages away local distinctions in the gradient proposal. A small kernel applied over many iterations appears to produce a cleaner basin of attraction: neighboring high-relevance neurons reinforce one another while weak isolated responses are suppressed. This interpretation matches the observed ordering, where FTN-Slow is closest to the FixedMask upper bound, FTN-Fast trades some retention for speed, and KWTA-only is competitive but less stable. It also clarifies the practical hyperparameter choice: kernel size and smoothing depth control a bias–variance tradeoff in mask recovery, rather than a purely biological modeling choice. Relationship to replay and mixture-of-experts. FTN is compatible with replay, but the reported results do not rely on it. The mask overlap ρj = (mcur · mj )/∥mcur ∥1 could provide a query for selective replay: examples from prior tasks whose masks overlap the current mask are precisely the examples most likely to constrain shared neurons. This would connect FTN to replay methods such as iCaRL and GEM [18, 32], but it remains future work. FTN is also close in spirit to sparse mixture-of-experts models [7, 37]. The main difference is where routing comes from. Standard MoE learns a feed-forward router over inputs; FTN performs an inner-loop optimization over a mask using the current batch loss. This makes routing slower, but it decouples task selection from a learned input-only router and lets the same mechanism be used for both training-time mask installation and evaluation-time mask recovery. Limitations. Several limitations remain. First, the present benchmarks are small and largely synthetic. Second, the baseline set is intentionally minimal: NoMask, EWC, KWTA-only, and FixedMask isolate the contribution of spatial
10
mask recovery, but potential comparisons include related baselines such as HAT, PackNet, SupSup, GEM, and iCaRL under matched capacity and compute [18, 21, 32, 36, 41], though these methods differ in exactly what they are designed to accomplish, e.g., requiring task-labels or explicit store-recall decisions. Third, the architecture spends capacity by allocating k of H neurons to each recovered subnetwork. Long sequences of mutually incompatible tasks will eventually exhaust the grid unless k is adapted, masks are allowed to share safely, or new neurons are added. Fourth, the structural forgetting guarantee applies to masked gradient paths; implementations must also avoid unintended changes through optimizer state, decoupled weight decay, shared biases, normalization statistics, or other non-gradient updates. Finally, mask configuration adds inference-time compute and introduces kernel, k, and reconfiguration-step hyperparameters whose scaling behavior is not yet known. Outlook. The most important next experiments are therefore clear. FTN should be tested on longer task sequences with capacity pressure, under matched comparisons to stronger task-free and parameter-isolation baselines, and with support-batch size varied to quantify how much evidence is needed for reliable mask recovery. A second direction is to combine FTN with selective replay or adaptive neurogenesis so that overlap can be used constructively rather than avoided. A third direction is to replace the supervised configuration loss with a self-supervised objective in settings where such a loss is already native, such as continual language-model pretraining. The present results justify FTN as a strong recall mechanism for block-sequential concept shift; whether the same mask-search machinery can support open-ended transfer and OOD recombination remains the central open question.
6
Conclusion
We introduced Functional Task Networks, a continual-learning method that combines a parallel bank of independent per-neuron subnetworks with a cortically-organized three-stage mask configurer. The backbone gives a strong structural forgetting guarantee (Theorem 1); the configurer rapidly installs and recovers task-specific subnetworks on the cortical grid, turning the mask-selection problem from unconstrained combinatorial search ( H k ) toward a near-linear search over spatially contiguous blob locations (O(H) under the single-blob idealization). On three benchmarks spanning concept shift, domain shift, and realistic tabular data, FTN-Slow attains the best reported numbers among adaptive methods: FM ≤ 0.02 on MNIST Shuffled Labels, and Permuted MNIST, all under the realistic mask-recovery protocol with task identity hidden at inference time. The method is compatible with block-sequential training, requires no task labels, and provides a natural mask-overlap query for minimizing experience replay. The combinatorial flexibility that plain KWTA retains remains a promising basis for future work on knowledge transfer and out-of-distribution generalization, which we frame as a distinct, computationally harder problem whose co-integration with efficient recall is a key open question for continual learning.
References [1] Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. IEEE CVPR, 2019. [2] Shun-ichi Amari. Dynamics of pattern formation in lateral-inhibition type neural fields. Biological Cybernetics, 27(2):77–87, 1977. [3] David Beniaguev, Idan Segev, and Michael London. Single cortical neurons as deep artificial neural networks. Neuron, 109(17):2727–2739, 2021. [4] Joseph Cichon and Wen-Biao Gan. Branch-specific dendritic ca2+ spikes cause persistent synaptic plasticity. Nature, 520(7546):180–185, 2015. [5] Rodney J Douglas and Kevan AC Martin. Neuronal circuits of the neocortex. Annual Review of Neuroscience, 27: 419–451, 2004. [6] A. Aldo Faisal, Luc P. J. Selen, and Daniel M. Wolpert. Noise in the nervous system. Nature Reviews Neuroscience, 9(4):292–303, 2008. [7] William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23:1–39, 2022. 11
[8] Michael J. Frank, Bryan Loughry, and Randall C. O’Reilly. Interactions between frontal cortex and basal ganglia in working memory: A computational model. Cognitive, Affective, & Behavioral Neuroscience, 1(2):137–160, 2001. [9] Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128–135, 1999. [10] João Gama, Indrė Žliobaitė, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey on concept drift adaptation. ACM Computing Surveys, 46(4):1–37, 2014. [11] Charles D. Gilbert and Torsten N. Wiesel. Clustered intrinsic connections in cat visual cortex. Journal of Neuroscience, 3(5):1116–1133, 1983. [12] Ian J. Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211, 2013. [13] Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360. Association for Computational Linguistics, 2020. [14] David H Hubel and Torsten N Wiesel. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of Physiology, 160(1):106–154, 1962. [15] Mikail Khona and Ila R. Fiete. Attractor and integrator networks in the brain. Nature Reviews Neuroscience, 23 (12):744–766, 2022. [16] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526, 2017. [17] Teuvo Kohonen. Self-organized formation of topologically correct feature maps. Biological cybernetics, 43(1): 59–69, 1982. [18] David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017. [19] Wolfgang Maass. On the computational power of winner-take-all. Neural computation, 12(11):2519–2535, 2000. [20] Zachary F. Mainen and Terrence J. Sejnowski. Reliability of spike timing in neocortical neurons. Science, 268 (5216):1503–1506, 1995. [21] Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. [22] Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of learning and motivation, 24:109–165, 1989. [23] Jonathan W Mink. The basal ganglia: focused selection and inhibition of competing motor programs. Progress in Neurobiology, 50(4):381–425, 1996. [24] Gianluigi Mongillo, Simon Rumpel, and Yonatan Loewenstein. Inhibitory connectivity defines the realm of excitatory plasticity. Nature neuroscience, 21(10):1463–1470, 2018. [25] Jose G. Moreno-Torres, Troy Raeder, Rocío Alaiz-Rodríguez, Nitesh V. Chawla, and Francisco Herrera. A unifying view on dataset shift in classification. Pattern Recognition, 45(1):521–530, 2012. [26] Balas Kausik Natarajan. Sparse approximate solutions to linear systems. SIAM Journal on Computing, 24(2): 227–234, 1995.
12
[27] Randall C. O’Reilly and Michael J. Frank. Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia. Neural Computation, 18(2):283–328, 2006. [28] German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54–71, 2019. [29] Panayiota Poirazi and Athanasia Papoutsi. Illuminating dendritic function with computational models. Nature Reviews Neuroscience, 21(6):303–321, 2020. [30] Panayiota Poirazi, Terrence Brannon, and Bartlett W. Mel. Pyramidal neuron as two-layer neural network. Neuron, 37(6):989–999, 2003. [31] Joaquin Quiñonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence, editors. Dataset Shift in Machine Learning. MIT Press, 2009. [32] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. [33] Edmund T Rolls. Attractor networks. Wiley Interdisciplinary Reviews: Cognitive Science, 1(1):119–134, 2010. [34] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016. [35] Wolfram Schultz, Peter Dayan, and P Read Montague. A neural substrate of prediction and reward. Science, 275 (5306):1593–1599, 1997. [36] Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International Conference on Machine Learning, pages 4548–4557. PMLR, 2018. [37] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. ICLR, 2017. [38] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958, 2014. [39] Gido M. van de Ven and Andreas S. Tolias. arXiv:1904.07734, 2019.
Three scenarios for continual learning.
arXiv preprint
[40] Hugh R. Wilson and Jack D. Cowan. Excitatory and inhibitory interactions in localized populations of model neurons. Biophysical Journal, 12(1):1–24, 1972. [41] Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in superposition. Advances in Neural Information Processing Systems, 33:15173– 15184, 2020. [42] Guangyu Robert Yang, Madhura R Joglekar, H Francis Song, William T Newsome, and Xiao-Jing Wang. Task representations in neural networks trained to perform many cognitive tasks. Nature neuroscience, 22(2):297–306, 2019. [43] Çağatay Yıldız, Nishaanth Kanna Ravichandran, Nitin Sharma, Matthias Bethge, and Beyza Ermis. Investigating continual pretraining in large language models: Insights and implications. arXiv preprint arXiv:2402.17400, 2024. [44] Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. ICLR, 2018. [45] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pages 3987–3995. PMLR, 2017.
13
A
Per-method Performance Matrices: Stored vs Recovered
This appendix collects the full 3×3 performance matrices for Experiment 1 across the four headline mask configurers, NoMask (the null mask reuses the same 128-slot subnetwork for every task), KWTA-only, FTN-Fast, and FTN-Slow, under both evaluation protocols. Each panel shows two heatmaps side by side: the left heatmap (stored) evaluates each finished training block using the mask that was active when that task was trained; the right heatmap (recovered) evaluates with a mask that the configurer is asked to find from scratch on a fresh evaluation batch, with no task identity given. Cell (i, j) is the average performance on task j after training through task i, averaged over 8 seeds. Classification cells report ACC (range [0, 1], higher is better); regression cells report MSE clamped at 0.5 for visual comparability (lower is better; cells with mean MSE above 0.5 render in the saturated top color and are flagged with the colorbar’s → extension). Reading the panels. The two heatmaps separate two failure modes that get summed together in the headline ACC/MSE: • A drop in the stored (left) panel along an off-diagonal row i, column j < i measures how much training task j’s solution has been overwritten by subsequent gradient updates on tasks j+1 . . . i. With perfectly disjoint masks this drop is zero (the FixedMask appendix entry, omitted here for space, has identical diagonal and off-diagonal entries), so any drop here is attributable to shared neurons between tasks at training time. Methods with nondisjoint masks, including NoMask (full overlap by construction) and any adaptive method whose discovered masks happen to overlap, will show negative gradients on the stored column. • A drop from the stored panel to the corresponding cell in the recovered panel measures mask recall error: the underlying weights are unchanged, only the mask is being chosen by the configurer, so any difference is entirely attributable to picking the wrong subnetwork at evaluation time. NoMask and EWC have no recovery problem because they always use the same all-on-128 mask, but they pay this cost in the stored column. KWTA-only is the opposite: it writes nearly clean training masks (the stored panel for KWTA on classification looks almost FixedMask-like) yet collapses on recovery because its single-step gradient proposal does not converge to the same neuron set the next time it is run. The reader should therefore compare the two panels jointly. A method that loses 5% on the stored column has an irrecoverable problem (gradient conflict between tasks); a method that loses 5% from stored to recovered has a soluble problem (the trained subnetwork still exists, only the lookup is noisy). The aggregate decomposition was already given in Table 3 in the main text; the matrices below show where each loss is concentrated within the task sequence. Per-method matrices, classification. Per-method matrices, regression. Why classification and regression look different. The two configurers differ only in S (1 vs. 10) and ηm (1.0 vs. 0.2); architecture, training data, optimizer, capacity, and the zero mask cold-start are identical (cf. Section 4.2). On classification, a single high-LR step gives a usable but batch-noisy proposal whose support shifts between calls. FTN’s lateral kernel averages that noise into a stable spatial blob, so stored and recovered masks coincide; KWTA-only has no smoothing to stabilize the proposal across calls and therefore picks an inconsistent mask at evaluation time, producing the 0.350 ACC recall gap visible in Figure 6 (top right). On regression, ten gentler steps converge to a more reproducible proposal, so even KWTA-only recovers the right neurons reliably and the recall column does not collapse; the residual MSE is dominated by neuron-set overlap during training, exactly as the matrices show.
14
KWTA
0.540
0.991
0.4 After T2
0.436
T0
After T0
0.8
0.461
T1 Evaluated Task
0.988
T2
0.2
1.0
0.990
0.8 0.6
After T1
0.540
0.991
After T0
0.988
After T1
0.854
0.8 0.6 0.989
0.4 After T2
After T2
0.0
0.436
0.461
0.988
0.765
T0 T0
T1 Evaluated Task
T2
0.854
T1 Evaluated Task
After T1
0.956
0.8 0.6 0.991
0.4 After T2
0.905
0.952
0.989
T0
T1 Evaluated Task
T2
0.2 0.0
After T0
0.976
After T1
0.955
recovered
0.8 0.6 0.956
0.985
After T1
0.959
0.916
0.927
0.936
T0
T1 Evaluated Task
T2
0.958
After T1
0.684
1.0 0.8 0.6
0.567
0.4 After T2
0.0
0.541
0.501
0.538
T0
T1 Evaluated Task
T2
0.2 0.0
FTN-Slow
1.0 0.8 0.6
0.990
0.4 After T2
0.4 After T2
After T0
stored (oracle)
1.0 After Training
0.983
After Training
After Training
After T0
FTN-Fast
1.0
T2
0.2
After T0
recovered
0.2 0.0
(b) KWTA-only (clf): the stored panel is close to the FixedMask oracle, indicating the trained masks were nearly disjoint, but the recovered panel collapses to chance, a one-step gradient proposal with no spatial averaging picks an inconsistent mask at evaluation time. FTN-Slow
stored (oracle)
0.990
0.2 0.0
KWTA
1.0
0.4
(a) NoMask (clf): the same fixed first-128 mask is used for every task, so the stored panel shows full overwriting (rows 1–2 zero out task 0) and the recovered panel is identical because no recovery happens.
FTN-Fast
stored (oracle)
After Training
0.990
recovered
0.915
0.935
0.988
T0
T1 Evaluated Task
T2
0.2 0.0
After Training
1.0
0.6 After T1
NoMask
After Training
After Training
After T0
stored (oracle)
After Training
NoMask
After T0
0.959
After T1
0.953
recovered
1.0 0.8 0.6
0.927
0.4 After T2
0.896
0.912
0.731
T0
T1 Evaluated Task
T2
0.2 0.0
(d) FTN-Slow (clf): the smallest training-time overlap of any adaptive method (cleanest stored panel) but slightly larger recall noise than FTN-Fast, a small kernel applied for many steps produces the tightest spatial separation, at the cost of slightly higher recovery variance per call.
(c) FTN-Fast (clf): stored and recovered panels are nearly identical. The large 17 × 17 kernel with two iterations is enough to make recovery deterministic at the level needed for accuracy.
Figure 6: Stored vs. recovered 3×3 performance matrices on synthetic classification (mean ACC over 8 seeds; range [0, 1]). Cell (i, j) is performance on task j after training through task i.
15
KWTA
stored (oracle)
KWTA
recovered
0.5
0.690
0.441
0.000
T0
T1 Evaluated Task
T2
0.1
0.506
0.0
0.2 After T2
0.690
0.441
0.000
T0
T1 Evaluated Task
T2
0.1 0.0
0.417
0.265
0.001
T0
T1 Evaluated Task
T2
FTN-Slow stored (oracle)
FTN-Fast
0.099
T0
0.255
T1 Evaluated Task
0.001
T2
0.1 0.0
After T1
0.086
0.4
After Training
0.002
0.039
MSE (clamped at 0.5)
0.3 0.2
After T2
After T0 After Training
0.033
0.5
0.4 MSE (clamped at 0.5)
After Training
After T1
0.002
0.3
0.043
0.2 After T2
0.151
T0
0.616
T1 Evaluated Task
0.060
0.3
0.139
0.2 After T2
0.0
0.145
0.164
0.170
T0
T1 Evaluated Task
T2
stored (oracle)
FTN-Slow
0.212
T2
0.1 0.0
recovered
0.5
recovered
0.5 After T0
0.1
After T1
0.4
(b) KWTA-only (reg): with S=10 steps at ηm =0.2, the regression configurer has more time per call than the classification one (S=1 at ηm =1.0), so the recall column does not collapse the way it did on classification, the residual loss on regression is dominated by training-time overlap, not by recovery error.
(a) NoMask (reg): off-diagonal MSE saturates at the colorbar cap, mirroring the classification picture, the single shared subnetwork cannot retain incompatible regression targets. FTN-Fast
0.3
0.001
After Training
0.272
0.3
0.000
0.2 After T2
After T1
0.003
MSE (clamped at 0.5)
After T1
0.4
0.4
After T0
After T0
0.002
After T1
0.034
0.4 0.3
0.002
0.2 After T2
0.099
0.285
0.001
T0
T1 Evaluated Task
T2
0.1 0.0
0.5
0.1 0.0
After T0
0.030
After T1
0.070
0.4 MSE (clamped at 0.5)
0.3
0.000
0.001
MSE (clamped at 0.5)
0.506
0.2 After T2
After T0
0.4 After Training
After T1
0.001
MSE (clamped at 0.5)
After Training
After T0
0.5
MSE (clamped at 0.5)
recovered
0.5
After Training
NoMask
0.5
0.001
MSE (clamped at 0.5)
stored (oracle)
After Training
NoMask
After T0
0.3
0.058
0.2 After T2
0.121
0.092
0.053
T0
T1 Evaluated Task
T2
0.1 0.0
(d) FTN-Slow (reg): smallest off-diagonal MSE among adaptive methods on both panels; the fine-grained spatial diffusion produces the tightest neuron-set separation across tasks, and the 10-step regression configurer recovers it accurately.
(c) FTN-Fast (reg): stored and recovered matrices both stay close to the FixedMask oracle (max prior-task MSE around 0.13).
Figure 7: Stored vs. recovered 3×3 performance matrices on synthetic regression (mean MSE over 8 seeds; clamped at 0.5, lower is better, → on the colorbar denotes saturation).
16