TOOD: Task-Aware Out-of-Distribution Score Calibration for Continual Learners

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

TOOD: Task-Aware Out-of-Distribution Score Calibration for Continual Learners

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

machine learning, deep learning, neural networks

Preprint.

TOOD: TASK -AWARE O UT- OF -D ISTRIBUTION S CORE C ALIBRATION FOR C ONTINUAL L EARNERS Mostafa ElAraby DIRO, Mila - Quebec AI Institute Université de Montréal Montreal, Canada {elarabim}@mila.quebec

Samer B. Nashed DIRO, Mila - Quebec AI Institute Université de Montréal Montreal, Canada

arXiv:2607.29592v1 [cs.CV] 31 Jul 2026

Liam Paull CIFAR AI Chair DIRO, Mila - Quebec AI Institute Université de Montréal Montreal, Canada

A BSTRACT The primary challenge of continual learning (CL) systems is to learn new tasks while remaining performant on previously learned tasks. A similarly important though less well-studied aspect of CL systems is their ability to identify inputs that are unlikely to come from within the set of tasks the system has already encountered, often called out-of-distribution (OOD) detection. This paper presents several findings related to the dynamics of OOD detection in CL systems, causes of performance degradation over time which we call OOD forgetting (OODF), and proposed mitigation strategies for OODF. Chiefly, we find the unintuitive result that OODF is only weakly anti-correlated with classification performance on previous tasks, suggesting that the underlying mechanisms producing OODF are distinct. Moreover, this effect is observed for both energy-based and feature-based OOD detection methods. Energy-based detectors suffer a drop in logit scale as additional tasks are learned, which we term the Confidence Gap, while feature-based detectors also degrade under a complementary effect we call Manifold Crowding. Motivated by these observations, we propose Task-Aware OOD Score Calibration (TOOD), a training-free post-hoc method that decomposes logits into pertask energy scores and re-calibrates them using replay-buffer statistics. Experiments on CIFAR-10, CIFAR-100, and a 100-task ImageNet-1K stream show that TOOD improves OOD detection performance over uncalibrated energy in most settings and ranks first or second in eight of ten CIFAR configurations under 3-seed mean±std evaluation, with the largest gains when the confidence gap is most severe. These results suggest that a portion of OOD deterioration in continual learning arises from score miscalibration rather than from a complete loss of discriminative structure.

I NTRODUCTION

Reliable deployment of a Continual Learning (CL) system requires two abilities: 1) retaining performance on previously learned tasks and 2) recognizing when a test input falls outside the set of learned classes. The CL literature has made substantial progress on the first objective through regularization (Kirkpatrick et al., 2016), replay (Rebuffi et al., 2017; Chaudhry et al., 2019b), custom architectures (Rusu et al., 2016; Serrà et al., 2018) and distillation-based methods (Buzzega et al., 2020; Li & Hoiem, 2016). The second objective, robust out-of-distribution (OOD) detection across the learning stream, has received much less direct attention. Robust OOD detection requires stable data distributions and calibrated output scores (Hendrycks & Gimpel, 2017; Yang et al., 2021), assumptions that CL violates through continuous parameter updates. CL systems further exacerbate these challenges via two phenomena. First, the semantic boundary of what constitutes OOD data is a moving target: as new tasks are introduced, regions of the input space previously considered unknown are absorbed into the indistribution (ID) manifold. Second, as the model accommodates these new classes, the classifier head continuously expands, permanently altering the absolute scale and geometry of the output space. Such OOD separability degradation represents a silent, critical failure mode in continual learning (CL) systems. While standard metrics evaluate how well a system remembers past classes, they fail to capture its ability to identify novel, unseen data—a fundamental safety requirement for the responsible real-world deployment of lifelong learning systems. 1

Preprint.

This leads to the following research questions: RQ1: Are declines in OOD detection and classifier accuracy driven by different underlying mechanisms? RQ2: What underlying mechanisms drive the progressive deterioration of OOD detection during CL? RQ3: Can this deterioration be mitigated via a training-free, post-hoc intervention? To address these questions in light of the above challenges, we first measure out-of-distribution forgetting (OODF) for various CL and OOD detection methods in order to uncover the underlying mechanisms driving OODF. Our empirical results demonstrate that OODF is not merely a byproduct of forgetting old tasks, but a distinct failure of the model’s confidence calibration as its output capacity expands. A CL system that appears ‘well-preserved’ in its classification performance can simultaneously become overconfident on novel inputs. Specifically, our analysis points to two recurring mechanisms behind this deterioration. First, as the CL system is presented with new tasks and the classifier head expands, the logits associated with older tasks tend to shrink in absolute scale relative to the current task; we refer to this task-age-dependent score drift as the confidence gap. Second, as more classes are introduced, the latent feature space becomes more densely occupied, reducing the margin available to distance-based detectors; we refer to this as manifold crowding. Together, these observations suggest that OOD forgetting is not fully characterized by standard accuracy-based forgetting metrics (see Section 3). Existing evaluation protocols measure this degradation only at the final checkpoint, missing how detection erodes over the task stream. Motivated by this diagnosis, we propose Task-Aware OOD Score Calibration (TOOD), a training-free, post-hoc method that decomposes logits into per-task energy scores and re-calibrates those scores using task-wise ID statistics. Requiring only a single forward pass over a small calibration memory and no gradient updates, TOOD leaves the underlying CL algorithm unmodified and introduces no optimization overhead. Across several experiments on CIFAR-10 and CIFAR-100, TOOD reduces OODF compared to baselines in the majority of cases and yields its largest gains when score drift is severe and shows less benefit when the baseline already stabilizes output scale, indicating that it is a relatively effective strategy for addressing this phenomenon. Under 3-seed mean±std evaluation, at least one TOOD variant ranks first or second in eight of the ten CIFAR configurations; the two misses occur where the confidence gap is smallest and the gaps to the next detector are within seed variation. Overall, this paper presents three contributions. First, we formalize OOD Forgetting and introduce metrics that track OOD detection across the full learning trajectory rather than just end-state performance. Second, we identify two mechanisms that produce OODF: the confidence gap (task-age logit-scale drift) and manifold crowding (progressive latent space saturation), and we demonstrate these recur across a number of state-of-the-art CL and OOD detection methods. Finally, we propose TOOD, a training-free, post-hoc method that calibrates per-task energy scores using task-wise ID calibration statistics, requiring only a single forward pass over that calibration memory and no retraining.

R ELATED W ORK

OOD detection methods broadly assume a fixed model and a static data distribution (Hendrycks & Gimpel, 2017; Yang et al., 2021). Post-hoc output-based methods assign anomaly scores from model outputs: MSP (Hendrycks & Gimpel, 2017) uses the maximum softmax probability, ODIN (Liang et al., 2018) applies temperature scaling and input perturbation, and energy scoring (Liu et al., 2020) aggregates class logits into a scalar. Feature-based methods exploit geometric separation in representation space (Lee et al., 2018; Sun et al., 2022), while activation-shaping methods such as ReAct and ASH (Sun et al., 2021; Djurisic et al., 2023) suppress outlier activations before scoring. CL researchers have also explored gradient-based replay constraints (Lopez-Paz & Ranzato, 2017; Chaudhry et al., 2019a), synaptic consolidation (Zenke et al., 2017), and output-layer bias correction or weight alignment (Zhao et al., 2019; Wu et al., 2019). These methods improve old/new class balance for classification, but they are not designed to preserve OOD score separation over time or to control the feature-space geometry on which many OOD detectors depend. Moreover, all of these approaches assume a frozen backbone and a stable score distribution, which is not the case in continual learning, where parameters update sequentially and the set of known classes grows. OpenCIL (Miao et al., 2026) documents that standard CL methods struggle to reject OOD samples and benchmarks this problem across incremental steps; CLOM (Kim et al., 2022) uses OOD detection for task inference. A growing line of work studies OOD detection and CL jointly Gupta et al. (2025), but most of it adopts a different definition of OOD data. Gupta et al. (2026) use OOD scorers for task identity inference and open world recognition so that detection replaces a replay buffer. Separately, Harun et al. (2025) study how neural collapse affects OOD detection and generalization. The continual works largely define OOD data as derived from unknown or future classes. Our setting is complementary. We define OOD as data semantically disjoint from every learned task, and ask how the ability to reject it degrades along the learning trajectory, correcting the dominant score level component post hoc. Our work complements OpenCIL by moving from aggregate benchmarking to an explicit analysis of OOD forgetting: we formalize task-wise deterioration over the CL trajectory and use it to disentangle the mechanisms driving performance loss. In non-CL settings, Kumar et al. (2022) show that full fine-tuning can hurt OOD generalization relative to linear 2

Preprint.

probing, while Ming & Li (2024) show that the effect of fine-tuning on OOD detection can be strongly score- and method-dependent in vision-language models. We extend that line of inquiry to the continual setting, identifying taskdependent logit-scale drift and feature-space saturation as primary drivers of OOD deterioration in our experiments. Work on score calibration has shown that modern networks are overconfident and temperature scaling improves calibration on i.i.d. data (Guo et al., 2017); Minderer et al. (2021) find that calibration degrades under distribution shift. Crucially, any monotonic scalar transformation applied to all test samples preserves score ranking and therefore cannot improve AUROC (Fawcett, 2006), motivating TOOD’s per-task score decomposition rather than global calibration.

OOD F ORGETTING

Measuring OODF and evaluating potential solutions requires us to formalize the OOD detection problem in CL systems and then establish several metrics to assess OODF throughout the task stream. First, however, we briefly review some basic CL formalism. Given a model fθt trained on tasks T0 . . . Tt , we denote the accuracy of fθt on a test set from task Ti after learning Tt as At,i . Using this measure, task forgetting for task Ti after the learner has trained on N + 1 tasks is defined as Fi = max At,i − AN,i . (1) t∈{i,...,N −1}

Standard CL benchmarks such as Split-CIFAR-10 (Krizhevsky, 2009), Split-CIFAR-100 (Krizhevsky, 2009), and PN −1 Split-ImageNet-1k (Deng et al., 2009) are typically evaluated using Fi or its aggregate F̄ = N 1−1 i=1 Fi . This framing implicitly assumes that preserving accuracy also preserves the ability to detect inputs from outside the learned distribution. We show empirically that this assumption does not hold: Equation (1) captures classification robustness but is insensitive to OOD forgetting, which motivates the metrics we develop next. Consider a CL scenario with disjoint tasks T0 , . . . , TN . Each task Tt introduces dataset Dt = {(x, y)} from distribution Pt (x, y) over class set Ct , with Ci ∩ Cj = ∅ for i ̸= j. Let θt denote model parameters after training on Tt . After training on TN , the final task, we would like the model to both (1) correctly classify ID samples from any seen task, and (2) detect OOD samples DOOD that possess no semantic overlap with any Ct . Objective (1) is captured by Fi (Equation (1)); we now develop a measure for objective (2) by extending the evaluation framework of OpenCIL (Miao et al., 2026) to explicitly quantify task-wise OOD forgetting over the full CL trajectory. Let S : X → R be a score function that assigns a real-valued label to data items x ∈ X . By convention, we interpret scores closer to −∞ to indicate a lower probability of x being an ID sample and scores closer to +∞ to indicate a higher probability of x being an ID sample. We define the OOD detection performance of score function S(x; θt ), with respect to task Ti under model θt as AUROC(Ti | θt ) = Pr[S(x+ ; θt ) > S(x− ; θt )], where x+ ∼ Di is a random ID sample and x− ∼ DOOD is a random OOD sample. We define OOD forgetting w.r.t. task Ti as the drop in OOD detection performance between t = i and t = N : Di = AUROC(Ti | θi ) − AUROC(Ti | θN ). The average OOD forgetting summarizes this over all tasks: N −1 1 X Di . (2) Davg = N i=0 A positive forgetting value, Di > 0, indicates that the model’s ability to distinguish between ID samples from task Ti and OOD samples that are not present in any task has deteriorated since the model was first trained to perform task Ti ; if Di = 0, performance has remained constant though individual data items may be classified differently; and Di < 0 indicates the model has improved (unlikely). Di is distinct from Fi (Equation (1)): Fi measures erosion of decision boundaries between known classes, while Di measures erosion of the boundary between all known classes and the unknown. Both are driven by sequential gradient updates but respond to different aspects of the model’s geometry. CL systems are typically evaluated for OOD robustness only at t = N , against a fixed OOD dataset, which does not allow investigation of the process of degradation over sequential learning. Following the step-wise evaluation of Miao et al. (2026), we formalize average incremental AUROC, the OOD analogue of average incremental accuracy (Rebuffi et al., 2017). After learning task Tt , the checkpoint θt is evaluated on all tasks seen thus far, T0 . . . Tt . The per-task AUROC is averaged over the |O| OOD evaluation datasets O = {d1 , . . . , d|O| }: P 1 AUROC(Ti | θt ) = |O| d∈O AUROCd (Ti | θt ). The aggregate trajectory metric is Avg AUROC =

N t 1 X 1 X AUROC(Ti | θt ). N + 1 t=0 t + 1 i=0

(3)

We report average incremental AUROC (Equation (3)), Davg (Equation (2)), and false positive rate at 95% ID recall (FPR@95) as complementary metrics. 3

Preprint.

Figure 1: Feature drift explains accuracy forgetting but not OOD forgetting. Each point is a CL method on CIFAR10 (N = 5 tasks). Left: Feature drift (1 − CKA) vs. OOD deterioration (Davg ) shows no significant relationship; r = 0.365, p = 0.270. Right: Feature drift vs. classification forgetting (F̄ ); r = 0.880, p < 0.001. Features drift appears to cause classification forgetting but not OOD detection, suggesting the latter is driven by distinct mechanisms. 3.1

M ECHANISMS BEHIND OOD F ORGETTING

Accuracy forgetting is governed by decision boundary preservation, and CL methods seek to protect the relative class rankings needed to separate known classes. When they succeed, AN,i ≈ Ai,i and Fi ≈ 0. In contrast, all OOD detection methods, whether they compute scores from logit energy (Liu et al., 2020), feature distances (Lee et al., 2018; Sun et al., 2022), or activation statistics (Sun et al., 2021; Djurisic et al., 2023), rely on score distribution preservation. This creates a mismatch between important quantities for OOD detection methods and CL methods. Figure 1 illustrates this mismatch, showing that feature drift, measured using centered kernel alignment (CKA) (Kornblith et al., 2019) across CL methods on CIFAR-10 (N = 5), is highly correlated with accuracy forgetting, but much less so with OODF. This is not surprising considering that under the hood, energy-based OOD detection methods operate on the absolute scale of the logit vector, and feature-based detectors rely on unoccupied geometric margins. However, standard CL techniques do not constrain these properties. During task acquisition, as the output head expands, old-task logits shrink in absolute scale and the latent space fills with new embeddings. While the relative decision boundary between classes remains intact, preserving classification accuracy, the absolute metrics required to detect OOD inputs are altered. Moreover, methods with low feature drift (WA, iCaRL) still exhibit significant OODF, suggesting additional sources of degradation in the output scores, consistent with Ramasesh et al. (2021), who show that forgetting in deep networks is primarily an output-layer phenomenon. Manifold Crowding. Feature-based detectors such as MDS (Lee et al., 2018) rely on geometric margin, the distance between an OOD sample and the nearest ID cluster in representation space. During CL, we observe a degradation mechanism we term manifold crowding. As new tasks arrive, their embeddings colonize previously unoccupied latent regions, compressing the margin available for distance-based detection even when task representations remain intact. On a 5-task CIFAR-10 stream, the median nearest-neighbor distance from OOD samples to the ID manifold decreases by approximately 28% while CKA analysis confirms that feature drift for old tasks remains low (see Appendix B.1 for figures). This disassociation, shrinking margins without significant representation drift, shows that manifold crowding is driven by the occupancy of latent space by new classes rather than by the movement of old ones. This phenomenon is conceptually related to the superposition hypothesis, where limited representational capacity forces multiple features to share latent directions (Elhage et al., 2022). In our setting, however, the effect is temporal. Continual class growth progressively occupies regions that previously served as OOD margin, rather than having overlapping features in a jointly trained neural network that are disjoint from OOD representation. TOOD does not directly address manifold crowding. It operates on output logits and cannot restore geometric margin lost in feature space. We confirm this by isolating manifold crowding and the confidence gap in a toy task with fully known geometry (16 Gaussian-blob classes on a sphere, with OOD at the interior origin; Appendix C). Growing the classifier head while holding the data fixed exposes only the confidence gap, which TOOD re-centres for up to +5.1 Avg AUROC. However, forcing class overlap to create manifold crowding collapses the geometric margin, which TOOD cannot recover. Moreover, a feature-space analog of TOOD that applies the same task-wise normalization in distance space rather than logit space fails to recover separability (−0.76 Avg AUROC), since re-centring scores 4

Preprint.

Figure 2: Continual learners retain classification accuracy on old tasks while losing OOD separability. Left: Task 0 classifier acc. (blue) remains high, while uncalibrated energy-based OOD AUROC (red dash) drops by 35 points. TOOD calibration (green) recovers most of that gap. Right: The mean ID energy score (blue) drifts from its T0 value toward the OOD score (red dash), collapsing the confidence gap at T3 . TOOD (green) re-centers the task-specific score distribution throughout the stream. Both figures show iCaRL on a 5-task CIFAR-10 stream, tested on Task 0.

cannot restore margin that newly added classes have already consumed (Appendix B.2). Manifold crowding therefore represents a distinct and open challenge: improving OOD detection for feature-based methods in CL systems requires complementary techniques such as representation regularization or margin-preserving replay objectives. Additional results on the 100-task ImageNet-1K stream are reported in Appendix E.1. The Confidence Gap. The confidence gap is the dominant mechanism behind OOD forgetting for energy-based detectors. As new tasks arrive, gradient updates prioritize the current task’s classes, causing old-task logits to shrink in absolute scale relative to newer tasks. Replay methods illustrate this clearly: rehearsing k exemplars per class preserves the relative ordering of old-task logits but imposes no constraint on the absolute magnitude of the expanding logit vector. Gradient updates for new classes push current-task logits to higher scale, creating a confidence gap even when old-task accuracy is preserved. The confidence gap is summarized by the expression Ex∼Di [S(x; θN )] ≪ Ex∼Dj [S(x; θN )] for i < j, even when Ex∼Di [A(x; θN )] ≈ Ex∼Di [A(x; θi )]. Figure 2 (right) illustrates the confidence gap. The mean ID energy for Task 0 drops from approximately 18 at T0 to 6 by T4 , converging with the OOD scores anchored in the 6–8 range. Consequently, a global threshold τ established at T0 systematically invalidates ID samples from earlier tasks. Figure 3 demonstrates that this trend generalizes across four CL methods. In every case, the ID energy decays into the OOD region, and TOOD successfully restores separability. Crucially, because this failure mode occurs in the output scores rather than the latent representations, a post-hoc intervention can substantially recover the detection margin without modifying the underlying feature extractor.

TASK -AWARE OOD S CORE C ALIBRATION

This section motivates TOOD by showing why scalar post-hoc calibration cannot repair the confidence gap and then introduces TOOD’s per-task energy formulation. AUROC depends on the relative ranking of scores. Any monotonic transformation applied to all samples preserves rank and cannot change AUROC. Since ID and OOD samples are indistinguishable at test time, a per-task scalar recalibration applied equally would preserve their ordering. Improving AUROC thus requires a transformation that moves ID and OOD scores in different directions. Per-task energy decomposition achieves this. ID samples activate one task’s energy strongly while OOD samples activate none, so the max operation selects different task channels for different sample types—a differential transformation by construction. Operating purely on logits is a deliberate design choice. It allows TOOD to be training-free and architecture-agnostic, applicable to any Class-Incremental Learning (CIL) backbone with an expanding classifier head. See Appendix E.2 for details on ablations between ResNet and ViT-B/16 backbones (BatchNorm versus LayerNorm), where TOOD raises Avg AUROC by +3.5 on iCaRL (CIFAR-10) and by up to +7.4 on the longer CIFAR-100 stream. 4.1

P ER -TASK E NERGY D ECOMPOSITION

SN Let h(x) ∈ R|Cseen | be the logit vector, where Cseen =P t=1 Ct . We use the sign-reversed energy convention (Liu et al., 2020), so higher scores indicate ID: E(x) = log c∈Cseen exp(hc (x)). This score drifts as old-class logits shrink. 5

Preprint.

Figure 3: TOOD restores ID/OOD separability across CL methods. Each column corresponds to a different CL method. Top row: Uncalibrated Task 0 ID energy scores (blue) converge toward OOD scores (red) as new tasks are learned, eroding the detection margin. Bottom row: After TOOD’s per-task normalization, calibrated ID scores (green) maintain a clear separation from OOD scores throughout the task stream. Boxes span the interquartile range; whiskers mark the 5th–95th percentile. Shaded red regions indicate task steps where the ID and OOD medians are inverted. TOOD replaces this with N per-task energies: Et (x) = log

exp hc (x) ,

(4)

c ∈ Ct N yielding E(x) = [Et ]N t=1 ∈ R . An ID sample from task Ti activates Ei strongly and all other Et weakly; an OOD sample activates no task dominantly. This structure persists even after the global score collapses.

4.2

TASK -W ISE S CORE N ORMALIZATION

Per-task energies still suffer from the confidence gap. Ei (x) for an old task has lower absolute value than EN (x) for the current task. TOOD normalizes each task’s energy using task-wise ID calibration statistics. We write Bt for the calibration set (the replay buffer when available, or a small held-out ID split otherwise as in regularization-based methods like LwF (Li & Hoiem, 2016)). Two variants are provided. Mean Shift. For each task Tt , the buffer mean is µt = 1/|Bt |

x∈Bt Et (x) and the corrected energy is

Etms (x) = Et (x) + (µref − µt ),

(5)

where µref is the calibration-set mean of the most recent task TN . Mean shift corrects location drift but assumes stable spread; when calibration sets are small and distributions heavy-tailed, the mean is sensitive to outliers. Robust Anchor. Robust anchor substitutes the median and Median Absolute Deviation (MAD) (Huber, 2011): Etrob (x) =

Et (x) − ẽt [ · MADref + ẽref , [t MAD

(6)

[ t = medianx∈Bt Et (x) − ẽt , scaled to normal consistency. The reference where ẽt = medianx∈Bt (Et (x)) and MAD [ ref ) comes from TN ’s calibration set. We call this TN the reference task: its statistics set only the location (ẽref , MAD and scale (the units) of the normalized scores, not their ordering. Substituting a different reference task (e.g., the oldest \ ref ), which rescales and shifts task, a max-spread task, or the mean of all task statistics) merely replaces (ẽref , MAD every normalized channel by the same positive factor and offset. This common affine map commutes with the max 6

Preprint.

in Equation (7), so it preserves the ranking of scores. Because AUROC depends only on that ranking, it is unchanged (largest deviation 0.00015 across our full sweep; see Appendix D.5 for details). We keep TN as the default only because it gives an interpretable, current-scale decision threshold. Because the median resists outlier contamination from representation drift, robust anchor is the primary variant in all reported experiments. To test the effect of recentering, we ablate against the same per-task split but only rescale each task’s energy by its own spread, with no common reference point. This actually lowers Avg AUROC by 3.6 points below uncalibrated energy, while TOOD improves it by 3.1 (Appendix D.2). The recentering, not task-awareness on its own, is what drives the improvement. 4.3

TASK -N ORMALIZED E NERGY S CORE

After normalization, TOOD computes the final detection score as the maximum normalized energy across all tasks: STOOD (x) =

max

t∈{1,...,N }

Etnorm (x),

(7)

where Etnorm denotes whichever of Equation (5) or Equation (6) is selected. Importantly, TOOD does not require the task identity of a test sample. The task index in Equation (7) is not provided externally; instead, all task-wise energies are computed and the final score is obtained by taking the maximum over normalized task channels. TOOD only uses the training-time task-to-class partition, i.e., which logits correspond to classes introduced in each incremental task. This partition is already present in standard CIL, where the classifier head expands as new classes arrive. Thus, TOOD reuses training-time class grouping but does not assume task-incremental inference. It is also robust to a finer class partition: splitting each task’s classes into smaller nonempty groups reduces Avg AUROC by only 0.5 points on average across CIFAR-10 and CIFAR-100, five continual-learning methods, and three seeds (Appendix D.3). An ID sample activates one task’s energy dominantly; after normalization, that channel is corrected upward and STOOD is high. An OOD sample activates no channel strongly, so all normalized energies remain mediocre and STOOD is low. Because the normalization happens inside the max, the effective transformation differs for ID and OOD samples, allowing TOOD to alter AUROC unlike ordinary scalar calibration (see Appendix B.3 for full score distributions). Algorithm 1 summarizes the procedure. The calibration phase requires one forward pass P over |B| = t |Bt | task-wise calibration samples and runs once after each task. Inference adds O(N ) LogSumExp operations per sample. Both phases are training-free. Optional heuristic margin term. For ID samples the gap between the top two normalized energies is often large; for OOD samples it is usually smaller. A simple heuristic is therefore to add a margin term: norm norm norm Sλ (x) = E(1) (x) + λ E(1) (x) − E(2) (x) , (8) norm norm where E(1) ≥ E(2) are the two largest normalized energies. When λ = 0, this reduces to Equation (7). For λ > 0, ID samples with broad margins are pushed to higher scores. Because this term is heuristic rather than theory-derived, we treat it as optional and analyze its sensitivity in Appendix D.1; all main experiments use a fixed λ = 0.5. A sweep shows methods with a large confidence gap (iCaRL, DER) peak near λ = 0.5 while methods with native output-head correction (BiC) prefer λ = 0, so 0.5 is a robust default that requires no per-method tuning.

R ESULTS

CL encompasses three canonical settings (Van de Ven & Tolias, 2019): task-incremental learning (TIL), where a task identifier is available at test time; domain-incremental learning (DIL), where the input distribution shifts but the label space remains fixed; and class-incremental (CIL), where new classes arrive sequentially and no task identifier is provided at inference. We focus on CIL because it maximally stresses OOD detection: each new task both expands the classifier head and redefines the semantic boundary between ID and OOD data, producing the strongest confidence gap and the most acute manifold crowding. TIL, by contrast, sidesteps output-head interference through task-conditioned prediction, while DIL keeps the label space fixed and thus introduces no head expansion. Across datasets, TOOD recovers OOD performance lost in continual learning, especially under severe logit-scale drift; extension to task- and domain-incremental CL remains future work. 5.1

E XPERIMENTAL S ETUP

Datasets. For training the CL systems we primarily use CIFAR-10 (Krizhevsky, 2009) (N = 5 tasks, 2 classes each) and CIFAR-100 (Krizhevsky, 2009) (N = 10 tasks, 10 classes each). We also use ImageNet-1K (Deng et al., 2009) with N = 100 tasks, 10 classes each. For evaluating OOD detection, we use the OOD datasets from OpenOOD (Zhang 7

Preprint.

Algorithm 1 TOOD: Task-Aware OOD Score Calibration Require: Frozen model fθ , task-wise calibration sets {Bt }N t=1 , variant v ∈ {ms, rob}, margin λ ≥ 0 1: R ← ∅ 2: for t = 1 to N do ▷ Calibration phase 3: Collect logits {h(x)}x∈Bt using fθ 4: Compute task energies {Et (x)}x∈Bt via Equation (4) 5: if v = ms then 6: R[t] ← µt ▷ Calibration-set mean 7: else if v = rob then [ t) 8: R[t] ← (ẽt , MAD ▷ Calibration-set median and MAD 9: Rref ← R[N ] ▷ Reference statistics from the most recent task 10: for each test sample x do ▷ Inference phase 11: for t = 1 to N do 12: Compute Et (x) via Equation (4) 13: Compute Etnorm (x) using R[t] and Rref via Equation (5) or Equation (6) norm norm 14: Let E(1) (x) and E(2) (x) be the largest and second-largest normalized energies norm norm norm 15: STOOD (x) ← E(1) (x) + λ E(1) (x) − E(2) (x) 16: return {STOOD (x)}x

▷ OOD scores for all test samples

et al., 2024), and we follow the Near-Far OOD labeling from OpenOOD with respect to CIFAR-10, CIFAR-100, and ImageNet-1k. For CIFAR-10/100, near-OOD datasets are CIFAR-100/TinyImageNet, and far-OOD datasets are MNIST (Yann, 2010), SVHN (Netzer et al., 2011), Textures (Cimpoi et al., 2014), and Places365 (Zhou et al., 2018). For ImageNet-1k, near-OOD datasets are SSB-hard/NINCO (Bitterwolf et al., 2023), and far-OOD are iNaturalist (Huang & Li, 2021), Textures (Kylberg, 2011), and OpenImage-O (Wang et al., 2022). To generate replay buffers Bt for TOOD calibration, we use a randomly generated class-balanced buffer (Rebuffi et al., 2017), CL Methods and Curricula. We test TOOD on five representative CL methods spanning three paradigms: distillation / regularization: LwF (Li & Hoiem, 2016); replay: iCaRL (Rebuffi et al., 2017), DER (Buzzega et al., 2020); output correction: BiC (Wu et al., 2019), WA (Zhao et al., 2019). All methods are implemented via the Avalanche library (Lomonaco et al., 2021) using the hyper-parameters recommended by the authors. We train each CL method following the class-label curriculum established by prior CIL benchmarks (Rebuffi et al., 2017; Wu et al., 2019; Zhao et al., 2019). These curricula assign classes in label-index order with equal cardinality per task, with the goal that each task introduces enough classes to produce a meaningful shift in the output head while keeping per-task training cost manageable. CIFAR-10 is split into N = 5 tasks of 2 classes, CIFAR-100 into N = 10 tasks of 10 classes, and ImageNet-1K into N = 100 tasks of 10 classes. All models are trained for 170 epochs (SGD, momentum 0.9, batch size 128, Multi-Step LR from 0.1 with γ = 0.1 at milestones [60, 100, 140]). CIFAR-10/100 datasets use a ResNet-32 (He et al., 2016) backbone with a single expanding linear head, while ImageNet-1k uses a ResNet-18. OOD Detectors. We compare TOOD against eight post-hoc OOD detection baselines, spanning three detection paradigms: output-based: MSP (Hendrycks & Gimpel, 2017), Energy (Liu et al., 2020); Dice (Sun & Li, 2022), ADASCALE (Regmi, 2025); activation shaping: ASH (Djurisic et al., 2023); feature-based: ViM (Wang et al., 2022), MDS (Sun et al., 2022), NNGuide (Park et al., 2023). All methods are implemented via the OpenOOD benchmark (Zhang et al., 2024). The above baselines are compared against both TOOD variants, Energy + Mean Shift and Energy + Robust Anchor. We also compare TOOD against bi-directional energy regularization (BER) from OpenCIL (Miao et al., 2026), a finetuning method for improving OOD detection in CIL. Code. 5.2

Code is available at https://github.com/mostafaelaraby/tood-continual-ood.

M AIN R ESULTS

Table 1 summarizes the main results of our comparisons between TOOD and other OOD detectors. With 3-seed meanplus-std estimates, at least one TOOD variant ranks first or second in eight of ten dataset and CL method combinations, ranked by Avg AUROC. Many other methods lack consistency, for example ASH fluctuates across CL methods (e.g., 65.9 on iCaRL but 57.7 on DER). Some feature-based methods, such as MDS, which are typically strong in non-CL settings struggle relative to TOOD (e.g. 58.4 vs. 66.5 on iCaRL and CIFAR-10), corroborating our manifold crowding diagnosis. 8

Preprint.

Table 1: Main results across 2 benchmarks. Columns are CL methods; rows are OOD detectors. We report average CIL accuracy, Avg AUROC (Equation (3)) (↑), and Avg FPR@95 (↓), with AUROC and FPR@95 averaged over Near- and Far-OOD. Values are mean±std over 3 seeds. Bold marks every entry within one std of the best per metric subcolumn. (a) CIFAR-10 (N = 5) OOD Method

iCaRL AUC ↑ FPR ↓

BiC AUC ↑ FPR ↓

DER AUC ↑ FPR ↓

WA AUC ↑ FPR ↓

LwF AUC ↑ FPR ↓

Avg CIL Acc. (↑)

80.9 ±0.3

77.5 ±0.2

72.7 ±0.2

80.2 ±0.9

67.9 ±0.1

MSP Ash VIM Dice ADASCALE MDS NNGuide Energy (Uncalib.)

63.0 ±1.4 82.4 ±0.8 68.3 ±0.6 77.0 ±3.6 69.0 ±0.6 71.1 ±1.0 74.2 ±2.1 67.6 ±8.1 66.3 ±0.6 72.3 ±0.4 65.9 ±1.7 71.8 ±1.1 61.9 ±2.1 71.5 ±2.1 57.7 ±1.0 79.3 ±0.8 62.2 ±1.7 73.5 ±0.3 64.7 ±1.2 74.8 ±0.7 63.9 ±1.2 81.7 ±1.5 65.7 ±1.5 73.5 ±1.1 61.1 ±0.3 77.8 ±0.9 77.9 ±2.0 61.4 ±3.5 72.9 ±1.4 70.1 ±1.5 61.5 ±1.5 72.7 ±1.0 54.9 ±0.7 74.9 ±1.3 53.7 ±1.4 79.5 ±1.0 59.4 ±0.8 72.5 ±0.3 62.9 ±2.9 75.3 ±0.9 58.0 ±2.1 85.9 ±1.7 58.5 ±2.3 79.7 ±3.3 52.6 ±1.2 85.1 ±0.7 62.6 ±0.6 81.5 ±1.2 63.9 ±1.5 81.2 ±2.6 58.4 ±0.1 85.5 ±1.5 61.1 ±1.1 84.3 ±0.9 59.3 ±3.0 86.6 ±2.9 54.2 ±1.4 89.8 ±2.8 53.7 ±1.8 90.0 ±2.6 50.1 ±1.3 85.4 ±1.2 48.8 ±1.2 79.0 ±1.0 43.4 ±0.6 86.9 ±0.3 58.4 ±1.3 74.0 ±2.3 55.8 ±2.6 79.1 ±1.1 65.9 ±1.5 77.6 ±0.9 67.2 ±1.0 69.1 ±1.1 61.2 ±0.7 74.1 ±0.4 78.1 ±2.3 59.5 ±4.5 72.2 ±0.5 70.4 ±0.8

Energy + Mean Shift 66.5 ±1.5 75.4 ±1.0 71.6 ±0.2 66.9 ±0.3 69.3 ±0.9 68.3 ±0.7 78.3 ±2.1 59.3 ±4.6 73.2 ±0.3 68.6 ±0.5 Energy + Robust Anchor 66.4 ±1.4 75.6 ±0.9 70.6 ±0.3 69.0 ±0.8 68.6 ±0.9 68.9 ±0.8 78.2 ±2.4 59.8 ±4.9 73.0 ±0.7 68.4 ±0.8 (b) CIFAR-100 (N = 10) OOD Method

iCaRL AUC ↑ FPR ↓

BiC AUC ↑ FPR ↓

DER AUC ↑ FPR ↓

WA AUC ↑ FPR ↓

LwF AUC ↑ FPR ↓

53.0 ±0.1

62.3 ±0.1

55.1 ±0.4

63.8 ±0.4

50.7 ±0.4

Avg CIL Acc. (↑) MSP Ash VIM Dice ADASCALE MDS NNGuide Energy (Uncalib.)

61.5 ±1.0 81.0 ±1.0 64.8 ±0.4 79.1 ±0.3 65.9 ±0.2 75.6 ±0.6 69.8 ±0.2 71.7 ±0.1 64.4 ±1.0 80.8 ±1.3 59.0 ±2.7 85.0 ±1.3 51.1 ±1.0 87.7 ±0.7 46.4 ±2.0 91.0 ±1.2 53.5 ±1.8 86.9 ±1.0 64.9 ±0.8 78.0 ±1.6 57.9 ±0.8 83.8 ±0.7 60.1 ±0.3 81.0 ±0.7 61.4 ±0.4 80.2 ±0.5 70.3 ±0.2 71.1 ±0.2 67.3 ±1.9 76.7 ±2.5 60.4 ±0.7 83.4 ±1.1 52.1 ±0.4 86.4 ±0.6 50.2 ±1.7 86.6 ±1.1 55.1 ±2.6 85.3 ±2.1 60.2 ±1.4 80.8 ±2.0 58.3 ±0.9 87.4 ±1.9 57.2 ±0.5 88.5 ±0.5 57.6 ±2.1 88.7 ±1.4 67.8 ±1.2 80.8 ±1.6 64.6 ±1.3 84.6 ±1.4 57.7 ±1.8 84.7 ±1.4 62.3 ±0.4 79.5 ±0.5 59.1 ±1.7 83.2 ±1.4 58.5 ±0.9 82.5 ±1.1 53.9 ±0.4 87.9 ±0.5 58.1 ±1.6 81.8 ±0.9 52.0 ±1.6 85.4 ±0.7 55.3 ±0.3 83.8 ±0.4 53.5 ±1.4 83.5 ±0.5 52.9 ±0.5 84.3 ±0.3 58.1 ±1.3 83.0 ±0.9 65.6 ±0.5 78.2 ±0.4 64.6 ±1.2 77.3 ±1.2 71.9 ±0.1 68.3 ±0.5 67.0 ±1.6 76.3 ±2.0

Energy + Mean Shift 58.4 ±1.0 82.5 ±0.7 68.4 ±1.3 74.7 ±1.1 67.1 ±0.5 75.8 ±1.2 71.7 ±0.2 68.8 ±0.9 66.7 ±1.7 77.4 ±2.1 Energy + Robust Anchor 58.9 ±1.0 82.4 ±0.5 68.8 ±1.6 74.7 ±1.8 66.1 ±0.9 76.1 ±1.1 71.5 ±0.1 69.4 ±0.9 66.3 ±1.6 77.8 ±1.8

Table 1 also provides more evidence for the importance of the confidence gap. BiC (Wu et al., 2019), which applies explicit output-head bias correction, reaches 67.2 AUROC on CIFAR-10, higher than DER (61.2) despite lower classification accuracy. iCaRL and DER stabilize logit values for replayed samples but impose no constraint on absolute scale, showing correspondingly lower AUROC. Moreover, TOOD delivers its largest improvements when logit-scale drift is severe. On CIFAR-10, Mean Shift improves DER by +8.1 AUROC (61.2 → 69.3) and BiC by +4.4 (67.2 → 71.6). Conversely, for methods that natively stabilize output scales (e.g., WA), the confidence gap is narrower, yielding marginal or statistically insignificant changes (e.g., WA on CIFAR-100 shifts 71.9 → 71.7). Thus, TOOD acts as a targeted first-order repair for score drift rather than a universal enhancement across all CL methods. Table 2 summarizes our results comparing TOOD to the most related existing method, BER; the post-hoc rows (Energy, TOOD) are 3-seed means while the 10-epoch BER rows are single-seed. While the two methods are in theory compatible, combining them does not consistently outperform the better of the two alone, and in five of eight metric columns a post-hoc TOOD variant outperforms all BER rows. Moreover, BER exhibits instability across scales: while it improves iCaRL on CIFAR-10, it degrades it on CIFAR-100 (58.1 → 57.7). Furthermore, TOOD requires no retraining, making it a simpler and cheaper option in most cases. 5.3

A BLATION S TUDIES

[ t ). In our experiments Replay Buffer Size. TOOD relies on a replay buffer to estimate per-task statistics (µt , ẽt , MAD we use a buffer size of B = 200 (CIFAR-10) and B = 700 (CIFAR-100) to contain samples across all tasks T0 . . . TN . Figure 4 shows TOOD performance over B. Performance is unstable for B < 20 but stabilizes between B = 50 and B = 100, with negligible gains thereafter. Thus, TOOD requires only a few dozen samples per task; our default values 200 and 700 are well above this threshold. We also tested iCaRL-style herding instead of random class-balanced 9

Preprint.

Table 2: Post-hoc Calibration (TOOD) versus 10-Epoch fine-tuning (BER). We report Avg AUROC (Equation (3)) (↑) and Avg FPR@95 (↓), each averaged over Near- and Far-OOD. Bold indicates 1st place per metric subcolumn. The post-hoc rows (Energy, TOOD) are means over 3 seeds and match Table 1; the 10-epoch BER and BER + TOOD rows are reported from a single seed. CIFAR-10 (N = 5) Method

CIFAR-100 (N = 10)

iCaRL BiC iCaRL BiC AUC ↑ FPR ↓ AUC ↑ FPR ↓ AUC ↑ FPR ↓ AUC ↑ FPR ↓

No Retraining (Zero-Epoch) Energy (Uncalib.) TOOD (Mean Shift) TOOD (Robust Anchor)

65.9 66.5 66.4

77.6 75.4 75.6

67.2 71.6 70.6

69.1 66.9 69.0

58.1 58.4 58.9

83.0 82.5 82.4

65.6 68.4 68.8

78.2 74.7 74.7

10-Epoch Retraining Required BER (Uncalib. Energy) BER + TOOD (Mean Shift) BER + TOOD (Robust Anchor)

73.5 74.3 73.6

67.1 68.1 70.6

69.5 68.0 66.1

69.1 69.6 77.6

57.7 58.4 55.9

84.0 84.4 86.3

68.5 66.9 68.7

74.1 74.9 75.7

calibration samples. The average change in Avg AUROC is −0.02 points over the five Table 1 CL methods and two CIFAR datasets, with no consistent gain, so random class-balanced calibration is sufficient for the median/MAD statistics used by TOOD.

Figure 4: Effect of buffer size B (CIFAR-10, N = 5 tasks). Average area under the receiver operating characteristic curve (AUROC) (↑, top) and OOD deterioration (↓, bottom) saturate quickly. The default B = 200 (dashed line) is well within the stable performance plateau, confirming the robustness of our results to buffer constraints.

C ONCLUSION

In this paper we show that standard CL evaluations focused on classification forgetting and feature drift do not predict a CL system’s ability to reject OOD inputs, and that this capability degrades substantially over the task stream. This mismatch indicates OOD forgetting happens via alternate mechanisms, of which we identify two: manifold crowding and the confidence gap, which describe i) how newly learned tasks consume feature-space margin and weaken distance-based separation and ii) the tendency for old-task logits to shrink in absolute scale relative to newer tasks, respectively. Motivated by these insights, we propose TOOD, a post-hoc, training-free algorithm that decomposes logits into per-task energy scores and re-calibrates them using task-wise ID statistics. We show that TOOD compares favorably to many OOD detection methods on a variety of CL datasets and CL training methods. Of course, TOOD does not prevent OOD forgetting outright. It relies on the training-time task-to-class grouping that CIL models already maintain, rather than any test-time task identity, together with a modest amount of ID calibration data, so its efficacy likely drops when that grouping is ambiguous or calibration data is scarce; the margin term is optional and can be disabled at no cost. Moreover, while it effectively re-calibrates energy-based OOD detectors, it cannot fix feature-space degradation associated with manifold crowding. In the future, we plan to further analyze manifold crowding and develop OOD detection methods for CL systems that mitigate this problem directly, for example through representation regularization or margin-preserving replay objectives. 10

Preprint.

R EFERENCES Julian Bitterwolf, Maximilian Müller, and Matthias Hein. In or out? fixing imagenet out-of-distribution detection evaluation. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 2471–2506. PMLR, 2023. URL https: //proceedings.mlr.press/v202/bitterwolf23a.html. Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark Experience Replay for Continual Learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pp. 15320– 15330, 2020. Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with A-GEM. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019a. URL https://openreview.net/forum?id=Hkf2_sC5FX. Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc’Aurelio Ranzato. On tiny episodic memories in continual learning. ArXiv preprint, abs/1902.10486, 2019b. URL https://arxiv.org/abs/1902.10486. Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, OH, USA, June 23-28, 2014, pp. 3606–3613. IEEE Computer Society, 2014. doi: 10.1109/CVPR.2014.461. URL https://doi.org/10.1109/CVPR.2014.461. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA, pp. 248–255. IEEE Computer Society, 2009. doi: 10.1109/CVPR.2009. 5206848. URL https://doi.org/10.1109/CVPR.2009.5206848. Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out-of-distribution detection. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreview.net/pdf?id= ndYXTEL6cZz. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac HatfieldDodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition. arXiv preprint arXiv:2209.10652, 2022. Tom Fawcett. An introduction to roc analysis. Pattern recognition letters, 27(8):861–874, 2006. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 1321–1330. PMLR, 2017. URL http://proceedings.mlr.press/v70/guo17a.html. Srishti Gupta, Riccardo Balia, Daniele Angioni, Fabio Brau, Maura Pintor, Ambra Demontis, Alessandro Sebastian, Salvatore Mario Carta, Fabio Roli, and Battista Biggio. Out-of-distribution detection for continual learning: Design principles and benchmarking. arXiv preprint arXiv:2512.19725, 2025. doi: 10.48550/arXiv.2512.19725. URL https://arxiv.org/abs/2512.19725. Srishti Gupta, Daniele Angioni, Maura Pintor, Ambra Demontis, Lea Schönherr, Fabio Roli, and Battista Biggio. Buffer-free class-incremental learning with out-of-distribution detection. Pattern Recognition, 172:112441, 2026. doi: 10.1016/j.patcog.2025.112441. URL https://doi.org/10.1016/j.patcog.2025.112441. Md Yousuf Harun, Jhair Gallardo, and Christopher Kanan. Controlling neural collapse enhances out-of-distribution detection and transfer learning. arXiv preprint arXiv:2502.10691, 2025. doi: 10.48550/arXiv.2502.10691. URL https://arxiv.org/abs/2502.10691. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 770–778. IEEE Computer Society, 2016. doi: 10.1109/CVPR.2016.90. URL https://doi.org/ 10.1109/CVPR.2016.90. 11

Preprint.

Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 2426, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https://openreview.net/forum? id=Hkg4TI9xl. Rui Huang and Yixuan Li. MOS: towards scaling out-of-distribution detection for large semantic space. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pp. 8710–8719. Computer Vision Foundation / IEEE, 2021. doi: 10.1109/CVPR46437.2021.00860. URL https: //openaccess.thecvf.com/content/CVPR2021/html/Huang_MOS_Towards_Scaling_ Out-of-Distribution_Detection_for_Large_Semantic_Space_CVPR_2021_paper.html. Peter J Huber. Robust statistics. In International encyclopedia of statistical science, pp. 1248–1251. Springer, 2011. Gyuhak Kim, Sepideh Esmaeilpour, Changnan Xiao, and Bing Liu. Continual learning based on OOD detection and task masking. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2022, New Orleans, LA, USA, June 19-20, 2022, pp. 3855–3865. IEEE, 2022. doi: 10.1109/CVPRW56347.2022. 00431. URL https://doi.org/10.1109/CVPRW56347.2022.00431. James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. ArXiv preprint, abs/1612.00796, 2016. URL https://arxiv.org/abs/1612.00796. Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton. Similarity of neural network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 3519–3529. PMLR, 2019. URL http://proceedings.mlr. press/v97/kornblith19a.html. Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images. Technical report, University of Toronto, 2009. Ananya Kumar, Aditi Raghunathan, Robbie Matthew Jones, Tengyu Ma, and Percy Liang. Fine-tuning can distort pretrained features and underperform out-of-distribution. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https: //openreview.net/forum?id=UYneFzXSJWh. Gustaf Kylberg. Kylberg texture dataset v. 1.0. Centre for Image Analysis, Swedish University of Agricultural Sciences and . . . , 2011. Kimin Lee, Kibok Lee, Honglak Lee, and In So Kweon. A Simple and Effective Baseline for Out-of-distribution Detection. In Advances in Neural Information Processing Systems (NeurIPS), volume 31, 2018. Zhizhong Li and Derek Hoiem. Learning Without Forgetting. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (eds.), Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV, volume 9908 of Lecture Notes in Computer Science, pp. 614–629. Springer, 2016. doi: 10.1007/978-3-319-46493-0\ 37. URL https://doi.org/10.1007/978-3-319-46493-0_ 37. Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https://openreview.net/ forum?id=H1VGkIxRZ. Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based out-of-distribution detection. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL https://proceedings.neurips.cc/paper/ 2020/hash/f5496252609c43eb8a3d147ab9b9c006-Abstract.html. Vincenzo Lomonaco, Lorenzo Pellegrini, Andrea Cossu, Antonio Carta, Gabriele Graffieti, Tyler L Hayes, Matthias De Lange, Marc Masana, Jary Pomponi, Gido M van de Ven, et al. Avalanche: An End-to-end Library for Continual Learning. In 2021 IEEE. In CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 3595–3605, 2021. 12

Preprint.

David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6467–6476, 2017. URL https://proceedings. neurips.cc/paper/2017/hash/f87522788a2be2d171666752f97ddebb-Abstract.html. Wenjun Miao, Guansong Pang, Trong-Tung Nguyen, Ruohuan Fang, Jin Zheng, and Xiao Bai. OpenCIL: Benchmarking out-of-distribution detection in class incremental learning. Pattern Recognit., 171:112163, 2026. doi: 10.1016/J.PATCOG.2025.112163. URL https://doi.org/10.1016/j.patcog.2025.112163. Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 15682–15694, 2021. URL https://proceedings.neurips.cc/paper/2021/hash/ 8420d359404024567b5aefda1231af24-Abstract.html. Yifei Ming and Yixuan Li. How Does Fine-tuning Impact Out-of-distribution Detection for Vision-language Models? Int. J. Comput. Vis., 132(2):596–609, 2024. doi: 10.1007/S11263-023-01895-7. URL https://doi.org/10. 1007/s11263-023-01895-7. Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading Digits in Natural Images with Unsupervised Feature Learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, volume 2011, pp. 5, 2011. Jaewoo Park, Yoon Gyo Jung, and Andrew Beng Jin Teoh. Nearest neighbor guidance for out-of-distribution detection. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pp. 1686–1695. IEEE, 2023. doi: 10.1109/ICCV51070.2023.00162. URL https://doi.org/10.1109/ ICCV51070.2023.00162. Vinay Venkatesh Ramasesh, Ethan Dyer, and Maithra Raghu. Anatomy of catastrophic forgetting: Hidden representations and task semantics. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https://openreview.net/forum?id= LhY8QdUGSuw. Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pp. 5533–5542. IEEE Computer Society, 2017. doi: 10.1109/CVPR.2017. 587. URL https://doi.org/10.1109/CVPR.2017.587. Sudarshan Regmi. AdaSCALE: Adaptive Scaling for OOD Detection. ArXiv preprint, abs/2503.08023, 2025. URL https://arxiv.org/abs/2503.08023. Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. ArXiv preprint, abs/1606.04671, 2016. URL https://arxiv.org/abs/1606.04671. Joan Serrà, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 4555–4564. PMLR, 2018. URL http://proceedings. mlr.press/v80/serra18a.html. Yiyou Sun and Yixuan Li. DICE: Leveraging Sparsification for Out-of-distribution Detection. In Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision - ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXIV, volume 13684 of Lecture Notes in Computer Science, pp. 691–708. Springer, 2022. doi: 10.1007/978-3-031-20053-3\ 40. URL https://doi.org/10.1007/978-3-031-20053-3_40. Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-distribution detection with rectified activations. In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 144–157, 2021. URL https://proceedings.neurips. cc/paper/2021/hash/01894d6f048493d2cacde3c579c315a3-Abstract.html. 13

Preprint.

Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 20827–20840. PMLR, 2022. URL https://proceedings. mlr.press/v162/sun22d.html. Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arXiv preprint arXiv:1904.07734, 2019. Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. ViM: Out-Of-distribution with Virtual-logit Matching. ArXiv preprint, abs/2203.10807, 2022. URL https://arxiv.org/abs/2203.10807. Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 374–382. Computer Vision Foundation / IEEE, 2019. doi: 10.1109/CVPR.2019. 00046. URL http://openaccess.thecvf.com/content_CVPR_2019/html/Wu_Large_Scale_ Incremental_Learning_CVPR_2019_paper.html. Jize Yang, Kai Zhou, and Yixuan Li. Generalized out-of-distribution detection: A survey. abs/2110.11334, 2021. URL https://arxiv.org/abs/2110.11334.

ArXiv preprint,

LeCun Yann. MNIST handwritten digit database. ATT Labs., 2010. Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 3987– 3995. PMLR, 2017. URL http://proceedings.mlr.press/v70/zenke17a.html. Jingyang Zhang, Jingkang Yang, Pengyun Wang, Haoqi Wang, Yueqian Lin, Haoran Zhang, Yiyou Sun, Xuefeng Du, Yixuan Li, Ziwei Liu, Yiran Chen, and Hai Li. OpenOOD v1.5: Enhanced Benchmark for Out-of-distribution Detection. J. Data-centric Mach. Learn. Res., 2:(3):1–32, 2024. URL https://data.mlr.press/assets/ pdf/v02-3.pdf. Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shutao Xia. Maintaining discrimination and fairness in class incremental learning. 2020 IEEE. In CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13205–13214, 2019. Bolei Zhou, Àgata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 Million Image Database for Scene Recognition. IEEE Trans. Pattern Anal. Mach. Intell., 40(6):1452–1464, 2018. doi: 10.1109/TPAMI. 2017.2723009. URL https://doi.org/10.1109/TPAMI.2017.2723009.

Preprint.

E XPERIMENTAL AND I MPLEMENTATION D ETAILS

For completeness, all experiments use the datasets, task curricula, continual-learning methods, and OOD detectors specified in Section 5.1. CIFAR-10 and CIFAR-100 use ResNet-32 over 5 and 10 tasks, respectively, while ImageNet1K uses ResNet-18 over 100 tasks. Models are trained for 170 epochs with SGD (momentum 0.9, batch size 128, initial learning rate 0.1, and a factor-0.1 decay at epochs 60, 100, and 140). TOOD is applied post hoc using random class-balanced calibration sets of B = 200 for CIFAR-10 and B = 700 for CIFAR-100. Main CIFAR results use three seeds; ImageNet-1K, the controlled toy experiment, and the ViT experiments are single-seed scalability or diagnostic studies.

C ONFIDENCE -G AP AND M ANIFOLD -C ROWDING D IAGNOSTICS

(a) OOD samples become surrounded by ID clusters as tasks accumulate. t-SNE projections after each task step (iCaRL, CIFAR-10, OOD: SVHN). ID samples are filled circles (each color a different task); OOD samples are red crosses. At Task 0, OOD samples occupy peripheral space; by Task 4, new task embeddings have colonized the surrounding regions, leaving OOD samples less geometrically distinguishable from ID data.

(b) The OOD–ID geometric margin shrinks monotonically across tasks. Distribution of Euclidean distances from each OOD sample to its nearest ID neighbor, one curve per task step. The median (dashed line) decreases from 2.20 to 1.59 (∼28% reduction), with the entire distribution shifting leftward by Task 4.

Figure 5: Manifold Crowding: new task embeddings erode the OOD detection margin without displacing old representations (iCaRL, CIFAR-10, 5 tasks). Top: the feature space is colonized task by task (t-SNE). Bottom: the resulting margin collapse is measured by nearest-neighbor distances. B.1

M ANIFOLD CROWDING

This section provides a detailed characterization of the Manifold Crowding mechanism introduced in Section 3.1. While the Confidence Gap operates at the logit level (score compression), Manifold Crowding represents a degradation of the feature-space geometry itself. Geometric View: t-SNE Projections. Figure 5a visualizes the colonization of the feature space as tasks accumulate. Initially (Task 0), the model maintains a clear separation between ID and OOD samples. However, as the learning stream progresses, new task embeddings fill the surrounding latent space. By Task 4, OOD samples are surrounded by various ID clusters, even if the individual clusters remain cohesive. This “crowding” explains why distance-based detectors (e.g., MDS) deteriorate: the margin shrinks from the outside as new tasks occupy previously empty regions. Quantitative View: Feature-Space Margin. We quantify this erosion in Figure 5b by measuring the Euclidean distance from each OOD sample to its nearest ID neighbor. The median distance decreases monotonically from 2.20 15

Preprint.

to 1.59 (∼28% reduction). Notably, Centered Kernel Alignment (CKA) analysis (Figure 1) shows that feature drift remains low, suggesting that the margin loss is driven by the occupancy of the space by new classes rather than the movement of old ones. B.2

W HAT TOOD DOES NOT FIX : MANIFOLD CROWDING

The confidence gap affects output scores. Manifold crowding instead reduces the separation between ID and OOD samples in the learned representation. Since TOOD only recalibrates logits, it cannot restore this lost geometric margin. We test this by applying the same normalization separately within each task to nearest prototype distances. This changes Avg AUROC by −0.76 points relative to the original distance detector (Figure 9), showing that adjusting the score centers does not recover geometric separation. Addressing manifold crowding therefore requires changes to the learned representation, such as preserving margins through replay or regularization. We leave this direction to future work. B.3

C ONFIDENCE - GAP RIDGE PLOTS

Figure 6: TOOD selectively shifts the ID score distribution rightward while OOD scores remain anchored (iCaRL, CIFAR-10, N = 5 tasks). Each ridge is the energy distribution at one task step. Left (uncalibrated): the ID distribution (blue) shifts progressively leftward with each new task, converging with Near-OOD (coral) and Far-OOD (orange) by Task 4, explaining the AUROC collapse. Right (TOOD, Robust Anchor): per-task normalization stabilizes the ID distribution (blue) at high energy values, while OOD distributions (coral, orange) remain anchored at low values throughout the stream. Figure 6 provides a distributional view of the confidence gap mechanism introduced in Section 3.1. While the main text quantifies this phenomenon through aggregate metrics (AUROC, Davg ), the ridge plots expose the full shape of the score distributions at each task step, making both the failure mode and the mechanism by which TOOD corrects it directly visible. Uncalibrated Drift. Without calibration (left panel), the ID energy distribution (blue) shifts progressively leftward as tasks accumulate. This leftward shift reflects the confidence gap: gradient updates for new classes inflate P current task logits while old task logits shrink in absolute scale. Since the sign-reversed energy score E(x) = log c exp(hc (x)) aggregates all logits, the shrinkage of old task logit magnitudes directly reduces E(x) for old task ID samples. By Task 4, the ID density almost entirely overlaps with both Near OOD (coral) and Far OOD (orange) distributions. This overlap means no global threshold τ can separate ID from OOD samples, which is precisely the AUROC collapse reported in Table 1. Notably, the OOD distributions themselves remain approximately stationary throughout the stream, suggesting that the degradation is one sided: it is the ID scores that drift toward the OOD region, not the reverse. Restored Separability. After Robust Anchor normalization (right panel), the ID distribution is shifted rightward and stabilized across all task steps, while OOD distributions remain anchored at low values. The mechanism behind this selective realignment is the per-task decomposition in Equations (4) and (6). An ID sample from task Ti activates 16

Preprint.

the energy channel Ei dominantly; the Robust Anchor normalization (Equation (6)) corrects Ei upward using task[ i ), and the max operation in Equation (7) selects this corrected channel. Under the intended specific statistics (ẽi , MAD separation, an OOD sample typically activates no single channel strongly, so its per-task energies tend to remain low and normalization provides no substantial boost to any channel. The max over these values therefore tends to yield a low final score. Because the normalization occurs inside the max, the effective transformation differs between ID and OOD samples, enabling TOOD to alter AUROC in a way that scalar post-hoc calibration cannot (Section 4.1).

A C ONTROLLED T OY E XPERIMENT WITH K NOWN G EOMETRY

(a) Controlled CIL stream (2-D view)

(b) Confidence gap grows with the stream

task index

per-task energy

latent dim 2

toy OOD AUROC

current-task channel first-task channel confidence gap

35 30 25 mean gap ≈ 18 units

20 15

well-separated high overlap

60 chance

50 0

task index t OOD (origin)

latent dim 1 class centres

0.0

0.2

0.4

0.6

0.8

1.0

margin weight λ

ID shell

Figure 7: Controlled single-seed confidence-gap toy experiment. (a) the toy geometry: class clusters on a shell around a central OOD core; (b) the per-task confidence gap grows along the stream (≈ 18 energy units) and TOOD re-centres it; (c) the margin term (λ) saturates once classes overlap. Why a toy experiment. In a trained network the two mechanisms are entangled: as the stream grows the classifier head expands (driving the confidence gap) and new classes colonize latent space (driving manifold crowding), so neither can be switched off to see which one a score-level method actually repairs. A synthetic stream with a fixed, fully known data geometry lets us drive the two independently—grow the head while holding the data fixed, or perturb the geometry while holding the head fixed—and read off exactly what TOOD can and cannot recover. The base geometry is 16 Gaussian-blob classes (2 introduced per task over 8 tasks) whose centres lie on a sphere in a 16-dimensional latent space, with the OOD distribution placed as a blob at the interior origin (Figure 7, panel a). The classes are well separated, so in the frozen-geometry regime any detection collapse is attributable to score drift alone; we introduce crowding only later, by forcing the blobs to overlap. Protocol. This is a single-seed diagnostic. A small Multi-Layer Perceptron (MLP) whose classifier head grows by two units per task is trained sequentially on this stream and scored with the exact TOOD implementation from the main experiments. The expanding head is the structural cause of the confidence gap in CIL—every task appends logits and rescales the old ones—so replicating only this growth on top of frozen data reproduces the gap with no representation change whatsoever. Manipulating the head and the geometry independently is what isolates the two mechanisms below. Isolating the confidence gap. As tasks accumulate the current task’s per-task energy drifts well above that of the first task’s own channel—a mean gap of ≈ 18 energy units (Figure 7, panel b)—which is precisely the confidence gap. Because the data geometry is fixed, this drift is purely a score-level effect, and TOOD re-centres the channels, recovering up to +5.1 Avg AUROC over uncalibrated energy across the first five tasks. The toy experiment therefore exhibits the confidence gap, and TOOD’s repair of it, in the absence of any representation change. Isolating manifold crowding. To isolate the complementary mechanism we instead hold the head fixed and force the class blobs to physically overlap, collapsing the geometric margin while leaving the output-head dynamics benign. Detection now falls to chance (≈ 50% AUROC), and TOOD provides essentially zero recovery, because the lost separability lives in the representation geometry rather than the output scores. The two manipulations thus dissociate the mechanisms cleanly: a score-level fix recovers the frozen-geometry (confidence-gap) regime but not the overlap (manifold-crowding) regime, mirroring the feature-space-analog negative result in Appendix B.2. Margin failure mode. The toy experiment also reproduces the margin term’s boundary condition: the optional margin weight λ helps while classes are well separated but saturates and then degrades once they overlap (Figure 7, panel c), mirroring the trend in Appendix D.1. 17

Preprint.

C ALIBRATION -C OMPONENT, M ARGIN , PARTITION , B UFFER , AND A NCHOR A BLATIONS

D.1

E FFECT OF THE MARGIN PARAMETER

Figure 8: Sensitivity to the margin parameter λ (CIFAR-10, N = 5). Left: Mean AUROC (↑); Right: Average OOD deterioration Davg (↓). High-drift methods (iCaRL, DER) peak near λ = 0.5 (dotted line). Low-drift methods (BiC) decrease monotonically for λ > 0, suggesting that the margin is unnecessary when output head bias correction already stabilizes scores. The margin term λ in Equation (8) augments the base TOOD score (Equation (7)) by rewarding samples whose top norm norm normalized energy E(1) is well separated from the second largest E(2) . The intuition is straightforward: an ID sample from task Ti produces a dominant energy peak in channel i and weak activations elsewhere, yielding a large norm norm gap E(1) − E(2) . An OOD sample, which belongs to no learned task, produces diffuse, roughly uniform activations across channels, yielding a small gap. The parameter λ controls the weight of this discriminative signal. When λ = 0, Equation (8) reduces to the base TOOD score; as λ increases, large-gap samples receive a larger nonnegative bonus, whereas samples with ambiguous cross-task activation receive a smaller bonus. Figure 8 evaluates this sensitivity across λ ∈ [0, 1] in 0.1 increments for three CL methods, revealing two distinct behavioral regimes. Regime 1: High-Drift Methods (iCaRL, DER). These methods exhibit a large confidence gap (Section 3.1), meaning that old task logits have shrunk substantially relative to current task logits by the end of the stream. After TOOD’s pertask normalization corrects the location drift, residual noise in the normalized energies can still cause OOD samples norm to achieve moderately high E(1) values through spurious cross-task activations. The margin heuristic addresses these cases indirectly by preferentially boosting samples with a clearly dominant task channel rather than by lowering ambiguous scores. Increasing λ from 0 to 0.5 yields +0.9 AUROC points for iCaRL and +2.4 points for DER. Beyond λ = 0.5, performance saturates for iCaRL and continues to improve slightly for DER, indicating that excessive margin weight can distort the ranking of legitimate ID samples whose secondary task activations are non-negligible (e.g., semantically related classes spanning adjacent tasks). Regime 2: Low-Drift Methods (BiC). BiC applies explicit bias correction to the output head (Wu et al., 2019), which already reduces the confidence gap before TOOD intervenes. Consequently, the normalized per-task energies for BiC are already well separated, and the base TOOD score (λ = 0) captures the ID/OOD distinction effectively. Introducing the gap-dependent bonus with λ > 0 harms the ranking in this regime: the monotonic decline in AUROC (left panel) and increase in Davg (right panel) show that reweighting by the top-two gap is unnecessary when the underlying score distributions are already calibrated. The optimal value for BiC is λ = 0. Consensus Value and Practical Guidance. We set λ = 0.5 as a fixed default for all main experiments. This value maximizes gains for methods with significant logit drift (the primary use case for TOOD) while imposing only modest overhead on methods that do not require calibration. For BiC specifically, the AUROC reduction at λ = 0.5 is small (1.1 point), whereas the gains for DER exceed +2 points. In practice, if the CL method is known to include output head correction, setting λ = 0 is preferable; otherwise, λ = 0.5 provides a robust default without per-method tuning. The following analyses isolate the calibration component and the class-to-task partition used by TOOD. All analyses are post-hoc: TOOD adds no training and re-scores the same checkpoints used in the main results. Unless stated otherwise, we report the change in Avg AUROC (Eq. 3, the near/far mean) relative to uncalibrated energy, averaged over the five CL methods of Table 1 and both CIFAR benchmarks; Figure 9 summarizes the score-level comparisons. 18

Preprint.

D.2

T HE GAINS COME FROM PER - TASK ENERGY DECOMPOSITION Only logit-space TOOD recovers separability

Δ Avg AUROC vs. uncalibrated energy

+3.1

2 0 -0.8 −2 −4

-3.6 TOOD (logit)

feature-space analog

per-task temp. scaling

Figure 9: Score-level dissociation. Only logit-space TOOD (+3.1) recovers separability, while generic per-task temperature scaling (−3.6) and a feature-space port of the same normalization (−0.76) do not. In principle TOOD might improve detection simply by being task-aware—rescaling each task’s scores to a common range—rather than through its particular energy decomposition and reference anchoring. To separate the two, we add a per-task temperature-scaling baseline that sees the same partition but only rescales each task’s logits, without decomposing or anchoring the energies (Equations (4) and (6)). Concretely, it divides each task energy of Equation (4) by a per-task temperature Tt and scores by the maximum over tasks, Etts (x) =

Et (x) , Tt

Sts (x) =

max

t∈{1,...,N }

Etts (x),

(9)

where each per-task temperature Tt is the standard deviation of that task’s calibration energies, s 2 1 X Et (x′ ) − µt . Tt = |Bt | ′ x ∈Bt

Unlike Equations (5) and (6), this baseline has no common reference recentering (no µref or ẽref ): each task’s energy is only divided by its own buffer standard deviation and retains its own offset. It therefore isolates task-aware rescaling without reference anchoring. This baseline lowers Avg AUROC by 3.6 points, whereas TOOD raises it by 3.1 (Figure 9). Task-aware rescaling alone is thus insufficient, so the improvement is attributable to the per-task energy mechanism itself rather than to access to the partition. D.3

W HAT TOOD ASSUMES : THE CLASS PARTITION , NOT TASK IDENTITY

Because TOOD indexes scores by task, one might suspect it covertly requires task-incremental (Task-Incremental Learning (TIL)) inference. It does not, and the two inputs are different in kind. The partition is the fixed training-time map recording which task each class was introduced in; every CIL method already stores it to grow its classifier head, and it is shared by all test inputs. A TIL task label, by contrast, is a per-input signal revealing at inference which task the current sample came from. TOOD uses only the former: it groups the logits into per-task energy channels via this map and scores an input by the maximum over channels (Equation (7)), so no test sample is ever told its task. The method is also robust to a misspecified partition. To show this, we set the within-task split factor to three at deployment. Classes are assigned once to disjoint, balanced subgroups, and empty subgroups are discarded. This gives three pseudo-groups per 10-class CIFAR-100 task and two singleton pseudo-groups per 2-class CIFAR-10 task. Averaged across both datasets, five continual-learning methods, and three seeds, Avg AUROC shifts by only −0.5 points. Detection therefore tolerates a substantially finer-than-true grouping rather than requiring accurate task identity at test time, consistent with TOOD operating in the CIL regime. 19

Preprint.

D.4

B UFFER SIZE

The buffer-size ablation in Figure 4 shows that performance stabilizes between B = 50 and B = 100, with negligible gains thereafter. Random class-balanced samples also match iCaRL-style herding to within 0.02 Avg AUROC points on average. D.5

A NCHOR SENSITIVITY

[ ref ). The robust-anchor normalization (Equation (6)) maps every task’s energy onto a common reference (ẽref , MAD We default to the most-recent task TN , but the choice applies the same affine map to all normalized task channels and therefore does not change their ranking. Across five CL methods, both CIFAR datasets, and three seeds, the largest absolute Avg AUROC changes relative to the default are 0.0001 for the oldest-task anchor, 0.0000 for the max-spread anchor, and 0.00015 for the mean-statistics anchor. We retain TN only as an interpretable current-scale reference for absolute decision thresholds.

I MAGE N ET AND V I T S CALABILITY

E.1

S CALABILITY ON I MAGE N ET-1K

On ImageNet-1K (N = 100), TOOD (Mean Shift) substantially helps drift-sensitive methods such as BiC and DER, while WA changes only marginally, consistent with its stable output scale. Table 3: ImageNet-1K results (N = 100; single seed). Columns are CL methods; rows are OOD detectors. We report average CIL accuracy, Avg AUROC (Equation (3)) (↑), and Avg FPR@95 (↓), with detection metrics averaged over Near-OOD and Far-OOD. Bold, underline, and teal denote 1st, 2nd, and 3rd place within each metric column. ImageNet-1K (N = 100) OOD Method Avg CIL Acc. (↑)

E.2

BiC WA DER AUC ↑ FPR ↓ AUC ↑ FPR ↓ AUC ↑ FPR ↓ 28.9

26.8

30.8

MSP ODIN Ash ADASCALE (Activation) NNGuide Energy (Uncalib.)

57.7 56.4 56.3 44.8 42.0 59.2

86.3 89.5 91.3 95.0 92.7 86.1

62.8 63.9 67.5 58.1 55.4 70.1

82.5 82.0 81.6 88.1 86.5 76.4

62.4 66.0 68.7 54.1 47.0 70.2

84.6 81.2 80.4 93.4 91.9 76.9

Energy + Mean Shift Energy + Robust Anchor

65.2 65.1

76.5 75.2

65.7 65.6

77.4 79.0

71.4 70.5

76.2 77.8

V ISION -T RANSFORMER BACKBONE

Our main results use ResNet backbones, whose BatchNorm statistics could in principle drive the confidence gap. To test a different normalization regime, we ran single-seed experiments with a ViT-B/16 backbone (ImageNet-pretrained, LayerNorm) on the CIFAR-10 five-task stream and re-scored the checkpoints post hoc. The correction transfers across the normalization regime: for two methods, TOOD raises Avg AUROC over uncalibrated energy by +3.5 points on iCaRL (54.5 → 58.0) and +1.1 on WA, matching the direction and magnitude of the ResNet results and indicating the fix is not BatchNorm-specific. On CIFAR-100, where the confidence gap is most severe, the gains are larger (Table 4). TOOD improves WA by 7.4 Avg AUROC points (60.9 to 68.3) and BiC by 1.0, while DER (−0.1) and LwF (−0.8) are left essentially unchanged. That WA improves more on CIFAR-100 than on CIFAR-10 fits the picture of a gap that widens as the stream grows longer and the number of classes increases. As on ResNet (Section 5.1) the effect varies by method, because TOOD is a targeted repair for score drift: it helps where a confidence gap is present and stays neutral, or turns slightly negative, where it is not. These results also help assess whether the gain merely follows the imbalance of accuracy across tasks. WA and DER both collapse onto the most recent task, reaching roughly 86% accuracy there against at most 30% on the older tasks, yet TOOD moves them in opposite directions, by +7.4 for WA and −0.1 for DER. Two models with almost identical imbalance thus behave very differently, suggesting that the benefit follows the confidence gap in the output scores 20

Preprint.

rather than the balance of accuracy across tasks. We report these methods as a single-seed proof of concept and leave a fuller multi-seed ViT study to future work. Table 4: ViT-B/16 CIFAR-100 (10-task), post-hoc, single-seed. Avg CIL accuracy (as in Table 1) and Avg AUROC (near+far) under uncalibrated energy vs. TOOD (robust anchor). Method

Avg CIL Acc.

Uncalib.

TOOD

∆

WA BiC DER LwF

35.6 54.5 47.5 46.4

60.9 65.6 62.8 64.7

68.3 66.6 62.7 63.9

+7.4 +1.0 −0.1 −0.8

Record · ID 422241 · SHA-256 fd8788253aecd114
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.