Rethinking the Teacher-Student Framework for Test-Time Adaptation

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

R ETHINKING THE T EACHER -S TUDENT F RAMEWORK FOR T EST-T IME A DAPTATION Damian Sójka1,2∗ Marc Masana3

Bartłomiej Twardowski4,5

Sebastian Cygert6,7

1

arXiv:2609.02507v1 [cs.LG] 2 Sep 2026

4

2 3 Poznan University of Technology Akces NCBR Graz University of Technology 5 IDEAS Research Institute Computer Vision Center, Universitat Autonoma de Barcelona 6 7 NASK - National Research Institute Gdańsk University of Technology

A BSTRACT Test-Time Adaptation (TTA) has recently emerged as a promising strategy that allows the adaptation of pre-trained models to changing data distributions at deployment time, without access to any labels. To mitigate error accumulation, researchers have widely adopted the teacher-student framework, though its long-term stability is often taken for granted. In this work, we challenge the common strategy of setting the teacher weights to an exponential moving average of the student by showing that error accumulation still occurs, although it is mostly apparent on longer sequences compared to those commonly utilized. We analyze the stability-plasticity trade-off within the teacher-student framework and propose to use an intransigent teacher that does not update its weights. Surprisingly, we show that this simple change allows TTA methods to significantly improve their performance on multiple datasets with longer scenarios and result in increased robustness to changes in hyperparameters. Finally, we show that those changes can be seamlessly and effectively applied to various architectures and experimental setups, including semantic segmentation. The code is available at https://github.com/dmn-sjk/intransigent_teacher.

1

I NTRODUCTION

Machine learning models typically assume that training and testing data originate from a similar distribution. However, in real-world applications, distribution shifts between training (source) and testing (target) data domains are common and can lead to performance issues throughout inference (Geirhos et al., 2019; Hendrycks & Dietterich, 2019; Koh et al., 2021). Test-Time Adaptation (TTA) (Wang et al., 2021) allows for an online adaptation of a pre-trained model to the changing data distributions during testing, in an unsupervised way. While many TTA methods have been developed in recent years (Gong et al., 2022; Goyal et al., 2022; Marsden et al., 2024a; Niu et al., 2022; 2023; Rusak et al., 2022; Sun et al., 2020; Wang et al., 2022; Yuan et al., 2023), important challenges remain, such as adaptation over very long scenarios (Press et al., 2023), robustness to noisy data (Gong et al., 2023), and sensitivity to hyperparameter change (Boudiaf et al., 2022; Zhao et al., 2023; Cygert et al., 2026). The teacher-student paradigm is a popular framework in TTA (Chen et al., 2022; Döbler et al., 2022; Sójka et al., 2023; Wang et al., 2022; Yuan et al., 2023; Zhou et al., 2024), where the teacher weights are updated as the Exponential Moving Average (EMA) of the student weights (i.e., EMA teacher). This strategy follows pioneering works in semisupervised learning (Laine & Aila, 2017; Tarvainen & Valpola, 2017), representation learning (Grill et al., 2020; He et al., 2020; Oquab et al., 2024), and learning under label noise (Liu et al., 2020; Nguyen et al., 2020). The averaged model provides more accurate and consistent predictions, which the student uses for training. While recent TTA survey (Maharana et al., 2026) suggests this framework inherently prevents model collapse and enables state-of-theart performance, our findings challenge this consensus. In this work, we present experimental evidence indicating that the teacher-student framework does not strictly prevent error accumulation, which can result in model collapse (i.e., falling below the accuracy of the source model). Moreover, we demonstrate that state-of-the-art TTA methods relying on this stabilization (Chen et al., 2022; Wang et al., 2022; Yuan et al., 2023; Brahma & Rai, 2023) are prone to significant accuracy degradation on extended test sequences. The EMA teacher merely postpones the effects of error accumulation. This reveals a critical flaw in common evaluation protocols, which have previously failed to detect this issue. In search for a solution, we observe that employing a simple technique of making the teacher more intransigent (not updating the teacher’s weights) prevents ∗

Corresponding author: damian.sojka@doctorate.put.poznan.pl

1

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

model collapse over very long testing sequences (see Fig. 1). Interestingly, such a teacher can also enable students to fulfill the age-old cliche of surpassing their teacher (see Fig. 7). Based on that finding, we take a closer look at the stability-plasticity trade-off (Mermillod et al., 2013) within teacher-student frameworks and how it affects the final model performance. We show that while increased teacher plasticity can lead to better performance on the current data in the short run, using a more stable teacher leads to a better adaptation over longer scenarios. This is due to the error accumulation of the plastic model, which adapts well to the current data but fails to maintain generalization over time, as explored in Sec. 4.2. Our contributions are as follows:

Avg. Accuracy (%)

60

• We analyze the teacher-student framework commonly used in TTA, demonstrating that an EMA teacher fails to prevent model collapse on longer test sequences.

Long Scenarios (L)

55 50 45

• We observe that maintaining the teacher’s weights fixed during adaptation (Intransigent Teacher – IT), effectively prevents model collapse while allowing the student to surpass the teacher. We find that IT works well because it avoids negative flips and tends to keep the representation strength intact.

40 35

AdaContrast

RoTTA

PETAL

Figure 1: Average classification accuracy [%] for the baseline methods, with and without our Intransigent Teacher technique, evaluated across the original benchmarks and their long variants (20× repetitions). While baseline methods exhibit performance degradation as test sequence length increases, our Intransigent Teacher modification to the teacher-student framework effectively mitigates this decay and maintains robust accuracy.

• This simple solution (i.e., IT) can be combined with state-of-the-art methods, mitigating error accumulation by easily replacing any EMA teacher and obtaining reliable performance. IT can serve as a simple yet effective comparative baseline for future TTA works.

2

Original Benchmarks Intransigent Teacher

R ELATED WORK

Test-time adaptation. TTA (Wang et al., 2021) aims to adapt a pre-trained model to shifting data distributions during testing, without any labels. The adaptation is typically performed using unsupervised optimization objectives, often relying on noisy pseudo-labels, which can lead to error accumulation over multiple iterations. Therefore, numerous strategies have been developed to circumvent that issue, including various reset strategies (Wang et al., 2022; Zhang et al., 2022; Press et al., 2023), filtering unreliable samples (Niu et al., 2022; Zhou et al., 2025), and the teacher-student framework (Chen et al., 2022; Wang et al., 2022; Yuan et al., 2023; Sójka et al., 2023; Brahma & Rai, 2023; Shi et al., 2025; Zhou et al., 2025), among others. The teacher-student framework was introduced to TTA by the CoTTA method (Wang et al., 2022), where the use of exponential moving average (EMA) of weights was proposed. However, as EMA of the student’s weights forms the teacher, error accumulation is merely delayed rather than eliminated. In this work, we analyze the teacher-student framework to address these limitations. Other works have explored strategies that retain the fixed source model. ROID (Marsden et al., 2024a) continually ensembles source and gradient-updated model weights to stabilize the adaptation. GRoTTA (Li et al., 2023) and TRIBE (Su et al., 2024) expand on the teacher-student framework by incorporating the source model as a third one for additional regularization. While these methods employ complex designs with extra loss terms and models, we explore how a simple teacher-student framework modification can address the performance degradation problem in TTA, especially for longer sequences. Parallel works (Zhou et al., 2024; Hoang et al., 2024) also propose to evaluate existing TTA methods over very long scenarios. Our work is complementary, since they propose adaptation methods (which require parameter tuning and access to source data (Hoang et al., 2024)), while we focus on evaluating existing approaches on a more extensive experimental setup, and observe that a simpler modification improves the reliability in such conditions. Plasticity-stability trade-off. EMA ensemble of weights is parametrized by the β trade-off parameter, which determines the balance between retaining old averaged weights of the teacher and incorporating newly updated student ones, commonly set in TTA to 0.999 (Wang et al., 2022) (following temporal ensembling works (Laine & Aila, 2017)), where β = 1 means full stability (frozen model), and β = 0 means maximal plasticity. The trade-off has been widely analyzed in continual learning (Mermillod et al., 2013; Chaudhry et al., 2018; Masana et al., 2022), where the learner needs to balance learning of new tasks with the risk of forgetting the previously acquired knowledge. While a variety of strategies have been developed in continual learning, the most successful ones are those that promote stability. Many recent works freeze the feature extractor and learn only the classification head (Ma et al., 2023; Panos et al., 2

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

2023; Goswami et al., 2024; Tscheschner et al., 2025). Our work is inspired by those studies and aims to analyze the plasticity-stability trade-off within TTA. Teacher-student framework. In knowledge distillation (Hinton et al., 2015), usually a larger model (teacher) guides the optimization of the target model (student) by providing informative training signals (teachers’ outputs). Such a strategy is also commonly used in continual learning to prevent forgetting of previous tasks (Buzzega et al., 2020; Kirkpatrick et al., 2017). Self-distillation is a special case in which the teacher and student have the same architecture. It has been shown that in such a scenario, the student can outperform its teacher (Furlanello et al., 2018). In their work, the teacher is updated at the end of every training epoch by copying students’ weights. They show improvement gains until such a procedure is repeated three times. Since TTA lacks access to labeled data, updating the teacher might result in even more significant error accumulation.

3

P RELIMINARIES

This study addresses continual TTA. The goal is to adapt a pre-trained (source) model fθ0 (·), initially trained on a s labeled dataset Ds={(xi , yi )}N i=1 with xi ∼ Ps (x), to a sequence of unlabeled test data batches arriving sequentially during inference: (1)

(T )

Dt , · · · , Dt

(k)

(k)

(k)

(k) k where Dt = {xj }N (x), j=1 , xj ∼ Q

and test distributions evolve over time: Q(k) (x) ̸= Q(k+1) (x) for some k, with Q(k) (x) ̸= Ps (x) in general (i.e., test data is out-of-distribution and non-stationary). Adaptation must be performed on-the-fly using only the current and/or past unlabeled test batches, without access to source data or labels. Both the student and the teacher start the adaptation with the same set of weights θ0 pre-trained on the source data. During TTA, the student’s weights θns at step n are updated via backpropagation. The teacher’s weights are updated as an EMA of the student’s weights: t θnt = β · θn−1 + (1 − β) · θns (1) where β (e.g., 0.999) is the EMA factor controlling the momentum of teacher’s update. The teacher is usually used for the final predictions.

4

P RELIMINARY EXPERIMENTS

In this section, we aim to gain a deeper understanding of the adaptation process within the teacher-student framework, with particular emphasis on the plasticity-stability trade-off. To this end, we conduct two experiments. In the first experiment (Sec. 4.1), adaptation relies exclusively on a simple self-supervised loss combined with the teacher-student framework, deliberately excluding any additional regularization or stabilization techniques. This minimal setup allows us to isolate the intrinsic behavior of the teacher-student mechanism. In the second experiment (Sec. 4.2), we systematically vary the level of plasticity across the teacher-student frameworks used in existing TTA methods and analyze the effects of low-to-none plasticity. 4.1

E FFECT OF TEACHER - STUDENT FRAMEWORK ON THE SIMPLE SELF - SUPERVISED ADAPTATION

The teacher-student framework is widely studied in TTA, with great results being achieved by popular methods such as AdaContrast (Chen et al., 2022) and CoTTA (Wang et al., 2022). Although those techniques rely on other components (e.g., memory queue or weight restoration), they share a common trend of incorporating a self-supervision loss. We dissect the objectives directly related to the learning (their self-supervised losses), which allows for analyzing the teacher-student framework impacted only by the component the most related to the stability-plasticity trade-off. We probe standard EMA teachers (ET) as proposed in their original works (both using β = 0.999) and compare them with the presented intransigent teacher (IT) technique (β = 1). Intransigent teachers have all trainable parameters frozen. Their batch normalization statistics are calculated on a per-batch basis (Schneider et al., 2020b), as commonly done in TTA (Niu et al., 2022; 2023; Wang et al., 2021; 2022). The final predictions are taken from the student model. As a motivation example, we evaluate on the popular ImageNet-C corruption benchmark (Hendrycks & Dietterich, 2019) on a common level 5 corruption sequence from Wang et al. (2022). Furthermore, we evaluate on the CCC benchmark (Press et al., 2023) and introduce the setting with ImageNet-C repeated 20 times (ImageNet-C (L)), to better assess the teacher-student framework’s performance on longer scenarios. The source model is trained on the clean ImageNet dataset (Deng et al., 2009). Figure 2 shows accuracy over time and Table 1 shows numerical results, where the evaluated objectives are described as: 3

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

• Consistency: The loss from CoTTA (Wang et al., 2022), which minimizes the cross-entropy consistency between teacher and student predictions. The input of the teacher is transformed via augmentations. • Contrastive: The adaptation objective from AdaContrast (Chen et al., 2022), which uses a MoCo-inspired (He et al., 2020) contrastive task in which features from different views of the same image (positive pairs) are pulled closer, while features from different images (negative pairs) are pushed away. Input for both teacher and student is augmented by randomly drawing two strong augmentations. The results from the above experiment lead to the fol- Table 1: Mean accuracy [%] of student models on test-time lowing observations: adaptation benchmarks. ET stands for exponential moving Observation 1: Self-supervised objective combined average teacher, and IT indicates the intransigent teacher. with an intransigent teacher provides great reliability IN-C (L) stands for the ImageNet-C adaptation sequence on its own. This is a very positive result, as recent being repeated 20 times. work (Press et al., 2023) observed that on CCC, most of Loss Teacher IN-C IN-C (L) CCC the current adaptation methods result in model collapse. This is especially interesting in the case of Contrastive, Source none 18.0 18.0 16.8 where the contrastive loss changes only the backbone ET 27.3 7.9 1.6 Consistency of the model without changing the classification layer, IT 28.8 31.3 27.2 mostly relying on feature alignment. Contrastive

Observation 2: An EMA on its own does not prevent error accumulation. As clearly shown in Figure 2, the problems when using EMA take some time to become apparent and are only visible over long sequences. For both losses, performance degradation starts rather early, after the 2nd and 3rd loops, respectively.

22.1 36.9

5.8 31.8

35 30 25 20 15 10 5 0

4.2

35.5 35.4

40

Accuracy [%]

Observation 3: When using EMA, there is a small gap between the teacher and student accuracies. When an intransigent teacher with consistency loss is used, the student performance is reliable and comparable to the teacher, while in the case of the contrastive loss, the student is able to outperform their teacher significantly.

ET IT

Intransigent Teacher Contrastive (IT) - Student Contrastive (ET) - Teacher Contrastive (ET) - Student Consistency (IT) - Student Consistency (ET) - Teacher Consistency (ET) - Student Source 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

E FFECTS OF INTRANSIGENCE

Figure 2: Per-batch accuracy on repeated ImageNet-C (20 To better understand the plasticity-stability trade-off, we loops) with EMA teacher (ET) and intransigent teacher evaluate performance when varying β ∈ [0.9, 1.0], where (IT), both for teacher (solid) and student (dashed). Only 1.0 means using an intransigent teacher, and 0.999 is the the self-supervised losses, without any additional compodefault value for the two original methods we compare: nents (or restart mechanisms), allow for successful adaptaAdaContrast and CoTTA. We evaluate on CIFAR10-C tion over long sequences when the teacher is intransigent. (C10-C) and ImageNet-C (IN-C) (Hendrycks & Dietterich, 2019) benchmarks, with each common adaptation sequence from Wang et al. (2022) repeated 20 times (L). The source model is either trained on clean CIFAR10 (Krizhevsky, 2009) or ImageNet (Deng et al., 2009), depending on the benchmark. The average final performance is reported in Table A.11 (Appendix), while the accuracy evolution throughout the sequence is shown in Figure 3, with the accuracy on the common-length benchmarks indicated at the first tick of the x-axis. Results with different method–dataset combinations are presented in the Appendix (Fig. A.7 and Fig. A.8). Intransigent teachers guide students more consistently than lenient alternatives. While increasing plasticity (decreasing β) improves short-term student accuracy, performance tend to collapse over longer sequences. For example, using an EMA with β = 0.9999 initially outperforms IT, but accuracy degrades significantly toward the end of the test sequence. Consequently, fixed teacher weights provide the most reliable long-term performance. We further demonstrate this vulnerability in Figure A.4 in the Appendix using CoTTA on ImageNet-C. Although a slowly-updating teacher (β = 0.9999) achieves superior results over 20 repetitions, its performance falls significantly below the IT baseline when the sequence extends to 100 repetitions. This highlights a critical failure mode in TTA. Methods often depend on adaptation sequence lengths. While carefully tuned EMA parameters may mitigate this, hyperparameter selection for TTA remains an open challenge (Boudiaf et al., 2022; Zhao et al., 2023) and may necessitate specific assumptions about the deployment environment, such as the length of the adaptation sequence. 4

40 35 30 25 20 15 10 5 0 1

0.9 0.99 0.999 (Def.) 0.9995 0.9999 1.0 (Intransigent) Source 6

11 Loop No.

Accuracy [%]

Accuracy [%]

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

16

20

80 70 60 50 40 30 20 10 1

0.9 0.99 0.999 (Def.) 0.9995 0.9999 1.0 (Intransigent) Source 6

11 Loop No.

16

20

Figure 3: Mean accuracy [%] for each loop of common testing sequence on ImageNet-C (L) with AdaContrast (left) and CIFAR10-C (L) with CoTTA (right). The brown dashed line indicates the source model accuracy as a reference.

5

M ETHODOLOGY

As presented in previous sections, using an intransigent teacher provides a great model consistency throughout longer scenarios without encountering catastrophic error accumulation. Here, we describe how we extend existing TTA methods with an intransigent teacher strategy for further experiments (Sec. 5.1) and provide the theoretical grounding for its effectiveness (Sec. 5.2). 5.1

A PPOINTING AN INTRANSIGENT TEACHER Student

Backprop

We extend 4 popular TTA methods that originally utilize the teacherLoss student framework, AdaContrast (Chen et al., 2022), CoTTA (Wang et al., 2022), RoTTA (Yuan et al., 2023), and PETAL (Brahma & Rai, Teacher 2023), with our proposed intransigent teacher. The main difference between the classic teacher-student framework with EMA teacher and the intransigent teacher is the value of the β parameter. In the Figure 4: The Intransigent Teacher. The case of IT, the parameter β is equal to 1, so the weights are fixed at student’s weights are updated using gradientbased optimization, while the teacher’s weights their initial pre-trained values and Eq. 1 simplifies to: remain fixed throughout adaptation. t t t θn = θn−1 =⇒ θn = θ0 (2) Additionally, the student model is used to generate the final predictions, regardless of which predictions are used in the original method. We maintain the same usage and adaptation of batch normalization statistics as the underlying TTA method for both the teacher and student networks. Figure 4 illustrates our adaptation process. Descriptions of strategies extended with IT are provided in the Appendix (Sec. A.2). 5.2

A NALYTICAL FRAMEWORK FOR INTRANSIGENT TEACHERS

To understand why standard EMA teachers fail on extended sequences, we can formalize the feedback loop of noise inherent in TTA. The student’s weights θns at step n are updated via backpropagation on an adaptation loss L, relying on the teacher’s pseudo-labels: s s t θns = θn−1 − η · ∇θs L(θn−1 , θn−1 , xn ) (3) t where η is the learning rate, θn−1 represents the teacher weights at time n − 1, and xn is the test batch at time n. Due to the absence of ground-truth supervision, this update consist of a constructive adaptation direction and a harmful gradient noise. We can explicitly decompose the student’s weight change into these two directions: s θns = θn−1 + d n + ϵn

(4)

s LGT (θn−1 , yn∗ , xn ) represents the ideal gradient step dictated by an oracle providing ground-truth

Here, dn = −η∇ labels yn∗ , and ϵn captures the residual gradient noise injected by incorrect pseudo-labels generated by the teacher. By unrolling the recurrence relation over time n in the EMA teacher update (Eq. 1), we can express the teacher’s current weights as a geometrically weighted sum of the source model weights (θ0 ) and all past student states: n X θnt = β n θ0 + (1 − β) β n−i θis (5) θs

i=1

This unrolled equation reveals the critical flaw of using β < 1 for continuous adaptation. Over infinite time horizons (as n → ∞), the β n θ0 term decays to zero, meaning the teacher completely loses its anchor to the robust, pre-trained 5

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

P source distribution. Furthermore, the teacher permanently absorbs the accumulated student noise ( ϵi ). Because the student’s loss function uses the teacher as a reference, a degraded teacher generates lower-quality pseudo-labels for the next batch. This, in turn, increases the magnitude of the noise ϵn+1 in the student’s next update. This creates a destructive positive feedback loop, illustrating why EMA merely delays, rather than prevents, catastrophic error accumulation on long sequences. When we apply the IT strategy by setting β = 1, the EMA recurrence simplifies entirely: θnt = θ0 . While it might seem that the student network would merely mimic the frozen teacher, it is instead optimized using consistency or contrastive losses across distinct, augmented views of the data. We attribute this effective learning signal to the forced alignment between the two networks, where one processes a heavily augmented view of the input while the other operates on a clean version. By freezing the teacher, we decouple the pseudo-label generator from the student’s noisy updates. While the teacher’s supervision is inherently noisy due to the domain shift, this noise distribution remains stationary and non-compounding. This prevents the destructive positive feedback loop of error accumulation. We experimentally verify the error accumulation preventing behavior in Sec. 6.2.

6

E XPERIMENTS

Following the insights from preliminary experiments in Sec. 4, we evaluate the intransigent teacher over longer scenarios and with various state-of-the-art methods. The efficacy of the proposed baseline is validated across various model architectures and hyperparameter selections. We assess the error accumulation and the feature representation strength in student models for both original baselines and IT-enhanced approaches. Further details on the experimental setup are provided in Appendix Sec. A.1. Benchmarks. We adopt the popular corruption benchmarks from Wang et al. (2022); Niu et al. (2022); Döbler et al. (2022): CIFAR10-C (C10-C) and ImageNet-C (IN-C) (Hendrycks & Dietterich, 2019), using the standard corruption sequence at the highest severity. The adaptation on natural domain shifts is tested on DomainNet-126 (DN-126) (Peng et al., 2019) and ImageNet-R (IN-R) (Hendrycks et al., 2021). For DN-126, we use the real domain as the source data and conduct experiments on the remaining domains. Our goal is to focus on very long (L) adaptation sequences to evaluate the methods in terms of stability during long-time operation. For that reason, we repeat the standard test sequences 20 times, and additionally test on the CCC (Press et al., 2023) long sequence without repetition. Baselines. We apply our modification to 4 teacher-student state-of-the-art frameworks: AdaContrast (Chen et al., 2022), CoTTA (Wang et al., 2022), RoTTA (Yuan et al., 2023), PETAL (Brahma & Rai, 2023), and report their performance with IT using (I-) prefix. We compare the results with the original methods and the following strategies that do not utilize classic teacher-student framework: TENT (Wang et al., 2021), EATA (Niu et al., 2022), SAR (Niu et al., 2023), RDUMB (Press et al., 2023), MEMO (Zhang et al., 2022), and PeTTA (Hoang et al., 2024). The nonadapted model is indicated as Source, and TestBN is a fixed model using batch normalization statistics from the current batch (Li et al., 2016; Schneider et al., 2020b) as commonly done in TTA (Wang et al., 2021; 2022; Niu et al., 2022). Architectures. Consistent with previous works (Wang et al., 2022; Marsden et al., 2024a), we use WideResNet-28 (Zagoruyko & Komodakis, 2016) models with pre-trained weights from ROBUST B ENCH (Croce et al., 2021) for the main experiments on CIFAR10-C. Similarly, evaluation on ImageNet and DomainNet-126 employ the ResNet50 network with weights sourced from the same model zoo or those provided by Marsden et al. (2024a) for DomainNet-126. Additional experiments in Sec. 6.2 are carried out with ResNet-26GN (RN26GN) (Wu & He, 2018), ResNeXt-50 32x4d (RNXt-50) (Xie et al., 2017), ViT-B16 (Dosovitskiy et al., 2021), SwinViT-T (SViT-T) (Liu et al., 2021) and ConvNeXt tiny (CNXt tiny) (Liu et al., 2022) architectures. Weights for RN26GN are taken from Zhang et al. (2022), as in Marsden et al. (2024a). T ORCHVISION is used to initialize the rest of the mentioned models. 6.1

O N ADAPTATION OVER LONG SCENARIOS

The vulnerability of EMA teachers is revealed on extended test sequences. In long scenarios (see Tab. 2), methods based on teacher-student framework often achieve lower performance compared to other baselines or even cause the model to collapse, which is especially apparent on ImageNet-C and CCC benchmarks. Those shortcomings are amplified in the lower batch size setting, potentially caused by a more significant error accumulation due to difficulties in estimating batch statistics and a larger number of adaptation steps. While the goal of using EMA is to provide a more stable adjustment and more accurate pseudo-labels for adaptation, we find this to not hold true for the longer settings. This is in contrast to the evaluation on common sequence length (see Appendix Tab. A.4), where the EMA teacher performs well and the original methods don’t cause model collapses in most cases, except for the more challenging lower batch size setting. Results show that the performance of state-of-the-art TTA methods is clearly test sequence-length dependent. 6

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

Intransigent teachers are very effective at collapse prevention. The evaluated original teacher-student adaptation methods exhibit at least 1% decrease in model accuracy in 21 out of 40 cases for long sequences (Tab. 2, results of AdaContrast, CoTTA, RoTTA, and PETAL on five benchmarks and two batch sizes). When using an intransigent teacher, the collapse happens only three times for the small batch size. However, in these cases, performance is at least close to the source model and may still yield great improvements over the baseline - for instance, I-CoTTA achieves a 44.3% increase in accuracy on DomainNet-126 (L). Intransigent teachers provide great reliability, out-ofthe-box. Keeping the teacher model intransigent on a batch size 64 allows for accuracy improvements of 12.2 (AdaContrast), 2.4 (CoTTA), and 8.0 (RoTTA) percentage points on average. It is even more effective on a smaller batch size, where the respective improvements are 22.8, 25.2, and 7.4. Note that applying this strategy did not require any hyperparameter tuning nor additional parameters. In fact, it removes the need to set β.

Table 2: Classification accuracy [%] for long (L) scenarios. Superscript indicates improvements over the baseline. Bold indicates best performing method. Gray color indicates model collapse – performance worse than the nonadapting model (Source). Results averaged from 3 random seeds. * indicates approximated result, details in Appendix.

Teacher intransigence and the plasticity bottleneck. While the intransigent teacher provides a robust baseline, it prioritizes stability over plasticity. This trade-off limits adaptation to rapid distribution shifts and prevents the student from exploring parameter spaces distant from the source model. This limitation is most evident when using CoTTA on ImageNet-C (L) and ImageNet-R (L) with a batch size of 64 (Tab. 2). In these scenarios, the use of IT reduces accuracy by 17.4 and 11.0 percentage points, respectively. We attribute this difference to the high baseline performance of CoTTA in these settings, as it outperforms all other tested methods. Here, IT over-regularizes the student and hinders its adaptation capabilities. While an EMA teacher may be preferable when hyperparameters can be precisely tuned, IT remains effective as it still outperforms the source model. Crucially, IT prevents long-term model collapse. Negative flip rate analysis (Sec. 6.2) shows that for CoTTA, the number of samples correctly predicted by the source but incorrectly by the adapted model increases monotonically over repetitions. In contrast, the IT negative flip rate remains constant. This suggests that long enough sequences could eventually cause model collapse, similarly to the 100 repetitions experiment in Appendix Figure A.4.

Method

C10-C (L)

IN-C (L)

IN-R (L)

DN-126 (L)

CCC

Avg.

Source MEMO

56.5 65.6

18.0 25.0

36.2 40.9

54.7 53.2

16.8 19.3*

36.4 40.8

TestBN TENT EATA SAR RDUMB PeTTA AdaCont. I-AdaCont. CoTTA I-CoTTA RoTTA I-RoTTA PETAL I-PETAL

75.1 39.0 73.6 75.2 76.8 83.8 72.1 84.1+12.0 23.8 69.7+45.9 82.5 79.0-3.5 68.0 74.2+6.2

26.9 4.7 36.4 30.6 34.3 37.8 2.3 39.5+37.2 3.8 27.6+23.8 24.4 33.6+9.2 3.9 28.7+24.8

49.6 10.9 54.0 50.8 51.5 55.8 47.0 63.2+16.2 6.0 50.3+44.3 45.6 57.8+12.2 47.3 50.7+3.4

22.5 0.7 29.7 20.3 28.1 13.7 0.4 21.8+21.4 17.3 25.7+8.4 1.1 23.4+22.3 0.5 13.9+13.4

42.1 14.5 47.6 44.1 46.2 46.3 26.0 48.8+22.8 16.9 42.1+25.2 39.3 46.7+7.4 31.5 40.8+9.3

TestBN TENT EATA SAR RDUMB PeTTA AdaCont. I-AdaCont. CoTTA I-CoTTA RoTTA I-RoTTA PETAL I-PETAL

79.1 20.1 61.6 79.2 81.1 83.7 81.8 85.4+3.6 56.0 68.3+12.3 82.3 79.7-2.6 56.3 78.8+22.5

31.4 11.1 43.3 39.9 41.7 36.9 18.8 40.4+21.6 52.8 35.4-17.4 13.2 32.9+19.7 36.8 32.3-4.5

54.4 18.4 61.9 59.2 59.0 57.2 61.7 64.4+2.7 45.6 56.0+10.4 50.3 56.6+6.3 58.6 55.4-3.2

27.3 1.2 36.3 22.3 37.0 13.4 2.4 23.6+21.2 8.3 26.3+18.0 1.1 21.7+20.6 13.1 16.3+3.2

46.4 17.4 50.5 49.6 53.3 46.6 38.2 50.4+12.2 42.7 45.1+2.4 38.1 46.1+8.0 41.1 44.5+3.4

BATCH SIZE 10

36.2 17.4 44.0 43.6 40.1 40.5 8.0 35.3+27.3 33.7 37.4+3.7 43.0 39.9-3.1 37.9 36.5-1.4

BATCH SIZE 64

39.6 36.4 49.2 47.3 47.5 42.0 26.5 38.1+11.6 50.5 39.5-11.0 43.4 39.7-3.7 40.7 39.8-0.9

Intransigent teacher provides competitive stability. PeTTA (Hoang et al., 2024), a TTA method focused on longterm adaptation stability, employs a complex suite of techniques, including a class-balanced memory bank, robust batch normalization, weight- and prediction-based regularization from the source model, an adaptive EMA factor to control teacher updates, and the use of source feature statistics. Despite this complexity, our baseline methods, augmented with an intransigent teacher, achieve competitive or superior performance (Tab. 2). Specifically, I-AdaContrast outperforms PeTTA by 1.7 percentage points on CIFAR10-C (L) (batch size 64), while I-RoTTA leads by 2 percentage points on ImageNet-C (L) (batch size 10). On average, I-AdaContrast improves accuracy over PeTTA by 3.8 percentage points (50.4% vs. 46.6%). These results demonstrate that preventing error accumulation in the teacher model is a highly effective, yet simple, strategy for ensuring long-term stability. 6.2

D ETAILED ANALYSIS

Intransigent teachers work across different architectures. We verify the architectural universality of IT on CIFAR10-C (L) and ImageNet-C (L) using various network families (e.g., ViT, ConvNeXt). It should be noted that the RN26GN, ViT-B16, SViT-T, and CNX-tiny architectures do not include batch normalization layers, so IT’s performance is not impacted by statistics recalibration. Because default learning rates are architecture-specific and difficult to set without target labels in TTA, we explore an upper-bound scenario where the learning rate is tuned via an Oracle for 7

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

Table 3: Classification accuracy [%] with different neu- Table 4: Classification accuracy [%] for long (L) scenarral network architectures. Superscript indicates improve- ios on CIFAR10-C and ImageNet-C with 1×, 10×, 50× ments over the baseline. The learning rate is adjusted us- learning rate (LR) scaling. Superscript indicates improveing an Oracle on long (L) benchmarks. ments over the baseline.

Source AdaCont. I-AdaCont. CoTTA I-CoTTA RoTTA I-RoTTA PETAL I-PETAL

C10-C (L) RN26GN

RNXt-50

IN-C (L) ViT-B16 SViT-T

CNXt tiny

67.3 75.7 79.6+3.9 57.0 67.3+10.3 70.2 72.5+2.3 24.8 60.8+36.0

21.1 38.8 42.7+3.9 42.1 38.3-3.8 35.6 36.2+0.6 40.6 35.4-5.2

39.8 41.5 43.5+2.0 41.7 40.7-1.0 40.6 42.9+2.3 42.8 41.1-1.7

29.1 33.4 32.5-0.9 29.1 30.5+1.4 29.0 29.7+0.7 26.5 28.1+1.6

28.3 30.6 30.9+0.3 28.4 28.9+0.5 28.8 28.9+0.1 23.6 27.0+3.4

Source AdaCont. I-AdaCont. CoTTA I-CoTTA RoTTA I-RoTTA PETAL I-PETAL

1×LR

C10-C (L) 10×LR 50×LR

1×LR

IN-C (L) 10×LR 50×LR

81.8 85.4+3.6 56.0 68.3+12.3 82.3 79.7-2.6 56.3 78.8+22.5

56.5 83.6 86.1+2.5 10.3 51.9+41.6 80.6 77.9-2.7 15.9 59.7+43.8

18.8 40.4+21.6 52.8 35.4-17.4 13.2 32.9+19.7 36.8 32.3-4.5

18.0 19.6 36.3+16.7 4.1 25.7+21.6 0.2 4.1+3.9 0.4 36.2+35.8

80.4 85.2+4.8 10.1 41.4+31.3 28.9 70.6+41.7 18.0 35.8+17.8

12.4 32.7+20.3 0.1 17.3+17.2 0.1 2.1+2.0 0.1 8.1+8.0

both the baseline and IT-augmented methods (Tab. 3). This controlled comparison isolates architectural robustness by providing an optimal hyperparameter setting for less stability-oriented TTA methods. Under these conditions, IT consistently prevents catastrophic collapse and either improves upon or maintains competitive baseline accuracy across the tested architectures. The advantages of IT become even more pronounced in a more realistic evaluation setup, where the learning rate is tuned exclusively for the original methods on standard-length benchmarks and directly reused for IT-augmented versions (see Tab. A.10 in the Appendix). This demonstrates IT’s robustness to suboptimal, baselinespecific hyperparameters. While IT-augmented CoTTA and PETAL underperform their base counterparts specifically on ResNeXt50, both still exceed the source model accuracy, resulting in successful adaptation. We attribute this occasional underperformance to the plasticity bottleneck, discussed in Sec. 6.1. Further ablations in the Appendix confirm that IT maintains or improves robustness even when the source model’s initial quality is artificially degraded with noise (Sec. B.3). On robustness to learning rate selection. In Table 4, we verify the learning rate sensitivity. Default 75 AdaContrast CoTTA AdaContrast shows robustness when 10× learning rate I-AdaContrast 50 I-CoTTA is used, but collapses with a 50× learning rate on 50 25 ImageNet-C. IT allows increased robustness in all set25 tings and achieves a non-collapsed solution, even for the uniform 10.0 1.0 0.1 0.01 0.001 uniform 10.0 1.0 0.1 0.01 0.001 highest learning rate. CoTTA lacks learning rate robust80 75 PETAL ness, which is mitigated by our IT extension. Our stratI-PETAL 50 60 RoTTA egy improves RoTTA’s performance on CIFAR10-C. 25 I-RoTTA However, it rendered our approach almost ineffective on 40 ImageNet-C. Overall, IT does not fully prevent the coluniform 10.0 1.0 0.1 0.01 0.001 uniform 10.0 1.0 0.1 0.01 0.001 Dirichlet parameter lapse when significantly altering the learning rate, but seems to promote a better-performing parameter space. Further analyses of hyperparameter tuning complexity Figure 5: Classification accuracy on CIFAR10-C (L). Sam(using Oracle and Transfer setups) are detailed in Ap- ples are sorted by class for different levels of correlation, by pendix Sec. B.4, and we show that our observations varying the Dirichlet concentration parameter δ. extend to the semantic segmentation task in Appendix Sec. B.2. 1.0

Accuracy [%]

0.8

0.6

0.4

0.2

0.0

0.0

0.2

0.4

Stud. Acc. - IT Acc. [pp]

1.0

10

0.6

0.8

10

1.0

C10-C (L)

Effects varying temporal correlation. Figure 5 illusIN-C (L) 8 5 trates the impact of varying degrees of class temporal IN-R (L) DN-126 (L) 6 correlation (Gong et al., 2022) on methods enhanced 0 CCC with IT. The analysis reveals that as temporal correla4 5 tion increases, IT-enhanced methods consistently outper2 form their original versions, demonstrating improved ro10 0 bustness in these scenarios, and even when the original I-AdaCont. I-CoTTA I-RoTTA I-PETAL I-AdaCont. I-CoTTA I-RoTTA I-PETAL method achieved better results on uniform class distriMethod bution (RoTTA). However, it should be noted that while Figure 6: Accuracy difference between student and intranIT-enhanced methods show improved performance, they sigent teacher in percentage points, with default learning are also negatively impacted by the effects of correla- rate values (left) and after parameter search (right). tion. The degree of improvement varies across different experimental settings. 0.8

0.6

0.4

0.2

0.0

8

0.0

0.2

0.4

0.6

0.8

1.0

45 40 35 30 25 20 15

40

ET - Teacher ET - Student IT - Teacher IT - Student Source

0

2

4

6

Accuracy [%]

Accuracy [%]

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

30 20 10 0 0

8 10 12 14 16 18 20 Loop No.

ET - Teacher ET - Student IT - Teacher IT - Student Source 2 4 6 8 10 12 14 16 18 20 Loop No.

NFR

Figure 7: Per-batch accuracy on repeated ImageNet-C (20 loops) using ResNet50. AdaContrast (left) and RoTTA (right) are compared with an EMA teacher (ET, orange) and our intransigent teacher (IT, blue), both for teacher (solid) and student (dashed). The EMA teacher visibly suffers from error accumulation as the sequence gets longer. After the 3rd loop, IT manages to avoid model collapse and, surprisingly, the student performs significantly better.

0.4

C10-C (L)

IN-C (L)

ET IT

ET IT

0.4

0.0

0

4

8

12 16 20

Loop No.

0.0

0

4

8

12 16 20

Loop No.

0.0

DN-126 (L) ET IT

0.1

0.2

0.2

IN-R (L)

0.2

0

4

8

12 16 20

Loop No.

ET IT

0.2

0.0

0

4

8

12 16 20

Loop No.

Figure 8: Per-batch Negative Flips Rate (NFR) of the student model averaged over AdaContrast, CoTTA, RoTTA and PETAL methods (ET) on each dataset, along with their IT-enhanced versions (IT). The shaded regions represents ± standard deviation illustrating the variability across methods.

Are students better than intransigent teachers? When paired with intransigent teachers, student models often outperform their teachers, whereas EMA keeps teacher-student performance tightly coupled (Fig. 7). As shown in Figure 6, IT enables the student to exceed teacher average accuracy in nearly all configurations, with only two exceptions: I-AdaContrast on ImageNet-R (L) and I-CoTTA on CIFAR10-C (L). In those cases, IT-induced stability limits performance decline but is insufficient to prevent degradation entirely. However, when the learning rate is optimized via an Oracle, students outperform teachers in all cases (Fig. 6, right). Per-batch accuracy plots for remaining method–dataset combinations are presented in the Appendix (Figs. A.9 and A.10). Evaluating the error accumulation. We hypothesize that the IT-enhanced methods’ success in longer scenarios stems from their ability to mitigate error accumulation. We attempt to compare the degree of accumulated error between methods and employ the Negative Flip Rate (NFR) as defined in Yan et al. (2021). The NFR quantifies the fraction of samples initially predicted correctly by the source model but incorrectly by the adapted model. This metric captures degradation of the initial model’s knowledge, the direct effect of error accumulation. In Figure 8, we show per-batch NFR of the student model averaged over AdaContrast, CoTTA, RoTTA, and PETAL methods for each benchmark. For specific examples, see the Appendix (Fig. A.11). The NFR for IT-enhanced methods remains consistently stable, showing no increasing trend across any benchmark. It suggests that the fixed weights of the teacher prevent the student from accumulating errors, thereby preserving the initial knowledge. In contrast, the original methods exhibit increasing NFR in most experiments, indicating that the adaptation process degrades the model’s knowledge, which suggests error accumulation. In some experiments, the original methods exhibit a stable, low NFR, indicating reliable adaptation that correctly improves the model, which also correlates with higher accuracy in Table 2. Complementary to these findings, we observe that IT also prevents the internal degradation of the model’s feature space. As detailed in Appendix Sec. B.1, tracking the effective rank of the student’s representations reveals that baseline collapses correlate with a severe decrease in feature diversity. By anchoring the adaptation process, IT consistently maintains high effective rank and prevents dimensional collapse. Overall, IT-enhanced methods offer more consistent behavior and robustness across benchmarks. 9

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

7

C ONCLUSIONS

In this work, we explore the well-known strategy of using an exponential moving average teacher for TTA. Although effective for common sequence lengths, this strategy fails to mitigate long-term error accumulation, which can lead to model collapse. To address the resulting plasticity-stability trade-off, we propose an intransigent teacher strategy compatible with current state-of-the-art methods. Our approach often prevents model collapse across diverse experimental setups and improves robustness to hyperparameter choice without requiring manual tuning. Empirical results demonstrate that the intransigent teacher halts the accumulation of errors and helps to maintain high feature diversity. These findings suggest that TTA evaluation must encompass varied sequence lengths to ensure reliability. Our method provides a robust, simple baseline for future long-term adaptation research. Discussion and limitations. While the intransigent teacher strategy proved highly effective across various scenarios, it can result in slower adaptation in certain cases due to the teacher’s fixed behavior. To address this issue, we also present results where the teacher model was allowed to adapt for a fixed number of initial steps (Appendix Sec. B.8). Further exploration of this approach is left for future work. Furthermore, we note that by adding the intransigent teacher to existing methods, we inherit their limitations. ACKNOWLEDGMENTS This research was funded in whole or in part by National Science Centre, Poland, grants no 2024/53/N/ST6/03156 and 2023/51/D/ST6/02846. We gratefully acknowledge Polish high-performance computing infrastructure PLGrid (HPC Center: ACK Cyfronet AGH) for providing computer facilities and support within computational grants no. PLG/2025/018634, PLG/2025/018644, and PLG/2026/019363. This research was funded in whole or in part by the Austrian Science Fund (FWF) 10.55776/COE12. Bartłomiej Twardowski acknowledges the grant RYC2021-032765-I.

R EFERENCES Motasem Alfarra, Hani Itani, Alejandro Pardo, Shyma Alhuwaider, Merey Ramazanova, Juan C Pérez, Zhipeng Cai, Matthias Müller, and Bernard Ghanem. Evaluation of test-time adaptation under computational time constraints. In ICML, 2024. Malik Boudiaf, Romain Müller, Ismail Ben Ayed, and Luca Bertinetto. Parameter-free online test-time adaptation. In CVPR, pp. 8334–8343, 2022. Dhanajit Brahma and Piyush Rai. A probabilistic framework for lifelong test-time adaptation. In CVPR, pp. 3582– 3591, 2023. Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems, 33:15920–15930, 2020. Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018. Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In CVPR, pp. 295–305, 2022. Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 40(4):834–848, 2017. Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: a standardized adversarial robustness benchmark. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021. URL https://openreview.net/forum?id=SSKZPJCt7B. Sebastian Cygert, Damian Sójka, Tomasz Trzciński, and Bartłomiej Twardowski. Realistic evaluation of test-time adaptation: Unsupervised model selection. In Proceedings of the 21st International Conference on Computer Vision Theory and Applications (VISAPP 2026) - Volume 2, pp. 75–86. INSTICC, SciTePress, 2026. ISBN 978-989-758744-3. doi: 10.5220/0014320100004084. 10

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pp. 248–255, 2009. Mario Döbler, Robert A. Marsden, and Bin Yang. Robust mean teacher for continual and gradual test-time adaptation. CoRR, abs/2211.13081, 2022. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021. Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born-again neural networks. In ICML, 2018. Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In ICLR, 2019. Taesik Gong, Jongheon Jeong, Taewon Kim, Yewon Kim, Jinwoo Shin, and Sung-Ju Lee. NOTE: robust continual test-time adaptation against temporal correlation. In NeurIPS, 2022. Taesik Gong, Yewon Kim, Taeckyung Lee, Sorn Chottananurak, and Sung-Ju Lee. SoTTA: Robust test-time adaptation on noisy data streams. In NeurIPS, 2023. Dipam Goswami, Yuyang Liu, Bartłomiej Twardowski, and Joost van de Weijer. Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning. NeurIPS, 36, 2024. Sachin Goyal, Mingjie Sun, Aditi Raghunathan, and J Zico Kolter. Test time adaptation via conjugate pseudo-labels. NeurIPS, 35:6204–6218, 2022. Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, et al. Bootstrap your own latent - a new approach to self-supervised learning. In NeurIPS, 2020. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In CVPR, pp. 9729–9738, 2020. Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. ICLR, 2019. Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In ICCV, pp. 8340–8349, 2021. Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. ArXiv, 2015. Trung Hieu Hoang, MinhDuc Vo, and Minh Do. Persistent test-time adaptation in recurring testing scenarios. Advances in Neural Information Processing Systems, 37:123402–123442, 2024. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences (PNAS), 2017. Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran S. Haque, Sara M. Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A benchmark of in-the-wild distribution shifts. In ICML, 2021. Alex Krizhevsky. Learning multiple layers of features from tiny images. pp. 32–33, 2009. URL https://www. cs.toronto.edu/˜kriz/learning-features-2009-TR.pdf. Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. In ICLR, 2017. Shuang Li, Longhui Yuan, Binhui Xie, and Tao Yang. Generalized robust test-time adaptation in continuous dynamic scenarios. arXiv preprint arXiv:2310.04714, 2023. Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. Revisiting batch normalization for practical domain adaptation. ArXiv, abs/1603.04779, 2016. 11

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

Sheng Liu, Jonathan Niles-Weed, Narges Razavian, and Carlos Fernandez-Granda. Early-learning regularization prevents memorization of noisy labels. NeurIPS, 33, 2020. Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pp. 10012–10022, 2021. Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. CVPR, 2022. Chunwei Ma, Zhanghexuan Ji, Ziyun Huang, Yan Shen, Mingchen Gao, and Jinhui Xu. Progressive voronoi diagram subdivision enables accurate data-free class-incremental learning. In The Eleventh International Conference on Learning Representations, 2023. Sarthak Kumar Maharana, Shambhavi Mishra, Yunbei Zhang, Shuaicheng Niu, Taki Hasan Rafi, Jihun Hamm, Marco Pedersoli, Jose Dolz, and Yunhui Guo. Continual test-time adaptation: A comprehensive survey. 2026. TorchVision maintainers and contributors. Torchvision: Pytorch’s computer vision library. https://github. com/pytorch/vision, 2016. Robert A. Marsden, Mario Döbler, and Bin Yang. Universal test-time adaptation through weight ensembling, diversity weighting, and prior correction. In WACV, 2024a. Robert A Marsden, Mario Döbler, and Bin Yang. Introducing intermediate domains for effective self-training during test-time. In IJCNN, pp. 1–10. IEEE, 2024b. Marc Masana, Xialei Liu, Bartłomiej Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost van de Weijer. Classincremental learning: survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022. Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin. The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects. Frontiers in psychology, 4:54654, 2013. Duc Tam Nguyen, Chaithanya Kumar Mummadi, Thi Phuong Nhung Ngo, Thi Hoai Phuong Nguyen, Laura Beggel, and Thomas Brox. Self: Learning to filter noisy labels with self-ensembling. In ICLR, 2020. Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In ICML, volume 162, pp. 16888–16905, 2022. Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In ICLR, 2023. Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. Aristeidis Panos, Yuriko Kobe, Daniel Olmeda Reino, Rahaf Aljundi, and Richard E Turner. First session adaptation: A strong replay-free baseline for class-incremental learning. In ICCV, pp. 18820–18830, 2023. Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, et al. Moment matching for multi-source domain adaptation. In ICCV, pp. 1406–1415, 2019. Ori Press, Steffen Schneider, Matthias Kümmerer, and Matthias Bethge. Rdumb: A simple approach that questions our progress in continual test-time adaptation. In Advances in Neural Information Processing Systems 36:, 2023. Olivier Roy and Martin Vetterli. The effective rank: A measure of effective dimensionality. In 2007 15th European signal processing conference, pp. 606–610. IEEE, 2007. Evgenia. Rusak, Steffen Schneider, George Pachitariu, Luisa Eck, Peter Gehler, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. If your data distribution shifts, use self-learning. Transactions of Machine Learning Research, 2022. URL https://openreview.net/forum?id=vqRzLv6POg. Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Advances in neural information processing systems, 33:11539–11551, 2020a. 12

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. NeurIPS, 33:11539–11551, 2020b. Ziqi Shi, Fan Lyu, Ye Liu, Fanhua Shang, Fuyuan Hu, Wei Feng, Zhang Zhang, and Liang Wang. Controllable continual test-time adaptation. In 2025 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE, 2025. Damian Sójka, Bartłomiej Twardowski, Tomasz Trzciński, and Sebastian Cygert. Ar-tta: A simple method for realworld continual test-time adaptation. In BMVC, 2023. Yongyi Su, Xun Xu, and Kui Jia. Towards real-world test-time adaptation: Tri-net self-training with balanced normalization. In AAAI, volume 38, pp. 15126–15135, 2024. Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A. Efros, and Moritz Hardt. Test-time training with selfsupervision for generalization under distribution shifts. In ICML, volume 119, pp. 9229–9248, 2020. Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. NeurIPS, 30, 2017. Benedikt Tscheschner, Eduardo Veas, and Marc Masana. Incremental learning with repetition via pseudo-feature projection. In Proceedings of the 28th Computer Vision Winter Workshop, CVWW 2025, February 2025. doi: 10.3217/978-3-99161-022-9-004. 28th Computer Vision Winter Workshop, CVWW 2025 ; Conference date: 1202-2025 Through 14-02-2025. Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno A. Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In ICLR, 2021. Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In CVPR, pp. 7191– 7201, 2022. Yuxin Wu and Kaiming He. Group normalization. In ECCV, volume 11217, pp. 3–19, 2018. Zehao Xiao and Cees GM Snoek. Beyond model adaptation at test time: A survey. arXiv preprint arXiv:2411.03687, 2024. Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In CVPR, pp. 1492–1500, 2017. Sijie Yan, Yuanjun Xiong, Kaustav Kundu, Shuo Yang, Siqi Deng, Meng Wang, Wei Xia, and Stefano Soatto. Positivecongruent training: Towards regression-free model updates. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14299–14308, 2021. Longhui Yuan, Binhui Xie, and Shuang Li. Robust test-time adaptation in dynamic scenarios. In CVPR, pp. 15922– 15932, 2023. Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016. Marvin Zhang, Sergey Levine, and Chelsea Finn. MEMO: test time robustness via adaptation and augmentation. In NeurIPS, 2022. Hao Zhao, Yuejiang Liu, Alexandre Alahi, and Tao Lin. On pitfalls of test-time adaptation. In ICML, 2023. Xingzhi Zhou, Zhiliang Tian, Ka Chun Cheung, Simon See, and Nevin L. Zhang. Resilient practical test-time adaptation: Soft batch normalization alignment and entropy-driven memory bank. ArXiv, 2024. Xingzhi Zhou, Boyang Zhang, Zhiliang Tian, Yibo Zhang, Xin Niu, Ka Chun Cheung, Simon See, and Nevin L. Zhang. Resilient test-time adaptation by mitigating batch-normalization overfitting. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, 2025. doi: 10.1109/ ICASSP49660.2025.10888647.

13

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

A PPENDIX This appendix provides comprehensive evidence and analysis to support the findings presented in the main paper. Sec. A details the experimental framework, including the setup (Sec. A.1), baseline descriptions and implementations (Sec. A.2, A.3), and specific details for the intransigence experiments (Sec. A.4) and the MEMO results on CCC benchmark (Sec. A.5). It also covers the negative flip rate (NFR) calculation (Sec. A.6) and computational requirements (Sec. A.7). Sec. B presents several additional experiments to better understand the described problem and the limitations of the proposed technique. These include evaluation of feature representation strength (Sec. B.1), semantic segmentation experiment (Sec. B.2), analysis of the dependency on source model performance (Sec. B.3), experiment with tuning the learning rate value for long scenarios (Sec. B.4), wall-clock time comparison (Sec. B.5), discussion on single image adaptation methods (Sec. B.6), results on common-length benchmarks (Sec. B.7), an investigation of the potential of the adaptive β value for teacher momentum (Sec. B.8), an analysis of student-only adaptation performance (Sec. B.9), an extended experiment showing the effects of intransigence for over 100 loops with the CoTTA method (Sec. B.10), discussion on model reset mechanism (Sec. B.11), the novelty clarification with regard to RDumb (Sec. B.12), augmentation count ablation for CoTTA and PETAL methods B.13, decomposition of the IT student’s gain into BN statistics recalibration and teacher distillation B.14, and experiments with fixed batch normalization statistics B.15. We also include the remaining results of the experiments mentioned in the main paper (Sec. C).

A

A DDITIONAL D ETAILS

A.1

E XPERIMENTAL SETUP

Benchmarks. Our experiments include popular corruption benchmarks CIFAR10-C (C10-C) and ImageNet-C (INC) (Hendrycks & Dietterich, 2019), training on clean CIFAR10 (Krizhevsky, 2009) or ImageNet (Deng et al., 2009) images and testing on corrupted images with 15 corruption types at 5 severity levels. We adopt the setup from Wang et al. (2022); Niu et al. (2022); Döbler et al. (2022), using the standard corruption sequence at the highest severity. The adaptation on natural domain shifts is tested utilizing DomainNet-126 (DN-126) (Peng et al., 2019) and ImageNetR (IN-R) (Hendrycks et al., 2021) datasets. For DN-126, we use the real domain as the source data and conduct experiments on the remaining domains. The clean ImageNet dataset serves as the source data for experiments on ImageNet-R. Our goal is to focus on very long adaptation sequences to evaluate the methods in terms of stability during long-time operation. For that reason, we repeat the standard test sequences of benchmarks described above 20 times. We call this setup a Long (L) scenario. Additionally, we take advantage of the CCC (Press et al., 2023) benchmark, which has a very high sequence length without repetitions. We use a CCC-Medium sequence with a 1k transition speed, which consists of 7,500,000 images. For experiments with semantic segmentation, we utilize the CarlaTTA dataset (Marsden et al., 2024b). The source data consists of images from the clear domain, i.e., images captured during the day in clear weather. For test data, we use the day2night sequence, which goes from the clear domain, gradually transitioning to night. Consistent with image classification experiments, we used the same approach to create the Long (L) scenario. Architectures. Consistent with previous works (Wang et al., 2022; Marsden et al., 2024a), we use WideResNet-28 (Zagoruyko & Komodakis, 2016) models with pre-trained weights from the RobustBench (Croce et al., 2021) model zoo for the main experiments on CIFAR10-C. Similarly, the tests on ImageNet-based benchmarks and DomainNet-126 employ the ResNet50 network with weights sourced from the same model zoo or those provided by Marsden et al. (2024a) for DomainNet-126. Additional experiments in Sec. 6.2 are carried out with ResNet-26GN (RN26GN) (Wu & He, 2018), ResNeXt-50 32x4d (RNXt-50) (Xie et al., 2017), ViT-B16 (Dosovitskiy et al., 2021), SwinViT-T (SViT-T) (Liu et al., 2021) and ConvNeXt tiny (CNXt tiny) (Liu et al., 2022) architectures. Weights for RN26GN are taken from Zhang et al. (2022), as in Marsden et al. (2024a). The Torchvision (maintainers & contributors, 2016) library is used to initialize the rest of the mentioned models. For semantic segmentation, following Marsden et al. (2024b), we utilize the DeepLab-V2 (Chen et al., 2017) architecture with ResNet-101 backbone. We use the pre-trained source checkpoint provided by Marsden et al. (2024b). Implementation details. As a testbed for experiments, we adopt the framework from Marsden et al. (2024a) and use the parameters reported in the respective original papers. We use parameters from common sequence length settings while testing on the long (L) scenarios, unless otherwise stated. We utilize two batch sizes for image classification: a standard value of 64, as in multiple previous works (Marsden et al., 2024a; Niu et al., 2022; Press et al., 2023; Wang et al., 2021), and a lower one (equal to 10), resulting in a more challenging setup with more model updates. When running experiments on a smaller batch size, the learning rate is decreased accordingly. In the case of semantic 14

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

segmentation, we follow the usual batch size of 1 (Marsden et al., 2024b). In-depth implementation details regarding the baselines are provided in Sec. A.3). A.2

D ESCRIPTIONS OF BASELINES EXTENDED WITH THE I NTRANSIGENT T EACHER

AdaContrast conducts weight updates using a three-part loss function: cross-entropy loss, diversity regularization, and contrastive loss. Pseudo-labels are refined by keeping a buffer of previous image features and their predictions. Refined predictions are based on the nearest neighbors of the current feature within the buffer. Statistics in batch normalization layers are updated with EMA. CoTTA updates the student model by minimizing the cross-entropy consistency between the teacher and the student predictions. Depending on the prediction confidence, the pseudo-labels are the result of averaging predictions on multiple, differently augmented images. At each iteration, there is a small probability for each of the student’s weights to be reset to the source pre-trained value. It calculates batch normalization statistics on a per-batch basis. RoTTA keeps the class-balanced memory buffer of images and uses it to perform the optimization in constant intervals. The loss is weighted based on how long the sample has been stored. Cross-entropy-based consistency between the student and teacher models is utilized for a loss function. Batch normalization statistics are updated via EMA. PETAL is similar to CoTTA, but it enhances the learning objective by the regularizer term based on a posterior distribution over a source model weights and a data-dependent prior. Moreover, it improves CoTTA’s stochastic model reset scheme with the Fisher Information Matrix. A.3

BASELINES IMPLEMENTATION DETAILS

The experiments are conducted by adapting the code repository of previous test-time adaptation works (Marsden et al., 2024a; Döbler et al., 2022), which provide the implementation of all evaluated state-of-the-art methods, except PETAL and PeTTA. We integrated its original implementation into this unified codebase ourselves. In terms of hyperparameters, we followed the typical implementation for test of batch size of 64. TENT (Wang et al., 2021), EATA (Niu et al., 2022), SAR (Niu et al., 2023), and RDUMB (Press et al., 2023) use Adam optimizer with a learning rate of 0.001 for CIFAR10-C and SGD optimizer with a learning rate of 0.00025 for other benchmarks. AdaContrast (Chen et al., 2022) utilizes an SGD optimizer with a learning rate set to 0.0002 for all of the benchmarks. CoTTA (Wang et al., 2022) uses Adam optimizer with a learning rate of 0.001 for CIFAR10-C and SGD with a learning rate of 0.01 for the rest of the benchmarks. Adam optimizer with a learning rate set to 0.001 is used by RoTTA (Yuan et al., 2023) for all of the tested datasets. MEMO (Zhang et al., 2022) uses an SGD optimizer with a learning rate of 0.005 for CIFAR10-C and 0.00025 for other datasets. PETAL (Brahma & Rai, 2023) in the original paper uses Adam optimizer with a learning rate of 0.001 for CIFAR10-C and SGD with a learning rate of 0.01 for other datasets. However, since we often experienced poor performance using these values on long scenarios, we utilized 10 times lower learning rates. Due to PETAL’s regularizer term becoming unstable at higher learning rates, which leads to model corruption, we excluded PETAL from the learning rate scaling experiments shown in Tab. 4. This decision avoided presenting potentially compromised results. PeTTA (Hoang et al., 2024) use Adam optimizer with learning rate of 0.001 for all of the benchmarks. The learning rate used in experiments with batch size set to 10 was adjusted accordingly by scaling it linearly. CoTTA (Wang et al., 2022) and PETAL (Brahma & Rai, 2023) methods update the student network using a consistency loss between the student and teacher. If the prediction confidence of the source model is below a certain threshold, the teacher’s predictions are averaged over 32 different augmentations of the image which adds 31 additional forward operations per batch. It creates a significant computation overhead and causes the methods to be significantly slower, compared to other state-of-the-art methods. It is especially problematic for long adaptation scenarios, which make the main part of our experiments. Our ablations indicate that using a single augmentation does not alter the results notably (Sec. B.13). Therefore, for the ease of experimentation, we reduce the number of augmentations to 1. A.4

D ETAILS ON INTRANSIGENT TEACHER EXPERIMENT FROM S ECTION 4.1

Our preliminary intransigent teacher experiment is conducted on ImageNet-C, a long ImageNet-C scenario (20× loop of standard ImageNet-C testing sequence), and CCC. We utilize the same model as for our main experiments on ImageNet-based benchmarks - ResNet50 with pre-trained weights from the RobustBench (Croce et al., 2021) model zoo. We use a single loss within the teacher-student framework for the model adaptation during test time – either Consistency from CoTTA or Contrastive from AdaContrast. Any other components of the mentioned state-of-the15

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

art methods are not included. Batch normalization statistics are recalculated for each batch. For both of the tested approaches, we use the SGD optimizer with a learning rate of 0.00025. A.5

D ETAILS ON MEMO RESULTS ON CCC BENCHMARK FROM TAB . 2

The result is based on the first 623,000 images of the benchmark, providing an initial estimate of the method’s accuracy. However, due to the benchmark’s extensive size (7,500,000 images) and the method’s requirement for a batch size of 1, we cannot complete the full experiment in a reasonable amount of time. We estimate that processing the entire dataset requires approximately 972 hours on a single NVIDIA GeForce RTX 4080 GPU. This substantial time requirement underscores the method’s significant computational inefficiency. A.6

D ETAILS ON N EGATIVE F LIP R ATE

In Sec. 6.2, we attempt to compare the degree of accumulated error between methods and employ the Negative Flip Rate (NFR) as defined in Yan et al. (2021). The NFR quantifies the fraction of samples initially predicted correctly by the source model but incorrectly by the adapted model, calculated as: N

NFR =

1 X 1(ŷi0 = yi , ŷi ̸= yi ) N i=1

(6)

where yi is the ground truth label, ŷi0 is the source model’s prediction, ŷi is the current model’s prediction, and 1(·) is the indicator function. This metric captures degradation of the initial model’s knowledge, the direct effect of error accumulation. A.7

C OMPUTE DETAILS

All experiments are conducted on a single GPU. We utilize either NVIDIA A100 with 40GB of memory or NVIDIA GeForce RTX 4080 with 16GB of memory. The execution time of the experiment greatly varied and was dependent on the dataset, scenario (standard or long), tested method, and batch size. The fastest experiments take about 30 minutes, whereas the longest last up to 36 hours.

B

A DDITIONAL E XPERIMENTS

B.1

E VALUATION OF FEATURE REPRESENTATION STRENGTH .

To gain additional insight into why the IT works effectively, we assess changes in the representation strength of features extracted from the model’s encoder during the adaptation process. We focus on measuring the diversity of the features and compute the effective rank (Roy & Vetterli, 2007) of the feature representations during the adaptation process. We collect a domain-balanced set of n (n = 150) randomly sampled images from the adaptation sequence of a given benchmark. During TTA, we extract features for this entire set of images using the encoder of the adapted model and construct a feature matrix A. We then compute its singular σi values σ1 , · · · , σn . Given the distribution of singular values, defined as pi = |σ| , we calculate the effective rank of 1 feature representation matrix A as follows: E-rank(A) = exp(−

n X

pi log(pi )).

(7)

i=1

We track the effective rank of the feature representations of the student model every 50 batches throughout the adaptation sequence. The representative results are presented in Fig. A.1. We observe that a significant drop in effective rank (e.g., a two-fold decrease) correlates with a collapsed solution, where accuracy falls below that of the source model. This pattern is evident in four out of six collapse cases with a batch size of 64, as reported in Tab. 2: CoTTA and PETAL on C10-C (L), RoTTA on IN-C (L), and CoTTA on DN126 (L). The use of IT helps maintain higher feature diversity, thereby preventing collapse. For the remaining collapse cases, the effective rank also exhibits a slight downward trend: AdaContrast on IN-R (L) and RoTTA on DN-126 (L). 16

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

Effective Rank

C10-C (L) 100

IN-C (L)

IN-R (L)

100

50

AdaContrast I-AdaContrast 0

4

8

100

50

12 16 20

0

Loop No.

4

8

100

50

AdaContrast I-AdaContrast 12 16 20

AdaContrast I-AdaContrast 0

Loop No.

DN-126 (L)

4

8

50

12 16 20

AdaContrast I-AdaContrast 0

Loop No.

4

8

12 16 20

Loop No.

(a) AdaContrast

Effective Rank

C10-C (L) CoTTA I-CoTTA

100 50 0

4

8

IN-C (L)

IN-R (L)

DN-126 (L)

100

100

100

50

50

50

12 16 20

CoTTA I-CoTTA 0

Loop No.

4

8

12 16 20

CoTTA I-CoTTA 0

Loop No.

4

8

12 16 20

CoTTA I-CoTTA 0

Loop No.

4

8

12 16 20

Loop No.

(b) CoTTA

Effective Rank

C10-C (L) RoTTA I-RoTTA

100 50

IN-C (L) 100

100

50 0

4

8

IN-R (L)

12 16 20

0

Loop No.

4

8

100

50

RoTTA I-RoTTA 12 16 20

RoTTA I-RoTTA 0

Loop No.

DN-126 (L)

4

8

50

12 16 20

RoTTA I-RoTTA 0

Loop No.

4

8

12 16 20

Loop No.

(c) RoTTA

Effective Rank

C10-C (L) PETAL I-PETAL

100 50

IN-C (L) 100

100

50 0

4

8

IN-R (L)

12 16 20

Loop No.

0

4

8

100

50

PETAL I-PETAL 12 16 20

Loop No.

DN-126 (L)

PETAL I-PETAL 0

4

8

12 16 20

Loop No.

50

PETAL I-PETAL 0

4

8

12 16 20

Loop No.

(d) PETAL Figure A.1: Effective rank values of the features extracted from the student model during the TTA every 50 batches on each benchmark using AdaContrast (a), CoTTA (b), RoTTA (c) and PETAL (d) methods and IT-enhanced version.

B.2

S EMANTIC SEGMENTATION

We evaluate a broader range of tasks and report the mean intersection-over-union (mIoU) results for the semantic segmentation experiment with different learning rate values across the entire long (L) scenario in Tab. A.1, with results for each loop presented separately in Fig. A.2. We focus exclusively on the CoTTA method, as it is the only baseline developed specifically for this task by its authors. The results demonstrate that our initial observations extend to the semantic segmentation task: CoTTA, which utilizes EMA teacher, is highly sensitive to learning rate selection and degrades model performance over time for all chosen 17

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

values, including the default for the standard-length sequence (2.5e-4), except for the lowest value (where its plot line is occluded by the I-CoTTA plot). In contrast, I-CoTTA exhibits greater robustness to learning rate selection, and IT effectively mitigates the error accumulation issue.

60

2.5e-2 2.5e-3

40

Source CoTTA I-CoTTA

30

2.5e-4

20

Learning Rate

mIoU [%]

50

2.5e-5

10

2.5e-6

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

Figure A.2: Segmentation results (mIoU [%]) for each repeated day2night sequence on CarlaTTA (Marsden et al., 2024b). Orange and blue colors indicate CoTTA and I-CoTTA, respectively. The darker the color, the higher the learning rate value used to achieve the result. The black dashed line indicates the source model performance as a reference.

Table A.1: Average segmentation (mIoU [%]) on the long day2night sequence from CarlaTTA Marsden et al. (2024b) with different learning rate (LR) values. The superscript indicates the improvements over the baseline. LR

2.5e-2

2.5e-3

2.5e-4

2.5e-5

2.5e-6

Source CoTTA I-CoTTA

4.9 21.6+16.7

33.9 54.3+20.4

58.4 50.9 60.8+9.9

58.0 61.3+3.3

61.5 61.7+0.2

IN-R (L)

DN-126 (L) AdaContrast I-AdaContrast Source

50

CoTTA I-CoTTA Source

25

Accuracy [%]

0 75 50 25

Accuracy [%]

Accuracy [%]

Accuracy [%]

C10-C (L) 75 50 25

75

RoTTA I-RoTTA Source PETAL I-PETAL Source

50 25 0.00

0.15 0.20

0.30 0.00

0.10

0.15

0.20 0.00

0.10

0.15

0.20

Figure A.3: The overall average accuracy on each of the tested benchmarks of the original methods and IT-enhanced versions using source model weights perturbed using Gaussian noise with increasing standard deviation σ. The σ = 0 indicates the performance of the model not perturbed by noise. The brown dashed line indicated the source model accuracy. 18

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

B.3

D EPENDENCY ON SOURCE MODEL PERFORMANCE

Since the intransigent teacher strategy fixes the teacher model’s weights, its performance may heavily depend on the quality of the source model and could fail if the model is poorly suited to the target domain. To test this hypothesis, we evaluate adaptation performance using source models of varying quality. We generate these models by taking our default model weights and perturbing them with Gaussian noise of varying standard deviation, σ. For each layer, the noise’s standard deviation is scaled by the mean value of that layer’s weights. The average accuracies across the entire benchmark for each σ are presented in Fig. A.3. We compare the accuracy trends of the original methods with those enhanced by the IT strategy. The results indicate that, in most cases, weaker source models lead to poorer performance in the original baselines. IT-enhanced methods either follow the same trend as the original baselines (e.g., I-AdaContrast and I-RoTTA on C10-C (L), I-PETAL and I-RoTTA on IN-R (L), I-AdaContrast on DN-126 (L)) or exhibit greater robustness to weaker initial training (e.g., I-CoTTA and I-PETAL on C10-C (L), I-CoTTA and I-PETAL on DN-126 (L)). In other words, IT either maintains the original method’s sensitivity to source model quality or improves its robustness. We hypothesize that this behavior occurs because, even when the source model’s predictions are less accurate, the fixed teacher model is prevented from further performance degradation and error accumulation due to the student’s noisy adaptation, while still allowing the student model to update (via backpropagation) on the target data. Our observations align with those from Zhao et al. (2023), where the authors show that, in general, the performance of TTA methods is highly dependent on the quality of the initial source model. Table A.2: Classification accuracy [%] for long scenarios with the learning rate (LR) parameter tuned. LR value Default means that the default LR value for the method was used. Transfer IN-C indicates that the LR is tuned utilizing the ImageNet-C benchmark with ground truth labels. In the Oracle rows the learning rate is chosen separately for each of the datasets with Oracle. The batch size is equal to 64. Method AdaCont. I-AdaCont.

CoTTA I-CoTTA

RoTTA I-RoTTA

B.4

LR

C10-C (L)

IN-C (L)

IN-R (L)

DN-126 (L)

Avg.

Default Transfer IN-C Oracle Default Transfer IN-C Oracle

81.8 81.2 82.6 85.4 85.4 86.2

18.8 36.1 36.1 40.4 40.4 40.4

26.5 40.8 40.8 38.2 38.2 42.3

61.7 59.7 62.7 64.4 64.4 64.4

47.2 54.5 55.6 57.1+9.9 57.1+2.6 58.3+2.7

Default Transfer IN-C Oracle Default Transfer IN-C Oracle

56.0 11.2 75.8 68.4 52.0 79.2

52.8 52.8 52.8 35.4 35.4 35.4

50.5 50.5 50.5 39.6 39.6 40.8

45.6 45.6 58.7 56.8 56.8 56.8

51.2 40.0 59.5 50.1-1.1 46.0+6.0 53.1-6.4

Default Transfer IN-C Oracle Default Transfer IN-C Oracle

82.3 73.2 82.3 79.6 79.3 79.6

13.2 30.6 30.6 32.7 33.3 33.3

43.4 41.0 43.4 39.7 39.9 39.9

50.3 55.3 55.3 57.2 57.2 57.2

47.3 50.1 52.9 52.3+5.0 52.4+2.3 52.5-0.4

T UNING THE LEARNING RATE VALUE FOR LONG SCENARIOS

We investigate whether tuning the learning rate, arguably the most crucial hyperparameter, could enhance the performance of baseline methods in long adaptation scenarios. Following a realistic approach, we employ an Oracle technique on ImageNet-C (L) as a reference benchmark (inspired by Rusak et al. (2022), we name it Transfer IN-C) and apply the selected learning rate across all datasets. We also present results when the learning rate is chosen separately for each of the datasets with an Oracle. Tab. A.2 reports the complexity of hyperparameter optimization in test-time adaptation. Our findings show the challenges of hyperparameter tuning. For instance, CoTTA achieves superior accuracy with its default learning rate compared to the version tuned on ImageNet-C. While both AdaContrast and RoTTA show improvements when using Transfer IN-C, our IT approach consistently outperforms these methods, even when specifically tuned for long-sequence adaptation on ImageNet-C. The results of Oracle, an (unrealistic) upper bound technique for learning rate tuning, suggest that optimal learning rates could potentially enable the original methods to outperform the IT version in some cases (e.g. CoTTA). However, this is not always the case. When tuned using Oracle, 19

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

I-AdaContrast achieves superior accuracy compared to its original version, while Oracle-tuned I-RoTTA performs similarly to its original counterpart. These results underscore both the difficulty of hyperparameter selection and the robust performance of our IT method across varying conditions. B.5

WALL - TIME RESULTS

We evaluate computational efficiency by measuring the wall-clock time required to process 10,000 CIFAR10C images on a single RTX 4080 GPU, with results shown in Tab. A.3. Our technique maintains a processing speed comparable to baseline methods, adding no computational overhead. MEMO’s result highlights a significant issue with single-image adaptation methods, which exhibit substantially longer processing times. B.6

O N SINGLE IMAGE ADAPTATION METHODS .

Single image adaptation approaches, like MEMO (Zhang et al., 2022), are not susceptible to the studied problem of error accumulation. However, those methods have their own issues, e.g., they tend to be significantly more inefficient (Alfarra et al., 2024), as shown in Tab. A.3. MEMO is significantly slower than other techniques, with a wall-clock time around 20× higher than AdaContrast. Moreover, it is outperformed by IT-modified methods and the baselines on most datasets, except ImageNet-R (see Tab. 2 and Tab. A.4). B.7

Table A.3: Wall-clock time (seconds) for processing 10,000 images of CIFAR10-C on a single RTX 4080 GPU. The value in superscript indicates the performance improvements over the baseline. Method

Time [s]

Source MEMO

3.4 508.4

AdaCont. I-AdaCont. CoTTA I-CoTTA RoTTA I-RoTTA PETAL I-PETAL

25.3 25.0-0.3 40.7 40.2-0.5 27.7 27.5-0.2 26.5 26.2-0.3

R ESULTS ON COMMON - LENGTH BENCHMARKS

The overall accuracy of the tested TTA methods on common-length benchmarks is presented in Tab. A.4. The results for IT are mixed, sometimes outperforming the baseline methods and sometimes falling behind. The failures can be attributed to its slow adaptability, which stems from the intransigence issue discussed in Sec. 6.1. Moreover, it is worth noting that the original methods were specifically tuned for these benchmarks, whereas no such tuning was performed for IT. Our goal is to highlight the lack of robustness to longer adaptation sequence lengths exhibited by state-of-the-art teacher-student framework-based TTA methods and to provide a simple and robust solution in the form of the intransigent teacher. B.8

P OTENTIAL OF ADAPTIVE β VALUE

In Tab. A.5, we explore a dynamic approach to adjusting the teacher model’s momentum parameter (β). Our experiment begins with the default value of β = 0.999, allowing initial teacher model plasticity, then transitions to complete weight preservation of IT (β = 1.0) after one full cycle through the data. This hybrid approach outperforms our IT technique in several cases, demonstrating the potential of adaptive momentum strategies. However, the results are not uniformly positive with our standard IT outperforming the hybrid method in some cases (AdaContrast on ImageNet-C (L) and CoTTA on DomainNet-126 (L)). This suggests that the fixed period length is not a universal value and there is a need to adjust it correctly. B.9

U SING ONLY STUDENT MODEL

Given that the intransigent teacher strategy maintains fixed teacher model weights, one might question whether the teacher model is truly necessary. To address this, we conduct experiments using only the student network (see Tab. A.6). In these experiments, we take the student model’s outputs directly as pseudo-labels, without any teacher involvement. We included various averaged results based on the hyperparameter selection approach, acknowledging the non-trivial nature of the selection in TTA. Following a realistic approach, we employ an Oracle technique on ImageNet-C (L) as a reference benchmark (inspired by Rusak et al. (2022), we name it Transfer IN-C) and apply the selected learning rate across all datasets. Additionally, we choose the learning rate using Oracle on the first loop of the long (L) scenarios (Transfer 1×Loop). We also present results when the learning rate is chosen separately for each of the datasets with the Oracle. 20

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

Table A.4: Classification accuracy [%] for common-length benchmarks. The value in superscript indicates the improvements over the baseline. Method

C10-C

IN-C

IN-R

DN-126

Source MEMO

56.5 65.6

18.0 25.0

36.2 40.9

54.7 53.2

TestBN TENT EATA SAR RDUMB AdaContrast I-AdaContrast CoTTA I-CoTTA RoTTA I-RoTTA PETAL I-PETAL

75.0 75.7 77.4 75.8 77.2 81.3 82.0+0.7 75.1 69.8-5.3 79.0 73.2-5.8 68.6 74.3+5.7

36.6 38.9 43.1 41.9 41.3 39.5 39.8+0.3 41.1 35.6-5.5 38.6 39.3+0.7 37.6 37.6+0.0

46.5 52.4 54.4 52.8 52.0 56.5 59.6+3.1 52.0 49.5-2.5 55.9 56.6+0.7 51.9 54.0+2.1

39.7 42.6 45.8 42.8 46.2 40.9 41.0+0.1 42.8 35.9-6.9 39.2 39.3+0.1 40.0 39.6-0.4

54.5 58.0 59.7 57.2 58.9 62.0 61.7-0.3 58.9 54.4-4.5 56.8 56.3-0.5 56.4 54.4-2.0

BATCH SIZE 10

27.0 31.2 36.0 31.3 34.8 33.3 33.8+0.5 26.4 28.3+1.9 29.2 29.4+0.2 20.6 29.8+9.2

BATCH SIZE 64

TestBN TENT EATA SAR RDUMB AdaContrast I-AdaContrast CoTTA I-CoTTA RoTTA I-RoTTA PETAL I-PETAL

79.2 77.8 79.8 79.3 81.4 82.6 82.4-0.2 82.2 68.6-13.6 80.9 76.7-4.2 76.6 78.5+1.9

31.4 37.3 42.0 37.8 40.0 34.8 35.1+0.3 36.8 31.7-5.1 32.4 30.6-1.8 32.9 34.2+1.3

Table A.5: Classification accuracy [%] for long scenarios with the weights of the teacher fixed only after the 1st loop on the test sequence. The value in superscript indicates the improvements over the IT technique’s performance. The batch size is equal to 64. Method AdaCont. CoTTA RoTTA

C10-C (L) -0.1

85.2 72.0+3.7 80.4+0.7

IN-C (L)

IN-R (L)

-2.0

+0.1

38.4 45.0+9.6 36.1+3.2

38.2 42.8+3.3 41.0+1.3

DN-126 (L) +0.9

65.3 49.1-6.9 57.9+1.3

Avg. 56.8-0.3 52.2+2.4 53.9+1.7

While it appears that using only the student model can achieve reasonable performance with carefully tuned hyperparameters, it significantly underperforms compared to the intransigent teacher approach. The intransigent teacher delivers consistent results, demonstrating robustness across various hyperparameter selection methods, whereas the student-only model exhibits higher performance variance using different selection approaches. B.10

E FFECTS OF INTRANSIGENCE AMOUNT EXTENDED EXPERIMENT

To signify the point of Sec. 6.1, Fig. A.4 shows results where the test sequence is extended to 100 loops of common CIFAR10-C. It verifies that CoTTA with ET and β = 0.9999 degrades below the performance of IT, given enough samples in the test sequence. This observation highlights a significant issue of TTA methods, as they can face test sequences of arbitrary lengths after deployment. 21

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

Table A.6: Classification accuracy [%] for long (L) scenarios while utilizing only the student model for adaptation (S. only) and tuning the learning rate parameter (LR). The batch size is equal to 64. Avg. Oracle is the average accuracy when the LR value is chosen separately for each of the datasets with Oracle. Avg. Transfer IN-C is the average accuracy with a single LR value chosen on the ImageNet-C dataset using the Oracle method. Average accuracy with the learning rate chosen on standard unrepeated benchmarks using the Oracle is presented in Avg. Transfer 1×Loop column. The value in superscript indicates the decrease below the IT technique’s performance. Avg. Transfer IN-C Transfer 1×Loop

Method

LR

C10-C (L)

IN-C (L)

IN-R (L)

DN-126 (L)

AdaCont. S. only

1e-5 1e-6 1e-7

82.4 81.4 79.1

23.0 36.6 31.9

37.4 40.7 39.6

59.1 59.2 57.0

54.7-3.8

54.5-2.6

50.5-6.6

CoTTA S. only

1e-4 1e-5 1e-6

35.3 70.8 79.2

6.3 34.8 31.9

14.1 41.3 40.0

1.1 1.6 7.0

40.6-12.5

37.1-8.9

26.7-23.1

RoTTA S. only

1e-4 1e-5 1e-6

21.9 51.6 86.6

1.4 14.5 26.0

5.8 35.6 37.3

6.9 48.9 53.8

45.9-6.5

45.9-6.5

41.4-10.8

Accuracy [%]

80

Oracle

0.9999 1.0 (Intransigent) Source

75 70 65 60

1 10 20 30 40 50 60 70 80 90 100

Loop No.

Figure A.4: Mean accuracy [%] of CoTTA with varying β for each loop of common CIFAR10-C testing sequence repeated 100 times. The brown dashed line indicates the accuracy of the source model as a reference. Table A.7: Classification accuracy [%] for long scenarios with restoration probability parameter p of CoTTA method tuned. The batch size is equal to 64. Avg. Def. is the average accuracy with default p value. Avg. Transfer IN-C is the average accuracy with a single p value chosen on the ImageNet-C dataset using the Oracle method. Average accuracy when the p value is chosen separately for each of the datasets with Oracle is presented in Avg. Oracle column. p-value 0.1 0.01 0.001 (Def.) 0.0001 0.00001 0.0

B.11

C10-C (L)

IN-C (L)

IN-R (L)

DN-126 (L)

73.1 53.7 56.0 54.7 54.3 52.7

29.0 24.8 52.8 53.7 53.6 53.5

41.8 35.6 50.5 45.0 49.0 48.9

26.9 13.7 45.6 52.9 55.0 54.5

Def.

Avg. Transfer IN-C

Oracle

51.2

51.6

58.1

D ISCUSSION ON MODEL RESET MECHANISM

CoTTA’s proposed resetting mechanism aims to preserve source knowledge by stochastically restoring portions of the student model’s weights to their original source state during each update iteration. In principle, an effective source knowledge preservation technique should eliminate the need for our IT technique. However, CoTTA’s reset mechanism introduces a restoration probability parameter. To ensure our findings were not biased by suboptimal parameter selection, we conducted parameter tuning experiments, documented in Tab. A.7. These results reveal that the optimal restoration probability varies across datasets, with model performance dependent on this parameter. When following a realistic scenario of tuning on a single dataset, the performance improvements were marginal (Avg. Transfer IN-C). Only by using an Oracle approach on all benchmarks, we observe performance gains, highlighting the practical limitations of this approach. 22

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

Figure A.5: Batch-wise accuracy plots of RDumb and I-AdaContrast methods on ImageNet-C (L) benchmark. The accuracy values were smoothed to make the plot clearer. RDumb resets the model every 1000 iterations, which causes significant drops in accuracy after the reset.

B.12

D ISCUSSION ON RD UMB

Similar to us, RDumb focuses on long scenarios and provides a straightforward analysis. However, there are several key differences between our work and RDumb. First, RDumb experiments exclusively with common corruptions, while we demonstrate the phenomenon across various types of distribution shifts, including ImageNet-R and DomainNet. Additionally, we analyze this issue using several newer methods that incorporate the mean-teacher mechanism, which theoretically should address the problem (whereas only CoTTA among RDumb’s baselines used this mechanism). Furthermore, we highlight a potential, simple solution to mitigate the issue, which is not explored in RDumb. RDumb method mechanism of periodically resetting the model to its initial state leads to significant accuracy drops immediately following each reset, as illustrated in Fig. A.5. Such instability is particularly concerning since reliable test-time adaptation should maintain consistent performance throughout the adaptation process. Furthermore, the same constant reset interval is likely not optimal for every case, which adds a hyperparameter to select. In contrast, our IT approach achieves comparable performance without requiring parameter tuning. B.13

AUGMENTATION COUNT ABLATION

The CoTTA (Wang et al., 2022) and PETAL (Brahma & Rai, 2023) methods update the student network using a consistency loss between the student and teacher models. When the prediction confidence of the source model falls below a predefined threshold, the teacher generates pseudo-labels by averaging predictions across 32 distinct random augmentations of each image. This ensembling strategy requires 31 additional forward passes per batch, introducing a severe computational bottleneck that significantly reduces throughput compared to alternative state-of-the-art approaches. To evaluate the necessity of this computational overhead, Fig. A.6 compares pseudo-label generation using either 1 or 32 random augmentations. Across evaluations on ImageNet-C (L) and ImageNet-R (L), accuracy profiles and degradation trajectories remain remarkably consistent regardless of the augmentation count. On ImageNet-R (L), CoTTA with a single augmentation even exhibits a marginally slower degradation rate toward the end of the sequence, while on DomainNet-126 (L), it yields a 2% to 4% accuracy improvement over the 32-augmentation baseline. The largest divergence occurs on CIFAR10-C (L), where 32 augmentations yield higher absolute accuracy for both methods. However, the underlying degradation trend persists, and both configurations ultimately collapse. Critically, reducing the augmentation count 23

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

does not accelerate this collapse, demonstrating that the overall evaluation outcomes remain robust. While a definitive theoretical explanation for this behavior is outside the scope of this work, we hypothesize that averaging predictions across random transformations does not always yield a significant improvement in pseudo-label accuracy.

C10-C (L)

Accuracy [%]

80

IN-C (L)

60

IN-R (L)

50

50

40

45

30

40

50

40

40

20 1

6

11 Loop No.

16

20 CoTTA

1

6 PETAL

11 Loop No.

16

20

DN-126 (L)

60

1

#augs=1

6

11 Loop No. #augs=32

16

20

1

6

11 Loop No.

16

20

Source

Figure A.6: Mean accuracy [%] for each loop of the common testing sequence on CIFAR10-C (L), ImageNet-C (L), ImageNet-R (L), and DomainNet-126 for CoTTA and PETAL. Methods are tested with pseudo-labels generated by averaging across predictions on either 1 or 32 randomly augmented images. The brown dashed line indicates the source model accuracy as a reference.

B.14

D ISTILLATION VS . BATCH NORMALIZATION RECALIBRATION

To isolate whether the IT’s performance gains stem from the distillation process or from simply recalibrating batch normalization (BN) statistics, we evaluate variants of our IT-enhanced methods without backpropagation (w/o BP). We compare these variants against the original baselines and the full IT baselines (Tab. A.8). In the w/o BP configuration, the adaptation process degenerates to adjusting only the student BN statistics, removing all influence from the teacher. The specific technique for adjusting BN statistics varies across methods: CoTTA and PETAL utilize per-batch BN calculation, matching the TestBN baseline. Hence, their w/o BP results are identical to TestBN. AdaContrast continuously updates the EMA of the BN statistics, while RoTTA leverages its proposed robust batch normalization (RBN) module. The results show that updating BN statistics alone provides a substantial accuracy boost. However, this update is not the sole driver of optimization. Across all 32 combinations of benchmarks, batch sizes, and baseline methods, the full IT framework outperforms or matches the w/o BP variant in 26 cases. This demonstrates that distillation-based adaptation yields consistent, additive improvements across the majority of evaluations. In the few cases where full IT underperforms relative to w/o BP, the original baseline method itself underperforms compared to both ”I-” variants (e.g., PETAL on CIFAR10-C (L) with a batch size of 64, and AdaContrast on ImageNet-R (L) with a batch size of 10). This pattern indicates that when the underlying adaptation method fails over long horizons, stripping away all plasticity except for the BN statistics update yields the most effective results. Consequently, the average accuracy of the full IT method exceeds that of the w/o BP variants in all setups except for CoTTA. We attribute this exception to the poor baseline performance of CoTTA on CIFAR10-C (L), which underscores that the final performance of IT remains fundamentally constrained by its underlying baseline. In summary, updating batch normalization statistics alone (w/o BP) delivers significant performance gains. However, this approach is strictly limited to architectures containing batch normalization layers. By contrast, distilling knowledge from an intransigent teacher provides further, cross-architecture improvements in most scenarios, validating the broader competitiveness of our approach. B.15

IT TEACHER WITH FIXED SOURCE BN STATISTICS

Using batch normalization (BN) statistics computed at test time is a common strategy in TTA. In our case, we adopt the same strategy for computing BN statistics as the baseline methods: CoTTA utilizes the TestBN technique, whereas AdaContrast and RoTTA update BN statistics using exponential moving average-based approaches. To provide additional insight, we also experimented with fixed teacher statistics at test time, which are calculated during training on the source data (see Table A.9). The results suggest that adjusting the BN statistics in the teacher model improves overall performance, aligning with the findings of the TTA community (Yuan et al., 2023; Schneider et al., 2020a; Xiao & Snoek, 2024). This indicates that adapting these statistics can significantly mitigate performance drops on out-of-distribution data. 24

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

Table A.8: Classification accuracy [%] for long (L) scenarios. Superscript indicates improvements over the baseline. Bold indicates best performing method. Gray color indicates model collapse – performance worse than the nonadapting model (Source). Results averaged from 3 random seeds.

Method

C10-C (L)

IN-C (L)

IN-R (L)

DN-126 (L)

Avg.

Source

56.5

18.0

36.2

54.7

41.4

BATCH SIZE 10

TestBN

75.1

26.9

36.2

49.6

47.0

AdaCont. I-AdaCont. w/o BP. I-AdaCont.

72.1 77.7+5.6 84.1+12.0

2.3 29.6+6.3 39.5+37.2

8.0 39.0+31.0 35.3+27.3

47.0 54.4+7.4 63.2+16.2

32.4 50.2+17.8 55.5+23.1

CoTTA I-CoTTA. w/o BP. I-CoTTA

23.8 75.1+51.3 69.7+45.9

3.8 26.9+23.1 27.6+23.8

33.7 36.2+2.5 37.4+3.7

6.0 49.6+43.6 50.3+44.3

16.8 47.0+30.2 46.3+29.5

RoTTA I-RoTTA. w/o BP. I-RoTTA

82.5 70.7-11.8 79.0-3.5

24.4 24.2-0.2 33.6+9.2

43.0 36.8-6.2 39.9-3.1

45.6 53.47.8 57.8+12.2

48.9 46.3-2.6 52.6+3.7

PETAL I-PETAL. w/o BP. I-PETAL

68.0 75.1+7.1 74.2+6.2

3.9 26.9+23.0 28.7+24.8

37.9 36.2-1.7 36.5-1.4

47.3 49.6+2.3 50.7+3.4

39.3 47.0+7.7 47.5+8.2

BATCH SIZE 64

TestBN

79.1

31.4

39.6

54.4

51.1

AdaCont. I-AdaCont. w/o BP. I-AdaCont.

81.8 77.9-3.9 85.4+3.6

18.8 30.4+11.6 40.4+21.6

26.5 39.4+12.9 38.1+11.6

61.7 55.0-6.7 64.4+2.7

47.2 50.7+3.5 57.1+9.9

CoTTA I-CoTTA. w/o BP. I-CoTTA

56.0 79.1+23.1 68.3+12.3

52.8 31.4-21.4 35.4-17.4

50.5 39.6-10.9 39.6-10.9

45.6 54.4+8.8 56.0+10.4

51.2 51.1-0.1 49.8-1.4

RoTTA I-RoTTA. w/o BP. I-RoTTA

82.3 70.7-11.6 79.7-2.6

13.2 24.2+11.0 32.9+19.7

43.4 36.8-6.6 39.7-3.7

50.3 53.4+3.1 56.6+6.3

47.3 46.3-1.0 52.2+4.9

PETAL I-PETAL. w/o BP. I-PETAL

56.3 79.1+22.8 78.8+22.5

36.8 31.4-5.4 32.3-4.5

40.7 39.6-1.1 39.8-0.9

58.6 54.4-4.2 55.4-3.2

48.1 51.1+3.0 51.6+3.5

Table A.9: Classification accuracy [%] for long scenarios while utilizing fixed teacher statistics calculated on source data in batch normalization layers. The batch size is equal to 64. Superscript indicates improvements over the full IT approach. Method I-AdaCont. Src. BN I-CoTTA Src. BN I-RoTTA Src. BN

C10-C (L)

IN-C (L)

IN-R (L)

DN-126 (L)

Avg.

83.2 49.4 61.1

24.8 17.6 16.2

35.3 35.5 37.4

63.3 52.3 51.8

51.7-5.4 38.7-11.1 41.6-10.6

25

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

C

R EMAINING R ESULTS

This section provides additional results not included in the main text. These include results on various architectures when the learning rate is tuned exclusively for the original methods on standard-length benchmarks and directly reused for IT-augmented versions (Tab. A.10), baseline evaluations for different β parameters (Tab. A.11, Figs. A.7 and A.8) and accuracy plots incorporating the IT technique (Figs. A.9 and A.10) for various benchmarks and architectures. Additionally, we include per-batch negative flip rate plots (Fig. A.11). Table A.10: Classification accuracy [%] for long scenarios on CIFAR10-C and ImageNet-C with different neural network architectures. The value in superscript indicates the improvements over the baseline. The learning rate parameter is adjusted using the Oracle method on the original benchmarks exclusively for original methods and reused for IT-augmented ones.

Source AdaCont. I-AdaCont. CoTTA I-CoTTA RoTTA I-RoTTA PETAL I-PETAL

C10-C (L) RN26GN

RNXt-50

IN-C (L) ViT-B16 SViT-T

CNXt tiny

67.3 75.5 79.6+4.1 16.5 61.6+45.1 62.8 70.5+7.7 15.0 52.8+37.8

21.1 21.3 42.9+21.6 57.1 38.4-18.7 16.2 35.2+19.0 56.1 36.2-19.9

39.8 33.9 43.5+9.6 35.3 39.9+4.6 37.1 40.7+3.6 38.4 39.41.0

29.1 19.2 32.513.3 22.1 29.5+7.4 18.7 29.7+11.0 24.8 27.93.1

28.3 17.0 30.8+13.8 25.6 28.9+3.3 7.6 26.5+18.9 22.1 26.64.5

Table A.11: Mean accuracy [%] for different exponential moving average β parameter. AdaContrast and CoTTA default originally to 0.999. (L) stands for the adaptation sequence being repeated 20 times. Gray color indicates model collapse – performance worse than no adaptation (Source). Dataset

0.9

0.99

0.999

0.9995

0.9999

IT

AdaCont. 79.0 C10-C (L) CoTTA 10.5

79.3 14.9

81.9 55.9

83.2 68.6

85.8 78.3

85.4 68.4

AdaCont. 1.2 CoTTA 0.3

5.4 23.6

18.8 52.8

25.6 55.1

38.6 50.9

40.4 35.4

IN-C (L)

Method

26

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

60

85

0.9 0.99 0.999 (Def.) 0.9995 0.9999 1.0 (Intransigent) Source

40 30 20 10

80

Accuracy [%]

Accuracy [%]

50

75 70 65 60

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

0.9 0.99 0.999 (Def.) 0.9995 0.9999 1.0 (Intransigent) Source

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

Loop No.

35 30 0.9 25 0.99 20 0.999 (Def.) 15 0.9995 0.9999 10 1.0 (Intransigent) 5 Source 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Accuracy [%]

Accuracy [%]

Figure A.7: Mean accuracy [%] for each loop of common testing sequence on ImageNet-C (L) using CoTTA (left) and on CIFAR10-C (L) using AdaContrast (right). The brown dashed line indicates the source model accuracy.

Loop No.

80 70 0.9 0.99 60 0.999 (Def.) 50 0.9995 40 0.9999 1.0 (Intransigent) 30 Source 20 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

Figure A.8: Mean accuracy [%] for each loop of common testing sequence on ImageNet-C (L) (left) and CIFAR10-C (right) using PETAL. The brown dashed line indicates the source model accuracy.

27

55 50 ET - Teacher 45 ET - Student 40 IT - Teacher 35 IT - Student 30 Source 25 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

40

Accuracy [%]

Accuracy [%]

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

30

ET - Teacher ET - Student IT - Teacher IT - Student Source

20 10

00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

Loop No.

(b) CoTTA, ViT-B16

50 45 40 35 ET - Teacher ET - Student 30 IT - Teacher 25 IT - Student Source 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

45

Accuracy [%]

Accuracy [%]

(a) CoTTA, ResNet50

40 35

ET - Teacher ET - Student IT - Teacher IT - Student Source

30 25 20

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

Loop No.

(d) RoTTA, ViT-B16

40

45

35

40

Accuracy [%]

Accuracy [%]

(c) AdaContrast, ViT-B16

30

ET - Teacher ET - Student 25 IT - Teacher IT - Student 20 Source 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

35 30 25

ET - Teacher ET - Student IT - Teacher IT - Student Source

20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

Loop No.

(e) PETAL, ResNet50

(f) PETAL, ViT-B16

Figure A.9: Per-batch accuracy [%] on ImageNet-C (L). Comparison across different methods and architectures with EMA teacher (ET, orange) and intransigent teacher (IT, blue), for both teacher (solid) and student (dashed).

28

90 85 80 75 ET - Teacher ET - Student 70 IT - Teacher 65 IT - Student 60 Source 550 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

80

Accuracy [%]

Accuracy [%]

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

70 60 50 40

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

Loop No.

(b) CoTTA

85

80

80

70

Accuracy [%]

Accuracy [%]

(a) AdaContrast

75

ET - Teacher ET - Student IT - Teacher IT - Student Source

ET - Teacher ET - Student IT - Teacher IT - Student Source

60

550 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

ET - Teacher ET - Student IT - Teacher 40 IT - Student Source 30 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

(c) RoTTA

(d) PETAL

70 65 60

50

Loop No.

Loop No.

Figure A.10: Per-batch accuracy [%] on CIFAR10-C (L) using WideResNet-28. Comparison with EMA teacher (ET, orange) and intransigent teacher (IT, blue), for both teacher (solid) and student (dashed).

29

Published at 5th Conference on Lifelong Learning Agents (CoLLAs), 2026

0.2 0.0

0.0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

AdaContrast I-AdaContrast

0.2

AdaContrast I-AdaContrast

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No. DN-126 (L)

NFR

NFR

Loop No. IN-R (L)

0.0

IN-C (L)

0.5

NFR

NFR

C10-C (L) AdaContrast I-AdaContrast

0.1 0.0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

AdaContrast I-AdaContrast 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

C10-C (L)

0.5 0.0

IN-C (L)

CoTTA I-CoTTA

NFR

NFR

(a) AdaContrast 0.1 0.0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

0.05 0.00

CoTTA I-CoTTA

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No. DN-126 (L)

NFR

NFR

Loop No. IN-R (L)

CoTTA I-CoTTA

0.0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

CoTTA I-CoTTA

0.2

Loop No.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

(b) CoTTA

C10-C (L)

0.1

0.0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

RoTTA I-RoTTA

0.00

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No. DN-126 (L)

NFR

NFR

Loop No. IN-R (L)

0.05

RoTTA I-RoTTA

NFR

RoTTA I-RoTTA

NFR 0.0

IN-C (L)

0.5

0.0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

RoTTA I-RoTTA

0.2

Loop No.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

0.5 0.0

C10-C (L)

IN-C (L)

PETAL I-PETAL

NFR

NFR

(c) RoTTA

0.00

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

0.000

PETAL I-PETAL

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No. DN-126 (L)

NFR

NFR

Loop No. IN-R (L)

0.025

PETAL I-PETAL

0.05

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

0.1 0.0

Loop No.

PETAL I-PETAL 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Loop No.

(d) PETAL Figure A.11: Per-batch Negative Flips Rate (NFR) of the student model on each benchmark using AdaContrast (a), CoTTA (b), RoTTA (c) and PETAL (d) methods and IT-enhanced version.

30

Related documents

• Teacher data use and student learning: Insights from a NSW Catholic School system#331448
• Test-Time Generalized Category Discovery#1119010
• Collaborative learning practices: teacher and student perceived obstacles to effective student collaboration#633894
• Challenges for building up confirmatory science in lab biology: lessons learned from the Brazilian Reproducibility Initiative#203495
• MATE: Masked Autoencoders are Online 3D Test-Time Learners#464522