Reliable Neural Collapse Approximation for Open-World Test-Time Adaptation

Reliable Neural Collapse Approximation for Open-World Test-Time Adaptation Jia-Qi Lin, Yuangang Pan, Chang-Dong Wang, Haizhang Zhang, Ivor W. Tsang, and Joey Tianyi Zhou

arXiv:2608.19890v1 [cs.LG] 20 Aug 2026

Abstract Test-Time Adaptation (TTA) methods aim to bridge the domain gap between the source and target domains. However, traditional TTA methods become ineffective when the label distribution shift occurs, a challenge commonly referred to as an open-world scenario. In this paper, we introduce a new method named Reliable Neural Collapse approximation (ReNC) for Open-World Test-Time Adaptation (OWTTA). Specifically, we leverage neural collapse as a structural prior for reliable target-domain adaptation. Guided by this prior, we justify that the pre-trained classifier weights can serve as the prototypes of the source domain. By measuring the similarity between samples and prototypes, we filter out the Out-Of-Distribution (OOD) samples for reliable updates. Furthermore, we propose a neural collapse approximation mechanism to refine these prototypes, ensuring they can gradually adapt to the target domain while maintaining the neural collapse structure. Extensive experiments on several open-world benchmarks demonstrate the superiority of the proposed method. Our empirical analysis suggests that ReNC better preserves NC-related properties in the target domain, providing useful evidence for explaining reliable OWTTA and offering new insights for model design. Code is available at https://github.com/JiaqiLin-AI/ReNC. 1

1

Introduction

Test-Time Adaptation (TTA) has attracted significant attention [1, 2, 3] for its ability to improve the generalization of deep learning models without requiring access to the entire batch of data. It aims to bridge the distribution gap between training and testing data under conditions of data distribution shift, where the training and testing data originate from different domains. Existing TTA methods [4, 5, 6, 7] address domain adaptation by either aligning target features with the source distribution or minimizing the entropy of model outputs. Both approaches have shown impressive effectiveness in enhancing model performance under data distribution shifts. A basic assumption of TTA is that the training and testing samples share the same label space. However, this assumption may be violated in more realistic open-world scenarios [8]. Under the open-world scenarios, the test data may not only contain seen classes, known as In-Distribution (ID) data, but also unseen classes, referred to as Out-Of-Distribution (OOD) data. This phenomenon, which we refer to as a label distribution shift, can render existing TTA methods ineffective. For example, in medical imaging, data distribution shift and label distribution shift often occur due to differences in acquisition devices and new disease that the 1

Jia-Qi Lin, Yuangang Pan, Ivor W. Tsang, and Joey Tianyi Zhou are with the Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore 138632. E-mails: Lin Jiaqi@a-star.edu.sg, Pan Yuangang@astar.edu.sg, ivor tsang@a-star.edu.sg, joey zhou@a-star.edu.sg. Jia-Qi Lin completed this work during his Ph.D. study at Sun Yat-sen University. Chang-Dong Wang is with School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China, Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China, and Guangdong Province Key Laboratory of Computational Science, Guangzhou, China. E-mail: changdongwang@hotmail.com. Haizhang Zhang is with School of Mathematics (Zhuhai), Sun Yat-sen University, Zhuhai, China. E-mail: zhhaizh2@mail.sysu.edu.cn. Corresponding author: Chang-Dong Wang.

1

(a) Existing Methods

(b) ReNC

Figure 1: Motivation of ReNC. (a) Existing methods use fixed source-domain prototypes, which may mismatch the target domain. (b) ReNC improves feature-prototype consistency and adapts prototypes toward target-domain neural collapse. model has not encountered during training [9]. Adapting models to simultaneous data distribution shift and label distribution shift is the known as Open-World Test-Time Adaptation (OWTTA), which remains a challenge for the community. There are two primary challenges in OWTTA: 1) The label distribution shift caused by OOD samples renders the model update process unreliable. 2) The domain gap introduced by the data distribution shift further increases the difficulty of effective adaptation. Most recently, a limited number of studies [8, 10, 11] have explored OWTTA as an effective strategy to enhance the generalization abilities of deep neural networks. They typically employ a two-step process [10, 11] to address the aforementioned challenges. First, the OOD samples are filtered out from the target data to mitigate the negative effects of label distribution shift. This is typically achieved by measuring the similarity between features and their nearest prototypes to distinguish between ID and OOD samples. Next, TTA is applied to the remaining ID samples to address the distribution gap caused by data distribution shift. However, these methods optimize the model using fixed prototypes, which are typically derived either from the source domain or the pre-trained classifier. In practice, source prototypes are often unavailable due to computational constraints [12, 13], privacy concerns [14, 15], or copyright restrictions [16]. Moreover, even when accessible, prototypes from the source domain often fail to align well with the target domain due to the data distribution shift. As illustrated in Figure 1a, existing method [8] push features toward fixed prototypes of the source domain leading to suboptimal performance due to the domain gap with the target domain. In this paper, we introduce a new OWTTA method, Reliable Neural Collapse Approximation (ReNC), which addresses the aforementioned challenges by approximating the neural collapse structure in the target domain. Neural collapse [17] describes a terminal geometry of well-trained classification neural networks, where features become intra-class compact, class means align with classifier weights, and classification behaves like nearest-class-center prediction. Motivated by this property, we leverage neural collapse as a reliable structural prior to guide OWTTA. However, as neural collapse typically occurs only with labeled source data, it cannot be directly achieved on unlabeled target data. Therefore, our ReNC method is proposed to reliably approximate the neural collapse state in the target domain. As illustrated in Figure 1b, ReNC not only pushes target features toward their corresponding prototypes, but also updates the prototypes to approx-

2

imate the target-domain neural collapse state. Specifically, considering that the pre-trained source model has already collapsed, we introduce a parameter-free mechanism to filter out OOD samples for reliable updates. This mechanism evaluates the similarity of the target domain features to prototypes, ensuring the adaptation process focuses on ID samples. Furthermore, we formulate neural collapse approximation as a progressive adaptation process that encourages reliable target features to approach their corresponding prototypes, while updating the prototypes with prior-regularized adaptation to preserve the underlying neural collapse geometry. Benefiting from reliable and efficient updates, ReNC achieves a better neural collapse state in the target domain without access to the entire target data batch. We summarize the contributions as follows: • We are the first study to introduce the concept of neural collapse to OWTTA. And we justify that classifier weights are equivalent to source domain prototypes from the neural collapse perspective. • We introduce a new OWTTA framework, called ReNC, featuring an efficient updating mechanism to approximate the neural collapse state in the target domain, which provides a reliable structural prior for TTA. • Extensive experiments on several open-world benchmarks demonstrate the effectiveness of ReNC in OWTTA, and further analyses show that ReNC better preserves NC-related representation properties during target adaptation. The rest of this paper is organized as follows: In Section 2, we define the problem definition of OWTTA and provide a brief review of the related literature. Section 3 introduces our ReNC method, progressing from the basic concepts to the final approach. In Section 4, we present a thorough evaluation based on extensive experiments to validate the effectiveness of the ReNC approach. Finally, Section 5 provides a summary of the key findings and concludes the paper.

2

Problem Definition and Literature Review

In this section, we first introduce the setting of closed-world test-time adaptation. Then, we extend the problem to the open-world scenario, where data contains label distribution shifts. Lastly, we compare this setting with OOD detection and universal domain adaptation, discussing the differences between them and OWTTA.

2.1

Test-Time Adaptation

Data Distribution Shift: We consider the classification tasks where X and Y represent the data space and the label space, respectively. There are two distinct distributions over X × Y, referred to as the source domain DS and the target domain DT . The data distribution shift occurs because the source data set SS and the target data set ST are drawn independently from two different domains, DS and DT , respectively, resulting in differences in their distributions: SS = {(xi , yi ) |xi ∈ DS } ,

ST = {xi |xi ∈ DT } .

Various domain adaptation methods [18, 19] have been proposed to eliminate the domain gap between DS and DT . Among all these methods, Test-Time Adaptation (TTA) [20, 21] has gained significant attention for its assumption that the model cannot access source data and target labels, making it more suitable for real-world applications. A number of TTA methods are inspired by the self-training and employ various techniques for unlabeled target data adaptation [7, 22]. Test-time entropy minimization (TENT) [23] is proposed to update only the parameters in the batch normalization layer by minimizing the entropy of logits. Source Hypothesis 3

Transfer (SHOT) [12] freezes the classifier of the source model and updates the feature extraction module by minimizing entropy while maintaining balanced distribution in the target domain. Besides, the distribution alignment techniques are also utilized in TTA by treating the source domain distribution as an anchor for aligning the features of the target domain [24, 25, 26]. Test-time training method [27] modified pre-training procedure to acquire the statistic information of source domain without accessing source data when adaptation. Nevertheless, such a scheme is still impractical since it alternates the pre-training procedure. To tackle this issue, [28] improves TTA performance by saving the distribution of the source domain and aligning the high-confidence target features towards the distribution for updating without altering pre-training. Additionally, self-supervised learning techniques are also widely used in TTA since they do not need to consider the semantic information of the target data [29]. For example, contrastive learning is introduced [30] to refine the online pseudo labeling process, leveraging the self-supervised techniques to enhance the target feature adaptation. The self-supervised training branch is introduced [31] to fine-tune the parameters of the feature encoder during adaptation. While most test-time adaptation (TTA) methods have been developed under the assumption of static target distributions and focus primarily on visual data, recent works have begun exploring more complex and realistic settings. Some studies consider scenarios where the target domain distribution changes continuously over time [32, 33, 34, 35], and [36] further extend this to evolving environments where the proportions of existing classes vary dynamically, potentially leading to class imbalance. In parallel, [37] investigates TTA in the context of tabular data, and proposes a method that estimates the test-time label distribution using confident predictions and calibrates model outputs accordingly. Furthermore, Vision-Language Models (VLMs) [38] have also been explored for test-time adaptation through prompt tuning, where prompts are optimized without labels to improve both accuracy and calibration during inference [39].

2.2

Open-World Test-Time Adaptation

Label distribution shift: Existing TTA methods have achieved significant improvement, assuming that the source and target domain have the same label space. In the open-world scenario, however, this assumption may be violated [40], with the distribution shift existing in both the data space X and the label space Y, simultaneously. This shift is characterized by the divergence between the source label space and the target label space. The label set of the target domain YT contains not only the labels from the source domain YS , but also additional labels from new classes, denoted as YO : YT = {YS ∪ YO } ,

YS ∩ YO = ∅.

Based on the label of samples, the target data set ST can be further partitioned into the In-Distribution (ID) sample set STI and the Out-Of-Distribution (OOD) sample set DTO : STI = {xi |xi ∈ DTI , Ψ(xi ) ∈ YS } , STO = {xi |xi ∈ DTO , Ψ(xi ) ∈ YO } , where DTI represents the domain of ID samples, DTO represents the domain of OOD samples, and Ψ is the ground truth mapping. OWTTA requires a robust method to ensure that the adaptation process focuses on the ID samples while minimizing the adverse impact of the OOD samples. Few TTA methods have taken the open-world scenario into consideration. Open-World Test-Time Training (OWT32 ) [8] distinguishes OOD samples based on the distance between target data and prototypes from the source domain. It then expands the prototype pool and aligns the ID samples with statistical information from the source domain for reliable adaptation. OSTTA [10] measures sample confidence based on the 2

To avoid confusion, we renamed the OWTTT method to OWT3.

4

probability values of logits. It leverages the wisdom of crowds to select high-confidence samples and applies entropy minimization exclusively to these selected samples. The Unified Entropy Optimization (UniEnt) [11] method introduces two Gaussian Mixture Models (GMMs) to model ID and OOD samples based on their distance to the nearest prototypes. UniEnt then minimizes the entropy of ID samples while maximizing the entropy of OOD samples to enhance the TTA performance.

2.3

Related Branches

Out-of-Distribution Detection. OOD detection aims to identify samples outside the training distribution, which is important for reliable model deployment. Existing methods usually design post-hoc scoring functions based on logits [41, 42, 43], feature representations [44, 45, 46], or gradient information [47, 48]. Some studies further improve OOD detection by re-training classifiers with auxiliary OOD datasets [49, 50] or generated synthetic OOD samples [51, 52, 53]. OOD detection methods require access to entire batches of data at once and do not account for data distribution shifts. In contrast, OWTTA must adapt and make inferences instantly with new incoming batches, addressing both data and label distribution shifts simultaneously. Universal Domain Adaptation. Similar to TTA, existing domain adaptation methods focus on data distribution shifts while neglecting label distribution shifts in the target domain. To provide more flexibility in real-world scenarios, universal (or open-set) domain adaptation methods have been proposed [3, 54, 55, 56]. These methods require access to both source and target data, making them impractical in real-world scenarios where source data is inaccessible due to privacy or security concerns. To address this issue, source-free universal domain adaptation methods [57, 16, 58, 59] have been developed, which only require target data for adaptation. Unlike universal domain adaptation, which requires access to entire batches of data, OWTTA handles a more challenging scenario. In OWTTA, while all test data can be processed at once, inference and adaptation must be performed instantly with new incoming batches. Test-Time Augmentation. Test-time augmentation improves prediction robustness by generating multiple augmented views of each test sample and aggregating their predictions [60, 61, 62]. These methods mainly exploit input-level diversity to obtain more stable predictions. Test-time augmentation improves generalization under domain shifts by leveraging input-level diversity, whereas OWTTA focusing on adapting the model to a specific target domain in the presence of OOD interference.

3

Methodology

In this section, we propose Reliable Neural Collapse approximation (ReNC) for open-world test-time adaptation. The key motivation of ReNC is to exploit neural collapse as a structural prior for reliable targetdomain adaptation. Specifically, we theoretically justify that, when a well-trained source model exhibits an NC-like geometry, its classifier weights can be interpreted as class prototypes. Based on this prior, ReNC first identifies reliable target samples according to their distances to the nearest prototypes, thereby filtering out OOD samples for reliable updates. Then, it progressively refines the source-domain prototypes with reliable target features and updates the model parameters under a geometry-preserving constraint in an alternating manner, encouraging the target representation to evolve toward an NC-like structure. In this way, ReNC effectively adapts the model to the target domain while maintaining a compact, balanced, and well-structured representation space. We start by developing a simple, reliable update rule and then progressively refine our approach to handle more practical scenarios in an open-world setting.

3.1

Reliable Update by Filtering Out OOD Samples

Given a model pre-trained on the source domain F (θ, ·), its logits pti can be represented by:



pti = F θ, xti = σ wf θ, xti + b = σ wzti + b , 5

(a) CIFAR10-C & Noise

(b) CIFAR10-C & SVHN

Figure 2: Entropy of CIFAR10-C with OOD samples ((a) Gaussian noise and (b) SVHN) pre-trained on CIFAR10. Minimizing entropy facilitates adaptation to the target domain. However, indiscriminately minimizing all entropy in an open-world scenario would degrade the model performance.

(a) CIFAR10-C&Noise

(b) CIFAR10-C&SVHN

Figure 3: t-SNE visualization of extracted features on CIFAR10-C with OOD samples ((a) Gaussian noise and (b) SVHN) pre-trained on CIFAR10. Prototypes are generated using the feature class-means of source samples. where xti is the input at the t-th batch, f (θ, ·) is the encoder, zti is the feature, σ(·) is the activation function, and w and b denote the classifier weights and bias, respectively. Existing closed-world TTA methods assume that, despite the presence of data distribution shifts, the entropy values [63] for target domain samples corresponding to each class remain relatively low during inferring. Based on this assumption, a straightforward way to adapt F (θ, ·) to the target domain DT is to minimize the entropy of each logit: N
1 X Le (θ) = H pti , (1) N i=1

where H(·) is the entropy function. However, in the open-world scenario, label distribution shift exists in test samples. As shown in Figure 2, the entropy of ID and OOD samples are mixed together, complicating the process of minimizing entropy. Since the goal of minimizing entropy is to encourage each logit to form a sharp (one-hot) distribution, simply reducing the entropy of all test samples can disrupt the adaptation process. To enhance the reliability of the update process, we investigate a selective updating strategy to split the ID and OOD samples. As shown in Figure 3, ID and OOD samples exhibit distinct feature distributions, where ID samples tend to cluster around their corresponding class prototypes, computed by averaging the samples in each class. Inspired by few-shot learning [64], we aim to transform the problem of distinguishing between ID and OOD samples into a two-cluster clustering problem. To achieve this, we construct a 6

(a) CIFAR10-C & Noise

(b) CIFAR10-C & SVHN

Figure 4: Similarity scores of CIFAR10-C with OOD samples ((a) Gaussian noise and (b) SVHN) pre-trained on CIFAR10. The differences in the similarity scores between ID and OOD samples provide a basis for their discrimination. similarity-based measure using multiple prototypes to differentiate between ID and OOD samples. Specifically, given K prototypes from source domain as C = {cj }K j=1 , where cj is calculated by the mean value of the features from j-th class, the similarity score between the test sample and its nearest prototype is: ⊤ zti cj t si = 1 − max t . (2) cj ∈C ∥zi ∥∥cj ∥ Some existing method [11] utilize a symmetric Gaussian Mixture Models (GMMs) with two components to fit ID and OOD samples based on their similarity scores. However, it is inappropriate because each component does not adhere to a symmetric Gaussian distribution, as shown in Figure 4. Instead, we approach this problem by learning an adaptive threshold τt to partition ID and OOD samples, effectively treating it as a clustering task: min τt

i2 1 Xh t 1 X t t s − 1(s ≤ τ )s t j i j N+ N+ i j i2 1 X 1 Xh t si − − 1(stj > τt )stj , + − N N i

(3)

j

P P where N + = i 1 sti ≤ τt and N − = i 1 sti > τt represent the number of ID and OOD samples respectively. Optimizing Eq. (3) enables finding an optimal threshold τt∗ to minimize the intra-cluster variations. Then, the objective function can be revised into: N+

L+ e (θ, C) =

1 X t H p . i N+

(4)

i=1

By applying Eq. (4), reliable updates can be achieved by minimizing the entropy of N + samples. In this process, the prototypes C serve as reliable class anchors for distinguishing ID samples from OOD ones, and thus are crucial for stable target-domain updates.

3.2

Source-Free Update with Pre-trained Classifier Weights

As discussed in Subsection 3.1, reliable updates rely on access to class prototypes. However, in Open-World Test-Time Adaptation (OWTTA), such prototype information is unavailable, since only a pre-trained model

7

(a) K = 2

(b) K = 3

(c) K = 4

Figure 5: Simplex Equiangular Tight Frame (Simplex ETF) with different class numbers. is given and unlabeled target samples arrive batch-by-batch. Therefore, we derive the prototypes directly from the pre-trained model itself. Recent neural collapse theory [65] provides a principled explanation for the final state of supervised training. Specifically, when a sufficiently large neural network is well-trained on the training data, it tends to exhibit the following properties. Theorem 1 (Convergence to Simplex ETF [65]) For a sufficiently large neural network, the classmeans of the embeddings centered at the global mean become both linearly separable and maximally distant when the model converges. These class-means lie on a sphere centered at the origin, forming a Simplex ETF: ( 1, i=j ⟨ci − cG , cj − cG ⟩ , = 1 ∥ci − cG ∥2 ∥cj − cG ∥2 − K−1 , i ̸= j, ∥ci − cG ∥2 − ∥cj − cG ∥2 =0, ∀1 ≤ i, j ≤ K, PN P zi I(Ψ(zi )=j) where cj = Pi=1 is the class-mean of j-th class and cG = N1 N N i=1 zi is the global mean of feature, i=1 I(Ψ(zi )=j)

both are computed under the neural collapse state.

Theorem 2 (Convergence to Self-duality [65]) For a sufficiently large neural network, the last-layer linear classifiers reside in the dual vector space of the class-means when the model has converged:   WC 1 1 ⊤ N C3 := −√ IK − 1K 1K , (5) K K −1 WC F F

where C = [c1 − cG , . . . , cK − cG ] represents the centered class-mean matrix. Theorem 3 (Variability Collapse [65]) As training progresses, the features within each class converge to their respective class-means, causing within-class variation to collapse to zero. Let the within-class covariance matrix be denoted as ΣW ∈ Rd×d and the between-class covariance matrix as ΣB ∈ Rd×d , defined as follows: N

ΣW :=

K

1 XX dij dTij , NK i=1 j=1

K

ΣB :=

1 X (cj − cG ) (cj − cG )T , K j=1

8

where dij = I(Ψ(zi ) = j) · (zi − cj ) is the distance between the i-th feature and its class-mean. Then, we can measure the extent of within-class variability collapse by analyzing the relative magnitudes of ΣB and ΣW for the learned features: N C 1 := Σ†B ΣW → 0, (6) where † denotes the pseudo-inverse. These theorems indicate that, at convergence, both the class means and classifier weights tend to align with a Simplex ETF structure, as depicted in Figure 5. Since OWTTA starts from a pre-trained source model, we assume that the source model has been sufficiently trained and thus approximately satisfies this neural collapse property. See Table 7 for empirical verification. Assumption 1 The pre-trained model can be considered well-trained on the source domain, with its classifier weights collapsing into a Simplex ETF.
Assumption 2 Let F (θ ∗ , ·) = σ w∗ f (θ ∗ , ·) + b∗ be a classification network that has reached the neural collapse state, where w∗ and b∗ are its classifier weights and bias, respectively. Let z = f (θ ∗ , x) denote the feature representations of the training set SS ∈ {(xi , yi )|xi ∈ DS }. Then, for some sufficiently small positive real ϵ, it holds that ∥b∗j ∥ < ϵ, ∀j. (7) ∗ ,z ⟨w:,j Ψ(z)=j ⟩ Under Assumption 2, the optimal bias b∗ is negligible in the neural collapse state. See Table 4.5 for empirical verification. Accordingly, the bias term can be omitted, and the optimization problem max ⟨w:,j , zΨ(z)=j ⟩ + bj reduces to max ⟨w:,j , zΨ(z)=j ⟩. This indicates that the classifier weight of each w:,j ,bj

w:,j

class is optimized toward the same direction as its corresponding class prototype. We formalize this relation in the following lemma.
Lemma 1 Let F (θ ∗ , ·) = σ w∗ f (θ ∗ , ·) + b∗ be a sufficiently trained classification network that approximately satisfies the neural collapse property. Let f (θ ∗ , x) denote the feature representation of a source training sample x from SS = {(xi , yi ) | xi ∈ DS }. Let cj be the prototype of the j-th class. Then we have: ∗ w:,j cj = ∗ ∥, ∥cj ∥ ∥w:,j

∀j.

(8)

Following Lemma 1, we use the normalized classifier weights as class prototypes under the source-free setting. Therefore, Eq. (3) can be computed at each batch step without accessing source data, enabling reliable updates with selected target samples.

3.3

Neural Collapse Approximation

We further consider the effect of data distribution shifts. The neural collapse structure learned by the source model reflects the source-domain feature geometry, but may become misaligned with target-domain representations due to the data distribution shifts. From the neural collapse perspective, this featureprototype misalignment between shifted target representations and source-domain prototypes becomes a key reason for performance degradation. Proposition 1 Given a source domain training set SS ∈ {(xi , yi )|xi ∈ DS } and a target domain test set ST ∈ {xi |xi ∈ DT }. Under the neural collapse state, let c∗S and c∗T denote the prototypes of source and target domains, respectively, each forming a Simplex ETF:     ⊤ ∗ ∗⊤ ⊤ 1 1 c∗S c∗⊤ = α I − 1 1 , c c = β I − 1 1 K K K K S K T T K , K K where α and β denote the respective scaling factors for the source and target Simplex ETFs. 9

By Proposition 1, if DS ̸= DT , then c∗S ̸= c∗T , i.e., the source and target prototypes differ due to the data distribution shift, which leads to classification errors. Based on this understanding, we aim to guide target representations toward a target-domain NC-inspired geometry during adaptation.
Lemma 2 Let F (θ ∗ , ·) = σ w∗ f (θ ∗ , ·) + b∗ be a classification network that has reached the neural collapse state with the training set SS ∈ {(xi , yi )|xi ∈ DS }. Then, for each training sample xi , its logits vector p = F (θ ∗ , xi ) is a one-hot vector, i.e., ( 1, Ψ(xi ) = j, pj = 0, otherwise. As stated in Lemma 2, the logits of a classification network tend to degenerate into one-hot vectors under the neural collapse state. The entropy loss in Eq. (4) encourages confident predictions, which implicitly pushes each sample toward its corresponding class mean and promotes a more compact target-domain representation. This connection to neural collapse explains why entropy minimization can improve adaptation effectiveness in TTA. However, as indicated by Proposition 1, distribution shifts may alter the target-domain feature geometry, making direct entropy minimization in Eq. (4) blindly push target features toward source-domain prototypes. Such indiscriminate updates may distort the embedding space and damage the structured representations required for reliable adaptation. We therefore seek to approximate the neural collapse state in the target domain. Since exact neural collapse is difficult to achieve in the unsupervised setting, we use it as a reliable structural prior for target adaptation. Specifically, we jointly update the model parameters and prototypes, and further leverage embedding-structure maintenance and stochastic prototype updates to promote balanced and compact representations during adaptation. 3.3.1

Maintain Embedding Structure

According to neural collapse theory, learned class representations tend to form a balanced geometric structure, i.e., a Simplex ETF, at convergence. We regard this property as a structural prior and preserve the balanced latent embedding geometry during target adaptation. Let w∗ denote the classifier weights, and let zΨ(z)=j denote the feature representations assigned to the j-th class. Then we have: p̂∗ =

K

K

j=1

j=1

1 X 1 X σ(⟨w∗ , zΨ(z)=j ⟩) = σ(⟨w∗ , cj ⟩) K K K

=

1 1 1 1 X σ([− ,− , . . . , 1, . . . , − ]) K K −1 K −1 K −1 j=1

=

1 1K , K

where p̂∗ denotes the ideal class-averaged prediction under neural collapse, corresponding to a uniform distribution over classes. Guided by this uniform prior, we encourage the average logits of ID samples to approach K1 1K , thereby maintaining a balanced embedding structure during adaptation:   t 1 Ld (θ) = DKL p̂ 1K , (9) K P + t t where p̂t = N1+ N i=1 pi is the average of logits of t-th batch, DKL is the KL-divergence, p is the output of ID samples. Eq. (9) regularizes the adaptation by encouraging balanced class-wise predictions, which facilitates the preservation of the neural collapse-inspired embedding structure. 10

3.3.2

Stochastic Prototype Update

As stated in Proposition 1, source-domain and target-domain prototypes may differ under distribution shifts. Therefore, we further investigate how to update prototypes during adaptation. In OWTTA, target samples arrive in a single-pass stream, and each mini-batch usually covers only a subset of classes. This naturally introduces a class-coverage imbalance during prototype updates. For example, when the number of classes is much larger than the batch size, many classes are absent from the current batch. Updating prototypes only with the current batch may therefore produce biased estimates of the target distribution, hindering reliable adaptation and degrading OWTTA performance. To address this issue, we propose a stochastic prototype update mechanism that selectively updates only the prototypes corresponding to classes present in the current batch, while keeping others fixed. This strategy ensures that updates are more relevant to the incoming data and improves adaptation efficiency. To facilitate this, the objective function is extended as follows: L (θ, C) = L+ e (θ, C) + λLd (θ) + Ls (θ, C)   N+
1 X t 1 t = + H pi +λDKL p 1K N K i=1

+

1 N+

N+ X K X

(10)

2

t ξij zti − ctj 2 ,

i=1 j=1

where λ is the trade-off parameter and ξ ∈ {0, 1} is an indicator factor assigning zti into its nearest prototype:    t−1 2 t  1, if j = arg max exp −∥zi −cj ∥2  , PK t−1 2 t t ∥2 (11) ξij = j j exp −∥zi −cj  0, otherwise. The first term of Eq. (10) is guided by the compactness prior in Theorem 3, encouraging each sample to move closer to its corresponding prototype. The second term is inspired by the simplex-ETF geometry in Theorem 1, encouraging a well-structured inter-class embedding space. The third term alleviates the prototype mismatch described in Proposition 1 by progressively adapting the prototypes toward the targetdomain feature distribution. These three terms jointly incorporate the neural collapse prior into unsupervised target adaptation, enabling the model to form a compact and balanced target-domain representation without requiring label supervision. 3.3.3

Alternating Optimization

Unlike the network parameters θ, prototypes C can be unstable when directly updated by stochastic gradients. This instability occurs because the prototypes estimated from different mini-batches can vary significantly. To achieve the stochastic prototype update, we propose a decomposition-coordination optimization approach based on the Alternating Direction Method of Multipliers (ADMM) to optimize Eq. (10). In particular, we employ an auxiliary variable µ to alternately update the model parameters and prototypes: L (θ, µ, C) = L+ e (θ, µ) + λLd (θ) + Ls (θ, µ) +ρ

K X

∥µj − cj ∥22 ,

j=1

where ρ is a trade-off parameter controlling the prototype updates between batches. Our decomposition-

11

coordination optimization technique consists of three iterative steps:
θ t = arg min L θ, µt−1 , Ct−1 , θ
t µ = arg min L θ t , µ, Ct−1 , µ
t C = arg min L θ t , µt , C . C

We then develop efficient update solutions for each sub-problem, allowing for neural collapse approximation. For updating θ: Since no closed-form solution exists, we optimize θ t using gradient descent:
∂L θ, µt−1 , Ct−1 t t−1 θ =θ −ϵ , (12) ∂θ t−1 θ=θ

where ϵ is the step size. Eq. (12) can be executed using advanced gradient descent methods. For updating µ: The sub-problem involves a two-objective optimization. By setting the gradient of the objective function to zero, we can obtain an analytic solution: P + t t
ρct−1 + Nη+ N ∂L θ t , µ, Ct−1 i=1 ξij zi j t , = 0 =⇒ µj = P + ∂µj ρ + η+ N ξ t N

(13)

i=1 ij

where η is a balance factor. The numerator of Eq. (13) controls the sample weights of each prototype, ρ while the denominator represents the sample counts assigned to the class. Let κ = and η PN + t ρ+

c̄tj =

PN +

t t i=1 ξij zi PN + t i=1 ξij

N+

i=1 ξij

. Eq. (13) can further simplified to the following form: µtj = κct−1 + (1 − κ)c̄tj , j

j = 1, 2, . . . , K,

(14)

where κ acts as a self-learning parameter that balances the influence between the previous prototype ct−1 j and the prototype calculated in the current batch, c̄tj . It is noteworthy that in Eq. (14), c̄tj is calculated based not only on the features assigned to the class but also on the number of samples allocated to it. In fact, Eq. (14) corresponds to the exponential moving average (EMA), which has been demonstrated that the use of it ensures both stability and efficiency in scenarios of mini-batch updates [66]. For updating C: By setting the gradient of the objective function to zero, the analytic solution of C is: ∂L(θ t , µt , C) = 0 =⇒ ctj = µtj , j = 1, 2, . . . , K. (15) ∂cj To summarize, our ReNC method filters out the OOD samples at first and approximates the neural collapse state in the target domain by updating prototypes stochastically to achieve reliable and efficient OWTTA. Finally, the predictions ŷit for each batch can be obtained by: ( F (θ t , xti ), sti < τt∗ , t ŷi = (16) |YS | + 1, otherwise. A detailed overview of the proposed method is provided in Algorithm 1.

12

Algorithm 1 Reliable Neural Collapse approximation (ReNC) Require: Test samples x from the target domain ST , pre-trained model F (θ, ·); # Initialize prototypes 1: Initialization: Extract prototypes C from classifier weights w and initialize µ = C; 2: for t ← 1 to T do # Partition data into ID (N + ) and OOD (N − ) samples 3: Calculate sti and τt∗ by Eq. (2) and Eq. (3); # Neural collapse approximation 4: Update θ t by Eq. (12), update µt by Eq. (13), update ct by Eq. (15); # Inference 5: Obtain ŷit by Eq. (16). 6: end for

4

Experiments

In this section, we empirically validate the proposed ReNC on several open-world benchmarks. We begin by describing the datasets, evaluation metrics, baselines.Next, we conduct extensive experiments on five open-world benchmarks to demonstrate the effectiveness of our ReNC. Additionally, we validate Assumption 1 and Assumption 2, and further discuss the connection between TTA and neural collapse. Finally, we conduct comprehensive analyses, including ablation studies, computational cost analysis, parameter sensitivity analysis, different ID/OOD ratio settings, continual OWTTA evaluation, ViT-backbone evaluation, and significance tests, to further assess the robustness and effectiveness of our method. Table 1: In-distribution Datasets Information Datasets #Images #Classes Target Domain CIFAR10-C CIFAR100-C ImageNet-C ImageNet-R VisDA-C

4.1

10,000 10,000 50,000 30,000 55,388

10 100 1,000 200 12

Corruption Corruption Corruption Style Transfer Style Transfer

Open-world Benchmarks

The open-world benchmarks employed in our experiments include both In-Distribution (ID) and Out-OfDistribution (OOD) samples. Specifically, the ID samples are drawn from two types of target domains, i.e., corruption and style transfer. For corruption domain, CIFAR10-C [67], CIFAR100-C [67], ImageNetC [67] are introduced, while for style transfer domain, we have ImageNet-R [68] and VisDA-C [69]. Details of the ID samples are presented in Table 1. Besides, we select Gaussian noise and five real-world datasets with distributions different from the ID samples to serve as the OOD datasets: Noise, composed with random Gaussian noise, MNIST [70], SVHN [71], Tiny-ImageNet [72], CIFAR100-C and CIFAR10-C. |S

|

We concatenate ID and OOD samples at an OOD sample ratio of |STTO| = 1 to form the open-world I benchmarks. Specifically, for CIFAR10-C and CIFAR100-C, the OOD samples include Noise, MNIST, TinyImageNet, and CIFAR100-C/CIFAR10-C. For ImageNet-C, ImageNet-R, and VisDA-C, the OOD samples are Noise, MNIST, and SVHN.

13

4.2

Evaluation metrics

To evaluate the OWTTA performance, similar to [8], we introduce three evaluation metrics: P xi ∈DTI 1 (ŷi = Ψ (xi )) · 1(Ψ(xi ) ∈ Ys ) P ACCI = , xi ∈DTI 1(Ψ(xi ) ∈ Ys ) P xi ∈DTO 1(ŷi ∈ YO ) · 1(Ψ(xi ) ∈ YO ) P ACCO = , xi ∈DT 1(Ψ(xi ) ∈ YO ) O

ACCI · ACCO , ACCH = 2 · ACCI + ACCO where ACCI and ACCO evaluate the classification accuracy of ID and OOD samples, respectively. ACCH is the harmonic mean of ACCI and ACCO measuring the overall OWTTA performance.

4.3

Baselines

The baselines in our experiments are summarized as follows. TEST: The classification results are obtained directly from the pre-trained model without any further adaptation, serving as a baseline for comparison. BN [73]: The batch normalization layers of the pre-trained network are unfrozen, allowing them to adjust their parameters based on the test samples. TENT [23]: It introduces two affine parameters to adjust the batch normalization layers of the model and updates these parameters by minimizing the entropy loss of target samples. SHOT [12]: It keeps the classifier parameters fixed while using the pre-trained feature encoder as the initialization for learning in the target domain. OSTTA [10]: This method filters out low-confidence OOD samples and adapts the model by minimizing the entropy of high-confidence samples. EATA [5]: This method actively selects high-confidence test samples for adaptation while filtering out lowconfidence ones, and mitigates catastrophic forgetting through Fisher regularization. RMT [35]: It uses symmetrical cross-entropy as a consistency loss and leverages contrastive learning for feature alignment, reducing error accumulation in continual and gradual test-time adaptation. CoTTA[32]: This method mitigates error accumulation by generating weight-averaged and augmentation-averaged pseudo-labels and alleviates catastrophic forgetting via stochastic restoration of source pre-trained weights during continual test-time adaptation. UniEnt [11]: It uses a two-component GMMs to partition ID and OOD samples based on similarity scores between features and prototypes. The entropy of ID samples is minimized, while that of OOD samples is maximized for better adaptation. OWT3 [8]: This method adapts the model to the target domain by aligning it with statistical information from the source domain and generating new prototypes for OOD samples. However, in OWTTA, since source statistics are unavailable, we replace the prototypes in OWT3 with normalized classifier weights and remove the alignment component of the loss function. For the baselines not tailored for OWTTA, we follow the approach in [8] for comparison. Specifically, samples from each batch are first partitioned into the ID and OOD samples using τt∗ calculated by Eq. (3), with the prototypes extracted by pre-trained classifier weights. Then, these methods are applied to the ID samples for the adaptation. Following [11], we report results for OSTTA and UniEnt combined with TENT. Besides, our evaluation metric differs from those used in OSTTA and UniEnt 3 , since we apply the stricter evaluation metrics, identical to those used in OWT3 [8].

4.4

Overall Comparisons in the OWTTA Scenario

In this section, we validate the effectiveness of the proposed method in the OWTTA scenario. The performance is evaluated using ACCI , ACCO and ACCH . We compare the proposed method with ten baselines 3

We ensure the performance of OSTTA and UniEnt using the original metrics remains consistent to their respective papers.

14

across all the open-world benchmarks, including both corrupted and style transfer domains with OOD samples. Tables 2, 3 and 4 present the results for the corrupted domains. The results for the style transfer domain are shown in Tables 5 and 6. These experimental results reveal several noteworthy observations. 1) Our ReNC consistently achieves superior ACCH across all baselines, validating its capability to effectively adapt to diverse datasets in the open-world scenarios. Specifically, on the ImageNet-C dataset, where existing methods struggle to detect OOD samples using fixed prototypes, ReNC achieves consistently higher ACCH , highlighting the effectiveness of our prototype updating mechanism. Meanwhile, on ImageNetR and VisDA-C datasets, where all methods have relatively high ACCO , our method still improves ID classification accuracy by approximating the neural collapse state in the target domain. 2) UniEnt and OSTTA underperform compared to TENT on nearly all the open-world benchmarks, suggesting that GMMs and confidence-based partitioning mechanisms may not align well with their entropybased optimization goals. In contrast, our ReNC performs data partitioning and prototype updating within the same latent space, making it more robust since both processes are tightly coupled. 3) The classification accuracy decreases for all methods as the number of classes increases, especially for ImageNet-C, which contains 1000 classes. Our results in Table 4 indicate an improvement in ACCH from 21.37% to 31.02%. This improvement can be attributed to the neural collapse approximation mechanism of ReNC. For the ImageNet-C dataset, the number of classes exceeds the batch size, resulting in imbalanced or even missing classes in each batch. While ReNC updates the prototypes stochastically, it effectively addresses this issue and thus achieves better performance compared to other methods. In conclusion, our ReNC stochastically updates the prototypes to fit the target data through neural collapse approximation, operating without any source domain information. It achieves the best performance across all the open-world benchmarks. Table 2: Open-world test-time adaptation results on CIFAR10-C with OOD samples. Best marked in bold, second best underlined. Noise

CIFAR10-C

MNIST

SVHN

Tiny

CIFAR100-C

ACCI ACCO ACCH ACCI ACCO ACCH ACCI ACCO ACCH ACCI ACCO ACCH ACCI ACCO ACCH TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

69.19 39.44 42.75 42.71 53.43 40.65 45.50 35.89 39.29 69.41 79.14

99.96 81.78 60.57 47.77 61.99 50.60 60.72 50.15 13.69 21.80 64.14 49.76 44.32 44.90 68.20 47.03 59.97 47.48 99.96 81.93 99.98 88.35

63.61 60.19 63.21 64.57 53.88 58.04 73.35 60.20 59.70 63.89 82.23

91.11 74.92 69.56 64.54 72.80 67.67 73.51 68.75 36.94 43.83 72.63 64.52 99.66 84.50 69.25 64.41 69.79 64.35 91.10 75.11 98.12 89.47

63.54 69.21 70.84 71.12 62.76 66.62 71.97 69.20 68.47 63.72 77.00

15

90.91 74.80 80.17 74.29 82.92 76.41 84.37 77.18 41.20 49.74 82.29 73.63 88.65 79.44 80.11 74.26 80.33 73.93 90.97 74.94 91.54 83.64

61.09 69.02 69.70 69.51 74.85 66.72 68.65 69.07 68.58 61.05 70.76

77.48 68.32 75.15 71.95 76.23 72.82 77.06 73.09 32.38 45.20 77.37 71.65 78.11 73.08 75.14 71.98 75.46 71.86 77.56 68.32 77.63 74.04

61.27 68.83 69.22 69.10 76.39 66.83 66.84 68.67 68.80 61.37 70.26

76.58 68.07 73.33 71.01 74.07 71.56 74.76 71.82 29.80 42.87 75.00 70.68 74.51 70.47 73.65 71.07 73.50 71.07 76.47 68.09 75.06 72.58

Table 3: Open-world test-time adaptation results on CIFAR100-C with OOD samples. Best marked in bold, second best underlined. Noise

CIFAR100-C

MNIST

SVHN

Tiny

CIFAR10-C

ACCI ACCO ACCH ACCI ACCO ACCH ACCI ACCO ACCH ACCI ACCO ACCH ACCI ACCO ACCH TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

25.30 21.22 24.45 25.98 12.20 22.73 27.68 20.93 20.05 25.54 48.87

99.99 40.38 93.29 34.58 95.75 38.95 96.62 40.95 40.58 18.76 94.06 36.61 99.92 43.35 93.25 34.19 94.48 33.08 99.99 40.69 97.74 65.16

24.76 26.09 27.50 28.26 17.97 25.58 26.40 25.23 25.54 24.51 38.75

46.81 32.39 93.21 40.77 91.49 42.29 87.36 42.71 46.56 25.93 92.08 40.04 83.61 40.13 92.85 39.68 92.86 40.06 45.51 31.86 94.31 54.93

26.34 30.60 32.05 32.42 33.33 29.57 27.92 30.09 31.36 26.48 40.96

91.65 40.92 95.42 46.34 95.17 47.95 95.14 48.36 62.59 43.50 96.20 45.24 89.52 42.56 95.26 45.73 94.48 47.09 91.70 41.09 78.22 53.77

26.20 31.41 31.33 31.18 36.81 30.35 27.58 31.17 31.20 26.34 36.27

86.68 40.24 89.69 46.53 89.64 46.43 89.71 46.28 52.15 43.16 90.03 45.40 81.59 41.22 89.85 46.28 89.19 46.23 86.68 40.40 71.91 48.22

25.79 31.95 32.08 32.07 42.75 31.44 28.34 31.86 32.52 25.93 35.57

87.86 39.88 89.93 47.15 89.93 47.29 90.10 47.30 43.67 43.21 89.79 46.57 84.31 42.42 89.90 47.05 89.03 47.64 87.82 40.04 76.75 48.61

Table 4: Open-world test-time adaptation results on ImageNet-C with OOD samples. Best marked in bold, second best underlined. ImageNet-C

Noise

MNIST

SVHN

ACCI ACCO ACCH ACCI ACCO ACCH ACCI ACCO ACCH TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

11.79 40.85 18.30 11.33 59.57 19.04 11.62 82.04 20.36 9.73 66.50 16.98 15.20 74.24 25.23 16.24 77.93 26.88 9.31 68.49 16.39 15.26 72.67 25.22 17.28 78.79 28.34 9.70 64.57 16.87 14.76 71.69 24.48 17.70 79.63 28.96 13.90 65.34 22.92 20.54 57.34 30.25 21.93 59.43 32.04 15.92 88.86 27.00 20.72 93.21 33.90 20.69 93.14 33.86 30.38 99.76 46.58 11.92 90.93 21.08 12.30 96.85 21.83 8.75 67.48 15.49 13.78 82.82 23.63 14.13 74.66 23.76 8.29 73.44 14.90 14.13 79.71 24.00 14.81 82.88 25.13 9.76 68.03 17.07 15.41 75.29 25.58 16.45 78.67 27.21 36.96 99.78 53.94 35.61 98.79 52.35 36.73 97.87 53.41

Table 5: Open-world test-time adaptation results on ImageNet-R with OOD samples. Best marked in bold, second best underlined. ImageNet-R

Noise

MNIST

SVHN

ACCI ACCO ACCH ACCI ACCO ACCH ACCI ACCO ACCH TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

24.71 100.00 39.63 23.60 99.81 38.17 22.46 98.89 36.61 16.38 89.76 27.70 18.70 82.97 30.52 19.03 86.08 31.17 16.87 93.23 28.57 19.81 84.72 32.11 20.09 87.99 32.71 17.27 96.24 29.28 19.70 81.43 31.72 20.44 89.61 33.29 25.46 58.59 35.50 22.46 49.67 30.93 25.66 58.02 35.58 16.21 91.71 27.55 18.95 86.24 31.07 18.95 88.37 31.21 8.99 73.27 16.02 12.27 93.79 21.70 16.36 77.63 27.02 15.58 85.41 26.35 17.40 86.71 28.98 17.51 83.04 28.92 10.56 72.46 18.43 15.01 79.59 25.26 15.84 83.72 26.64 16.56 92.13 28.07 19.13 83.75 31.15 19.42 86.93 31.75 39.85 98.67 56.77 37.99 97.32 54.65 39.36 95.68 55.78

16

Table 6: Open-world test-time adaptation results on VisDA-C with OOD samples. Best marked in bold, second best underlined. Noise

VisDA-C

MNIST

SVHN

ACCI ACCO ACCH ACCI ACCO ACCH ACCI ACCO ACCH TEST 41.27 100.00 58.43 43.12 97.41 59.78 42.06 99.46 59.12 BN 45.48 99.64 62.45 42.06 79.96 55.12 42.80 88.77 57.75 TENT 51.57 100.00 68.05 41.50 79.96 54.64 44.57 88.40 59.26 SHOT 42.39 99.94 59.53 42.69 68.08 52.48 43.45 72.69 54.33 OSTTA 39.60 54.33 45.81 46.41 29.81 36.30 44.91 45.55 45.23 EATA 43.12 99.68 60.20 41.49 81.13 54.90 41.75 89.25 56.89 RMT 25.87 74.63 38.42 26.09 99.98 41.38 30.36 63.64 41.11 CoTTA 44.01 99.56 61.04 40.71 83.91 54.82 40.89 89.86 56.20 UniEnt 54.84 95.99 69.80 41.92 80.13 55.04 42.83 88.53 57.73 OWT3 46.12 99.99 63.12 42.14 77.39 54.57 43.04 91.60 58.56 Ours 55.96 99.63 71.67 47.42 99.17 64.16 53.93 98.23 69.63

4.5

Connections between TTA and Neural Collapse

In this section, we investigate the connection between TTA and neural collapse to validate that the proposed method aligns with the intrinsic properties of well-trained neural networks. Neural collapse in the source domain: Assumption 1 states that the pre-trained model has already collapsed in the source domain. To validate this, we calculate the N C 1 and N C 3 of the pre-trained model using its training data. N C 1 reflects how well the features collapse to their class-means and N C 3 measures the self-duality between class-mean and the classifier weights. The lower the values of these two metrics, the closer the model is to achieving the neural collapse state. As shown in Table 7, the models trained on different datasets have indeed collapsed on their respective datasets as indicated by low N C 1 and N C 3 values4 . Table 7: N C 1 and N C 3 on each training dataset. Lower values indicate closer to the neural collapse state. CIFAR10 N C1

N C3

CIFAR100

ImageNet

N C1

N C1

N C3

N C3

VisDA N C1

N C3

0.1712 0.2466 0.7541 0.6235 2.3275 1.1968 0.3031 0.5146

Table 8: The ratio b to ⟨w, c⟩ quantifies their relative influence. Lower values imply that the bias term is increasingly negligible in the neural collapse state. Dataset Ratio

CIFAR10

CIFAR100

ImageNet

VisDA

1.526 × 10−2 7.693 × 10−3 1.116 × 10−3 1.873 × 10−3

Empirical validation of Assumption 2: To verify the validity of Assumption 2, we conduct experiments on a series of classification neural networks. According to Theorem 3, in the context of neural collapse, features z have collapsed to their class-mean, i.e., prototype c. Therefore, to empirically validate this assumption, we measure the ratio of the bias term bj to the weight-prototype interaction ⟨w:,j , cj ⟩, i.e., 1 PK ∥bj ∥ j ⟨w:,j ,cj ⟩ . As shown in Table 8, the weight-prototype interaction significantly outweighs the bias term, K P ∥bj ∥ i.e. K1 K j ⟨w:,j ,cj ⟩ ≪ 1. This empirical result validates Assumption 2, which posits that the effect of the bias term is negligible in the neural collapse state. Neural collapse in the target domain: TTA algorithms update model parameters to align with the data distribution of the target domain. Their primary goal is to correct the inaccuracies caused by target 4

Due to hardware limitations, we assess the neural collapse state of the ImageNet dataset using its validation set.

17

Table 9: N C 1 and N C 3 on CIFAR10-C and CIFAR100-C with OOD samples. Best results are marked in bold, second best are underlined. The gray background indicates the highest ACCS . Noise

CIFAR10-C TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

SVHN

Tiny

CIFAR100-C

N C3

N C1

N C3

N C1

N C3

N C1

N C3

N C1

N C3

0.7386 1.5176 1.4132 1.3871 1.4157 1.5056 1.3723 1.9769 1.4784 0.7358 0.5731

0.3548 0.4034 0.3917 0.3914 0.4056 0.4046 0.4611 0.4753 0.4017 0.3545 0.3202

0.7386 0.8026 0.7314 0.7057 0.7922 0.8125 0.6609 0.7999 0.8185 0.7358 0.5782

0.3548 0.3637 0.3497 0.3461 0.3633 0.3665 0.3963 0.3626 0.3666 0.3545 0.3174

0.7386 0.6209 0.5924 0.5815 0.6154 0.6307 0.6399 0.6208 0.6270 0.7350 0.5667

0.3548 0.3189 0.3151 0.3147 0.3194 0.3205 0.3979 0.3184 0.3195 0.3545 0.3203

0.7386 0.6052 0.5946 0.5914 0.5944 0.6157 0.6970 0.6074 0.6101 0.7366 0.5795

0.3548 0.3204 0.3179 0.3180 0.3212 0.3224 0.4060 0.3206 0.3201 0.3545 0.3195

0.7386 0.5970 0.5884 0.5863 0.5852 0.6065 0.6858 0.5963 0.6039 0.7368 0.5761

0.3548 0.3193 0.3179 0.3185 0.3203 0.3215 0.4143 0.3189 0.3200 0.3545 0.3202

Noise

CIFAR100-C TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

MNIST

N C1

MNIST

SVHN

Tiny

CIFAR10-C

N C1

N C3

N C1

N C3

N C1

N C3

N C1

N C3

N C1

N C3

9.5278 14.1927 11.6919 10.5667 13.3452 13.0616 8.1638 14.6210 14.2197 9.4840 7.7409

0.8884 0.9529 0.9177 0.9027 0.9586 0.9414 0.9008 0.9554 0.9363 0.8877 0.8606

9.5278 10.6814 9.4333 8.8852 10.3678 10.9230 8.2756 11.2101 11.7456 9.5121 7.6476

0.8884 0.9139 0.8909 0.8815 0.9193 0.9124 0.9038 0.9200 0.9174 0.8882 0.8700

9.5278 8.2031 7.7855 7.6591 7.9534 7.9622 8.2989 8.4420 8.3867 9.4805 8.0640

0.8884 0.8764 0.8649 0.8608 0.8755 0.8725 0.9036 0.8792 0.8729 0.8876 0.8604

9.5278 7.8558 7.7511 7.7561 7.4570 7.6159 8.5444 7.9485 8.2405 9.4835 7.7153

0.8884 0.8689 0.8631 0.8610 0.8673 0.8682 0.9172 0.8704 0.8713 0.8877 0.8557

9.5278 7.7276 7.6456 7.6657 6.9739 7.3165 8.0033 7.8608 8.1938 9.4799 7.6808

0.8884 0.8709 0.8652 0.8631 0.8634 0.8675 0.9041 0.8727 0.8757 0.8876 0.8577

Table 10: N C 1 and N C 3 on ImageNet-C with OOD samples. Best results marked in bold, second best underlined. The gray background indicates the highest ACCS . Noise MNIST SVHN ImageNet-C N C1 N C3 N C1 N C3 N C1 N C3 TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

3.7824 7.6031 7.4605 7.6685 6.0655 6.9685 1.9977 7.7565 8.0681 7.4846 5.6142

1.2762 1.2733 1.2755 1.2759 1.2586 1.2501 1.4037 1.2733 1.2695 1.2724 1.2418

3.7824 6.0601 5.6129 5.8075 4.1373 5.3900 2.7490 7.2648 5.6453 5.9790 5.3901

18

1.2762 1.2620 1.2635 1.2610 1.2554 1.2427 1.3911 1.2652 1.2506 1.2609 1.2436

3.7824 6.7761 6.3444 6.3542 4.1407 5.4280 2.0345 7.9593 5.8017 6.7146 5.3158

1.2762 1.2581 1.2574 1.2557 1.2557 1.2527 1.4074 1.2616 1.2496 1.2571 1.2434

Table 11: N C 1 and N C 3 on ImageNet-R with OOD samples. Best results marked in bold, second best underlined. The gray background indicates the highest ACCS . Noise

ImageNet-R

N C1 TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

MNIST N C3

N C1

N C3

SVHN N C1

N C3

20.6384 1.0441 20.6384 1.0441 20.6384 1.0441 30.1058 1.0541 28.6891 1.0577 27.8901 1.0540 27.4597 1.0478 27.5288 1.0549 26.6687 1.0507 26.1513 1.0437 27.2900 1.0539 25.9399 1.0480 25.8711 1.0483 24.8492 1.0584 24.3873 1.0555 29.5161 1.0521 28.0359 1.0552 27.4065 1.0521 44.8059 1.1647 37.8196 1.1126 39.1313 1.1318 32.4597 1.0595 31.2893 1.0632 30.3840 1.0595 33.4932 1.0751 26.7478 1.0536 26.0775 1.0478 28.9696 1.0513 27.6794 1.0548 27.1448 1.0512 19.7501 1.0345 19.3365 1.0369 19.2924 1.0383

Table 12: N C 1 and N C 3 on VisDA-C with OOD samples. Best results marked in bold, second best underlined. The gray background indicates the highest ACCS . Noise

VisDA-C TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

MNIST

SVHN

N C1

N C3

N C1

N C3

N C1

N C3

1.8591 1.3001 1.1664 1.0559 1.4202 1.2970 6.2168 1.3695 1.1522 1.1679 1.1548

0.8081 0.7268 0.7034 0.6944 0.7503 0.7272 0.8892 0.7298 0.7019 0.7083 0.7097

1.8591 1.2937 1.3417 1.1516 1.2309 1.2910 6.0925 1.3806 1.3653 1.1764 1.3135

0.8081 0.7217 0.7287 0.7098 0.748 0.7215 0.8393 0.7229 0.6829 0.7031 0.7055

1.8591 1.2915 1.1755 1.1388 1.2792 1.2934 4.8335 1.3659 1.3198 1.1955 1.2832

0.8081 0.7126 0.7005 0.7003 0.7308 0.7130 0.8780 0.7158 0.6778 0.6983 0.7046

19

samples producing incorrect labels when passed through the pre-trained model. Since the pre-trained model has well-collapsed in the source domain, following Theorem 2 and Theorem 3, each feature should collapse to its corresponding classifier weight vector. The inaccuracies occur because the features generated by the encoder do not align well with the pre-trained classifier. From this perspective, effective TTA should guide target representations toward a target-domain NC-like geometry. To support this claim, we measure the N C 1 and N C 3 on all the open-world benchmarks, and the results are presented in Tables 9-12. For the corrupted domain, we observe that stronger ID classification performance is often accompanied by better NC-related properties. For example, on CIFAR10-C with Noise and MNIST as OOD samples, ReNC achieves the lowest N C 1 and N C 3 values among all methods, together with the highest ACCS . For CIFAR10-C with Tiny-ImageNet and CIFAR100-C as OOD samples, OSTTA obtains the best ACCS , while ReNC still maintains competitive NC-related metrics. These results suggest that although classification accuracy is not solely determined by NC metrics, preserving compact and well-aligned representations can effectively facilitate the target adaptation process. For the style-transfer domain, as shown in Tables 11 and 12, ImageNet-R follows a similar trend to the corrupted domain. VisDA-C presents a more challenging case, where SHOT achieves relatively strong collapse metrics but still suffers from poor ID classification performance, even falling below TEST in some settings. This discrepancy may be due to the distinct training and testing sets in VisDA-C, which cause the source-domain pre-trained classifier to poorly align with the target domain features. ReNC achieves comparable NC-related properties and strong adaptation performance, supporting the effectiveness of neural collapse approximation. In general, neural collapse provides an informative perspective for understanding classification representations. The improved NC-related properties of ReNC offer a plausible explanation for its better performance.

4.6

Ablation Study

In this section, we evaluate the impact of entropy minimization (L+ e ), the one-hot distribution prior (Ld ), the alignment of samples with their prototypes (Ls ), and finally, the effectiveness of Updating Prototypes (U.P.). The TTA performance is reported using ACCH on all the open-world benchmarks, as shown in Table 13 and Table 14. From these tables, we have the following observations: 1) Ls may degrade classification performance if the prototypes are not updated. For example, on the CIFAR100-C dataset, applying Ls leads to a drop in ACCH with OOD samples such as MNIST, SVHN, Tiny-ImageNet and CIFAR10-C. This may be because the prototypes, initialized by the weights of the pretrained classifier, do not fit well with the target domain. Pushing samples towards these fixed prototypes can mislead the classification process. 2) Updating prototypes generally enhances TTA performance on all the open-world benchmarks. This can be attributed to the proposed neural collapse approximation mechanism, which updates prototypes stochastically, enabling the prototypes to well align with the target domain. 3) Simply minimizing L+ e or Ld can lead to degraded TTA performance. As shown in Table 14, when updating prototypes, optimizing the model with L+ e or Ld alone does not provide optimal adaptation results. Overall, using any of these losses individually may not yield better adaptation results. Our ReNC integrates all the losses and updating mechanisms into a unified pipeline. It encourages variability collapse by pushing samples toward their corresponding prototypes, while preserving balanced embedding structure, which supports more robust and effective adaptation.

20

Table 13: Ablation study on the CIFAR10-C and CIFAR100-C with OOD samples. Best results marked in bold, second best underlined.

CIFAR10-C

Noise MNIST SVHN

Tiny

L+ e Ld Ls U.P. ACCH ACCH ACCH ACCH ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓

CIFAR100-C

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

4.7

✓ ✓ ✓ ✓

ACCH

86.05 86.07 86.05

78.60 78.56 78.62

82.10 82.06 82.06

69.25 69.27 69.27

68.04 68.00 68.00

88.02 88.07 88.04 88.35

89.21 89.24 89.21 89.47

83.22 73.77 83.30 73.42 83.32 73.34 83.64 74.04

72.13 71.47 71.81 72.58

Noise MNIST SVHN

Tiny

L+ e Ld Ls U.P. ACCH ACCH ACCH ACCH ✓ ✓ ✓ ✓ ✓ ✓

CIFAR100-C

CIFAR10-C ACCH

48.99 49.28 49.36

43.49 43.50 43.34

46.21 46.29 46.21

46.22 46.43 46.26

46.51 46.61 46.44

57.06 63.19 54.10 65.16

54.58 52.57 51.39 54.93

53.31 47.05 52.22 47.22 50.85 46.36 53.77 48.22

47.46 47.56 46.45 48.61

Computational Cost Analysis

In this section, we evaluate the computational cost of each method. Since OOD detection is performed in a batch-wise manner, we set the batch size to 64 and measure the per-sample running time across all benchmarks with Noise OOD samples. The results are reported in Table 15. Under the same experimental settings, TEST and BN show the lowest computational overhead. The proposed ReNC method, while incurring approximately 2–3 times higher cost, achieves significantly better performance. Compared with other OWTTA-specific methods, ReNC demonstrates competitive efficiency relative to UniEnt (0.003270 vs. 0.003955) while achieving superior results. Notably, ReNC runs substantially faster than OWT3 on the ImageNet-C dataset (0.003270 vs. 0.006559). These results suggest that ReNC achieves a favorable balance between computational cost and performance.

21

Table 14: Ablation study on the ImageNet-C, ImageNet-R and VisDA-C with OOD samples. Best results marked in bold, second best underlined.

ImageNet-C

Noise

MNIST

SVHN

Ld

ACCH

ACCH

ACCH

33.11 33.25 32.27

34.96 35.00 35.33

36.56 36.47 36.44

52.84 51.67 51.64 53.94

52.10 51.23 51.17 52.35

52.45 51.53 51.23 53.41

ImageNet-R

Noise

MNIST

SVHN

L+ e

Ld

ACCH

ACCH

ACCH

✓ ✓ ✓

✓ ✓

✓

42.54 42.62 42.74

39.69 39.69 39.66

36.64 36.64 36.58

✓ ✓

✓ ✓ ✓ ✓

56.45 55.55 55.54 56.77

54.16 52.51 52.49 54.65

55.26 54.55 54.59 55.78

Noise

MNIST

SVHN

ACCH

ACCH

ACCH

47.07 47.08 47.32

52.81 52.83 52.95

52.94 52.90 52.85

70.39 69.81 69.72 71.67

63.22 62.68 62.31 64.16

68.41 67.86 67.83 69.63

L+ e ✓ ✓ ✓

✓ ✓

✓

✓ ✓

✓ ✓ ✓ ✓

✓ ✓

✓ ✓

Ls

Ls

U.P.

✓ ✓ ✓ ✓

U.P.

✓ ✓ ✓ ✓

VisDA-C L+ e

Ld

✓ ✓ ✓

✓ ✓

✓

✓ ✓

✓ ✓ ✓ ✓

✓ ✓

Ls

U.P.

✓ ✓ ✓ ✓

Table 15: Per-sample running time (in seconds) across all datasets, measured with Noise OOD samples. Running Time CIFAR10-C CIFAR100-C ImageNet-C ImageNet-R VisDA-C TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

0.000623 0.000632 0.003121 0.002406 0.001202 0.001726 0.002328 0.005638 0.001898 0.003217 0.001978

0.000626 0.000658 0.003208 0.002496 0.001230 0.001830 0.002382 0.004163 0.001905 0.003776 0.001872

22

0.001139 0.001163 0.005682 0.004015 0.002643 0.002824 0.003816 0.036937 0.003955 0.006559 0.003270

0.001188 0.001242 0.005821 0.003910 0.002656 0.002860 0.003570 0.009987 0.004019 0.006480 0.003493

0.001165 0.001203 0.005640 0.003733 0.003850 0.003195 0.003591 0.019757 0.003683 0.005694 0.004059

4.8

Parameter Analysis

This section explores the model stability with respect to hyper-parameters. Given that identifying the optimal balance parameter λ is still an unresolved issue in unsupervised learning, we adopt a parameter-tuning approach. Figure 6 reports experiments with varying λ ∈ {1e − 3, 1e − 5, 1e − 2, 5e − 2, 1e − 1, 5e − 5} using the ACCH metric. We observe that the performance remains stable across different parameter values on almost all the open-world benchmarks, with only minor variations. For the VisDA-C dataset, the proposed method also shows improved performance when λ is larger than 5e − 3 for different OOD samples. In brief, the proposed method is stable under different settings of λ, indicating its robustness.

(a) CIFAR10-C

(b) CIFAR100-C

(c) ImageNet-C

(d) ImageNet-R

(e) VisDA-C

Figure 6: Parameter analysis of the optimal balance parameter λ in Eq. (10).

4.9

Model Performance across Different OOD Sample Ratios

This section validates the robustness of the proposed method against varying numbers of OOD samples. Since the ratio between ID and OOD samples can vary in real-world scenarios, we examine the impact of |S | different OOD sample ratios. Specifically, we control the OOD sample ratio |STTO| from 0.2 to 1.0 across all I the open-world benchmarks, evaluating their ACCH values. The results, presented in Figure 7, show that the proposed method maintains consistent performance across different OOD sample ratios, demonstrating its applicability to a variety of data ratio scenarios.

(a) CIFAR10-C

(b) CIFAR100-C

(c) ImageNet-C

(d) ImageNet-R

(e) VisDA-C

Figure 7: Open-world test-time adaptation results of ReNC at different OOD sample ratios.

4.10

Evaluation of Effectiveness on Continual OWTTA

In this section, we evaluate the effectiveness of ReNC under the continual OWTTA setting, where data distribution shifts occur and OOD samples continuously change during the adaptation process. Specifically, CIFAR10-C is used as the ID dataset. For the OOD samples, we introduce the Noise, MNIST, SVHN, Tiny ImageNet, and CIFAR100-C datasets, all presented sequentially in an online manner. The comparison results against all baseline methods are presented in Table 16, using ACCH as the evaluation metric. As shown in the table, the proposed ReNC achieves the highest average performance on the continual CIFAR10C open-world benchmark, demonstrating its strong adaptability and effectiveness in handling the continual OWTTA scenario.

23

Table 16: Continual OWTTA results on CIFAR10-C with OOD samples. Best marked in bold, second best underlined. t −−−−−−−−−−−−−−−−−−−−−−−−−→

CIFAR10-C Noise MNIST SVHN Tiny CIFAR100-C Average TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

4.11

81.78 47.77 50.60 50.15 21.80 49.76 44.90 47.03 47.48 81.93 88.35

75.10 64.19 67.37 67.74 43.70 64.36 88.10 59.90 63.99 75.18 86.60

74.85 74.18 76.44 77.16 46.89 74.09 82.39 73.79 73.94 74.94 88.62

68.38 71.72 72.75 72.97 48.22 71.69 76.32 71.41 71.77 68.45 75.63

68.07 71.32 71.76 71.89 47.47 71.34 76.32 71.01 71.28 68.08 72.11

73.64 65.83 67.78 67.98 41.62 66.25 73.61 64.63 65.69 73.72 82.26

Effectiveness with ViT backbone

In this section, we present additional experiments using a ViT [74] backbone. Specifically, we first pretrain a ViT model on the CIFAR10 training set as the source domain model, and then perform OWTTA experiments on the CIFAR10-C dataset with OOD samples. Table 17 summarizes the results in terms of ACCH . As shown, ReNC consistently outperforms all baseline methods, demonstrating its robust generalization capability across different backbone architectures. Table 17: OWTTA results under ViT backbone, evaluated by ACCH . Best marked in bold, second best underlined. CIFAR10-C Noise MNIST SVHN Tiny CIFAR100-C TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 Ours

4.12

92.99 91.81 89.15 93.61 37.21 90.26 90.49 93.04 93.46 93.67 94.88

91.04 90.57 92.47 92.37 44.75 89.55 90.03 91.81 90.12 92.64 93.40

90.40 87.89 90.89 90.36 49.78 88.94 89.32 89.90 85.71 90.94 91.61

86.16 84.80 86.23 86.21 50.36 85.55 85.85 83.58 83.58 86.21 87.06

81.60 80.34 82.12 82.11 50.60 81.12 81.86 81.66 79.65 81.92 82.72

Vision Language Models in OWTTA

In this section, we explore the applicability of Vision-Language Models (VLMs) [38] to OWTTA. While VLMs are well known for their strong zero-shot capabilities, they lack native OOD detection mechanisms and thus cannot be directly applied to the OWTTA setting. To address this limitation, we extend C-TPT [39], a CLIP-based test-time adaptation method, by incorporating a pre-trained ResNet-50 as the OOD detection module, resulting in C-TPT+. As shown in Table 18, C-TPT+ ranks second on the ImageNet-R open-world benchmark, demonstrating its strong generalization ability. This result highlights the potential of VLM-based approaches in OWTTA. Notably, our ReNC is specifically designed for this task, with tailored components that support more effective 24

Table 18: Comparison of vision-language model extensions on ImageNet-R with OOD samples, evaluated by ACCH . Best marked in bold, second best underlined. ImageNet-R

Noise

MNIST

SVHN

TEST BN TENT SHOT OSTTA EATA RMT CoTTA UniEnt OWT3 C-TPT+ Ours

39.63 27.70 28.57 29.28 35.50 27.55 16.02 26.35 18.43 28.07 45.39 56.77

38.17 30.52 32.11 31.72 30.93 31.07 21.70 28.98 25.26 31.15 42.40 54.65

36.61 31.17 32.71 33.29 35.58 31.21 27.02 28.92 26.64 31.75 40.38 55.78

learning and adaptation. As a result, it achieves the best overall performance. These findings suggest that while VLMs offer impressive zero-shot and generalization capabilities, they still require task-specific adaptation to perform reliably in specialized scenarios like OWTTA.

4.13

Friedman Test for Significant Test

In this section, we conduct a significant test to evaluate the performance variations among the eight methods. Specifically, we utilize the Friedman test [75] under the assumption of equal performance across all methods as the null hypothesis. Pairwise comparisons are subsequently carried out using the Nemenyi post-hoc test [76]. The dataset for this evaluation consists of 627 instances, with each method independently assessed across nineteen datasets using three evaluation metrics. The Friedman test yields a test statistic of τF = 18.266, which surpasses the critical value of F10,560 = 1.847 at a significance level of α = 0.05. Thus, the null hypothesis is rejected, confirming that there are statistically significant differences among the eight methods at a significance level of 0.05. Further examination via the Nemenyi post-hoc test (See Figure 8) shows that our ReNC method demonstrates a clear performance advantage over the seven baseline methods.

Figure 8: Nemenyi post-hoc analysis of all the baselines.

25

5

Conclusion

This paper introduces Reliable Neural Collapse approximation (ReNC), a method designed to address OWTTA reliably and effectively. We justify that pre-trained classifier weights can be viewed as prototypes within the context of neural collapse and develop a reliable update mechanism to filter out Out-OfDistribution (OOD) samples during adaptation. Furthermore, we propose a neural collapse approximation mechanism for updating prototypes, enabling the model to gradually approach the neural collapse state in the target domain while maintaining the balanced structure. Our empirical results validate the effectiveness of ReNC and suggest that NC-related properties may provide useful evidence for understanding its improved classification performance. In future work, we will explore the potential of foundation models for the OWTTA.

References [1] S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira, “Analysis of representations for domain adaptation,” in NeurIPS, 2006, pp. 137–144. [2] S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan, “A theory of learning from different domains,” Mach. Learn., vol. 79, no. 1-2, pp. 151–175, 2010. [3] K. You, M. Long, Z. Cao, J. Wang, and M. I. Jordan, “Universal domain adaptation,” in CVPR, 2019, pp. 2720–2729. [4] M. Boudiaf, R. Müller, I. B. Ayed, and L. Bertinetto, “Parameter-free online test-time adaptation,” in CVPR, 2022, pp. 8334–8343. [5] S. Niu, J. Wu, Y. Zhang, Y. Chen, S. Zheng, P. Zhao, and M. Tan, “Efficient test-time model adaptation without forgetting,” in ICML, vol. 162, 2022, pp. 16 888–16 905. [6] Z. Chi, Y. Wang, Y. Yu, and J. Tang, “Test-time fast adaptation for dynamic scene deblurring via meta-auxiliary learning,” in CVPR, 2021, pp. 9137–9146. [7] S. Goyal, M. Sun, A. Raghunathan, and J. Z. Kolter, “Test time adaptation via conjugate pseudolabels,” in NeurIPS, 2022. [8] Y. Li, X. Xu, Y. Su, and K. Jia, “On the robustness of open-world test-time training: Self-training with dynamic prototype expansion,” in ICCV, 2023, pp. 11 802–11 812. [9] M. Yuan, Y. Xia, H. Dong, Z. Chen, J. Yao, M. Qiu, K. Yan, X. Yin, Y. Shi, X. Chen, Z. Liu, B. Dong, J. Zhou, L. Lu, L. Zhang, and L. Zhang, “Devil is in the queries: Advancing mask transformers for real-world medical image segmentation and out-of-distribution localization,” in CVPR, 2023, pp. 23 879–23 889. [10] J. Lee, D. Das, J. Choo, and S. Choi, “Towards open-set test-time adaptation utilizing the wisdom of crowds in entropy minimization,” in ICCV, 2023, p. 16334. [11] Z. Gao, X.-Y. Zhang, and C.-L. Liu, “Unified entropy optimization for open-set test-time adaptation,” in CVPR, 2024, pp. 23 975–23 984. [12] J. Liang, D. Hu, and J. Feng, “Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation,” in ICML, vol. 119, 2020, pp. 6028–6039. [13] J. N. Kundu, N. Venkat, R. M. V., and R. V. Babu, “Universal source-free domain adaptation,” in CVPR, 2020, pp. 4543–4552. 26

[14] S. Liu, D. Zhang, and X. Hao, “Efficient deformable convolutional prompt for continual test-time adaptation in medical image segmentation,” in AAAI, 2025, pp. 5550–5557. [15] R. Li, Q. Jiao, W. Cao, H. Wong, and S. Wu, “Model adaptation: Unsupervised domain adaptation without source data,” in CVPR, 2020, pp. 9638–9647. [16] Z. Feng, C. Xu, and D. Tao, “Open-set hypothesis transfer with semantic consistency,” IEEE Trans. Image Process., vol. 30, pp. 6473–6484, 2021. [17] J. Zhou, C. You, X. Li, K. Liu, S. Liu, Q. Qu, and Z. Zhu, “Are all losses created equal: A neural collapse perspective,” in NeurIPS, 2022. [18] Y.-W. Luo, C.-X. Ren, X.-L. Xu, and Q. Liu, “Geometric understanding of discriminability and transferability for visual domain adaptation,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–16, 2024. [19] F. Huang, S. Song, and L. Zhang, “Gradient harmonization in unsupervised domain adaptation,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–17, 2024. [20] Y. Wu, Z. Chi, Y. Wang, K. N. Plataniotis, and S. Feng, “Test-time domain adaptation by learning domain-aware batch normalization,” in AAAI, 2024, pp. 15 961–15 969. [21] Y. Su, X. Xu, and K. Jia, “Towards real-world test-time adaptation: Tri-net self-training with balanced normalization,” in AAAI, 2024, pp. 15 126–15 135. [22] Y. Gandelsman, Y. Sun, X. Chen, and A. A. Efros, “Test-time training with masked autoencoders,” in NeurIPS, 2022. [23] D. Wang, E. Shelhamer, S. Liu, B. A. Olshausen, and T. Darrell, “Tent: Fully test-time adaptation by entropy minimization,” in ICLR, 2021. [24] Y. Su, X. Xu, and K. Jia, “Revisiting realistic test-time training: Sequential inference and adaptation by anchored clustering,” in NeurIPS, 2022. [25] T. Kojima, Y. Matsuo, and Y. Iwasawa, “Robustifying vision transformer without retraining from scratch by test-time class-conditional feature alignment,” in IJCAI, 2022, pp. 1009–1016. [26] M. J. Mirza, P. Jané-Soneira, W. Lin, M. Kozinski, H. Possegger, and H. Bischof, “Actmad: Activation matching to align distributions for test-time-training,” in CVPR, 2023, pp. 24 152–24 161. [27] Y. Liu, P. Kothari, B. van Delft, B. Bellot-Gurlet, T. Mordan, and A. Alahi, “TTT++: when does self-supervised test-time training fail or thrive?” in NeurIPS, 2021, pp. 21 808–21 820. [28] Y. Su, X. Xu, T. Li, and K. Jia, “Revisiting realistic test-time training: Sequential inference and adaptation by anchored clustering regularized self-training,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 8, pp. 5524–5540, 2024. [29] N. Hansen, R. Jangir, Y. Sun, G. Alenyà, P. Abbeel, A. A. Efros, L. Pinto, and X. Wang, “Selfsupervised policy adaptation during deployment,” in ICLR, 2021. [30] D. Chen, D. Wang, T. Darrell, and S. Ebrahimi, “Contrastive test-time adaptation,” in CVPR, 2022, pp. 295–305. [31] Y. Sun, X. Wang, Z. Liu, J. Miller, A. A. Efros, and M. Hardt, “Test-time training with self-supervision for generalization under distribution shifts,” in ICML, vol. 119, 2020, pp. 9229–9248.

27

[32] Q. Wang, O. Fink, L. V. Gool, and D. Dai, “Continual test-time domain adaptation,” in CVPR, 2022, pp. 7191–7201. [33] S. Niu, J. Wu, Y. Zhang, Z. Wen, Y. Chen, P. Zhao, and M. Tan, “Towards stable test-time adaptation in dynamic wild world,” in ICLR, 2023. [34] T. Gong, J. Jeong, T. Kim, Y. Kim, J. Shin, and S. Lee, “NOTE: robust continual test-time adaptation against temporal correlation,” in NeurIPS, 2022. [35] M. Döbler, R. A. Marsden, and B. Yang, “Robust mean teacher for continual and gradual test-time adaptation,” in CVPR, 2023, pp. 7704–7714. [36] Z. Zhou, L. Guo, L. Jia, D. Zhang, and Y. Li, “ODS: test-time adaptation in the presence of open-world data shift,” in ICML, vol. 202, 2023, pp. 42 574–42 588. [37] Z. Zhou, K. Yu, L. Guo, and Y. Li, “Fully test-time adaptation for tabular data,” in AAAI, 2025, pp. 23 027–23 035. [38] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” in ICML, vol. 139, 2021, pp. 8748–8763. [39] H. S. Yoon, E. Yoon, J. T. J. Tee, M. A. Hasegawa-Johnson, Y. Li, and C. D. Yoo, “C-TPT: calibrated test-time prompt tuning for vision-language models via text feature dispersion,” in ICLR, 2024. [40] Y. Qian, Y. Bai, Z. Zhang, P. Zhao, and Z. Zhou, “Handling new class in online label shift,” in ICDM, 2023, pp. 1283–1288. [41] D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” in ICLR, 2017. [42] W. Liu, X. Wang, J. D. Owens, and Y. Li, “Energy-based out-of-distribution detection,” in NeurIPS, 2020. [43] Y. Sun, C. Guo, and Y. Li, “React: Out-of-distribution detection with rectified activations,” in NeurIPS, 2021, pp. 144–157. [44] Y. Sun, Y. Ming, X. Zhu, and Y. Li, “Out-of-distribution detection with deep nearest neighbors,” in ICML, vol. 162, 2022, pp. 20 827–20 840. [45] C. S. Sastry and S. Oore, “Detecting out-of-distribution examples with gram matrices,” in ICML, vol. 119, 2020, pp. 8491–8501. [46] H. Wang, Z. Li, L. Feng, and W. Zhang, “Vim: Out-of-distribution with virtual-logit matching,” in CVPR, 2022, pp. 4911–4920. [47] R. Huang, A. Geng, and Y. Li, “On the importance of gradients for detecting distributional shifts in the wild,” in NeurIPS, 2021, pp. 677–689. [48] C. Igoe, Y. Chung, I. Char, and J. Schneider, “How useful are gradients for OOD detection really?” CoRR, vol. abs/2205.10439, 2022. [49] Y. Li and N. Vasconcelos, “Background data resampling for outlier-aware classification,” in CVPR, 2020, pp. 13 215–13 224.

28

[50] Y. Ming, Y. Fan, and Y. Li, “POEM: out-of-distribution detection with posterior sampling,” in ICML, vol. 162, 2022, pp. 15 650–15 665. [51] S. Vernekar, A. Gaurav, V. Abdelzad, T. Denouden, R. Salay, and K. Czarnecki, “Out-of-distribution detection in classifiers via generation,” CoRR, vol. abs/1910.04241, 2019. [52] K. Lee, H. Lee, K. Lee, and J. Shin, “Training confidence-calibrated classifiers for detecting out-ofdistribution samples,” in ICLR, 2018. [53] X. Du, Z. Wang, M. Cai, and Y. Li, “VOS: learning what you don’t know by virtual outlier synthesis,” in ICLR, 2022. [54] W. Chang, Y. Shi, H. Tuan, and J. Wang, “Unified optimal transport framework for universal domain adaptation,” in NeurIPS, 2022. [55] G. Li, G. Kang, Y. Zhu, Y. Wei, and Y. Yang, “Domain consensus clustering for universal domain adaptation,” in CVPR, 2021, pp. 9757–9766. [56] K. Saito, D. Kim, S. Sclaroff, and K. Saenko, “Universal domain adaptation through self supervision,” in NeurIPS, 2020. [57] S. Qu, T. Zou, F. Röhrbein, C. Lu, G. Chen, D. Tao, and C. Jiang, “Upcycling models under domain and category shift,” in CVPR, 2023, pp. 20 019–20 028. [58] J. Liang, D. Hu, J. Feng, and R. He, “UMAD: universal model adaptation under domain and category shift,” CoRR, vol. abs/2112.08553, 2021. [59] S. Qu, T. Zou, L. He, F. Röhrbein, A. Knoll, G. Chen, and C. Jiang, “LEAD: learning decomposition for source-free universal domain adaptation,” in CVPR, 2024, pp. 23 334–23 343. [60] D. Shanmugam, D. W. Blalock, G. Balakrishnan, and J. V. Guttag, “Better aggregation in test-time augmentation,” in ICCV, 2021, pp. 1194–1203. [61] I. Kim, Y. Kim, and S. Kim, “Learning loss for test-time augmentation,” in NeurIPS, 2020. [62] A. Jelea, A. N. Belbachir, and M. Leordeanu, “Learning from random subspace exploration: Generalized test-time augmentation with self-supervised distillation,” CoRR, 2025. [63] M. I. Belghazi, A. Baratin, S. Rajeswar, S. Ozair, Y. Bengio, R. D. Hjelm, and A. C. Courville, “Mutual information neural estimation,” in ICML, vol. 80, 2018, pp. 530–539. [64] J. Snell, K. Swersky, and R. S. Zemel, “Prototypical networks for few-shot learning,” in NeurIPS, 2017, pp. 4077–4087. [65] Z. Zhu, T. Ding, J. Zhou, X. Li, C. You, J. Sulam, and Q. Qu, “A geometric analysis of neural collapse with unconstrained features,” in NeurIPS, 2021, pp. 29 820–29 834. [66] S. Xie, Z. Zheng, L. Chen, and C. Chen, “Learning semantic representations for unsupervised domain adaptation,” in ICML, vol. 80, 2018, pp. 5419–5428. [67] D. Hendrycks and T. G. Dietterich, “Benchmarking neural network robustness to common corruptions and perturbations,” in ICLR, 2019. [68] D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, D. Song, J. Steinhardt, and J. Gilmer, “The many faces of robustness: A critical analysis of out-of-distribution generalization,” in ICCV, 2021, pp. 8320–8329. 29

[69] X. Peng, B. Usman, N. Kaushik, J. Hoffman, D. Wang, and K. Saenko, “Visda: The visual domain adaptation challenge,” CoRR, 2017. [70] L. Deng, “The MNIST database of handwritten digit images for machine learning research [best of the web],” IEEE Signal Process. Mag., vol. 29, no. 6, pp. 141–142, 2012. [71] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, A. Y. Ng et al., “Reading digits in natural images with unsupervised feature learning,” in NIPS workshop on deep learning and unsupervised feature learning, vol. 2011, no. 5, 2011, p. 7. [72] Y. Le and X. Yang, “Tiny imagenet visual recognition challenge,” CS 231N, vol. 7, no. 7, p. 3, 2015. [73] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in ICML, vol. 37, 2015, pp. 448–456. [74] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in ICLR, 2021. [75] J. Demsar, “Statistical comparisons of classifiers over multiple data sets,” J. Mach. Learn. Res., vol. 7, pp. 1–30, 2006. [76] P. B. Nemenyi, Distribution-free multiple comparisons.

30

Princeton University, 1963.

Related documents

• The Mechanics of Delta Learning: Target Design for Generalizable Scientific Machine Learning#1076438
• The miracle dilemma: demystifying a common challenge in neuroprognostication.#360713
• Recovery-First Metacognitive Science: A Preregisterable Measurement-Standard and Claim-Licensing Protocol#74729
• Deep neural collapse is provably optimal for the deep unconstrained features model#384960
• Investigating the Impact of Image Reconstruction Pretraining on Adversarial Robustness#442198