VAE-Inf: A statistically interpretable generative paradigm for imbalanced classification∗ Hongfei Wu
Department of Data Science and Artificial Intelligence The Hong Kong Polytechnic University
arXiv:2604.25334v1 [cs.LG] 28 Apr 2026
Ruijian Han
Department of Data Science and Artificial Intelligence The Hong Kong Polytechnic University
Yancheng Yuan
Department of Applied Mathematics The Hong Kong Polytechnic University
Abstract Imbalanced classification remains a pervasive challenge in machine learning, particularly when minority samples are too scarce to provide a robust discriminative boundary. In such extreme scenarios, conventional models often suffer from unstable decision boundaries and a lack of reliable error control. To bridge the gap between generative modeling and discriminative classification, we propose a two-stage framework VAE-Inf that integrates deep representation learning with statistically interpretable hypothesis testing. In the first stage, we adopt a one-class modeling perspective by training a variational autoencoder (VAE) exclusively on majority-class data to capture the underlying reference distribution. The resulting latent posteriors are aggregated via a Wasserstein barycenter to construct a global Gaussian reference model, providing a geometrically principled baseline for the majority class. In the second stage, we transform this generative foundation into a discriminative classifier by fine-tuning the encoder with limited minority samples. This is achieved through a novel distribution-aware loss that enforces probabilistic separation between classes based on variance-normalized projection statistics. For inference, we introduce a projection-based score that admits a natural hypothesis testing interpretation, allowing for a distributionfree calibration procedure. This approach yields exact finite-sample control of the Type-I error (false positive rate) without relying on restrictive parametric assumptions. Extensive experiments on diverse real-world benchmarks demonstrate that our framework achieves competitive performance against other approaches. The codes are available upon request.
1
Introduction
Imbalanced classification is a fundamental challenge in machine learning and statistics, where the majority class contains far more samples than the minority class. This phenomenon commonly arises in real-world applications such as pattern recognition (Dong et al., 2019), object detection (Oksuz et al., 2021) and medical diagnosis (Guo et al., 2017). In these domains, correct identification of minority samples is essential, while misclassifying them can be costly or even harmful. For instance, in disease screening, missing even a few true positive cases may lead to severe medical consequences or even loss of life. However, standard learning methods typically focus on maximizing overall accuracy (Pei et al., 2024) and assume equal importance among all samples, which causes the learned model to be dominated by the majority class and perform poorly in rare but crucial cases. ∗ Corresponding authors: Ruijian Han and Yancheng Yuan.
1
Many strategies have been developed to address class imbalance in traditional learning frameworks. Existing approaches can generally be categorized into three groups: (1) Resampling methods focus on resampling the training data to rebalance class proportions (He & Garcia, 2009). Typical approaches include random under-sampling of the majority class (Tahir et al., 2009) and over-sampling of the minority class, as well as synthetic data generation techniques such as the Synthetic Minority Over-sampling Technique (SMOTE) and its variants (Chawla et al., 2002; Zheng et al., 2016). (2) Cost-sensitive learning methods (Elkan, 2001; Thai-Nghe et al., 2010; Castro & Braga, 2013) modify the loss function to impose higher penalties on misclassified minority samples. This allows the model to pay greater attention to rare examples. (3) Ensemble learning methods combine multiple classifiers or resampled subsets to improve performance (Galar et al., 2012). Representative ensemble-based algorithms include AdaBoost (Freund & Schapire, 1996), SMOTEBoost (Chawla et al., 2003), and Bagging (Galar et al., 2013), which integrate sampling or reweighting strategies within the ensemble framework. However, these traditional approaches may struggle to effectively capture complex structures in high-dimensional data (Huang et al., 2025). Recently, deep learning has achieved remarkable success due to its exceptional learning capacity, and a growing number of methods have applied deep architectures to solve imbalance classification problems. In particular, deep generative models are integrated to offer high-quality synthetic samples for the minority class (Mullick et al., 2019; Wang et al., 2020). Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) have been widely used as oversampling tools for image data (Sampath et al., 2021), with variants such as BAGAN (Mariani et al., 2018) and CGAN (Nazari & Branco, 2021) designed to improve generation quality. Similarly, Variational Autoencoders (VAEs) (Kingma & Welling, 2013) have also been adopted to generate minority samples in a probabilistic latent space (Wan et al., 2017; Zhang et al., 2018). Moreover, new loss functions, such as Focal loss (Lin et al., 2017) and Label-Distribution-Awareness margin (LDAM) (Cao et al., 2019), have been proposed to replace the commonly used Cross Entropy (CE) loss to improve discrimination. Specifically, the Focal loss reduces the weight of well-classified samples to focus training on difficult instances, while the LDAM loss explicitly adjusts the classification margin according to the frequency of the class. Despite the progress of deep learning-based approaches, extreme class imbalance remains a fundamental challenge: the minority class is often too sparse to faithfully characterize its underlying distribution. While data-level generation can alleviate sample scarcity, its effectiveness is strictly capped by the fidelity of synthetic examples due to the very limited available training samples of the minority class. For instance, GAN-based oversampling frequently suffers from mode collapse (Arjovsky et al., 2017), while existing latentspace models often rely on restrictive symmetric assumptions (Guo et al., 2019) that fail to capture the complex, asymmetric feature structures of real-world data. These limitations underscore a critical insight: in extreme scenarios, defining what the minority class “is” is significantly harder than defining what it “is not”. Building on this insight, we reframe extreme imbalanced classification through the lens of statistical inference modeling, drawing inspiration from anomaly detection (Ruff et al., 2018; Pang et al., 2019; Ruff et al., 2020; Zhou et al., 2022). Instead of constructing fragile models for a sparse minority class, we focus on establishing a robust profile of the majority class (normality) and identifying minority samples as significant deviations from this regularity. In particular, we propose a two-stage framework VAE-Inf that integrates deep representation learning with statistical inference. In Stage 1, a VAE is pretrained exclusively on majority-class data to learn a continuous probabilistic latent manifold. In Stage 2, the encoder is fine-tuned using limited minority samples through a distribution-aware objective, which enhances class separability while preserving the learned majority structure. This design enables a statistically interpretable inference strategy, allowing for decision rules with controllable error behavior and finite-sample guarantees. Our numerical results demonstrate that even a small amount of labeled minority information can be utilized to improve the detection performance of the minority class, making it particularly suitable for imbalanced tasks. The main contributions of this paper are summarized as follows:
• We propose a two-stage VAE-based framework VAE-Inf that constructs a reference distribution of the majority class and effectively utilizes limited minority samples through a distribution-aware fine-tuning process. 2
• We develop a projection-score calibration mechanism that provides finite-sample Type-I error control under exchangeability, while empirically achieving competitive Type-II trade-offs. • Extensive experiment results on several important datasets demonstrate that our VAE-Inf achieves superior classification performance while maintaining reliable Type-I/II error control compared with imbalanced learning and anomaly detection baselines. The remainder of this paper is organized as follows. We start with the notation and problem formulation. Section 2 presents the proposed method. Section 3 reports the experimental setup and results, and Section 4 concludes the paper. 1.1
Notation and Problem Formulation
Formally, consider a binary classification problem with input–label pairs (x, y), where x ∈ X ⊆ Rp and y ∈ {1, 2}. Assign y = 1 to the majority class and y = 2 to the minority class. We formulate the problem from a hypothesis testing perspective. Given an observation x, we consider the hypotheses H0 : x ∼ P1
vs.
H1 : x ∼ P2 ,
(1)
where P1 and P2 denote the data distributions of the majority and minority classes, respectively. A decision rule ψ : X → {1, 2} assigns each Type-I and Type-II errors are defined sample to one of the two classes. as R1 (ψ) = Px∼P1 ψ(x) = 2 and R2 (ψ) = Px∼P2 ψ(x) = 1 . In general, R1 (ψ) and R2 (ψ) can not be minimized simultaneously, leading to an inherent trade-off between false positive and false negative rates. We adopt a single-error control perspective that prioritizes reliable regulation of one error type. Specifically, given a user-specified tolerance level δ ∈ (0, 1), our goal is to construct a decision rule ψ such that R1 (ψ) ≤ δ. This formulation emphasizes controlled statistical decision-making, rather than purely optimizing classification accuracy.
2
Methodologies
In this paper, we propose a novel framework VAE-Inf that addresses imbalanced classification by combining majority distribution modeling with statistically controlled decision-making. Our core idea is to model and learn the distribution of the majority class in latent space, and utilize it to perform error-aware classification with limited minority data. As illustrated in Figure 1, the framework consists of two stages. In Stage 1, a VAE is pretrained exclusively on the majority-class samples to learn a latent reference distribution. In Stage 2, the encoder is fine-tuned using a distribution-aware objective that promotes separation between majority and minority representations while preserving the learned reference structure. At inference time, we propose a projection-based statistic in the latent space, which serves as a decision score for hypothesis testing and enables controlled error trade-offs. 2.1
Stage 1: Pretraining VAE on the Majority Class
In the first stage, we pretrain a VAE exclusively on majority-class samples, aiming to learn a latent representation of the normal data manifold. This stage serves to construct a statistical reference that characterizes the majority distribution in latent space. d 1 VAE Training. Let D1 = {xi }N i=1 denote the set of majority-class samples with xi ∈ R . We employ a standard VAE with encoder qϕ (z|x) and decoder pθ (x|z), where the encoder defines a diagonal Gaussian posterior qϕ (z|x) = N z; µϕ (x), diag(σϕ2 (x)) . Latent variables are sampled via the reparameterization trick: z = µϕ (x) + σϕ (x) ⊙ ϵ, ϵ ∼ N (0, I), (2)
where ⊙ is the element-wise product. The model is trained by maximizing the evidence lower bound (ELBO) (Kingma & Welling, 2013): LVAE (x) = Ez∼qϕ (z|x) [log pθ (x|z)] − DKL (qϕ (z|x) ∥ p(z)) , 3
Stage 1. VAE Pretrain
Imbalanced Data
3. Inference
Calibration Set
Decoder
Encoder
Unknown
Reconstruction
Majority
Score Function
Global barycenter
Encoder
Calibration Threshold
Majority Sample Minority Sample
Majority
Encoder
Projection Vectors
Minority
update
Majority
Minority
≤ Penalty term for Minority
Penalty term for Majority
Rejection Region Rejection Region
<
Rejection Region Rejection Region
Acceptance Region
Acceptance Region
Stage 2. Fine-tuning with Distribution-aware Loss
Figure 1: The overview of the proposed two-stage VAE-Inf for imbalanced classification. (1) Stage 1 learns a latent reference distribution of the majority class in latent space using a VAE. (2) Stage 2 refines the representation via a distribution-aware objective to enhance separation between majority and minority samples. (3) At inference, a projection-based score is computed with respect to the learned reference distribution, enabling statistically principled decision-making with controlled error trade-offs. where p(z) = N (0, I) is the standard Gaussian prior. This training procedure yields a collection of posterior 1 distributions {qϕ (z|xi )}N i=1 , providing local probabilistic descriptions of majority samples in latent space. Estimates of Global Mean and Variance While individual posteriors qϕ (z|x) provide local descriptions of each sample, we aim to construct a global characterization of the majority class in latent space. To this 1 end, we aggregate the collection of Gaussian posteriors {N (z; µxi , diag(σϕ2 (xi )))}N i=1 into a single latent reference distribution. We define this reference as the 2-Wasserstein barycenter of the individual posteriors: N (z; µref , Σref ) = arg
min
N (z;µ,Σ)
N1 X
W22 N (z; µxi , diag(σϕ2 (xi ))), N (z; µ, Σ) ,
i=1
where µxi ∈ Rk and diag(σϕ2 (xi )) ∈ Rk×k denote the encoder-predicted posterior mean and corresponding diagonal covariance for each sample xi ∈ D1 . For two Gaussians N (z; µ1 , Σ1 ) and N (z; µ2 , Σ2 ) with diagonal covariances, the 2-Wasserstein distance reduces to 1/2
W22 (N (z; µ1 , Σ1 ), N (z; µ2 , Σ2 )) = ∥µ1 − µ2 ∥22 + ∥Σ1
1/2
− Σ2 ∥2F ,
where ∥ · ∥F is the Frobenius norm. Following Mallasto & Feragen (2017), the Wasserstein barycenter admits closed-form estimates for the mean and diagonal covariance: N
1 1 X µx , N1 i=1 i !2 N1 q X 1 σϕ2 (xi ) . Σref = diag N1 i=1
µref =
4
(3)
(4)
2.2
Stage 2: Fine-tuning with a Distribution-aware Statistical Margin
While Stage 1 provides a reference distribution for the majority class, the resulting latent space is not yet optimized for discrimination between the majority class and minority class. The objective of Stage 2 is therefore to fine-tune the encoder under a distribution-aware statistical criterion: we seek a representation in which majority samples remain compatible with the reference distribution learned in Stage 1, whereas minority samples are systematically pushed away from it. 2.2.1
Projection-based Statistics for Hypothesis Testing
It is challenging to conduct hypothesis testing in the latent space due to the high dimensionality of z ∈ Rk . To address this, we leverage the projection property of multivariate Gaussians to reduce the high-dimensional comparison problem to a family of one-dimensional statistical comparisons. Under the Stage 1 reference model, z ∼ N (µref , Σref ), for any deterministic direction a ∈ Rk , the projection a⊤ z is univariate Gaussian: a⊤ z ∼ N (a⊤ µref , a⊤ Σref a). To quantify the deviation of a latent sample from the majority reference along direction a, we define the squared projected deviation 2 d(z, a) := a⊤ z − a⊤ µref . Normalizing this quantity by the projected variance yields the projection statistic a⊤ z − a⊤ µref d(z, a) = T (z, a) = ⊤ a Σref a a⊤ Σref a
2 .
(5)
When z ∼ N (µref , Σref ) and a is fixed, the standardized projection satisfies a⊤ z − a⊤ µref p ∼ N (0, 1). a⊤ Σref a Therefore, T (z, a) provides a direction-wise test statistic for assessing whether a latent representation is statistically consistent with the majority reference distribution. 2.2.2
Distribution-aware Statistical Margin Loss
Let x(1) ∈ D1 be a sample from the majority class and x(2) ∈ D2 be a sample from the minority class. Their latent representations are sampled from the pretrained encoder posteriors via the reparameterization trick: z (i) = µϕ (x(i) ) + σϕ (x(i) ) ⊙ ϵ,
ϵ ∼ N (0, I),
i = 1, 2.
Under the Gaussian reference model, T (z, a) follows a chi-square(χ2 ) distribution with one degree of freedom. Therefore, for a prescribed threshold α > 0, the event T (z, a) ≤ α defines a direction-wise acceptance region with respect to the majority reference distribution. Based on the hypotheses in Eq. (1), this yields a natural testing interpretation: if T (z, a) lies in the acceptance region, H0 is not rejected; otherwise, H0 is rejected. For a fixed projection direction a, the desired class separation can be expressed as T (z (1) , a) ≤ α,
T (z (2) , a) > α,
(6)
where α serves as a direction-wise statistical margin. Moreover, α admits a direct probabilistic interpretation: choosing α = c2 corresponds to a c-sigma tolerance in the standardized projected space. For example, the critical region T (z, a) > 9 corresponds to the 3σ rule, which rejects H0 with significance level ≈ 0.27%. Since T (z, a) is defined as Eq. (5), Eq. (6) can be equivalently written as d(z (1) , a) ≤ α (a⊤ Σref a),
d(z (2) , a) > α (a⊤ Σref a). 5
Rather than relying on a single global discrepancy such as the Mahalanobis distance, we sample random directions a ∼ U(Sk−1 ). This allows the model to detect departures from the majority manifold across multiple latent-space orientations and provides diverse gradient signals during training, thereby discouraging overfitting to a single global metric. We therefore design the following distribution-aware regularization loss: h i Lreg = Ex(1) ∼D1 , x(2) ∼D2 Ea∼U (Sk−1 ) d(z (1) , a) − α(a⊤ Σref a) + + β α(a⊤ Σref a) − d(z (2) , a) + , (7) where (x)+ := max(0, x) for any x ∈ R, and β > 0 controls the relative strength of the minority penalty. The first term is a majority-tightening term, which penalizes majority samples whose projected deviations exceed the prescribed high-probability region of the reference model. The second term is a minority-pushing term, which penalizes minority samples that remain too close to the majority reference and therefore fail to enter the rejection region. 2.3
Inference with Distribution-Free Calibration
The final component of VAE-Inf is a distribution-free calibration procedure that converts latent anomaly scores into statistically valid decision rules. While Stage 2 uses projection-based statistical structure to shape the latent space, the resulting scores are not automatically calibrated for decision-making. We employ an empirical calibration scheme that provides finite-sample control of the Type-I error under exchangeability, without requiring parametric assumptions on the score distribution. In short, exchangeability means that the joint distribution is invariant under permutations of the random variables (Chow & Teicher, 2003). Let D = D1 ∪ D2 denote the full dataset, where D1 and D2 correspond to the majority and minority classes, respectively. D1 and D2 are further partitioned as D1 = D1,train ∪ D1,val ∪ D1,test , D2 = D2,train ∪ D2,val ∪ D2,test . The majority validation split D1,val is used as a held-out calibration set for threshold selection, and we denote ncal := |D1,val |. Given an input x, the encoder defines qϕ (z | x) = N z; µϕ (x), diag(σϕ2 (x)) . We obtain a latent code z by sampling from this posterior via the reparameterization trick in Eq. (2). The statistic T (z, a) in Eq. (5) measures deviation from the majority reference along the projection direction a. To obtain a more stable anomaly score, we aggregate over a set of directions A = {a1 , . . . , aM } sampled uniformly from the unit sphere, we define S(x) =
1 X T (z, a), M
(8)
a∈A
where larger values indicate stronger deviation from the majority distribution. In practice, the projection set A is fixed after sampling. We then formulate classification as the one-sided hypothesis test in Eq. (1) using S(x) as the test value. To control the Type-I error at a prescribed level δ ∈ (0, 1), we calibrate the decision threshold using the empirical distribution of calibration scores from the majority class. Specifically, for each xi ∈ D1,val , we compute Si = S(xi ), and denote the sorted scores by S(1) ≤ · · · ≤ S(ncal ) . The threshold is chosen as the empirical upper quantile, defined by ( S(k) , k ≤ ncal , k = ⌈(1 − δ)(ncal + 1)⌉ , τδ = (9) +∞, k = ncal + 1. Under the sole assumption that the calibration scores and the score of any future majority sample are exchangeable (Vovk et al., 2005; Lei et al., 2018; Romano et al., 2019). this threshold yields the following finitesample marginal guarantee. This result is formalized in Theorem 1, with proof deferred to Appendix A.2. Theorem 1 (Finite-Sample Type-I Error Control) Let S1 , . . . , Sncal be the scores of the majority-class calibration samples, and let Sncal +1 be the score of a future majority sample. Assume that the scores are exchangeable, i.e., the joint distribution of (S1 , . . . , Sncal , Sncal +1 ) is invariant under permutations. Equivalently, for every permutation π of {1, . . . , ncal + 1}, d
(S1 , . . . , Sncal , Sncal +1 ) = (Sπ(1) , . . . , Sπ(ncal +1) ). 6
Let τδ be the empirical upper quantile defined in Eq. (9) from the calibration scores. Then P Sncal +1 ≤ τδ ≥ 1 − δ. Theorem 1 guarantees that a future majority score is accepted with probability at least 1 − δ. Based on the calibrated threshold τδ , we define the final prediction rule for a test sample x by ( 1, S(x) ≤ τδ , ψ(x) = 2, S(x) > τδ , where class 1 denotes the majority class and class 2 denotes the minority class. For a future majority sample x ∼ P1 , the Type-I error occurs if and only if S(x) > τδ . Therefore, R1 (ψ) = Px∼P1 ψ(x) = 2 = Px∼P1 S(x) > τδ = 1 − Px∼P1 S(x) ≤ τδ ≤ δ, where the last inequality follows from Theorem 1. Hence, the resulting classifier is explicitly calibrated to control the Type-I error at level δ, yielding a transparent and statistically interpretable decision mechanism. In particular, this error-control guarantee is distribution-free and does not depend on the particular modeling choices used to construct the latent score. As a result, the significance level δ serves as a reliable and operationally meaningful indicator of the Type-I error in practice. For clarity, the overall training and inference procedures are summarized in Algorithms 1 and 2, respectively. Algorithm 1 Training algorithm of the proposed two-stage VAE-Inf 1: Input: D = Dtrain ∪ Dval ∪ Dtest ; α > 0; β > 0; number of projections M ; training epochs E1 , E2 2: Initialize: encoder qϕ (z|x), decoder pθ (x|z), latent dimension dl 3: Stage 1: VAE Training on Majority Class 4: for epoch = 1 to E1 do 5: Sample the mini-batch B1 ⊂ D1,train 6: (µ, σ 2 ) ← qϕ (B1 ) ▷ Encoding 7: ϵ ∼ N (0, I) 8: z ←µ+σ⊙ϵ 9: Compute LVAE and update (ϕ, θ) 10: end for 11: Compute (µref , Σref ) from D1,train using Eq. (3) and Eq. (4)
▷ Reparameterization ▷ ELBO optimization ▷ Wasserstein barycenter (diagonal)
12: Stage 2: Encoder Fine-tuning with Distribution-aware Loss 13: Fix decoder parameters θ 14: for epoch = 1 to E2 do k−1 15: Sample A = {aj }M ) j=1 , where aj ∼ U(S 16: Sample B1 ⊂ D1,train and B2 ⊂ D2,train 17: Draw latent codes z for B1 ∪ B2 via reparameterization 18: Compute Lreg in Eq. (7) and update ϕ 19: end for 20: return (ϕ, θ), (µref , Σref )
3
▷ Freeze decoder ▷ Random projections
▷ Distribution-aware separation
Experiments
This section provides a comprehensive evaluation of our proposed VAE-Inf on multiple benchmark datasets, focusing on classification performance under severe class imbalance. In addition, we examine the trade-off between Type-I and Type-II errors to evaluate the effectiveness of the proposed error-control mechanism. 3.1
Experimental Settings
Datasets. We evaluate the proposed VAE-Inf on three data domains: tabular, image, and high-dimensional biomedical data. In the main text, we focus on three tabular datasets (Credit Card Fraud Detection, Backdoor 7
Algorithm 2 Inference with distribution-free calibration 1: Input: trained (ϕ, θ); (µref , Σref ); D1,val ; projections A; target Type-I level δ; test sample x 2: Calibration: Empirical Quantile Threshold 3: Compute calibration scores Si ← S(xi ) for all xi ∈ D1,val 4: Sort {Si } increasingly as S(1) ≤ · · · ≤ S(ncal ) 5: k ← ⌈(1 − δ)(ncal + 1)⌉ , τδ ← S(k) 6: Inference: Projection-based Scoring and Decision 7: Encode x and obtain z via reparameterization 8: for a ∈ A do 9: Compute T (z, a) in Eq. (5) 10: end for 11: Compute S(x) in Eq. (8) 12: if S(x) ≤ τδ 13: ψ(x) ← 1 14: else 15: ψ(x) ← 2 16: end if
▷ Upper quantile threshold ▷ Latent representation ▷ 1-D normalized deviation ▷ Aggregated anomaly score ▷ Majority class ▷ Minority class
17: return ψ(x) and S(x)
Attack Detection, and Census) and the TCGA Pan-Cancer dataset, which together cover diverse application domains, imbalance levels, and feature dimensions. We report the minority-class proportion to characterize 2 the degree of class imbalance, defined as ρ = N1N+N , where N1 and N2 denote the numbers of majority- and 2 minority-class samples, respectively. A smaller value of ρ indicates a higher class imbalance level. For TCGA, we use a one-vs-rest protocol in which each cancer type is treated in turn as the minority class. For data preprocessing, we use a 6:2:2 train/validation/test split with preserved class imbalance. Continuous features are standardized using training-set statistics and the same transformation is applied to the validation and test sets. Additional image-domain experiments on MNIST and CIFAR-10, together with detailed dataset descriptions and experimental protocols, are reported in Appendix A.3. Methods for Comparison. We compare our method with five representative deep anomaly detection baselines: DeepSVDD, DeepSAD, DevNet, FeaWAD, and PReNet (Ruff et al., 2018; Pang et al., 2019; Zhou et al., 2022; Pang et al., 2023). These methods are selected to cover the main paradigms of deep anomaly detection under limited supervision; detailed descriptions are provided in Appendix A.1. Implementation Details. All methods are trained under the same clean-label protocol for fair comparison. For implementation, all baselines follow their original papers or standardized open-source implementations from DeepOD and ADBench (Xu et al., 2023; 2024; Han et al., 2022). For our method VAE-inf, the Stage 2 hyperparameters α and β are selected on the validation set from a predefined grid and then fixed for test-set evaluation. We use a batch size of 128 and sample 32 random projection directions for projection-based scoring. Additional architectural choices, hyperparameter settings, and training details are given in Appendix A.3. Evaluation metrics. We evaluate model performance using three widely adopted metrics: the Area Under the Receiver Operating Characteristic Curve (AUC-ROC), the Area Under the Precision–Recall Curve (AUC-PR) and F1-score. Since AUC-PR is generally more informative than AUC-ROC under severe imbalance (Saito & Rehmsmeier, 2015), we treat it as a primary threshold-free metric. For F1-score, we adopt a unified thresholding strategy: for each dataset, all methods predict the same fraction of samples as minority-class instances, where this fraction is set to the dataset-specific minority-class proportion. Additional discussion on the choice and interpretation of evaluation metrics is provided in Appendix A.3. 8
3.2 3.2.1
Main Experiment Results Tabular Datasets
Table 1 summarizes the performance of all methods on five tabular datasets with varying degrees of class imbalance. Across these benchmarks, our method achieves highly competitive AUC-ROC and consistently strong AUC-PR and F1-score performance, demonstrating its strength in detecting rare anomalous events. In particular, AUC-PR—the most informative metric under severe imbalance—shows substantial and stable gains over all baselines. For instance, on the Credit Card dataset with only 0.17% minority-class samples, our method achieves the highest AUC-PR and F1-score, outperforming strong discriminative approaches such as DeepSAD and PReNet. To further evaluate robustness under extreme class imbalance, we construct more challenging variants of the Backdoor and Census datasets by reducing the minority-class proportion in the training set to approximately 0.20%, while keeping the validation and test sets fixed for evaluation. A clear trend emerges as the training class imbalance becomes more severe. When the available minority-class samples are reduced to this extremely low level, the performance of existing methods decreases substantially, especially in terms of AUC-PR and F1-score. In contrast, our method maintains robust performance and delivers the best results across all metrics in the most imbalanced settings. These improvements suggest that learning a distributionaware latent representation and using projection-based statistical anomaly scores remain effective even when discriminative baselines become less stable due to the scarcity of minority-class training samples. In general, the tabular results demonstrate that our VAE-Inf performs on par with or better than stateof-the-art discriminative detectors under moderate imbalance, while offering pronounced gains when the minority-class proportion falls far below 1%. 3.2.2
TCGA Pan-Cancer Results
Table 2 summarizes the performance across 33 one-vs-rest cancer detection tasks on the TCGA pan-cancer dataset. This benchmark is particularly challenging due to its extremely high dimensionality (20,531 genomic features) and highly skewed class distribution. Across all three metrics, our method ranks consistently among the top-performing approaches, achieving the highest AUC-PR (95.58) and F1-score (93.52), while also attaining the second-highest AUC-ROC. To further evaluate performance under extreme imbalance in realistic rare disease scenarios, we examine five cancer types with prevalence below 1% in the cohort. As shown in Table 2, several baselines struggle in this setting. DeepSAD and PReNet remain competitive, though their performance varies considerably across metrics. PReNet obtains the highest AUC-ROC (99.55) but yields lower AUC-PR, indicating a less favorable precision–recall balance when minority-class samples are exceedingly scarce. In contrast, our method achieves the best AUC-PR (79.66) and F1-score (76.24), and the second-best AUC-ROC, demonstrating improved stability across all three evaluation measures under ultra-imbalanced conditions. These findings highlight that the proposed generative–statistical modeling approach is competitive on average across all 33 tasks and becomes particularly advantageous when minority classes are extremely rare. By explicitly learning the majority-class distribution and grounding anomaly detection in projection-based deviation scoring, our method maintains strong discriminative ability even in high-dimensional, ultra-imbalanced biomedical settings. 3.2.3
Metrics Discussion
While our method is not always the top performer in AUC-ROC, it consistently ranks strongest in AUC-PR and F1-score across nearly all datasets, which are more informative in severely imbalanced settings. As noted in Saito & Rehmsmeier (2015), when the minority-class proportion is extremely small, ROC-based evaluation can be overly influenced by the large number of true negatives. Under such conditions, even large differences in false positives produce only minimal changes in the false-positive rate, causing AUC-ROC to remain high and to be relatively insensitive to improvements in minority-class detection. In contrast, the precision–recall curve directly captures the trade-off between true positives and false positives, making AUC-PR far more informative under severe imbalance. Therefore, the consistently superior AUC-PR and F1 results of our 9
Table 1: Performance comparison (AUC-ROC / AUC-PR / F1-score, all in percentage). Here, ρ = N2 /(N1 + N2 ) denotes the minority-class proportion. Best and second-best results for each metric per dataset are highlighted in bold and underlined, respectively. All results are averaged over ten independent runs and presented as mean ± standard deviation. Metric
DevNet
FeaWAD
DeepSAD
PReNet
DeepSVDD
VAE-inf
86.19±6.82 25.95±15.13 35.40±13.61
97.48±0.31 85.61±1.45 83.57±1.87
78.01±7.06 36.88±5.36 40.70±4.25
99.31±0.13 97.41±0.63 93.60±1.01
86.25±6.50 54.74±8.17 52.66±9.57
99.26±0.34 96.07±0.95 93.73±1.88
56.18±6.39 7.42±0.83 7.61±1.03
93.88±0.08 59.04±0.39 55.70±0.42
57.44±4.08 7.95±0.61 9.21±0.95
90.61±0.28 52.39±1.26 51.08±1.05
Credit Card (ρ = 0.17%) AUC-ROC ↑ AUC-PR ↑ F1-score ↑
96.06±0.57 45.32±6.93 51.53±4.26
96.86±0.43 56.85±6.00 59.59±4.45
95.46±0.80 84.11±1.38 81.94±1.45
98.03±0.30 70.50±4.92 71.22±2.57
Backdoor (ρ = 2.44%) AUC-ROC ↑ AUC-PR ↑ F1-score ↑
98.76±0.44 93.61±0.59 89.55±0.83
99.09±0.21 91.92±1.85 88.52±1.75
99.79±0.10 97.61±0.32 94.14±0.29
95.74±0.27 89.58±0.48 86.91±0.35
Backdoor (ρ = 0.20%) AUC-ROC ↑ AUC-PR ↑ F1-score ↑
99.17±0.34 94.14±0.24 89.96±0.76
97.60±0.88 83.35±3.89 83.51±2.45
99.24±0.27 95.49±0.80 91.81±0.47
95.65±0.34 86.66±0.44 85.71±0.34
Census (ρ = 6.20%) AUC-ROC ↑ AUC-PR ↑ F1-score ↑
90.49±0.65 39.72±3.38 43.82±2.54
89.80±0.65 41.27±4.45 44.99±3.63
90.21±0.37 49.93±0.53 50.30±0.37
92.66±0.04 54.07±0.19 52.67±0.17
Census (ρ = 0.21%) AUC-ROC ↑ AUC-PR ↑ F1-score ↑
88.88±0.50 42.16±0.37 44.12±0.30
86.76±1.07 30.98±2.23 37.01±2.23
75.12±1.01 26.43±0.86 28.37±1.28
88.00±0.47 46.68±0.50 47.23±0.43
method provide a more reliable indication of practical detection capability, even in cases where AUC-ROC differences are small. 3.3
Error Control Results
To evaluate the statistical reliability of the proposed decision rule, we examine how the Type-I and Type-II errors vary with the inference-time threshold τ applied to the score S. After training the model, we keep all network parameters fixed and sweep the inference threshold τ over 100 uniformly sampled values. For each threshold, we compute the corresponding errors on both the validation and test sets. In Figure 2, solid lines denote Type-I errors and dashed lines denote Type-II errors, with validation and test curves plotted together to assess the stability of the calibration rule across data splits. For the TCGA dataset, we select Rectum Adenocarcinoma as the minority class, corresponding to an minority-class proportion of 1%. Even in this high-dimensional biomedical setting, the validation and test Type-I curves remain closely matched, with a mean absolute deviation of only 0.0031, indicating that the proposed calibration mechanism remains stable beyond tabular benchmarks. A similar pattern is observed on the tabular datasets. On Credit Card (Figure 2b), the mean absolute deviation between the validation and test Type-I curves is only 0.0003, corresponding to roughly 17 majority samples out of more than 56,000. Similar stability is observed on the Backdoor (Figure 2c) and Census (Figure 2d) datasets. Overall, these results indicate that the proposed score induces stable error profile 10
Table 2: Performance comparison on TCGA datasets (AUC-ROC / AUC-PR / F1-score, all in percentage). Best and second-best results for each metric per dataset are highlighted in bold and underlined, respectively. Results for TCGA-Full are averaged over all 33 one-vs-rest experiments, while results for TCGA-Rare are averaged over five selected rare-cancer experiments with minority proportion ρ ≤ 1%. Metric
DevNet
FeaWAD
DeepSAD
PReNet
DeepSVDD
VAE-inf
99.85±0.24 94.06±12.31 91.00±14.27
50.60±7.06 3.47±2.33 3.51±3.50
99.58±0.99 95.58±8.16 93.52±9.17
55.19±2.91 1.82±1.40 3.61±4.94
98.32±1.93 79.66±10.35 76.24±10.05
TCGA-Full AUC-ROC ↑ AUC-PR ↑ F1-score ↑
99.08±2.32 88.51±20.96 85.19±21.95
98.60±1.84 82.66±20.35 79.00±19.39
98.62±3.06 94.53±11.18 92.51±11.28
TCGA-Rare (five selected types, ρ ≤ 1%) 96.35±4.70 63.62±25.16 60.05±23.72
97.82±1.27 51.83±27.68 51.82±26.77
Type-I & II Error
Type-I & II Error
0.25
1.0
valid Type-I valid Type-II test Type-I test Type-II
0.20
0.8
99.55±0.33 73.73±19.74 67.27±21.75
93.12±4.86 73.00±15.89 71.73±15.21
0.175 0.8
0.15
0.2
0.05
0.2
0.0
0.00
0.0
20
30
40
50
60
70
0.125
Type-I
Type-I
0.10
10
0.4
80
0.075 0.050 0.025 0.000 0
(a) TCGA (MAD = 0.0031). 0.040 0.035 0.8
0.030
0.0 0
5
10
15
20
25
20
25
30
1.0 0.9
0.08
valid Type-I valid Type-II test Type-I test Type-II
0.8 0.06
0.7
Type-II
Type-I
Type-I 0.2
15
Type-I & II Error valid Type-I valid Type-II test Type-I test Type-II
0.020 0.4
10
(b) Credit Card (MAD = 0.0003).
0.025
0.6
5
0.6
0.015
0.5
0.010
0.4
0.005
0.3
0.000
0.2
30
Type-II
Type-I & II Error 1.0
valid Type-I valid Type-II test Type-I test Type-II
0.100
0.4
0
0.150
0.6
Type-II
0.6
0.200
1.0
Type-II
AUC-ROC ↑ AUC-PR ↑ F1-score ↑
0.04 0.02 0.00 0
(c) Backdoor (MAD = 0.0010).
5
10
15
20
25
30
(d) Census (MAD = 0.0004).
Figure 2: Type-I (solid) and Type-II (dashed) errors as functions of the inference threshold τ across datasets. The reported mean absolute deviations (MAD) between the validation and test Type-I curves are uniformly small, indicating that the calibrated decision rule generalizes stably across data splits.
across validation and test splits, enabling reliable threshold calibration without overfitting to a particular validation split. Across the entire range of τ , the Type-I curves for different data splits remain closely aligned, indicating that the learned majority reference induces a highly stable decision boundary under threshold variation. By contrast, the Type-II curves exhibit more noticeable variation. This is expected because the minority 11
class is substantially smaller and therefore more sensitive to sampling variability and the stochasticity of fine-tuning. At the same time, increasing τ enlarges the acceptance region for the majority class, which monotonically reduces the Type-I error but increases the miss-detection rate. Together, these patterns illustrate the fundamental trade-off between statistical conservativeness and anomaly sensitivity. Table 3: Type-II (at Type-I = 0.01) and Type-I (at Type-II = 0.1) errors on test set after calibrating the threshold on the validation set. Best results for each metric are highlighted in bold. Dataset
Error Type
DeepSVDD
DevNet
FeaWAD
DeepSAD
PReNet
VAE-inf
Credit Card (0.17%)
Type-II ↓ Type-I ↓
0.3571 0.4461
0.1939 0.0595
0.1122 0.1205
0.1122 0.1441
0.1020 0.0683
0.1020 0.0455
Backdoor (2.44%)
Type-II ↓ Type-I ↓
0.8712 0.8792
0.0730 0.0030
0.0901 0.0052
0.0300 0.0002
0.1202 0.0549
0.0300 0.0003
Backdoor (0.20%)
Type-II ↓ Type-I ↓
0.5021 0.4312
0.0708 0.0025
0.1524 0.0476
0.0708 0.0008
0.1288 0.0489
0.0536 0.0007
Census (6.20%)
Type-II ↓ Type-I ↓
0.9906 0.9416
0.8810 0.2502
0.8333 0.1933
0.7052 0.2958
0.6877 0.1740
0.6330 0.1647
Census (0.21%)
Type-II ↓ Type-I ↓
0.9833 0.6724
0.7833 0.2587
0.8883 0.3289
0.8331 0.5624
0.7491 0.2661
0.6828 0.2494
TCGA (1.00%)
Type-II ↓ Type-I ↓
1.0000 0.7933
0.6190 0.0333
0.5714 0.0304
0.1429 0.0087
0.4286 0.0251
0.1429 0.0082
Table 3 reports the Type-II error under a prescribed Type-I error level (δ = 0.01) and the Type-I error under a prescribed Type-II error level (δ = 0.1), where the corresponding decision threshold is calibrated on the validation set and then evaluated on the test set. For completeness, we also report the test Type-I error at the validation-selected operating point whose validation Type-II is set to 0.1. The proposed method consistently achieves the best or near-best performance across all datasets, demonstrating its effectiveness in meeting target error constraints while maintaining stable generalization under the distribution shift from validation to test data. Overall, these results verify that the proposed inference procedure provides reliable and interpretable error control: for a user-specified target error level δ, the calibration step selects the corresponding decision threshold so as to satisfy the desired constraint. This enables practitioners to specify operational requirements directly in terms of error tolerance, rather than tuning an ad hoc threshold parameter.
4
Conclusion
This paper presented VAE-Inf: a two-stage generative framework for imbalanced classification that combines deep representation learning with statistically principled decision-making. By training a VAE solely on majority-class data, the proposed method constructs a reliable latent reference distribution that captures the intrinsic structure of normal samples without contamination from scarce minority data. A distributionaware fine-tuning stage then exploits limited minority information to enhance separability in latent space through variance-normalized projection statistics. A key advantage of the proposed approach lies in its statistical interpretability. The projection-based score naturally corresponds to a hypothesis testing statistic under the learned majority distribution, enabling a transparent acceptance–rejection mechanism. Moreover, the introduction of an empirical quantile calibration procedure provides exact finite-sample control of the Type-I error under minimal exchangeability assumptions, independent of the correctness of the learned generative model. This distinguishes our method from existing imbalanced learning approaches that rely on heuristic thresholds or asymptotic approximations. Experiment results across diverse datasets demonstrate that the proposed VAE-Inf achieves superior classification performance for highly imbalanced data while offering explicit Type-I error guarantees. Beyond 12
imbalanced classification, the methodology developed in this work suggests a broader perspective on integrating deep generative models with distribution-free statistical inference. Future work may explore extensions to multi-class imbalance, adaptive projection strategies, and alternative generative priors, as well as applications to other anomaly-sensitive domains such as biomedical analysis and safety-critical systems.
References Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pp. 214–223, 2017. Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label-distribution-aware margin loss. In Advances in Neural Information Processing Systems, volume 32, 2019. Cristiano L. Castro and Antônio P. Braga. Novel cost-sensitive approach to improve the multilayer perceptron performance on imbalanced data. IEEE Transactions on Neural Networks and Learning Systems, 24:888– 899, 2013. Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321–357, 2002. Nitesh V. Chawla, Aleksandar Lazarevic, Lawrence O. Hall, and Kevin W. Bowyer. Smoteboost: Improving prediction of the minority class in boosting. In Knowledge Discovery in Databases: PKDD 2003, pp. 107–119, 2003. Yuan Shih Chow and Henry Teicher. Probability theory: independence, interchangeability, martingales. Springer Science & Business Media, 2003. Qi Dong, Shaogang Gong, and Xiatian Zhu. Imbalanced deep learning by minority class incremental rectification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41:1367–1381, 2019. Charles Elkan. The foundations of cost-sensitive learning. In Proceedings of the 17th International Joint Conference on Artificial Intelligence - Volume 2, pp. 973–978, 2001. Yoav Freund and Robert E. Schapire. Experiments with a new boosting algorithm. In Proceedings of the Thirteenth International Conference on International Conference on Machine Learning, pp. 148–156, 1996. Mikel Galar, Alberto Fernandez, Edurne Barrenechea, Humberto Bustince, and Francisco Herrera. A review on ensembles for the class imbalance problem: Bagging-, boosting-, and hybrid-based approaches. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42:463–484, 2012. Mikel Galar, Alberto Fernández, Edurne Barrenechea, and Francisco Herrera. Eusboost: Enhancing ensembles for highly imbalanced data-sets by evolutionary undersampling. Pattern Recognition, 46:3460–3471, 2013. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, volume 27, 2014. Haixiang Guo, Yijing Li, Jennifer Shang, Mingyun Gu, Yuanyue Huang, and Bing Gong. Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications, 73:220–239, 2017. Ting Guo, Xingquan Zhu, Yang Wang, and Fang Chen. Discriminative sample generation for deep imbalanced learning. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pp. 2406–2412, 2019. Songqiao Han, Xiyang Hu, Hailiang Huang, Minqi Jiang, and Yue Zhao. Adbench: Anomaly detection benchmark. In Advances in Neural Information Processing Systems, volume 35, pp. 32142–32159, 2022. 13
Haibo He and Edwardo A. Garcia. Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21:1263–1284, 2009. Haoqi Huang, Ping Wang, Jianhua Pei, Jiacheng Wang, Shahen Alexanian, and Dusit Niyato. Deep learning advancements in anomaly detection: A comprehensive survey. IEEE Internet of Things Journal, 12: 44318–44342, 2025. Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. Jing Lei, Max G’Sell, Alessandro Rinaldo, Ryan J. Tibshirani, and Larry Wasserman. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113(523):1094–1111, 2018. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988, 2017. Anton Mallasto and Aasa Feragen. Learning from uncertain curves: The 2-wasserstein metric for gaussian processes. In Advances in Neural Information Processing Systems, volume 30, 2017. Giovanni Mariani, Florian Scheidegger, Roxana Istrate, Costas Bekas, and Cristiano Malossi. Bagan: Data augmentation with balancing gan. arXiv preprint arXiv:1803.09655, 2018. Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 Military Communications and Information Systems Conference (MilCIS), pp. 1–6, 2015. Sankha Subhra Mullick, Shounak Datta, and Swagatam Das. Generative adversarial minority oversampling. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1695–1704, 2019. Ehsan Nazari and Paula Branco. On oversampling via generative adversarial networks under different data difficulty factors. In Proceedings of the Third International Workshop on Learning with Imbalanced Domains: Theory and Applications, volume 154, pp. 76–89, 2021. Kemal Oksuz, Baris Can Cam, Sinan Kalkan, and Emre Akbas. Imbalance problems in object detection: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43:3388–3415, 2021. Guansong Pang, Chunhua Shen, and Anton van den Hengel. Deep anomaly detection with deviation networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 353–362, 2019. Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel. Deep learning for anomaly detection: A review. ACM Comput. Surv., 54, 2021. Guansong Pang, Chunhua Shen, Huidong Jin, and Anton van den Hengel. Deep weakly-supervised anomaly detection. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1795–1807, 2023. Wenbin Pei, Bing Xue, Mengjie Zhang, Lin Shang, Xin Yao, and Qiang Zhang. A survey on unbalanced classification: How can evolutionary computation help? IEEE Transactions on Evolutionary Computation, 28:353–373, 2024. Yaniv Romano, Evan Patterson, and Emmanuel Candes. Conformalized quantile regression. In Advances in Neural Information Processing Systems, volume 32, 2019. Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In Proceedings of the 35th International Conference on Machine Learning, volume 80, pp. 4393–4402, 2018. 14
Lukas Ruff, Robert A. Vandermeulen, Nico Görnitz, Alexander Binder, Emmanuel Müller, Klaus-Robert Müller, and Marius Kloft. Deep semi-supervised anomaly detection. In International Conference on Learning Representations, 2020. Takaya Saito and Marc Rehmsmeier. The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. PloS one, 10:e0118432, 2015. Vignesh Sampath, Iñaki Maurtua, Juan Jose Aguilar Martin, and Aitor Gutierrez. A survey on generative adversarial networks for imbalance problems in computer vision tasks. Journal of big Data, 8:27, 2021. Hinrich Schütze, Christopher D Manning, and Prabhakar Raghavan. Introduction to information retrieval, volume 39. Cambridge University Press Cambridge, 2008. Muhammad Atif Tahir, Josef Kittler, Krystian Mikolajczyk, and Fei Yan. A multiple expert approach to the class imbalance problem using inverse random under sampling. In Multiple Classifier Systems, pp. 82–91, 2009. Nguyen Thai-Nghe, Zeno Gantner, and Lars Schmidt-Thieme. Cost-sensitive learning methods for imbalanced data. In The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1–8, 2010. Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Springer, 2005.
Algorithmic learning in a random world.
Zhiqiang Wan, Yazhou Zhang, and Haibo He. Variational autoencoder based synthetic data generation for imbalanced learning. In 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–7, 2017. Xinyue Wang, Yilin Lyu, and Liping Jing. Deep generative model for robust imbalance classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14124–14133, 2020. Hongzuo Xu, Guansong Pang, Yijie Wang, and Yongjun Wang. Deep isolation forest for anomaly detection. IEEE Transactions on Knowledge and Data Engineering, 35:12591–12604, 2023. Hongzuo Xu, Yijie Wang, Songlei Jian, Qing Liao, Yongjun Wang, and Guansong Pang. Calibrated one-class classification for unsupervised time series anomaly detection. IEEE Transactions on Knowledge and Data Engineering, 36:5723–5736, 2024. Chunkai Zhang, Ying Zhou, Yingyang Chen, Yepeng Deng, Xuan Wang, Lifeng Dong, and Haoyu Wei. Over-sampling algorithm based on vae in imbalanced classification. In Cloud Computing – CLOUD 2018, pp. 334–344, 2018. Zhuoyuan Zheng, Yunpeng Cai, and Ye Li. Oversampling method for imbalanced classification. Computing and Informatics, 34:1017–1037, 2016. Yingjie Zhou, Xucheng Song, Yanru Zhang, Fanxing Liu, Ce Zhu, and Lingqiao Liu. Feature encoding with autoencoders for weakly supervised anomaly detection. IEEE Transactions on Neural Networks and Learning Systems, 33:2454–2465, 2022.
A
Appendix
A.1
Related Work
Deep anomaly detection (AD) under limited supervision has received increasing attention in recent years. In practice, some labeled data is often available and should be fully exploited. Leveraging such labels effectively to learn expressive representations of normality and abnormality is therefore essential for accurate anomaly detection (Pang et al., 2021). Existing methods exploit this weak supervision in different ways, which can 15
be broadly grouped into geometric compactness modeling, statistical score learning, reconstruction-guided representation learning, and relational supervision. A representative line of work focuses on geometric compactness of normal data in latent space. DeepSAD (Ruff et al., 2020) extends Deep SVDD (Ruff et al., 2018) to the semi-supervised setting by incorporating labeled normal and anomalous samples into a unified objective. Unlabeled and normal samples are encouraged to concentrate around a latent center, while labeled anomalies are pushed away, yielding lowentropy representations for normal data. This framework effectively leverages limited supervision to improve robustness over purely unsupervised approaches. Another line of work formulates AD as direct statistical score learning. DevNet (Pang et al., 2019) proposes to optimize anomaly scores against a predefined Gaussian reference distribution using a deviation loss. Normal samples are constrained to stay near the reference mean, while labeled anomalies are encouraged to deviate significantly, resulting in statistically interpretable decision thresholds. This design enables efficient training and scalability, and provides an explicit connection between anomaly scores and significance levels. To better capture the intrinsic structure of normal data, recent methods integrate reconstruction-based representation learning with discriminative objectives. FeaWAD (Zhou et al., 2022) employs an autoencoder to model the normal data manifold and represents each sample using a three-factor feature tuple (h, r, e), corresponding to latent embedding, reconstruction residual, and reconstruction error magnitude. These features are subsequently fed into a discriminative scoring network. More recently, relational supervision has been proposed to further amplify weak supervision. PReNet (Pang et al., 2023) constructs pairwise relationships among samples (normal–normal, normal–anomaly, anomaly– anomaly) and learns to predict their relative anomaly ordering. This pairwise formulation significantly enlarges the effective supervision signal and improves generalization to unseen anomalies. In summary, existing AD methods leverage limited labels through diverse mechanisms, ranging from geometric constraints and reference-based scoring to manifold-aware representations and relational learning. While effective, most approaches either rely on simplified assumptions about the normal distribution or produce heuristic anomaly scores without an explicit, data-driven statistical decision rule. These limitations motivate approaches that jointly learn faithful majority-class representations and enable interpretable, statistically grounded inference under severe class imbalance. A.2
Finite-Sample Guarantee via Empirical Quantile Calibration
In this section, we provide a rigorous proof for the Type-I error control achieved by the calibration scheme. Theorem 1 establishes a finite-sample guarantee under the sole assumption of exchangeability of the scores. Proof. If k = ncal + 1, then by definition τδ = +∞, and therefore P Sncal +1 ≤ τδ = 1 ≥ 1 − δ. Thus, it remains to consider the case k ≤ ncal , for which τδ = S(k) . By exchangeability and the assumption that the scores are almost surely distinct, the rank of Sncal +1 among {S1 , . . . , Sncal , Sncal +1 } is uniformly distributed over {1, . . . , ncal + 1}. That is, for every j ∈ {1, . . . , ncal + 1}, P rank(Sncal +1 ) = j =
1 . ncal + 1
Since τδ = S(k) , the event Sncal +1 ≤ τδ is equivalent to the event that the rank of Sncal +1 is at most k. Hence P Sncal +1 ≤ τδ = P rank(Sncal +1 ) ≤ k =
16
k . ncal + 1
Substituting k = ⌈(1 − δ)(ncal + 1)⌉ and the elementary inequality ⌈x⌉ ≥ x, we obtain k (1 − δ)(ncal + 1) ≥ = 1 − δ. ncal + 1 ncal + 1 Therefore, P Sncal +1 ≤ τδ ≥ 1 − δ. This completes the proof. A.3
Experimental Setting
A.3.1
Dataset Description
Tabular datasets. We consider three representative tabular imbalance classification datasets that cover financial risk assessment, network intrusion detection, and socioeconomic analysis. The Credit Card Fraud Detection dataset from Kaggle exhibits an extreme minority-class proportion of 0.17%, reflecting realistic fraud detection scenarios. The Backdoor Attack Detection dataset is derived from the UNSW-NB15 benchmark (Moustafa & Slay, 2015) and contains 2.44% anomalous traffic instances, representing a modern network security challenge. The Census dataset is constructed from the U.S. Census Bureau database, where the task is to identify individuals with income exceeding 50K dollars per year; only 6.2% of the population belongs to this high-income group. These datasets span diverse domains and imbalance levels, providing a broad evaluation of our method. A summary of dataset description is presented in Table 4. Table 4: Overview of the tabular datasets. The minority-class proportion is defined as ρ = N2 /(N1 + N2 ). Dataset Credit Card Backdoor Census
#Samples
Features
#Minority Samples
Minority Prop.
Category
284,807 95,329 299,285
30 196 500
492 2,329 18,568
0.17% 2.44% 6.20%
Finance Network Sociology
All tabular features are standardized to zero mean and unit variance. For consistency across benchmarks, each dataset is partitioned into training, validation, and test splits with a ratio of 6:2:2, and the minority-class proportion is preserved within each split. Image datasets. Following the experimental protocol of DeepSAD (Ruff et al., 2020), we evaluate our method on two benchmark image datasets: MNIST, and CIFAR-10. In the no-pollution setting, one class is designated as the majority class, while one of the remaining nine classes is randomly selected as the minority class. This procedure is repeated for all 10 majority classes and all 9 minority classes, resulting in a total of 90 experiments per dataset. To simulate realistic extreme imbalance scenarios, the fraction of labeled training anomalies is set to 2 = 0.005. The same sampling ratio is applied consistently to both the training and test sets. ρ = N1N+N 2 Performance is reported as the average values across all 90 runs. TCGA Pan-Cancer dataset. We further evaluate our method on a large-scale biomedical dataset: the TCGA Pan-Cancer cohort obtained from the UCSC Xena platform (http://xena.ucsc.edu). We use the genomicMatrix representation, where each row corresponds to a patient sample and each column corresponds to a genomic feature (molecular identifier). The dataset consists of 10,459 samples and 20,531 genomic features across 33 distinct cancer types. The class distribution is highly imbalanced, as shown in Table 5. 17
Table 5: Distribution of phenotype classes in the TCGA pan-cancer dataset. Class Breast invasive carcinoma Kidney clear cell carcinoma Lung adenocarcinoma Thyroid carcinoma Head & neck squamous cell carcinoma Lung squamous cell carcinoma Prostate adenocarcinoma Brain lower grade glioma Skin cutaneous melanoma Stomach adenocarcinoma Bladder urothelial carcinoma Liver hepatocellular carcinoma Colon adenocarcinoma Kidney papillary cell carcinoma Cervical & endocervical cancer Ovarian serous cystadenocarcinoma Sarcoma Uterine corpus endometrioid carcinoma Esophageal carcinoma Pheochromocytoma & paraganglioma Pancreatic adenocarcinoma Acute myeloid leukemia Glioblastoma multiforme Testicular germ cell tumor Thymoma Rectum adenocarcinoma Kidney chromophobe Mesothelioma Uveal melanoma Adrenocortical cancer Uterine carcinosarcoma Diffuse large B-cell lymphoma Cholangiocarcinoma
Occurrences
Proportion
1218 606 576 572 566 553 550 530 474 450 426 423 329 323 308 308 265 201 196 187 183 173 172 156 122 105 91 87 80 79 57 48 45
11.65% 5.79% 5.51% 5.47% 5.41% 5.29% 5.26% 5.07% 4.53% 4.30% 4.07% 4.04% 3.15% 3.09% 2.94% 2.94% 2.53% 1.92% 1.87% 1.79% 1.75% 1.65% 1.64% 1.49% 1.17% 1.00% 0.87% 0.83% 0.76% 0.76% 0.54% 0.46% 0.43%
The imbalance is substantial: the most common subtype (breast invasive carcinoma) constitutes over 11% of the dataset, whereas several rare malignancies, such as cholangiocarcinoma and Mesothelioma, each account for fewer than 1% of samples. This makes TCGA a highly challenging and biologically meaningful benchmark. For evaluation, we adopt a one-vs-rest protocol. Each of the 33 cancer types is treated in turn as the minority class, while samples from the remaining 32 types form the majority class. This yields 33 separate experiments. In addition to reporting the average performance over all 33 one-vs-rest tasks, we also report results on a selected rare-cancer subset with minority proportion ρ ≤ 1%, consisting of rectum adenocarcinoma, uterine carcinosarcoma, kidney chromophobe, cholangiocarcinoma, and adrenocortical cancer. 18
A.3.2
Implementation Details
All baseline methods are implemented following the hyperparameters and model configurations reported in their original papers. For consistency and reproducibility, we adopt the codes from original papers and open-source implementations from DeepOD (Xu et al., 2023; 2024) and ADBench (Han et al., 2022), which provide standardized training pipelines and widely validated configurations for deep anomaly detection. Our approach employs a vanilla VAE backbone in Stage 1, with the architecture adapted to the structure of each data domain. For tabular datasets, we use a lightweight fully connected encoder–decoder, where both the encoder and decoder contain two hidden layers. For image datasets, MNIST uses a convolutional encoder– decoder with residual blocks, whereas CIFAR-10 adopts a deeper residual convolutional architecture. For the TCGA Pan-Cancer dataset, which contains over 20,000 genomic features, we use a high-capacity MLPbased VAE with residual feedforward blocks. During Stage 2, the model is fine-tuned using the proposed projection-based regularization. The hyperparameters α and β are selected on the validation set from a predefined grid and then fixed for test-set evaluation. The latent dimension dl and the hyperparameters α and β are summarized in Table 6. Table 6: Hyperparameter settings used in the experiments, including the latent dimension dl and the validation-selected Stage 2 hyperparameters α and β. Dataset
dl
α
β
Credit Card Backdoor Census
16 20 50
16 25 25
2 16 10
MNIST CIFAR-10
16 64
16 16
10 10
TCGA Pan-Cancer
128
16
2
For all experiments, we set the batch size to 128. The numbers of training epochs in Stage 1 and Stage 2 are set to E1 = 200 and E2 = 100, respectively. The corresponding learning rates are 1 × 10−4 for Stage 1 and 2 × 10−3 for Stage 2. For projection-based scoring, we sample 32 random projection vectors to estimate the anomaly score. All models are optimized using Adam with default momentum parameters, and early stopping is applied based on the validation loss to prevent overfitting. A.3.3
Evaluation Metrics
We evaluate model performance using three widely adopted metrics: the Area Under the Receiver Operating Characteristic Curve, the Area Under the Precision–Recall Curve (AUC-PR) and F1-score. AUC-ROC measures the trade-off between true positive and false positive rates across all decision thresholds. Although widely used, AUC-ROC can be misleading under severe class imbalance, as its value may be dominated by the abundant normal samples rather than the minority-class samples. In contrast, AUC-PR focuses explicitly on the performance of the positive (minority) class by measuring precision and recall. As demonstrated in Saito & Rehmsmeier (2015), AUC-PR is a more informative indicator than AUC-ROC in highly imbalanced settings, where even a large improvement in identifying minority-class samples may produce only marginal changes in AUC-ROC. We compute AUC-PR following the standard average-precision formulation in Schütze et al. (2008). Finally, we report the F1-score as a threshold-dependent metric that balances precision and recall. Since most baseline methods do not provide explicit decision thresholds, we adopt a unified strategy based on the minority-class proportion reported in Table 4 for threshold selection. For example, in Credit Card Fraud Detection dataset, where the minority-class proportion is 0.17%, the top 0.17% of samples ranked by anomaly scores are predicted as minority class. This strategy ensures a consistent and fair comparison on classification performance across methods by fixing the predicted minority proportion. 19
A.4
Ablation and Sensitivity Analysis
A.4.1
Effect of Stage-2 fine-tuning
Table 7 reports the performance of the proposed VAE-Inf before and after Stage 2 on tabular datasets. A consistent and substantial improvement is observed in all evaluation metrics. In particular, Stage 1 alone yields limited discriminative performance, especially on highly imbalanced datasets. This is consistent with the observation in the latent space, where minority samples are not well separated from the majority manifold. After incorporating Stage 2, all metrics improve significantly, demonstrating that Stage 2 effectively enhances the discriminative capability of the learned representation. Table 7: Performance comparison between Stage 1 and Stage 2 (AUC-ROC / AUC-PR / F1-score, all in percentage). Best results for each metric are highlighted in bold. All results are averaged over ten independent runs and presented as mean ± standard deviation. Method
AUC-ROC ↑
AUC-PR ↑
F1-score ↑
Credit Card (ρ = 0.17%) Stage 1 Stage 1&2
87.51±1.48 97.48±0.31
4.30±0.58 85.61±1.45
8.27±1.25 83.57±1.87
Backdoor (ρ = 2.44%) Stage 1 Stage 1&2
77.89±2.71 99.31±0.13
29.30±2.17 97.41±0.63
41.52±1.45 93.60±1.01
Census (ρ = 6.20%) Stage 1 Stage 1&2
A.4.2
42.80±1.72 93.88±0.08
5.51±0.17 59.04±0.39
5.37±0.27 55.70±0.42
Sensitivity to Hyperparameters
We examine the influence of the hyperparameters α and β in the proposed distribution-aware regularization term Eq. (7). The parameter α controls the extent of the latent confidence region of the majority distribution, while β regulates the strength with which minority embeddings are encouraged to lie outside this region. The larger α allows for a wider admissible range for majority samples, whereas the larger β increases the penalty for minority embeddings that remain close to the majority reference region. Figure 3 presents a sensitivity analysis of AUC-PR with respect to β under different choices of α on the Credit Card dataset. Within the considered grid, the most favorable performance on this dataset is observed at (α = 16, β = 2). The results show that the performance is sensitive to the balance between majority tolerance and minority separation. Extremely small α (overly restrictive boundaries) or excessively large β (over-separation) both lead to performance degradation, indicating that balanced regularization is essential for stable discrimination under high imbalance. The selected values of α and β vary across datasets and are determined through validation set selection in the main experiments. A.5
Image Datasets Results
Table 8 reports results on MNIST and CIFAR-10 under the ρ = 0.005 protocol. Since most baselines are designed for tabular inputs, we compare our method with DeepSAD, the only baseline offering official support for image anomaly detection. All results are averaged over 90 experiments spanning all normal–anomaly class combinations. Across both datasets, our method achieves competitive AUC-ROC and consistently stronger AUC-PR and F1-score performance. On MNIST, where anomaly structure is relatively simple, both methods obtain 20
Best: ( = 16, = 2.0)
0.8
AUC-PR
0.6
=1 =4 =9 = 16 = 25
0.4 0.2 0.0
0
2
4
6
8
10
12
14
16
Figure 3: Sensitivity of AUC-PR to β under different choices of α on the Credit Card dataset. Optimal performance occurs at (α = 16, β = 2). Table 8: Comparison of AUC-ROC and AUC-PR on image datasets ( ρ = 0.005). Results are averaged over 90 experiments. Method
AUC-ROC ↑
AUC-PR ↑
F1-score ↑
MNIST DeepSAD VAE-inf (Ours)
95.22±3.61 95.25±3.43
67.36±18.20 73.94±14.38
62.59±16.38 68.04±13.74
CIFAR-10 DeepSAD VAE-inf (Ours)
72.54±8.35 71.46±7.62
18.53±10.18 62.07±11.60
21.31±10.06 55.55±9.24
similar AUC-ROC, but our approach yields noticeably higher AUC-PR (+6.6 points on average) and F1score, indicating more precise identification of rare minority-class samples. The advantage becomes larger on CIFAR-10 dataset. While DeepSAD attains a slightly higher AUC-ROC, its AUC-PR drops sharply due to poor precision under extreme imbalance. In contrast, our distribution-aware scoring achieves a large improvement in AUC-PR (from 18.53 to 62.07) and F1-score, reflecting a significantly enhanced ability to rank and classify anomalous image samples. Overall, the image results demonstrate that the proposed VAE-Inf remains effective beyond tabular domains. A.6
Additional Error-Control and Calibration Results
A.6.1
Stability across Training, Validation, and Test Splits
We evaluate the stability of the proposed decision rule across the training, validation, and test splits of the Credit Card dataset, using hyperparameters α = 16 and β = 2. The three subsets preserve the same minority-class proportion (approximately 0.17%), enabling a controlled examination of error behaviors. Sample statistics are summarized below: • Training: 170,883 samples, 295 minority (0.1726%) 21
• Validation: 56,962 samples, 99 minority (0.1738%) • Test: 56,962 samples, 98 minority (0.1720%) Table 9 reports the quantitative results obtained by using the same threshold parameter τ = 16 during training and directly deploying it for inference on three sets. A notable observation is the remarkable stability of the Type-I error, which remains within the narrow range of 0.11%–0.16% across all splits. This confirms that the latent boundary learned in Stage 1 is well calibrated: normal samples consistently fall within the estimated majority region, and the decision threshold based on statistical score preserves its validity across independent partitions. Type-II error exhibits larger variability, increasing from 6.44% in the training set to 17.17% on the validation split, and decreasing again to 12.24% on the test set. This fluctuation is expected due to the limited number of minority samples and the stochasticity inherent in fine-tuning. Table 9: Confusion matrices for training, validation, and test sets. Rows correspond to true labels and columns to predicted labels. True\Pred. Maj. Min.
Maj.
Min.
170,393 195 19 276
(a) Training set
A.6.2
True\Pred.
Maj.
Min.
True\Pred.
Maj.
Min.
Maj. Min.
56,785 17
78 82
Maj. Min.
56,772 12
92 86
(b) Validation set
(c) Test set
Calibrated Type-I and Type-II Error Control
To further assess the practical utility of empirical calibration, we select τ on the validation set to achieve a target Type-I error level of 0.01 and evaluate the resulting Type-II error on both validation and test sets. As shown in Table 10, the calibrated test-set Type-I errors remain tightly concentrated around the desired level across all datasets, confirming that the rule provides effective false-positive control. Likewise, the corresponding Type-II errors on the test set closely match those on the validation split. A complementary analysis using a target Type-II level of 0.1 in Table 11 shows the same pattern. Since the empirical Type-II error is discrete for finite minority samples, exact matching to 0.1 is generally impossible. We select the threshold that yields the closest attainable validation Type-II error to 0.1. The resulting test-set errors remain closely aligned with the validation-set calibrations, further highlighting the robustness of the projection-based score. Table 10: Type-I and Type-II errors after calibrating the threshold on the validation set to achieve Type-I = 0.01. Results are reported for both the validation and test sets across all datasets. Numbers in parentheses indicate the corresponding error count divided by the relevant class size. Dataset
Split
Type-I Error ↓
Type-II Error ↓
Credit Card (0.17%)
Val Test
0.0100 (568/56863) 0.0107 (607/56864)
0.1313 (13/99) 0.1020 (10/98)
Backdoor (2.44%)
Val Test
0.0100 (186/18600) 0.0084 (156/18600)
0.0193 (9/466) 0.0300(14/466)
Backdoor (0.20%)
Val Test
0.0100 (186/18600) 0.0084 (156/18600)
0.0536(25/466) 0.0536 (25/466)
Census (6.20%)
Val Test
0.0100 (561/56144) 0.0102 (572/56143)
0.6154 (2285/3713) 0.6330 (2351/3714)
Census (0.21%)
Val Test
0.0100 (561/56144) 0.0103 (577/56143)
0.6601 (2451/3713) 0.6828 (2536/3714)
22
Table 11: Type-I and Type-II errors after calibrating the threshold on the validation set to obtain the closest attainable empirical Type-II error to 0.1. Results are reported for both the validation and test sets. Numbers in parentheses indicate the corresponding error count divided by the relevant class size. Dataset
Split
Type-I Error ↓
Type-II Error ↓
Credit Card (0.17%)
Val Test
0.0455(2586/56863) 0.0460(2617/56864)
0.0909(9/99) 0.0918(9/98)
Backdoor (2.44%)
Val Test
0.0001(2/18600) 0.0003(5/18600)
0.0987(46/466) 0.0987(46/466)
Backdoor (0.20%)
Val Test
0.0007(13/18600) 0.0007(13/18600)
0.0987(46/466) 0.0901(42/466)
Census (6.20%)
Val Test
0.1650(9264/56144) 0.1647(9246/56143)
0.0999(371/3713) 0.1061(394/3714)
Census (0.21%)
Val Test
0.2506(14069/56144) 0.2494(14002/56143)
0.0999(371/3713) 0.1010(375/3714)
23