Confusion-Geometry Rebalancing for Long-Tailed Adversarial Training

Confusion-Geometry Rebalancing for Long-Tailed Adversarial Training Mengnan Zhao1 , Geyong Min3 , Lihe Zhang2 , Tianhang Zheng4 , Jie Cui1 1

2

School of Computer Science, Anhui University, China School of Information and Communication Engineering, Dalian University of Technology, China 3 Department of Computer Science, University of Exeter, UK 4 College of Computer Science, Zhejiang University, China

1

Top Head classes

Source classes

Adversarial training under long-tailed distributions suffers from a dual imbalance: the class imbalance skews the training objective toward head classes, and the adversarial inner maximization may further amplify this bias. Existing methods mitigate this issue by correcting class priors or adapting class-wise robust supervision, yet they treat each class in isolation and fail to identify which boundaries drive longtailed collapse. We propose a Confusion-Geometry Rebalancing method (CGRm) for long-tail adversarial training, a plug-in framework that leverages directed robust errors as training signals. CGRm leverages periodic robust evaluations to derive source-class loss weights, class-wise robust coefficients, and a directed confusion-geometry graph. The method then couples feedback-weighted robust optimization with graph-guided margin correction, thereby boosting the robustness of vulnerable classes and sharpening the critical boundaries that drive long-tailed performance degradation. Experiments on long-tailed benchmarks show that CGRm achieves consistent robust performance gains over existing methods, with ablations validating the contribution of each component. We provide the code in the supplement.

Robust Confusion Matrix (Row-normalized %)

Row percentage (%)

arXiv:2608.09688v1 [cs.LG] 10 Aug 2026

Abstract

Top Tail classes Predicted classes under PGD attack

Figure 1: Robust confusion matrix on CIFAR10-LT. Adversarial examples from tail classes are frequently misclassified to head classes, often in a directed manner, highlighting the importance of targeted margin correction.

Introduction

The vulnerability of deep neural networks to imperceptible adversarial perturbations has raised critical concerns about their deployment in safety-critical systems. Adversarial training (AT) is widely regarded as one of the most effective defenses against this threat. It is typically formulated as a min-max optimization problem (Zhao et al. 2024b, 2023, 2024a, 2026). Despite its empirical success, AT has been predominantly studied under an implicit assumption of balanced class priors. However, this assumption rarely holds in real-world scenarios, where sampled datasets are intrinsically long-tailed: a few head classes account for the majority of samples, while numerous tail classes are represented by very few instances (Du et al. 2024). When AT is applied directly to such skewed data, it introduces a two-fold imbalance: head classes not only dominate the training objective, but also disproportionately influence the adversarial decision boundaries during the inner maximization (Jiang et al. 2026; Qin et al. 2026; Li et al. 2025). Consequently, the learned model may achieve satisfactory average robust accuracy, yet its performance on tail classes remains notably inferior.

Recent studies improve the long-tailed AT from complementary perspectives. One line of work focuses on prior correction, adjusting logits, losses, or sample weights to counteract label imbalance (Ren et al. 2020; Wu et al. 2021; Yue et al. 2024). Another thread develops class-aware robust calibration, adapting perturbation budgets, regularization strengths, or class-specific weights based on each class’s robust behavior (Wei et al. 2023; Lee et al. 2024). Most recently, RobustLT shows that the adversarial inner maximization itself should also be class-dependent, reallocating perturbation radii and attack schedules across the imbalance spectrum (Zhang et al. 2026). Despite these advances, existing methods share a common limitation: each class’s robust error is treated in isolation, without considering where those errors are directed, as shown in Figure 1. This directional information matters in long-tailed AT. Under adversarial attack, an example from a source class is driven across the decision boundary toward different target classes. In long-tailed recognition, these errors are often

structured rather than random: tail-class examples tend to collapse into frequent classes, especially when their feature representations lie close in the adversarial space. Consequently, treating all errors from an under-robust class equally may waste robust supervision on irrelevant negatives, while scalar class feedback alone cannot identify which target classes actually absorb these errors. We argue that robust long-tailed training should consider not only whether a class is vulnerable, but also where its adversarial examples are directed and whether the resulting confusion is geometrically plausible. Motivated by this observation, we propose a ConfusionGeometry Rebalancing method (CGRm) for long-tail AT, a plug-in framework that explicitly accounts for directed robust errors. The method periodically evaluates the model under attack and collects three forms of feedback—class-wise robust accuracy, confusion counts, and adversarial feature centers—which jointly characterize each class’s vulnerability and its error direction. This feedback serves two complementary purposes. On one hand, it produces class-adaptive weights and robust coefficients that amplify the contribution and regularization of under-robust classes. On the other hand, it builds a directed confusion-geometry graph whose edges capture tail-to-head misclassifications that are both frequent and geometrically plausible; a margin loss is then applied exclusively to these selected pairs. In this way, CGRm rebalances not only how much each class is supervised, but also which boundaries are explicitly hardened. Our contributions are summarized as follows: • We formulate long-tailed AT as a directed robust errorcorrection problem and propose CGRm, modeling not only per-class vulnerability but also the target direction and geometric plausibility of robust confusions. • We introduce a dynamic feedback mechanism that decouples per-class gradient contribution from clean-robust regularization by separately updating loss weights and robust coefficients based on evolving robust-error statistics. • We design a confusion-geometry boundary correction module that builds a directed graph from confusion frequency, target-class headness, and adversarial feature affinity, and then applies a margin loss exclusively to systematic source-target confusions. • Extensive experiments on long-tailed benchmarks show that CGRm consistently outperforms existing methods.

2

Related Work

Adversarial training (AT). AT is typically formulated as a min-max optimization problem (Goodfellow, Shlens, and Szegedy 2015; Robey et al. 2024; Yue et al. 2023; Jia et al. 2024). Let fθ be a classifier, ℓ be the loss, and Sϵ (x) = {x′ : ∥x′ − x∥∞ ≤ ϵ} be the perturbation set around an input x. Standard AT minimizes the worst-case risk,   ′ min E(x,y)∼Dtrain ′max ℓ(fθ (x ), y) , (1) θ

x ∈Sϵ (x)

where the inner maximization is usually approximated by PGD (Madry et al. 2018). TRADES (Zhang et al. 2019)

decouples the learning objective into a standard classification term and a robust regularization term:   ′ E ℓ(fθ (x), y) + β ′max DKL (pθ (·|x)∥pθ (·|x )) , (2) x ∈Sϵ (x)

where pθ (·|x) denotes the predictive distribution and β governs the trade-off between clean accuracy and adversarial robustness. AWP (Wu, Xia, and Wang 2020) improves robust generalization by perturbing both inputs and model weights during training, expressed as   ′ min max E(x,y)∼D ′max ℓ(fθ+v (x ), y) , (3) θ

∥v∥≤γ∥θ∥

x ∈Sϵ (x)

where v denotes the weight perturbation and γ controls the perturbation magnitude. Long-tailed adversarial training. Unlike standard AT, long-tailed AT inherits two coupled biases: 1) the empirical risk is dominated by head classes, and 2) the adversarial examples generated around the learned decision boundary tend to further amplify head-class preferences. One family of methods addresses long-tailed learning as a prior-correction problem, adjusting losses, logits, or sample weights to reduce the dominance of class frequency in robust training (Cao et al. 2019; Ren et al. 2020). Given an adversarial example x′ ∈ Sϵ (x) and logits z = fθ (x′ ), RoBal (Wu et al. 2021) integrates AT with dynamic logit adjustment, exp(zy + ∆y ) LRoBal = − log P , j exp(zj + ∆j )

(4)

where ∆j is estimated from per-class robust confusion patterns during training. Instead of relying on dynamic confusion statistics, BSL (Yue et al. 2024) uses static class frequencies as the correction prior, formulated as nτyb exp(zy ) LBSL = − log P τb , i ni exp(zi )

(5)

where τb tunes the strength of class-prior correction. TAET (Wang et al. 2025) argues that static frequency correction may still overfit underrepresented or hard classes. It therefore introduces a two-stage adversarial equalization strategy. Stage I trains the model with standard cross-entropy to stabilize clean representations. Stage II optimizes adversarial examples with a hierarchical equalization loss. Similarly, REAT (Li, Xu, and Zhang 2023) down-weights the loss contribution of adversarial examples that are frequently predicted as head classes, encouraging the adversary to focus more on underrepresented categories. Beyond prior-correction, another line of work adapts robustness mechanisms on a per-class basis, such as perturbation radii and regularization coefficients. CFA (Wei et al. 2023) scales perturbation radii and regularization coefficients according to each class’s current robust accuracy rc : ϵc = (λ1 + rc )ϵ,

βc =

(λ2 + rc )β , 1 + (λ2 + rc )β

(6)

where ϵ and β are base values, and λ1 , λ2 are hyperparameters. This per-class scaling relies on a scalar accuracy summary per class but does not capture cross-class confusion.

UDR (Bui et al. 2022) instead abstracts the issue to the distribution level, formulating AT as Wasserstein distributional robustness. RobustLT (Zhang et al. 2026) tackles long-tailed AT through a perturbation rebalancing mechanism coupled with an adversarial iteration weighting schedule. For each class c, it defines the imbalance ratio Kc = nmax /nc and formulates the maximum perturbation budget as: ! √ α log Kc ϵ̄c = 1 − α + P nj p ϵ, (7) log Kj j N where nc denotes the sample count of class c, nmax = maxc nc , and the hyperparameter α governs the extent to which perturbation budgets are redistributed from head to tail classes. The method further introduces a progressive warm-up schedule to enable more effective application of these budgets. At epoch t, the actual perturbation radius and the corresponding PGD step size are given by   (t) t−1 ϵc (t) ϵc = min , 1 ϵ̄c , ηc(t) = η, (8) ρT ϵ with T denoting the total number of epochs, η the base step size, and ρ the warm-up ratio. Difference from existing methods. Existing methods treat all misclassifications from a vulnerable class equally, ignoring both which competing class the error goes to and whether that confusion is geometrically plausible in feature space. CGRm addresses these two missing dimensions. It jointly tracks robust errors and their source-target geometry: class frequency only initializes adversarial exploration, while subsequent corrections are driven by observed robust errors, their target classes, and the feature-space proximity of the confused classes.

3

Method

This work propose CRGm, which periodically evaluates the model under attack and translates the robust errors into optimization objectives: class-adaptive weights and coefficients that rebalance per-class loss contribution and regularization toward clean-robust consistency, and a directed graph that applies margin loss to systematic source-target confusions.

Optimization with Error Feedback To mitigate the adverse effects of uniformly assigned perturbation budgets in imbalanced data scenarios, we reuse Eqs. (t) (7) and (8) to calculate the attack budgets ϵc and step size (t) ηc . For a sample (x, y), we then generate xadv using such (t) (t) class-conditioned budget ϵy and step size ηy . Given these adversarial examples, CGRm optimizes three coupled signals: a prior-calibrated natural classification term for long-tailed recognition, a feedback-weighted robust consistency term for vulnerable source classes, and a confusiongeometry margin term for the target classes that absorb adversarial errors. The objective is expressed as LCGR = E [ℓbal (fθ (x), y)] + Lrob + λm Lcgm .

(9)

The first term denotes a balanced prior-calibrated classification term. Let πc = nc /N and bc = log(πc ). For logits z = fθ (x), we use exp(zy + by ) . (10) ℓbal (z, y) = − log PC j=1 exp(zj + bj ) Lrob reallocates the consistency between clean and adversarial predictions across classes. We introduce two independent (t) factors: the source-class weight wy modulates the overall gradient contribution from samples of class y, while the (t) class-wise robust coefficient βy controls the strength of the consistency constraint imposed on that class. This allows the model to emphasize an under-robust class and independently enforce a stronger clean-adversarial prediction agreement on it. For example, when adopting TRADES as the base learner, Lrob is expressed as h
i (11) Lrob = E wy(t) βy(t) DKL fθ (x) ∥ fθ (xadv ) . where DKL penalizes the discrepancy between clean and adversarial predictions. The confusion-geometry margin complements this source-class correction with target-class information. A low robust accuracy indicates that class y is vulnerable, but it does not identify which wrong class should be separated from y. The graph G(t) selects target classes that adversarial examples from y are frequently confused with and that are geometrically plausible competitors. CGRm then applies a margin penalty only to these selected targets:   X (t)   Lcgm = E(x,y) wy(t) Gyj m − zyadv + zjadv +  , j̸=y

(12) where m denotes the margin threshold, z adv = fθ (xadv ). At the beginning of training, wy = 1, βy = β, and Gy,j = 0. As training progresses, these variables are updated from robusterror statistics, which we define next.

Robust-Error Statistics The frequency prior is only a proxy for robust difficulty. Some rare classes may be clearly separable, whereas some medium-frequency classes may remain vulnerable due to feature overlap or semantic ambiguity. CGRm thus updates its training emphasis using the model’s own robust behavior. At an evaluation epoch t, we attack a monitoring split (subset of the training dataset) and compute the class-wise robust (t) (t) accuracy ac and robust confusion matrix M (t) , where Mij counts examples from class i predicted as class j under attack. We convert these measurements into a feedback score using two complementary signals. The first measures how far a class falls below the mean robust accuracy, C   1 X (t) (t) gc(t) = max ā(t) − a(t) , 0 , ā = a . (13) c C j=1 j The second measures how often adversarial examples from class c leave the correct class: (t)

e(t) c = 1 − PC

Mcc

(t) j=1 Mcj + ξ

,

(14)

where ξ is a negligible positive constant for numerical sta(t) bility, Mij counts examples from source class i that are predicted as target class j under attack. We combine the two (t) signals as sc , (t)

s(t) c =

(t)

gc + ec 1 C

(t) (t) j=1 (gj + ej ) + ξ

PC

.

(15)

Because robust evaluation can be noisy, especially for tail classes, we use an exponential moving average (t−1) s̃(t) + (1 − µ)s(t) c = µs̃c c .

(16)

(t)

Based on s̃c , we calculate the per-class loss weight in Eq. (11), which answers the question of how much examples from class c should contribute to the optimization objective: ! (t) 1 + s̃c (t) wc = clip , 0, wmax . (17) PC (t) 1 j=1 (1 + s̃j ) C Clipping prevents noisy tail estimates from producing excessive gradients, while normalization keeps the average weight stable across epochs. (t) Additionally, s̃c is used to determine the regularization strength for each class. A single global coefficient β applies the same clean-robust trade-off across all classes, which becomes suboptimal when robust difficulty varies unevenly: increasing β universally may hurt clean accuracy on easier classes, while decreasing it leaves harder classes underregularized. To address this, we adapt β per class based on the proposed robust-error score:   (t) clip 1 + s̃c /s̄(t) , 1, rmax   , (18) βc(t) = β · P (t) (t) C 1 j=1 clip 1 + s̃j /s̄ , 1, rmax C (t) (t) j s̃j . Unlike wc , which modulates a (t) class’s contribution to the overall objective, βc adjusts the

where s̄(t) = C1

P

Class-wise feedback identifies vulnerable source classes, yet it treats each class independently, overlooking the inherently relational nature of robust failure. An adversarial example from class i becomes harmful precisely because it crosses the decision boundary toward a specific target class j. In longtailed recognition, the most detrimental errors are often not random misclassifications, but rather tail classes being absorbed by neighboring head classes. To address this, CGRm constructs a directed confusion-geometry graph that selects source-target pairs requiring explicit margin correction. The graph integrates three complementary signals: empirical confusion probabilities, class-frequency bias, and featuregeometry affinity. The graph at epoch t is formulated as the element-wise product of these three terms: (t)

(t)

i ̸= j,

(t)

Ĝii = 0.

model fθ , base attack parameters (ϵ, η), base robust coefficient β, total epochs T . 2: Output: robust model fθ . (0) (0) 3: Initialize feedback scores s̃c = 1, class weights wc = (0) 1, robust coefficients βc = β, and graph G(0) = 0; 4: for epoch t = 1, . . . , T do (t) 5: Obtain class-wise perturbation budgets ϵc and step (t) sizes ηc from Eq. (8); 6: for mini-batch (x, y) sampled from D do (t) (t) 7: Generate xadv using ϵy and ηy ; 8: Compute prior-calibrated loss ℓbal by Eq. (10); 9: Compute the feedback-weighted robust loss Lrob (t−1) (t−1) with wy and βy by Eq. (11); 10: Compute the confusion-geometry margin Lcgm with G(t−1) by Eq. (12); 11: Update θ by minimizing LCGR in Eq. (9); 12: end for 13: if t is an evaluation epoch then 14: Attack a monitoring split to compute robust gaps (t) (t) gc , confusion errors ec , and smoothed feedback (t) scores s̃c by Eqs. (13)–(16); (t) (t) (t) (t) 15: Update wc and βc based on ec and s̃c ; (t) 16: Compute robust confusion P , headness h, and feature affinity A(t) by Eqs. (20)–(22); 17: Update the smoothed confusion-geometry graph G(t) by Eq. (23); 18: end if 19: end for (t)

Here, the row-normalized robust confusion probability Pij reflects the empirical tendency of adversarial examples from i to cross the decision boundary toward j. (t)

(t)

Mij

(t) k=1 Mik + ξ

Confusion-Geometry Boundary Correction

(t)

1: Input: long-tailed training set D, class counts {nc }C c=1 ,

Pij = PC

clean-robust trade-off within the robust term.

Ĝij = Pij hj Aij ,

Algorithm 1 CGRm Training

(19)

.

(20)

The headness score hj aims to penalize confusions that are more damaging under long-tailed imbalance: r nj hj = , (21) nmax where nc is the sample count of class c and nmax = maxc nc . (t) The geometry affinity Aij measures the geometric proximity between the two classes in the adversarial feature space. Let (t) mc denote the feature center of class c computed from adversarial examples, and define (t)

Aij =

1 (t) (t) ∥m̂i − m̂j ∥2 + ξ

,

(22)

where m̂ is ℓ2 -normalized. To reduce noise and maintain sparsity, we retain only the top-k outgoing edges for each source class and normalize

Table 1: Natural and robust accuracies of various base AT algorithms using ResNet. Bold digits denotes the best result. CIFAR10-LT

Methods

CIFAR100-LT

TinyImageNet-LT

Nat.(all)

Nat.(tail)

Rob.(all)

Rob.(tail)

Nat.(all)

Nat.(tail)

Rob.(all)

Rob.(tail)

Nat.(all)

Nat.(tail)

Rob.(all)

Rob.(tail)

REAT TAET RoBal BSL

69.59% 65.45% 72.43% 70.04%

63.79% 59.80% 68.14% 64.35%

31.41% 33.32% 33.47% 31.98%

21.91% 26.12% 26.71% 22.75%

45.49% 43.65% 44.88% 45.90%

43.35% 40.05% 43.79% 43.53%

17.54% 17.36% 18.61% 18.10%

16.59% 15.94% 18.10% 17.10%

37.91% 34.99% 37.12% 38.90%

35.42% 31.77% 35.95% 36.36%

12.80% 12.04% 12.28% 13.08%

11.33% 10.29% 11.56% 11.41%

PGD-AT + UDR + CFA + DAFA + RobustLT + Ours

56.33% 57.60% 57.08% 64.14% 56.71% 71.80%

46.27% 47.71% 47.23% 56.91% 46.81% 68.15%

26.28% 29.71% 29.76% 30.51% 30.88% 35.31%

13.19% 16.94% 16.31% 21.26% 18.60% 27.93%

43.03% 42.58% 43.31% 44.17% 46.09% 48.25%

38.91% 38.26% 38.60% 40.39% 42.14% 46.66%

15.40% 15.72% 17.58% 16.78% 17.73% 18.83%

13.34% 13.86% 15.54% 15.55% 16.01% 18.19%

34.80% 34.90% 37.08% 36.39% 36.67% 39.81%

30.56% 29.86% 32.00% 32.19% 31.89% 38.01%

9.16% 12.17% 11.96% 12.22% 12.26% 13.03%

7.25% 9.64% 9.55% 10.35% 10.08% 11.83%

TRADES + UDR + CFA + DAFA + RobustLT + Ours

54.48% 55.09% 53.32% 61.25% 56.44% 68.29%

43.91% 44.64% 42.43% 54.24% 44.01% 62.39%

30.13% 29.79% 30.45% 31.54% 31.38% 34.92%

16.66% 16.59% 16.41% 21.93% 18.57% 27.06%

43.57% 41.01% 43.07% 44.77% 45.50% 48.81%

38.51% 36.08% 37.52% 40.83% 40.80% 46.80%

20.07% 19.21% 21.16% 19.43% 20.68% 21.54%

17.25% 16.93% 18.18% 18.04% 18.25% 20.73%

35.41% 33.72% 37.60% 36.07% 36.46% 38.84%

30.18% 28.73% 32.48% 32.21% 31.66% 36.82%

13.85% 12.61% 14.82% 13.21% 13.78% 14.12%

10.78% 9.85% 11.61% 11.19% 10.82% 13.05%

AWP + UDR + CFA + DAFA + RobustLT + Ours

48.04% 43.59% 53.28% 52.67% 57.74% 68.19%

35.68% 30.28% 42.38% 42.66% 44.35% 64.71%

31.50% 28.00% 32.51% 32.45% 36.69% 40.70%

17.77% 14.22% 18.81% 22.45% 27.06% 36.44%

43.45% 43.58% 46.72% 44.50% 45.67% 50.30%

37.76% 38.45% 41.71% 39.30% 39.65% 48.67%

22.72% 21.08% 21.81% 23.03% 22.96% 24.22%

19.45% 18.25% 18.93% 20.42% 19.86% 23.85%

34.27% 31.54% 38.56% 35.03% 34.38% 39.99%

28.32% 25.79% 32.94% 29.78% 28.10% 37.65%

16.73% 14.10% 16.02% 16.99% 16.50% 18.43%

12.95% 10.78% 12.53% 13.94% 12.62% 17.09%

the nonzero entries to unit mean. The graph is then updated smoothly across epochs via exponential moving average: G(t) = µG(t−1) + (1 − µ)Ĝ(t) .

(23)

The target classes selected by Eq. (19) are those that (i) are frequently confused with adversarial examples from the source class, as measured by Eqs. (20) and (21), and (ii) serve as geometrically plausible competitors, as quantified by Eq. (22). Thus, this graph serves as a targeted description of long-tailed robust collapse. The training procedure is summarized in Algorithm 1.

4

Experiments

Experimental details Datasets. Following RobustLT, we apply the exponential long-tailed sampling strategy (Cao et al. 2019) to derive CIFAR10-LT, CIFAR100-LT, and TinyImageNet-LT (200 classes) from CIFAR10, CIFAR100 (Krizhevsky and Hinton 2009), and TinyImageNet (Le and Yang 2015), respectively. For C classes with imbalance ratio K, the number of training samples for class i is i

ni = nmax K − C−1 ,

(24)

where classes are sorted from head to tail. We set the imbalance ratio to 50 for CIFAR10-LT and 10 for CIFAR100-LT and TinyImageNet-LT. The test sets retain the original balanced class distributions. Compared Methods. We compare against advanced AT techniques, including conventional debiased AT methods (RoBal (Wu et al. 2021), REAT (Li, Xu, and Zhang 2023), BSL (Yue et al. 2024), TAET (Wang et al. 2025)) and plug-in methods (UDR (Bui et al. 2022), CFA (Wei et al. 2023), DAFA (Lee et al. 2024), RobustLT (Zhang et al. 2026)) that are applied on base AT algorithms such as PGD-AT (Madry

et al. 2018), TRADES (Zhang et al. 2019), and AWP (Wu, Xia, and Wang 2020). For fair comparison, each plug-in method shares the same learner, backbone, and attack setting as its corresponding baseline. Evaluation Metrics. We report natural and robust accuracy on all test classes (Nat.(all) and Rob.(all)), as well as on the 80% of classes with fewest training samples (Nat.(tail) and Rob.(tail)). Robust accuracy is measured under a 20-step ℓ∞ PGD attack with radius 8/255 and step size 2/255. As an additional sanity check, we also evaluate the final models with AutoAttack to rule out gradient masking. Implementation Details. The main experiments use ResNet18, PreActResNet (He et al. 2016), WideResNet (Zagoruyko and Komodakis 2016) as backbones, optimized with SGD (momentum 0.9, weight decay 5 × 10−4 ) and an initial learning rate of 0.1. All models are trained for 110 epochs. The perturbation radius is set to ϵ = 8/255. Adversarial examples are generated by 10-step PGD during training and evaluated by 20-step PGD at test time. The base AT configuration is kept unchanged. For CGRm-specific settings, the robust-error feedback and the confusion-geometry graph are updated every 10 epochs. For each source class, the graph retains the top-3 most relevant target classes. The resulting graph is used only in the margin regularization term with weight λm = 10/C, where C denotes the number of classes. All other CGRm-specific hyperparameters are fixed across datasets and listed in the appendix.

Comparative Results Tables 1-3 present a comprehensive comparison across datasets, backbones, and base learners. Compared with existing plug-in methods (e.g., UDR, CFA, DAFA, and RobustLT), CGRm consistently enhances both clean and adversarial performance, with the most stable gains observed on tail classes. For example, under ResNet-based PGD-AT

Table 2: Natural and robust accuracies of various AT algorithms using PreActResNet. CIFAR10-LT

Methods

CIFAR100-LT

TinyImageNet-LT

Nat.(all)

Nat.(tail)

Rob.(all)

Rob.(tail)

Nat.(all)

Nat.(tail)

Rob.(all)

Rob.(tail)

Nat.(all)

Nat.(tail)

Rob.(all)

Rob.(tail)

REAT TAET RoBal BSL

70.89% 66.54% 72.27% 70.49%

65.21% 61.24% 68.10% 64.88%

32.26% 33.73% 34.45% 32.15%

23.04% 26.88% 27.38% 22.85%

45.46% 43.32% 45.04% 45.13%

42.99% 39.42% 44.38% 42.89%

17.88% 18.04% 18.29% 17.60%

16.89% 16.35% 17.61% 16.75%

38.40% 35.12% 36.39% 38.31%

35.76% 31.70% 35.25% 35.96%

12.94% 12.19% 13.31% 13.31%

11.55% 10.44% 12.26% 12.00%

PGD-AT + UDR + CFA + DAFA + RobustLT + Ours

57.38% 59.62% 57.30% 66.05% 57.36% 73.03%

47.64% 50.34% 47.54% 59.31% 47.59% 68.59%

26.93% 29.69% 30.28% 31.36% 31.70% 35.08%

13.85% 16.99% 16.90% 22.30% 19.46% 28.11%

42.67% 42.24% 43.25% 43.69% 46.24% 47.75%

38.46% 37.85% 38.75% 40.08% 42.23% 46.35%

15.36% 15.76% 17.13% 16.91% 17.88% 18.39%

13.53% 13.95% 15.06% 15.72% 16.20% 17.76%

34.39% 34.87% 36.93% 37.13% 37.47% 39.68%

30.03% 30.20% 32.06% 32.94% 32.86% 37.75%

9.34% 10.59% 12.19% 12.49% 12.48% 12.68%

7.46% 8.56% 9.71% 10.81% 10.29% 11.71%

TRADES + UDR + CFA + DAFA + RobustLT + Ours

56.19% 56.44% 54.39% 65.06% 57.33% 69.15%

46.09% 46.38% 43.70% 58.35% 47.55% 62.59%

30.68% 30.38% 30.51% 32.13% 33.12% 35.28%

17.32% 17.15% 16.59% 22.51% 22.57% 26.03%

43.49% 40.78% 43.53% 44.62% 45.21% 49.09%

38.52% 35.95% 38.01% 41.09% 40.33% 46.67%

19.85% 18.82% 20.66% 19.69% 21.22% 21.23%

17.29% 16.48% 17.65% 18.31% 18.88% 20.21%

35.74% 33.82% 37.32% 36.61% 36.25% 39.78%

30.56% 28.50% 31.79% 32.59% 31.08% 38.08%

14.00% 12.09% 14.60% 13.46% 14.08% 13.96%

10.96% 9.21% 11.09% 11.43% 11.22% 13.28%

AWP + UDR + CFA + DAFA + RobustLT + Ours

50.78% 46.16% 54.86% 56.78% 45.82% 69.59%

39.05% 33.46% 44.26% 47.60% 32.91% 66.25%

32.18% 29.64% 32.84% 33.48% 31.52% 41.42%

18.50% 15.81% 19.21% 23.60% 18.05% 37.25%

43.53% 45.60% 47.06% 44.62% 45.87% 51.11%

37.86% 40.65% 41.93% 39.59% 39.95% 49.64%

23.04% 21.17% 22.19% 22.89% 23.20% 24.28%

19.74% 18.60% 19.25% 20.38% 20.29% 23.97%

33.99% 34.39% 38.63% 35.15% 34.65% 39.85%

28.20% 28.78% 32.85% 29.81% 28.16% 37.56%

16.96% 15.44% 16.35% 16.74% 16.70% 18.85%

13.31% 12.10% 12.90% 13.75% 12.97% 17.68%

on CIFAR10-LT, CGRm increases Nat.(tail)/Rob.(tail) from 46.81%/18.60% (RobustLT) to 66.15%/27.93%. On CIFAR100-LT with TRADES, it improves the best competitor’s Rob.(all)/Rob.(tail) from 20.68%/18.25% to 21.54%/20.73%, while also raising Nat.(all) from 45.50% to 48.81%. These gains indicate that the proposed method allocates robust learning pressure more effectively toward vulnerable classes and ambiguous boundaries. The improvement persists when switching from ResNet to Pre-ResNet and WideResNet. On Pre-ResNet, CGRm attains strong tail robustness. In the sole case where RobustLT marginally surpasses CGRm in overall robustness—TinyImageNet-LT with TRADES (14.08% vs. 13.96% Rob.(all))—CGRm still improves Rob.(tail) from 11.22% to 13.28%. On WideResNet, the benefits scale with model capacity: CGRm boosts PGD-AT on CIFAR10-LT from 31.54%/19.11% to 35.64%/26.41% in Rob.(all)/Rob.(tail), and AWP combined with CGRm reaches 43.72%/37.79% on CIFAR10-LT and 26.06%/25.85% on CIFAR100-LT. Collectively, these comparisons demonstrate that CGRm effectively addresses the core robustness failure induced by class imbalance. Table 4 compares different methods under more stronger and diverse attacks, C&W (Carlini and Wagner 2017) and AutoAttack (Croce and Hein 2020). Similarly, CGRm improves tail robustness across base learners and datasets, while preserving competitive all-class robustness. Under PGDAT, CGRm increases CIFAR10-LT C&W.(tail)/AA.(tail) from 20.51%/17.75% with RobustLT to 28.04%/25.09%, and improves TinyImageNet-LT from 12.10%/10.96% to 13.99%/12.29%. Similar gains are observed with TRADES, where CGRm obtains stronger tail robustness than RobustLT on CIFAR10-LT (24.99%/22.65% vs. 19.98%/18.44%) and CIFAR100-LT (20.26%/18.71% vs. 19.61%/18.31%). With AWP, CGRm achieves the most consistent improvement,

giving the best result on all reported metrics, including 40.35%/33.32% C&W accuracy and 37.37%/30.55% AutoAttack accuracy on CIFAR10-LT in all/tail classes. We note that a few all-class entries remain close or slightly favor competing methods, such as CIFAR100-LT AA.(all) under TRADES. However, the corresponding tail metrics are consistently improved by CGRm.

Ablation Study Table 5 ablates the four components of CGRm: the priorcalibrated classification term (PCC) in Eq. (10), feedback weight wy and class-wise coefficient βy in Eq. (11), and confusion-geometry margin in Eq. (9). 1) Removing PCC drops natural accuracy from 68.29% to 56.10% (all) and from 62.39% to 45.56% (tail), and tail robust accuracy from 27.06% to 20.49%. 2) The two feedback components are complementary. Removing wy lowers Rob.(tail) to 25.04%, indicating vulnerable source classes need stronger optimization contribution. Removing βy gives a similar drop to 24.96%, showing that loss reweighting alone is insufficient; consistency regularization must also be strengthened for under-robust classes. 3) Without the margin term, Rob.(all) slightly increases to 35.41%, but Rob.(tail) falls to 24.61%. Class-wise feedback alone improves average robustness, yet tail robustness requires correcting specific source-target confusions. The graph-guided margin thus trades a small average gain for substantial tail improvement. 4) Retaining PCC while disabling all three robust modules yields 64.60% Nat.(all), 58.79% Nat.(tail), 32.96% Rob.(all), and 22.31% Rob.(tail). Compared to this baseline, the full model improves all metrics, especially Rob.(tail) by 4.75 points.

5

Conclusion

This paper presents CGRm, a confusion-geometry rebalanced framework for AT under long-tailed distributions. Our

Table 3: Natural and robust accuracies of various base AT algorithms using WideResNet. CIFAR10-LT

Methods

CIFAR100-LT

TinyImageNet-LT

Nat.(all)

Nat.(tail)

Rob.(all)

Rob.(tail)

Nat.(all)

Nat.(tail)

Rob.(all)

Rob.(tail)

Nat.(all)

Nat.(tail)

Rob.(all)

Rob.(tail)

REAT TAET RoBal BSL

67.37% 66.89% 72.50% 67.65%

60.31% 60.40% 67.71% 60.61%

30.19% 34.51% 34.62% 30.64%

18.73% 25.25% 25.77% 19.09%

48.08% 45.58% 48.84% 47.92%

44.93% 40.79% 47.38% 45.05%

19.48% 19.07% 21.85% 20.11%

18.31% 17.12% 21.20% 18.84%

42.30% 38.26% 41.26% 41.65%

39.96% 33.85% 40.05% 36.90%

15.19% 13.16% 16.05% 14.71%

13.51% 10.87% 15.06% 12.17%

PGD-AT + UDR + CFA + DAFA + RobustLT + Ours

58.85% 59.44% 57.52% 65.06% 60.74% 73.83%

49.33% 50.06% 47.59% 57.39% 51.60% 68.59%

26.86% 28.11% 30.07% 29.60% 31.54% 35.64%

13.12% 14.96% 16.41% 18.71% 19.11% 26.41%

46.20% 45.47% 45.44% 46.87% 49.07% 50.95%

41.65% 40.90% 40.55% 43.23% 44.55% 48.85%

17.22% 17.29% 19.19% 18.06% 18.55% 19.54%

15.01% 15.20% 16.90% 16.89% 16.74% 18.78%

37.74% 39.23% 38.69% 39.78% 41.65% 44.14%

33.50% 34.45% 34.07% 35.89% 36.90% 42.45%

10.53% 12.83% 10.94% 13.80% 14.71% 14.77%

8.43% 10.34% 9.12% 11.81% 12.17% 13.74%

TRADES + UDR + CFA + DAFA + RobustLT + Ours

59.04% 58.73% 55.00% 65.58% 60.53% 68.37%

49.53% 49.09% 44.41% 58.48% 51.36% 61.54%

32.42% 31.36% 31.85% 33.54% 33.60% 35.81%

19.09% 18.07% 17.99% 22.94% 21.00% 25.16%

47.60% 44.95% 47.15% 48.52% 49.98% 52.71%

42.19% 39.85% 41.98% 44.84% 45.32% 49.20%

22.10% 20.32% 22.58% 21.98% 23.22% 23.60%

19.60% 17.91% 19.62% 20.45% 21.12% 22.04%

38.83% 40.85% 39.57% 39.15% 39.76% 42.84%

33.60% 35.90% 34.14% 34.91% 34.76% 41.24%

16.07% 13.67% 16.80% 15.91% 16.78% 17.07%

12.72% 11.19% 13.11% 13.45% 13.71% 16.15%

AWP + UDR + CFA + DAFA + RobustLT + Ours

53.39% 57.87% 57.81% 63.27% 68.91% 74.07%

42.35% 47.89% 47.77% 55.31% 56.60% 69.98%

31.21% 32.68% 34.47% 35.93% 38.70% 43.72%

16.96% 19.50% 20.70% 25.94% 27.07% 37.79%

50.03% 50.71% 50.50% 49.78% 50.83% 55.95%

44.60% 45.74% 45.25% 45.05% 45.16% 54.60%

24.81% 22.94% 24.31% 25.09% 25.34% 26.06%

21.29% 20.26% 21.30% 22.78% 22.64% 25.85%

39.10% 41.25% 39.96% 35.13% 38.01% 43.99%

32.86% 35.71% 33.48% 29.71% 31.29% 41.79%

14.87% 12.68% 15.80% 16.95% 19.29% 21.40%

11.26% 10.74% 11.65% 13.96% 15.31% 20.13%

Table 4: Robust accuracies against different attacks using WRN-28-10. Better results are bolded. CIFAR10-LT

Methods

CIFAR100-LT

TinyImageNet-LT

C&W.(all)

C&W.(tail)

AA.(all)

AA.(tail)

C&W.(all)

C&W.(tail)

AA.(all)

AA.(tail)

C&W.(all)

C&W.(tail)

AA.(all)

AA.(tail)

RoBal BSL

35.52% 32.62%

27.82% 25.76%

32.29% 29.64%

24.53% 22.59%

19.42% 19.23%

19.08% 18.25%

17.99% 17.34%

17.66% 16.56%

12.98% 15.05%

12.12% 13.50%

12.06% 13.30%

11.29% 11.96%

PGD-AT + UDR + DAFA + RobustLT + Ours

31.18% 28.93% 31.14% 32.70% 35.91%

17.95% 15.89% 20.97% 20.51% 28.04%

28.90% 26.95% 28.73% 29.97% 33.01%

15.48% 13.80% 18.64% 17.75% 25.09%

19.52% 17.47% 18.54% 18.88% 19.39%

17.40% 15.60% 17.69% 17.25% 19.04%

17.42% 15.83% 16.76% 16.91% 17.25%

15.53% 14.12% 16.08% 15.54% 17.03%

14.38% 13.31% 14.13% 15.03% 15.06%

11.61% 10.60% 12.06% 12.10% 13.99%

12.74% 11.93% 12.92% 13.21% 13.45%

10.18% 9.49% 11.09% 10.96% 12.29%

TRADES + UDR + DAFA + RobustLT + Ours

31.81% 30.92% 33.01% 32.69% 34.69%

18.39% 17.54% 22.45% 19.98% 24.99%

30.78% 29.54% 31.22% 31.22% 32.24%

17.29% 16.15% 20.66% 18.44% 22.65%

20.84% 18.98% 20.41% 21.64% 21.79%

18.35% 16.83% 19.20% 19.61% 20.26%

19.40% 17.45% 19.03% 20.21% 19.94%

17.14% 15.50% 18.01% 18.31% 18.71%

13.81% 12.23% 13.25% 14.19% 14.28%

10.91% 9.95% 11.15% 11.45% 12.76%

12.91% 11.09% 12.33% 13.06% 12.92%

10.18% 9.08% 10.34% 10.53% 11.54%

AWP + UDR + DAFA + RobustLT + Ours

30.15% 32.35% 35.37% 37.11% 40.35%

15.81% 19.05% 25.14% 28.38% 33.32%

29.15% 29.92% 32.99% 35.01% 37.37%

14.78% 16.41% 22.76% 24.70% 30.55%

22.06% 21.84% 23.54% 23.80% 24.44%

19.05% 19.34% 21.26% 21.22% 24.10%

20.67% 20.03% 21.37% 21.50% 22.02%

18.01% 17.84% 19.45% 19.23% 21.83%

12.48% 10.46% 16.54% 17.14% 18.33%

9.41% 8.38% 13.05% 13.41% 16.95%

11.23% 9.80% 11.67% 15.46% 16.24%

8.34% 7.63% 8.86% 12.15% 14.90%

Table 5: Ablation study of CGRm on CIFAR10-LT. ‘PCC’ denotes the balanced prior-calibrated classification term in (t) Eq. (10), where the feedback reweighting wy and class-wise (t) factor βy are given in Eq. (11), and λm in Eq. (9) indicates whether the confusion-geometry margin term is used. CIFAR10-LT

PCC wy(t) βy(t) λm

Nat.(all) Nat.(tail) Rob.(all) Rob.(tail) × ✓ ✓ ✓ ✓

✓ × × ✓ ✓

✓ × ✓ × ✓

✓ × ✓ ✓ ×

45.56% 58.79% 62.01% 61.51% 61.67%

32.38% 32.96% 34.78% 34.47% 35.41%

20.49% 22.31% 25.04% 24.96% 24.61%

✓

✓

✓

✓ 68.29% 62.39%

34.92%

27.06%

56.10% 64.60% 68.27% 67.78% 67.92%

key insight is that robust imbalance cannot be adequately captured by class-wise accuracy alone: adversarial errors tend to originate from vulnerable source classes and flow toward specific target classes, with the errors often arising from geometrically plausible tail-to-head confusions. To address this, CGRm converts the periodic robust evaluation into three components: source-class loss weights, class-wise robust coefficients, and a directed confusion-geometry graph. By integrating feedback-weighted robust optimization with graph-guided margin correction, CGRm effectively strengthens under-robust classes and hardens the decision boundaries most responsible for long-tailed robust collapse. Extensive experiments and ablation studies demonstrate that our design improves robust performance over existing long-tailed AT methods, confirming the effectiveness of both robust-error feedback and confusion-geometry boundary correction.

References Bui, A. T.; Le, T.; Tran, Q. H.; Zhao, H.; and Phung, D. 2022. A Unified Wasserstein Distributional Robustness Framework for Adversarial Training. In International Conference on Learning Representations. Cao, K.; Wei, C.; Gaidon, A.; Arechiga, N.; and Ma, T. 2019. Learning Imbalanced Datasets with Label-DistributionAware Margin Loss. In Advances in Neural Information Processing Systems. Carlini, N.; and Wagner, D. 2017. Towards Evaluating the Robustness of Neural Networks. In 2017 IEEE Symposium on Security and Privacy (SP), 39–57. IEEE. Croce, F.; and Hein, M. 2020. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In Proceedings of International Conference on Machine Learning (ICML), 1126–1135. Du, C.; Wang, Y.; Song, S.; and Huang, G. 2024. Probabilistic contrastive learning for long-tailed visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(9): 5890–5904. Goodfellow, I. J.; Shlens, J.; and Szegedy, C. 2015. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations. He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Identity mappings in deep residual networks. In European conference on computer vision, 630–645. Springer. Jia, X.; Zhang, Y.; Wei, X.; Wu, B.; Ma, K.; Wang, J.; and Cao, X. 2024. Improving fast adversarial training with priorguided knowledge. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(9): 6367–6383. Jiang, C.; Dong, M.; Wang, Z.; Gui, J.; Jia, J.; Tang, Y. Y.; and Kwok, J. T.-Y. 2026. Rethinking Frequency Modeling: Tail-Aware Dynamic Adversarial Training for Long-Tailed Robustness. IEEE Transactions on Information Forensics and Security. Krizhevsky, A.; and Hinton, G. 2009. Learning multiple layers of features from tiny images. Technical report, University of Toronto. Le, Y.; and Yang, X. 2015. Tiny imagenet visual recognition challenge. CS 231N, 7(7): 3. Lee, H.; Lee, S.; Jang, H.; Park, J.; Bae, H.; and Yoon, S. 2024. DAFA: Distance-Aware Fair Adversarial Training. In International Conference on Learning Representations. Li, G.; Xu, G.; and Zhang, T. 2023. Alleviating the Effect of Data Imbalance on Adversarial Training. arXiv:2307.10205. Li, Z.; Zhang, H.; Wang, J.; Chen, M.; Hu, H.; Yi, W.; Xu, X.; Yang, M.; and Ma, C. 2025. From head to tail: efficient black-box model inversion attack via long-tailed learning. In Proceedings of the Computer Vision and Pattern Recognition Conference, 29288–29298. Madry, A.; Makelov, A.; Schmidt, L.; Tsipras, D.; and Vladu, A. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations.

Qin, Y.; Zhou, Y.; Wang, J.; Xie, X.; and Qi, H. 2026. FedCART: Tackling Long-Tailed Distributions in Federated Adversarial Training via Classifier Refinement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 24557–24566. Ren, J.; Yu, C.; Ma, X.; Zhao, H.; Yi, S.; et al. 2020. Balanced Meta-Softmax for Long-Tailed Visual Recognition. In Advances in Neural Information Processing Systems, 4175– 4186. Robey, A.; Latorre, F.; Pappas, G.; Hassani, H.; and Cevher, V. 2024. Adversarial training should be cast as a non-zerosum game. In International Conference on Learning Representations, volume 2024, 57834–57854. Wang, Y.-H.; Guo, J.; Liu, A.; Wang, K.; Wu, Z.; Liu, Z.; Yin, W.; and Liu, J. 2025. TAET: Two-Stage Adversarial Equalization Training on Long-Tailed Distributions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15476–15485. Wei, Z.; Wang, Y.; Guo, Y.; and Wang, Y. 2023. CFA: ClassWise Calibrated Fair Adversarial Training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8193–8201. Wu, D.; Xia, S.-T.; and Wang, Y. 2020. Adversarial Weight Perturbation Helps Robust Generalization. In Advances in Neural Information Processing Systems, 2958–2969. Wu, T.; Liu, Z.; Huang, Q.; Wang, Y.; and Lin, D. 2021. Adversarial Robustness under Long-Tailed Distribution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8659–8668. Yue, X.; Mou, N.; Wang, Q.; and Zhao, L. 2024. Revisiting adversarial training under long-tailed distributions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 24492–24501. Yue, X.; Ningping, M.; Wang, Q.; and Zhao, L. 2023. Revisiting adversarial robustness distillation from the perspective of robust fairness. Advances in Neural Information Processing Systems, 36: 30390–30401. Zagoruyko, S.; and Komodakis, N. 2016. Wide residual networks. arXiv preprint arXiv:1605.07146. Zhang, H.; Yu, Y.; Jiao, J.; Xing, E. P.; El Ghaoui, L.; and Jordan, M. I. 2019. Theoretically Principled Trade-off between Robustness and Accuracy. In International Conference on Machine Learning, 7472–7482. Zhang, L.; Guo, Y.; Li, Y.; Shi, J.; and Liu, X. 2026. Taming the Long Tail: Rebalancing Adversarial Training via Adaptive Perturbation. arXiv:2506.13395. Zhao, M.; Zhang, L.; Kong, Y.; and Yin, B. 2023. Fast Adversarial Training with Smooth Convergence. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 4720–4729. Zhao, M.; Zhang, L.; Kong, Y.; and Yin, B. 2024a. Catastrophic Overfitting: A Potential Blessing in Disguise. In European Conference on Computer Vision, 293–310. Zhao, M.; Zhang, L.; Wang, B.; Zheng, T.; Zhong, H.; and Min, G. 2026. Mitigating Error Amplification in Fast Adversarial Training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Zhao, M.; Zhang, L.; Ye, J.; Lu, H.; Yin, B.; and Wang, X. 2024b. Adversarial Training: A Survey. arXiv:2410.15042.

Related documents

• Improving Clean Accuracy via a Tangent-Space Perspective on Adversarial Training#147018
• Understanding and Improving Continuous Adversarial Training for LLMs via In-context Learning Theory#13012
• Information-Geometric Inverse Distillation for Enhancing Adversarial Transferability#1013401
• Guided Adversarial Robust Transfer Learning with Source Mixing#999617
• DUA-D2C: Dynamic Uncertainty Aware Method for Overfitting Remediation in Deep Learning#972887