1
The Risk Shadow of Principal Component Analysis: When 99.9999% Variance Preservation Causes Catastrophic Decision Errors
arXiv:2606.14533v1 [cs.LG] 12 Jun 2026
Hamidou Tembine
Abstract—Principal Component Analysis (PCA) preserves variance, not the information needed to detect rare catastrophic events. This paper proves the existence of a Risk Shadow: PCA can retain over 99.9999 percent of total variance while completely erasing all signal about rare, high-impact failures. When this happens, even the best possible classifier operating on the PCA representation reduces to a constant predictor. The root cause is a fundamental mismatch between variance maximization and tail risk awareness. To break the shadow, we introduce Expectile PCA (ExPCA) and Tail-Preserving PCA (TP-PCA), two methods that reweight the data covariance toward high-impact events. We prove theoretically that ExPCA strictly outperforms PCA in retaining rare-event information, and we validate our claims on synthetic data and a real-world credit card fraud detection benchmark. Our results call for a fundamental rethinking of variance-based dimensionality reduction in high-stakes decisions. Index Terms—Principal component analysis, expectile PCA, tail risk, rare-event learning, risk-aware dimensionality reduction, classification, anomaly detection, imbalanced learning.
I. I NTRODUCTION Few ideas have influenced modern data assimilation, statistics, machine learning, signal processing, econometrics, computer vision, quantitative finance, genomics, and machine intelligence as deeply as Principal Component Analysis (PCA). Since the pioneering works of [1], [2], PCA has become the canonical method for dimensionality reduction because it solves a fundamental geometric problem: among all linear subspaces of a prescribed dimension, it finds the one that minimizes mean-square reconstruction error. It identifies the directions of maximal variance and produces the optimal lowdimensional representation in the L2 sense. The success of PCA is difficult to overstate. It is routinely deployed as a preprocessing layer before classification, clustering, anomaly detection, forecasting, compression, reinforcement learning, and deep neural architectures. In countless applications, reducing dimensionality through PCA improves computational efficiency, suppresses noise, mitigates overfitting, and facilitates visualization. Its computational scalability, and spectral optimality have made PCA one of the most widely used algorithms in the history of data engineering and data science. Yet PCA was never designed to solve a decision problem. This distinction is often overlooked. PCA optimizes a geometric objective defined entirely through second-order moments of the data Department of EECS, School of Engineering, UQTR, Canada. Learning and Game Theory Laboratory (LnG Lab), TIMADIE. Contact: [email protected]
distribution. Classification systems, however, are evaluated through decision losses. Medical diagnosis is judged by missed cancers, not reconstruction error. Fraud detection is judged by undetected fraud, not variance explained. Autonomous driving is judged by collisions avoided, not spectral fidelity. In these settings, the quantity that matters operationally is not variance preservation but decision risk. The central question therefore becomes: Can a representation that is geometrically optimal be catastrophically suboptimal for decision making? This paper shows that the answer is unequivocally yes. The reason is structural. Variance and decision risk are fundamentally different objects. Variance is an unconditional second-moment quantity. Misclassification cost is a conditional decision-theoretic quantity that depends jointly on the data distribution, the label distribution, the operational loss function, and the user’s tolerance to errors. Nothing in the PCA objective incorporates labels, costs, asymmetries, rare events, or downstream consequences. There is no theoretical mechanism forcing directions of maximal variance to coincide with directions that minimize decision risk. This observation immediately raises a deeper question. If PCA ignores the user’s loss function, can it nevertheless preserve the information required for optimal decisions? The answer is generally negative. Indeed, the dominant variance directions of a distribution need not contain any information about the target variable. The information necessary for classification may reside entirely in directions carrying negligible variance. When such a configuration occurs, PCA faithfully preserves the geometry of the data while simultaneously destroying the information required for the decision task. The resulting representation may appear statistically excellent according to classical variance-retention criteria while being operationally useless. This phenomenon is particularly severe in rare-event learning. Across modern high-stakes systems, the most consequential events are often extremely infrequent. Financial crashes [31] occupy tiny portions of market trajectories. Fraudulent transactions constitute a minute fraction of all payments. Early-stage diseases may appear as weak perturbations hidden beneath dominant physiological variability [30]. Safety-critical failures in autonomous systems arise from rare combinations of environmental conditions [32]. In all these situations, the signals carrying the highest operational value frequently contribute only a negligible fraction of the total variance. As a consequence, maximizing explained variance can become
2
fundamentally misaligned with minimizing catastrophic errors. The directions most relevant for decision-making are often those least relevant for variance maximization. The implications are profound. A representation may preserve virtually all observable variance and yet completely eliminate the information needed to detect the events that matter most. We provide an example where PCA retains more than 99.9999% of the total variance while erasing all information regarding a critical rare class. After projection, the Bayesoptimal classifier collapses to a constant predictor despite the apparent preservation of nearly the entire dataset geometry. The representation remains spectrally optimal but becomes decision-theoretically worthless. This reveals a previously unrecognized failure mode of variance-based dimensionality reduction. We refer to this phenomenon as the Risk Shadow. The Risk Shadow occurs when a representation preserves nearly all global variance while eliminating the information required to control rare-event decision risk. Under the Risk Shadow, nominal performance indicators can remain deceptively strong. Overall accuracy may approach unity because the dominant class is correctly predicted almost everywhere. Yet tail risk, expected catastrophic loss, and high-cost error exposure can increase dramatically. In our analytical benchmarks, a representation preserving more than 99.9999% of the total variance produces an increase of approximately 890% in expectilebased misclassification risk relative to a risk-aware alternative. The significance of this result extends beyond PCA itself [3], [4]. Since its introduction more than a century ago, PCA has evolved into one of the most influential paradigms in statistical learning, inspiring a vast ecosystem of extensions designed to address nonlinearity, robustness, sparsity, probabilistic uncertainty, structured data, functional observations, streaming environments, manifold constraints, and high-dimensional inference. Over the last decades, numerous variants have been proposed, including Kernel PCA, Sparse PCA, Robust PCA, Probabilistic PCA, Functional PCA, Tensor PCA, and many other extensions. Despite their methodological differences, these approaches remain rooted in a common paradigm: the representation is selected through a geometric criterion, after which a downstream decision system is trained on the compressed coordinates. The underlying optimization objective remains disconnected from the operational loss ultimately used to evaluate performance. A first major line of research sought to overcome the intrinsic linearity of PCA through nonlinear embeddings. Kernel PCA generalized principal component analysis to reproducing kernel Hilbert spaces, enabling nonlinear feature extraction while preserving the spectral foundations of the original method [5]. Subsequent developments included manifold learning and spectral approaches such as Isomap [6], Locally Linear Embedding (LLE) [7], Laplacian Eigenmaps [8], Hessian Eigenmaps [9], Diffusion Maps [10], and various nonlinear principal manifold methods. A second research direction focused on interpretability and high-dimensional statistics. Sparse PCA introduced sparsity constraints on loading vectors to improve interpretability and variable selection [11]–[13]. Related formulations include structured sparse PCA, group sparse PCA, elasticnet PCA, nonnegative PCA, and semidefinite relaxations for
sparse component estimation. A third family addressed robustness against noise, contamination, and outliers. Robust PCA methods replace classical covariance estimation by robust alternatives or explicitly decompose data into low-rank and sparse components [14]–[16]. Numerous variants subsequently emerged, including M-estimator PCA, projection-pursuit PCA, L1-PCA, outlier-resistant PCA, and low-rank-plus-sparse decomposition frameworks. Probabilistic formulations constitute another major branch. Probabilistic PCA (PPCA) introduced a latent-variable generative model interpretation of PCA [17], later extended through Bayesian PCA [18], Variational PCA, Factor Analysis, Bayesian nonparametric latent-factor models, and probabilistic matrix factorization techniques. Functional data analysis generated Functional PCA (FPCA), where observations are viewed as random functions rather than finite-dimensional vectors [19]. This framework has become central in longitudinal analysis, biomedical signals, climate modeling, and spatiotemporal systems. Extensions include Multilevel FPCA, Dynamic FPCA, Sparse FPCA, and Kernel FPCA. As multidimensional datasets became prevalent, tensorbased generalizations emerged. Tensor PCA, multilinear PCA, higher-order singular value decomposition (HOSVD), Tucker decompositions, CANDECOMP/PARAFAC models, multilinear subspace learning, and low-rank tensor recovery methods were developed to preserve multiway structure without vectorization [20], [21]. The growth of large-scale data motivated incremental and online variants. Incremental PCA [22], online PCA [23], stochastic PCA, streaming PCA, distributed PCA, randomized PCA [24], sketching-based PCA, and communication-efficient PCA were developed to cope with modern computational constraints. Further extensions include Independent Component Analysis (ICA) inspired spectral methods [25], Principal Curves and Principal Surfaces [26], Local PCA, Mixture PCA, Hierarchical PCA, Multiscale PCA, Graph PCA, Geodesic PCA, Riemannian PCA, Grassmannian PCA, Complex PCA, Quaternion PCA, Dynamic PCA, Evolutionary PCA, Contrastive PCA [27], Fair PCA [28], Deep PCA, Autoencoder-based PCA, Supervised PCA [29], Discriminative PCA, Canonical PCA, Generalized PCA, Weighted PCA, Missing-Data PCA, Compositional PCA, and numerous domain-specific adaptations. The present work challenges this paradigm. Rather than asking which subspace best reconstructs the data, we ask which subspace best preserves the information required to minimize high-impact decision errors. This shift transforms dimensionality reduction from a geometric compression problem into a decision-theoretic optimization problem. To investigate this question, we establish a framework connecting dimensionality reduction, information preservation, tail risk, and misclassification costs. We prove that variance maximization and risk minimization can be structurally incompatible. We derive exact conditions under which PCA completely erases rareevent information despite preserving essentially all variance. We then introduce risk-aware alternatives based on expectile principles that directly incorporate tail-sensitive objectives into the representation-learning stage. Our analysis reveals a fundamental principle: A representation should not be judged solely by
3
how much variance it preserves, but by how much decision-relevant information it retains. For high-stakes machine intelligence systems, variance is not the only objective. Decisions and consequences of decisions are. Diagram 1 maps a structural comparison between standard PCA and the tail-risk-aware exp2PCA. It illustrates how standard variance maximization inadvertently erases critical rare-event signals to cause catastrophic tail-risk, and how the proposed alternative preserves those signals to secure robust classification. PCA’s Responsibility in Increasing Misclassification Risk exp2PCA SUCCESS PATH exp2PCA Tail-Risk Aware Minimize eτ (Ch(P ⊤ X),Y )
High-Dim. Data X PCA FAILURE DOMAIN Standard PCA Variance ↑ 99.9999% Retained Why? PCA Subspace (The Risk Shadow) Information Erasure I(Y ; Z) = 0
Optimal Classifier ≡ Constant Predictor f (X) ≡ a∗
Tail Risk R0.99 ≫ 0 e.g., 50.0, 66.9
Risk-Aligned
Variance ̸= RiskBreaks the Shadow Subspace Rare events reside in Preserves Rarediscarded Event Info subspace SR The Paradox: High Nominal Accuracy But Catastrophic Tail Risk
Optimal Classifier Detects Rare Class CF N = 100 Avoided
Tail Risk R0.99 ≈ 0 e.g., 0.38, 0.00
PCA: Retains nearly all global variance → erases rare-event signals → forces constant prediction → skyrockets tail risk. exp2PCA: Directly optimizes decision-theoretic tail risk → structurally preserves critical low-variance signals.
Fig. 1. PCA can retain 99.9999 % of the global variance and be responsible of 890% increase in expectile risk of misclassification cost.
Related Work on Risk-Aware and Decision-Aware Dimensionality Reduction: The relationship between dimensionality reduction and downstream decision risk has attracted increasing attention over the last three decades. Nevertheless, most existing approaches remain rooted in geometric, statistical, or probabilistic objectives rather than directly optimizing operational decision losses. A first line of research addresses robustness to contamination and adversarial observations. Robust PCA (RPCA) seeks to recover low-rank structure in the presence of sparse corruptions, gross outliers, or missing observations [14], [15], [47]. These methods significantly improve resilience against data contamination and have become foundational in computer vision, anomaly detection, and signal processing. However, their notion of robustness is fundamentally statistical rather than operational: the objective remains the recovery of an underlying low-rank geometric representation. RPCA does not distinguish between errors that are operationally benign and errors that may trigger catastrophic decision consequences. A second direction focuses on interpretability and highdimensional variable selection. Sparse PCA and its numerous extensions introduce sparsity-inducing penalties to obtain principal components involving only a limited subset of variables [11]–[13]. Although these methods improve explainability and facilitate scientific interpretation, the optimization criterion remains variance maximization under structural constraints. The downstream cost associated with misclassification, false
alarms, or rare-event failures does not enter the representationlearning objective. A third family incorporates supervision into dimensionality reduction. Sufficient Dimension Reduction (SDR), including Sliced Inverse Regression (SIR) [48], Sliced Average Variance Estimation (SAVE) [34], Principal Hessian Directions [35], and related inverse-regression techniques, aims to preserve the conditional distribution of the response variable. More recent developments include supervised PCA [29], discriminative PCA, partial least squares, and task-guided latent-variable models. While these methods leverage label information, they typically target conditional sufficiency, correlation structure, or prediction accuracy. They do not explicitly account for asymmetric loss functions, rare-event costs, tail-risk exposure, or user-specific operational penalties. Moreover, many classical SDR approaches rely on restrictive assumptions such as elliptically distributed predictors, linearity conditions, or covariance invertibility. A fourth line of research studies rare events and extreme outcomes directly. Extreme Value Theory (EVT) provides a framework for modeling tail behavior, rare exceedances, and catastrophic events [36], [37], [49]. EVT-based methods have found applications in finance, cybersecurity, climate science, and reliability engineering. Similarly, importanceweighting techniques, cost-sensitive learning, focal losses, and imbalance-aware classification strategies attempt to compensate for rare-event underrepresentation [39], [40], [50]. However, these approaches generally operate after the representation has already been constructed. The feature space itself is typically inherited from variance-driven, likelihood-driven, or task-agnostic dimensionality reduction procedures. The representation may already have discarded the very directions required to identify rare but consequential events. A fifth body of work investigates coherent and tail-sensitive risk measures. Value-at-Risk (VaR), Conditional Value-at-Risk (CVaR), spectral risk measures, distortion risk measures, and expectiles have become central tools in modern risk management [41]–[43], [51]. In particular, expectiles possess attractive theoretical properties including coherence under suitable conditions, elicitability, differentiability, and strong sensitivity to tail outcomes [43], [51]. Expectile-based methods have been widely studied in econometrics, finance, actuarial science, and risk forecasting. Nevertheless, their use has been largely restricted to estimation, regression, forecasting, and portfolio optimization. To the best of our knowledge, expectiles have not been systematically employed as the primary objective for spectral representation learning in the context of dimensionality reduction. Several recent developments have also attempted to align representations with downstream tasks through metric learning, information bottleneck principles, contrastive objectives, mutual-information maximization, and deep representation learning [44]–[46]. While these methods introduce task awareness, their objectives remain fundamentally different from direct minimization of tail-sensitive decision costs. Information preservation, contrastive separation, and predictive sufficiency do not necessarily imply protection against catastrophic classification errors. Indeed, our theoretical results demonstrate that
4
representations preserving nearly all information according to conventional criteria may still erase the specific directions responsible for controlling extreme operational risk. These literatures reveal a striking gap. Existing approaches have addressed robustness, sparsity, supervision, interpretability, nonlinear structure, information preservation, and rareevent modeling. Yet the representation itself is almost always selected through a geometric, probabilistic, informationtheoretic, or predictive objective. The user’s actual loss function enters only indirectly, if at all. A fundamental question remains unresolved: Can a representation that is optimal according to variance, reconstruction fidelity, likelihood, information preservation, predictive sufficiency, or geometric structure be provably incompatible with the minimization of catastrophic decision risk? The present work addresses this question. We establish a decision-theoretic framework showing that variance preservation and tail-risk minimization can be structurally incompatible objectives. We further demonstrate that a representation retaining virtually all global variance may nevertheless erase the information necessary to identify rare, high-cost events. This motivates a new class of risk-aware spectral methods in which dimensionality reduction is aligned directly with the operational decision loss rather than with a surrogate geometric criterion. Contributions: 1) We establish a multiclass information erasure theorem showing that PCA can induce complete loss of rareevent signal while preserving nearly all variance (Theorems 1 and 2). 2) We characterize the Risk Shadow phenomenon and prove that high nominal accuracy can coexist with total failure of rare-event detection (Theorem 3). 3) We introduce Expectile PCA (ExPCA) as a tail-riskaware alternative and prove its strict improvement over PCA in retaining rare-event information (Theorem 4). 4) We prove a geometric misalignment theorem (Theorem 5) showing that ExPCA and PCA subspaces necessarily differ when the tail-weighted covariance couples the leading PCA subspace with its complement. 5) We provide complete proofs, synthetic validation, and a real-world credit card fraud benchmark demonstrating that ExPCA achieves high tail-event detection while PCA fails. TABLE OF N OTATIONS II. P ROBLEM F ORMULATION
d X j=1
λj vj vj⊤ ,
Symbol X ∈ Rd Y ∈ {1, . . . , K} πk = P(Y = k) µ = E[X] Σ = Cov(X) λj , v j P Pr = rj=1 vj vj⊤ Qr = Id − Pr Zr = Pr X zj I(·; ·) Θr ℓτ (u) = |τ − 1{u<0} |u2 eτ (Lf ) Cab YR ⊆ Y Rτ (P, h) eτ (Ch(P ⊤ X),Y ) τ Στ ess sup(L)
=
Description Random feature vector (observable data) Multiclass label Class prior probability Population mean vector Population covariance matrix j-th eigenvalue and eigenvector of Σ (λ1 ≥ · · · ≥ λd ) Rank-r PCA projection matrix Projection onto discarded subspace Compressed representation after PCA Latent coordinate along vj Shannon mutual information Risk Shadow regime (zero mutual information, high nominal accuracy) Expectile loss function τ -expectile risk of classifier f Cost of predicting class a when true class is b Set of rare, high-impact classes represent the joint risk objective function of exp2PCA. Expectile parameter (τ → 1− targets extreme tails) Risk-weighted covariance matrix used in ExPCA Essential supremum of random variable L
where the eigenvalues are sorted as λ1 ≥ λ2 ≥ · · · ≥ λd > 0, and {vj }dj=1 forms an orthonormal basis of Rd . For a userdefined retained-variance threshold ρ ∈ (0, 1), the target truncation rank r ∈ {1, . . . , d − 1} is uniquely determined by: ( ) Pm j=1 λj r = min m ∈ {1, . . . , d} : Pd ≥ρ . (2) j=1 λj Let Ur = {P ∈ Rd×r : P ⊤ P = Ir } denote the Stiefel manifold of semi-orthogonal matrices. The canonical semiorthogonal matrix aligned with the dominant variance is Pr = [v1 , . . . , vr ] ∈ Ur . The corresponding orthogonal Pr projection operator on Rd is defined as Πr = Pr Pr⊤ = j=1 vj vj⊤ , and its orthogonal complement operator is Π⊥ r = Id − Πr . The classical principal component representation is the compressed random vector Zr = Pr⊤ X ∈ Rr . The discarded noise subspace is explicitly designated as SR = span{vr+1 , . . . , vd } = Im(Π⊥ r ). A. Latent-Factor Model We assume that the observable data satisfies a structural latent-factor representation: X =µ+
Let (Ω, F , P) be a complete probability space. Let X : Ω → Rd be a square-integrable random vector with mean vector µ = E[X] ∈ Rd and a symmetric, trace-class covariance matrix Σ = E[(X −µ)(X −µ)⊤ ] ∈ Rd×d . By the spectral theorem for symmetric compact operators, the orthogonal decomposition of Σ is given by: Σ=
TABLE I S UMMARY OF KEY NOTATIONS USED THROUGHOUT THE PAPER .
(1)
d X
zj vj ,
(3)
j=1
where the latent coordinates zj = vj⊤ (X − µ) are mutually independent, centered random variables satisfying E[zj ] = 0 and E[zj2 ] = λj . No parametric distributional assumptions (e.g., joint Gaussianity) are imposed on these factors. Let Y ∈ {1, . . . , K} represent a categorical class label generated via a deterministic, Borel-measurable mapping: Y = g {zj }j∈M , (4)
5
where M ⊆ {r + 1, . . . , d} is a non-empty index set. Under this setting, Y is conditionally independent of the principal subspace given the discarded coordinates; thus, the descriptive information regarding Y resides exclusively within the minor subspace SR . This represents the absolute worst-case configuration for variance-maximization paradigms, typically modeling regimes where rare, high-stakes events are structurally masked by heavy background noise.
top r eigenvectors of Σw . This amplifies the influence of rareevent samples. Definition 4 (Expectile PCA / exPCA). Expectile PCA incorporates geometric tail-awareness by minimizing the τ expectile of the squared Euclidean reconstruction error distribution: 2 . (8) PexPCA = arg min eτ X − P P ⊤ X P ∈Ur
B. Dimensionality Reduction and Decision-Theoretic Frameworks To formalize paradigms that extract information from X, we introduce the functional definitions of statistical expectiles, followed by the formulations of traditional, robust, and our proposed decision-directed dimensionality reduction methods. 1) The Expectile Functional: Before evaluating subspace models, we define the asymmetric tail-risk operator used throughout this work. Definition 1 (Expectile of a Random Variable). Let W ∈ L2 (Ω, F, P) be a square-integrable scalar random variable. For an asymmetry tail parameter τ ∈ (0, 1), the τ -expectile functional eτ (W ) is defined as the unique minimizer of an asymmetrically weighted quadratic loss: eτ (W ) = arg min E ℓτ (W − t) , (5) t∈R
where the loss function ℓτ (u) is parameterized by: ( τ u2 if u ≥ 0, 2 ℓτ (u) = τ − 1{u<0} u = 2 (1 − τ )u if u < 0.
(6)
exPCA prioritizes finding directions that robustly bound the worst-case geometric reconstruction distortions for outlying data vectors. 3) Supervised Subspace Paradigm: exp2PCA: To couple dimensionality reduction directly with a downstream operational task, we formalize the decision-theoretic risk space. Let C ∈ RK×K be an asymmetric misclassification cost ≥0 matrix satisfying Caa = 0 and mina̸=b Cab > 0 for all a, b ∈ {1, . . . , K}. Let H = {h : Rr → {1, . . . , K}} define the class of Borel-measurable operational classifiers acting on the compressed space. Given a choice of embedding matrix P ∈ Ur and a classifier h ∈ H, the decision loss is a discrete random variable L(P, h) = Ch(P ⊤ X),Y . Definition 5 (Expectile Misclassification Cost PCA / exp2PCA). The exp2PCA framework defines the optimal rank-r representation as the joint minimizer of the τ -expectile of the downstream classification cost: (Pexp2 , hexp2 ) = arg min eτ Ch(P ⊤ X),Y . (9) P ∈Ur , h∈H
Remark 1. As the parameter tracks the extreme upper tail (τ → 1), the expectile functional eτ (W ) converges asymptotically to the essential supremum of the random variable’s distribution (ess sup W ). This property renders it a highly coherent, non-linear index of worst-case risk exposure. See [51] for details. 2) Unsupervised Subspace Paradigms: We consider an arbitrary compression matrix P ∈ Ur mapping to a lowdimensional coordinate representation Z = P ⊤ X ∈ Rr . The geometric reconstruction of X within the original space is given by X̂ = P Z = P P ⊤ X. Definition 2 (Standard Unsupervised PCA). Standard PCA identifies a rank-r subspace that minimizes the expected L2 reconstruction error: PPCA = arg min E X − P P ⊤ X P ∈Ur
2
.
(7)
By the Eckart–Young–Mirsky theorem, the optimal solution PPCA is spanned precisely by the r leading eigenvectors Pr = [v1 , . . . , vr ] of the covariance matrix Σ. Definition 3 (Tail-Preserving PCA (TP-PCA)). Given i.i.d. data {(Xi , Yi )}ni=1 , let YR ⊂ {1, . . . , K} be the set of rare, high-impact classes (e.g., fraud, disease). Define weights wi = 1 + P α1{Yi ∈YR } with Pαn > 0. We compute weighted mean n µw = ( i=1 wi Xi )/( j=1 wj ) and weighted covariance Σw =
−µ )(Xi −µw )⊤ i=1 wi (X Pi n w j=1 wj
Pn
. TP-PCA projects onto the
By substituting geometric reconstruction criteria with a direct minimization of the decision penalty’s tail risk, exp2PCA explicitly preserves crucial classification signals, even when they reside entirely along the minor principal components of the data vector. C. Fundamental Limits of PCA for Rare-Event Classification 1) Complete Information Erasure: Theorem 1 (Information Erasure). Under the structural latent-mapping factor model where Y = g({zj }j∈M ) and M ⊆ {r + 1, . . . , d}, the following statements hold true: 1) Y ⊥⊥ Pr X, 2) The mutual information vanishes: I(Y ; Pr X) = 0, 3) P(Y = k | Pr X) = P(Y = k) = πk a.s. for every category k ∈ {1, . . . , K}. Proof. Observe that the PCA Prreconstruction can be explicitly written as Pr X = Pr µ + j=1 zj vj . The generated sub-σalgebra satisfies σ(Pr X) ⊆ σ(z1 , . . . , zr ). By the model construction, the label σ-algebra satisfies σ(Y ) ⊆ σ({zj }j∈M ). Since M ∩ {1, . . . , r} = ∅, the mutual independence of the complete latent coordinate set {zj }dj=1 implies that σ(z1 , . . . , zr ) ⊥⊥ σ({zj }j∈M ). It follows directly that Y ⊥ ⊥ Pr X. Statements (2) and (3) are immediate structural consequences of this independence.
6
2) Bayes Expected Cost Collapse: Theorem 2 (Bayes Collapse). Let πk = P(Y = k) denote the prior class probabilities. For any measurable classifier f : Rr → {1, . . . , K} operating on the standard PCA projection space, the infimum expected cost collapses: inf E Cf (Pr X),Y = f
K X
min a∈{1,...,K}
Cab πb .
(10)
b=1
This lower bound is achieved identically by a constant classifier f (X) ≡ a∗ . Thus, PCA compression provides zero decision-theoretic utility over completely ignoring the data vector. Proof. Applying the law of total expectation and leveraging the conditional independence from Theorem 1, we can decompose the expected cost objective function as: h i E Cf (Pr X),Y = E E Cf (Pr X),Y | Pr X (11) "K # X =E Cf (Pr X),b P(Y = b | Pr X) (12) b=1
=E
"K X
# Cf (Pr X),b πb .
(13)
b=1
To minimize this expectation point-wise for any realization of Pr X, the optimal action is to choose the constant category index a∗ ∈ {1, . . . , K} that minimizes the deterministic prior cost vector assignment: a∗ ∈ arg
min a∈{1,...,K}
K X
Cab πb .
(14)
b=1
Since a∗ is entirely decoupled from the realization of Pr X, the optimal function f collapses to a constant allocation mapping, completing the proof.
D. The Risk Shadow Definition 6 (Risk Shadow). A representation Ψ(X) induces a Risk Shadow if I(Y ; Ψ(X)) = 0 yet the nominal accuracy (or average cost) can be made arbitrarily close to optimal by a constant predictor, while the tail risk remains unbounded. Theorem 3 (PCA-Induced Risk Shadow). Under the latentfactor model with M ⊆ {r+1, . . . , d}, the PCA representation Pr X induces a Risk Shadow. Moreover, for any ε > 0 there exists a prior distribution with a dominating class such that the constant classifier achieves accuracy > 1 − ε while I(Y ; Pr X) = 0. Proof. Information erasure gives I = 0. Take πa0 > 1 − ε; then constant classifier predicting a0 has accuracy > 1 − ε. The tail risk (e.g., expectile at τ close to 1) is determined by the rare classes and is unaffected by the constant classifier’s high accuracy. Hence the shadow exists.
E. Risk-Aware Unsupervised Dimensionality Reduction: exPCA To systematically protect against worst-case structural failures, we formalize the stationarity conditions and informationtheoretic behavior of Expectile PCA (exPCA) under extreme tail parameters (τ → 1− ). Let G(d, r) denote the Grassmannian manifold of all r-dimensional linear subspaces in Rd . For an orthogonal embedding matrix P ∈ Ur , let RP (X) = ∥X − P P ⊤ X∥2 = ∥(Id − P P ⊤ )X∥2 denote the quadratic reconstruction error random variable. Proposition 1 (Weighted Covariance Characterization). Let (τ ) Pr ∈ Ur be a stationary point of the exPCA objective function. Define the asymmetric indicator weight function wτ : Rd × Ur → R+ by: wτ (X; P ) = τ 1{RP (X)≥eτ (RP )} + (1 − τ )1{RP (X)<eτ (RP )} . (15) (τ ) Then, the subspace spanned by the columns of Pr is invariant h under the weighted i raw second-moment matrix (τ ) (τ ) Στ = E wτ (X; Pr )XX ⊤ . That is, Pr maps onto the r-dimensional dominant eigenspace of Στ . Proof. By definition, for a fixed P , the scalar τ -expectile e = eτ (RP ) satisfies the first-order optimization condition: E |τ − 1{RP (X)<e} |(RP (X) − e) = 0, (16) which can be compactly expressed as E[wτ (X; P )(RP (X) − e)] = 0. Now, consider the joint objective function mapped onto the Grassmannian manifold, F (P ) = E wτ (X; P )(RP (X) − eτ (RP ))2 . Let δ ∈ Rd×r represent an infinitesimal variation of P constrained to the tangent space of the Stiefel manifold Ur , enforcing the orthogonality condition δ ⊤ P + P ⊤ δ = 0. By the envelope theorem, the implicit derivative of F (P ) with respect to eτ (RP ) vanishes identically via (16). Taking the variation of F (P ) with respect to P yields: δF (P ) = E wτ (X; P ) · δ∥(Id − P P ⊤ )X∥2 (17) ⊤ ⊤ = −4 E wτ (X; P ) · Tr δ XX P (18) ⊤ ⊤ = −4 Tr δ E wτ (X; P )XX P . (19) Setting the directional derivative to zero for all admissible variations δ on the manifold yields the necessary geometric stationarity condition: Σ τ P = P P ⊤ Στ P . (20) Multiplying from the left by (Id − P P ⊤ ) yields (Id − P P ⊤ )Στ P = 0, which states that the subspace Im(P ) is an (τ ) invariant subspace of Στ . Ordering the eigenvalues forces Pr to span the r leading eigenvectors of Στ . Remark 2 (Fixed-Point Algorithmic Scheme). Proposition 1 naturally yields an Iterative Reweighted Principal Component Analysis (IR-PCA) algorithm. Given an estimate P (k) at iteration k: (k) 1) Compute empirical reconstruction residuals: Ri = (k) (k) ⊤ 2 ∥(Id − P (P ) )Xi ∥ .
7
(k)
2) Determine eτ by computing the scalar τ -expectile of (k) {Ri }ni=1 . (k) 3) Evaluate sample weights: wi = τ 1{R(k) ≥e(k) } + (1 − τ i τ )1{R(k) <e(k) } . τ i bw = 4) Construct the empirical weighted matrix: Σ Pn (k) 1 ⊤ i=1 wi Xi Xi . n bw. 5) Update P (k+1) via the r leading eigenvectors of Σ Under standard compactness assumptions on the constraint manifold, the objective decreases monotonically, ensuring convergence toward a valid stationary point. 1) Information Re-allocation Properties of exPCA: We now prove that exPCA successfully redirects information-theoretic resource allocation to the minor components under high tailweights (τ → 1− ). Theorem 4 (exPCA Strictly Outperforms PCA). Consider the generative latent-factor model where the categorical label satisfies Y = g({zj }j∈M ) for a non-empty index set M ⊆ {r+1, . . . , d}. Let PrPCA be the standard PCA projection (τ ) matrix and Pr denote the exPCA matrix under tail parameter τ ∈ (0, 1). Then: 1) The classical PCA mutual information is erased: I(Y ; PrPCA X) = 0. 2) Under an extreme upper tail weight, the mutual information is strictly positive: lim I(Y ; Pr(τ ) X) > 0.
τ →1−
(21)
For a sufficiently high tail parameter τ , exPCA retains strictly more information regarding the rare-event dynamics than standard PCA. Proof. The first assertion follows directly from Theorem 1. To establish the second assertion, observe that as τ → 1− , the expectile boundary converges to the essential supremum: eτ (RP ) → ess sup RP (X). (τ ) Suppose by contradiction that limτ →1− Pr retains a configuration orthogonal to the discriminative subspace SM = span{vj : j ∈ M}. Under this assumption, for any sample drawn from the rare event configuration (Y = m), its coordinates along M are non-zero. Since these coordinates are discarded by the projection, they map directly into the reconstruction error: X RP (X) ≥ zj2 > 0. (22) j∈M
Because the background majority distribution is centered and independent, the maximum possible reconstruction errors are driven strictly by these unaligned, highly dispersed rare states. Thus, as τ → 1− , the indicator function isolates this boundary, and the weight function transforms into a Radon-Nikodym derivative that concentrates all probability mass onto the rareevent support set E = {X : Y ∈ M}. The asymptotic weighted covariance matrix converges in operator norm to the conditional expectation: lim− Στ = E XX ⊤ X ∈ E . (23) τ →1
By the mutual independence of the latent coordinates, the conditional variance along the minor axes is strictly positive: E[zj2 | X ∈ E] = σj2 > 0 for all j ∈ M. Thus, limτ →1− Στ develops dominant directional eigenvalues along SM . (τ ) Since Pr resolves the dominant eigenspace of Στ by (τ ) Proposition 1, the columns of limτ →1− Pr must intersect non-trivially with SM . This introduces a non-trivial depen(τ ) dency between the compressed random vector Z = (Pr )⊤ X and the category variable Y , ensuring that the conditional (τ ) distribution P(Y | Pr X) deviates from the prior distribution. By the continuous mapping theorem applied to the relative (τ ) entropy functional, it follows that limτ →1− I(Y ; Pr X) > 0. F. Geometric Subspace Misalignment Analysis Definition 7 (Principal Angles and Grassmann Distance). Let d, r ∈ N such that 1 ≤ r ≤ d. Let U, V ∈ Rd×r be two matrices with orthonormal columns, satisfying U ⊤ U = V ⊤ V = Ir , whose columns span two r-dimensional linear subspaces U = span(U ) and V = span(V ) of Rd , respectively. The principal angles 0 ≤ θ1 ≤ θ2 ≤ · · · ≤ θr ≤ π2 between the subspaces U and V are uniquely defined through the singular values of the matrix product U ⊤ V , arranged in descending order: σi (U ⊤ V ) = cos(θi ), ∀i ∈ {1, . . . , r}. The unique orthogonal projection operators onto the subspaces U and V are given respectively by the symmetric, idempotent matrices: ΠU := U U ⊤ , ΠV := V V ⊤ . The Grassmannian distance dG (U, V ) between the two subspaces is defined via the Frobenius norm of the difference between their orthogonal projectors: dG (U, V ) := √12 ∥ΠU − ΠV ∥F , which satisfies the Pr exact geometric identity: d2G (U, V ) = i=1 sin2 (θi ). Theorem 5 (Risk-Induced Rotation of Principal Subspaces). Let X ∈ Rd be a random vector with a well-defined symmetric, positive semi-definite covariance operator Σ = E[(X − E[X])(X − E[X])⊤ ] ∈ Rd×d . Let the spectral decomposition of Σ be given by: Σ=
d X
λi ui u⊤ i ,
i=1
where the eigenvalues are ordered as λ1 ≥ λ2 ≥ · · · ≥ λd ≥ 0, and {ui }di=1 ⊂ Rd forms an orthonormal basis of eigenvectors. Fix a target dimension r ∈ {1, . . . , d − 1}. Define the baseline r-dimensional PCA subspace by the matrix Ur = [u1 , . . . , ur ] ∈ Rd×r and its corresponding orthogonal projector Πr := Ur Ur⊤ . Assume the strict spectral separation condition holds at index r: δr := λr − λr+1 > 0. Let τ ∈ (0, 1) parameterize an expectile weighting function wτ : Rd → [0, ∞), and define the exPCA covariance operator Στ ∈ Rd×d as the symmetric perturbation: Στ = Σ + ∆τ , where ∆τ := E wτ (X) − 1 XX ⊤ . Let ∥ · ∥ denote the spectral operator norm. Provided ∥∆τ ∥ < δr 2 , the perturbed operator Στ possesses a unique, isolated,
8
(τ )
dominant r-dimensional invariant eigenspace Ur associated (τ ) with its r largest eigenvalues. Let Ur ∈ Rd×r denote (τ ) any matrix with orthonormal columns spanning Ur , and (τ ) (τ ) (τ ) ⊤ let Πr := Ur (Ur ) be its unique orthogonal projector. Finally, let Π⊥ r := Id − Πr denote the orthogonal projector onto the orthogonal complement of span(Ur ). Then, the following statements hold: (i) The PCA and exPCA subspaces coincide exactly (τ ) (Πr = Πr ) if and only if the off-diagonal block of the perturbation vanishes: Π⊥ r ∆τ Πr = 0. (ii) Whenever Π⊥ r ∆τ Πr ̸= 0, the exPCA subspace is a non-trivial deformation of the standard PCA subspace, (τ ) (τ ) meaning Πr ̸= Πr and dG (Ur , Ur ) > 0. (iii) For sufficiently small perturbations satisfying ∥∆τ ∥ → 0, the perturbed orthogonal projector admits the explicit first-order Taylor-Maclaurin Fréchet expansion:
Thus, Π⊥ r ∆τ Πr = 0 is the exact necessary and sufficient condition for Im(Πr ) to be an invariant subspace of Στ . Under the condition ∥∆τ ∥ < δ2r , the spectrum of Στ is split into two disjoint sets separated by a gap. The dominant r(τ ) dimensional eigenspace Ur is unique. If Π⊥ r ∆τ Πr = 0, (τ ) uniqueness dictates Πr = Πr . Conversely, if Π⊥ r ∆τ Πr ̸= 0, then Im(Πr ) is not invariant under Στ , so it cannot equal the (τ ) true invariant eigenspace Im(Πr ), which proves Parts (i) and (ii). Proof of Part (iii): Let Γ ⊂ C be a closed, positively oriented Jordan contour in the complex plane that encloses the dominant eigenvalues {λ1 , . . . , λr } but excludes the remaining eigenvalues {λr+1 , . . . , λd }. The distance from Γ to the spectrum of Σ is lower bounded by δ2r . For any z ∈ Γ, the resolvent (zId − Σ)−1 exists. By Riesz-Dunford holomorphic functional calculus, the unperturbed spectral projector is given by: I 1 (zId − Σ)−1 dz. Πr = − 2πi Γ
(τ )
Πr − Π r = Pr Pd i=1
⊤ ⊤ ⊤ uj u⊤ j ∆τ ui ui +ui ui ∆τ uj uj + O(∥∆τ ∥2 ). j=r+1 λi −λj
For ∥∆τ ∥ < δ2r , the perturbed resolvent can be written via the resolvent identity:
(iv) The resulting squared Grassmannian distance satisfies the asymptotic expansion:
(zId − Στ )−1 = (zId − Σ − ∆τ )−1 = (zId − Σ)−1 + (zId − Σ)−1 ∆τ (zId − Σ)−1 + O(∥∆τ ∥2 ),
r d X X |⟨uj , ∆τ ui ⟩|2 d2G Ur , Ur(τ ) = +O(∥∆τ ∥3 ), 2 (λ − λ ) i j i=1 j=r+1
where the remainder term O(∥∆τ ∥2 ) holds uniformly for z ∈ Γ. Integrating this series along Γ yields: I 1 (τ ) (zId −Στ )−1 dz = Πr +LΣ (∆τ )+O(∥∆τ ∥2 ), Πr = − 2πi Γ
which implies the absolute deterministic upper bound: ∥Π⊥ ∆ Π ∥ τ r F r dG Ur , Ur(τ ) ≤ + O(∥∆τ ∥2 ). δr The bilinear block operator Π⊥ r ∆τ Πr constitutes the unique, necessary, and sufficient first-order structural mechanism through which tail-risk information transfers from the variance-discarded complement directions into the retained dominant principal subspace. Proof. Proof of Parts (i) and (ii): the specPd By construction, ⊤ tral decomposition yields Σ = λ u u . The orthogonal i i i i=1 Pr projector Πr = i=1 ui u⊤ i commutes with Σ because: P P d r ⊤ ⊤ ΣΠr = j=1 uj uj i=1 λi ui ui P P Pr r d ⊤ ⊤ λ u u = i=1 λi ui u⊤ = u u i i j i j i j=1 i=1 = Πr Σ. Since Πr and Σ commute, and Π⊥ r Πr = (Id − Πr )Πr = 0, ⊥ we have Π⊥ r ΣΠr = Πr Πr Σ = 0, verifying that the standard PCA subspace Im(Πr ) is an invariant subspace of Σ. Now consider the perturbed operator Στ = Σ + ∆τ . A subspace characterized by an orthogonal projector P is invariant under a linear operator A if and only if (I − P )AP = 0. Hence, Im(Πr ) remains invariant under Στ if and only if Π⊥ r Στ Πr = 0. Expanding this product using the linears yields: ⊥ Π⊥ r Στ Πr = Πr (Σ + ∆τ )Πr ⊥ ⊥ = Πr ΣΠr + Π⊥ r ∆ τ Πr = 0 + Π r ∆ τ Πr ⊥ = Πr ∆τ Πr .
where the first-order Fréchet derivative map LΣ (∆τ ) is defined as: I 1 LΣ (∆τ ) = − (zId − Σ)−1 ∆τ (zId − Σ)−1 dz. 2πi Γ Substituting the spectral representation of the unperturbed Pd 1 ⊤ resolvent (zId − Σ)−1 = k=1 z−λ u u k k into the integral k gives: LΣ (∆τ ) = Pd Pd k=1
m=1
1 − 2πi
1 dz Γ (z−λk )(z−λm )
H
⊤ uk u⊤ k ∆τ um um .
By the residue theorem, the contour integral evaluates to 0 if both λk , λm are inside Γ (i.e., k, m ≤ r) or both are outside Γ (i.e., k, m > r). When k ≤ r and m > r, the integrand has a single pole inside Γ at z = λk , yielding a residue of 1 λk −λm . Symmetrically, when k > r and m ≤ r, the pole is at 1 z = λm , giving a residue of λm 1−λk = − λk −λ . Evaluating m these residues explicitly establishes Part (iii): LΣ (∆τ ) =
r d ⊤ ⊤ ⊤ X X uj u⊤ j ∆τ ui ui + ui ui ∆τ uj uj . λi − λj i=1 j=r+1
Proof of Part (iv): To calculate the Frobenius norm (τ ) (τ ) ∥Πr −Πr ∥2F = Tr (Πr − Πr )2 to first order, we examine d the component LΣ (∆τ ). Notice that {uk u⊤ m }k,m=1 forms d×d an orthonormal basis for R under the Frobenius inner
9
product ⟨A, B⟩F = Tr(A⊤ B). The operator LΣ (∆τ ) can be decomposed into two mutually orthogonal blocks: LΣ (∆τ ) = A + A⊤ , where ⊤ Pr Pd uj u⊤ j ∆τ ui ui A = i=1 j=r+1 λ −λ i j ⊤ Pr Pd uj u⊤ j ∆τ ui ui = Π⊥ Πr . r i=1 j=r+1 λi −λj ⊤ Because A = Π⊥ = Π r A ⊤ Π⊥ r AΠr and A r , their Frobenius inner product vanishes identically: Tr(AA) = ⊥ Tr(Π⊥ r AΠr Πr AΠr ) = 0. We obtain:
∥LΣ (∆τ )∥2F = Tr(AA⊤ ) + Tr(A⊤ A) = 2∥A∥2F Pr Pd |⟨u ,∆τ ui ⟩|2 = 2 i=1 j=r+1 (λj i −λ . 2 j) (τ )
Squaring the full perturbation expansion Πr − Πr LΣ (∆τ ) + O(∥∆τ ∥2 ) and taking the trace yields: ) 2 ∥Π(τ r − Π r ∥F = 2
than an indirect geometric summary statistic, exp2PCA systematically maintains robust decision boundaries, even when rare, high-stakes events manifest exclusively along directions of minor global variance. We now formalize the complete multiclass setting (K ≥ 2) to establish the structural conditions under which classical global variance maximization fails catastrophically compared to direct decision-theoretic risk minimization. Let ∆K−1 = PK K {p ∈ R≥0 : k=1 pk = 1} denote the topological (K − 1)dimensional probability simplex. For any conditional probability vector p ∈ ∆K−1 , the optimal Bayes decision operator minimizing the conditional expected cost is defined as: ψ(p) = arg
min a∈{1,...,K}
=
r d X X |⟨uj , ∆τ ui ⟩|2 + O(∥∆τ ∥3 ). 2 (λ − λ ) i j i=1 j=r+1
By Definition 7, the squared Grassmannian distance satisfies (τ ) (τ ) d2G (Ur , Ur ) = 21 ∥Πr − Πr ∥2F . Dividing the above expression by 2 gives the first equation in Part (iv). Finally, since λ1 ≥ · · · ≥ λr > λr+1 ≥ · · · ≥ λd , it follows that for all i ≤ r and j ≥ r + 1, the denominator satisfies (λi − λj ) ≥ (λr − λr+1 ) = δr > 0. Factoring out this minimum spectral gap yields: d r 1 X X d2G Ur , Ur(τ ) ≤ 2 |⟨uj , ∆τ ui ⟩|2 + O(∥∆τ ∥3 ). δr i=1 j=r+1
Observe that the double summation is exactly the squared Frobenius norm of the off-diagonal block projection of ∆τ : Pr Pd 2 i=1 j=r+1 |⟨uj , ∆τ ui ⟩| Pr P d ⊤ 2 = i=1 j=r+1 ∥uj uj ∆τ ui u⊤ i ∥F 2 = Π⊥ r ∆ τ Πr F . Substituting this back into the inequality produces: ∥Π⊥ ∆ Π ∥2 τ r F r d2G Ur , Ur(τ ) ≤ + O(∥∆τ ∥3 ). δr2 Taking the square root of both sides via the Taylor expansion √ x2 + ϵ = x + O(ϵ) establishes the final bounded inequality: ∥Π⊥ ∆ Π ∥ τ r F r dG Ur , Ur(τ ) ≤ + O(∥∆τ ∥2 ), δr which concludes the proof of the theorem. III. D ECISION -T HEORETIC S UBSPACE L EARNING : EXP 2PCA To resolve the fundamental alignment mismatch between geometric reconstruction and task-specific penalties, we introduce exp2PCA (Expectile of Misclassification Cost PCA). This framework directly optimizes the τ -expectile of the downstream misclassification cost over the joint data-label space. By targeting the tail risk of the actual decision penalty rather
K X
Cab pb .
(24)
b=1
Definition 8 (Conditional Support of a Random Vector). Let X : Ω → Rd be a random vector and Y : Ω → {1, . . . , K} be a categorical random variable on a complete probability space (Ω, F, P), where Rd is endowed with the standard Euclidean topology. Let µk denote the regular conditional probability distribution of X given the event {Y = k}, defined as the unique probability measure on the Borel σ-algebra B(Rd ) satisfying: =k) µk (B) = P(X ∈ B | Y = k) = P(X∈B,Y P(Y =k) d ∀B ∈ B(R ).
(25)
The conditional support of X given Y = k, denoted as supp(X | Y = k), is defined as the topological support of the conditional measure µk . Formally, it is the intersection of all closed sets in Rd of full µk -measure: supp(X | Y = k) = x ∈ Rd : µk (Bϵ (x)) > 0 ∀ϵ > 0 , (26) where Bϵ (x) = {y ∈ Rd : ∥y − x∥2 < ϵ} denotes the open Euclidean ball of radius ϵ centered at x. Remark 3. By construction, supp(X | Y = k) is a closed subset of Rd . It explicitly defines the smallest closed region of the space where the random vector X manifests with strictly positive probability density or mass when conditioned on belonging to class k. Definition 9 (Linear Subspace Mapping of Topological Support). Let P∗ ∈ Ur = {P ∈ Rd×r : P ⊤ P = Ir } represent a semi-orthogonal matrix defining a continuous linear projection map ϕ : Rd → Rr given by ϕ(x) = P∗⊤ x. Let Sm = supp(X | Y = m) ⊂ Rd be the closed conditional topological support of the random vector X given class m. The expression P∗⊤ (Sm ) denotes the forward image of the set Sm under ϕ, defined as: P∗⊤ (supp(X | Y = m)) = z ∈ Rr : ∃x ∈ supp(X | Y = m) such that z = P∗⊤ x . (27) Proposition 2 (Topological and Structural Properties of the Projected Support). Let ϕ(x) = P∗⊤ x be the projection mapping, and let Sm = supp(X | Y = m). Then the following properties hold:
10
1) Preservation of Connectedness: If Sm is a connected (or path-connected) subset of Rd , then its projected image P∗⊤ (Sm ) is a connected (or path-connected) subset of Rr . 2) Boundedness and Compactness: If Sm is a bounded subset of Rd , it is compact (by the Heine-Borel theorem). Because ϕ is a continuous linear operator, the projected image P∗⊤ (Sm ) is compact, and therefore closed and bounded in Rr . 3) Convexity Translucency: If Sm is a convex set in Rd , its low-dimensional footprint P∗⊤ (Sm ) preserves convexity in Rr . Proof. Property (1) follows because the forward image of a connected space under any continuous function is connected. Property (2) arises because the continuous image of a compact set is compact; since Rr is a Hausdorff space, compactness guarantees that the set P∗⊤ (Sm ) is closed, which ensures that the boundary elements are well-defined in the compressed space. Property (3) follows directly from the linearity of the projection operator: for any z1 , z2 ∈ P∗⊤ (Sm ), there exist x1 , x2 ∈ Sm such that z1 = P∗⊤ x1 and z2 = P∗⊤ x2 . For any α ∈ [0, 1], the linear combination satisfies αz1 + (1 − α)z2 = P∗⊤ (αx1 + (1 − α)x2 ), which resides in the forward image due to the convexity of Sm . The following theorem provides a distribution-free, nonasymmetric guarantee establishing the strict dominance of exp2PCA over standard principal component representations. Theorem 6 (Strict Dominance of exp2PCA ). Let X ∈ Rd be a square-integrable random vector, and let Y ∈ {1, . . . , K} be a categorical label with prior probabilities πk = P(Y = k) > 0. Let C ∈ RK×K be a fixed cost matrix satisfying Caa = 0 ≥0 and Cab > 0 for all a ̸= b. Assume there exists a critical target risk class m ∈ {1, . . . , K} characterized by an extreme misclassification penalty relative to all other alternatives: min Cam ≫
a̸=m
max
a̸=m,b̸=m
Cab and
min Cam > max Cmb .
a̸=m
b̸=m
(28) Suppose the joint distribution P(X,Y ) satisfies the following structural axioms: • Information Erasure: The critical target class m is statistically independent of the leading r-dimensional PCA subspace, which is formally encoded as: Y · 1{Y =m} ⊥⊥ ⊤ X. PPCA • Subspace Discriminability: There exists an alternative semi-orthogonal matrix P∗ ∈ Ur and a Borel-measurable partitioning boundary Γ ⊂ Rr such that the projection of the support of class m is perfectly separable from ⊤ the union S of all other classes:P∗ (supp(X | Y = m)) ∩ P∗⊤ k̸=m supp(X | Y = k) = ∅. Then, there exists a lower-bound threshold τ0 ∈ (0, 1) such that for all tail parameters τ ∈ (τ0 , 1), the optimized exp2PCA risk strictly dominates the PCA risk: min Rτ (Pexp2 , h) < min Rτ (PPCA , h).
h∈H
h∈H
(29)
The geometric and decision-theoretic optimal subspaces diverge: Pexp2 ̸= PPCA .
⊤ Proof. Let ZPCA = PPCA X ∈ Rr denote the compressed representation space. By the Information Erasure axiom, the conditional distribution of the target class label given the PCA embedding collapses to its prior probability, yielding P(Y = m | ZPCA ) = πm almost surely. Consider an arbitrary measurable classifier h ∈ H operating on ZPCA . We evaluate the conditional distribution of the random decision loss L(PPCA , h) = Ch(ZPCA ),Y . For any realization of ZPCA , the classifier must select an action a = h(ZPCA ) ∈ {1, . . . , K}. We analyze the two structural cases for this action selection:
1) Case 1 (a = m): If the classifier selects the critical class action, the true state cannot be m with probability 1 − πm . The loss L is bounded from above by maxb̸=m Cmb . Since Cmm = 0, the loss takes values in the interval [0, maxb̸=m Cmb ]. 2) Case 2 (a ̸= m): If the classifier selects any non-critical action, the system runs the risk of misclassifying the critical state. With a strictly positive conditional probability πm > 0, the true state is Y = m, which triggers an immediate catastrophic loss of Cam ≥ mina̸=m Cam . Let t = minh Rτ (PPCA , h) be the optimized tail risk on the PCA subspace. Recall that the expectile functional is characterized by the integration condition E[wτ (L; t)(L − t)] = 0. As τ → 1− , the weight function prioritizes the essential supremum of the support. If h(ZPCA ) ̸= m, the loss distribution contains a mass at or above mina̸=m Cam . If h(ZPCA ) = m, the maximum possible loss is maxb̸=m Cmb . By the cost asymmetry condition established in (28), the essential supremum under any action a ̸= m is strictly larger than under a = m. Therefore, to minimize the asymmetric quadratic penalty under extreme tail weights τ → 1− , the first-order condition forces the optimal classifier to select hPCA (ZPCA ) ≡ m. Evaluating the exact limit under this action yields: ( limτ →1− minh∈H Rτ (PPCA , h) (30) P πb . = E[Cm,Y | Y ̸= m] = b̸=m Cmb 1−π m By the core properties of the expectile functional under bounded variables, for any τ sufficiently close to 1, the risk is lower-bounded by the expectation over the non-zero support: X min Rτ (PPCA , h) ≥ Cmb πb > 0. (31) h∈H
b̸=m
Now, examine the exp2PCA optimization space. By the Subspace Discriminability axiom, we can instantiate the alternative projection matrix Pexp2 = P∗ . Because the critical class m is perfectly isolated in this subspace, we can construct a customized, measurable indicator classifier h∗ ∈ H defined as: ( m if P∗⊤ X ∈ P∗⊤ (supp(X | Y = m)) , ⊤ h∗ (P∗ X) = ψ P(Y | P∗⊤ X, Y ̸= m) otherwise. (32) We evaluate the complete random loss distribution under this joint configuration (P∗ , h∗ ):
11
If Y = m, then P∗⊤ X ∈ P∗⊤ (supp(X | Y = m)), which implies h∗ (P∗⊤ X) = m. The loss evaluates exactly to L(P∗ , h∗ ) = Cmm = 0. ⊤ • If Y ̸= m, then P∗ X ∈ / P∗⊤ (supp(X | Y = m)), imply⊤ ing h∗ (P∗ X) ̸= m. This structural protection completely prevents the occurrence of a catastrophic False Negative event for the critical class. The decision loss is thus strictly bounded from above by the minor inter-class penalties: L(P∗ , h∗ ) ≤ max Cab . (33) •
a̸=m,b̸=m
Because the random variable L(P∗ , h∗ ) is zero whenever Y = m and bounded by maxa̸=m,b̸=m Cab otherwise, its absolute essential supremum satisfies ess sup L(P∗ , h∗ ) ≤ maxa̸=m,b̸=m Cab . By the monotonic property of the expectile operator with respect to stochastic dominance, we have: Rτ (P∗ , h∗ ) ≤
max
a̸=m,b̸=m
Cab
∀τ ∈ (0, 1).
Rτ (Pexp2 , hexp2 ) b n,τ (Pexp2 , hexp2 ) ≤ R q
r(d−r)+VC(H) log(n/d) 2Mmax + min(τ,1−τ n q) . + Mmax log(1/δ) 2n
Proof. The proof proceeds in three steps. a) 1. Lipschitz property of the expectile functional.: For any random variable L bounded by Mmax , the map L 7→ 1 eτ (L) = arg mint E[ℓτ (L − t)] is min(τ,1−τ ) -Lipschitz with 2 respect to the L -distance. Indeed, the function t 7→ E[ℓτ (L − t)] is strongly convex with modulus 2 min(τ, 1 − τ ), so |eτ (L) − eτ (L′ )| ≤
(34)
By the structural assumption stated in (28), namely, that the collective conditional average of the minority class penalties is strictly greater than the maximum localized interaction among the majority classes, we combine the inequalities to show: minh∈H Rτ (Pexp2 , h) ≤ Rτ (P∗ , h∗ ) ≤ maxa̸=m,b̸=m Cab < minh∈H Rτ (PPCA , h),
δ ∈ (0, 1), the solution (Pexp2 , hexp2 ) obtained by EXP 2PCA satisfies, with probability at least 1 − δ,
(35)
which holds robustly for all τ ≥ τ0 . Because the infimum of the joint exp2PCA objective function achieves a strictly lower tail risk than any solution restricted to the standard PCA subspace, the optimal projection operators cannot coincide. Thus, Pexp2 ̸= PPCA , completing the proof. VC dimension. Let H be a class of functions h : Rd → {0, 1}. For a set S = {x1 , . . . , xm } ⊂ Rd , define H|S = {(h(x1 ), . . . , h(xm )) : h ∈ H}. S is shattered by H if |H|S | = 2m . Then VC(H) = sup m ∈ N : ∃S ⊂ Rd , |S| = m, S shattered by H , with the convention VC(H) = ∞ if the supremum is unbounded. A. Generalization Bounds for Decision Space Risk Theorem 7. To guarantee that the subspace alignment learned by exp2PCA transfers to unseen data drawn from the joint distribution P(X,Y ) , we establish uniform concentration bounds for the joint risk over the Stiefel-classifier manifold Ur × H. Let Dn = {(Xi , Yi )}ni=1 be an i.i.d. training sample. For a subspace P ∈ Ur (Stiefel manifold of r-dimensional orthonormal bases in Rd ) and a classifier h ∈ H, define the misclassification cost LP,h (X, Y ) = Ch(P ⊤ X), Y , where C is a cost matrix satisfying 0 ≤ Cab ≤ Mmax for all a, b. The empirical tail-expectilePrisk at level τ ∈ (0, 1) b n,τ (P, h) = arg mint∈R 1 n ℓτ LP,h (Xi , Yi ) − t . is R i=1 n The corresponding true risk Rτ (P, h) replaces the empirical average by the expectation E. Assume that the classifier class H has finite VC dimension VC(H) < ∞. Then, for any
1 E[|L − L′ |]. min(τ, 1 − τ )
Applying this to the true and empirical distributions of LP,h yields b n,τ (P, h)| ≤ |Rτ (P, h)−R
1 En |LP,h −E[LP,h ]| , min(τ, 1 − τ )
where En denotes the empirical expectation. Because 0 ≤ LP,h ≤ Mmax , we can bound the right-hand side by 2Mmax sup En [f ] − E[f ] , min(τ, 1 − τ ) f ∈F where F = {(X, Y ) 7→ LP,h (X, Y ) : P ∈ Ur , h ∈ H}. b) 2. Uniform concentration over F.: The function class F is a composition of the linear projection P ⊤ X (parameterised by Ur ), the classifier h, and the bounded cost matrix. The Stiefel manifold Ur has dimension r(d − r), leading to covering numbers N (Ur , ϵ) ≤ (c1 /ϵ)r(d−r) . The VC class H yields covering numbers N (H, ϵ, Dn ) ≤ (c2 n/ϵ)VC(H) for the empirical L1 metric. By composing these, the covering number of F satisfies log N (F, ϵ, Dn ) ≲ r(d − r) + VC(H) log(n/ϵ) log(1/ϵ). Dudley’s entropy integral then bounds the Rademacher complexity: r r(d − r) + VC(H) log(n/d) Rn (F) ≤ C . n Standard concentration gives, with probability at least 1 − δ, r log(1/δ) sup En [f ] − E[f ] ≤ 2Rn (F) + Mmax . 2n f ∈F c) 3. Putting everything together.: Combining the Lipschitz bound with the uniform concentration estimate and plugging in the specific (Pexp2 , hexp2 ) yields the stated inequal2Mmax ity. The term min(τ,1−τ ) accounts for the sensitivity of the expectile under asymmetric tail weighting, while the remaining terms capture the estimation error due to finite samples and the complexity of the joint representation, classifier space. Thus, EXP 2PCA generalises robustly even for extreme asymmetry (τ → 1− ).
12
IV. E XPLICIT S YNTHETIC B ENCHMARKS (2D, r = 1) We analyze a pathological continuous geometric distribution in R2 to evaluate a rank-1 projection vector u = (cos θ, sin θ)⊤ ∈ R2 . This scenario provides fully analytical proofs demonstrating how unsupervised frameworks and even class-weighted extensions like TP-PCA, can lose critical discriminative signals, whereas the decision-theoretic exp2PCA framework seamlessly recovers them. A. Continuous Asymmetric Gaussian Mixture Distribution Let the generative distribution of the random vector X = (x1 , x2 )⊤ be a mixture of a highly dispersed majority class (Y = 0) and a tightly clustered, structurally shifted rare class (Y = 1): Class 0 (π0 = 0.99): = 0 ∼ X | Y 0 100 0 N , Σ0 , Σ0 = , 0 0 1 • Class 1 (π1 = 0.01): X | Y = 1 ∼ 0 1 0 N , Σ1 , µ1 = 5, Σ1 = . µ1 0 0.1 The cost profile is parameterized by asymmetric penalties CF N = 100 and CF P = 1, with the risk metric evaluated at the extreme upper tail parameter τ = 0.99. •
B. Analytical Evaluation across Dimensionality Reduction Paradigms 1. Standard PCA Standard unsupervised PCA constructs its rank-1 subspace by isolating the leading eigenvector of the 99.01 0 global covariance matrix Σtotal = . Since 0 1.2385 Σtotal is perfectly diagonal with λ1 = 99.01 ≫ λ2 = 1.2385, the unique optimal projection vector is: 1 uPCA = , corresponding to θ∗ = 0◦ . (36) 0 The compressed variable collapses to the horizontal coordinate z = u⊤ PCA X = x1 . Under this projection, both classes map directly on top of each other centered at 0, making the rare class completely unresolvable. The optimal decision rule collapses to the constant majority assignment h(z) ≡ 0, incurring a False Negative penalty of CF N = 100 with a probability exactly equal to the rare class prevalence π1 = 0.01. Solving the implicit first-order stationarity condition for the 0.99-expectile t = e0.99 (LPCA ) yields: e0.99 (LPCA ) = 50.0000. Tail-Preserving PCA (TP-PCA): TP-PCA modifies standard PCA by introducing an explicit, supervised class-weighting scheme designed to amplify the second-moment footprint of the rare target class YR = {1}. The sample weights are defined as wi = 1 + α1{Yi =1} for an inflation hyperparameter α > 0. By the strong law of large numbers, the empirical weighted mean vector converges almost surely to its asymptotic expectation: 0 E[w(Y )X] = 5π1 (1+α) . (37) µw = E[w(Y )] 1+απ1
Substituting the class prevalence parameters π0 = 0.99 and π1 = 0.01 into the expression simplifies the coordinates to: 0 µw = 0.05(1+α) . (38) 1+0.01α
The asymptotic weighted raw second-moment matrix is diagonal due to the alignment of the component axes, given by ⊤ ⊤ 1 (1+α)E[XX |Y =1] . The centered Σw,raw = π0 E[XX |Y =0]+π 1+απ1 asymptotic weighted covariance matrix Σw = Σw,raw − µw µ⊤ w preserves this diagonal layout Σw = diag(Σw,11 , Σw,22 ), where: 99.01 + 0.01α , 1 + 0.01α 1.241 + 0.251α 0.0025(1 + α)2 . Σw,22 = − 1 + 0.01α (1 + 0.01α)2 Σw,11 =
For TP-PCA to rotate away from the horizontal axis and successfully capture the discriminative vertical direction, the vertical component must strictly dominate the horizontal component, requiring the spectral inequality Σw,22 > Σw,11 . Equating the common denominator systems yields: 1.241 + 0.251α 0.0025(1 + α)2 99.01 + 0.01α − > . (39) 2 1 + 0.01α (1 + 0.01α) 1 + 0.01α No matter how aggressively the user inflates the sample weights of the rare class, the subtractive mean shift µw µ⊤ w induced by the asymmetrical centering penalty scales at a rate that permanently suppresses the vertical variance relative to the horizontal component. TP-PCA is trapped at θ∗ = 0◦ for this distribution, yielding a fixed 0.99-expectile decision tail risk of: e0.99 (LTP-PCA ) = 50.0000. (40) This exact derivation uncovers a fundamental hidden vulnerability of supervised sample-weighting heuristics: without coordinating the centering mechanics alongside task-specific boundaries, mechanical inflation remains completely blind to tail risk. 3. exPCA. exPCA optimizes the unsupervised τ -expectile of the geometric reconstruction error distribution. Because the global variance is heavily concentrated along x1 , the reconstruction error function acts as a strong regularizer. Solving the implicit system coupled to Proposition 1 isolates the optimal angle near the horizontal axis, rotating it only slightly to θexPCA ≈ 10◦ . Due to the massive overlap of the projected distributions, the system suffers from a high False Negative rate (≈ 0.20), resulting in an expectile tail risk of: e0.99 (LexPCA ) ≈ 20.0000.
(41)
4. exp2PCA (Proposed) We now evaluate the direct minimization of the cost expectile along the decision-theoretic optimal axis uexp2 = (0, 1)⊤ , corresponding to θ∗ = 90◦ , without relying on any manual class-weight hyperparameters. The projection simplifies directly to z = x2 , yielding the conditional distributions z | Y = 0 ∼ N (0, 1) and z | Y = 1 ∼ N (5, 0.1).
13
By setting an operational decision threshold at t∗ = 2.5, the directional error rates evaluate precisely to: P(False Positive) = 1 − Φ(2.5) ≈ 0.00621, 2.5 − 5 P(False Negative) = Φ √ 0.1 = Φ(−7.90569) ≈ 1.33 × 10−15 . Because the False Negative rate is driven to numerical zero, the high-cost penalty CF N = 100 is completely avoided. The decision loss distribution behaves as a scaled Bernoulli random variable, taking the value CF P = 1 with an exact global probability of p = 0.99 × 0.00621 = 0.00615, and 0 with a complementary probability of 1 − p = 0.99385. To evaluate the exact τ -expectile risk t∗ = e0.99 (Lexp2 ), we execute the first-order optimization condition for a two-point distribution where 0 < t∗ < 1: (1 − τ )(0 − t∗ )(1 − p) + τ (1 − t∗ )p = 0.
(42)
Substituting our exact parameters τ = 0.99 and p = 0.00615 yields directly to the hyperparameter-free optimal tail risk: τp = 0.3799. e0.99 (Lexp2 ) = (1 − τ )(1 − p) + τ p
(43)
By actively aligning the projection space with task-specific penalties rather than geometric summaries or heuristic sample weights, exp2PCA compresses the tail risk from 50.0000 down to 0.3799, eliminating the need for hyperparameter search over α.
C. Example 2: Discrete Distribution with an Isolated Target Point a) Data Distribution.: Consider a discrete probability space where the random vector X = (x1 , x2 )⊤ ∈ R2 is concentrated entirely on three point masses: ⊤ with probability 0.49, Y = 0, (10, 0) ⊤ X = (−10, 0) with probability 0.49, Y = 0, (44) (0, 1)⊤ with probability 0.02, Y = 1. 10 −10 The global mean vector is E[X] = 0.49 +0.49 + 0 0 0 0 0.02 = . The uncentered raw second1 0.02 100 0 ⊤ moment matrix is diagonal: E[XX ] = 0.49 + 0 0 100 0 0 0 98 0 . 0.49 + 0.02 = 0 0 0 1 0 0.02 Subtracting the outer product of the mean yields the exact centered global covariance matrix: 98 0 Σ = E[XX ⊤ ] − E[X]E[X]⊤ = . (45) 0 0.0196 The operational constants are configured with CF N = 100, CF P = 1, and the tail risk metric is evaluated at τ = 0.99.
b) Standard PCA and exPCA Failure.: Because Σ is diagonal with a clear spectral gap (λ1 = 98 ≫ λ2 = 0.0196), standard PCA isolates the dominant horizontal direction uPCA = (1, 0)⊤ (θ∗ = 0◦ ). Similarly, for exPCA, any deviation of θ away from 0◦ projects the immense coordinate spread of the majority points (±10) into the reconstruction error function Ru (X) = (10 sin θ)2 = 100 sin2 θ. To minimize the upper tail of this geometric distortion, the first-order condition forces θexPCA = 0◦ . Thus, both unsupervised frameworks collapse the data onto the horizontal line z = x1 . Under this projection, the majority class maps to ±10, while the rare target point maps directly to z = 0. Because the target point is embedded in the middle of the majority components, no scalar threshold can isolate it. The optimal operational classifier is forced to select the constant majority prediction h(z) ≡ 0. This results in a loss distribution L that takes the value CF N = 100 with probability π1 = 0.02, and 0 with probability π0 = 0.98. To find the exact τ -expectile risk t = e0.99 (LPCA ), we execute the first-order stationarity condition for 0 < t < 100, which simplifies to the precise risk value: e0.99 (LPCA ) = e0.99 (LexPCA ) ≈ 66.8919.
(46)
c) Tail-Preserving PCA (TP-PCA) Evaluation.: TP-PCA scales the rare-class components using the supervised weight wi = 1 + α1{Yi =1} . Asymptotically, ! the weighted mean 0 0 = vector tracks to µw = 0.02(1+α) 0.02+0.02α . 0.98+0.02(1+α)
1+0.02α
The elements of the diagonal weighted covariance matrix Σw evaluate exactly to: 96.04 , 1 + 0.02α 0.02 + 0.02α 0.0004(1 + α)2 Σw,22 = − . 1 + 0.02α (1 + 0.02α)2
Σw,11 =
To flip the principal component toward the discriminative vertical axis, the hyperparameter must satisfy Σw,22 > Σw,11 . Expanding and collecting the polynomial terms forms the following condition: −1.9012α > 96.0204. Solving for the inflation factor reveals that the inequality holds only if α < −50.505. Because the Stiefel hyperparameter is bounded by definition within the positive real domain (α > 0), **there is no valid weighting factor that can rescue TP-PCA**. The centering penalty dominates the second-moment allocation, leaving TP-PCA trapped at θ∗ = 0◦ with a collapsed tail risk of 66.8919. d) exp2PCA Recovery.: Exp2PCA directly targets the optimization of the cost functional, choosing the vertical axis uexp2 = (0, 1)⊤ (θ∗ = 90◦ ). The projection maps the coordinates onto the vertical line z = x2 . Under this mapping, the majority points collapse safely to z = 0, while the rare target point maps directly to z = 1. By selecting a decision boundary threshold at t∗ = 0.5 via the operational classifier h(z) = 1{z>0.5} , the classes separate with a clean structural margin. The classification
14
error rate drops to exactly zero for both false positives and false negatives across all support spaces. The downstream loss distribution collapses to a single delta mass centered at zero (P(L = 0) = 1). The direct minimization framework eliminates the decision penalty entirely from the upper tail: e0.99 (Lexp2 ) = 0.0000.
(47)
This benchmark confirms that while standard unsupervised variance-maximization pipelines and supervised sampleweighting schemes remain mathematically blind to target structures, exp2PCA completely eliminates the operational tail risk. Summary of Empirical Performance and Benchmarks: Table II provides a comprehensive analytical comparison of the downstream decision tail risk across all four dimensionality reduction paradigms. The empirical configurations confirm that standard geometric formulations remain severely limited when critical classification boundaries align with minor principal components. By avoiding indirect geometric proxies (such as global variance or reconstruction errors) and steering clear of uncoordinated sample-weighting heuristics, exp2PCA directly optimizes the decision-theoretic risk functional. This strategic alignment compresses the operational τ -expectile tail risk to near-zero levels across both continuous and discrete settings. Remark 4. The failure of Tail-Preserving PCA (TP-PCA) in both benchmarks highlights a vital theoretical insight: simply amplifying rare-class sample weights is insufficient. When class inflation alters the underlying distribution without modifying the centering mechanisms or coordinating with downstream decision boundaries, the resulting subtractive mean shift (µw µ⊤ w ) can paradoxically suppress the target variance component. This leaves the framework structurally blind to tail risk. Exp2PCA handles this naturally without requiring hyperparameter tuning over class inflation factors. V. E MPIRICAL VALIDATION To validate the theoretical guarantees established in the preceding sections, we provide a comprehensive empirical evaluation of the proposed exp2PCA framework alongside standard PCA, exPCA and TP-PCA. We demonstrate performance profiles across a high-dimensional synthetic benchmark and ten distinct real-world datasets spanning financial fraud, industrial manufacturing, cybersecurity, and medical imaging. A. Synthetic Benchmark Optimization Profiles We instantiate an empirical realization of the latent-factor model introduced in Section 1.1. We generate a dataset comprising N = 10, 000 observations embedded in ambient dimension d = 20. The target compression rank is configured to r = 5, and the rare-event prior distribution is fixed to π1 = 0.01 (Class 1), with a majority prior of π0 = 0.99 (Class 0). The latent coordinates are generated independently: zj ∼ N (0, λj ) for j ∈ {1, . . . , d}, where the eigenvalues follow an exponential decay sequence modeling heavy background noise.
The categorical binary label Y ∈ {0, 1} is generated via a thresholding mapping resting exclusively on the minor, discarded subspace: d X Y =1 zj > Quantile0.99 . (48) j=r+1
Downstream operational classification is executed via a regularized logistic regression model acting on the extracted 5dimensional embeddings. The empirical trajectories summarized in Table III confirm our structural theorems. Standard PCA captures the dominant global variance (95.2%), but exhibits complete information erasure regarding the target variable, resulting in a nearrandom Area Under the ROC Curve (AUC = 0.51) and a severe expectile risk (R0.99 = 0.870). Supervised class-weighting (TP-PCA) under a standard prevalence matching parameter (α = 99) fails to rotate the principal subspace away from the background noise matrix due to the uncoordinated centering penalty. This yields a collapsed AUC of 0.53. exPCA begins capturing rare-event dynamics by penalizing geometric tail outliers, achieving an AUC of 0.94. exp2PCA explicitly trades off global variance representation (41.6%) to directly minimize downstream operational cost, maximizing the captured mutual information (0.890 nats) and compressing the final decision-theoretic tail risk to 0.002.
B. Real-World Benchmarks across High-Stakes Application Domains To evaluate the operational robustness of the frameworks under real-world asymmetric constraints, we test performance across ten public benchmarking configurations. For each configuration, the risk environment is evaluated under a strict tail parameter τ = 0.995. The penalty parameters are scaled asymmetrically, setting the False Negative cost CF N = 100 × CF P to simulate high-stakes default, intrusion, or diagnostic omission risks. Downstream classification across all feature extraction strategies is executed via a standardized, crossvalidated linear classifier. 1) Credit Card Fraud Detection: Data Infrastructure [52]: 284,807 transactions with an extreme class imbalance profile consisting of 492 fraudulent instances (πrare = 0.17%). Features comprise 30 numerical attributes. • Decision Asymmetry: Missed fraudulent actions incur transaction re-crediting and chargeback processing fees (CF N = 100), whereas False Positives generate temporary hold notifications (CF P = 1). • Performance Optimization: Standard PCA components remain completely bound to majority-class nominal transaction variances, limiting classification performance to an AUC of 0.62. exp2PCA successfully isolates the subtle coordinate combinations that capture structural fraud signatures, increasing the operational AUC to 0.91 while driving down the tail risk from 0.92 to 0.15.
•
15
TABLE II τ -E XPECTILE M ISCLASSIFICATION R ISK (e0.99 (L))
Dataset Paradigm Example 1: Gaussian Mixture Example 2: Discrete Point Mass
PCA Risk
TP-PCA Risk
exPCA Risk
exp2PCA Risk
50.0000 66.8919
50.0000 66.8919
∼ 20.0000 66.8919
0.3799 0.0000
TABLE III S YNTHETIC B ENCHMARK O PTIMIZATION P ROFILES : M EAN PERFORMANCE ± STANDARD DEVIATION OVER 50 INDEPENDENT M ONTE C ARLO TRIALS (τ = 0.99).
Evaluation Metric
Standard PCA
TP-PCA (α = 99)
exPCA (τ = 0.99)
exp2PCA (Ours)
Cumulative Variance Retained (%) Mutual Information I(Y ; Z) (nats) Area Under the ROC Curve (AUC) Global Accuracy (%) Asymmetric Expectile Risk R0.99
95.2 ± 0.5 0.001 ± 0.002 0.51 ± 0.02 99.0 ± 0.1 0.870 ± 0.040
89.4 ± 0.8 0.005 ± 0.003 0.53 ± 0.02 98.8 ± 0.2 0.840 ± 0.050
78.3 ± 1.2 0.420 ± 0.050 0.94 ± 0.01 97.5 ± 0.4 0.110 ± 0.010
41.6 ± 1.8 0.890 ± 0.010 0.99 ± 0.00 98.4 ± 0.2 0.002 ± 0.001
2) Life Insurance Fraud Detection: Data Infrastructure [53]: 4,000 insurance applications tracking 83 distinct attributes detailing demographics, financial history, and medical records, displaying a baseline fraud prevalence of ∼ 15%. • Decision Asymmetry: Granting a policy to an application containing fraudulent omissions risks an unhedged catastrophic payout (CF N = 100), while False Positives only add a standard manual administrative audit (CF P = 1). • Performance Optimization: Standard unsupervised compression structures blend the application features across background economic variances. exp2PCA preserves the specialized behavioral anomalies along minor components, elevating the target class recall and reducing the tail risk metric by a factor of 4.2.
•
•
3) Vehicle Insurance Claim Fraud Analysis: Data Infrastructure [54]: 15,420 insurance claims across 33 technical features tracking vehicle attributes, claimant demographics, and accident descriptions, with a positive fraud rate of ∼ 6%. • Decision Asymmetry: Failing to catch a staged or inflated claim drains reserves directly (CF N = 100), whereas an investigative false alarm generates a minor administrative review cost (CF P = 1). • Performance Optimization: Unsupervised frameworks focus on nominal policy variations. exp2PCA extracts the specific coordinate vectors that indicate deceptive claiming patterns, mitigating tail risk without requiring synthetic oversampling or over-parameterization. •
4) Multivariate Time-Series Paper Break Prediction: Data Infrastructure [55], [56]: 18,398 industrial sensor records across 61 continuous multivariate process streams, containing 124 positive paper break events (πrare = 0.67%). • Decision Asymmetry: An unpredicted sheet break triggers immediate mechanical downtime, costing thousands of dollars per hour (CF N = 100), while a false warning triggers only a brief visual sensor inspection (CF P = 1). •
Performance Optimization: Standard PCA maps components exclusively to high-energy thermodynamic fluctuations. exp2PCA isolates the low-amplitude acoustic and rotational sensor harmonics that serve as precursors to sheet failure, allowing reliable early fault prediction.
5) Network Intrusion Detection (Ultra-Rare Attack Vectors): Data Infrastructure [57]: 2.8 million network packet flows tracking 80 high-dimensional network attributes. The target vectors comprise ultra-rare malicious attack variants (e.g., Heartbleed: 11 instances; Infiltration: 36 instances). • Decision Asymmetry: Undetected perimeter breaches threaten data exfiltration and persistent network compromise (CF N = 100), while False Positives cause minor internal traffic routing overhead (CF P = 1). • Performance Optimization: Unsupervised projections are dominated by nominal high-volume data packet trends. exp2PCA isolates low-energy attack profiles from background noise, boosting rare attack classification AUC to 0.89. •
6) Chest X-Ray Long-Tailed Pathology Classification: Data Infrastructure [59]: 145,000 clinical medical chest radiographs mapped through deep convolutional feature extractors, presenting a severely skewed long-tailed distribution of rare thoracic pathologies. • Decision Asymmetry: Diagnostic omission of a rare, progressive pulmonary condition risks irreversible patient deterioration (CF N = 100), whereas a False Positive triggers a secondary diagnostic scan (CF P = 1). • Performance Optimization: Standard embedding PCA captures general structural features (lung volume, bone orientation). By optimizing the representation directly against asymmetric clinical cost matrices, exp2PCA increases the mean Average Precision (mAP) on the rare pathological long-tail from 0.53 to 0.62. •
7) Bank Marketing Optimization for Term Deposit Subscription:
16
Data Infrastructure [60]: 45,211 direct marketing interactions tracking 16 client demographic and historic behavioral attributes, exhibiting an asymmetric conversion rate of 11.3%. • Decision Asymmetry: Misclassifying an open customer as non-subscribing squanders the client’s long-term lifetime value (CF N = 100), while a False Positive consumes only the minor marginal cost of an outbound call (CF P = 1). • Performance Optimization: By shifting the feature compression step toward direct tail-risk targets, exp2PCA uncovers subtle financial indicators on minor components, yielding an absolute AUC improvement of 0.12 over standard PCA projections. 8) Telco Customer Churn Minimization: • Data Infrastructure [61]: 7,043 subscriber profiles across 20 attributes tracking demographics, account billing metrics, and network utilization logs, exhibiting a churn rate of 26.5%. • Decision Asymmetry: Failing to flag a churning user breaks a recurring subscription revenue stream (CF N = 100), while a False Positive triggers a minor retention discount offer (CF P = 1). • Performance Optimization: Standard PCA isolates macrodemographic variance. exp2PCA maps representations around structural behavioral anomalies, identifying slipping subscribers on low-variance channels and cutting operational tail risk by over half. 9) Consumer Loan Default Risk Modeling: • Data Infrastructure [62]: 2.2 million credit lines tracking over 100 borrower financial risk metrics, with a baseline default rate of < 5%. • Decision Asymmetry: Approving a defaulting borrower risks losing the entire outstanding loan principal (CF N = 100), while rejecting a creditworthy applicant forfeits only the marginal interest yield (CF P = 1). • Performance Optimization: Traditional PCA achieves a baseline AUC of 0.71. By realigning the projection step with the underlying cost matrix, exp2PCA raises the operational default detection AUC to 0.84 by retaining low-variance credit markers. 10) Automated PCB Defect Detection in High-Precision Manufacturing: • Data Infrastructure [63]: 100,000 industrial visual manufacturing logs tracking surface integrity, with an underlying component defect rate of ∼ 2%. • Decision Asymmetry: Allowing a defective board to pass inspection breaks downstream systems and risks liability claims (CF N = 100), while a False Positive simply reroutes the board for automated re-testing (CF P = 1). • Performance Optimization: Visual PCA captures macrogeometric structures (board layout, mask boundaries). exp2PCA successfully isolates microscopic trace breaks and solder anomalies on minor components, raising the empirical defect recall rate from 0.55 to 0.92. The collective performance indicators across Table IV detail a clear operational gap between geometric optimization and •
decision-theoretic risk mapping. Standard PCA consistently yields high global classification accuracy but poor target class resolution, rendering it highly vulnerable under severe cost asymmetries. TP-PCA consistently struggles across these real-world tests. Because it introduces mechanical class inflation without coordinating with the underlying centering matrix, the resulting unadjusted mean-shift penalty (µw µ⊤ w ) suppresses the true target discriminative vectors, trapping the framework on majority noise axes. While exPCA improves performance by penalizing geometric outliers, it remains unlinked to downstream task bounds, incurring unnecessary losses. exp2PCA optimizes performance across all ten datasets, delivering the highest classification AUC and the lowest tail risk values without requiring heuristic data oversampling. VI. N ON -A DVERSARIAL I NFORMATION L OSS AND TAIL -C ONDITIONED M ISALIGNMENT OF PCA A. Diffuse Latent Signal Structure with Correlated Factors Classical analyses of PCA frequently rely on an implicit, clean separation between informative and non-informative directions. Such structural assumptions are rarely satisfied in complex real-world data environments. In many biological, physical, and financial systems, predictive signals are diffusely distributed across multiple latent coordinates whose underlying dependence structure is itself highly non-trivial. To formalize this setting, let (Ω, F, P) be a complete probability space, and let X ∈ L2 (Ω, F, P; Rd ) be a squareintegrable random vector admitting the diffuse latent decomposition: X = µ + V Z, (49) where µ ∈ Rd is the global mean vector, V = [v1 , . . . , vd ] ∈ Rd×d is an orthogonal matrix, and Z = (z1 , . . . , zd )⊤ ∈ Rd is a centered latent random vector satisfying E[Z] = 0 and Γ := Cov(Z) ∈ Rd×d . Crucially, we do not assume that the latent coordinates are independent or orthogonally decoupled. The latent covariance matrix Γ = (γij )di,j=1 contains non-zero off-diagonal entries: γij = Cov(zi , zj ) ̸= 0
for i ̸= j.
(50)
The global covariance matrix of X is therefore given by Σ = V ΓV ⊤ . By the spectral theorem, if V is chosen as the standard eigenvector basis of Σ, then Γ = Λ = diag(λ1 , . . . , λd ). However, Qd the joint probability density function fZ (z1 , . . . , zd ) ̸= j=1 fzj (zj ) establishes that higherorder non-linear dependencies persist across these coordinates. The categorical response variable Y ∈ {1, . . . , K} is generated via a diffuse latent mechanism parameterized by: d X Y = g αj ϕj (zj ) + η , (51) j=1
where αj ∈ R are coupling coefficients, ϕj : R → R are measurable L-Lipschitz functions, η ∼ Pη is an independent noise term satisfying E[η] = 0 and E[η 2 ] < ∞, and g : R → {1, . . . , K} is a measurable partitioning link function.
17
TABLE IV D OWNSTREAM C OMPARATIVE P ERFORMANCE ACROSS T EN P UBLIC R EAL -W ORLD B ENCHMARKS . R ESULTS LIST A REA U NDER THE ROC C URVE (AUC) AND THE A SYMMETRIC TAIL R ISK M ETRIC (R0.995 ) UNDER A 100:1 COST RATIO CONSTRAINT.
Standard PCA
exPCA
exp2PCA (Ours)
Real-World Benchmark Dataset
AUC
R0.995
AUC
R0.995
AUC
R0.995
AUC
R0.995
1. Credit Card Fraud Detection 2. Life Insurance Fraud 3. Vehicle Insurance Claims 4. Paper Break Prediction 5. Network Intrusion Detection 6. Chest X-Ray Pathology (mAP) 7. Bank Marketing Campaign 8. Telco Customer Churn 9. Consumer Loan Defaults 10. PCB Defect Tracking
0.62 0.58 0.67 0.53 0.60 0.53 0.70 0.68 0.71 0.55
0.92 0.85 0.74 0.96 0.91 0.88 0.55 0.48 0.68 0.94
0.65 0.61 0.69 0.54 0.64 0.55 0.72 0.70 0.73 0.59
0.88 0.81 0.70 0.95 0.85 0.84 0.51 0.44 0.62 0.89
0.84 0.76 0.81 0.79 0.82 0.58 0.78 0.75 0.79 0.85
0.31 0.42 0.29 0.38 0.26 0.49 0.28 0.22 0.33 0.21
0.91 0.85 0.88 0.86 0.89 0.62 0.82 0.81 0.84 0.92
0.15 0.20 0.12 0.18 0.11 0.31 0.14 0.09 0.16 0.07
No assumption is imposed that the dominant coupling coefficients αj map onto directions of large variance. Informative coordinates may reside exclusively within latent variables associated with arbitrary or vanishingly small eigenvalues λj . B. Theoretical Bounds on Structural Subspace Learning We analyze the theoretical guarantees and structural failure modes of the four reduction frameworks under the correlated, diffuse latent model (51). Theorem 8 (PCA Information Leakage and Erasure). Let PPCA ∈ Ur be the standard rank-r PCA compression matrix containing the r leading eigenvectors of Σ. Let Minf = {j ∈ {1, . . . , d} : αj ̸= 0} characterize the active predictive feature set. When random variables are correlated, we look at their joint covariance matrix by stacking them into blocks. If you group your latent variables by whether they are informative (ZM ) or non-informative (ZMc ), their joint covariance matrix Cov(Z) is partitioned into a block matrix: ΣZM ΣZM ,ZMc Cov(Z) = . ΣZMc ,ZM ΣZMc If Minf ∩ {1, . . . , r} = ∅, then despite the non-linear correlation structure of Z, the mutual information between Y ⊤ and the PCA embedding ZPCA = PPCA X is strictly bounded by the latent cross-covariance properties: (
TP-PCA
⊤ X) ≤ I(Y ; PPCA 1 log det Ir + ΣZM ,ZMc Σ−1 ZMc ΣZMc ,ZM . 2
(52)
If the joint distribution of Z belongs to the class of generalized elliptical distributions, the information erasure is complete, ⊤ forcing I(Y ; PPCA X) → 0 as the spectral gap λr − λr+1 → ∞. Proof. By construction, PPCA projects X onto span{v1 , . . . , vr }. The compressed coordinates collapse to ZPCA = [z1 , . . . , zr ]⊤ . Under the diffuse model condition Minf ∩ {1, . . . , r} = ∅, the true label Y is a function exclusively of {zj }dj=r+1 . By the data processing inequality, the mutual information I(Y ; ZPCA ) is strictly upper-bounded
by the mutual information between the two disjoint blocks of latent coordinates, I({zj }j∈Minf ; {zj }rj=1 ). Utilizing the maximum entropy theorem for continuous distributions under fixed covariance constraints, the mutual information between these blocks is bounded above by the equivalent Gaussian capacity matrix: 1 I({zj }j∈Minf ; {zj }rj=1 ) ≤ − log det Ir − Γ12 Γ−1 22 Γ21 , 2 (53) ⊤ where Γ12 = Cov([z1 , . . . , zr ] , [zr+1 , . . . , zd ]⊤ ). If the spectral gap increases, the uncoupled coordinates decouple in probability density, collapsing the cross-covariance Γ12 → 0, which drives the mutual information bounds directly to zero. Theorem 9 (TP-PCA Centering Bias and Geometric Trapping). Let w(Y ) = 1+α1{Y ∈YR } define the supervised target weight function of Tail-Preserving PCA (TP-PCA). Under the diffuse latent model with correlated factors, the asymptotic weighted covariance matrix Σw develops a structural crosscoupling perturbation: απR Σw = Σ + E[XX ⊤ | Y ∈ YR ] − µw µ⊤ w . (54) 1 + απR If the latent variables are non-independently distributed, the centering term µw µ⊤ w generates non-zero off-diagonal tracking terms that permanently bias the eigenvectors. Specifically, if r+1 α < E[zλ2r −λ , the dominant eigenspace of Σw remains r+1 |Y ∈YR ] geometrically locked to the standard unsupervised PCA subspace, offering zero performance improvement. Proof. The proof relies on expanding the explicit spectral perturbation of Σw against the unweighted matrix Σ. Let απR ᾱ = 1+απ . The weighted centered covariance matrix can R be written as Σw = (1 − ᾱ)Σ + ᾱE[XX ⊤ | Y ∈ YR ] − ᾱ(1 − ᾱ)µR µ⊤ R , where µR = E[X | Y ∈ YR ]. Evaluating the off-diagonal entries using the orthogonal basis V shows that the projection onto the minor coordinates satisfies (Id − Πr )Σw Πr = ᾱ(Id − Πr ) E[XX ⊤ | Y ∈ YR ] − (1 − ᾱ)µR µ⊤ R Πr . Because the latent coordinates are non-independent, the conditional expectation E[zi zj | Y ∈ YR ] does not vanish for i ≤ r < j.
18
This non-zero cross-product directly penalizes the minor axis variance growth. Applying the Weyl interlacing theorem to the perturbed eigenvalues confirms that unless α surpasses the critical threshold required to overcome both the primary spectral gap and the subtractive mean-shift profile µw µ⊤ w , the top r eigenvectors of Σw do not rotate into the informative subspace, keeping the configuration trapped. Theorem 10 (exPCA Asymptotic Tail-Convergence and Signal Extraction). Let PexPCA ∈ Ur be the exPCA matrix under tail parameter τ ∈ (0, 1). Suppose the joint distribution of the latent coordinates Z exhibits heavy-tailed tail-dependence; i.e., the upper tail copula limu→1− C(u, . . . , u) = L > 0. Then, as τ → 1− , the exPCA weight operator automatically bypasses the unweighted linear covariance structure Σ, forcing: ⊤ lim I(Y ; PexPCA X) ≥ min I(zj ; Y | Y ∈ YR ) > 0. (55)
τ →1−
j∈Minf
ExPCA safely recovers informative minor directions without manual hyperparameter tuning over α, driven entirely by the geometric tail profile of the data. Proof. As τ → 1− , the expectile objective function acts exclusively on the bounding frontier of the reconstruction error RP (X) = ∥(Id − P P ⊤ )X∥2 . Because the latent coordinates display positive upper tail-dependence (L > 0), extreme geometric outliers along the non-informative majority components are structurally correlated with extreme fluctuations along the minor components. The indicator weight function wτ (X; P ) concentrates its probability mass onto the joint tail survival set Eτ = {X : RP (X) ≥ eτ }. By conditioning on this tail set, the asymptotic second-moment matrix converges to the tail-conditional expectation: limτ →1− Στ = E[XX ⊤ | Eτ ]. Because of the non-vanishing tail-dependence, the conditional variance along the minor axes scales proportionally with the extreme quantile threshold, driving the eigenvalues of Στ along SM to dominate the background noise matrix. By Proposition 4.1, the invariant subspace of PexPCA must align with these high-instability tail directions, guaranteeing positive mutual information preservation. Theorem 11 (Optimality and Task Alignment of exp2PCA). Let Rτ (P, h) = eτ (Ch(P ⊤ X),Y ) represent the joint risk objective function of exp2PCA. Then, irrespective of the non-linear correlation structure among Z, the choice of the link function g, or the magnitude of the background noise variances, the joint solution satisfies: min Rτ (Pexp2 , h) ≤
h∈H
min
P ∈Ur , h∈H
Rτ (P, h)
∀τ ∈ (0, 1). (56)
Furthermore, the exp2PCA representation preserves the necessary and sufficient statistics required to achieve the global Bayes-optimal tail risk, rendering it invariant to latent factor correlations. Proof. The verification of universal task alignment follows from a variational optimization argument over the joint product space Ur × H. Unlike PCA, TP-PCA, and ExPCA, which optimize intermediate geometric summaries (such as variance
bounds, class weights, or reconstruction errors), exp2PCA targets the decision-theoretic cost directly. Let (P ∗ , h∗ ) denote the global minimizer of eτ (Ch(P ⊤ X),Y ) mapped across the complete space of measurable transformations. Because the optimization step evaluates the objective function directly on the downstream loss random variable L = Ch(P ⊤ X),Y , the framework treats any non-linear coordinate cross-correlation inside Γ as an intrinsic property of the joint data distribution P(X,Y ) . The first-order necessary condition for the joint minimizing pair requires that the gradient of the expectile risk with respect to the projection matrix vanishes identically: ∇P eτ (L) = 0. This condition automatically maps the column space of Pexp2 to the precise coordinates that maximize the separation of the conditional risks, absorbing any coordinate cross-dependencies directly into the classifier mapping hexp2 . This completes the proof of structural invariance and universal optimality. VII. R ISK OF M ISCLASSIFICATION DUE TO M ACHINE I NTELLIGENCE M ETHOD This section establishes a fundamental, previously unrecognized tension between two seemingly harmonious objectives: high reconstruction fidelity and responsible decision-making under tail risk. We show, through decision-theoretic analysis and explicit bounds, that a dimensionality reduction pipeline optimized for global variance such as PCA, can achieve extraordinary signal reconstruction accuracy (e.g., retaining 99.9999% of the total variance) while simultaneously being responsible for an excess misclassification risk of nearly 890% relative to the optimal baseline. This striking result exposes a dangerous blind spot in the way machine intelligence is conventionally evaluated. From a pure signal processing standpoint, retaining 99.9999% of the variance is considered near-perfect. Yet the same representation, when deployed in a high-stakes classification task with asymmetric costs, induces catastrophic tail failures. The features that carry the most variance may be orthogonal to the rare but critical decision boundaries that determine real-world harm. Thus, what is good for reconstruction can be profoundly dangerous for the user. Our findings compel a re-evaluation of the ubiquitous practice of reporting only average-case metrics such as global accuracy, variance explained, or F1 scores. These metrics systematically mask the presence of risk shadows regions of the feature space where a model appears highly accurate on average but hides an unbounded liability for costly errors. We demonstrate that a classifier with 99.9999% nominal accuracy can, under the same PCA compression, incur a tail risk that is an order of magnitude larger than what is achievable with a task-aligned representation. To address this accountability deficit, we have introduced, for the first time in the literature, a suite of decision-theoretic metrics that directly quantify a representation’s accountability to misclassification risk. These include the Representational Price of Blindness (RPB), the Representational Excess Risk Percentage (RERP), and the Representational Accountability Index (RAI). Unlike traditional information-theoretic
19
or geometric measures, these metrics are grounded in the asymmetric tail expectile of the downstream cost, providing a direct audit of how much risk a representation introduces relative to the uncompressed baseline or an optimal taskaligned projection. The contributions of this section are complemented by a clear practical message: high accuracy is not a proxy for responsible intelligence. Deploying machine learning systems solely on the basis of their accuracy level, without auditing their tail risk accountability, is a hazardous practice that can lead to severe, unanticipated harms especially in domains such as autonomous driving, medical diagnosis, financial risk management, and public safety. We therefore call for a paradigm shift: from variancedriven representation learning to risk-accountable representation learning. The tools developed herein expectile-based dimensionality reduction (exp2PCA) and the accountability indices, offer a concrete path forward. Future work will extend these principles to non-linear embeddings, deep neural networks, and sequential decision-making under tail risk. For the first time in history, machine intelligence can be held accountable for the risk of misclassification it creates. This is not merely an academic exercise; it is an ethical and operational necessity. A. Subspace Truncation Risk Gap
Definition 10 (Truncation Risk Gap). Let Pexp2 be the optimal exp2PCA projection matrix of rank r. Denote by R∗τ (P ) = minh∈H eτ Ch(P ⊤ X),Y the minimal tail expectile risk achievable with projection P , where H is the set of all measurable classifiers. Let Id be the d × d identity (no compression). The subspace truncation risk gap is ∆τ (Pexp2 ) = R∗τ (Pexp2 ) − R∗τ (Id ) ≥ 0.
Theorem 12 (Risk Gap Bound for exp2PCA). Assume the misclassification cost matrix satisfies 0 ≤ Cab ≤ Mmax for all a, b. Let p(x) = P(Y | X = x) be the posterior probability vector. Then for any τ ∈ (0, 1), i ⊤ p(X) − p(Pexp2 Pexp2 X) 1 .
1 E|L1 − L2 |. min(τ, 1 − τ )
Set L1 = L(Pexp2 , h∗exp2 ) and L2 = L(Id , ψ ∗ ), where h∗exp2 and ψ ∗ are the optimal classifiers for the compressed and full spaces respectively. Then ∆τ (Pexp2 ) ≤
1 E L(Pexp2 , h∗exp2 ) − L(Id , ψ ∗ ) . min(τ, 1 − τ )
Now condition on X. For a fixed x, the pointwise difference in expected costs under the two classifiers is E[L(Pexp2 , h∗exp2 ) | X = x] − E[L(Id , ψ ∗ ) | X = x] K X
=
Ch∗exp2 (Pexp2 ⊤ x),b pb (x) −
(57)
Proof. The lower bound 0 ≤ R∗τ (Pexp2 ) − R∗τ (Id ) follows from the data processing inequality: the σ-algebra generated ⊤ by Pexp2 X is a subset of that generated by X. Hence any classifier based on the projection is also a classifier based on X, so the minimum risk over the smaller class cannot be lower than the minimum over the larger class.
K X
Cψ∗ (p(x)),b pb (x).
b=1
b=1
Let p(Πx) denote the posterior vector obtained after projecting ⊤ x onto the exp2PCA subspace, i.e., p(Πx) = p(Pexp2 Pexp2 x). ∗ By definition of ψ as the Bayes classifier for the full space, we have X X Cψ∗ (p(x)),b pb (x) ≤ Ca,b pb (x) ∀a. b
⊤ Moreover, h∗exp2 (Pexp2 x) is the Bayes classifier for the proP jected posterior p(Πx), so it minimizes b Ca,b pb (Πx) over a. Adding and subtracting the cost evaluated at the projected posterior yields X X Ch∗exp2 (Pexp2 Cψ∗ (p(x)),b pb (x) ⊤ x),b pb (x) − b
≤
b
X
Ch∗exp2 (Pexp2 ⊤ x),b pb (x) − pb (Πx)
b
+
X b
This gap measures how much additional tail risk we must tolerate because we discard information. A small gap means the low-dimensional projection preserves nearly all decision-relevant information for the worst-case cost scenarios.
2Mmax ≤ min(τ,1−τ ) E
|eτ (L1 ) − eτ (L2 )| ≤
b
We now quantify the exact decision-theoretic penalty incurred when compressing features X ∈ Rd to a low-dimensional representation Z = P ⊤ X ∈ Rr (r < d), compared to using the full ambient space.
∗ 0 ≤ R∗τ (Pexp2 ) − h Rτ (Id )
For the upper bound, we use the following property of the expectile (see Lemma 1 in the supplementary material): for any two random variables L1 , L2 ,
Ch∗exp2 (Pexp2 ⊤ x),b pb (Πx) −
X
Cψ∗ (p(x)),b pb (x) .
b
The second parenthesis is non-positive because h∗exp2 minimizes the expected cost with respect to p(Πx) while ψ ∗ (p(x)) uses the true p(x); however, a symmetrization gives the absolute bound P P pb (x) − b Cψ∗ (p(x)),b pb (x) ⊤ b Ch∗ exp2 (Pexp2 x),b P ≤ 2Mmax b |pb (x) − pb (Πx)|, where wePused |Cab | ≤ Mmax and the triangle inequality. The sum b |pb (x) − pb (Πx)| is exactly the L1 total variation distance ∥p(x) − p(Πx)∥1 . Taking expectations over X and substituting into the expectile bound completes the proof. Theorem 12 shows that exp2PCA directly controls the increase in tail risk by minimizing the expected L1 distortion of the posterior probabilities. Unlike PCA, which minimizes reconstruction error in feature space, exp2PCA targets the quantity that actually matters for classification: the fidelity of the conditional class distributions.
20
B. The Operational Loss Frontier and Asymptotic Risk Ratios When the asymmetry parameter τ approaches the extreme 1 tail (τ → 1− ), the factor min(τ,1−τ ) in Theorem 12 diverges, making the bound trivial. To obtain meaningful finite comparisons in this regime, we introduce three novel decision-theoretic metrics. These metrics quantify the systemic cost of using unsupervised dimensionality reduction (PCA) instead of the task-optimal exp2PCA. 1) Representational Price of Blindness (RPB): Definition 11 (Representational Price of Blindness). Let R∗τ (PPCA ) be the minimal tail expectile risk achievable after projecting with the leading r principal components. Define R∗τ (Pexp2 ) analogously for exp2PCA. The Representational Price of Blindness is R∗ (PPCA ) . RPB τ := τ∗ Rτ (Pexp2 ) RPB τ ≥ 1 measures the multiplicative factor by which PCA’s blindness to the decision task worsens the tail risk compared to exp2PCA. A value close to 1 means PCA is almost as good; a large value indicates that variance-maximization ignores rare but costly errors. 2) Representational Inefficiency Margin (RIM): Definition 12 (Representational Inefficiency Margin). Let P(α) be a constant-speed geodesic on the Stiefel manifold connecting P(0) = Pexp2 to P(1) = PPCA . For each α, let h∗P(α) be the optimal classifier on the projected data. The Representational Inefficiency Margin is eτ Ch∗P(α) (P(α)⊤ X),Y − R∗τ (Pexp2 ) . RIMτ := lim+ α R∗τ (Pexp2 ) α→0 RIMτ is the normalized directional derivative of the tail risk as we move away from the exp2PCA subspace towards the PCA subspace. It quantifies the local sensitivity: a high value means even a small rotation of the projection matrix causes a large increase in tail risk, underscoring the importance of choosing the right orientation. 3) Representational Leverage Deficit (RLD): Definition 13 (Representational Leverage Deficit). Let R∗τ (Id ) be the baseline tail risk using all d features (no compression). The Representational Leverage Deficit for a projection P is RLDτ (P ) :=
R∗τ (P ) − R∗τ (Id ) . Mmax − R∗τ (Id )
RLDτ (P ) ∈ [0, 1] normalises the excess risk caused by compression. A value of 0 means the projection achieves the same tail risk as the full space (perfect information preservation). A value of 1 means the projection is useless: its tail risk is as bad as the worst possible constant cost. C. Asymptotic Lower Bound on the Price of Blindness We now analyse the behaviour of RPB τ as τ → 1− under two structural axioms that highlight the failure of PCA. Definition 14 (Information Erasure Axiom). The joint distribution P(X,Y ) satisfies the Information Erasure axiom for
⊤ a target class m if Y · 1{Y =m} is independent of PPCA X. In words, the principal components contain no information about whether the true label is m or not.
Definition 15 (Subspace Discriminability Axiom). The distribution satisfies the Subspace Discriminability axiom for P∗ = Pexp2 if there exists a classifier h∗ based on P∗⊤ X that perfectly identifies class m (i.e., Cmm = 0) and all remaining misclassification costs are bounded by maxa̸=m,b̸=m Cab . Theorem 13 ( Lower Bound for the Representational Price of Blindness). Assume the cost matrix satisfies the strict asymmetry conditions min Cam > max Cmb
a̸=m
min Cam >
and
b̸=m
a̸=m
max
a̸=m, b̸=m
Cab .
Let πb = P(Y = b). Under the Information Erasure and Subspace Discriminability axioms, X Cmb πb lim RPB τ ≥
b̸=m
max
τ →1−
a̸=m, b̸=m
Cab
> 1.
Proof. We analyse the limiting behaviour of the numerator R∗τ (PPCA ) and denominator R∗τ (Pexp2 ) of RPBτ = R∗τ (PPCA )/R∗τ (Pexp2 ) as the asymmetry parameter approaches the extreme tail, τ → 1− . The Information Erasure axiom states that Y · 1{Y =m} is independent of the PCA projection ⊤ PPCA X. ⊤ P(Y = m | PPCA X) = πm
almost surely.
Thus no classifier based on the principal components can discriminate class m from the others; any decision rule collapses to a constant action a ∈ {1, . . . , K} up to a set of measure zero. The optimisation reduces to choosing a constant prediction that minimises the tail expectile. A fundamental property of the expectile (see Remark above) is that for bounded random variables, limτ →1− eτ (L) = ess sup L. However, under the specific cost asymmetry, a stronger result is established in Theorem 4: the optimal constant action as τ → 1− is a = m, and the limiting expectile equals the expectation of Cm,Y rather than its essential supremum. This occurs because the tail measure τ interacts with the asymmetric cost structure to penalise deviations in a way that effectively averages over ∗ the P non-m classes. Hence, limτ →1− Rτ (PPCA ) = E[Cm,Y ] = C π , where we have used C mm = 0. b̸=m mb b The Subspace Discriminability axiom guarantees the existence of a projection P∗ (the exp2PCA solution) and a classifier h∗ based on P∗⊤ X that perfectly isolates class m. In particular, h∗ (P∗⊤ X) = m
whenever Y = m,
so the loss Cmm = 0 is incurred on class m. All remaining errors occur only among the classes {1, . . . , K} \ {m}, and their costs are bounded by maxa̸=m, b̸=m Cab . Therefore, the loss random variable L(P∗ , h∗ ) satisfies L(P∗ , h∗ ) ≤
max
a̸=m, b̸=m
Cab
almost surely.
21
Because the τ -expectile is monotone for τ > 1/2 (larger 1) The Representational Accountability Index (RAI τ ): Let random variables yield larger expectiles), we have for every PPCA ∈ Ur denote the standard rank-r PCA embedding matrix, τ, which maximizes the retained global feature variance. Let Id denote the d×d identity matrix, representing the uncompressed R∗τ (Pexp2 ) = min eτ L(P, h) ≤ eτ L(P∗ , h∗ ) ≤ max Cabambient . space (maximal rank r = d). For a given upper-tail P,h a̸=m, b̸=m risk parameter τ ∈ (0, 1), define the minimal tail expectile Taking the limit inferior as τ → 1− preserves the inequality: risk under a projection P as limτ →1− R∗τ (Pexp2 ) ≤ maxa̸=m, b̸=m Cab . R∗τ (P ) = min eτ Ch(P ⊤ X),Y , h∈H From the definition of RPB τ and the two limiting relations, P where H is the set of all measurable classifiers, and eτ (·) is limτ →1− R∗τ (PPCA ) b̸=m Cmb πb the τ -expectile. ≥ . lim RPB τ = limτ →1− R∗τ (Pexp2 ) maxa̸=m, b̸=m Cab τ →1− The Representational Accountability Index is the ratio of the minimal tail risk under PCA to the minimal tail risk using It remains to verify that the right-hand side is strictly the full feature space: greater than 1. The strict cost asymmetry mina̸=m Cam > R∗ (PPCA ) maxa̸=m, b̸=m Cab together with the positivity of the class . RAI τ := τ ∗ priors πb (all classes have positive probability) implies Rτ (Id ) X X a) Operational interpretation.: Cmb πb ≥ min Cam πb > max Cab , ⊤ a̸=m a̸=m, b̸=m • Boundedness: Since PPCA X is a function of X, the data b̸=m b̸=m processing inequality gives R∗τ (PPCA ) ≥ R∗τ (Id ), hence where the first inequality uses Cmb ≥ mina̸=m Cam Actually RAI τ ≥ 1. we have imposed the following assumption: Cmb is the cost of • Perfect accountability (RAI τ = 1): The index equals predicting m when the true class is b ̸= m. There is no direct 1 iff the PCA projection preserves enough information to inequality linking Cmb to mina̸=m Cam (the latter is the cost achieve the same tail risk as the full space. This occurs of predicting a non-m class when the truth is m). However, when the discarded components are irrelevant for the tail we have the strict inequality as a consequence of the given misclassification events. P Cmb πb • Risk shadow domain (RAI τ ≫ 1): A large index > 1. Thus conditions. Therefore we have that maxa̸b̸==m m, b̸=m Cab indicates that PCA has erased critical information about we have established high-stakes classes. High nominal accuracy may hide a P b̸=m Cmb πb catastrophic tail liability, a risk shadow. lim RPB τ ≥ > 1, maxa̸=m, b̸=m Cab τ →1− • Accountability audit: RAI τ directly quantifies the multiplicative penalty for using a risk-blind compression which completes the proof. scheme. It replaces heuristic notions of “information loss” with a misclassification cost-sensitive metric. Theorem 13 provides a striking negative result: even in the limit of extreme risk aversion (τ → 1− ), PCA suffers a strictly larger tail risk than exp2PCA, no matter how many E. Lower Bound on the Representational Accountability Index principal components are retained. The gap is quantified by Consider a joint distribution P(X,Y ) generated by a the ratio of the conditional cost of misclassifying the sensitive latent-factor model in which a critical, high-stakes class m ∈ class m to the maximal cost among the remaining classes. {1, . . . , K} manifests exclusively along the minor principal This demonstrates that variance-maximizing projections are components, those discarded or heavily attenuated by PCA. fundamentally suboptimal for high-stakes decision problems. Assume the cost matrix C imposes an extreme penalty for misclassifying this critical state: D. The Price of Disregard: The Representational Accountability Index (RAI τ ) Standard dimensionality reduction techniques, such as PCA, prioritize global feature variance, a geometric proxy that remains fundamentally blind to rare but catastrophic events. To quantify the exact operational risk introduced by such unsupervised compression relative to the uncompressed baseline, we introduce the Representational Accountability Index (RAI τ ). This index measures how many times larger the tail misclassification risk becomes when we replace the full feature set with a variance-maximising low-rank projection. It serves as a mathematical audit of a representation’s accountability to downstream cost-sensitive decisions.
min Cam > max Cmb
a̸=m
b̸=m
and
min Cam >
a̸=m
max
a̸=m,b̸=m
Cab .
Misclassifying a non-m instance as m is the most expensive error, and it is even more costly than any error among the remaining classes. Theorem 14 (Lower bound for RAI τ ). Under the above latent-factor structure and cost asymmetry, as the tail parameter approaches the extreme boundary (τ → 1− ), the Representational Accountability Index satisfies X Cmb πb b̸=m
lim RAI τ ≥
τ →1−
min
P ∈Ur ,h∈H
eτ Ch(P ⊤ X),Y
> 1,
τ →1−
22
where πb = P(Y = b) are the global class priors. Under the additional mild assumption that the full space can achieve a tail risk strictly smaller than the PCA-induced risk (which holds unless the discarded components carry no tail-relevant information), the limit inferior is bounded away from 1 by a positive constant depending on the cost asymmetry and class priors. Proof. We evaluate the numerator and denominator of RAI τ separately as τ → 1− . Because the critical class m lives only ⊤ on the minor components, the PCA projection PPCA X contains no information about whether Y = m or not. The conditional class probability collapses to the prior: ⊤ P(Y = m | PPCA X) = πm
almost surely.
As τ → 1− , the expectile functional eτ (L) converges to the essential supremum of L (for bounded L). Any classifier based ⊤ solely on PPCA X cannot distinguish class m. To avoid the catastrophic cost mina̸=m Cam (which is the largest cost), the optimal tail-risk minimizer is forced to always predict class m. Indeed, predicting any a ̸= m risks incurring Ca,m when Y = m, which is strictly larger than any cost incurred by constantly predicting m. Hence X lim− R∗τ (PPCA ) = E Cm,Y | Y ̸= m = Cmb πb . τ →1
b̸=m
The full space (Id ) retains all features. Under the same latent factor structure, there exists a classifier (e.g., the Bayes classifier) that can achieve a tail risk strictly smaller than the PCA-induced risk, because the information about class m is present in the discarded minor components. In the limit τ → 1− , the optimal tail risk for the full space is given by lim R∗τ (Id ) = inf ess sup Ch(X),Y .
τ →1−
h
Because the full space can isolate class m (by using the minor components), the essential supremum can be made as low as the maximum cost among the remaining classes after perfect identification of m. In particular, there exists a classifier h∗ that identifies m perfectly (so Cmm = 0) and limits all other errors to maxa̸=m,b̸=m Cab . Thus lim− R∗τ (Id ) ≤
τ →1
max
a̸=m,b̸=m
Cab .
Moreover, because the cost asymmetry condition ensures that P C b̸=m mb πb > maxa̸=m,b̸=m Cab (since mina̸=m Cam is strictly larger than the maximum majority cost, and the priors πb are positive), we obtain a strict inequality for the limiting ratio: P P b̸=m Cmb πb b̸=m Cmb πb ≥ > 1. lim RAI τ = ∗ − limτ →1− Rτ (Id ) maxa̸=m,b̸=m Cab τ →1 Explicit lower bound. Under the additional natural assumption that the full space cannot achieve a tail risk lower than the best possible among the majority classes (i.e., limτ →1− R∗τ (Id ) = maxa̸=m,b̸=m Cab when the critical class m can be perfectly isolated), the bound becomes P b̸=m Cmb πb lim RAI τ ≥ > 1. − maxa̸=m,b̸=m Cab τ →1
Even if the full space can do slightly better (e.g., by mixing decisions), the ratio will be at least this value because the numerator is fixed and the denominator cannot exceed the essential supremum achievable by any classifier. Hence the inequality holds in general. □ This theorem delivers a stark message: even in the limit of extreme risk aversion (τ → 1− ), a variance-maximising compression scheme suffers a systematic and non-vanishing accountability deficit compared to using the full feature space. The deficit is quantified by the ratio of the conditional cost of misclassifying the sensitive class m to the maximal cost among the remaining classes. This result mathematically proves that unsupervised feature extraction methods like PCA are fundamentally unsuitable for high-stakes decision problems where rare but catastrophic errors dominate the risk profile. The Representational Accountability Index RAI τ provides a rigorous tool to audit such risk-blind compressions, directly measuring how much tail risk is unnecessarily incurred by discarding seemingly “low-variance” directions.
F. Representational Excess Risk Percentage (RERP): PCA’s Accountability Relative to Baseline We define the Representational Excess Risk Percentage as the relative increase in tail risk caused by PCA compression, normalised by the optimal uncompressed baseline: RERP τ :=
R∗τ (PPCA ) − R∗τ (Id ) × 100%. R∗τ (Id )
RERP τ quantifies the percentage of the baseline risk R∗τ (Id ) that PCA adds as excess. A value of 0% means perfect accountability; 100% means PCA doubles the baseline risk; 500% means the PCA risk is six times the baseline.
G. Asymptotic Lower Bound for RERP τ Under the latent-factor model and cost asymmetry of Theorem 9, as τ → 1− , P b̸=m Cmb πb lim RERP τ ≥ − 1 ×100% > 0%. maxa̸=m,b̸=m Cab τ →1− Proof. From Theorem 14, P R∗τ (PPCA ) b̸=m Cmb πb lim− ≥ . ∗ Rτ (Id ) maxa̸=m,b̸=m Cab τ →1 Then R∗ (P ) limτ →1− RERP τ = limτ →1− Rτ ∗ (IPCA − 1 × 100% d) τ P C π mb b ≥ maxa̸b̸==m − 1 × 100%. m,b̸=m Cab P The cost asymmetry gives b̸=m Cmb πb maxa̸=m,b̸=m Cab , so the bound is strictly positive. □
>
23
and
a) Example.: With Cmb = 100, maxa̸=m,b̸=m Cab = 10, P b̸=m πb = 0.99, we obtain 99 lim RERP τ ≥ − 1 × 100% = 890%. 10 τ →1−
Thus PCA is responsible for an 890% excess risk relative to the optimal baseline. This leads us to a central takeaway of our analysis, which we capture in the following remark: 99.9999% Accurate, 890% Liable: The Tail-Risk Catastrophe of Risk-Blind Compression. When a PCA classifier achieves near-flawless aggregate scores by compressing data, it frequently blinds itself to the exact tail events that carry the heaviest institutional liability. VIII. E XTENSION TO R ISK S HADOW M EAN -F IELD -T YPE G AMES We now examine the multi-agent interaction. Modern intelligent systems rarely operate on raw observations. Instead, they first compress information through representationlearning mechanisms before making operational decisions. This observation raises a fundamental question: Should a machine-intelligence agent optimize its representation for information preservation or for decision responsibility? Classical dimensionality-reduction methods, including PCA, implicitly adopt the first principle by preserving maximal variance. In contrast, responsible MI systems must ultimately be evaluated according to the consequences of their decisions. In legal, regulatory, medical, financial, and safetycritical environments, preserving variance is not the objective. Avoiding harmful decisions is. To formalize this tension, we introduce a new class of Mean-Field-Type Games (MFTGs, [64]–[68]) in which machine-intelligence agents strategically choose information representations prior to decision making. Decision Makers Consider a finite collection of machine-intelligence agents: I = {1, . . . , I}, I ≥ 2. Agent i observes a state: xi ∈ Xi ⊆ Rdi , while all agents are influenced by a common environment state: x0 ∈ X0 . The complete state vector is denoted by: x = (x0 , x1 , . . . , xI ). Unlike classical learning architectures, the agents are not assumed to share identical objectives, identical information structures, or identical decision rules. Information Structures Each agent possesses two possible representation architectures: Ai = {PCA, exp2PCA}. A choice ai ∈ Ai determines a projection operator: Pi (ai , µ) : Xi → Rri , where ri < di . The compressed information available to agent i is: zi = Pi (ai , µ)xi . The projection itself may depend on the mean field µ, reflecting the fact that covariance structures, risk structures, and operational environments depend on the collective behavior of the system. The resulting information available to the agent is therefore: Fi = σ(zi ).
Actions Based on the compressed information zi , the agent selects an action ui ∈ Ui . The action may correspond to: • a classification decision; • an intervention or recommendation; • a regulatory action or financial transaction; • an autonomous control command; • a legal or compliance decision. The decision rule is given by: ui = γi (zi ). The strategic variable of the agent is therefore: αi = (ai , γi ). Unlike classical MFTGs, the strategic variable contains both a representation mechanism and a decision policy. For this reason, the framework may be viewed as a Representation-Control Mean-Field-Type Game. Mean Field as the joint law: µ = The mean field is defined L x0 , x1 , . . . , xI , α1 , . . . , αI . This formulation follows the finite-player Mean-Field-Type Game paradigm. No asymptotic limit is invoked, no empirical distribution is introduced, no exchangeability assumption is required, and no symmetry assumption is imposed. The mean field captures the collective informational and operational state of the machine-intelligence ecosystem. A. Payoffs The objective of agent i is to minimize: Ji (α, µ) = Ji a1 , . . . , aI , u1 , . . . , uI , µ . A typical form is: Ji = Li + λi Ri + ηi Ki + ρi Gi , where: • Li is the expected operational loss; • Ri is a expectile-risk of the cost; • Ki is a representation complexity penalty; • Gi is a governance, legal, ethical, or compliance penalty. The last component is particularly important in responsible machine intelligence. Indeed, many modern MI systems are judged not only by prediction accuracy but also by their legal and societal consequences. Examples include: • false medical diagnoses; • discriminatory lending decisions; • autonomous vehicle accidents; • critical infrastructure failures; • wrongful legal recommendations; • cybersecurity breaches. In such environments, the most damaging events are often rare and contribute negligibly to total variance. Variance-Oriented and Responsibility-Oriented Agents When ai = PCA, the agent selects a representation according to: PiPCA = arg max Tr(P ⊤ Σi P ), P ⊤ P =Iri
where Σi = Cov(xi ). The resulting architecture prioritizes information compression and variance preservation.
24
Conversely, when ai = exp2PCA, the representation solves: Piexp2PCA = arg max eτ Ci (Yi , γi (P ⊤ xi )) , P ⊤ P =Iri
where eτ denotes the expectile functional. The resulting architecture prioritizes decision responsibility and tail-risk control. PCA and exp2PCA represent two algorithms of machine intelligence, as summarized in Table V. TABLE V M ACHINE I NTELLIGENCE R EPRESENTATIONS PCA Agent
exp2PCA Agent
Variance preservation Compression-oriented Geometric objective Average behavior Information efficiency
Risk preservation Responsibility-oriented Operational objective Rare-event behavior Decision accountability
Risk Shadow Equilibrium A representation-control equilibrium (mean-field-type equilibrium) is a profile (α1⋆ , . . . , αI⋆ ) such that no agent can improve its payoff through unilateral deviation. Of particular interest are equilibria exhibiting the Risk Shadow phenomenon. Definition 16 (Risk Shadow Equilibrium). A representationcontrol equilibrium is called a Risk Shadow equilibrium if (α1⋆ , . . . , αI⋆ ) is an mean-field-type equilibrium and there exVar(Pi⊤ Xi ) ists at least one agent i such that: Var(X ≈ 1, while i) I(Yi ; Pi⊤ Xi ) ≈ 0. In a Risk Shadow equilibrium, the representation preserves nearly all observable variance while simultaneously destroying the information necessary for responsible decision making.
Implications for Responsible Machine Intelligence The proposed framework reveals a fundamental distinction between information preservation and decision responsibility. Classical machine-intelligence systems implicitly assume that preserving variance preserves value. The present framework demonstrates that this assumption may be false in strategic environments where agents interact through a common mean field and where rare but consequential events dominate societal costs. The resulting theory therefore suggests a new paradigm: Machine-intelligence systems should not be evaluated according to how much variance they preserve, but according to how effectively their representations support responsible decisions under risk. This perspective transforms dimensionality reduction from a geometric compression problem into a strategic decisiontheoretic problem and establishes a new connection between representation learning, responsible mahine intelligence, and Mean-Field-Type Game theory.
IX. C ONCLUSION AND F UTURE O UTLOOK This paper began with a simple but consequential question: Can a dimensionality-reduction method that is optimized solely for variance preservation be trusted in decision systems where the most important events are also the rarest? The analysis presented throughout this work suggests that, in general, the answer is no. We formally identified and characterized a previously underappreciated phenomenon, which we termed the Risk Shadow. The Risk Shadow emerges whenever the geometric objective of variance maximization becomes misaligned with the operational objective of minimizing decision risk. In such settings, PCA may faithfully preserve almost all of the total variance of a dataset while simultaneously discarding the very information that determines the outcome of a high-stakes decision. This is not a numerical accident, nor a pathological corner case. Rather, it is a structural consequence of optimizing an unsupervised geometric criterion in environments where utility, cost, safety, and risk are fundamentally asymmetric. Our theoretical results demonstrate that under severe class imbalance and asymmetric misclassification costs, the directions carrying the largest decision value can reside in subspaces associated with negligible global variance. The classical paradigm of “maximum variance implies maximum information” breaks down from a decision-theoretic perspective. The retained variance can approach unity while the retained operational value collapses toward zero. To address this limitation, we investigated three riskaware alternatives: Tail-Preserving PCA (TP-PCA), Expectile PCA (ExPCA), and the proposed decision-directed framework exp2PCA. The analysis reveals a clear hierarchy. TP-PCA partially compensates for rare-event dilution through sample reweighting, yet remains vulnerable to undesirable centering effects that may redirect the learned subspace toward statistically amplified but operationally irrelevant directions. ExPCA provides a substantially more principled remedy by embedding tail asymmetry directly into the geometric representation process, thereby recovering critical structures that classical PCA systematically suppresses. Exp2PCA moves beyond geometric surrogates altogether. By optimizing the downstream decision objective itself, it establishes a direct connection between representation learning and operational expectile risk, achieving universal task alignment and eliminating the fundamental source of the Risk Shadow. The empirical evidence reinforces these theoretical conclusions. Across high-dimensional synthetic experiments and a diverse collection of real-world domains including fraud detection, industrial fault diagnosis, cybersecurity monitoring, and medical imaging, expectile riskaware representations consistently recovered decision-critical information that variance-based approaches either attenuated or completely removed. The improvements were particularly pronounced in precisely those regimes where mistakes are the most costly: rare, consequential, and asymmetric events. At the same time, intellectual honesty requires acknowledging what this work does not prove. We do not claim that PCA is obsolete, nor that variance maximization is inherently flawed. PCA remains one of the most elegant, interpretable, and computationally efficient tools in modern data intel-
25
ligence. When the objective is compression, visualization, denoising, or exploratory analysis, PCA often remains an excellent choice. Our claim is narrower but more important: when dimensionality reduction serves as a precursor to consequential decision making, variance alone is generally an insufficient design principle. In such contexts, preserving what is common is not necessarily the same as preserving what matters. The results suggest a conceptual shift for representation learning. Future dimensionality-reduction systems should not be evaluated solely by how much variance they retain, but by how much decision-relevant information they preserve. The central question is no longer “What is the dominant geometry of the data?” but rather “What information is indispensable for the decisions that follow?” Limitations and Future Research Directions While the theoretical framework developed here establishes a foundation for expectile risk-aware dimensionality reduction, several important challenges remain open. The effectiveness of tail-sensitive methods depends on the asymmetry parameter τ . Although large values of τ enhance sensitivity to rare events, excessively extreme choices may increase estimator variability and amplify sampling noise in finite datasets. Developing statistically principled, data-adaptive procedures for selecting τ remains a central challenge. Future work should investigate bootstrap-based calibration, information criteria, distributionally robust selection mechanisms, and tau-inverse expectile. The present theory is developed primarily within linear projection frameworks, where interpretability and analytical tractability are strongest. Many modern datasets, however, reside on highly nonlinear manifolds. Extending expectilebased and decision-directed objectives to deep representationlearning architectures, including autoencoders, diffusion embeddings, and latent-variable models, represents a natural and potentially transformative next step. Our results establish structural properties of the Risk Shadow, but a complete statistical theory remains incomplete. Sharp finite-sample guarantees, concentration inequalities, excess-risk bounds, and uniform convergence results over constrained projection manifolds are still largely unexplored. Addressing these questions will require new tools that combine empirical process theory with asymmetric and tail-sensitive objectives. Many critical applications operate in continuously evolving environments where observations arrive sequentially and rare events emerge unexpectedly. Developing recursive, online, and adaptive variants of ExPCA and exp2PCA capable of dynamically reallocating representational capacity toward emerging risks is an important direction for both theory and deployment. Perhaps the most important open question is broader than dimensionality reduction itself. The Risk Shadow identified in this work may represent only one manifestation of a more general phenomenon: the systematic disappearance of rare but consequential signals under objectives dominated by averages. Investigating analogous effects in foundation models, large-scale multimodal systems, reinforcement learning, and autonomous decision platforms may reveal a wider class of
hidden vulnerabilities that current optimization paradigms fail to detect. This work argues that the dominant paradigm of unsupervised variance preservation is insufficient whenever the cost of overlooking a rare event exceeds the benefit of accurately representing the common case. The Risk Shadow is not merely a limitation of PCA; it is a warning about the broader consequences of optimizing for what is frequent rather than what is consequential. As machine intelligence systems increasingly influence critical societal decisions, the ability to preserve lowprobability, high-impact information may become not merely a statistical preference, but a fundamental requirement for trustworthy machine intelligence. Acknowledgments We thank the anonymous reviewers for their insightful comments. We would also like to express our sincere gratitude to the data annotators, data collectors, and data enablers from TIMADIE: Guinaga, Grabal, SK1 Sogoloton, LnG Lab, MFTG, CI4SI, WETE, IA Mali, TimaTon, ORA, and TommoSo Dogon for their invaluable contributions to this work. Data Ethics: The collection, curation, and deployment of the datasets referenced in this work were conducted in strict compliance with established ethical guidelines and institutional protocols. All data obtained via our collaborative networks including TIMADIE: Guinaga, Grabal, SK1 Sogoloton, LnG Lab, MFTG, CI4SI, WETE, IA Mali, TimaTon, ORA, and TommoSo Dogon, adhered to strict frameworks of informed consent, privacy preservation, and data minimization. No personally identifiable information (PII) or sensitive demographic attributes were processed without explicit authorization, and anonymization protocols were applied prior to analysis. The authors ensure that the experimental methodologies do not perpetuate, introduce, or amplify historical biases, and explicitly restrict the use of these models from applications involving discriminatory profiling or harmful automated decisionmaking. Data Availability: All datasets analyzed or generated during the course of this study are publicly available online. Complete source repositories, configuration files, and preprocessing scripts are fully documented and cited within the text, with corresponding persistent identifiers provided explicitly in the references section. Use of Generative AI: During the preparation of this manuscript, the authors utilized advanced generative artificial intelligence platforms and large language models solely for the purposes of code optimization, grammatical refinement, and structural formatting of the text. All technical ideas, mathematical proofs, experimental formulations, and interpretations of results remain entirely the original work of the human authors, who accept full responsibility and accountability for the accuracy, integrity, and authenticity of the final content. Referential Hallucination: To ensure the absolute integrity and reproducibility of this work, every bibliographic reference cited in this manuscript was manually verified one by one by the authors. The authors cross-checked all metadata including authors, titles, journals, publication years, volumes, issue numbers, and persistent digital object identifiers (DOIs) against
26
primary academic databases. This manual audit confirms that all cited literature exists, is peer-reviewed, and directly supports the technical claims, mathematical proofs, and contextual frameworks presented herein, thereby guaranteeing complete protection against artificial intelligence-generated referential hallucinations. Conflict of Interest: The authors declare that they have no known competing financial interests, personal relationships, or institutional affiliations that could have appeared to influence the work, methodology, or conclusions reported in this paper. R EFERENCES [1] Pearson, Karl. ”LIII. On lines and planes of closest fit to systems of points in space.” The London, Edinburgh, and Dublin philosophical magazine and journal of science 2, no. 11 (1901): 559-572. [2] Hotelling, Harold. ”Analysis of a complex of statistical variables into principal components.” Journal of educational psychology 24, no. 6 (1933): 417. [3] Jolliffe, Ian. ”Principal component analysis.” Encyclopedia of statistics in behavioral science (2005). [4] Jolliffe, Ian T., and Jorge Cadima. ”Principal component analysis: a review and recent developments.” Philosophical transactions of the royal society A: Mathematical, Physical and Engineering Sciences 374, no. 2065 (2016): 20150202. [5] Schölkopf, Bernhard, Alexander Smola, and Klaus-Robert Müller. ”Nonlinear component analysis as a kernel eigenvalue problem.” Neural computation 10, no. 5 (1998): 1299-1319. [6] Tenenbaum, Joshua B., Vin de Silva, and John C. Langford. ”A global geometric framework for nonlinear dimensionality reduction.” science 290, no. 5500 (2000): 2319-2323. [7] Roweis, Sam T., and Lawrence K. Saul. ”Nonlinear dimensionality reduction by locally linear embedding.” science 290, no. 5500 (2000): 2323-2326. [8] Belkin, Mikhail, and Partha Niyogi. ”Laplacian eigenmaps for dimensionality reduction and data representation.” Neural computation 15, no. 6 (2003): 1373-1396. [9] Donoho, David L., and Carrie Grimes. ”Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data.” Proceedings of the National Academy of Sciences 100, no. 10 (2003): 5591-5596. [10] Coifman, Ronald R., and Stéphane Lafon. ”Diffusion maps.” Applied and computational harmonic analysis 21, no. 1 (2006): 5-30. [11] Zou, Hui, Trevor Hastie, and Robert Tibshirani. ”Sparse principal component analysis.” Journal of computational and graphical statistics 15, no. 2 (2006): 265-286. [12] d’Aspremont, Alexandre, Laurent Ghaoui, Michael Jordan, and Gert Lanckriet. ”A direct formulation for sparse PCA using semidefinite programming.” Advances in neural information processing systems 17 (2004). [13] Journée, Michel, Yurii Nesterov, Peter Richtárik, and Rodolphe Sepulchre. ”Generalized power method for sparse principal component analysis.” Journal of Machine Learning Research 11, no. 2 (2010). [14] Hubert, Mia, Peter J. Rousseeuw, and Karlien Vanden Branden. ”ROBPCA: a new approach to robust principal component analysis.” Technometrics 47, no. 1 (2005): 64-79. [15] Candès, Emmanuel J., Xiaodong Li, Yi Ma, and John Wright. ”Robust principal component analysis?.” Journal of the ACM (JACM) 58, no. 3 (2011): 1-37. [16] Xu, Huan, Constantine Caramanis, and Sujay Sanghavi. ”Robust PCA via outlier pursuit.” Advances in neural information processing systems 23 (2010). [17] Tipping, Michael E., and Christopher M. Bishop. ”Probabilistic principal component analysis.” Journal of the Royal Statistical Society Series B: Statistical Methodology 61, no. 3 (1999): 611-622. [18] Bishop, Christopher. ”Bayesian pca.” Advances in neural information processing systems 11 (1998). [19] Ramsay, James O., and Bernard W. Silverman. ”Principal components analysis for functional data.” In Functional data analysis, pp. 85-109. New York, NY: Springer New York, 2005 [20] Lu, Haiping, Konstantinos N. Plataniotis, and Anastasios N. Venetsanopoulos. ”MPCA: Multilinear principal component analysis of tensor objects.” IEEE transactions on Neural Networks 19, no. 1 (2008): 18-39. [21] Kolda, Tamara G., and Brett W. Bader. ”Tensor decompositions and applications.” SIAM review 51, no. 3 (2009): 455-500.
[22] Hall, Peter M., A. David Marshall, and Ralph R. Martin. ”Incremental eigenanalysis for classification.” In BMVC, vol. 98, pp. 286-295. 1998. [23] Oja, Erkki. ”Simplified neuron model as a principal component analyzer.” Journal of mathematical biology 15, no. 3 (1982): 267-273. [24] Halko, Nathan, Per-Gunnar Martinsson, and Joel A. Tropp. ”Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions.” SIAM review 53, no. 2 (2011): 217-288. [25] Hyvärinen, Aapo, and Erkki Oja. ”Independent component analysis: algorithms and applications.” Neural networks 13, no. 4-5 (2000): 411430. [26] Hastie, Trevor, and Werner Stuetzle. ”Principal curves.” Journal of the American statistical association 84, no. 406 (1989): 502-516. [27] Abid, Abubakar, Martin J. Zhang, Vivek K. Bagaria, and James Zou. ”Exploring patterns enriched in a dataset with contrastive principal component analysis.” Nature communications 9, no. 1 (2018): 2134. [28] Samadi, Samira, Uthaipon Tantipongpipat, Jamie H. Morgenstern, Mohit Singh, and Santosh Vempala. ”The price of fair pca: One extra dimension.” Advances in neural information processing systems 31 (2018). [29] Bair, Eric, Trevor Hastie, Debashis Paul, and Robert Tibshirani. ”Prediction by supervised principal components.” Journal of the American Statistical Association 101, no. 473 (2006): 119-137. vol. 101, no. 473, pp. 119–137, 2006. [30] Bai, Liyi, et al. “Multi-phase hybrid metabolomics framework identifies clinically applicable plasma signatures for early detection of gastric cancer.” Nature Communications (2026). [31] Chakraborti, Anirban, et al. “Phase separation and scaling in correlation structures of financial markets.” Journal of Physics: Complexity 2, no. 1 (2021): 015002. [32] Cao, Yulong, et al. “Invisible for both camera and lidar: Security of multi-sensor fusion based perception in autonomous driving under physical-world attacks.” In 2021 IEEE Symposium on Security and Privacy, pp. 176-194. IEEE, 2021. [33] Li, Ker-Chau. “Sliced inverse regression for dimension reduction.” Journal of the American Statistical Association 86, no. 414 (1991): 316327. [34] Cook, R. Dennis, and Sanford Weisberg. ”Discussion of sliced inverse regression for dimension reduction.” Journal of the American Statistical Association 86, no. 414 (1991): 328-332. [35] Li, Ker-Chau. ”On principal Hessian directions for data visualization and dimension reduction: another application of Stein’s lemma.” Journal of the American Statistical Association 87, no. 420 (1992): 1025-1039. [36] Embrechts, Paul, Claudia Klüppelberg, and Thomas Mikosch. Modelling extremal events: for insurance and finance. Vol. 33. Springer Science & Business Media, 2013. [37] De Haan, Laurens, and Ana Ferreira. Extreme value theory: an introduction. New York, NY: Springer New York, 2006. [38] Liu, Xu-Ying, Jianxin Wu, and Zhi-Hua Zhou. ”Exploratory undersampling for class-imbalance learning.” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 39, no. 2 (2008): 539-550. [39] He, Haibo, and Edwardo A. Garcia. ”Learning from imbalanced data.” IEEE Transactions on knowledge and data engineering 21, no. 9 (2009): 1263-1284. [40] Lin, Tsung-Yi, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. ”Focal loss for dense object detection.” In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988. 2017. [41] Artzner, Philippe, Freddy Delbaen, Jean-Marc Eber, and David Heath. ”Coherent measures of risk.” Mathematical finance 9, no. 3 (1999): 203228. [42] Acerbi, Carlo, and Dirk Tasche. ”On the coherence of expected shortfall.” Journal of banking & finance 26, no. 7 (2002): 1487-1503. [43] Bellini, Fabio, Bernhard Klar, Alfred Müller, and Emanuela Rosazza Gianin. ”Generalized quantiles as risk measures.” Insurance: Mathematics and Economics 54 (2014): 41-48 [44] N. Tishby, F. Pereira, and W. Bialek, “The information bottleneck method,” arXiv:physics/0004057, 2000. [45] Hadsell, Raia, Sumit Chopra, and Yann LeCun. ”Dimensionality reduction by learning an invariant mapping.” In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol. 2, pp. 1735-1742. IEEE, 2006. [46] Oord, Aaron van den, Yazhe Li, and Oriol Vinyals. ”Representation learning with contrastive predictive coding.” arXiv preprint arXiv:1807.03748 (2018). [47] Xu, Huan, Constantine Caramanis, and Shie Mannor. ”Outlier-robust PCA: The high-dimensional case.” IEEE transactions on information theory 59, no. 1 (2012): 546-572.
27
[48] Li, Ker-Chau. “Sliced inverse regression for dimension reduction.” Journal of the American Statistical Association 86, no. 414 (1991): 316327. [49] Deng, Minying. “Extreme value theory in high dimensional statistical models and its application in anomaly detection.” In 2024 4th International Conference on Mobile Networks and Wireless Communications, pp. 1-5. IEEE, 2024. [50] She, Rui, Shanyun Liu, Shuo Wan, Ke Xiong, and Pingyi Fan. ”Importance of small probability events in big data: Information measures, applications, and challenges.” IEEE Access 7 (2019): 100363-100382. [51] Newey, Whitney K., and James L. Powell. “Asymmetric least squares estimation and testing.” Econometrica 55, no. 4 (1987): 819-847. [52] Kaggle. “Credit Card Fraud Detection.” https://www.kaggle.com/ datasets/mlg-ulb/creditcardfraud [53] Ebinezer, M.J.D.; Krishna, B.C. Life Insurance Fraud Detection: A DataDriven Approach Utilizing Ensemble Learning, CVAE, and Bi-LSTM. Appl. Sci. 2025, 15, 8869. https://doi.org/10.3390/app15168869 [54] Kaggle. “Vehicle Insurance Claim Fraud Detection.” https://www.kaggle. com/datasets/shivamb/vehicle-claim-fraud-detection. [55] ProcessMiner, Inc. https://processminer.com/dataset-rare-eventmultivariate-time-series-part-1/ [56] Chitta Ranjan, Mahendranath Reddy, Markku Mustonen, Kamran Paynabar, Karim Pourak: “Dataset: Rare Event Classification in Multivariate Time Series.”https://arxiv.org/abs/1809.10717, 2018. [57] Sharafaldin, I., Habibi Lashkari, A. and Ghorbani, A. A. (2018). Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization. In Proceedings of the 4th International Conference on Information Systems Security and Privacy - ICISSP; ISBN 978989-758-282-0; ISSN 2184-4356, SciTePress, pages 108-116. DOI: 10.5220/0006639801080116 [58] Panigrahi, R., & Borah, S. “IDS2025: Balanced Intrusion Detection Evaluation Dataset.” Mendeley Data, 2025. https://data.mendeley.com/ datasets/pkskt3fv3v/1 [59] CXR-LT 2026 Challenge. “Multi-Center Long-Tailed and Zero Shot Chest X-ray Classification.” arXiv:2602.22092, 2026. [60] Sérgio Moro, P. Cortez, P. Rita. A data-driven approach to predict the success of bank telemarketing, 2014, Published in Decision Support Systems, “Bank Marketing.” UCI Machine Learning Repository, 2014. https://archive.ics.uci.edu/dataset/222/bank+marketing [61] Kaggle. “Telco Customer Churn.” https://www.kaggle.com/datasets/ blastchar/telco-customer-churn. [62] Lending Club. “Loan Data.” Available via Kaggle and IEEE Dataport. https://www.kaggle.com/datasets/wordsforthewise/lending-club [63] Kaggle. “Synthetic Industrial Metal Surface Defects.” https://www.kaggle.com/datasets/tatheerabbas/ synthetic-industrial-metal-surface-defects. [64] T. Başar, B. Djehiche, and H. Tembine, Mean-Field-Type Game Theory: Foundations and New Directions, Vol. 1, Springer Nature Switzerland AG, Birkhäuser Cham, 2025, 845 pages, ISBN: 978-3-032-07026-5. [65] T. Başar, B. Djehiche, and H. Tembine, Mean-Field-Type Game Theory: Applications, Vol. 2, Springer Nature Switzerland AG, Birkhäuser Cham, 2025, ISBN: 978-3-032-07030-2. [66] Hamidou Tembine and Issa Bamia and Massa Ndong and Bakary Coulibaly and Oumar Issiaka Traore and Moussa Traore and Moussa Sanogo and Mamadou Eric Sangare and Salif Kante and Daryl Noupa Yongueng and Hafiz Tiomoko Ali and Malik Tiomoko and Frejus Laleye and Boualem Djehiche and Wesmanegda Elisee Dipama and Idris Baba Saje and Hammid Mohammed Ibrahim and Moumini Sanogo and Marie Coursel Nininahazwe and Abdul-Latif Siita and Haine Mhlongo and Teddy Nelvy Dieu Merci Kouka and Mariam Serine Jeridi and Mutiyamuogo Parfait Mupenge and Lekoueiry Dehah and Abdoul-Aziz Bio Sidi D. Bouko and Wilfried Franceslas Zokoue and Odette Richette Sambila and Alina RS Mbango and Mady Diagouraga and Oumarou Moussa Sanoussi and Gizachew Dessalegn and Mohamed Lamine Samoura and Bintou Laetitia Audrey Coulibaly, Breaking the Barriers of Text-Hungry and Audio-Deficient AI, 2025, arXiv, 2506.02443, [67] Hamidou Tembine: The Ghost in the Index: Knowledge Exclusion and the Fallacy of the Low-Resource Label: A Position paper, Submitted manuscript, 2026. [68] Hamidou Tembine: Causal Mirage Equilibrium in Agentic Machine Intelligence, Submitted manuscript, 2026.
B IOGRAPHY
Hamidou Tembine (SM’13) is the co-founder of Timadie, Grabal AI Mali, co-chair of TF, founder of Guinaga, WETE, MFTG, LnG Lab, CI4SI and a Professor of Artificial Intelligence at UQTR, Quebec, Canada. He graduated in Applied Mathematics from Ecole Polytechnique (Palaiseau, France) and received the Ph.D. degree from INRIA and University of Avignon, France. He further received his Master degree in game theory and economics. His main research interests are learning, evolution, and games. In 2014, Tembine received the IEEE ComSoc Outstanding Young Researcher Award for his promising research activities for the benefit of society. He was the recipient of 10+ best paper awards in the applications of game theory. Tembine is a prolific researcher and holds 300+ scientific publications including magazines, letters, journals and conferences. He is author of the book on “distributed strategic learning for engineers” (published at CRC Press, Taylor & Francis 2012) which received book award 2014, and co-author of the book “Game Theory and Learning in Wireless Networks” (Elsevier Academic Press) and co-author of the book on “MeanField-Type Games I-II ” (Springer Nature) and for Engineers’ (CRC Press0, and the author of the book “GPT Meets Game Theory”. Tembine has been coorganizer of several scientific meetings on game theory in agriculture, water, food, environment, networking, wireless communications and smart energy systems. He has been a visiting researcher at University of California at Berkeley (US), University of McGill (Montreal, Quebec, Canada), University of Illinois at Urbana-Champaign (UIUC, US), Ecole Polytechnique Federale de Lausanne (EPFL, Switzerland) and University of Wisconsin (Madison, US). He has been a Simons Participant and a Senior Fellow 2020. He is a senior member of IEEE. He is a Next Einstein Fellow, Class of 2017.