Representation Similarity Does Not Predict Transformation Stability in Neural Networks: An Empirical and Structural Study

Note—Updated and extended version of a previously published paper titled “Representation Similarity And Transformation Stability: An Empirical Study of Structural Anti-Correlation in Neural Networks.” We investigate the relationship between representation similarity and transformation-dependent stability in neural networks. While Centered Kernel Alignment (CKA) is widely used to measure representational similarity, its relationship to functional robustness under input transformations remains insufficiently understood across architectures. Across multiple experimental regimes—including architecture variation, dataset shifts, and input perturbations—we observe a statistically stable coupling between CKA and a stability metric S(M) measuring sensitivity to structured transformations. This relationship is stable under bootstrap analysis and persists across perturbation types, while exhibiting architecture-dependent sign inversion. These results suggest that representation similarity is not a sufficient predictor of transformation-dependent stability in the evaluated settings. Instead, their relationship appears structured and is consistent with a role of architectural inductive bias. We further propose a minimal structural interpretation in which this coupling depends on alignment between representation subspaces and transformation-sensitive directions.