Empirical Decoupling of Representation Similarity and Transformation Stability in Neural Networks

Representation similarity metrics such as CKA are widely used to compare neural network representations across architectures and training conditions. In parallel, transformation-dependent stability measures quantify sensitivity to structured input perturbations. This work empirically investigates the relationship between these two notions. We find that representation similarity and stability exhibit a consistent positive correlation under standard evaluation conditions. However, this relationship becomes significantly weaker and more variable under structured distribution shifts. Rather than a fixed equivalence or independence, our results indicate a regime-dependent coupling between representation similarity and transformation-dependent stability. These findings suggest that representation geometry alone is insufficient to fully characterize functional stability under distributional perturbations.