Neural collapse is globally optimal in deep regularized ResNets and transformers

The empirical emergence of neural collapse—a surprising symmetry in the feature representations of the training data in the penultimate layer of deep neural networks—has spurred a line of theoretical research aimed at its understanding. However, existing work focuses on data-agnostic models or, when