Growing to See: Efficient Pretraining of Self-Supervised Vision Transformers

Progressive depth growth builds large networks by starting with shallow models and inserting layers during training. Recent advances in large language models have renewed interest in this longstanding approach, demonstrating benefits for training efficiency and reasoning alongside changes in interna