Official Review · ICLR 2026

The paper considers a weak-to-strong generalization setup with ridgeless regression under spurious correlations with teacher and student models having fixed sets of features that entangle the core and group features differently. They show that in the appropriate limit of sample sizes and core feature dimension, weak-to-strong generalization happens (i) whenever the spurious correlations are the same and there is enough teacher labeled samples, (ii) when there is a mismatch of spurious correlations between the teacher and the student the the W2S gain decreases with this difference, and (iii) algorithmic improvement to standard W2S training pipeline that improves the W2S student performance when there are spurious correlations. The main significance of the paper is: 1. Consider regression with spurious correlation in the context of W2S is an interesting way to capture different pre-trainings and capacities of student and teacher models 2. Provides crisp characterization of weak-to-strong generalizations in regression with spurious correlations in the appropriate limit of sample sizes and feature dimensions in terms of mismatch in level of spurious correlation, similarity between student and teacher features, and the number of samples 3. Alternative way of performing W2S training when the spurious correlations don’t align with retraining the student model on high confidence points which improves over the vanilla W2S. The idea of considering spurious correlation is original. The writing is clear and the mathematical arguments seem sound. 1. The main theorem, Theorem 3, does not formally establish weak-to-strong generalizations. It only gives a sufficient condition under which weak-to-strong generalization happens in this case in terms of $p_{T S}$. It would be more interesting to find natural choices of $\phi_S$ and $\phi_T$ for which we can understand $p_{T S}$ and therefore W2S improvement. 2. Here the W2S improvement, like in other ridgeless regression setups, comes “artifically” from the fact that the teacher is trained with too little samples and thus is not optimally. The student is able to improve over the teacher because it has sufficient samples, and so the effect of student having better features than the teacher is confounded with the effect of the teacher being suboptimal. 3. The paper asks whether W2S still happens in data with spurious correlation, but the definition of what makes a teacher “weak” and a student “strong” (Assumption 2) is based on the spurious correlations, i.e. that the weak model entangles the core and group features and the strong model doesn’t. The result could be further strengthened with a more agnostic way of defining weak and strong which implies Assumption 2. That way, we could use this setup to get insights on the mechanisms underpinning W2S generalization. 4. It is unclear how broadly applicable algorithmic insights are to other non-vision related W2S setups. 1. Is there a choice of parameters in Assump