Peer review SmsCcr5j

The authors empirically study, for various transformer-based vision models, the relationship between the two concepts _human-machine alignment_ and _concept convexity_. For both of these concepts they evaluate an existing proxy/metric, namely _graph convexity_ and the _odd one out accuracy_, and evaluate these on the latent representations of pictures from based classes after various transformer layers in the model. The authors mention that this study provides the first evidence of a relationship between the two concepts. Although it is difficult to draw conclusions from correlation coefficients, the authors also performed an intervention study where they optimize for odd-one-out-accuracy to see the influence on convexity, showing that especially for models that are not finetuned, also the convexity increases. Especially at the high level, I believe the question on the relationship between the representations in machine-learning models and human-machine alignment is interesting. The article is neatly written, and on a high level the story becomes clear quickly. It is also interesting to study these numbers, and one can probably get more interesting statistics out of them. It is difficult to analyze correlation scores to interpret relationships, but the authors alleviate this at least partly by doing an intervention study as well: in one of their experiments, they optimize for human-alignment, and measure the effect on convexity. In contrast to the larger story, the details were more difficult to figure out for me, and I am still uncomfortable pointing out exactly how various quantities are computed. The authors mention for instance "We performed a correlation analysis of the two scores using Pearson’s R on a layer-wise basis across all models." It was originally unclear to me whether there the number of datapoints in the computation of such a correlation would be just be the twelve averaged scores corresponding to the twelve trained models, or whether there would be a data point for every triplet of classes, for instance. A formula might help, or an even more precise wording. Similarly, the appendix mentions "The latent representations were extracted according to the way they were used by the classifier in the original implementation. Hence, we took the vector corresponding to the classifier token for ViT and averaged over all the other tokens for data2vec and BEiT." I have difficulty understanding what this means. In general, I am unsure how and where the latent representation of a class is actually computed. This is relevant for the interpretation of the convexity concept. Although not necessarily a weakness of the paper, it is in a way difficult to connect the proxy to the intended concept, i.e. graph convexity to a measure of convexity, and out-of-order-alignment with human-machine alignment. For instance, whereas the authors in the conclusion talk about the formation of convex regions, it is not necessary for achieving a high correlati