Official Review · ICLR 2026

This paper introduces a map navigation benchmark based on 2D sparse grid navigation tasks, and uses it to evaluate transformer models with LLaMA-style architecture, The authors analyze model performance on this benchmark using two dimensions: - Transfer (systematicity): solving the same class of problems in new environments, and - Scaling (productivity): solving harder problems after mastering simpler ones. They consider two model training paradigms: (1) Supervised fine-tuning, with paths encoded as movement directions; model predicts continuation given start/end prompt. (2) Reinforcement learning, giving binary reward for valid shortest paths. They find that for supervised fine-tuning distinct questions drive transfer more than multiple solutions per question. To determine which types of questions best support spatial transfer they focus on coverage (fraction of unique nodes in the training map included in training) vs diversity (average number of distinct endpoints each start node is paired with.) of training questions. The authors focus on defined generalization dimensions (transfer and scaling) and the experimental design appears to be sound, separating coverage vs diversity. Overall, this work clearly *intended as* a methodologically solid framework with careful experiments, however poor presentation makes this conceptual design very hard to understand and evaluate. Presentation is very poor, making it hard to follow this paper. Beginning as early as the abstract, the writing is often superficially coherent, but conceptually incoherent - reflecting perhaps the lack of conceptual depth, or perhaps a last minute writing of work that is intended as well thought out, but obscured by poor presentation. For instance: "_Someone who learns to walk shortest paths in New York can, upon receiving a map of Paris, immediately apply the same rule to navigate, despite never practicing there._" Same rule (as in, the same heuristic) or same algorithm (as in, Djkstra's algorithm)? Much of the field would not agree with this statement, as New York has a grid-like North American design, and Paris is an older city - meaning that different rules/heuristics and approximations will be optimal. The next sentence then jumps topic to combining rules - "_This ability to recombine known rules to solve novel problems exemplifies compositional generalization_.." There is no sense how this is related to the previous sentence, or the remainder of the abstract. I am unable to evaluate the paper, because I can not unearth the content from the ineffective presentation. I feel that this work is not yet ready for an archival publication, but after a careful revision it might be. N/A