Official Review · ICLR 2025

The paper introduces a new protein ligand benchmark: * Data: It uses existing established datasets (posebusters, casp15 etc) and also adds apo data to them by using AF3, compares different datasets in terms of pocket-ligand interaction similarity. * Benchmarking tasks: It targets well established benchmarking task of single ligand docking and adds a multi ligand task, including molecular fragment docking. * Benchmarked models: A wide and diverse set of methods across different categories, from more traditional to more recent generative ones: ADVina, Tulip, FABind, RF-AA, DynamicBind, NeuralPLexer, Chai-1, DiffDock-L. * Benchmark evaluations: The well-accepted RMSD <2A criterion and PoseBusters structural validity is used for single ligand evals, for multiligand docking CASP15 criteria (% RMSD<2A and lDDT-PLI) are given. * Results: Single ligand docking case - 1) training on diverse data boosts generalizability. 2)Generative methods outperform conventional docking tools in single ligand docking. 3)in multi ligand or fragment docking case, the situation is more complex, success rate of all models are lower, NeuralPlexer seems to do better than others, authors attribute this to the steric clash loss used in this model. The following are the contributions that make this paper significant to the field of protein-ligand docking * Predicted protein unbound (predicted-apo) state benchmark * Multiligand/fragment docking benchmark * Use of relatively recent models like DiffDock-L and Chai-1 in the benchmarks as well as AF3 structures * Hybrid models such as combination of different pocket-finder ML tools with pocket-requiring conventional tools. This is because benchmarks of these kinds have mostly been performed within model development efforts and not systematically across models. * There was been a recent attempt at extensive docking benchmarking called _Plinder_ and it brought an important understanding to the field about the amount of leak from train to widely used benchmark sets. There is no analysis in this work that makes use of that highly relevant work, one would expect at least a leak measurement for different ML models. * Despite apo state benchmark being the strength of this paper, there is no apo vs holo analysis in the main text. Some relevant context is given in the appendix. Should be summarized in the main text, e.g. analysis based on apo-to-holo RMSD of the _pocket_ * In multiple occasions important nuance about results is not given in text, I will highlight few examples: * Chai-1 significantly outperforms others in the single ligand docking task if PB validity is to be considered as a metric, however the same model performs very poorly in the DockGen dataset according to Fig 13 in Appendix; and in pocket-only case, results get better however the error bar is pretty large. So although the final conclusion is correct, that “are we there yet?” question is answered negatively, details like this should be surfaced from app