Official Review · ICLR 2024

The work is related to applying reinforcement learning to decide how a join query (a query that cross-references multiple tables) can be executed most efficiently by a database system which is an NP-hard problem where existing solutions still leave a lot of room for improvement. The paper proposes a simulator/benchmark to evaluate the performance of reinforcement learning approaches and features an extensive empirical study that compares some existing approaches. S1) Originality: Thorough investigation of existing reinforcement learning approaches and useful tool/benchmark for future works. S2) Significance: Reinforcement learning is a reasonable approach for the join order problem and shedding light on what works best can contribute towards faster join execution not just in general, but also for particular data domains / systems. S3) Presentation: The paper presents ideas clearly, particularly figures seem to illuminate concepts well and the literature review appears to be thorough. W1) Significance: The conclusion of compared reinforcement learning approaches could be clearer in terms of which kind of predictions it enables. A non-learning baseline would also help to root the results. W2) Originality: The work does not seem to (explicitly) propose any novel approach. W3) Relevance: The focus of the paper seems to be less on reinforcement learning and more about a particular application of reinforcement learning in database management systems. Q1) What can be learned about the performance of different reinforcement learning approaches in this benchmark and how does it compare to the performance of a non-learning approach (as used in modern commercial systems)? Q2) Do the any of the results conclude any new approach or variation of an approach that has not been previously considered? Q3) What kind of progress does this work make in terms of reinforcement learning approaches (e.g., via trial and error) beyond measuring their empirical performance?