Official Review · ICLR 2026

The paper presents a new benchmark for system identification methods called SysIdBench. Having a good benchmark in any field is crucial, as it enables the systematic study of existing methods and facilitates the transfer of this knowledge to real-world problems. The benchmark was created based on synthesized developer requirements across various applications of system identification methods, including accuracy, frequency recovery, heterogeneity, out-of-distribution generalization, and robustness. It includes the results of four different system identification methods evaluated on four datasets, using five performance metrics across both in-distribution and out-of-distribution test sets. The main contribution lies in defining five evaluation metrics related to the five developer requirements. However, the paper lacks sufficient detail to justify the design choices made in the experimental setup—for example, the selection of datasets, the choice of benchmarked methods, and the approach used for splitting the data into training, validation, and test sets. These aspects require more careful consideration, and different strategies should be analyzed to assess sensitivity. Furthermore, the performance analysis is not adequately conducted. A deeper investigation is needed to demonstrate the importance of the proposed benchmark, including whether the datasets are correlated based on the newly defined metrics. If the dataset distributions in the meta-space are too similar, the benchmark may yield biased results. The main issue with benchmarking today is that many contributions introduce new benchmarks without a clear understanding of their strengths and weaknesses. These are crucial steps that are missing from the paper. Therefore, I recommend that the paper be rejected. The paper introduces a new benchmark for system identification methods, comprising four methods evaluated on four datasets using five performance metrics. These metrics correspond to key development requirements for system identification across different application domains, making the contribution highly relevant for advancing the field. The originality of the paper lies in defining five evaluation metrics aligned with the five development requirements for system identification—accuracy, frequency recovery, heterogeneity, out-of-distribution generalization, and robustness. This represents a significant improvement over existing benchmarks, which typically focus on a single metric, most often accuracy. The paper lacks sufficient arguments regarding the criteria used to select the four datasets included in the benchmark. Could you please also elaborate on how these datasets differ from those used in other benchmarks for system identification? The paper also does not provide justification for the choice of the method used for determining the truncation length. It reads as if one method was arbitrarily chosen—what would happen if a different method were used, and how would that influence