Official Review · ICLR 2026

This paper addresses the problem of tracing unauthorized fine-tuning of diffusion models by embedding imperceptible watermarks into training datasets. The authors claim to propose a unified evaluation framework for dataset watermarking methods, introducing three dimensions, i.e., "Universality," "Transmissibility," and "Robustness", to assess their performance. They further conduct experiments across different fine-tuning settings on 4 existing methods on Stable Diffusion v1.4. Finally, the paper presents a so-called "practical" watermark removal method that is said to expose vulnerabilities in existing approaches. - The topic of this paper, i.e., tracing unauthorized fine-tuning or dataset usage in diffusion models, is important and relevant. - The overall evaluation design appears reasonable, though this is mainly because it adheres closely to what previous works have already done. - The claimed "unified evaluation framework" lacks sufficient detail and support. The authors repeatedly claim to evaluate all watermarking methods "under a unified threat model" and "in a fair manner", yet provide no experimental protocol or hyperparameter details on how the baselines are trained and evaluated. As far as I know, the baselines themselves are fundamentally different in nature. DIAGNOSIS and SIREN use a hypothesis testing mechanism (binary or one-class classification results on an image distribution) to determine model infringement, while WatermarkDM and DiffusionShield decode multibit watermarks and measure bit accuracy. These paradigms are not directly comparable, yet the paper does not explain how they were put under the same evaluation framework, nor how the "ACC" metric is computed. This raises strong concerns about the fairness and validity of the comparisons. - The reliability and validity of the experimental protocal and experimental results are highly questionable. To be honest, I am shocked by some results reported in the paper, as many findings are drastically inconsistent with the well-established consensus in the community: - According to the evaluations in DIAGNOSIS, SIREN, and other literature [8, 9], WatermarkDM (which reuses the same decoder from Yu et al.) should perform close to random guessing (around 50% watermark bit accuracy) in the fine-tuning setting due to the limited learnability of image watermarks. In contrast, in my understanding, SIREN is explicitly designed for diffusion model dataset tracing. It introduces a learnability loss to address the unlearnable nature of prior watermarking methods. The original SIREN paper also reports to achieve nearly 100% detection accuracy across multiple datasets (even under FPR = $10^{-9}$). The original SIREN paper also evaluated settings with partial clean data and under certain attack conditions, still obtaining strong results. This makes the claim in the current paper that "SIREN exhibits the lowest performance, with a detection accuracy of approximately 50% across all datasets an