Official Review · ICLR 2026

This paper studies AR diffusion, which reverses a patch-wise diffusion model with conditioning from already sampled patches. The authors establish an error bound for AR diffusion models, and discusses the conditions on which AR diffusions outperform conventional diffusion models. They also empirically compare AR and conventional diffusion models through two toy examples, confirming that a certain conditional structure can be better learned by AR diffusions. The paper is well written, and the mathematical derivations are easy to follow. Theorem 1 decomposes the error/complexity into $K$-related term (coming from $K$-stage AR formulation) and standard term (coming from usual diffusion error estimates), which helps the understanding of AR diffusion modeling. The toy examples also confirm the intuition that the AR formulation explicitly learning conditional distributions can be helpful when the patch decomposition aligns with the data-generation process. A large portion of the paper (pages 1-6) is just rewriting the standard formulations of diffusion models (such as Chen et al. (2023b)) into AR-based formulations. The essential contributions of the paper seems to be the results in Section 4, but it has the following weaknesses: - [W1] The complexity obtained in Theorem 1 (and its remark) seems to be just $K\times$ complexity of standard diffusion. Technically, it seems to be just a combination of stage-wise estimates of standard diffusion models. Furthermore, while the authors state "Moreover, we prove that, in certain regimes, AR diffusion can better capture some conditional dependencies among patches compared to standard diffusion" in Introduction, the "regime" is described only as $\epsilon(K)\ll K^{1/2}\epsilon(1)$, and it is far from characterizing the situation (in terms of data and/or network) where AR diffusion is more efficient. - [W2] **The results in Section 4.2 is quite misleading.** First, for Lemma 4, the conditional KL divergence is bounded by using the conditional loss $L^{SM}(\theta|x_{[1:k]})$ and the authors state it can be arbitrarily small. However, in the loss functions such as Eq. 14, this conditional loss is taken expectation regarding the choice of $x_{[1:k]}$, and there is no reason that $L^{SM}(\theta|x_{[1:k]})$ is uniformly small for each $x_{[1:k]}$. Similarly, in Lemma 5, the right-hand side can be large when $\lVert x_{1:k}\rVert$ is moderately large, but in the example constructed by authors (in Appendix C), $\lVert x_{1:k}\rVert$ **tends to be very small**, since it follows a centered Gaussian with variance of order $O(\epsilon^2/M^2)$. Thus, "unbounded KL divergence" stated by the authors in Introduction is an inappropriate exaggeration of the result. Indeed, from the chain rule of KL divergence, the conditional KL divergence is small in expectation, if the joint KL divergence is small. Overall, I believe that the paper's theoretical contribution is only in Section 4.1, which gives a bound for AR diffusion models