Official Review · ICLR 2025

This paper proposes CubeDiff, a method that generates panoramas from text or single-image condition. The core of CubeDiff is to represent a panorama with cubemaps, 6 images representing the 6 faces of a cube that seamlessly capture the scene in an inside-out way. With this representation, CubeDiff adapts a pre-trained text-to-image diffusion model to generate the 6 cube-face images simultaneously. Similar to previous works like MVDream and Wonder3D, the proposed method extends the self-attention layer in the pre-trained denoising network to operate on multiple images for learning holistic multi-view consistency. Experiments on Laval Indoor and Sun360 datasets show that CubeDiff outperforms previous panorama generation methods (MVDiffusion, Diffusion360, etc,) across various generation settings (text-conditioned, image-condition, or both). 1. CubeDiff's design choices are reasonable and backed up with a detailed discussion and comparison of different panorama representations. Extending the original self-attention over image patches of a single image to handle multiple views is a common strategy in the literature of multi-view diffusion models and is a natural choice here, considering the cube map representation. 2. The paper provides a detailed discussion on different representations for the panorama in Sec.3. Together with the related works (Sec.2), choosing cube maps as the generation targets of the multi-view diffusion model is well-motivated. 3. The experiments cover a rich set of conditioning settings, and CubeDiff demonstrates supreme performance both qualitatively and quantitatively. The anonymous website provides an interactive qualitative comparison, which helps better understand the quality of different approaches. 1. The experiment section of the paper lacks some critical ablation studies and is not complete: 1) Sec4.2 states the importance of the Synchronized GroupNorm, but there is no quantitative analysis for this design; 2) Sec4.4 claims the design of overlapping predictions between the nearby faces can reduce the generation artifacts, but again lack a corresponding ablation study. 2. The overall technical contribution is relatively limited - there are no new designs from the modeling perspective. However, given the good empirical performance and the simplicity of the approach, this is not necessarily a vital weakness. Therefore, I hope the authors can address the first point above to complete the ablation studies for the core modeling components described in Sec.3 and Sec.4. 3. It is unclear to me if the training data of the baselines (e.g., MVDiffusion and DIffusion360) is the same as the proposed CubeDiff. If they use different training data, would the comparisons be unfair? Ideally, the authors should re-train those diffusion-based baselines with the same training data. Please refer to the Weaknesses section for the questions. In general, the proposed CudeDiff is simple and obtains good experimental results. However, t