GameMind
Native3D: End-to-End 3D Scene Generation via Unified Mesh-Texture Modeling and Semantic Alignment
arXiv:2606.07117v1 [cs.CV] 5 Jun 2026
Yibo Liu∗ , Ziwei Zhang∗ , Haozhou Pang, Menghao Li, Lanshan He, Gan QiB Kuaishou GameMind Lab {liuyibo10, zhangziwei09, panghaozhou, limenghao, helanshan, qigan}@kuaishou.com *
Abstract This paper presents Native3D, the first end-to-end 3D scene generation framework that completely bypasses 2D intermediate representations. Traditional approaches typically require adapting 3D representations to the 2D domain to leverage pre-trained diffusion models, which inevitably introduces domain adaptation issues including geometric structural distortion and texture detail degradation. To address these limitations, we design a unified mesh-texture joint representation that simultaneously models both geometric structures and texture features through a Transformer-based scene encoder, effectively maintaining spatial relationships and visual consistency among objects within scenes. We further propose the 3D Representation Alignment Loss (3D REPA Loss), which employs an improved contrastive learning mechanism to align multi-level semantic representations in the latent space, significantly enhancing geometric and textural fidelity. Experimental results demonstrate that Native3D outperforms existing methods in both generation quality and editing flexibility, providing a novel solution for 3D scene editing.
1. Introduction The accelerating adoption of virtual reality (VR), augmented reality (AR), gaming, and digital media has created unprecedented demand for automated generation of highquality 3D indoor scenes. Traditional 3D modeling workflows impose substantial barriers. These include specialized expertise, expensive software licenses, protracted production timelines, and prohibitive costs that preclude scalable deployment. Practitioners across interior design, virtual production, film industry, and architectural visualization require systems capable of synthesizing photorealis* Equal contribution. B Corresponding author.
tic, stylistically coherent, and geometrically detailed indoor environments from minimal input, such as textual descriptions, reference imagery, or rough sketches. Recent advances in deep generative models present promising avenues for addressing these challenges. Largescale pretrained text-to-image diffusion models [30] exhibit remarkable proficiency in image generation and editing, establishing new paradigms for texture generation, style transfer, and content creation in architectural contexts. Concurrently, self-supervised vision transformers [2, 25, 28] demonstrate superior capabilities in semantic feature extraction and cross-image correspondence, enabling more precise scene-level appearance manipulation. Existing approaches to 3D indoor scene editing fall into two paradigms, each with fundamental limitations. The first treats scene editing as purely a 2D pixel-space problem [13, 45], leveraging semantic correspondences from pretrained diffusion models or multi-view sampling mechanisms. The core limitation of these methods lies in the absence of explicit 3D geometric constraints. 2D representations fundamentally lack the capacity to reason about 3D structures and depth relationships, resulting in cross-view inconsistencies, blurring artifacts, and visible seams under large viewpoint variations. The second paradigm adopts hybrid 2D-processing-3D-reconstruction strategies [7, 9, 33, 35], where inputs and outputs exist in 3D space but core operations remain anchored in 2D through iterative image editing coupled with NeRF or 3D gaussian splatting. These approaches face critical challenges in consistency propagation between 2D editing and 3D reconstruction. Lacking multi-view geometric constraints, 2D diffusion models produce cross-view inconsistencies when lifted to 3D, resulting in fine-grained misalignment and slow convergence. These limitations converge on a central challenge: multiview consistency, rooted in a fundamental representation mismatch. Both pure 2D methods and hybrid approaches suffer from an inherent domain gap. 2D models cannot rea-
Accepted to CVPR 2026 Findings
"Please add a sofa to the appropriate location."
"Move the sofa to the appropriate location"
"Remove the bed from the room"
"Apply cartoon-style to the entire room"
Remove
Style
Native3D
Add
Move
Figure 1. Native3D supports multiple scene editing tasks: object addition, spatial rearrangement, object removal, and appearance style transfer.
son about 3D structure, depth relationships, or spatial topology. Without explicit geometric constraints across viewpoints, they fail to maintain global consistency in geometry and appearance. This becomes particularly problematic when handling intricate details, preserving long-range coherence, or processing scenes with multiple objects, manifesting as cumulative errors, visual artifacts, and temporal flicker. The problem is further complicated by a persistent trade-off between efficiency and flexibility. Optimizationbased techniques require known camera poses, dense captures, and substantial computation, making them impractical for sparse or casual scenarios. Forward-inference methods operating in 2D space offer speed but sacrifice 3D reasoning, struggling with occlusion handling and substantial viewpoint shifts. What the field needs are solutions that bridge this domain gap while achieving robust multi-view consistency without compromising computational practicality or editing versatility. In this paper, we purpose Native3D, the first end-toend 3D-native generation framework that fundamentally circumvents the inherent domain gap issues prevalent in traditional 2D feature projection approaches. Our method directly takes 3D mesh geometry and texture information as input, achieving unified representation of global semantic information and local geometric details through a joint modeling mechanism and hierarchical feature encoder. This hybrid feature representation not only comprehensively encodes the geometric topology and visual appearance of 3D models, but also preserves precise spatial positioning infor-
mation and object-level fine-grained features. For the generation module design, we adopt Diffusion Transformer (DiT) as the core architecture, which directly learns the diffusion process in high-level 3D feature space, completely eliminating information loss caused by 2D-3D cross-domain conversion. To enable flexible scene editing capabilities, our system introduces natural language as conditional control signals, allowing users to precisely specify complex scene generation and modification requirements through textual descriptions. The output from the DiT module is reconstructed into standard 3D mesh and texture representations through a specifically designed decoder. To further enhance the local detail fidelity of generation results, we propose the 3D REPA Loss. This loss function leverages a frozen Direct3D [38]Encoder to extract 3D feature representations from both generated results and ground truth samples: for generated models, it extracts global features of the entire scene; for ground truth, it simultaneously extracts scene-level global features and object-level local features. By aligning these multi-scale representations in semantic feature space, the 3D REPA Loss guides the model to accurately restore geometric details and texture quality of individual target objects while maintaining global consistency. Our work makes three primary contributions to 3D scene generation. (1) We present Native3D that completely bypasses 2D intermediate representations, eliminating domain gap issues at the architectural level. (2) We design a unified feature representation that jointly models mesh geom-
etry and texture appearance, enabling simultaneous capture of global layouts and local details. (3) We introduce the novel 3D REPA Loss that aligns multi-level semantic representations to significantly improve geometric and textural fidelity.
2. Related Works 2.1. Text-to-3D Scene Generation Early text-to-3D generation methods [5, 16, 23] primarily relied on paired text-3D datasets for training, employing GANs or latent diffusion models to learn feature representations of 3D objects. These approaches improved generation quality by scaling up training datasets, but were fundamentally limited by the scarcity of 3D data and high annotation costs, making it challenging to extend them to complex scene generation. To address the scarcity of 3D training data, recent methods [20, 26, 27, 37, 41] leveraged powerful 2D text-to-image diffusion models as priors for 3D generation. Among these, DreamFusion pioneered the Score Distillation Sampling loss function, which optimized randomly initialized 3D representations through gradient descent. Magic3D further accelerated the generation process by adopting a sparse 3D hash grid structure, while ProlificDreamer effectively alleviated issues of over-saturation and low diversity through Variational Score Distillation. Despite the success of these methods in object-level generation, extending them to complete indoor scenes with layout constraints remained challenging. Methods such as Text2Room [15] and Text2NeRF [43] employed incremental generation frameworks that progressively generated different viewpoints and reconstructed 3D meshes frame by frame. However, these approaches often failed to effectively model the global layout of rooms, resulting in generated outputs that lacked spatial consistency.
2.2. Indoor Scene Texturing and Stylization Several works [3, 18, 19, 33] represented indoor scenes as panoramic images, avoiding explicit 3D geometry modeling. These methods constructed complete panoramic views by generating and stitching multiple image patches, with MVDiffusion proposing correspondence-aware attention blocks to maintain multi-view consistency. However, such approaches could produce incorrect room layouts due to the lack of explicit geometric constraints. For the task of re-texturing given 3D scenes, RoomPainter [13] proposed a view-integrated diffusion framework that effectively adapted 2D diffusion models to 3D-consistent texture synthesis through zero-shot techniques. This method addressed inconsistencies caused by independent view stylization and achieved high-fidelity indoor scene texture generation. While Text2Room could incrementally synthesize images and recover depth maps, it struggled with geomet-
ric and textural consistency across views, often resulting in contradictory structures.
2.3. Semantic-Aware Appearance Transfer Establishing accurate semantic correspondences was crucial for achieving high-quality appearance transfer. ReStyle3D [45] leveraged open-vocabulary segmentation techniques to establish dense instance-level correspondences between style and scene images, ensuring that each object was stylized with semantically matched textures. This explicit semantic correspondence mechanism could handle complex multi-object scenes and avoided semantic confusion issues present in traditional methods. Attention mechanisms in diffusion models [10, 34, 40] provided effective pathways for semantic control. Prompt-toPrompt achieved text-based local editing by manipulating cross-attention, while Plug-and-Play maintained layout consistency using spatial features and self-attention maps. ReStyle3D further proposed a cross-image semantic attention mechanism that simultaneously transferred the appearance of all semantic categories through explicit correspondence masks, enabling efficient scene-level stylization without requiring text prompts or 3D priors.
3. Methods We introduce Native3D, the first end-to-end native 3D indoor scene editing method. After reviewing foundational work in Sec. 3.1, we present our unified scene representation framework in Sec. 3.2. Sec. 3.3 details the 3D diffusion process for scene manipulation. Finally, Sec. 3.4 examines our alignment loss, which ensures geometric consistency and semantic coherence.
3.1. Preliminaries Diffusion Models. Diffusion models [10, 29, 34] generate data by reversing a forward process that gradually adds noise to clean samples. The forward process is typically defined as √ q(xt | x0 ) = N (xt ; ᾱt x0 , (1 − ᾱt )I), Qt where ᾱt = s=1 αs and αt = 1 − βt , with βt denoting a coefficient that controls the noise strength at step t. Training involves learning a denoising network ϵθ to predict the added noise, via the simplified objective: h i √ √ 2 Lsimple = Et,x0 ,ϵ ϵ − ϵθ ( ᾱt x0 + 1 − ᾱt ϵ, t) , where ϵ ∼ N (0, I). For conditional generation given text c, the reverse process uses ϵθ (·, t, c) to guide sampling. Flow Matching (FM) [21] provides an alternative framework that learns a probability density path between the prior distribution p0 (x) and the data distribution p1 (x). It is
Diffusion Process
Mesh and Texture Output
Mesh and Texture Input
"Apply cartoon-style to the entire room"
Mesh and Texture GT
Figure 2. Native3D Overview. We begin by encoding the input mesh through a joint mesh-texture encoder Ē, which is initialized with the HunYuan3dShapeVAE. A trainable adapter Adpt then transforms the encoded latent representation to match the target dimensionality. To facilitate the diffusion process, we apply a customized flow matching scheduler that progressively corrupts the adapter output Zinit into pure noise N . This noise N , together with text embeddings generated by the umT 5 encoder, serves as both input and conditioning signal for the standard diffusion process. The diffusion output is subsequently decoded by the decoder D̄ to reconstruct the final mesh and texture. During training, both Ē and D̄ are jointly optimized. We introduce an auxiliary supervision mechanism (indicated by the red line): both the model prediction and ground truth are encoded via a frozen pretrained Direct3D encoder Ē3d , and their latent representations are compared using 3D REP A Loss. This auxiliary loss mitigates detail loss inherent in joint mesh-texture modeling and contributes to the overall training objective.
trained to predict the velocity field vθ (xt , t) by the ordinary t differential equation: dx dt = vθ (xt , t), where xt is the state at time t ∈ [0, 1]. The training objective minimizes the difference between the learned field and a target velocity field ut (xt ): h i 2 Ldif f = Et,pt (x) ∥vθ (xt , t) − ut (xt )∥ . For conditional generation given a condition c, the vector field becomes vθ (xt , t, c). Sampling proceeds by solving the ODE from a sample x0 ∼ p0 : Z 1 x1 = x0 + vθ (xt , t, c)dt, 0
yielding a sample x1 distributed according to pdata (· | c). 3D Representation. A 3D shape is represented as a point cloud sampled from its surface, where each point combines its 3D coordinate pi ∈ R3 . The input to the encoder is thus a set P = {pi }N i=1 . The encoder E maps this point cloud to a probabilistic latent code z ∈ Rd via a variational formulation: z ∼ q(z | P) = N (µ, σ 2 I),
µ, σ = E(P),
where µ and σ denote the predicted mean and standard deviation. The decoder D reconstructs the shape by predicting a Signed Distance Function (SDF): fSDF (x) = D(z, x),
x ∈ R3 ,
where fSDF (x) gives the signed distance from query point x to the nearest surface. A mesh is then extracted from the zero-level set {x | fSDF (x) = 0} using the marching cubes algorithm [22]. This continuous representation enables differentiable and topology-flexible 3D reconstruction, forming the basis for latent-space generative modeling.
3.2. Unified Scene Representation Modeling In 3D scene editing tasks, traditional methods typically process individual objects independently, leading to results that lack global consistency and inter-object coordination. To address this challenge, we propose a unified Mesh-Texture joint representation approach that simultaneously models both the geometric structure and texture features of the entire 3D scene. Given a 3D scene S containing N objects, we represent it as the union of Mesh and Texture features from all objects:
S=
N [
(Mi , Ti )
(1)
i=1
where Mi ∈ RVi ×3 represents the mesh grid of the i-th object containing Vi vertex coordinates, and Ti ∈ RUi ×3 represents the corresponding texture features containing Ui texture elements. This joint representation advantageously maintains spatial relationships and visual consistency between different objects in the scene.
Hierarchical Feature Encoder. To extract hierarchical features from 3D scenes, we design a scene encoder Eϕ based on a Transformer architecture with local encoding and global aggregation modules: z = Eϕ (S) = Adpt (E(M1 , T1 ), . . . , E(MN , TN )) (2) where z ∈ Rdz denotes the global scene feature, and E represents a local encoder adapted from the pre-trained HunYuan3dShapeVAE [14]. This encoder processes both mesh structures and texture information for each object. The adapter network Adpt employs self-attention mechanisms to model inter-object relationships and aggregate features across the scene.
3.3. 3D Representation Aligned Diffusion Process Cross-Domain Representation Adaptation. Many existing works utilize pre-trained 2D diffusion models by adapting 3D representations to the 2D domain. However, this cross-domain mapping inevitably introduces information loss, primarily manifested in two aspects: geometric structural distortion due to the compression of 3D spatial relationships, and texture detail degradation caused by the projection of surface properties onto 2D representations. To address these limitations, we introduce 3D domain features into the diffusion process as a regularization alignment term. This approach preserves essential 3D structural and textural information that would otherwise be lost during domain adaptation, while still leveraging the powerful generative capabilities of pre-trained 2D diffusion models. 3D Feature Alignment Strategy. We establish featurelevel alignment between the diffusion process and 3D geometric representations. Specifically, we extract rich 3D geometric features and constrain their consistency with the diffusion model’s latent representations: min Lalign = D(z, f 3d) θ
(3)
where z denotes the diffusion latent representation, f3d represents the extracted 3D geometric features, and D(·, ·) is a feature distance metric function. This alignment strategy ensures that the diffusion process maintains structural fidelity to the 3D domain while benefiting from 2D pretrained models, effectively mitigating the information loss inherent in cross-domain adaptation.
3.4. 3D REPA Loss Multi-Object Feature Alignment Strategy. Considering the unique characteristics of 3D scene editing tasks, where scene spaces typically contain multiple objects while requiring spatial continuity, we explicitly model the independent features of each object during the feature extraction stage.
Assuming a scene contains K objects, we obtain objectlevel features through a feature extractor: (1)
(2)
(K)
F3d = [f3d , f3d , . . . , f3d ] ∈ RK×df
(4)
To facilitate alignment with the diffusion latent variables z ∈ RB×dz , we merge the object dimension with the batch dimension, resulting in reshaped feature tensors F3d ∈ R(B·K)×df and corresponding latent representations. Improved Contrastive Alignment Loss. Inspired by REPA [40] and DispersiveLoss [36] from the 2D domain, we employ a modified InfoNCE loss for feature alignment. This loss imposes contrastive constraints at each level of the feature space:
Lalign = −
B·K exp(s(zi , fi+ )/τ ) 1 X log PB·K (5) + B · K i=1 j=1 exp(s(zi , fj )/τ )
where s(·, ·) denotes the cosine similarity function, τ is the temperature coefficient, and fi+ represents the positive sample feature corresponding to zi . This loss function encourages the model to maintain both the discriminative nature of individual object features and semantic consistency at the scene level. Training Objective and Regularization. The contrastive alignment between diffusion latents and semantic features may introduce training instability due to distribution mismatches and gradient conflicts. To mitigate this issue, we incorporate latent space regularization to stabilize the training process: Latent Space Regularization imposes an ℓ2 -norm constraint on the diffusion latent representations to ensure training stability and improve convergence: Lreg = λ|z|22
(6)
This regularization term effectively controls the scale of latent representations and prevents excessive deviation during the alignment process, thereby maintaining numerical stability throughout training. Overall Optimization Objective The complete training objective integrates all components through a balanced combination: Ltotal = Ldif f + η1 Lalign + η2 Lreg where: • Ldif f : Standard diffusion denoising objective
(7)
• Lalign: Contrastive feature alignment loss • Lreg : Latent space regularization term • η1 , η2 : Balancing hyper-parameters This optimized objective ensures stable training while maintaining effective feature alignment, enabling robust 3D scene editing with consistent semantic preservation.
4. Experiments Dataset. Our collection consists of 9,001 room pairs, each accompanied by editing instructions categorized into four types: adding, removing, rearranging furniture, and style transfer. These instructions are generated using Qwen-VL2.5 [1], ensuring precise and varied modifications. The original data is sourced from the 3D-FRONT dataset [8]. For furniture manipulation, we utilize a custom-built Unity program, while style transfer data is synthesized utilizing InstructScene [17]. The dataset includes various room types: 5,749 bedrooms, 653 dining rooms, 694 libraries, 305 living rooms, and 1600 other rooms. For each room type, we use 95% of the rooms for training and 5% for testing. Evaluation Metrics. We employ two sets of metrics to comprehensively evaluate our 3D scene editing method. For assessing the overall quality of generated 3D scenes, we adopt commonly used quantitative metrics including CLIP Score (CS) [11] for measuring semantic alignment between images and text prompts, Aesthetic Score (AS) [31] for evaluating perceptual aesthetic quality, BRISQUE (BQ) [6] for assessing naturalness based on locally normalized luminance, Inception Score (IS) [32] for measuring quality and diversity via KL divergence, and Fréchet Inception Distance (FID) [12] for computing the distance between feature distributions of real and generated images. To evaluate finegrained editing capabilities within 3D scenes, we leverage ImgEdit-Bench [39], assessing four critical tasks: Add, Remove, Move, and Style. We render edited 3D scenes from four fixed viewpoints at 0◦ , 30◦ , 60◦ and 90◦ azimuthal angles, and these multi-view renderings are evaluated using Qwen-VL-Max and GPT-4.1, measuring Instruction Adherence, Editing Quality, and Detail Preservation on a 0-5 scale, with the final score averaged across all dimensions and viewpoints.
4.1. Implementation Details In our implementation, we adopt the HunYuanDiT [44] as diffusion generation backbone and employ Flow Matching as the sampler. For 3D shape representation, we extract geometric features using the Direct3D [38] encoder. All experiments are conducted on high performance GPUs, with a total computational cost of approximately 3100 GPU hours. Additional implementation details are provided in the supplementary material.
4.2. Quantitative Results Method
CS ↑
AS ↑
BQ ↓
IS ↑
FID ↓
Text2tex [4] SceneTex [3] RoomTex [35] RoomPainter [13] Ours
27.8 26.9 30.2 31.7 32.1
4.91 4.53 5.34 6.12 5.89
61.8 57.9 43.2 47.7 52.8
5.41 4.88 5.62 5.81 5.14
56.5 61.3 48.2 35.7 39.8
Table 1. Quantitative comparison of quality metrics (CS, AS, BQ, IS, FID). Our method achieves the best CS score and highest average ranking (1.6), demonstrating superior overall performance. While RoomPainter leads in AS, IS, FID and RoomTex excels in BQ, our approach shows exceptional consistency in semantic understanding.
Table 1 presents the quantitative comparison using five metrics: CS, AS, BQ, IS, and FID. Our method achieves the best performance in CS and demonstrates competitive results in IS, highlighting its superior capability in semantic understanding and 3D scene representation. This advantage primarily stems from our native 3D modeling approach, which enables more accurate geometric representation and enhanced semantic comprehension. However, our method shows relatively lower performance in FID and AS, which can be attributed to its limited training on opendomain and complex datasets. Furthermore, the absence of comprehensive illumination, viewpoint, and camera parameters in our framework leads to inferior performance in BQ compared to RoomTex, which specializes in texture synthesis under such conditions. These results indicate that while our method excels in structural and semantic accuracy, there remains room for improvement in photorealistic rendering and domain generalization. Method
Add ↑
Remove ↑
Move ↑
Style ↑
Text2tex [4] SceneTex [3] RoomTex [35] RoomPainter [13] Ours
1.45 2.87 2.66 2.59 4.01
2.18 2.99 2.78 2.71 4.14
1.11 1.91 1.82 2.24 2.61
3.52 3.66 4.14 4.05 3.63
Table 2. Quantitative comparison of editing capabilities across four fundamental operations: Add, Remove, Move, and Style. Our method achieves state-of-the-art performance on all structural editing tasks (Add, Remove, and Move), and competitively on Style editing.
To evaluate the specific capabilities for interactive scene editing, we compare the performance on four fundamental editing operations in Table 2: Add, Remove, Move, and Style. All metrics follow the principle that higher scores indicate better performance. The experimental results demonstrate that our proposed method achieves superior perfor-
Remove
Move
Style
Dining Room
Bed Room
Input
Figure 3. Qualitative editing results across two bedroom and two dining room scenes. Each row corresponds to one scene, with the first column showing the input scenes, the second demonstrating object removal, the third illustrating object movement, and the fourth showcasing style transfer.
mance over all baseline models in the three core structural editing tasks: Add, Remove, and Move. This strongly validates the exceptional capability of our approach in understanding and executing complex structural modifications while maintaining scene coherence. Notably, RoomTex obtains the highest score on the Style transfer task, which can be attributed to its architecture specifically optimized for texture synthesis and style manipulation. In conclusion, these results confirm the distinct advantage of our method in handling editing tasks that require precise object manipulation and structural integrity preservation.
4.3. Qualitative Results To qualitatively evaluate the editing capabilities of our method, we conducted extensive experiments across four distinct scenes - two bedroom and two dining room environments. The visual results, as shown in Fig. 3, demonstrate our approach’s effectiveness in handling three fundamental editing operations: removal, movement, and style transfer. For remove operations, our method cleanly eliminates target objects while plausibly reconstructing the underlying surfaces and maintaining scene coherence. In move operations, the system successfully relocates objects to new positions with appropriate spatial relationships and lighting consistency. Most notably, the style transfer results showcase
w/o feat
w/o 3D REPA Loss
Ours
Remove
Move
Input
Figure 4. Ablation study on 3D feature alignment and 3D REPA loss. Move task instruction is “Move a chair to the side of the cabinet”. Remove task instruction is “Remove the desk”
our method’s ability to comprehensively alter the visual appearance of scenes - transforming both textures and materials while preserving the fundamental layout and geometry. The consistent performance across different room types and editing tasks validates the robustness and generalizability of our proposed framework.
4.4. Ablation Study Method
CS ↑
AS ↑
Add ↑
Remove ↑
Feature Extractor Ablation 3DShape2VecSet [42] Direct3DEncoder [38]
29.8 31.2
5.34 5.67
3.45 3.82
3.62 3.95
Loss Function Ablation REPA [40] infoNCE [24] 3D REPA
30.5 31.0 31.5
5.42 5.58 5.76
3.61 3.78 3.89
3.72 3.88 3.98
Ours (Full)
32.1
5.89
4.01
4.14
The full model achieves the best results, validating our component design. To more intuitively demonstrate the value of our introduced 3D feature alignment and 3D REPA loss, as shown in Fig. 4 presents visual comparisons of the Move and Remove tasks. From the results, we can clearly observe that without incorporating 3D features to align the diffusion process, the model fails to effectively follow either type of instruction and exhibits artifacts such as penetration and visual corruption. This indicates that introducing 3D features enables more coherent modeling for generation and editing tasks. When the 3D REPA loss is absent, the edited objects either fail to comply with instructions or misinterpret the editing targets, demonstrating that additional loss supervision enhances semantic understanding and expression during the diffusion process.
5. Conclusion Table 3. Ablation studies on feature extractors and loss functions. We select four key metrics (CS, AS, Add, Remove) for ablation studies based on their alignment with 3D feature enhancement and semantic-spatial modeling objectives.
To establish a focused evaluation protocol for our ablation studies, we select CS, AS, Add, Remove as metrics. To validate the effectiveness of core components in our framework, we conduct systematic ablation studies focusing on feature extractors and loss functions. As shown in Table 3, Direct3DEncoder surpasses 3DShape2VecSet across all metrics, demonstrating stronger 3D understanding. Our 3D REPA loss also outperforms both REPA and infoNCE.
In this paper, we present an Native3D for indoor scene editing that fundamentally eliminates the domain gap inherent in traditional 2D projection-based approaches. By directly modeling mesh geometry and texture features through a unified representation and DiT, our method achieves comprehensive capture of both global scene layouts and fine-grained object details while enabling intuitive natural language-based control. The proposed 3D REPA Loss aligns multi-scale semantic features to preserve local geometric and textural fidelity during joint modeling. Experimental results demonstrate state-of-the-art performance in multi-view consistency and editing precision, outperform-
ing existing 2D or hybrid methods, with promising applications in interior design, virtual reality content creation, and professional 3D design workflows.
References [1] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-VL Technical Report, 2025. arXiv:2502.13923 [cs]. 6 [2] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging Properties in Self-Supervised Vision Transformers, 2021. arXiv:2104.14294 [cs]. 1 [3] Dave Zhenyu Chen, Haoxuan Li, Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießner. SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors, 2023. arXiv:2311.17261 [cs]. 3, 6 [4] Dave Zhenyu Chen, Yawar Siddiqui, Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießner. Text2tex: Text-driven texture synthesis via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 18558–18568, 2023. 1, 2, 3, 4, 6, 8, 12. 6 [5] Kevin Chen, Christopher B. Choy, Manolis Savva, Angel X. Chang, Thomas Funkhouser, and Silvio Savarese. Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings, 2018. eprint: arXiv:1803.08495. 3 [6] Xiaoqiao Chen, Qingyi Zhang, Manhui Lin, Guangyi Yang, and Chu He. No-reference color image quality assessment: From entropy to perceptual quality, 2018. 6 [7] Chuan Fang, Yuan Dong, Kunming Luo, Xiaotao Hu, Rakesh Shrestha, and Ping Tan. Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints, 2025. arXiv:2310.03602 [cs]. 1 [8] Huan Fu, Bowen Cai, Lin Gao, Lingxiao Zhang, Jiaming Wang Cao Li, Zengqi Xun, Chengyue Sun, Rongfei Jia, Binqiang Zhao, and Hao Zhang. 3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics, 2021. arXiv:2011.09127 [cs]. 6 [9] Ayaan Haque, Matthew Tancik, Alexei A. Efros, Aleksander Holynski, and Angjoo Kanazawa. InstructNeRF2NeRF: Editing 3D Scenes with Instructions, 2023. arXiv:2303.12789 [cs]. 1 [10] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-toPrompt Image Editing with Cross Attention Control, 2022. arXiv:2208.01626 [cs]. 3 [11] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A Reference-free Evaluation Metric for Image Captioning, 2022. arXiv:2104.08718 [cs]. 6 [12] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by
a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium, 2018. arXiv:1706.08500 [cs]. 6 [13] Zhipeng Huang, Wangbo Yu, Xinhua Cheng, ChengShu Zhao, Yunyang Ge, Mingyi Guo, Li Yuan, and Yonghong Tian. RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturing, 2025. arXiv:2412.16778 [cs]. 1, 3, 6 [14] Team Hunyuan3D, Shuhui Yang, Mingxin Yang, Yifei Feng, Xin Huang, Sheng Zhang, Zebin He, Di Luo, Haolin Liu, Yunfei Zhao, Qingxiang Lin, Zeqiang Lai, Xianghui Yang, Huiwen Shi, Zibo Zhao, Bowen Zhang, Hongyu Yan, Lifu Wang, Sicong Liu, Jihong Zhang, Meng Chen, Liang Dong, Yiwen Jia, Yulin Cai, Jiaao Yu, Yixuan Tang, Dongyuan Guo, Junlin Yu, Hao Zhang, Zheng Ye, Peng He, Runzhou Wu, Shida Wei, Chao Zhang, Yonghao Tan, Yifu Sun, Lin Niu, Shirui Huang, Bojian Zheng, Shu Liu, Shilin Chen, Xiang Yuan, Xiaofeng Yang, Kai Liu, Jianchen Zhu, Peng Chen, Tian Liu, Di Wang, Yuhong Liu, Linus, Jie Jiang, Jingwei Huang, and Chunchao Guo. Hunyuan3d 2.1: From images to high-fidelity 3d assets with production-ready pbr material, 2025. 5 [15] Lukas Höllein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models, 2023. arXiv:2303.11989 [cs]. 3 [16] Heewoo Jun and Alex Nichol. Shap-E: Generating Conditional 3D Implicit Functions, 2023. eprint: arXiv:2305.02463. 3 [17] Chenguo Lin and Yadong Mu. InstructScene: InstructionDriven 3D Indoor Scene Synthesis with Semantic Graph Prior, 2024. arXiv:2402.04717 [cs]. 6 [18] Chieh Hubert Lin, Chia-Che Chang, Yu-Sheng Chen, DaCheng Juan, Wei Wei, and Hwann-Tzong Chen. COCOGAN: Generation by Parts via Conditional Coordinating, 2020. arXiv:1904.00284 [cs]. 3 [19] Chieh Hubert Lin, Hsin-Ying Lee, Yen-Chi Cheng, Sergey Tulyakov, and Ming-Hsuan Yang. InfinityGAN: Towards Infinite-Pixel Image Synthesis, 2022. arXiv:2104.03963 [cs]. 3 [20] Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3D: High-Resolution Text-to-3D Content Creation, 2023. arXiv:2211.10440 [cs]. 3 [21] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022. 3 [22] William E Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. In Seminal graphics: pioneering efforts that shaped the field, pages 347–353. 1998. 4 [23] Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-E: A System for Generating 3D Point Clouds from Complex Prompts, 2022. arXiv:2212.08751 [cs]. 3 [24] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation Learning with Contrastive Predictive Coding, 2019. arXiv:1807.03748 [cs]. 8
[25] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning Robust Visual Features without Supervision, 2024. arXiv:2304.07193 [cs]. 1 [26] Hao Ouyang, Kathryn Heal, Stephen Lombardi, and Tiancheng Sun. Text2Immersion: Generative Immersive Scene with 3D Gaussians, 2023. arXiv:2312.09242 [cs]. 3 [27] Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. DreamFusion: Text-to-3D using 2D Diffusion, 2022. eprint: arXiv:2209.14988. 3 [28] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision, 2021. arXiv:2103.00020 [cs]. 1 [29] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 3 [30] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-Resolution Image Synthesis with Latent Diffusion Models, 2022. arXiv:2112.10752 [cs]. 1 [31] Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. LAION400M: Open Dataset of CLIP-Filtered 400 Million ImageText Pairs, 2021. arXiv:2111.02114 [cs]. 6 [32] Edward Smith and David Meger. Improved Adversarial Systems for 3D Object Generation and Reconstruction, 2017. arXiv:1707.09557 [cs]. 6 [33] Shitao Tang, Fuyang Zhang, Jiacheng Chen, Peng Wang, and Yasutaka Furukawa. MVDiffusion: Enabling Holistic Multiview Image Generation with Correspondence-Aware Diffusion, 2023. arXiv:2307.01097 [cs]. 1, 3 [34] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation, 2022. arXiv:2211.12572 [cs]. 3 [35] Qi Wang, Ruijie Lu, Xudong Xu, Jingbo Wang, Michael Yu Wang, Bo Dai, Gang Zeng, and Dan Xu. RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpainting, 2024. eprint: 2406.02461. 1, 6 [36] Runqian Wang and Kaiming He. Diffuse and Disperse: Image Generation with Representation Regularization, 2025. arXiv:2506.09027 [cs]. 5 [37] Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation, 2023. arXiv:2305.16213 [cs]. 3
[38] Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer, 2024. arXiv:2405.14832 [cs]. 2, 6, 8 [39] Yang Ye, Xianyi He, Zongjian Li, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, and Li Yuan. ImgEdit: A Unified Image Editing Dataset and Benchmark, 2025. arXiv:2505.20275 [cs]. 6 [40] Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think, 2025. arXiv:2410.06940 [cs]. 3, 5, 8 [41] Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis, 2024. arXiv:2409.02048 [cs]. 3 [42] Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models, 2023. arXiv:2301.11445 [cs]. 8 [43] Jingbo Zhang, Xiaoyu Li, Ziyu Wan, Can Wang, and Jing Liao. Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields, 2024. arXiv:2305.11588 [cs]. 3 [44] Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang, Sheng Zhang, Xianghui Yang, Huiwen Shi, Sicong Liu, Junta Wu, Yihang Lian, Fan Yang, Ruining Tang, Zebin He, Xinzhou Wang, Jian Liu, Xuhui Zuo, Zhuo Chen, Biwen Lei, Haohan Weng, Jing Xu, Yiling Zhu, Xinhai Liu, Lixin Xu, Changrong Hu, Shaoxiong Yang, Song Zhang, Yang Liu, Tianyu Huang, Lifu Wang, Jihong Zhang, Meng Chen, Liang Dong, Yiwen Jia, Yulin Cai, Jiaao Yu, Yixuan Tang, Hao Zhang, Zheng Ye, Peng He, Runzhou Wu, Chao Zhang, Yonghao Tan, Jie Xiao, Yangyu Tao, Jianchen Zhu, Jinbao Xue, Kai Liu, Chongqing Zhao, Xinming Wu, Zhichao Hu, Lei Qin, Jianbing Peng, Zhan Li, Minghui Chen, Xipeng Zhang, Lin Niu, Paige Wang, Yingkai Wang, Haozhao Kuang, Zhongyi Fan, Xu Zheng, Weihao Zhuang, YingPing He, Tian Liu, Yong Yang, Di Wang, Yuhong Liu, Jie Jiang, Jingwei Huang, and Chunchao Guo. Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation, 2025. arXiv:2501.12202 [cs]. 6 [45] Liyuan Zhu, Shengqu Cai, Shengyu Huang, Gordon Wetzstein, Naji Khosravan, and Iro Armeni. ReStyle3D: SceneLevel Appearance Transfer with Semantic Correspondences, 2025. arXiv:2502.10377 [cs]. 1, 3