Masked Generative Transformer Is What You Need for Image Editing Wei Chow1,2, * Linfeng Li1, * Xian Sun3 Lingdong Kong1,2 Zefeng Li1 Qi Xu1 Hang Song1 Tian Ye5 Xian Wang1 Jinbin Bai2 Shilin Xu1 Xiangtai Li1 Junting Pan1 Shaoteng Liu1 Ran Zhou1 Tianshu Yang1 Songhua Liu4,† 1
ByteDance
2
National University of Singapore
3
Duke University
4
5
Shanghai Jiao Tong University
HKUST(GZ)
arXiv:2605.10859v1 [cs.CV] 11 May 2026
Project Page, Code & Dataset: https://weichow23.github.io/EditMGT Score
7 6 5 Turning a lion into a deer.
The environment became cloudy.
FluxKontext
Ours (0.96B) (0.96B) Ours
Bagel
8B
7B
VAREdit
OminiGen2
4
14B
12B
6B
GoT
Inference Time (s)
3 1
11
21
31
(b) Speed vs. Score of Existing Models 11 Added a puppy looking at the astronaut.
Wear a pair of round-frame glasses.
#Types AnyEdit ImgEdit 9+
9
2.5M
1M
6B
OmniEdit
7
PromptFix 1.2M 1.2M 1.5M GPT-I-Edit
5 Steampunk style.
Chinese ink.
Impressionism.
Ghibli style.
Byte Morph
3 250
HQ-Edit
6M 0.5M
Ours
Resolution
InstructP2P
500
2M 0.2M
750
1000
1250
(c) Scales of Open-Sourced Datasets
(a) Editing Results from EditMGT
Figure 1. Overview of EditMGT and CrispEdit-2M. We introduce the first MGT-based editing model that performs editing in 2s with 960M parameters, 6× faster than existing models of comparable performance while surpassing 8B models. We also contribute CrispEdit2M, providing 2M high-resolution (≥1024) editing samples across 7 categories.
Abstract
ters, EditMGT achieves state-of-the-art image similarity on multiple benchmarks while delivering 6× faster editing, demonstrating that MGTs offer a compelling alternative to diffusion-based editing.
Diffusion models dominate image editing, yet their global denoising mechanism entangles edited regions with surrounding context, causing modifications to propagate into areas that should remain intact. We propose a fundamentally different approach by leveraging Masked Generative Transformers (MGTs), whose localized token-prediction paradigm naturally confines changes to intended regions. We present EditMGT, an MGT-based editing framework that is the first of its kind. Our approach employs multilayer attention consolidation to aggregate cross-attention maps into precise edit localization signals, and regionhold sampling to explicitly prevent token flipping in nontarget areas. To support training, we construct CrispEdit2M, a 2M-sample high-resolution (≥1024) editing dataset spanning seven categories. With only 960M parame-
1. Introduction Instruct a diffusion model to “put a birthday hat on the dog”, and you may find the dog’s fur subtly recolored, the background shifted, or the lighting altered in ways you never requested. This edit leakage stems from the global denoising dynamics of diffusion models (DMs) [8]: every step refines the entire image simultaneously, making it inherently difficult to confine modifications to a specific region. Prior solutions based on large-scale training [21], predefined masks [22], or inversion techniques [12] mitigate but do not fully resolve this entanglement. 1
EditMGT
Micro Condition CLIP
CLS Token
R C C
Proj-Out
LLM Enc
Convert the unicorn into a lion
Linear
Attention
Single-Modal Transformer Block
K
V
R C
RoPE Linear
M
Linear
M M
Q
Timestep N
VQ Enc
Multi-Modal Transformer Block
M M
+ Mask
MLP-Out
Set N=0
xM
xN
M
Prompt Tokens
Original Tokens
Iterating Tokens
Figure 2. EditMGT framework. The original image conditions generation via attention injection. Right panel: token interactions inside the multi-modal and single-modal transformer blocks.
tion and confidence-based resampling [3]. Let CI ∈ RN ×d and CT ∈ RM ×d denote image and text tokens. Building upon Meissonic [1], each transformer block applies RoPE [15] for spatial encoding, then computes multi-modal √ attention over C = [CI ; CT ]: W = softmax(QK ⊤ / d). This token-level control makes MGTs naturally suited for editing: tokens in non-target regions can remain fixed throughout decoding. Image-Conditioned Editing via Attention Injection. To condition the generation on the original image, we introduce image condition tokens CV ∈ RN ×d with the same shape as CI (Fig. 2). Critically, CV shares all parameters with CI and uses identical RoPE positions, i.e., (i, j)CV = (i, j)CI , ensuring precise spatial alignment between the source and edited images. The key distinction is that the timestep assigned to CV is held constant at zero, preventing temporal drift and maintaining a stable reference signal. The model θ is trained by minimizing the cross-entropy loss over masked token reconstruction, jointly conditioned on visible tokens and the image reference:
We observe that a different class of generative models sidesteps this problem entirely. Masked Generative Transformers (MGTs) [3] synthesize images by iteratively predicting masked tokens in parallel rather than performing holistic refinement. This localized decoding paradigm naturally supports selective token replacement: tokens corresponding to non-target regions can simply be held fixed, providing an architectural guarantee against edit leakage that DMs lack. The question here is whether this token-level control can be harnessed for high-quality, instruction-driven image editing. We answer affirmatively with EditMGT [5], an MGTbased editing framework that is, to our knowledge, the first of its kind. We aim to address two essential capabilities for effective editing. First, for adaptive localization, we find that MGT cross-attention maps encode informative semantic correspondences between text instructions and visual regions, but lack sufficient prominence for direct use. We devise multi-layer attention consolidation that aggregates and refines these maps across network layers, producing sharp, accurate localization of edit-relevant areas (Fig. 3). Second, for region preservation, we propose region-hold sampling, a strategy that restricts token updates within low-attention areas during iterative decoding, explicitly preventing modifications from propagating into non-target regions. To train EditMGT, we assemble CrispEdit-2M, a high-resolution (≥1024) editing dataset with 2M rigorously filtered samples across seven distinct categories. We adapt a pretrained text-to-image MGT (Meissonic [1]) into an editing model via attention injection, requiring no additional parameters.
L = \mathbb {E}_{(x, t) \sim \mathcal {D},\, \mathbf {m} \sim \mathcal {M}} \bigg [- \sum \nolimits _{i \in \mathbf {m}} \log p_\theta (v_i \mid v_{\neg i}, C_T; C_V)\bigg ], (1) where m is a binary mask selecting tokens to predict, v¬i denotes visible tokens, and the masking rate follows a cosine schedule sampled from a truncated arccos distribution. To control CV ’s influence at inference, we introduce a bias matrix E into the attention weights as Wnew = W+E, where E scales attention between CI and CV by log(γ) while leaving intra-modality patterns unchanged. Setting γ = 0 removes conditioning; γ > 1 strengthens it. This mechanism transforms the pretrained text-to-image MGT into an editor via attention injection alone, without introducing any additional parameters. Multi-Layer Attention Consolidation. We find that the cross-attention weights between instruction tokens CT and
2. Methodology Preliminary: Masked Generative Transformers. MGTs synthesize images by starting from a fully masked token grid and progressively revealing tokens via parallel predic2
Input
Text Tokens Textto-Text
black cat
Text-to-Image Attention
Raw
Stacked
Output
Filtered
Input orange cat
Image-to-Text Attention
Add a birthday hat on the dog tail
Image-to-Image Attention
Remove the cup from the table orange cat
ground
Replace the airplane with a hot air balloons
Figure 3. Attention mechanism in EditMGT. Cross-attention maps encode semantic correspondences between instructions and visual regions. Multi-layer consolidation sharpens these maps for region-hold sampling. Table 1. Comparative results for instructive image editing on the test sets of EMU Edit [14] and MagicBrush [22]. We list the task-specific models in the first block and some concurrent universal models in the second block. EMU Edit Test Set CLIPout ↑ L1 ↓
Venue
Size
CLIPim ↑
• InstructPix2Pix [2] • MagicBrush [22] • PnP [17] • Null-Text Inv. [12] • UltraEdit [23] • EMU Edit [14] • AnyEdit [21]
CVPR’23 NeurIPS’23 CVPR’23 CVPR’23 NeurIPS’24 CVPR’24 CVPR’25
1B 1B 1B 1B 2B 1B
0.834 0.838 0.521 0.761 0.793 0.859 0.872
0.219 0.222 0.089 0.236 0.283 0.231 0.285
• OmniGen [20] • PixWizard [9] • UniReal [4] • GoT [7] • OminiGen2 [18] • EditAR [13] • NEP [19] • VAREdit [11]
arXiv’24 ICLR’25 CVPR’25 NeurIPS’25 arXiv’25 ICLR’25 arXiv’25 arXiv’25
4B 2B 5B 6B 7B 3B 3B 8B
0.836 0.845 0.851 0.864 0.876 0.871 0.876
Ours
1B
0.878
Model
• E DIT MGT
MagicBrush Test Set CLIPout ↑ L1 ↓
DINO↑
CLIPim ↑
0.121 0.100 0.304 0.075 0.071 0.094 0.070
0.762 0.776 0.153 0.678 0.844 0.819 0.821
0.837 0.883 0.568 0.752 0.868 0.897 0.898
0.245 0.261 0.101 0.263 0.261 0.275
0.093 0.058 0.289 0.077 0.088 0.052 0.051
0.767 0.871 0.220 0.664 0.792 0.879 0.881
0.233 0.248 0.285 0.276 0.309 0.307 0.280
0.069 0.099 0.078 0.094
0.804 0.798 0.790 0.822 0.844 0.825
0.884 0.903 0.867 0.901
0.265 0.308 0.287
0.063 0.081 0.103 0.083
0.876 0.837 0.804 0.844
0.308
0.093
0.832
0.911
0.301
0.058
0.881
DINO↑
image tokens CI naturally encode the spatial location of intended edits (Fig. 3). Notably, even within the initial iterations, the model can delineate the contours of objects to be added or modified, establishing precise text-to-image correspondence. However, individual layer activations lack sufficient prominence and exhibit unclear boundaries with internal discontinuities, making them unreliable for direct use as localization masks.
consolidated map provides a reliable per-token localization signal that distinguishes edit-relevant regions from those that should remain intact. Region-Hold Sampling. Given the consolidated attention map, we compute a localization score for each image token:
To remedy this, we aggregate attention from the most coherent single-modality processing layers (blocks 28–36), which we empirically identify as producing the most semantically aligned activations. The aggregated map is further refined via adaptive filtering [6] to suppress spatial discontinuities and sharpen object boundaries. The resulting
where Wiℓ ∈ RM ×N is the normalized instruction-to-image attention at layer ℓ, L is the set of selected layers, and M indexes the relevant instruction tokens (which can be restricted to specific keywords for finer control). During iterative decoding, the MGT reveals high-confidence tokens while re-masking uncertain ones. With region-hold sam-
s_L = \frac {1}{|\mathcal {L}||\mathcal {M}|}\sum \nolimits _{\ell \in \mathcal {L},\, m \in \mathcal {M}} \mathcal {W}^{\ell }_i[m,:] \;\in \mathbb {R}^N,
3
(2)
References
pling, tokens satisfying sL < λ are additionally restored to their original values from CV after each step, where λ is a user-controllable threshold that trades off edit extent against preservation. Training Pipeline. We assemble CrispEdit-2M comprising 2M high-resolution (≥1024) editing samples across seven categories with rigorous quality filtering. Training proceeds in three stages: (1) base model adaptation with Gemma22B [16] as text encoder (5K steps), (2) full fine-tuning on 4M editing samples (50K steps), and (3) high-quality refinement for human preference alignment (1K steps). Additional details are available in the main paper [5].
[1] Jinbin Bai et al. Meissonic: Revitalizing masked generative transformers for efficient high-resolution text-to-image synthesis. arXiv preprint arXiv:2410.08261, 2024. [2] Tim Brooks et al. InstructPix2Pix: Learning to follow image editing instructions. In CVPR, pages 18392–18402, 2023. [3] Huiwen Chang et al. MaskGiT: Masked generative image transformer. In CVPR, pages 11315–11325, 2022. [4] Xi Chen et al. UniReal: Universal image generation and editing via learning real-world dynamics. arXiv preprint arXiv:2412.07774, 2024. [5] Wei Chow et al. EditMGT: Unleashing potentials of masked generative transformers in image editing. In CVPR, 2026. [6] Paulo SR Diniz et al. Adaptive filtering. Springer, 1997. [7] Rongyao Fang et al. GoT: Unleashing reasoning capability of multimodal large language model for visual generation and editing. arXiv preprint arXiv:2503.10639, 2025. [8] Amir Hertz et al. Prompt-to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. [9] Weifeng Lin et al. PixWizard: Versatile image-to-image visual assistant with open-language instructions. arXiv preprint arXiv:2409.15278, 2024. [10] Shiyu Liu et al. Step1x-Edit: A practical framework for general image editing. arXiv preprint arXiv:2504.17761, 2025. [11] Qingyang Mao et al. Visual autoregressive modeling for instruction-guided image editing. arXiv preprint arXiv:2508.15772, 2025. [12] Ron Mokady et al. Null-text inversion for editing real images using guided diffusion models. In CVPR, 2023. [13] Jiteng Mu et al. EditAR: Unified conditional generation with autoregressive models. In CVPR, pages 7899–7909, 2025. [14] Shelly Sheynin et al. Emu edit: Precise image editing via recognition and generation tasks. In CVPR, pages 8871– 8879, 2024. [15] Jianlin Su et al. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024. [16] Gemma Team et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024. [17] Narek Tumanyan et al. Plug-and-play diffusion features for text-driven image-to-image translation. In CVPR, pages 1921–1930, 2023. [18] Chenyuan Wu et al. OmniGen2: Exploration to advanced multimodal generation. arXiv preprint arXiv:2506.18871, 2025. [19] Huimin Wu et al. NEP: Autoregressive image editing via next editing token prediction. arXiv preprint arXiv:2508.06044, 2025. [20] Shitao Xiao et al. OmniGen: Unified image generation. In CVPR, pages 13294–13304, 2025. [21] Qifan Yu et al. AnyEdit: Mastering unified high-quality image editing for any idea. In CVPR, 2025. [22] Kai Zhang et al. MagicBrush: A manually annotated dataset for instruction-guided image editing. NeurIPS, 36, 2024. [23] Haozhe Zhao et al. UltraEdit: Instruction-based fine-grained image editing at scale. arXiv preprint arXiv:2407.05282, 2024.
3. Experiments We evaluate EditMGT on standard benchmarks: Emu Edit [14], MagicBrush [22], AnyBench [21], and GEditEN-full [10]. Details are available in the main paper [5]. Quantitative Results. As reported in Table 1, EditMGT achieves state-of-the-art CLIPim scores on both Emu Edit (0.878) and MagicBrush (0.911), with a 1.1% improvement over the next best method on MagicBrush. DINO scores are state-of-the-art on MagicBrush and second-best on Emu Edit, while instruction adherence (CLIPout ) remains consistently competitive. On the GPT-based GEditEN-full benchmark [10], EditMGT achieves competitive performance with FluxKontext.dev (12B) and surpasses VAREdit-8B, GoT-6B, and OminiGen2-7B, with notable margins of +9.8% on color alteration and +17.6% on style transfer. These results confirm that the MGT’s localized decoding paradigm effectively prevents modifications from propagating into non-target regions. Qualitative Results & Efficiency. Fig. 4 compares EditMGT against UltraEdit (SD3), GoT (6B), OminiGen2 (7B), and VAREdit (8B). Key advantages include superior instruction comprehension (correctly reducing warm tones rather than increasing yellow) and faithful structural preservation (maintaining subject pose during style transfer). At 1024×1024 resolution, EditMGT completes an edit in ∼2 seconds with 13.8 GB memory, 6× faster than models of comparable quality.
4. Conclusion We presented EditMGT, the first MGT-based framework that resolves the edit leakage problem inherent in diffusion models by leveraging localized token prediction. Multilayer attention consolidation provides precise, mask-free edit localization, while region-hold sampling preserves nontarget content during decoding. With 960M parameters and 2-second inference, EditMGT achieves state-of-the-art similarity across multiple benchmarks and competitive quality with models up to 12× larger, establishing MGTs as an efficient paradigm for interactive image editing. 4
Input
UltraEdit-2B
GoT-6B
OminiGen2-7B VAREdit-8B
Input
EditMGT
UltraEdit-2B
My photo looks a bit yellowish; please adjust the color.
GoT-6B
OminiGen2-7B VAREdit-8B
EditMGT
Enhance the clarity of this photo.
Replace the pencil in the hand of the boy in orange shirt with an egg.
Change '2022' to '2024'
Replace the cat with a fish.
Remove the person from the top of the boat cabin.
Add long black stockings.
Light the all the candles to enhance the candlelight.
Can you add penguin eyes to this image?
Add a book in her hand
Generate a Pixar-style animation with a cheerful spring background.
Add a robot bird in the sky.
Switch the image to a high-resolution Impressionist painting.
Make it an oil painting.
Adjust the background to the ocean.
Make the clothing fabric from premium linen.
Figure 4. Qualitative comparisons. EditMGT (960M) outperforms larger models across diverse editing tasks such as object transformation, scene replacement, and material substitution.
5