arXiv:2604.18320v1 [cs.CV] 20 Apr 2026
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations Yongrui Heng∗
Chaoya Jiang∗†
Han Yang
National Engineering Research Center for Software Engineering, Peking University Beijing, China [email protected]
School of Control Science and Engineering, Shandong University Jinan, China [email protected]
Zeekr, Geely Auto Shanghai, China [email protected]
Shikun Zhang
Wei Ye†
National Engineering Research Center for Software Engineering, Peking University Beijing, China [email protected]
National Engineering Research Center for Software Engineering, Peking University Beijing, China [email protected]
Abstract Self-evolution of multimodal large language models (MLLMs) remains a critical challenge: pseudo-label-based methods suffer from progressive quality degradation as model predictions drift, while template-based methods are confined to a static set of transformations that cannot adapt in difficulty or diversity. We contend that robust, continuous self-improvement requires not only deterministic external feedback independent of the model’s internal certainty, but also a mechanism to perpetually diversify the training distribution. To this end, we introduce EVE (Executable Visual transformation based self-Evolution), a novel framework that entirely bypasses pseudo-labels by harnessing executable visual transformations continuously enriched in both variety and complexity. EVE adopts a Challenger-Solver dual-policy architecture. The Challenger maintains and progressively expands a queue of visual transformation code examples, from which it synthesizes novel Python scripts to perform dynamic visual transformations. Executing these scripts yields VQA problems with absolute, execution-verified groundtruth answers, eliminating any reliance on model-generated supervision. A multi-dimensional reward system—integrating semantic diversity and dynamic difficulty calibration—steers the Challenger to enrich its code example queue while posing progressively more challenging tasks, preventing mode collapse and fostering reciprocal co-evolution between the two policies. Extensive experiments demonstrate that EVE consistently surpasses existing ∗ These authors contributed equally to this work. † Corresponding authors.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. Conference acronym ’XX, Woodstock, NY © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-XXXX-X/2018/06 https://doi.org/XXXXXXX.XXXXXXX
self-evolution methods, establishing a robust and scalable paradigm for verifiable MLLM self-evolution. The code is available at https://github.com/0001Henry/EVE.
CCS Concepts • Computing methodologies → Artificial intelligence.
Keywords Multimodal Large Language Models, Self-Evolution, Reinforcement Learning ACM Reference Format: Yongrui Heng, Chaoya Jiang, Han Yang, Shikun Zhang, and Wei Ye. 2026. EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations. In Proceedings of Make sure to enter the correct conference title from your rights confirmation email (Conference acronym ’XX). ACM, New York, NY, USA, 17 pages. https://doi.org/XXXXXXX.XXXXXXX
1
Introduction
The ability of a model to improve itself without human supervision has become a central goal in machine learning. For multimodal large language models (MLLMs), self-evolution refers to iteratively generating training data from the model’s own capabilities and using that data to further improve the model—ideally without any human-labeled supervision or access to stronger external models. Recent efforts have made meaningful progress in this direction, but two fundamental problems persist. Pseudo-label degradation. Methods such as VisPlay [11], MMZero [20], and EvoLMM [36] construct training data by having the model make predictions and treating high-confidence outputs as pseudo-labels. This creates a positive feedback loop that works early in training but degrades rapidly later: as the model’s predictions shift, pseudo-label quality degrades and error accumulates, as shown in Figure 1(b). Confidence scores do not measure groundtruth correctness. Recent theoretical analyses [10, 24] confirm that unsupervised reinforcement learning based on model confidence or entropy cannot reliably scale MLLM training. Furthermore, while difficulty filtering at intermediate success rates (around 50%) has
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Yongrui Heng et al.
Table 1: Comparison of human-annotation-free MLLM training methods. EVE is an external environment-based approach that simultaneously achieves accurate supervision, expandable question types, and no reliance on external models. Accurate Expandable No External Supervision Questions Model
Method Pseudo-label-based [11, 20, 36]
no
yes
yes/no
Template-based [15, 39, 44]
yes
no
yes
EVE (Ours)
yes
yes
yes
100
68 67 EVE (Ours) VisPlay Qwen3-VL-8B-Instruct
66 65
Iter 0
Iter 1
Iter 2
Iter 3
Iter 4
(a) Performance on 8 benchmarks
Answer Accuracy (%)
Overall Accuracy (%)
69
space of possible transformations is as rich as the space of programs, and the resulting questions are as diverse as the images and parameters fed to those programs. Crucially, code execution provides an external, deterministic oracle: every question has a unique, verifiable answer independent of model confidence, resolving the pseudo-label degradation problem at its root. Moreover, because the model itself generates the transformation programs, the question space is expandable and adaptive—unconstrained by the several fixed templates—thereby overcoming the template diversity limitation. Based on this insight, we propose EVE (Executable Visual transformation based self-Evolution), a MLLM self-evolution framework built on a Challenger-Solver dual-policy architecture. Two policies are initialized from the same pretrained MLLM:
90
• The Challenger receives an image and generates a Python program that defines an image editing function together with multiple candidate parameter sets. Executing the program produces the corresponding edited images. • The Solver receives the original image, the code, the edited images, and a multiple-choice question, and tries to identify the correct parameter-to-image or image-to-parameter correspondence.
80 70 60 50
EVE (Ours) VisPlay Iter 1
Iter 2
Iter 3
(b) Reward Signal Accuracy
Figure 1: Quantitative comparison with VisPlay. (a) Overall accuracy on 8 benchmarks across iterations. (b) Reward signal accuracy (answer labels vs. ground truth) across iterations. been theoretically justified for maximizing learning efficiency [1], this result critically assumes accurate training labels. Pseudo-labelbased methods violate this assumption, undermining their theoretical foundation and explaining their performance degradation over iterations. Template diversity and difficulty limitations. An alternative to pseudo-labels is the proxy task paradigm, which constructs training questions from unlabeled images using predefined transformation templates. Agentic Jigsaw [44] and JigsawR1 [39] use the jigsaw puzzle task as a proxy, where the model needs to reorder image patches to reconstruct the original image. PuzzleCraft [15] extends this with rotation and patch-fitting. These approaches offer verifiable answers without human labels, but they are constrained to fixed, human-designed transformation templates. The question types, visual complexity, and difficulty levels are determined by the template designer, not by the model or the data, limiting the model’s generalization capability to explore complex visual reasoning. Executable visual transformations as an environment. We argue that achieving expandable self-evolution requires a mechanism that (1) automatically generates diverse, image-adaptive questions, (2) provides accurate and verifiable correctness signals without human labels, and (3) adaptively controls task difficulty. Satisfying all three conditions requires the model to learn from an external environment—one where deterministic feedback is grounded in the environment’s dynamics rather than the model’s own predictions. We identify executable visual transformations—Python programs that edit images—as a natural realization of such an environment. A Python function that transforms an image defines an infinite variety of visual puzzles whose ground-truth answers are determined entirely by program execution, not by the model. The
From each Challenger program, we automatically construct VQA questions with ground-truth labels determined entirely by code execution, independent of model confidence. The Challenger is guided by a multi-dimensional reward system that balances code validity, difficulty calibration (targeting ∼50% Solver accuracy for optimal learning), and semantic diversity. To sustain the growth of question diversity, EVE maintains a visual transformation example queue that is progressively expanded with high-quality, semantically diverse code examples across iterations. At each iteration, the Challenger samples and composes codes from this queue to synthesize novel programs, ensuring that the question space continuously broadens rather than collapses to a narrow subset. The two policies co-evolve through alternating training, forming a self-sustaining loop of mutual improvement. Our main contributions are: • We propose EVE, a self-evolution framework grounded in executable visual transformations as a programmatic external environment, where code execution provides verifiable ground-truth answers independent of model confidence, directly eliminating pseudo-label degradation. • We design a Challenger-Solver dual-policy architecture with multi-dimensional rewards that enable adaptive question generation, overcoming the diversity and difficulty rigidity of template-based methods. • Comprehensive experiments on 8 benchmarks demonstrate consistent improvements across both 4B and 8B model scales, validating that EVE continuously enhances visual perception and reasoning abilities.
2
Related Work
Self-Evolution of Large Language Models. Self-evolution aims to improve model capabilities without human supervision or external models [5]. In language models, this paradigm has been explored
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
through reinforcement learning: Absolute Zero [47] uses code execution as a verifiable reward signal for unsupervised RL on coding and math tasks, while R-Zero [13] adopts a Challenger-Solver framework where majority vote serves as pseudo-labels. These approaches demonstrate the potential of self-evolution, but remain limited to text-only reasoning. Self-Evolution of Multimodal Large Language Models. For multimodal models, existing methods can be categorized into two paradigms, as shown in Table 1: pseudo-label-based and templatebased approaches. Pseudo-label-based methods. These methods generate training data by having the model make predictions and treating certain outputs as supervision signals. This category can be further divided into two subcategories based on the source of supervision: (1) Internal signal guided methods [11, 20, 23, 34, 36, 38] rely on the model’s own confidence or entropy as supervision. VisPlay [11] follows RZero’s framework, constructing VQA pairs from unlabeled images and filtering by confidence scores. EvoLMM [36] introduces continuous reward signals based on model certainty, while MM-Zero [20] extends this to settings without real images. However, confidence scores measure internal certainty rather than ground-truth correctness, causing error accumulation across iterations [10, 11]. (2) External model guided methods [4, 16, 45] leverage external models as specialized verifiers for supervision. ViPER [45] uses diffusion models to externalize reasoning into visual snapshots, enabling the model to critique and refine its understanding. RRVF [4] employs CLIP [29] similarity as a reward signal, but remains limited to image-to-code tasks. While these methods can provide more reliable signals than internal confidence, they require access to proprietary models or domain-specific verifiers, limiting generalizability and violating the self-evolution principle. Template-based methods. These methods [15, 25, 39, 44] construct proxy tasks with deterministic answers using predefined transformation templates. Agentic Jigsaw [44] and JigsawR1 [39] shuffle image patches and ask models to predict the correct arrangement. Puzzle Curriculum GRPO [15] extends this with rotation and patch-fitting tasks, introducing curriculum learning to gradually increase difficulty. Spatial-SSRL [25] is a similar paradigm that derives verifiable signals directly from ordinary RGB or RGB-D images. While these methods provide accurate supervision without pseudolabel degradation, they are constrained by fixed human-designed transformation templates. The question types, visual complexity, and difficulty levels are predetermined by template designers, preventing adaptive difficulty control and limiting the diversity of training data. Code Generation for Vision-Language Tasks. Early work [9, 22, 35] has explored using code as an intermediate representation for vision-language reasoning. Some recent "think with images" methods [12, 33, 46, 48, 49] use code as a tool to process the input image during the reasoning process, enabling more accurate and interpretable reasoning. However, these methods focus on inference-time program synthesis rather than using code execution as a training signal. In contrast, EVE leverages code execution as a self-evolution mechanism, providing deterministic supervision grounded in a programmatic external environment.
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
3 Method 3.1 Overview EVE addresses two core challenges in MLLM self-evolution: pseudolabel degradation and the diversity and difficulty rigidity of templatebased proxy tasks, through a unified framework (Figure 2). The framework consists of two policies initialized from the same pretrained MLLM: a Challenger policy 𝜋𝐶 that generates executable Python code defining visual transformations, and a Solver policy 𝜋𝑆 that answers visual questions constructed from code execution results. Given an image library D, the Challenger samples an image 𝐼 ∈ D and generates code 𝑃 that produces multiple edited images. From these transformations, we automatically construct visual questions 𝑞 with verifiable ground-truth answers 𝑦 ∗ determined by code execution. The Solver then attempts to answer these questions. The two policies co-evolve through alternating optimization over 𝑇 iterations: the Challenger learns to generate diverse, moderately difficult questions that challenge the Solver, while the Solver learns to answer these questions correctly. This adversarial-cooperative dynamic drives continuous improvement without external supervision.
3.2
Challenger Policy
Task Definition. Given an input image 𝐼 , the Challenger generates executable Python code 𝑃 containing: (1) an edit_image(img, **args) function that applies one or more transformations to the image, and (2) a list args_list of 𝑁 distinct parameter sets A = {𝑎 1, . . . , 𝑎 𝑁 }. Executing the code produces 𝑁 edited images {𝐼 1, . . . , 𝐼 𝑁 } where 𝐼 𝑗 = edit_image(𝐼, 𝑎 𝑗 ) for 𝑗 ∈ {1, . . . , 𝑁 }. Code Generation with Few-Shot Examples. To guide the Challenger, we maintain a priority queue Q of high-quality code examples. The priority queue is initialized with four seed examples covering basic transformations (jigsaw puzzles, rotation, cropping, bounding box drawing) and evolves during training to accumulate diverse, high-reward programs. For each generation, we randomly sample 𝑁 e examples from Q and construct a prompt that includes: (1) task instructions and requirements, (2) the example codes, and (3) the input image 𝐼 . The Challenger is prompted to output code in the Python code block. Execution-verified task synthesis. From the generated code 𝑃 and execution results {𝐼 𝑗 }𝑁𝑗=1 , we automatically construct two types of multiple-choice questions: Type-0 (Parameter-to-Image). Given the original image 𝐼 , the code 𝑃, one parameter set 𝑎 𝑗 ∈ A, and 𝑁 candidate images {𝐼 1, . . . , 𝐼 𝑁 }, identify which image corresponds to 𝑎 𝑗 . This tests whether the Solver can execute the code mentally and predict the visual outcome. Type-1 (Image-to-Parameter). Given the original image 𝐼 , one transformed image 𝐼 𝑗 , the code 𝑃, and 𝑁 parameter options A = {𝑎 1, . . . , 𝑎 𝑁 }, identify which parameters produced 𝐼 𝑗 . This tests whether the Solver can perform inverse reasoning from visual observations to parameter values. Both question types have unique, verifiable answers determined entirely by code execution, eliminating reliance on model confidence or human judgment. We apply random shuffling to answer
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Yongrui Heng et al.
update # Task Write a simple Python function named `edit_image` that edit the user's image, and then design 4 different parameter sets. ... # Examples
Code Example Queue
rollout
from PIL import Image def edit_image(img: Image.Image, angle: float) -> Image.Image: return img.rotate(angle,expand=True, resample=Image.Resampling.BICUBIC) args_list = [ {'angle': 15}, {'angle': 45}, {'angle': 90}, {'angle': 180}, ]
optimize
Question:
...def edit_image... After applying the `edit_image` function to image_0 with `{'crop_coords': (0, 0, 1000, 800)}`, which candidate image will be produced? Options: A. image_1 B. image_2 C. image_3 D. image_4
image_1 (rotated)
image_2 (cropped)
Solver Reward Format Accuracy
rollout
Ground Truth: B Question:
Challenger
... # Input Image
Image Library
from PIL import Image, ... def edit_image(img: Image.Image, ... ... return img args_list= [ {'angle': -5}, {'crop_coords': (0, 0, 1000, 800)}, {'contrast_factor': 1.3}, {'blur_radius': 1.5} ]
optimize image_0
Code Sandbox
image_3 (enhanced)
Challenger Reward Format Valid Difficulty Diversity
Automatic VQA Construction
...def edit_image... Which set of arguments, when passed to `edit_image` for image_0, produces image_1? Options: A. {'angle': -5} B. {'crop_coords': (0, 0, 1000, 800)} C. {'contrast_factor': 1.3} D. {'blur_radius': 1.5}
rollout
Solver
Ground Truth: A
…
image_4 (blurred)
Visual Q&A Bank
…
Edited Images
Challenger Prompt
Pipline for Challenger Training & VQA Generation
Pipline for Solver Training
Figure 2: Overview of EVE. The model learns from executable visual transformations as a programmatic external environment: the Challenger generates Python scripts defining various visual transformations; script execution produces edited images and determines execution-verified ground-truth answers; VQA questions are automatically constructed to train the Solver; the Solver’s accuracy feeds back as a difficulty signal to the Challenger. All supervision is derived from deterministic code execution, eliminating reliance on pseudo-labels or fixed templates. options while maintaining the one-to-one correspondence between parameters and their resulting images. This ensures the model must attend to visual content rather than exploiting positional biases. Challenger Reward Function. The Challenger’s total reward combines four components: 𝑟𝐶 = 𝜆format𝑟 format + 𝜆valid𝑟 valid + 𝜆diff𝑟 diff + 𝜆div𝑟 div
(1)
Format Reward 𝑟 format ∈ {0, 1} encourages parseable code blocks and penalizes irregular formatting or comments. To avoid solvers taking shortcuts from comments or being misleading, we require Challenger to generate uncommented code. Validity Reward 𝑟 valid ∈ {0, 1} requires successful sandbox execution with constraints: no random functions, no duplicate parameters in args_list, and image quality filtering (size bounds, no identical outputs). Specifically, we compute the similarity between the edited images through perceptual hashing, and if the similarity exceeds a threshold of 0.95, the output is considered to be a duplicate. Difficulty Reward 𝑟 diff ∈ [0, 1] calibrates question difficulty. For each generated question 𝑞, we sample the Solver 𝐾 times to compute average accuracy: 𝐾
acc𝑠 (𝑞) =
1 ∑︁ ⊮[𝑦ˆ𝑘 = 𝑦 ∗ ] 𝐾
(2)
𝑘=1
where 𝑦ˆ𝑘 ∼ 𝜋𝑆 (𝑞) is the 𝑘-th sampled answer and 𝑦 ∗ is the ground truth. The difficulty reward is: 𝑟 diff (𝑞) = 1 − 2|acc𝑠 (𝑞) − 0.5|
(3)
This function peaks at acc𝑠 = 0.5 (moderately difficult) and decreases toward 0 or 1 (too easy or too hard), encouraging questions that maximize learning gain [1]. The difficulty reward for code 𝑃 is computed as the average difficulty reward over all questions it generates.
Diversity Reward 𝑟 div ∈ [−1, 0] prevents mode collapse. For a batch of 𝐺 generated samples X𝐼 = {𝑥 1, . . . , 𝑥𝐺 } given image 𝐼 , we cluster them by pairwise BLEU similarity [27] into 𝐾𝐼 clusters {𝐶 1(𝐼 ) , . . . , 𝐶𝐾(𝐼𝐼) }. For sample 𝑥𝑖 in cluster 𝐶𝑘(𝐼 ) , the redundancy density is: |𝐶 (𝐼 ) | (4) 𝑝𝑖 = 𝑘 𝐺 After min-max normalization over the batch, the diversity reward is: 𝑝𝑖 − min 𝑗 𝑝 𝑗 𝑟 div (𝑥𝑖 ) = − (5) max 𝑗 𝑝 𝑗 − min 𝑗 𝑝 𝑗 This penalizes samples in large clusters, encouraging exploration of diverse programs.
3.3
Solver Policy
The Solver receives a question 𝑞 (including images, code, and opˆ Its reward function is: tions) and generates an answer 𝑦. 𝑟 𝑆 = 𝜔 format𝑟 format + 𝜔 acc𝑟 acc
(6)
The format reward 𝑟 format ∈ {0, 1} ensures the answer is extractable and properly formatted, while the accuracy reward 𝑟 acc ∈ {0, 1} measures semantic correctness against the ground-truth answer 𝑦 ∗ determined by code execution.
3.4
Self-Evolution Loop
The Challenger and Solver co-evolve through alternating optimization over 𝑇 iterations. Algorithm 1 presents the complete training procedure. The priority queue Q maintains the top-𝑀 code examples ranked by 𝑟 diff . For codes with equal 𝑟 diff , later additions have priority. BLEUbased deduplication ensures semantic diversity, codes exceeding 𝜎high are discarded. This alternating optimization drives continuous improvement: the Challenger learns to generate diverse, moderately
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Algorithm 1 Self-Evolution Loop of EVE
the token length of output 𝑜𝑖 . The probability ratio is:
Require: Initial policies 𝜋𝐶0 , 𝜋𝑆0 ; Image library D; Priority queue Q; Total iterations 𝑇 ; Length of args_list 𝑁 ; Number of in-context code examples 𝑁 e ; Training steps per iteration 𝑁 step ; Batch size 𝐵; Solver’s sampling size 𝐾; Ensure: Evolved policies 𝜋𝐶𝑇 , 𝜋𝑆𝑇 1: for 𝑡 = 1 to 𝑇 do 2: Initialize question bank Y𝑡 ← ∅ 3: // Phase 1: Train Challenger & VQA generation 4: for step = 1 to 𝑁 step do 𝑀 5: Sample image 𝐼 ∼ D and 𝑁 e examples {𝑃𝑚 }𝑚=1 ∼Q 𝑀 ) 6: Generate code 𝑃 ∼ 𝜋𝐶𝑡 (𝐼, {𝑃𝑚 }𝑚=1 7: Execute 𝑃 to obtain edited images {𝐼 𝑗 }𝑁𝑗=1 8: Construct questions 𝑞 0, 𝑞 1 from (𝐼, 𝑃, {𝐼 𝑗 }𝑁𝑗=1 ) 9: // Evaluate difficulty 10: for each question 𝑞 ∈ {𝑞 0, 𝑞 1 } do 𝐾 ∼ 𝜋 𝑡 (𝑞) 11: Sample answers {𝑦ˆ𝑘 }𝑘=1 𝑆 12: Compute accuracy acc𝑠 (𝑞) // see Eq. 2 13: Compute difficulty reward 𝑟 diff (𝑞) // see Eq. 3 14: Add (𝑞, 𝑟 diff (𝑞)) to question bank Y𝑡 15: end for 16: 𝑟¯diff ← 21 (𝑟 diff (𝑞 0 ) + 𝑟 diff (𝑞 1 )) 17: Add (𝑃, 𝑟¯diff ) to priority queue Q Í 18: 𝑟𝐶 ← 𝑘 𝜆𝑘 𝑟𝑘 // see Eq. 1 19: Update 𝜋𝐶𝑡 with reward 𝑟𝐶 // see Eq. 7 20: end for 21: // Phase 2: Train Solver 22: Sort Y𝑡 by 𝑟 diff in descending order 23: Y𝑡 ← Y𝑡 [: 𝐵 · 𝑁 step ] 24: Train 𝜋𝑆𝑡 on question bank Y𝑡 for 𝑁 step steps with 𝑟 𝑆 25: Obtain updated Solver 𝜋𝑆𝑡 +1 26: end for 𝑇 , 𝜋𝑇 27: return 𝜋𝐶 𝑆
difficult questions, while the Solver learns to answer them correctly, creating a virtuous cycle of self-evolution.
3.5
Training Objective
Both the Challenger and Solver policies are optimized using Reinforcement Learning (RL). Following DAPO [42], we apply global normalization averaging over all tokens while retaining the KL divergence regularization term from GRPO [30]. The optimization objective is: " 𝐽RL (𝜃 ) = E
𝑞∼D 𝐺 ∼𝜋 {𝑜𝑖 }𝑖=1 𝜃 old (· |𝑞)
1 Í𝐺
|𝑜𝑖 | 𝐺 ∑︁ ∑︁
𝑖=1 |𝑜 𝑖 | 𝑖=1 𝑡 =1
min 𝑟𝑖,𝑡 (𝜃 )𝐴ˆ𝑖,𝑡 ,
clip(𝑟𝑖,𝑡 (𝜃 ), 1 − 𝜖low, 1 + 𝜖high )𝐴ˆ𝑖,𝑡 # − 𝛽𝐷 KL (𝜋𝜃 ||𝜋ref )
(7)
𝑟𝑖,𝑡 (𝜃 ) =
(8)
and the advantage function is computed as: 𝐴ˆ𝑖,𝑡 =
𝑅𝑖 − mean({𝑅 𝑗 }𝐺𝑗=1 ) std({𝑅 𝑗 }𝐺𝑗=1 )
(9)
where 𝑅𝑖 is the total reward for output 𝑜𝑖 (either 𝑟𝐶 for Challenger or 𝑟 𝑆 for Solver). The advantage is shared across all tokens in the same output, encouraging coherent generation. The KL divergence penalty prevents the policy from deviating too far from the reference policy 𝜋 ref , maintaining stability during training.
4 Experiments 4.1 Experimental Setup Implementation Details. Both Challenger and Solver policies are initialized from the same base checkpoint. For the Challenger, we set the length of args_list 𝑁 = 4, the number of in-context code examples 𝑁 e = 2, sampling rounds 𝐾 = 6 for difficulty estimation, and batch size 𝐵 = 128. The priority queue maintains the top50 (𝑀 = 50) code examples. We train for 𝑇 = 3 iterations with 𝑁 step = 10 training steps per iteration. In the task synthesis phase, we randomly select a parameter set-edited image pair and construct one problem each for Type-0 and Type-1 categories. For reward weights, we set 𝜆format = 0.2, 𝜆valid = 0.4, 𝜆diff = 0.4, 𝜆div = 0.3 for the Challenger, and 𝜔 format = 0.2, 𝜔 acc = 0.8 for the Solver. The image library is derived from the Vision-SR1-47K dataset[21], which contains high-quality images from diverse domains. More details are provided in the Appendix. Baselines. We first instantiate EVE on Qwen3-VL-Instruct models [2]. We compare with pseudo-label-based self-evolution methods VisPlay [11], MM-Zero [20], and ViPER [45]; and template-based methods Jigsaw-R1 [39] and Spatial-SSRL [25]. For fair comparison, we reproduce two representative methods, VisPlay and Jigsaw-R1 on Qwen3-VL-8B-Instruct using the same training hyperparameters and image corpus as our method. We also compare against closedsource models GPT-5 mini/nano [32] and GPT-4o [14]; and larger open-source MLLMs InternVL3-9B [50] and LLaVA-OneVision72B [18]. Evaluation Setting. We adopt VLMEvalKit [6], a widely used evaluation framework, for all assessments. We evaluate performance across diverse benchmarks covering multiple capabilities: MMStar [3] and MMVet [43] (general VQA); HallusionBench [8] (visual hallucination); MIA-Bench [28] (complex instruction following); VisuLogic [41] and MathVista (testmini subset) [26] (mathematical and logical reasoning); BLINK [7](visual perception); MuirBench [37] (multi-image understanding). In total, the evaluation suite comprises more than 10,000 test samples, comprehensively covering a wide variety of multimodal understanding and reasoning tasks.
4.2 where 𝑞 is the input query (image and prompt for Challenger, or question for Solver), {𝑜𝑖 }𝐺 𝑖=1 are 𝐺 sampled outputs, and |𝑜 𝑖 | denotes
𝜋𝜃 (𝑜𝑖,𝑡 | 𝑞, 𝑜𝑖,<𝑡 ) 𝜋𝜃 old (𝑜𝑖,𝑡 | 𝑞, 𝑜𝑖,<𝑡 )
Main Results
Comparison with VisPlay over 3 iterations. We first evaluate on Qwen3-VL-8B-Instruct and compare with the base model as well as
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Yongrui Heng et al.
Table 2: Performance comparison of EVE with VisPlay on Qwen3-VL-8B over 3 iterations. General VQA Alignment Math Reasoning Perception Multi-Image Overall MMStar MMVet HalluBench MIA-Bench VisuLogic MathVista BLINK Muirbench
Method Qwen3-VL-8B-Instruct
72.07
66.79
61.18
91.98
24.6
76.9
65.02
73.27
66.48
VisPlay-iter1 VisPlay-iter2 VisPlay-iter3
73.40 72.67 73.13
69.82 72.29 69.22
61.07 60.57 61.56
91.46 92.35 93.02
25.6 23.7 26.3
78.5 77.7 78.8
65.83 65.12 64.49
75.38 75.23 74.96
67.63 67.45 67.69
Ours-iter1 Ours-iter2 Ours-iter3
73.47 74.20 74.53
70.00 71.74 71.10
62.70 62.19 62.78
91.05 93.31 92.99
24.6 25.3 27.2
79.2 79.3 77.8
65.44 65.49 66.81
73.23 74.12 74.23
67.46 68.21 68.43
Table 3: Extended training results on Qwen3-VL-8B up to 5 iterations. General VQA Alignment Math Reasoning Perception Multi-Image Overall MMStar MMVet HalluBench MIA-Bench VisuLogic MathVista BLINK Muirbench
Method Qwen3-VL-8B-Instruct
72.07
66.79
61.18
91.98
24.6
76.9
65.02
73.27
66.48
Ours-iter3 Ours-iter4 Ours-iter5
74.53 73.93 73.53
71.10 71.88 70.87
62.78 62.99 62.78
92.99 93.31 93.19
27.2 27.8 27.8
77.8 77.9 78.1
66.81 67.33 67.18
74.23 74.38 76.00
68.43 68.69 68.68
Table 4: Comparison with state-of-the-art MLLMs and self-evolution methods. Method
General VQA Alignment Math Reasoning Perception Multi-Image Overall MMStar MMVet HalluBench MIA-Bench VisuLogic MathVista BLINK Muirbench Closed-Source MLLMs
GPT-5 mini (minimal) GPT-5 nano (high) GPT-4o-20240513
61.3 68.6 64.7
69.1
55.0
InternVL3-9B LLaVA-OneVision-72B Qwen3-VL-8B-Instruct
66.3 65.8 72.1
76.2 60.6 66.8
51.2 49.0 61.2
92.3 89.9 -
27.6 24.5 26.3
59.6 71.5 63.8
56.7 58.3 68.0
57.5 65.7 68.0
-
71.5 67.1 76.9
58.6 55.4 65.0
51.4 54.8 73.3
66.5
77.9
56.2 64.7
73.2
67.5
Open-Source MLLMs 92.0
24.6
Template-based Self-evolution Methods Spatial-SSRL-7B Jigsaw-R1-8B
74.3
69.4
53.2 62.1
91.3
27.4
Qwen-ViPER-7B VisPlay-8B-iter3 MM-Zero-8B-iter3
66.2 73.1 70.7
65.8 69.2 69.5
54.4 61.6 61.7
93.0 92.9
26.3 25.7
78.8 74.7
57.6 64.5 65.9
75.0 72.2
67.7 66.7
Ours-8B-iter4
73.9
71.9
63.0
93.3
27.8
77.9
67.3
74.4
68.7
Pseudo-label-based Self-evolution Methods
VisPlay [11], a representative pseudo-label-based self-evolution method. Results are shown in Table 2. After 3 iterations, EVE achieves an overall score of 68.43, outperforming both the base model (66.48, +1.95) and VisPlay-iter3 (67.69, +0.74). On individual benchmarks, EVE at iteration 3 obtains the best results on MMStar (74.53), HallusionBench (62.78), VisuLogic (27.2), and BLINK (66.81), demonstrating strong improvements in comprehensive evaluation, hallucination resistance and visual logical reasoning. Addressing pseudo-label degradation. A key distinction lies in the training dynamics. Our method exhibits monotonically increasing overall performance across iterations (67.46 → 68.21 → 68.43), whereas VisPlay fluctuates (67.63 → 67.45 → 67.69), with a notable drop at iteration 2. Furthermore, EVE maintains stable or
improved performance across all individual benchmarks without any regression. For instance, on HallusionBench, VisPlay suffers from performance fluctuations and even degradation, suggesting that pseudo-labels generated by the evolving model may become less reliable over time and further amplify hallucinations, whereas EVE mitigates hallucination by grounding learning in code execution feedback. This empirically confirms that learning from accurate reward signals resolves the fundamental limitation of pseudo-label methods. Extended Training Analysis. To analyze long-term evolution trends, we extend training up to 5 iterations, with results shown in Table 3 and Figure 1(a). Our method achieves the highest overall score of
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Table 5: Ablation study on challenger reward components. MMStar HalluBench MathVista BLINK
Overall
Qwen3-VL-8B-Instruct
72.07
61.18
76.90
65.02
68.79
Full Method ours-iter1 ours-iter2 ours-iter3
73.47 74.20 74.53
62.70 62.19 62.78
79.20 79.30 77.80
65.44 65.49 66.81
70.20 70.30 70.48
w/o 𝑟 div ours-iter1 ours-iter2 ours-iter3
73.80 74.47 73.93
61.53 62.38 61.16
77.30 78.10 77.70
63.23 68.97 ( − 1.23) 63.18 69.53 ( − 0.77) 64.80 69.40 ( − 1.08)
w/o 𝑟 div + w/o 𝑟 diff ours-iter1 ours-iter2 ours-iter3
73.20 73.87 72.73
61.36 61.28 61.03
77.10 77.40 76.70
64.14 68.95 ( − 1.25) 63.02 68.89 ( − 1.41) 63.80 68.57 ( − 1.91)
Table 6: Results on Qwen3-VL-4B across 3 iterations. Method
MMStar MMVet HalluBench VisuLogic BLINK Overall
70.5
70.39
70.30 70.08
Overall Score
Method
70.0 69.5 69.41 69.0 68.5
Qwen3-VL-8B-Instruct Ours-iter2 0
25
50
100
Priority Queue Size M
Figure 3: Ablation on priority queue size 𝑀. Overall score (average of MMStar, HallusionBench, MathVista, and BLINK) at iteration 2 under different 𝑀 values. Table 7: Results on MiMo-VL-7B-SFT-2508 across 3 iterations.
Qwen3-VL-4B-Instruct
70.33
67.89
60.08
24.2
65.18
57.54
Method
Ours-iter1 Ours-iter2 Ours-iter3
71.20 70.53 70.86
68.99 68.58 69.36
59.98 61.82 61.21
26.1 27.2 27.3
65.33 65.20 65.35
58.32 58.67 58.82
MiMo-VL-7B-SFT-2508
72.87
66.67
60.93
20.3
62.34
56.62
Ours-iter1 Ours-iter2 Ours-iter3
73.07 73.40 73.47
68.53 67.52 69.63
61.28 61.09 61.89
22.1 23.4 23.7
62.65 62.96 62.39
57.53 57.68 58.22
68.7, outperforming the base model by +2.2. The most substantial gains are observed on MMVet (+5.1), VisuLogic (+3.2), and BLINK (+2.3). Performance stabilizes at iteration 5 (68.68), with some benchmarks exhibiting marginal trade-offs. Crucially, the absence of significant performance collapse over 5 iterations demonstrates the stability of code-execution-based rewards, contrasting sharply with pseudo-label methods that typically degrade after 2–3 iterations. Comparison with Existing Advanced MLLMs and Self-Evolution Methods. Table 4 compares our 8B model against recent advanced closed-source MLLMs, open-source models, and self-evolution baselines. For self-evolution methods, we report results from the bestperforming iteration checkpoint. Our method achieves the highest overall score of 68.7, surpassing all compared self-evolution approaches. Relative to self-evolution baselines, EVE outperforms VisPlay (67.7) by +1.0 and MM-Zero (66.7) by +2.0. Particularly noteworthy is the performance on BLINK, where EVE achieves 67.3, exceeding both the base model (+2.3) and all other self-evolution methods, demonstrating substantial enhancement in visual perception capabilities. While Jigsaw-R1 leads on MMStar and VisPlay leads on MathVista and MuirBench, EVE achieves the best performance on all remaining benchmarks, indicating broader and more balanced improvements. Addressing template limitations. Jigsaw-R1’s fixed templates improve MMStar and VisuLogic but constrain visual perception and multi-image understanding, leading to weaker performance on BLINK and MuirBench. By contrast, the Challenger autonomously discovers diverse editing operations through code generation, exploring richer visual reasoning patterns beyond fixed, human-designed transformation templates.
4.3
Ablation Studies
Ablation on challenger reward components. We ablate the diversity reward 𝑟 div and difficulty reward 𝑟 diff to assess their contributions, as shown in Table 5. Removing 𝑟 div leads to a consistent
MMStar MMVet HalluBench VisuLogic BLINK Overall
performance drop across all iterations, with the overall score at iteration 3 decreasing from 70.48 to 69.40 (−1.08). This degradation is particularly pronounced on BLINK, where the score drops from 66.81 to 64.80 (−2.01), indicating that without diversity regularization, the Challenger generates repetitive transformations that fail to adequately challenge the Solver’s capabilities. Further removing 𝑟 diff exacerbates the decline, with the overall score at iteration 3 falling to 68.57 (−1.91 compared to the full method). The cumulative effect is evident across all benchmarks: MMStar drops by 1.80 (74.53→72.73), HallusionBench by 1.75 (62.78→61.03), and BLINK by 3.01 (66.81→63.80). Without difficulty calibration, the Challenger produces questions that are either trivially easy or excessively hard, both of which provide weak training signals for the Solver. These results confirm that both diversity and difficulty rewards are essential for maintaining a rich, appropriately challenging question distribution that drives effective self-evolution. We do not ablate 𝑟 format and 𝑟 valid , as removing them would prevent the Challenger from generating valid executable code, making it impossible to construct questions for Solver training. Ablation on Priority queue size 𝑀. We ablate the priority queue size 𝑀 to examine the effect of retaining high-quality code examples as few-shot context across iterations, as shown in Figure 3. Performance increases monotonically with 𝑀, confirming that a larger pool of high-reward historical programs provides richer fewshot context and enables the Challenger to generate more diverse transformations over iterations. Ablation on Different Model Scales. To verify the generalizability of our framework, we apply it to the smaller Qwen3-VL-4B model, with results shown in Table 6. Our method consistently improves the 4B model across iterations, raising the overall score from 57.54 to 58.82 (+1.28) after 3 iterations. The smaller gain compared to the 8B model likely reflects the greater difficulty smaller models face in generating sufficiently diverse and high-quality code.
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
image_0 (original)
image_1
Yongrui Heng et al.
image_2
from PIL import Image, ImageEnhance def edit_image(img: Image.Image, mode: str, param1: float, param2: float) -> Image.Image: if mode == 'rotate_mirror': img = img.rotate(90, expand=True) img = img.transpose(Image.FLIP_LEFT_RIGHT) elif mode == 'brighten': enhancer = ImageEnhance.Brightness(img) img = enhancer.enhance(param1) elif mode == 'saturate': enhancer = ImageEnhance.Color(img) img = enhancer.enhance(param1) elif mode == 'crop_zoom': width, height = img.size left = width * param1 / 1000 upper = height * param2 / 1000 right = width * (1 - param1 / 1000) lower = height * (1 - param2 / 1000) img = img.crop((left, upper, right, lower)) return img
image_3
EVE (Ours) Response ... `rotate_mirror` means: first, rotate the image 90 degrees then flip it horizontally. Let's examine the images: - image_0: A top-down view of penne pasta with sauce and a basil leaf on a white plate. - image_1: The plates, pasta, basil, lights and shadows are all the same, but the locations seem to be different. - image_2: The background is now white, not gray. The shadow around the plate is also altered. - image_3: The plate and food look the same, but the background is drastically changed, suggesting excessive color saturation. - image_4: This image shows only a portion, likely cropped from the left side. The pasta, basil, and plate are visible but cut off at the edges. …only image_1 appear to be mirrored or rotated. Thus, candidate is image_1. \boxed{A}
image_4
The given images are image_0, image_1, image_2, image_3, and image_4, respectively. Images image_1 through image_4 are the results of applying the `edit_image` function to image_0 with different arguments.
Qwen3-VL (Base model) Reponse Looking at the code … rotate_mirror means that the image is rotated 90 degrees clockwise then lipped horizontally. - image_1: Shows the same pasta dish... It does not appear to be mirrored or rotated. - image_2: This image has a completely white background instead of the gray one from image_0. - image_3: This image also has a white background similar to image_2, meaning it's likely not the result of just rotating and mirroring. … In fact, I notice that image_1 is identical to image_0. So that cannot be it. image_2 and image_3 have white backgrounds, differing from the original's gray, so they don't match the `rotate_mirror` effect which preserves the background color. That leaves image_4. Thus, we choose D. \boxed{D}
After applying the `edit_image` function to image_0 with `{'mode': 'rotate_mirror', 'param1': 500, 'param2': 500}`, which candidate image will be produced? Options: A. image_1 B. image_2 C. image_3 D. image_4
0.88
0.220
0.86
0.215
0.84
Code Emb. Diversity
Diversity Score (BLEU & Visual)
Figure 4: A qualitative example of the Parameter-to-Image task. By generating executable scripts, the Challenger tries visual operations beyond the seed examples (such as color enhancement and mirroring) and combines them to create complex visual transformations. While the baseline model suffers from visual hallucinations, our evolved Solver accurately grounds the code logic in precise visual evidence to make the correct deduction.
0.210
0.82 0.80
Code (BLEU) Visual (Embedding) Code (Embedding)
0.78 1
2
3
Iteration
4
0.205 0.200
5
Figure 5: Diversity evolution across iterations. Left axis: code BLEU diversity and visual embedding diversity share the same scale. Right axis: code embedding diversity. Ablation on Different Model Families. We instantiate EVE on MiMo-VL-7B-SFT-2508 [40] to validate the generalizability of our approach across different model families. As shown in Table 7, EVE consistently improves over the base model across iterations, achieving an overall score of 58.22 at iteration 3, representing a +1.60 gain over MiMo-VL-7B-SFT-2508 (56.62). Gains are observed across most benchmarks, with notable improvements on MMVet (+2.96) and HallusionBench (+0.96), demonstrating that EVE generalizes beyond a single model family.
4.4
Analysis
Code Diversity Evolution. We track two code-level metrics across iterations (Figure 5): (1) code BLEU diversity, defined as the mean pairwise BLEU distance between code strings, and (2) code embedding diversity, defined as the mean pairwise cosine distance between embedding features (encoded by Qwen3-VL-Embedding-2B [19]) of code strings. Both metrics increase consistently across iterations, confirming that the Challenger generates increasingly varied programs over time. The co-movement of the two metrics further
validates that BLEU serves as a reliable and computationally efficient proxy for semantic code diversity. Visual Diversity Evolution. As shown in Figure 5, We also measure visual diversity as the mean pairwise cosine distance between Qwen3-VL-Embedding-2B features of edited images. This metric rises in parallel with the code-level diversity metrics, demonstrating that code-level variation directly drives perceptual variation in generated images. The consistent growth confirms that EVE’s self-evolution produces not only structurally diverse programs but also visually distinct training samples. Qualitative Analysis. Crucially, the observed diversity stems not merely from varying parameters within a fixed transformation type (e.g., adjusting rotation angles within the same edit image function, as in the Challenger’s prompt template shown in Figure 2), but from the Challenger actively discovering and composing entirely new operation types. As illustrated in Figure 4, the Challenger autonomously invents compound transformations—such as combining rotation with mirroring—that go beyond the 4 seed code examples provided at initialization. These results validate the BLEU-based diversity reward as a semantically grounded signal for promoting cross-modal diversity during RL training.
5
Conclusion
We presented EVE, a self-evolution framework for multimodal large language models that leverages executable visual transformations as a programmatic external environment. By grounding all supervision in deterministic code execution, EVE eliminates the pseudolabel degradation that plagues confidence-based methods. Meanwhile, the Challenger-Solver dual-policy architecture with multidimensional reward-driven co-evolution enables an expandable, adaptive question space, overcoming the diversity and difficulty constraints of template-based approaches. Comprehensive experiments demonstrate consistent improvements across 8 benchmarks, with particularly strong gains on MMVet, VisuLogic and BLINK.
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
References [1] Sanghwan Bae, Jiwoo Hong, Min Young Lee, Hanbyul Kim, JeongYeon Nam, and Donghyun Kwak. 2026. Online difficulty filtering for reasoning oriented reinforcement learning. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 700–719. [2] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. 2025. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025). [3] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. 2024. Are we on the right way for evaluating large vision-language models? Advances in Neural Information Processing Systems 37 (2024), 27056–27087. [4] Yang Chen, Yufan Shen, Wenxuan Huang, Sheng Zhou, Qunshu Lin, Xinyu Cai, Zhi Yu, Jiajun Bu, Botian Shi, and Yu Qiao. 2025. Learning only with images: Visual reinforcement learning with reasoning, rendering, and visual feedback. arXiv preprint arXiv:2507.20766 (2025). [5] Shijian Deng, Kai Wang, Tianyu Yang, Harsh Singh, and Yapeng Tian. 2025. SelfImprovement in Multimodal Large Language Models: A Survey. In Findings of the Association for Computational Linguistics: EMNLP 2025. 1987–2006. [6] Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. 2024. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM International Conference on Multimedia. 11198–11201. [7] Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. 2024. Blink: Multimodal large language models can see but not perceive. In European Conference on Computer Vision. Springer, 148–166. [8] Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. 2024. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 14375–14385. [9] Tongkun Guan, Zhibo Yang, Jianqiang Wan, Mingkun Yang, Zhengtao Guo, Zijian Hu, Ruilin Luo, Ruize Chen, Songtao Jiang, Peng Wang, et al. 2026. CodePercept: Code-Grounded Visual STEM Perception for MLLMs. arXiv preprint arXiv:2603.10757 (2026). [10] Bingxiang He, Yuxin Zuo, Zeyuan Liu, Shangziqi Zhao, Zixuan Fu, Junlin Yang, Cheng Qian, Kaiyan Zhang, Yuchen Fan, Ganqu Cui, et al. 2026. How Far Can Unsupervised RLVR Scale LLM Training? arXiv preprint arXiv:2603.08660 (2026). [11] Yicheng He, Chengsong Huang, Zongxia Li, Jiaxin Huang, and Yonghui Yang. 2025. Visplay: Self-evolving vision-language models from images. arXiv preprint arXiv:2511.15661 (2025). [12] Jack Hong, Chenxiao Zhao, ChengLin Zhu, Weiheng Lu, Guohai Xu, and Xing Yu. 2025. DeepEyesV2: Toward Agentic Multimodal Model. arXiv preprint arXiv:2511.05271 (2025). [13] Chengsong Huang, Lantao Yu, Yicheng He, Zongxia Li, Jiaxin Huang, and Yonghui Yang. 2025. R-Zero: Self-Evolving Reasoning LLM from Zero Data. arXiv preprint arXiv:2508.05004 (2025). [14] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024). [15] Ahmadreza Jeddi, Hakki Can Karaimer, Hue Nguyen, Zhongling Wang, Ke Zhao, Javad Rajabi, Ran Zhang, Raghav Goyal, Babak Taati, and Radek Grzeszczuk. 2025. Puzzle Curriculum GRPO for Vision-Centric Reasoning. arXiv preprint arXiv:2512.14944 (2025). [16] Zaid Khan, Elias Stengel-Eskin, Jaemin Cho, and Mohit Bansal. 2024. Dataenvgym: Data generation agents in teacher environments with student feedback. arXiv preprint arXiv:2410.06215 (2024). [17] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles. 611–626. [18] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326 (2024). [19] Mingxin Li, Yanzhao Zhang, Dingkun Long, Chen Keqin, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2026. Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking. arXiv preprint arXiv:2601.04720 (2026). [20] Zongxia Li, Hongyang Du, Chengsong Huang, Xiyang Wu, Lantao Yu, Yicheng He, Jing Xie, Xiaomin Wu, Zhichao Liu, Jiarui Zhang, et al. 2026. MM-Zero: SelfEvolving Multi-Model Vision Language Models From Zero Data. arXiv preprint arXiv:2603.09206 (2026). [21] Zongxia Li, Wenhao Yu, Chengsong Huang, Rui Liu, Zhenwen Liang, Fuxiao Liu, Jingxi Che, Dian Yu, Jordan Boyd-Graber, Haitao Mi, and Dong Yu.
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
2025. Self-Rewarding Vision-Language Model via Reasoning Decomposition. arXiv:2508.19652 [cs.CV] https://arxiv.org/abs/2508.19652 [22] Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. 2023. Code as policies: Language model programs for embodied control. In 2023 IEEE International conference on robotics and automation (ICRA). IEEE, 9493–9500. [23] Wei Liu, Junlong Li, Xiwen Zhang, Fan Zhou, Yu Cheng, and Junxian He. 2024. Diving into self-evolving training for multimodal reasoning. arXiv preprint arXiv:2412.17451 (2024). [24] Wei Liu, Siya Qi, Yali Du, and Yulan He. 2026. Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain. arXiv preprint arXiv:2603.02218 (2026). [25] Yuhong Liu, Beichen Zhang, Yuhang Zang, Yuhang Cao, Long Xing, Xiaoyi Dong, Haodong Duan, Dahua Lin, and Jiaqi Wang. 2025. Spatial-ssrl: Enhancing spatial understanding via self-supervised reinforcement learning. arXiv preprint arXiv:2510.27606 (2025). [26] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating Math Reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255 (2023). [27] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311–318. [28] Yusu Qian, Hanrong Ye, Jean-Philippe Fauconnier, Peter Grasch, Yinfei Yang, and Zhe Gan. 2024. Mia-bench: Towards better instruction following evaluation of multimodal llms. arXiv preprint arXiv:2407.01509 (2024). [29] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PmLR, 8748–8763. [30] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. 2024. Deepseekmath: Pushing the limits of Math Reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024). [31] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. HybridFlow: A Flexible and Efficient RLHF Framework. arXiv preprint arXiv: 2409.19256 (2024). [32] Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. 2025. Openai gpt-5 system card. arXiv preprint arXiv:2601.03267 (2025). [33] Qi Song, Honglin Li, Yingchen Yu, Haoyi Zhou, Lin Yang, Song Bai, Qi She, Zilong Huang, and Yunqing Zhao. 2025. CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoning. arXiv preprint arXiv:2512.17312 (2025). [34] Meghana Sunil, Manikandarajan Venmathimaran, and Muthu Subash Kavitha. 2026. iReasoner: Trajectory-Aware Intrinsic Reasoning Supervision for SelfEvolving Large Multimodal Models. arXiv preprint arXiv:2601.05877 (2026). [35] Dídac Surís, Sachit Menon, and Carl Vondrick. 2023. Vipergpt: Visual inference via python execution for reasoning. In Proceedings of the IEEE/CVF international conference on computer vision. 11888–11898. [36] Omkar Thawakar, Shravan Venkatraman, Ritesh Thawkar, Abdelrahman Shaker, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, and Fahad Khan. 2025. Evolmm: Self-evolving large multimodal models with continuous rewards. arXiv preprint arXiv:2511.16672 (2025). [37] Fei Wang, Xingyu Fu, James Y Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, et al. 2024. Muirbench: A comprehensive benchmark for robust multi-image understanding. arXiv preprint arXiv:2406.09411 (2024). [38] Han Wang, Yi Yang, Jingyuan Hu, Minfeng Zhu, and Wei Chen. 2026. V-Zero: Self-Improving Multimodal Reasoning with Zero Annotation. arXiv preprint arXiv:2601.10094 (2026). [39] Zifu Wang, Junyi Zhu, Bo Tang, Zhiyu Li, Feiyu Xiong, Jiaqian Yu, and Matthew B. Blaschko. 2025. Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles. [40] LLM-Core-Team Xiaomi. 2025. MiMo-VL Technical Report. arXiv:2506.03569 [cs.CL] https://arxiv.org/abs/2506.03569 [41] Weiye Xu, Jiahao Wang, Weiyun Wang, Zhe Chen, Wengang Zhou, Aijun Yang, Lewei Lu, Houqiang Li, Xiaohua Wang, Xizhou Zhu, et al. 2025. Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models. arXiv preprint arXiv:2504.15279 (2025). [42] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. 2025. Dapo: An opensource llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476 (2025). [43] Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. 2023. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490 (2023). [44] Yu Zeng, Wenxuan Huang, Shiting Huang, Xikun Bao, Yukun Qi, Yiming Zhao, Qiuchen Wang, Lin Chen, Zehui Chen, Huaian Chen, et al. 2025. Agentic Jigsaw
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Interaction Learning for Enhancing Visual Perception and Reasoning in VisionLanguage Models. arXiv preprint arXiv:2510.01304 (2025). [45] Juntian Zhang, Song Jin, Chuanqi Cheng, Yuhan Liu, Yankai Lin, Xun Zhang, Yufei Zhang, Fei Jiang, Guojun Yin, Wei Lin, et al. 2025. Viper: Empowering the self-evolution of visual perception abilities in vision-language model. arXiv preprint arXiv:2510.24285 (2025). [46] Yi-Fan Zhang, Xingyu Lu, Shukang Yin, Chaoyou Fu, Wei Chen, Xiao Hu, Bin Wen, Kaiyu Jiang, Changyi Liu, Tianke Zhang, et al. 2025. Thyme: Think Beyond Images. arXiv preprint arXiv:2508.11630 (2025). [47] Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. 2025. Absolute zero: Reinforced self-play reasoning with zero data. arXiv preprint arXiv:2505.03335 (2025). [48] Shitian Zhao, Shaoheng Lin, Ming Li, Haoquan Zhang, Wenshuo Peng, Kaipeng Zhang, and Chen Wei. 2026. PyVision-RL: Forging Open Agentic Vision Models via RL. arXiv preprint arXiv:2602.20739 (2026). [49] Shitian Zhao, Haoquan Zhang, Shaoheng Lin, Ming Li, Qilong Wu, Kaipeng Zhang, and Chen Wei. 2025. Pyvision: Agentic vision with dynamic tooling. arXiv preprint arXiv:2507.07998 (2025). [50] Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. 2025. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479 (2025).
Yongrui Heng et al.
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
A Implementation Details A.1 Sandbox Execution
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Table 8: Training hyperparameters for the self-evolution loop.
Code execution is performed in a restricted Python sandbox, which is adapted from the Thyme [46] sandbox environment, with the following constraints: • Timeout: 5 seconds per execution • Prohibited operations: network access, subprocess calls, random module usage • Random seed is fixed to ensure reproducibility • Code length limit: 2000 characters for the generated code
A.2
Training Hyperparameters
The hyperparameters used for training the self-evolution loop are detailed in Table 8.
A.3
Training Infrastructure
Our implementation is based on the VERL [31] reinforcement learning framework with vLLM [17] for efficient inference. The training pipeline consists of: • Challenger training: Uses 4 A100 GPUs with FSDP (Fully Sharded Data Parallel) strategy. The Solver model runs as a vLLM service on 4 separate GPUs to evaluate question difficulty in real-time during rollout. • Solver training: Uses 8 A100 GPUs with FSDP strategy.
A.4
Seed Code Examples
The code example queue 𝑄 is initialized with four seed examples, as shown in Figures 6-9: (1) Jigsaw Puzzle (Figure 6): divides the image into a 2 × 2 grid and permutes blocks according to a specified order; (2) Rotation (Figure 7): rotates the image by a specified angle with bicubic resampling; (3) Cropping (Figure 8): extracts different regions defined by normalized bounding box coordinates; (4) Bounding Box Drawing (Figure 9): draws red rectangles on the image at specified coordinates.
A.5
B Additional Experimental Results B.1 Qualitative Examples We present qualitative examples in Figures 14-16 to illustrate how the EVE Solver reasons through the synthesized VQA tasks. Each
Value
General Training Total iterations 𝑇 Training steps per iteration 𝑁 step Learning rate Training batch size
3 10 1e-6 128
RL Optimization (Eq. 7) KL coefficient 𝛽 KL loss type Clip range 𝜖low Clip range 𝜖high
0.01 low_var_kl 0.28 0.20
Sampling Parameters Rollout batch size 𝐵 Temperature Top-p Top-k Presence penalty Max prompt length Max response length
128 1.0 1.0 40 2.0 8192 2048
Challenger Training Solver sampling rounds 𝐾 Rollout sampling 𝐺 GPUs Format weight 𝜆format Validity weight 𝜆valid Difficulty weight 𝜆diff Diversity weight 𝜆div
6 4 4 0.2 0.4 0.4 0.3
Solver Training Rollout sampling 𝐺 GPUs Format weight 𝜔 format Accuracy weight 𝜔 acc
8 8 0.2 0.8
Method-related BLEU threshold for deduplication 𝜎high Code length limit Length of args_list 𝑁 Number of in-context examples 𝑁 e Priority queue size 𝑀
Templates
We present the prompt templates used in our framework. The Challenger prompt (Figure 10) instructs the model to generate an executable edit_image function along with four parameter sets, using two in-context examples sampled from the priority queue 𝑄. The Solver prompt (Figure 11) is intentionally minimal, presenting the VQA question and requesting the answer in a boxed format for easy extraction. The two VQA synthesis templates (Figures 12 and 13) construct the two types of questions from the Challenger’s code execution results: Parameter-to-Image asks the Solver to identify which output image corresponds to a given parameter set, while Image-to-Parameter requires the Solver to determine which parameters produced a specific output image.
Hyperparameter
0.25 2000 chars 4 2 50
example shows the complete question on the left and the Solver’s full reasoning process on the right.
C
Limitations and Future Directions.
While EVE transfers to higher-level reasoning tasks, 2D pixel operations have inherent limitations in encoding physical commonsense or complex compositional relationships. Future work can integrate richer executable environments—3D rendering engines for physical reasoning, or UI rendering code for hierarchical layout understanding. Performance plateaus after 4-5 iterations suggest that continued improvement may require more sophisticated curriculum strategies or larger base models. Extending to other modalities (video, 3D, audio) also remains promising.
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Yongrui Heng et al.
Jigsaw Puzzle from PIL import Image def edit_image(img: Image.Image, N: int, order: list) -> Image.Image: width, height = img.size block_w = width // N block_h = height // N adjusted_size = (N * block_w, N * block_h) img = img.resize(adjusted_size) out_img = Image.new("RGB", adjusted_size) for new_idx in range(N * N): new_row = new_idx // N new_col = new_idx % N orig_idx = order[new_idx] orig_row = orig_idx // N orig_col = orig_idx % N left = orig_col * block_w upper = orig_row * block_h right = left + block_w lower = upper + block_h block = img.crop((left, upper, right, lower)) out_img.paste(block, (new_col * block_w, new_row * block_h)) return out_img args_list = [ {'N': 2, 'order': [3, 0, 1, 2]}, {'N': 2, 'order': [1, 0, 3, 2]}, {'N': 2, 'order': [0, 3, 2, 1]}, {'N': 2, 'order': [2, 1, 0, 3]}, ]
Figure 6: Seed code example: Jigsaw Puzzle. Rotation from PIL import Image def edit_image(img: Image.Image, angle: float) -> Image.Image: return img.rotate(angle, expand=True, resample=Image.Resampling.BICUBIC) args_list = [ {'angle': 15}, {'angle': 45}, {'angle': 90}, {'angle': 180}, ]
Figure 7: Seed code example: Rotation.
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Cropping from PIL import Image def edit_image(img: Image.Image, bbox_2d: list) -> Image.Image: real_width, real_height = img.size bbox_2d = [int(b * real_width / 1000) if i % 2 == 0 else int(b * real_height / 1000) for i, b in enumerate(bbox_2d)] return img.crop(bbox_2d) args_list = [ {"bbox_2d": [205, 220, 335, 422]}, {"bbox_2d": [103, 94, 378, 210]}, {"bbox_2d": [452, 603, 565, 750]}, {"bbox_2d": [154, 752, 357, 958]}, ]
Figure 8: Seed code example: Cropping. Bounding Box Drawing from PIL import Image, ImageDraw def edit_image(img: Image.Image, bbox_2d: list) -> Image.Image: draw = ImageDraw.Draw(img) bbox_2d = [int(b * real_width / 1000) if i % 2 == 0 else int(b * real_height / 1000) for i, b in enumerate(bbox_2d)] draw.rectangle(bbox_2d, outline="red", width=5) return img args_list = [ {"bbox_2d": [205, 220, 335, 422]}, {"bbox_2d": [103, 94, 378, 510]}, {"bbox_2d": [452, 610, 556, 850]}, {"bbox_2d": [154, 750, 357, 958]}, ]
Figure 9: Seed code example: Bounding Box Drawing.
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Yongrui Heng et al.
Prompt Template for Challenger # Task Write a simple Python function named `edit_image` that edit the user's image, and then design 4 different parameter sets. # Requirements 1. The `edit_image` function must accept a PIL Image object and specific parameters, returning a modified PIL Image object. 2. The Python code must include necessary imports, the `edit_image` function, and a list of dictionaries named `args_list`. 3. Ensure that the 4 sets of parameters in `args_list` produce 4 visually distinct editing results. 4. No comments must be added to the `edit_image` function. 5. The examples below are for format reference only. The parameters must be designed according to the specific content of the user's image; do not copy them directly. # Examples ## Example 1 ```python {code_str1} ``` ## Example 2 ```python {code_str2} ``` Observe the given image, design the code with reference to the image content. Output your reasoning process inside <thinking>...</thinking> tags, followed by the final Python code in ```python ... ```
Figure 10: Prompt template for the Challenger. Prompt Template for Solver {question} Please put your final answer within \\boxed{}.
Figure 11: Prompt template for the Solver. Template for VQA Synthesis (Parameter-to-Image) The given images are image_0, image_1, image_2, image_3, and image_4, respectively. Images image_1 through image_4 are the results of applying the `edit_image ` function to image_0 with different arguments. ```python {code_str} ``` Question: After applying the `edit_image` function to image_0 with `{arg_chosen}`, which candidate image will be produced? Options: A. image_1 B. image_2 C. image_3 D. image_4 Guidelines: First, understand the the code, then carefully observe the images, and solve the problem based on the key elements or details in the images.
Figure 12: VQA synthesis template for Parameter-to-Image questions.
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
Template for VQA Synthesis (Image-to-Parameter) The given images are image_0, image_1, image_2, image_3, and image_4, respectively. Images image_1 through image_4 are the results of applying the `edit_image ` function to image_0 with different arguments. ```python {code_str} ``` Question: Which set of arguments, when passed to `edit_image` for image_0, produces image_1? Options: A. {arg_1} B. {arg_2} C. {arg_3} D. {arg_4} Guidelines: First, understand the the code, then carefully observe the images, and solve the problem based on the key elements or details in the images.
Figure 13: VQA synthesis template for Image-to-Parameter questions. EVE Solver Response To determine which image results from the specified `edit_image` function call, let's break down the operation step-by-step.\n\n### Step 1: Analyze the Cropping Operation\n\nThe `crop_rect` is `(0, 0, 1000, 1000)`. Let's see how this translates to pixel coordinates using the provided code:\n\n- `left = int(0 * width / 1000) = 0`\n- `upper = int(0 * height / 1000) = 0`\n- `right = int(1000 * width / 1000) = width`\n- `lower = int(1000 * height / 1000) = height`\n\nThis means image_0 (original)
image_1
image_2
image_3
image_4
the crop operation selects the entire original image, from `(0, 0)` to `(width, height)`. Therefore, the `cropped_img` is identical to the original `image_0`. No part of the image is
The given images are image_0, image_1, image_2, image_3, and image_4, respectively. Images image_1 through image_4 are the results of applying the `edit_image` function to image_0 with different arguments.
removed.\n\n### Step 2: Analyze the Rotation Operation\n\nThe next step is `rotated_img =
```python from PIL import Image
degrees. In PIL (Pillow), a negative angle corresponds to a **clockwise** rotation. So, the
def edit_image(img: Image.Image, crop_rect: tuple, rotation_degrees: float) -> Image.Image: width, height = img.size left = int(crop_rect[0] * width / 1000) upper = int(crop_rect[1] * height / 1000) right = int(crop_rect[2] * width / 1000) lower = int(crop_rect[3] * height / 1000) cropped_img = img.crop((left, upper, right, lower)) rotated_img = cropped_img.rotate(rotation_degrees, expand=True, resample=Image.Resampling.BICUBIC) return rotated_img ``` Question: Which candidate image results from `{'crop_rect': (0, 0, 1000, 1000), 'rotation_degrees': -90}`? Options: A. image_1 B. image_2 C. image_3 D. image_4 Guidelines: First, understand the the code, then carefully observe the images, and solve the problem based on the key elements or details in the images.
cropped_img.rotate(-90, expand=True, ...)`.\n\n- **Rotation Angle:** The angle is `-90` image will be rotated 90 degrees clockwise.\n- **Expand Parameter:** The `expand=True` parameter ensures that the output image is large enough to contain the entire rotated source image. This prevents any part of the image from being cut off. For a 90-degree rotation, the width and height of the image will be swapped.\n\n### Step 3: Predict the Resulting Image\n\nApplying a 90-degree clockwise rotation to `image_0` will have the following effects:\n- The top of the original image (the hill with people on it) will become the right side of the new image.\n- The bottom of the original image (where the two photographers are standing) will become the left side of the new image.\n- The left side of the original image will become the top.\n- The right side of the original image will become the bottom.\n\nLet's examine the candidate images:\n- **image_1:** This is a slightly cropped version of the original image, but it is not rotated.\n- **image_2:** This is a rotated and heavily cropped section of the original image.\n- **image_3:** This shows the original image rotated counter-clockwise by about 45 degrees and placed on a black background. This does not match our prediction.\n**image_4:** This image shows the scene from `image_0` rotated 90 degrees clockwise. The photographers are now on the left, and the crowd on the hill is on the right. The orientation of all elements matches the predicted transformation perfectly.\n\nBased on the analysis, the correct option is \\boxed{D}.
Figure 14: A Parameter-to-Image example. Left: the synthesized question with code, parameters, and candidate images. Right: the EVE Solver’s step-by-step reasoning process leading to the correct answer.
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
image_0 (original)
image_1
image_2
Yongrui Heng et al.
image_3
image_4
The given images are image_0, image_1, image_2, image_3, and image_4, respectively. Images image_1 through image_4 are the results of applying the `edit_image` function to image_0 with different arguments. ```python from PIL import Image def edit_image(img: Image.Image, crop_rect: tuple, rotation_degrees: float) -> Image.Image: width, height = img.size left = int(crop_rect[0] * width / 1000) upper = int(crop_rect[1] * height / 1000) right = int(crop_rect[2] * width / 1000) lower = int(crop_rect[3] * height / 1000) cropped_img = img.crop((left, upper, right, lower)) rotated_img = cropped_img.rotate(rotation_degrees, expand=True, resample=Image.Resampling.BICUBIC) return rotated_img ``` Question: Which set of arguments, when passed to `edit_image` for image_0, produces image_1? Options: A. {'crop_rect': (50, 100, 950, 700), 'rotation_degrees': 0} B. {'crop_rect': (0, 0, 300, 1000), 'rotation_degrees': 180} C. {'crop_rect': (0, 0, 1000, 1000), 'rotation_degrees': -90} D. {'crop_rect': (150, 200, 850, 600), 'rotation_degrees': 45} Guidelines: First, understand the the code, then carefully observe the images, and solve the problem based on the key elements or details in the images.
EVE Solver Response To determine which set of arguments produces image_1, we need to analyze the visual content of image_1 and compare it with the expected transformations applied by each argument.\n\nRecall that:\n- `crop_rect`: Defines a rectangular region `(left, upper, right, lower)` as a percentage of the original image's width and height (0–1000 scale).\n- `rotation_degrees`: Rotates the cropped image by that many degrees (positive = counter-clockwise, negative = clockwise).\n\nLet's examine image_1:\n\nImage_1 is a vertical, narrow view of the scene. The people are arranged vertically in the frame, and the overall orientation suggests an extreme rotation — specifically, if you rotate the original image 90 degrees clockwise (which is equivalent to -90 degrees), you get a vertical image where the horizontal dimension becomes the vertical one.\n\nLooking at the specific crops: In image_1, we see a tall, narrow column of people on the right side (originally the top of the hill) and the \"photographers\" (visible as silhouettes in the bottom-left area from the original photo) appear inverted relative to their positions in the original image, but more importantly, they are positioned at the top of this rotated rectangle.\n\nComparing this to the options:\n\nA. `{'crop_rect': (50, 100, 950, 700), 'rotation_degrees': 0}` → This would result in the image being cropped to the middle section and then not rotated. Our target, image_1, is clearly rotated, so A is incorrect.\n\nB. `{'crop_rect': (0, 0, 300, 1000), 'rotation_degrees': 180}` → This would crop the leftmost third of the image and rotate it 180 degrees. That would invert the people, which doesn't match image_1. The people are upright, albeit flipped vertically. So B is incorrect.\n\nC. `{'crop_rect': (0, 0, 1000, 1000), 'rotation_degrees': -90}` → This crops the entire image and rotates it -90 degrees (clockwise 90). This means the original sky/foreground becomes the background and the image becomes vertically oriented, with the hillside appearing as a vertical strip on the right and the crowd along the side. Importantly, the photographer in the bottom left appears near the top of the rotated image, but in the correct orientation for a -90 degree rotation. **This matches image_1 perfectly**: the layout is tall, with the original left edge becoming the bottom in a vertical orientation after -90 degree rotation.\n\nD. `{'crop_rect': (150, 200, 850, 600), 'rotation_degrees': 45}` → This crops a central region and rotates it 45 degrees, resulting in a diamond-shaped image or diagonal perspective. Image_1 is vertically aligned and does not show the diagonal aspect seen in option D, so D is incorrect.\n\nTherefore, the only option that correctly describes the appearance of image_1 is C.\n\n\\boxed{C}
Figure 15: An Image-to-Parameter example. Left: the synthesized question with code, candidate parameters, and the target output image. Right: the EVE Solver’s reasoning process that correctly identifies the parameters.
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
image_0 (original)
image_1
image_2
image_3
Conference acronym ’XX, June 03–05, 2026, Woodstock, NY
image_4
The given images are image_0, image_1, image_2, image_3, and image_4, respectively. Images image_1 through image_4 are the results of applying the `edit_image` function to image_0 with different arguments. ```python from PIL import Image def edit_image(img: Image.Image, N: int, order: list) -> Image.Image: width, height = img.size block_w = width // N block_h = height // N adjusted_size = (N * block_w, N * block_h) img = img.resize(adjusted_size) out_img = Image.new("RGB", adjusted_size) for new_idx in range(N * N): new_row = new_idx // N new_col = new_idx % N orig_idx = order[new_idx] orig_row = orig_idx // N orig_col = orig_idx % N left = orig_col * block_w upper = orig_row * block_h right = left + block_w lower = upper + block_h block = img.crop((left, upper, right, lower)) out_img.paste(block, (new_col * block_w, new_row * block_h)) return out_img ``` Question: Which set of arguments, when passed to `edit_image` for image_0, produces image_1? Options: A. {'N': 2, 'order': [1, 0, 3, 2]} B. {'N': 2, 'order': [3, 2, 1, 0]} C. {'N': 2, 'order': [0, 1, 2, 3]} D. {'N': 2, 'order': [2, 3, 0, 1]} Guidelines: First, understand the the code, then carefully observe the images, and solve the problem based on the key elements or details in the images.
EVE Solver Response Let's carefully analyze the problem and the code.\n\nWe are given `image_0`, which is a circle with diameter AB, center O, and a vertical chord CD intersecting AB at M (with a right angle indicated). The code defines a function `edit_image` that takes an image and resizes it to N×N blocks, then rearranges these blocks using a permutation given by `order`.\n\nThe goal is to determine which set of arguments produces `image_1`.\n\nLooking at the images:\n**image_0**: A single circle with horizontal diameter AB, vertical chord CD perpendicular to AB at point M, with right angle shown.\n- **image_1**: Shows the same circle but split into two halves vertically — top half has a semicircle above chord CD? Wait, no. Let me reexamine image_1.\n\nActually, looking closely at image_1: It appears to be the original image split into 4 equal quadrants, then rearranged so that:\n- Top-left quadrant (originally top-left) moves to topright\n- Top-right quadrant (originally top-right) moves to bottom-left\n- Bottom-left quadrant (originally bottom-left) moves to bottom-right\n- Bottom-right quadrant (originally bottom-right) moves to top-left\n\n…Wait, let me consider what the function does.\n\nAlternative approach: visually compare the position of elements.\n\nIn **image_1**, the element that was originally at top-left (quadrant 0) is now at top-right? Or bottom-left?\n\nLooking at image_1: Since it is described as a semicircular composition, and compared to image_0, if we imagine swapping the top two quadrants with bottom two quadrants.\n\nTry `order = [2, 3, 0, 1]`:\n\nThis maps:\n\n- Out[0] ← In[2] (bottom-left)\n- Out[1] ← In[3] (bottom-right)\n- Out[2] ← In[0] (top-left)\n- Out[3] ← In[1] (top-right)\n\nSo output grid:\n\nRow 0: bottom-left | bottomright\n\nRow 1: top-left | top-right\n\nSo we get:\n\nFirst row: bottom half\n\nSecond row: top half\n\nWhich matches the appearance of **image_1** — a top half and bottom half, now arranged with bottom half on top and top half on bottom.\n\nAlternatively, if `order = [3, 2, 1, 0]`:\n\nOut[0] ← In[3] (bottom-right)\n\nOut[1] ← In[2] (bottom-left)\n\nOut[2] ← In[1] (topright)\n\nOut[3] ← In[0] (top-left)\n\nThen:\n\nRow 0: bottom-right | bottom-left\n\nRow 1: topright | top-left\n\nStill bottom then top.\n\nBut in image_1, the top part is shaped like a semicircle above, and bottom part is below — and they appear to have the same size and shape.\n\nLooking back at the options:\n\nA. [1, 0, 3, 2]\n\nB. [3, 2, 1, 0]\n\nC. [0, 1, 2, 3]\n\nD. [2, 3, 0, 1]\n\nOption C gives us the original image, which is image_0, so not image_1.\n\nOption A: [1,0,3,2]\n\nNew positions:\n\n- 0→1 (top-right)\n\n- 1→0 (top-left)\n\n- 2→3 (bottomright)\n\n- 3→2 (bottom-left)\n\nOutput grid:\n\nRow0: top-right | top-left → which is mirrored horizontally\n\nRow1: bottom-right | bottom-left\n\nThis might create a mirror image, but image_1 isn't a mirror; it's reversed vertically in some way.\n\nOption B: [3,2,1,0]\n\n- 0→3 (bottom-right)\n\n- 1→2 (bottom-left)\n\n- 2→1 (top-right)\n\n- 3→0 (top-left)\n\nSo Row0: bottom-right | bottom-left\n\nRow1: top-right | top-left → same as before, just reversed columns.\n\nOption D: [2,3,0,1]\n\n- 0→2 (bottom-left)\n\n- 1→3 (bottom-right)\n\n- 2→0 (top-left)\n\n- 3→1 (top-right)\n\nSo:\n\nRow0: bottom-left | bottom-right\n\nRow1: topleft | top-right\n\nThis puts bottom half on top, top half on bottom — which exactly matches the appearance of **image_1**.\n\nIn image_1, you can see the bottom half of the picture (which should contain points B, D, etc.) is now on the top row, and top half (points A, C) is on bottom row.\n\nTherefore, option D correctly produces image_1.\n\nFinal Answer : \n\\boxed{D}
Figure 16: A Jigsaw Puzzle example (Image-to-Parameter) demonstrating multi-step reasoning and self-correction. Red highlights indicate reasoning errors that the model subsequently identifies and corrects (green highlights), ultimately reaching the correct answer through reflection.