Blending Concepts: Benchmarking Visual Metaphor Generation in Text-to-Image Models

Blending Concepts: Benchmarking Visual Metaphor Generation in Text-to-Image Models

Chuer Chen1

arXiv:2609.02502v1 [cs.CV] 2 Sep 2026

1

Zichen Wang1

Tongji University

Yi He1 2

Zhengxi Yu1

Nan Cao1,2∗

Shanghai Innovation Institute

Abstract Text-to-image (T2I) models have achieved remarkable success at faithfully rendering specified objects and attributes, yet their ability to produce visual metaphors—images that convey abstract ideas by combining elements from two distinct domains—remains largely unexamined. To bridge this gap, we introduce VMetaphor-Bench, the first benchmark for evaluating visual metaphor generation in T2I models. It comprises 1,500 visual metaphors curated from real-world creative imagery, organized into three levels and ten categories, with each sample paired with two prompts of differing specificity. For evaluation, we develop a hybrid framework within an MLLM-as-judge paradigm, combining a multiplechoice question (MCQ) based protocol of 9,594 questions across four levels of metaphorical fidelity with a dimension-based scoring protocol along three perceptual dimensions. Extensive evaluation of 11 representative T2I models reveals that even the strongest proprietary models struggle with compositional structuring and cross-domain mapping, key aspects of metaphorical expression, highlighting visual metaphor generation as an important frontier for future T2I research.

1

Introduction

Recent years have witnessed remarkable progress in text-to-image (T2I) generation, driven by advances across three main architectural directions: diffusion models [20, 35], autoregressive models [40, 43], and unified multimodal architectures [8, 9, 48]. Powered by large-scale training, a wide range of T2I systems, from leading proprietary models such as GPT-Image-1 [30] and Gemini 3.1 Flash Image [16] to open-source counterparts [27, 47], now produce photorealistic images that faithfully render specified objects, attributes, and spatial layouts, achieving strong performance on basic semantic prompt following. However, their ability to express abstract concepts and convey meaning beyond what is literally depicted remains far less explored. A particularly important form of such abstract expression is the visual metaphor, where one concept is understood through another via visual analogy [14]. Drawing on conceptual blending theory [13], a visual metaphor combines elements from two distinct domains — a source that lends its properties, and a target that receives them — to produce a single image carrying meaning richer than either domain alone. Generating such images is far from straightforward: the model must (i) blend the two given domains coherently using a specific compositional strategy (e.g., fusion or juxtaposition), (ii) establish meaningful element-level mappings between the two, and (iii) ensure that the resulting image conveys the intended meaning to viewers. As shown in Figure 1, even state-of-the-art T2I models frequently struggle with these tasks, producing images that miss one of the domains, adopt the wrong structural form, blend the two awkwardly, or fail to convey the intended meaning. Yet this challenge has been largely overlooked by existing T2I benchmarks, which remain dominated by literal prompt following. Mainstream benchmarks such as GenEval [15] and T2I-CompBench [22] ∗ Corresponding author.

Preprint.

Missing Domain

Wrong Structure

A seamless hybrid of an eggplant and a collection of books suggests that a single raw ingredient holds an entire library of...

Poor Integration

Deep palm creases lie in place of massive paved roads to create a vast transportation network for tiny vehicles...

Unclear Meaning

A vodka bottle and scissor hybrid cuts through a family photograph, showing how alcoholism severs domestic bonds.

Coherent Metaphor

Human hands grip and twist a sea turtle in place of a disposable plastic bottle, ... singleuse plastics directly harms marine life.

A broken chain that transforms seamlessly into an ascending flock of birds, showing that the shattering of systemic oppression...

Figure 1: Common failure modes in visual metaphor generation from GPT-Image-1.5 [30], Seedream 5.0 Lite [4], and FLUX.2-dev [28].

evaluate object, attribute, and basic spatial fidelity from short prompts, while more recent reasoningaware benchmarks [29, 37, 55] extend toward factual or physical inference, leaving the cross-domain conceptual blending required by visual metaphors unexamined. On the metaphor side, existing efforts have focused predominantly on the understanding direction [2, 26, 52]; the only generation evaluation we are aware of, conducted as a small probe within MetaCLUE [2], is limited to 300 samples assessed via FID, CLIP similarity, and a small user study — none of which can systematically diagnose where and how T2I models fail at metaphorical expression. Together, these limitations leave a clear gap: no existing benchmark systematically evaluates how well T2I models generate visual metaphors across diverse themes and metaphorical structures. To address this gap, we introduce VMetaphor-Bench, a new benchmark designed for evaluating visual metaphor generation in T2I models. It comprises 1,500 visual metaphors curated from realworld creative imagery, organized into three levels and ten categories, as illustrated in Figure 2. Each sample is annotated with its source and target domains, structure type, metaphorical meaning, element-level mappings, and paired with two prompts of differing specificity: a conceptual prompt that conveys only the metaphorical idea, and a descriptive prompt that details its visual realization. For evaluation, we develop a hybrid framework consisting of an MCQ-based protocol — 9,594 questions targeting four levels of metaphorical fidelity (domain presence, structure type, meaning, and element mapping) — and a dimension-based scoring protocol rating each image on metaphoric efficacy, metaphor logic, and perceptual harmony. Both protocols are realized within an MLLM-as-judge paradigm, enabling fine-grained, interpretable diagnosis of T2I models’ strengths and weaknesses in visual metaphor generation. Using VMetaphor-Bench, we conduct a comprehensive evaluation of 11 representative T2I models, spanning leading proprietary systems, open-source diffusion models, and unified multimodal architectures. Our results reveal that proprietary models such as GPT Image 1.5 and Nano Banana 2 achieve substantially better performance than open-source counterparts. However, even the strongest models struggle with compositional structuring and cross-domain mapping—key aspects of metaphorical expression—highlighting visual metaphor generation as an important frontier for future T2I research.

e

tiv ea gn Cr esi D

Cog Pro nitive

ces ses

3 Levels & 10 Thematic Categories

Cr ea Le tive vel

ual

ivid Ind evel L

Te ch no Lif logic e al

MCQ Overall Accuracy

75% 65%

l nta Me es t Sta

Healthy Living Per Dev son elo al

pm ent

55%

Individual

External

Creative

2 el ev ge age -Pro age -dev Lite a 2 1.5 en ag -d e n ar niG B UX.1 .5 L Z-Im nus n-Im X.2 5.0 ana mag U 3 B I Ja we FL SD FL ream no T Q a d GP e N Se

Om

Dimension Overall Score

3 Types of Visual Metaphor Structure Fusion

Replacement

Juxtaposition

4.0 3.5

l cia So ues Iss

l n cia tio So rac e t In

External

Level Natu Envir ral onme nt

tising Adver

VMetaphor -Bench

Performance 85%

3.0 2.5

A hybrid of a businessman's head and a slot machine

Miniature bed pillows spill out of an overturned medicine bottle

An image of a car body suspended directly above a group of horses

2 el ev ge age -Pro age -dev Lite a 2 1.5 en ag -d e n ar niG B UX.1 .5 L Z-Im nus n-Im X.2 5.0 ana mag U 3 B I Ja we FL SD FL ream no T Q GP ed Na Se

Om

Figure 2: Overview of VMetaphor-Bench. Left: hierarchical taxonomy with three levels and ten thematic categories. Middle: representative examples spanning the three levels and the three structure types of visual metaphors. Right: performance of 11 evaluated T2I models on MCQ accuracy and dimension scoring under conceptual prompts. 2

In summary, our contributions are threefold: • We introduce VMetaphor-Bench, the first benchmark (to the best of our knowledge) dedicated to evaluating visual metaphor generation in T2I models, comprising 1,500 high-quality, carefully curated samples organized into a hierarchical taxonomy of three levels and ten categories. • We design a hybrid evaluation framework with an MCQ-based protocol of 9,594 questions across four levels of metaphorical fidelity and a dimension-based protocol along three perceptual dimensions, enabling fine-grained, interpretable assessment. • We conduct a comprehensive evaluation of 11 representative T2I models, providing insights into their capabilities in visual metaphor generation and highlighting key directions for future research.

2

Related Works

2.1

Text-to-Image Generation Models

Text-to-image (T2I) generation has advanced rapidly along three main architectural directions. Diffusion models [10, 20, 35] remain the dominant paradigm, with recent progress driven by scaling transformer-based denoisers [12, 27, 32]. Autoregressive models [38, 40, 43] cast image generation as sequential token prediction, offering fine-grained compositional control through explicit modeling of inter-token dependencies. More recently, unified multimodal models [8, 9, 48, 49] integrate visual understanding and generation within a single architecture, leveraging multimodal reasoning to better interpret complex prompts. State-of-the-art systems such as GPT-Image-1 [30] and Gemini 3.1 Flash Image [16] now produce images of remarkable fidelity and aesthetic quality for everyday scenes. However, generating images that convey abstract semantic relationships, such as visual metaphors, remains a significant challenge across all three paradigms. 2.2

Text-to-Image Generation Benchmarks

Early evaluation of text-to-image generation relied on distribution-level metrics such as FID [19] and feature-based similarity scores like CLIPScore [18], which capture perceptual quality but fail to assess fine-grained semantic correctness. The recent rise of multimodal large language models (MLLMs) [23, 41] has enabled a paradigm shift toward MLLM-as-judge evaluation [6], where a vision-language model directly examines the generated image against the prompt, providing a more flexible and semantically grounded assessment framework. Under this paradigm, a line of benchmarks targets foundational semantic alignment — verifying whether generated images faithfully depict the specified objects [24], attributes [51], and spatial relationships [45, 46]. Representative works include GenEval [15], T2I-CompBench [22], and TIFA [21], which decompose prompts into atomic verification questions to systematically evaluate compositional fidelity. More recently, benchmarks have shifted toward higher-order reasoning capabilities, probing whether T2I models can incorporate world knowledge [29, 53] and reasoning [7, 37, 55] into their generations, reflecting a broader trend from evaluating object-level fidelity toward scene-level semantic understanding. 2.3

Visual Metaphor

Visual metaphor communicates abstract ideas by mapping one conceptual domain onto another through visual imagery [11, 14, 44]. On the understanding side, MetaCLUE [2] introduces a comprehensive benchmark for metaphor classification, localization, interpretation, along with a small probe on generation; ImageMet [26] benchmarks VLMs on metaphor captioning and VQA, and MetaphorStar [52] applies visual reinforcement learning to improve metaphor reasoning. On the generation side, several recent methods leverage text-to-image models as the visual backbone: Creative Blends [39] fuses disparate visual concepts via commonsense knowledge, The Mind’s Eye [25] proposes a source-target-meaning decomposition with self-evaluation rewards, and Beyond Pixels [50] introduces schema-driven reasoning for metaphor transfer. However, evaluation of these methods has been limited—typically conducted on small-scale ad hoc test sets via FID, CLIP similarity, or user studies, none of which can diagnose where and how T2I models fail at metaphorical expression. While benchmarks exist for metaphor understanding, no counterpart has been established for systematically evaluating the generation quality of visual metaphors. 3

3

VMetaphor-Bench

We present VMetaphor-Bench, the first benchmark, to the best of our knowledge, for evaluating visual metaphor generation in text-to-image models, featuring multi-dimensional assessments of cross-domain conceptual blending. We describe the dataset construction process in Section 3.1 and detail the evaluation framework in Section 3.2. An overview of the complete pipeline is illustrated in Figure 3. Dataset Construction Image Collection

Ev

Annotation

aluation Framework

QAs Generation

Evaluation Dimension

Scoring

Web Images

Crawling

Manual

Filtering

Visual Metaphor

Images

Gemini-3.1-pro

Manual

Verification

Prompts

& Annotation

Gemini-3.1-pro

Structured Annotations - Target domain: Earth - Source domain: Bomb

- Structure type: Fusion

- Meaning: The world is currently in a highly volatile and dangerous state, akin to a ticking bomb, requiring urgent intervention to cut off the impending threat of global destruction and achieve peace.

- Mappings: [

{Spherical bomb casing → Earth's globe},

{Black grid structure → Latitude and longitude lines}

] Prompts Domain:

Social Issues - Descriptive prompt: A central circular shape combines the visual characteristics of a terrestrial globe and a classic round explosive device......

- Conceptual prompt: An Earth globe merged with a ticking bomb, conveying that the world is currently in a highly volatile state......

Manual Multiple-Choice

Verification QA pairs

Generated

Images

MLLM

Evaluator

MCQ

Evaluation

Multiple-Choice QA pairs

Original

Image

enerated

Image

G

- L1 – Domain Presence:

Q1: What is the target domain of this visual metaphor?

A. peace

B. scissors

C. earth

D. cannot be determined

Q2: What is the source domain of this visual metaphor? ....

- L2 – Structure Type:

Q3: What visual strategy is used to combine the metaphorical elements?

A. replacement B. fusion C. juxtaposition D. cannot be determined

- L3 – Meaning:

Q4: What meaning does this image most likely convey?

icacy: 4 ogic: 5 Harmony: 4 Eff L

A. The world is currently in a highly volatile and dangerous state...

B. The world's resources are being rapidly depleted...

- L4 – Element Mappings:

Q5-1: What does spherical bomb casing'visually correspond to in this image?

A. earth's atmosphere B. earth's globe C. continents D. not clear

Q5-2: What does black grid structure visually correspond to in this image?...

Figure 3: Overview of the VMetaphor-Bench pipeline. Left: Dataset Construction — images are collected and annotated through a two-stage pipeline, producing structured annotations and two granularity levels of prompts. Right: Evaluation Framework — generated images are assessed via MCQ-based evaluation (L1–L4 levels) and dimension-based scoring (Efficacy, Logic, Harmony). 3.1

Dataset Construction

Image Collection. To construct a benchmark grounded in real-world visual metaphors, we collect images from Pinterest [1], a platform rich in creative visual content such as advertisements, editorial illustrations, and conceptual art. We query the platform using over ten keywords spanning both general and domain-specific terms, e.g. visual metaphor, creative ads. This process yields approximately 5,000 candidate images. Two authors then review the entire collection, filtering out duplicates, images that do not contain a clear visual metaphor, and low-quality or ambiguous samples, resulting in approximately 1,600 images for subsequent annotation. Data Annotation. After collecting images, we employ Gemini 3.1 Pro [17] as the annotation backbone for its strong visual reasoning capabilities. Since visual metaphors involve multi-layered semantics [5], generating accurate annotations and high-quality prompts simultaneously in a single pass is challenging. We therefore adopt a two-stage annotation pipeline. In the first stage, the model is presented with each image and tasked with extracting structured annotations: • Target domain: the subject being described — what the image is ultimately “about”. • Source domain: the concept used to characterize the target— it lends a property to the target through visual analogy. • Structure type: how the two domains are visually combined. Following Phillips et al. [33], we categorize it as replacement (one domain is absent and inferred from context), fusion (both domains are merged into a single object), or juxtaposition (both domains appear as separate objects in the same scene). • Meaning: the overall message conveyed through the interaction of the two domains. • Element mappings: fine-grained correspondences specifying which source element maps to which target element. In the second stage, the model receives both the image and the extracted annotations, and generates two prompts at different levels of granularity: (1) The descriptive prompt provides a detailed depiction of the visual content and how the metaphor is physically rendered in the image; (2) The conceptual 4

prompt specifies the target and source domains, implicitly conveys the structure type, and states the intended meaning, without describing specific visual details or element-level mappings. Additionally, a style suffix describing the visual style of the original image (e.g., photorealistic, illustration) is extracted and appended to both prompts during generation to ensure stylistic consistency. This two-stage design allows the structured annotations to guide prompt generation, ensuring consistency between the structured annotations and the textual descriptions. Three annotators, after training on visual metaphor theory and annotation guidelines, perform rigorous manual inspection on all images, annotations, and prompts, and cross-validate each other’s decisions. Images where the metaphor is ambiguous or unrecognizable are removed. Inaccurate annotations (e.g., misidentified target or source domains) and imprecise prompts are manually corrected to align with the visual content. This verification process yields our final set of 1,500 high-quality images, each accompanied by structured annotations and two granularity levels of prompts. 3.2

Evaluation Framework

Evaluating visual metaphor generation requires assessing both metaphorical fidelity — whether the generated image faithfully captures the intended cross-domain mapping — and expressive quality — how effectively and aesthetically the metaphor is conveyed. To this end, we adopt a hybrid evaluation framework with MLLM-as-judge [54], comprising two complementary approaches: (1) MCQ-based evaluation that measures the correctness of metaphorical elements through multiple-choice questions, and (2) dimension-based scoring that rates the overall quality of the generated metaphor on a 1-5 Likert scale. An overview of the evaluation framework is shown in Figure 3 (Right). MCQ-based Evaluation. To objectively assess the correctness of generated images at each level of the metaphorical structure, we automatically construct a set of multiple-choice questions (MCQs) for all 1,500 samples in the dataset. Specifically, we leverage Gemini 3.1 Pro to generate MCQs based on each sample’s structured annotations along with its descriptive prompt. The generated MCQs assess four hierarchical levels of metaphorical fidelity: • L1 – Domain Presence: Whether the target and source domains are visually recognizable in the image (one question per domain). • L2 – Structure Type: Whether the visual combination strategy matches the ground-truth structure type (replacement, fusion, or juxtaposition). • L3 – Meaning: Whether the image conveys the intended metaphorical meaning. • L4 – Element Mapping: Whether each source element correctly corresponds to its intended target element (one question per mapping pair; scores are averaged). This process yields a total of 9,594 MCQs across all samples, which are manually verified by the same annotators to ensure correctness. Each question consists of four options: one ground-truth answer derived from the annotation, two plausible but incorrect distractors, and a "cannot be determined" option. All option orders are randomized to eliminate positional bias. For each image, we compute per-level accuracy: L1 averages over the two domain questions, L2 is a single binary score, L3 is a single binary score, and L4 averages over all mapping questions. The overall MCQ score is the mean across all four levels. Dimension-based Scoring. While MCQ-based evaluation measures the correctness of metaphorical elements, it does not capture the overall perceptual quality of the generated image. We therefore complement it with dimension-based scoring, where an MLLM evaluator is presented with the generated image along with its structured annotations and asked to rate the image on a 1–5 Likert scale across three independent dimensions: • Metaphoric Efficacy: How clearly the image communicates the intended metaphorical meaning to a viewer without any textual explanation. • Metaphor Logic: Whether the cross-domain mapping between source and target is conceptually coherent and logically sound. • Perceptual Harmony: How naturally and coherently the two domains are visually presented, considering blending, composition, and overall visual cohesion. The overall dimension score is the average across all three dimensions. Together with the MCQ-based evaluation, this hybrid framework provides a comprehensive assessment of both the metaphorical 5

Table 1: Evaluation results on VMetaphor-Bench with conceptual prompts. MCQ Accuracy (%) Model Domain Structure Closed-source T2I Models Seedream 5.0 Lite 90.6 65.6 Nano Banana 2 93.0 76.3 GPT Image 1.5 93.6 78.1 Open-source T2I Models FLUX.1-dev 82.3 54.9 SD 3.5 Large 83.0 60.1 Z-Image 84.3 59.6 Qwen-Image 87.3 66.1 FLUX.2-dev 88.6 63.7 Open-source Unified MLLMs OmniGen2 74.8 48.1 Bagel 79.8 54.8 Janus-Pro 83.4 59.1

Dimension Score (1–5)

Meaning

Mapping

Overall

Efficacy

Logic

Harmony

Overall

96.0 98.5 97.6

66.5 75.9 76.5

78.5 84.8 85.4

3.78 4.05 4.27

3.74 4.08 4.23

4.08 4.21 4.38

3.86 4.11 4.30

85.6 85.7 89.5 90.5 92.1

55.2 58.9 60.8 61.7 63.9

68.4 70.8 72.4 74.9 76.0

2.90 2.99 2.93 3.39 3.48

2.87 3.01 3.03 3.31 3.46

3.82 3.75 3.67 3.84 3.93

3.19 3.25 3.21 3.52 3.62

77.1 84.2 89.1

49.1 54.2 63.2

61.4 67.0 73.0

2.53 2.69 2.90

2.44 2.72 3.02

3.45 3.56 3.43

2.74 2.99 3.12

fidelity and expressive quality of generated visual metaphors. Full evaluation prompts are provided in Appendix A.7.

4

Experiments

In this section, we evaluate representative T2I models on VMetaphor-Bench and analyze their behavior under different prompt granularities. We further validate the reliability of our MLLMas-judge framework through cross-judge consistency and human alignment studies. Additional experimental details and results are provided in the Appendix. 4.1

Setup

Models. We evaluate 11 representative text-to-image models spanning three categories: (1) Closedsource T2I Models. GPT Image 1.5 [30], Nano Banana 2 (Gemini 3.1 Flash Image) [16], and Seedream 5.0 Lite [4]. These proprietary models represent the current state of the art in text-toimage generation; (2) Open-source T2I Models. Stable Diffusion 3.5 Large [36], FLUX.1-dev [27], FLUX.2-dev [28], Z-Image [42], and Qwen-Image [47]. These models represent the leading opensource text-to-image models; (3) Open-source Unified MLLMs. OmniGen2 [48], Bagel [9], and Janus-Pro [8]. These models employ a unified architecture for both image generation and multimodal understanding. All models are evaluated using their default generation configurations. MLLM Evaluator. For both MCQ-based evaluation and dimension-based scoring, we adopt an opensource MLLM, Qwen3.5-27B [34], as the default judge model for its strong visual reasoning capability and full reproducibility. We additionally report results with GPT-5.4 [31] in the Appendix A.5 to validate cross-judge consistency. 4.2

Main Results on VMetaphor-Bench

We report the main evaluation results with conceptual prompts in Table 1. All metrics are assigned by Qwen3.5-27B as the default judge. Based on the results across 11 models, we highlight several key findings regarding model capabilities and core challenges in visual metaphor generation. Closed-source models lead by a substantial margin. GPT Image 1.5 achieves the best overall performance with 85.4% MCQ accuracy and a dimension score of 4.30, followed by Nano Banana 2 (84.8%, 4.11). In contrast, the best-performing open-source T2I model, FLUX.2-dev, trails by 9.4 percentage points (76.0%, 3.62). While FLUX.2-dev narrows the gap with the weakest closed-source model Seedream 5.0 Lite (78.5%), a clear performance tier remains, suggesting that visual metaphor generation—which demands both semantic comprehension and creative compositional ability—still substantially benefits from the larger-scale training and architectural advantages of proprietary systems. Unified MLLMs show mixed performance: while Janus-Pro (73.0%) is competitive with 6

GPT Image 1.5

Nano Banana 2

FLUX.2-dev

Z-Image

SD 3.5 Large

Janus-Pro

OmniGen2

Conceptual Prompt:

A hybrid of a child's backpack and an uprooted house, showing how a child carries the heavy, dragging weight of a displaced or troubled home life everywhere they go

Conceptual Prompt:

A takeaway coffee cup formed from a stack of soft pillows, expressing that the convenience of coffee on the go grants the luxury of extra sleep

Conceptual Prompt:

Instead of typical stuffed prizes, an arcade claw machine is filled with business professionals to portray the corporate employee selection process as an arbitrary, dehumanizing game of chance driven by a detached authority

Conceptual Prompt:

A hand holding a cigarette casts a dark shadow shaped like a handgun pointed back at the smoker, showing that smoking is a lethal and self-destructive act

Figure 4: Qualitative comparison of generated images across seven representative models on four visual metaphors. Each row corresponds to one metaphor, with the conceptual prompt shown on the left. The four metaphors cover three structure types: fusion (rows 1–2), replacement (row 3), and juxtaposition (row 4). Blue and purple text highlight the target and source domains, respectively. mid-tier open-source T2I models, OmniGen2 (61.4%) and Bagel (67.0%) rank at the bottom despite their multimodal understanding capabilities. Structure and Mapping are the primary bottleneck. All models exhibit a consistent difficulty hierarchy across MCQ levels: Domain and Meaning accuracies are consistently the two highest among the four levels for every model, indicating that models can generate images containing the correct conceptual elements and conveying the intended metaphorical meaning. However, performance drops sharply on Structure (48.1–78.1%) and Mapping (49.1–76.5%), revealing that the core challenge lies in compositional structuring—models can depict the right elements but struggle to compose them into coherent metaphorical structures. This pattern is particularly evident in Seedream 5.0 Lite, which achieves 90.6% on Domain and 96.0% on Meaning yet drops to 65.6% on Structure and 66.5% on Mapping. Notably, a deeper analysis of structure type errors reveals that replacement is far more challenging than fusion or juxtaposition, a pattern that mirrors the cognitive complexity hierarchy in visual rhetoric theory [33] (see Appendix A.3). Visually harmonious but metaphorically shallow. Across all models, Perceptual Harmony is consistently the highest-scoring dimension (3.43–4.38), while Metaphoric Efficacy and Metaphor Logic generally trail behind. This suggests that although the intended meaning can be inferred (as reflected by high meaning accuracy), the metaphor is not conveyed with sufficient clarity and immediacy through visual structure—models produce aesthetically pleasing images without powerful metaphorical expression. The gap between Harmony and Efficacy widens for weaker models: OmniGen2 scores 3.45 on Harmony but only 2.53 on Efficacy (gap = 0.92), whereas GPT Image 1.5 maintains a gap of only 0.11 (4.38 vs. 4.27). 4.3

Qualitative Analysis

To complement the quantitative results above, Figure 4 presents representative outputs from seven models on four visual metaphors covering different structure types (fusion, replacement, and juxtaposition). Several patterns emerge that are consistent with our quantitative findings. Closed-source models such as GPT Image 1.5 and Nano Banana 2 generally produce coherent compositions that integrate the two domains in meaningful ways. In contrast, weaker open-source models often struggle to compose the two domains correctly. Mid-tier models such as FLUX.2-dev and Z-Image fall in between, occasionally capturing the metaphorical concept but producing visually disharmonious compositions or failing on structurally demanding cases. 7

Table 2: Performance comparison between conceptual and descriptive prompts on representative models. “Conc.” and “Desc.” denote the conceptual and descriptive prompt settings, respectively. ∆ denotes the change when switching from conceptual to descriptive prompts. MCQ Overall (%)

Dimension Overall

Model

Conc.

Desc.

∆

Conc.

Desc.

∆

GPT Image 1.5 Nano Banana 2 FLUX.2-dev Z-Image OmniGen2

85.4 84.8 76.0 72.4 61.4

88.5 89.2 87.3 85.9 78.0

+3.1 +4.4 +11.3 +13.5 +16.6

4.30 4.11 3.62 3.21 2.74

4.24 4.18 4.09 3.90 3.35

−0.06 +0.07 +0.47 +0.69 +0.61

A closer look at typical failure cases reveals three recurring issues that map directly to our quantitative findings. The most common is incorrect structural composition: models depict both domains but fail to combine them in the structural form implied by the prompt. For example, in row 4, where the prompt suggests a juxtaposition of a cigarette and a handgun-shaped shadow, SD 3.5 Large, Z-Image, and Janus-Pro instead fuse the two into a single physical object—echoing the sharp performance drop on L2 Structure across all models. A second issue is poor visual integration, where the two domains are placed together but blend awkwardly, such as a coffee cup stacked on visibly distinct pillows without any unified form (row 2). Although the resulting images can still appear visually polished, the awkward integration weakens the metaphor’s expressive force, which is consistent with our finding that Metaphoric Efficacy lags behind Perceptual Harmony across most models. In the most severe cases, models omit one of the domains entirely: Z-Image, for instance, fills the claw machine with ordinary toy balls rather than business professionals (row 3), making the metaphor impossible to recover. Together, these failure modes underscore the value of a multi-level evaluation framework for diagnosing the specific weaknesses of current T2I models. 4.4

Effect of Prompt Granularity

In addition to the conceptual prompts reported in our main experiments, we also generate and evaluate images using the descriptive prompts paired with each sample. Full results under descriptive prompts are provided in the Appendix A.4. Here we analyze the differences between the two prompt settings on a representative subset of five models spanning the three model categories, summarized in Table 2. We first observe that all models benefit from descriptive prompts on MCQ accuracy, and most models also see improvements on dimension scores, indicating that explicit visual specification reduces the difficulty of metaphor generation. However, the magnitude of improvement varies dramatically across models. GPT Image 1.5 gains only +3.1 percentage points on MCQ overall and shows essentially no change on dimension score (−0.06), whereas Z-Image improves by +16.6 points and +0.69, respectively. A clear inverse relationship emerges between a model’s baseline capability and the gain it receives: stronger models exhibit smaller gains, while weaker models benefit substantially more. This pattern suggests that capable models such as GPT Image 1.5 and Nano Banana 2 already possess the ability to reason about how a metaphorical idea should be visually composed without explicit specification. In contrast, weaker models lack this inferential capacity and rely heavily on the explicit visual guidance provided by descriptive prompts. This aligns with our earlier observation that visual metaphor generation requires more than literal rendering: it demands a degree of conceptual reasoning, on which current open-source models still fall short. Beyond the per-model gains, descriptive prompts also narrow the capability gap between models. Under conceptual prompts, the dimension score gap between GPT Image 1.5 (4.30) and Z-Image (3.21) is 1.09; under descriptive prompts, this gap narrows to 0.34 (4.24 vs. 3.90). For practitioners working with open-source models, providing fine-grained visual descriptions is therefore an effective strategy to compensate for the limited metaphorical reasoning capability. 4.5

Reliability of the Evaluator

To validate the reliability of using Qwen3.5-27B as our default evaluator, we conduct two studies: a cross-judge consistency study assessing robustness to the choice of evaluator, and a human alignment study measuring agreement with human judgment. 8

Table 3: Cross-judge comparison. Model rankings by MCQ Overall and Dimension Overall under two judges. MCQ Model GPT Image 1.5 Nano Banana 2 Seedream 5.0 Lite FLUX.2-dev Qwen-Image

Dimension

Qwen3.5

GPT-5.4

Qwen3.5

GPT-5.4

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

Table 4: Alignment between MLLM evaluators (Qwen3.5-27B and GPT-5.4) and human annotators on 200 sampled images. (a) MCQ alignment by balanced accuracy (%); (b) dimension alignment by mean absolute error (MAE) on a 1–5 scale. (a) MCQ alignment (Bal.Acc, %)

(b) Dimension alignment (MAE)

Judge

Domain

Structure

Meaning

Mapping

Overall

Judge

Qwen3.5-27B GPT-5.4

87.61 88.79

71.25 73.89

83.97 84.50

76.45 76.51

80.24 81.16

Qwen3.5-27B GPT-5.4

Efficacy

Logic

Harmony

0.65 0.71

0.71 0.89

0.64 0.75

Cross-Judge Consistency. To test whether our conclusions depend on the specific choice of evaluator, we re-run the full evaluation pipeline with GPT-5.4 [31] as an alternative judge and compare model rankings and scores against those produced by Qwen3.5-27B. As shown in Table 3, the top-5 models receive identical rankings from both judges across both metrics. Over all 11 models, Spearman’s ρ reaches 0.991 for MCQ and 0.936 for dimension scores, confirming that our findings are not dependent on the choice of evaluator. Human Alignment Study. We further evaluate how well the two MLLM judges align with human annotators. We randomly sample 100 images each from FLUX.2-dev and Qwen-Image, yielding 200 images in total. Five human annotators, trained on visual metaphor theory and our annotation guidelines, independently evaluate each image using the same protocol applied to the MLLM evaluator. We measure alignment using balanced accuracy [3] for MCQ evaluation and Mean Absolute Error (MAE) for dimension-based scoring, following established practice in recent benchmark studies [46, 55]. Results are reported in Table 4. The MCQ alignment is consistently high across all four levels, with both Qwen3.5-27B and GPT5.4 achieving over 80% balanced accuracy overall. Alignment is highest on the more concrete L1 (Domain) and L3 (Meaning) levels (above 83%), and lower on L2 (Structure) and L4 (Mapping), which require finer-grained judgments about how source and target are visually combined. For dimension-based scoring, Qwen3.5-27B yields a low MAE across all three dimensions (below 0.8 on a 1–5 scale), indicating that its ratings are close to those of human annotators. GPT-5.4 shows slightly larger errors (up to 0.89 on Logic) but remains within a reasonable range (< 1.0). The fact that an open-source evaluator achieves alignment comparable to or better than a leading proprietary model further supports the choice of Qwen3.5-27B as our default judge.

5

Conclusion

In this paper, we introduce VMetaphor-Bench, the first benchmark for evaluating visual metaphor generation in text-to-image models. Built upon 1,500 carefully curated visual metaphors with structured annotations, our benchmark provides a hierarchical taxonomy of three levels and ten categories and a hybrid evaluation framework that combines MCQ-based fidelity assessment with dimensionbased scoring. Extensive evaluation of 11 representative T2I models reveals that proprietary models substantially outperform open-source counterparts, yet even the strongest models continue to struggle with compositional structuring and cross-domain mapping—key aspects of metaphorical expression. We hope VMetaphor-Bench serves as a foundation for future research on bridging the gap between literal rendering and creative, meaning-rich visual generation. 9

References [1] Pinterest. https://www.pinterest.com. [2] Arjun R Akula, Brendan Driscoll, Pradyumna Narayana, Soravit Changpinyo, Zhiwei Jia, Suyash Damle, Garima Pruthi, Sugato Basu, Leonidas Guibas, William T Freeman, et al. Metaclue: Towards comprehensive visual metaphors research. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 23201–23211, 2023. [3] Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann. The balanced accuracy and its posterior distribution. In 2010 20th international conference on pattern recognition, pages 3121–3124. IEEE, 2010. [4] ByteDance Seed. Seedream 5.0 lite. https://seed.bytedance.com/en/seedream5_0_ lite, 2026. [5] Silvia Cappa, Anna Sofia Lippolis, and Stefano Zoia. Meanings are like onions: A layered approach to metaphor processing. arXiv preprint arXiv:2507.10354, 2025. [6] Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. Mllm-as-a-judge: Assessing multimodal llm-asa-judge with vision-language benchmark. In Forty-first International Conference on Machine Learning, 2024. [7] Kaijie Chen, Zihao Lin, Zhiyang Xu, Ying Shen, Yuguang Yao, Joy Rimchala, Jiaxin Zhang, and Lifu Huang. R2i-bench: Benchmarking reasoning-driven text-to-image generation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 12606–12641, 2025. [8] Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811, 2025. [9] Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683, 2025. [10] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021. [11] Francesca Ervas. Metaphor, ignorance and the sentiment of (ir) rationality. 198(7):6789–6813, 2021.

Synthese,

[12] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024. [13] Gilles Fauconnier and Mark Turner. Conceptual blending, form and meaning. Recherches en communication, 19:57–86, 2003. [14] Charles Forceville et al. Metaphor in pictures and multimodal representations. The Cambridge handbook of metaphor and thought, pages 462–482, 2008. [15] Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36:52132–52152, 2023. [16] Google. Gemini 3.1 Flash Image Preview. https://ai.google.dev/gemini-api/docs/ models/gemini-3.1-flash-image-preview, 2026. [17] Google DeepMind. Gemini 3.1 pro. https://deepmind.google/models/gemini/pro/, 2026. 10

[18] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 7514–7528, 2021. [19] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. [20] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. [21] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20406–20417, 2023. [22] Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723–78747, 2023. [23] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. [24] Amita Kamath, Kai-Wei Chang, Ranjay Krishna, Luke Zettlemoyer, Yushi Hu, and Marjan Ghazvininejad. Geneval 2: Addressing benchmark drift in text-to-image evaluation. arXiv preprint arXiv:2512.16853, 2025. [25] Girish A Koushik, Fatemeh Nazarieh, Katherine Birch, Shenbin Qian, and Diptesh Kanojia. The mind’s eye: A multi-faceted reward framework for guiding visual metaphor generation. arXiv preprint arXiv:2508.18569, 2025. [26] Manishit Kundu, Sumit Shekhar, and Pushpak Bhattacharyya. Looking beyond the pixels: Evaluating visual metaphor understanding in vlms. Findings of the Association for Computational Linguistics: EMNLP, pages 23137–23158, 2025. [27] Black Forest Labs. Flux. https://github.com/black-forest-labs/flux, 2024. [28] Black Forest Labs. FLUX.2: Frontier Visual Intelligence. https://bfl.ai/blog/flux-2, 2025. [29] Yuwei Niu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Kunpeng Ning, Bin Zhu, et al. Wise: A world knowledge-informed semantic evaluation for text-to-image generation. arXiv preprint arXiv:2503.07265, 2025. [30] OpenAI. GPT-Image-1.5. gpt-image-1.5, 2025.

https://developers.openai.com/api/docs/models/

[31] OpenAI. Introducing gpt-5.4. introducing-gpt-5-4/, 2025.

https://openai.com/zh-Hans-CN/index/

[32] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023. [33] Barbara J Phillips and Edward F McQuarrie. Beyond visual metaphor: A new typology of visual rhetoric in advertising. Marketing theory, 4(1-2):113–136, 2004. [34] Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. [35] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. [36] Stability AI. Stable diffusion 3.5 large. stable-diffusion-3.5-large, 2024. 11

https://huggingface.co/stabilityai/

[37] Kaiyue Sun, Rongyao Fang, Chengqi Duan, Xian Liu, and Xihui Liu. T2i-reasonbench: Benchmarking reasoning-informed text-to-image generation. arXiv preprint arXiv:2508.17472, 2025. [38] Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. [39] Zhida Sun, Zhenyao Zhang, Yue Zhang, Min Lu, Dani Lischinski, Daniel Cohen-Or, and Hui Huang. Creative blends of visual concepts. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–17, 2025. [40] Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818, 2024. [41] Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. [42] Z-Image Team. Z-image: An efficient image generation foundation model with single-stream diffusion transformer. arXiv preprint arXiv:2511.22699, 2025. [43] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in neural information processing systems, 37:84839–84865, 2024. [44] Margot Van Mulken, Andreu Van Hooft, and Ulrike Nederstigt. Finding the tipping point: Visual metaphor and conceptual complexity in advertising. Journal of Advertising, 43(4):333–343, 2014. [45] Zehan Wang, Jiayang Xu, Ziang Zhang, Tianyu Pang, Chao Du, Hengshuang Zhao, and Zhou Zhao. Genspace: Benchmarking spatially-aware image generation. arXiv preprint arXiv:2505.24870, 2025. [46] Zengbin Wang, Xuecai Hu, Yong Wang, Feng Xiong, Man Zhang, and Xiangxiang Chu. Everything in its place: Benchmarking spatial intelligence of text-to-image models. arXiv preprint arXiv:2601.20354, 2026. [47] Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu. Qwen-image technical report, 2025. [48] Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, et al. Omnigen2: Exploration to advanced multimodal generation. arXiv preprint arXiv:2506.18871, 2025. [49] Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. Show-o2: Improved native unified multimodal models. arXiv preprint arXiv:2506.15564, 2025. [50] Yu Xu, Yuxin Zhang, Juan Cao, Lin Gao, Chunyu Wang, Oliver Deussen, Tong-Yee Lee, and Fan Tang. Beyond pixels: Visual metaphor transfer via schema-driven agentic reasoning. arXiv preprint arXiv:2602.01335, 2026. [51] Junyan Ye, Dongzhi Jiang, Zihao Wang, Leqi Zhu, Zhenghao Hu, Zilong Huang, Jun He, Zhiyuan Yan, Jinghua Yu, Hongsheng Li, et al. Echo-4o: Harnessing the power of gpt-4o synthetic images for improved image generation. arXiv preprint arXiv:2508.09987, 2025. [52] Chenhao Zhang, Yazhe Niu, and Hongsheng Li. Metaphorstar: Image metaphor understanding and reasoning with end-to-end visual reinforcement learning. arXiv preprint arXiv:2602.10575, 2026. 12

[53] Daoan Zhang, Che Jiang, Ruoshi Xu, Biaoxiang Chen, Zijian Jin, Yutian Lu, Jianguo Zhang, Liang Yong, Jiebo Luo, and Shengda Luo. Worldgenbench: A world-knowledge-integrated benchmark for reasoning-driven text-to-image generation. arXiv preprint arXiv:2505.01490, 2025. [54] Zicheng Zhang, Junying Wang, Farong Wen, Yijin Guo, Xiangyu Zhao, Xinyu Fang, Shengyuan Ding, Ziheng Jia, Jiahao Xiao, Ye Shen, et al. Large multimodal models evaluation: a survey. Science China Information Sciences, 68(12):221301, 2025. [55] Xiangyu Zhao, Peiyuan Zhang, Kexian Tang, Xiaorong Zhu, Hao Li, Wenhao Chai, Zicheng Zhang, Renqiu Xia, Guangtao Zhai, Junchi Yan, et al. Envisioning beyond the pixels: Benchmarking reasoning-informed visual editing. arXiv preprint arXiv:2504.02826, 2025.

A

Appendix

A.1

Benchmark Statistics

Table 5 summarizes the dataset statistics. Conceptual prompts are concise (mean 25.6 words), while descriptive prompts provide richer detail (mean 65.0 words); appending the style suffix roughly doubles the length in both cases. We organize the 1,500 images into 10 thematic categories across three levels: Individual (495 images) covers topics related to personal cognition, emotion, health, and growth; External (499 images) encompasses societal, environmental, technological, and interpersonal themes; and Creative (506 images) includes artistic designs and commercial advertisements. The distribution is approximately balanced across levels, with the largest single category being Creative Design (21.07%). Table 5: Left: Prompt length statistics (word count). Right: Distribution of thematic categories.

Conceptual Prompt + Style Suffix Descriptive Prompt + Style Suffix

A.2

Level

Category

Count

Prop.(%)

Cognitive Processes Mental States Healthy Living Personal Development

198 161 72 64

13.20 10.73 4.80 4.27

Mean

Min

Max

Individual (495)

25.6 68.4 65.0 107.7

10 29 20 38

56 133 142 226

External (499)

Social Issues Natural Environment Technological Life Social Interaction

243 119 88 49

16.20 7.93 5.87 3.27

Creative (506)

Creative Design Advertising

316 190

21.07 12.67

Experimental Setup Details

We conduct all experiments on a single server equipped with 4 × NVIDIA H200 GPUs. Detailed information about each evaluated model and MLLM evaluators is provided in Table 6. A.3

Per-Structure Analysis

To better understand where models struggle on the L2 Structure level, we examine the structure type predicted by the MCQ evaluation against the ground-truth structure type annotated in our dataset. Each L2 MCQ question asks the judge to identify the structure type of the generated image among fusion, replacement, juxtaposition, and cannot be determined, allowing us to construct a confusion matrix between ground-truth and predicted structure types. Aggregated results across all 11 evaluated models are shown in Figure 5. The confusion matrix reveals a striking asymmetry across structure types. Models achieve reasonable accuracy on fusion (67.9%) and juxtaposition (69.2%), where the two domains are either merged into a single object or placed alongside each other. In contrast, replacement accuracy drops dramatically to only 18.9%. When prompted with replacement metaphors, models tend to either fuse the two domains together (37.8%) or place them as separate objects (36.6%), rather than substituting one for the other. 13

Table 6: Detailed information about the evaluated T2I models and MLLM evaluators. Model

Resolution

Source

Version / HF Checkpoint

Closed-source T2I Models Seedream 5.0 Lite 2026 2K Nano Banana 2 2026 1024 × 1024 GPT Image 1.5 2025 1024 × 1024

Year

API API API

doubao-seedream-5-0-260128 gemini-3.1-flash-image-preview gpt-image-1.5-2025-12-16

Open-source T2I Models FLUX.1-dev 2024 1024 × 1024 Checkpoint black-forest-labs/FLUX.1-dev SD 3.5 Large 2024 1024 × 1024 Checkpoint stabilityai/stable-diffusion-3.5-large Z-Image 2025 1024 × 1024 Checkpoint Tongyi-MAI/Z-Image-Turbo Qwen-Image 2025 1328 × 1328 Checkpoint Qwen/Qwen-Image-2512 FLUX.2-dev 2025 1024 × 1024 Checkpoint black-forest-labs/FLUX.2-dev Open-source Unified MLLMs OmniGen2 2025 1024 × 1024 Bagel 2025 1024 × 1024 Janus-Pro 2025 384 × 384

Checkpoint OmniGen2/OmniGen2 Checkpoint ByteDance-Seed/BAGEL-7B-MoT Checkpoint deepseek-ai/Janus-Pro-7B

MLLM Evaluators Qwen3.5-27B 2026 GPT-5.4 2026

Checkpoint API

– –

Qwen/Qwen3.5-27B gpt-5.4-2026-03-05

Ground-Truth Structure Type

Interestingly, this difficulty hierarchy mirrors the cognitive complexity ordering proposed in visual rhetoric theory [33], where replacement is identified as the most demanding structure type for human viewers because the missing element must be inferred from its absence. In contrast, fusion requires disentangling two co-present elements, while juxtaposition simply pairs them side by side. Our results extend this ordering from comprehension to generation: T2I models exhibit the same complexity gradient when producing visual metaphors. This suggests that the cognitive demands of replacement, which require preserving context while substituting a localized element, pose a fundamental challenge for current models. This finding points to compositional precision, the ability to perform localized, structure-aware modifications, as a key direction for advancing visual metaphor generation.

Replacement

Fusion

Juxtaposition Replacement

Fusion

Juxtaposition

Predicted Structure Type

Cannot be

determined

Figure 5: Confusion matrix of predicted vs. ground-truth structure types, aggregated across all models.

A.4

Full Results with Descriptive Prompts

Table 7 reports the full evaluation results under descriptive prompts, complementing the conceptual prompt results in Table 1. Most models show substantial improvements under descriptive prompts, as the detailed visual descriptions provide more explicit guidance for generation. A notable exception is GPT Image 1.5, whose dimension score remains essentially unchanged (−0.06), suggesting that the strongest proprietary models can already infer appropriate visual realizations from abstract conceptual 14

prompts and do not require further specification. Despite these magnitude differences, the overall model ranking remains largely consistent across the two prompt types. Table 7: Evaluation results on VMetaphor-Bench with descriptive prompts. MCQ Accuracy (%) Model Domain Structure Closed-source T2I Models Seedream 5.0 Lite 89.3 69.2 GPT Image 1.5 89.6 72.9 Nano Banana 2 90.8 73.6 Open-source T2I Models SD 3.5 Large 83.0 62.3 FLUX.1-dev 83.8 64.1 Z-Image 87.9 67.7 Qwen-Image 88.6 70.9 FLUX.2-dev 89.4 70.3 Open-source Unified MLLMs OmniGen2 82.6 59.9 Bagel 83.2 62.7 Janus-Pro 84.0 65.9

A.5

Dimension Score (1–5)

Meaning

Mapping

Overall

Efficacy

Logic

Harmony

Overall

96.5 96.6 97.1

87.4 90.6 91.2

86.6 88.5 89.2

4.02 4.12 4.06

4.14 4.24 4.21

4.27 4.34 4.28

4.15 4.24 4.18

93.4 93.5 96.4 95.9 96.4

76.0 79.6 87.5 88.5 88.9

78.8 80.7 85.9 87.0 87.3

3.36 3.45 3.76 3.98 4.00

3.45 3.55 3.90 4.03 4.09

3.82 3.97 4.04 4.08 4.18

3.54 3.66 3.90 4.03 4.09

93.2 92.6 94.9

75.3 76.8 79.1

78.0 79.1 81.0

3.07 3.28 3.31

3.26 3.36 3.52

3.71 3.77 3.62

3.35 3.47 3.48

Cross-Judge Results with GPT-5.4

Table 8 reports the full evaluation results using GPT-5.4 as the judge, complementing the Qwen3.527B results in Table 1. Across all 11 models, GPT-5.4 produces highly consistent rankings with Qwen3.5-27B, with Spearman rank correlations of 0.991 for MCQ and 0.936 for dimension scoring. The two judges differ slightly in absolute scores, with GPT-5.4 tending to assign somewhat more lenient dimension scores. However, the relative performance ordering of models is preserved, confirming that the conclusions reported in our main experiments do not depend on the specific choice of evaluator. Table 8: Evaluation results on VMetaphor-Bench with conceptual prompts, judged by GPT-5.4. MCQ Accuracy (%) Model Domain Structure Closed-source T2I Models Seedream 5.0 Lite 93.3 66.7 Nano Banana 2 94.3 75.6 GPT Image 1.5 94.4 77.2 Open-source T2I Models FLUX.1-dev 91.3 63.8 SD 3.5 Large 91.5 65.7 Z-Image 93.1 66.5 Qwen-Image 91.9 70.3 FLUX.2-dev 94.0 69.3 Open-source Unified MLLMs OmniGen2 90.2 62.0 Bagel 91.7 64.7 Janus-Pro 92.3 67.1

A.6

Dimension Score (1–5)

Meaning

Mapping

Overall

Efficacy

Logic

Harmony

Overall

96.3 98.3 97.6

73.0 76.9 77.2

82.0 85.5 85.8

3.72 4.08 4.13

3.85 4.29 4.37

3.94 4.14 4.33

3.84 4.17 4.27

91.9 91.4 94.8 93.9 94.7

62.8 64.6 65.6 67.3 69.0

76.4 77.4 78.9 79.6 80.9

2.95 2.97 3.05 3.32 3.45

3.07 3.14 3.25 3.45 3.62

3.70 3.68 3.56 3.73 3.84

3.24 3.26 3.28 3.50 3.64

90.9 92.6 93.9

57.6 62.1 67.6

73.7 76.6 79.4

2.51 2.78 3.06

2.65 2.92 3.34

3.42 3.44 3.49

2.86 3.05 3.30

Benchmark Construction Prompts.

Figure 6 and Figure 7 show the prompts for the two-stage annotation pipeline: structured annotation extraction (Stage 1) and descriptive/conceptual prompt generation (Stage 2). Figure 8 shows the prompt for MCQ distractor generation.

15

Stage 1: Prompt for Extracting Structured Annotations You are analyzing a single image that may contain a visual metaphor.

First, check the following conditions. If ANY of them apply, return {{"has_metaphor": false}}:

[conditions]

If it DOES contain a visual metaphor, decompose it into structured components. Only extract structural and semantic information grounded in the image.

Please identify:

1. Target domain: What is the primary object being reconstructed?

2. Source domain: What external domain is visually applied to the target?

3. Structure type:

- Choose one: (a) Fusion: [explainations] (b) Replacement: [explainations] (c) Juxtaposition: [explainations]

4. Element-level mappings

Identify visually supported correspondences between elements of the source and target. Format as pairs: source_element→ target_element

5. Meaning summary

Provide a 1–2 sentence summary of the overall metaphorical meaning. Focus on abstract interpretation, not visual description.

Output in the following JSON format:

{{

"has_metaphor": true,

"target_domain": "",

"source_domain": "",

"structure_type": "",

"mappings": [

{{"source_element": "", "target_element": ""}},

...

],

"meaning": ""

}}

Figure 6: Prompt for structured annotation generation (Stage 1).

Stage 2: Prompt for Generating T2I Prompts You are a vision-language expert. Your task is to write high-quality generation prompts for the provided image, which contains a visual metaphor.

### Reference Annotations:

{structured_annotations}

### Instructions:

You must analyze the ATTACHED IMAGE as your primary reference to ensure visual fidelity. Use the Annotations only to anchor the metaphorical logic.

You will produce three prompts. IMPORTANT: The first two prompts must NOT contain any style-related information (artistic style, color palette, lighting, mood, texture). All style information goes exclusively into style_suffix.

### 1. Descriptive Prompt (Visual Reconstruction):

Write a text-to-image generation prompt that describes the visual content needed to recreate the image.

[requirements]

### 2. Conceptual Prompt (Abstract Level):

Write a concise, natural-sounding generation prompt at the conceptual level.

[requirements]

### 3. Style Suffix:

Describe ONLY the artistic style and visual aesthetics of the image.

[requirements]

Return strictly valid JSON without markdown formatting or code fences.

### Output Format (Strict JSON):

{{

"descriptive_prompt": "...",

"conceptual_prompt": "...",

"style_suffix": "..."

}}

Figure 7: Prompt for descriptive and conceptual prompt generation (Stage 2).

16

Prompt for Generating MCQs You are an expert in visual metaphor analysis. Given the following annotation and descriptive prompt for a visual metaphor image, generate plausible but INCORRECT distractors for multiple-choice questions.

## Annotation

[structured_annotations]

## Descriptive Prompt (describes the visual content of the image)

[descriptive_prompt]

## Distractor Rules

[rules]

## Categories

Q1 (target_domain): "What is the target domain (tenor) of this visual metaphor?"

- Correct answer: "{target}"

- Generate 2 distractors following the rules above.

Q2 (source_domain): "What is the source domain (vehicle) of this visual metaphor?"

- Correct answer: "{source}"

- Generate 2 distractors following the rules above.

Q4 (meaning): "What meaning does this image most likely convey?"

- Correct answer: "{meaning}"

- Generate 2 plausible but wrong interpretations. They must differ in substance from the correct meaning, not just in phrasing.

Q5 (element_mapping): For EACH mapping below, generate 2 wrong target elements.

{mappings_text}

- For each mapping, the question asks: "What does [source_element] visually correspond to in this image?"

- Correct answer is the target_element.

- Generate 2 plausible wrong target elements (NOT synonyms/hypernyms/hyponyms of the correct target_element).

## Output Format

Return strictly valid JSON without markdown formatting or code fences:

{{

"Q1_distractors": ["distractor1", "distractor2"],

"Q2_distractors": ["distractor1", "distractor2"],

......

}}

Figure 8: Prompt for MCQ distractor generation.

17

A.7

Evaluation Prompts

Figure 9 and Figure 10 show the prompts for MCQ evaluation, which is split into two calls: Q1– Q4 (target, source, structure, and meaning) are answered together, while Q5 (element mapping) is answered separately to prevent domain leakage from its answer options. Specifically, Q5 (element mapping) options may reveal the source domain (e.g., a question stem mentioning ’spherical bomb casing’ would expose ’bomb’ as the source), which would compromise the independence of L1 and L4 evaluations. Figure 11 shows the prompt for dimension scoring with 1–5 Likert-scale rubrics. Prompt for MCQ-Based Evaluation (Q1-Q4) You are an expert in visual metaphor analysis.

## Key Concepts

A visual metaphor compares two concepts by visually mapping one onto another, implying a shared property or relationship between them.

- Target domain: The subject being described — what the image is ultimately "about". It receives a property from the source domain. For example, in "french fries are deadly", french fries are the target domain.

- Source domain: The concept used to describe or characterize the target — it lends a property to the target. For example, in "french fries are deadly", bullets serve as the source domain, lending the property "deadly" to french fries.

Note: Either domain may appear literally or be implied in the image. There is no fixed rule about which one is visually depicted.

- Structure type: How the two domains are visually presented in the image. There are three strategies:

- Replacement: Only one domain's element is directly visible. The other domain is ABSENT and must be inferred from the surrounding visual context. For example, a light bulb's position is occupied by a tree — the tree is visible, but the light bulb is absent and inferred from the lamp context.

- Fusion: Two normally distinct objects are physically merged into a single gestalt. Both domains are simultaneously present in ONE object, combined in a way that is physically impossible. For example, a globe with an ice-cream surface — one single object exhibiting features of both earth and ice cream.

- Juxtaposition: Both domains are depicted as separate, complete objects within the same scene. Their metaphorical connection is suggested through visual similarity in form, position, or spatial arrangement. For example, a car placed alongside a cheetah to suggest speed.

- Meaning: The message the image conveys through the comparison of the two domains. For example, an image blending earth and ice cream might convey "the earth is melting".

## Task

Look at the image carefully and answer each of the following multiple-choice questions. Base your judgment ONLY on what is visually present in the image.

For EACH question, respond with ONLY the question ID and the letter answer. Use this exact format (one per line):

Q1: <letter>

Q2: <letter>

Q3: <letter>

Q4: <letter>

Do not explain. Do not add any other text.

{Q1-Q4 questions}

Figure 9: Prompt for MCQ evaluation (Q1–Q4: target domain, source domain, structure type, and meaning).

18

Prompt for MCQ-Based Evaluation (Q5) You are an expert in visual metaphor analysis.

## Key Concepts

A visual metaphor communicates an abstract idea by visually mapping one concept onto another.

- Element mapping: The specific visual correspondence between a element from the source domain and a concept or element from the target domain. For

example, if a rose represents love, the "thorns" (source element) might visually correspond to "emotional pain" (target element).

## Context

- Target domain: {target_domain}

- Source domain: {source_domain}

## Task

Given the target and source domains above, look at the image carefully and answer each of the following multiple-choice questions. Base your judgment ONLY on what is visually present in the image.

For EACH question, respond with ONLY the question ID and the letter answer. Use this exact format (one per line):

Q5_1: <letter>

Q5_2: <letter>

Do not explain. Do not add any other text.

{Q5 questions}

Figure 10: Prompt for MCQ evaluation (Q5: Element mapping).

19

Prompt for Dimension-based Scoring You are an expert evaluator of visual metaphor images.

## Annotation (Ground Truth)

This image is intended to convey a visual metaphor with the following properties:

- Target domain: {target}

- Source domain: {source}

- Structure type: {structure}

- Element mappings: {mappings_text}

- Intended meaning: {meaning}

## Task

Important: The annotation above describes what the image is intended to express — it is a reference, NOT what you are evaluating. Your job is to assess how well the IMAGE ITSELF visually realizes these intentions. A well-written annotation does NOT imply the image is good. If the metaphor described in the annotation is NOT visually perceivable in the image, you must score low regardless of how coherent the annotation sounds.

Evaluate the image on the following three dimensions independently, each on a 1-5 scale.

### 1. Metaphoric Efficacy (1-5)

Can a viewer understand the intended metaphorical meaning from the image alone, WITHOUT reading any textual explanation?

- 5: Any viewer, regardless of background, would immediately and effortlessly understand the metaphor; the meaning is unmistakable and powerfully conveyed.

- 4: A viewer can understand the metaphor with only brief thought; the core meaning is clear and the visual expression shows creativity.

- 3: A viewer can tell the image is attempting a metaphor and may grasp the general direction, but would need the annotation to fully understand the intended meaning.

- 2: The metaphorical intent is very difficult to interpret from the image alone; only vague or ambiguous hints are present.

- 1: The metaphor is completely unrecognizable; a viewer would have no idea what is being communicated.

### 2. Metaphor Logic (1-5)

Is the cross-domain mapping between source and target conceptually coherent and logically sound as depicted in the image?

- 5: The mapping is exceptionally well-constructed; the cross-domain connection feels natural, insightful, and almost inevitable — it is hard to imagine a better visual mapping.

- 4: The mapping is coherent and convincing; the connection is logical, though one could imagine equally good alternative mappings.

- 3: The mapping is understandable but requires some effort to accept; there are noticeable gaps or stretches in the cross-domain logic.

- 2: The mapping is weak; the connection between visual elements and the intended metaphor feels forced or highly unintuitive.

- 1: The mapping makes no sense; there is no logical connection between the visual elements and the intended metaphor.

### 3. Perceptual Harmony (1-5)

As a whole, does this image work as a visual metaphor — does it look natural, coherent, and easy to appreciate?

Consider: Do the combined elements form a coherent whole, or does the image feel strange, confusing, or visually uncomfortable despite good rendering quality?

- 5: The image works perfectly as a whole; the metaphorical fusion feels natural, visually pleasing, and immediately appreciable.

- 4: The fusion is seamless with only very minor imperfections; the image feels cohesive and natural at first glance.

- 3: The image mostly works; the fusion is coherent and the metaphor is appreciable, but with some noticeable awkwardness.

- 2: The image is technically well-rendered, but the metaphorical fusion feels unnatural, confusing, or uncomfortable to view.

- 1: The image feels visually incoherent or grotesque; the combination of elements is jarring and incomprehensible.

## Output Format

Respond with ONLY the three scores in this exact format (one per line):

metaphoric_efficacy: <score>

metaphor_logic: <score>

perceptual_harmony: <score>

Do not explain. Do not add any other text.

Figure 11: Prompt for dimension score evaluation.

20

A.8

Interface for Human Annotators

We provide screenshots of the annotation interfaces used in our study. Figure 12 shows the interface for manual verification during data curation (Section 3.1), where annotators review and edit metadata fields such as domains, structure type, element mappings, and meaning. Figures 13 and 14 show the two pages of the interface used in the human alignment study (Section 4.5): one for answering MCQ questions alongside the generated image, and another for rating dimension scores on a 1–5 scale.

Figure 12: Manual verification interface for data curation

Figure 13: Human evaluation interface for the alignment study: MCQ answering

21

Figure 14: Human evaluation interface for the alignment study: dimension rating

22

A.9

Samples

We present representative samples in Figure 15. For each category, we select one visual metaphor and display the generated images from all evaluated models under the conceptual prompt setting. These examples illustrate the diversity of metaphorical themes in our benchmark and highlight the varying capabilities of different models in generating visual metaphors.

Samples of Generated Results Across Categories PT Image 1.5 Seedream 5.0 Lite

G

LUX.2-dev

F

en-Image

Qw

SD 3.5 Large

agel

B

Cognitive Processes An exclamation mark with a human brain in place of its bottom dot, showing that the sudden impact of a great idea originates from the mind. Mental States A human head merging into a chaotic mass of paper documents portrays the cognitive overload of a mind overwhelmed by an excess of external tasks and information. Healthy Living An alcohol bottle and cigarettes blended into a prison cell demonstrates how destructive habits create a selfimposed trap that strips away freedom. Personal Development A pencil-staircase hybrid representing education as a foundational pathway that elevates individuals from early growth to academic success. Social Issues A hybrid object merging a microphone and an open birdcage releasing a dove, illustrating that unhindered communication liberates voices and ideas from suppression. Natural Environment Plastic bottles falling from the sky in place of bombs show that improperly discarded plastic waste is a destructive, deadly force threatening the survival of vulnerable wildlife. Technological Life A hybrid playground and smartphone environment critiques 'sharenting', illustrating how parents increasingly turn physical childhood play into mere content for social media consumption. Social Interaction Speech bubbles fused with interlocking puzzle pieces show that effective communication relies on sharing complementary ideas to achieve a complete understanding. Creative Design A tea infuser shaped like a tiny human figure relaxing in a hot tub, bringing humor and a sense of leisure to the mundane task of brewing tea. Advertising A hybrid object merging a Toblerone chocolate bar and a Swiss Army knife, presenting the candy as an iconic national invention with authentic Swiss heritage.

Figure 15: Generated visual metaphors across ten categories from VMetaphor-Bench.

23

A.10

Limitations

We acknowledge several limitations. First, the distribution of samples across thematic categories is uneven, reflecting the natural prevalence of visual metaphors in different topics in real-world creative imagery. Second, our evaluation relies on an MLLM-as-judge framework, which, despite strong alignment with human annotators, inherits the fundamental limitations of automatic visual reasoning on the most subjective judgments. Finally, our prompts and source imagery are predominantly English and from Western creative traditions, leaving multilingual prompts and cross-cultural metaphors as future directions. A.11

Broader Impacts

This work advances the evaluation of visual metaphor generation, a challenging aspect of text-toimage models that requires coherent cross-domain reasoning and expressive visual composition. By providing a structured benchmark with fine-grained evaluation criteria, it can support the development of more capable and interpretable generative models. Such progress may benefit real-world applications including creative design, education, and advertising, where visual metaphors are widely used to communicate messages effectively and engage audiences. We also acknowledge potential negative implications. Improved visual metaphor generation could be misused to produce persuasive misleading imagery (e.g., manipulative advertising or political propaganda) that exploits the strong emotional and rhetorical effects of visual analogies. We encourage future work to consider safeguards against such misuse, and our benchmark itself is intended for evaluation rather than direct deployment. A.12

Data Licensing and Release

To respect the rights of original content creators on Pinterest, we do not redistribute the source images. Our public release consists of: (1) image URLs pointing to the original Pinterest pages, (2) all structured annotations (target/source domains, structure types, meaning, element mappings), (3) descriptive and conceptual prompts, and (4) the 9,594 multiple-choice questions. Users can retrieve images via the URLs for research purposes following Pinterest’s terms of service. The annotations and prompts are released under CC-BY-4.0.

24

Related documents

• DataSway: Vivifying Metaphoric Visualization with Animation Clip Generation and Coordination#173333
• Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoning#684345
• Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoning#682570
• VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning#629971
• VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning#628109