arXiv CS open access

A Framework for Evaluating Zero-Shot Image Generation in Concept-based Explainability

2026 · arxiv_cs

arXiv CS · Papers · 2026

artificial-intelligence knowledge-representation reasoning

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

arXiv:2605.19855v1 [cs.CV] 19 May 2026

A Framework for Evaluating Zero-Shot Image Generation in Concept-based Explainability Giacomo Astolfi1 *

Matteo Bianchi1 *

Riccardo Campi1 *

giacomo.astolfi@mail.polimi.it

matteo.bianchi@polimi.it

riccardo.campi@polimi.it

Antonio De Santis1

Marco Brambilla1

antonio.desantis@polimi.it

marco.brambilla@polimi.it

Politecnico di Milano, DEIB, Italy

Abstract

Real Concept Images

Concept-based Explainable Artificial Intelligence (XAI) interprets deep learning models using human-understandable visual features (e.g., textures or object parts) by linking internal representations to class predictions, thereby bridging the gap between low-level image data and high-level semantics. A major challenge, however, is the reliance on large sets of labeled images to represent each concept, which limits scalability. In this work, we investigate the use of zero-shot Text-to-Image (T2I) generative models as a source of synthetic concept datasets for concept-based XAI methods. Specifically, we generate concepts using predefined prompts and evaluate their faithfulness to real ones through four complementary analyses: (1) comparing synthetic vs. real concept images via concept representation similarity; (2) evaluating their intra-similarity by comparing pairs of subsets of the same concept with progressively increasing size; (3) evaluating their performance for downstream explanation tasks using relevant class images; (4) evaluating how removing a concept from tested class images affects explanations of generated concepts. While current T2I generative models promise a shortcut to concept-based XAI, our study highlights challenges and raises open questions about the use of synthetic data generated by zero-shot pipelines in model analyses. The resulting dataset is available at https://github.com/ DataSciencePolimi/ZeroShot-T2I-Concepts.

(e.g. striped textures) Synthetizing Concept Images with Zero-Shot T2I Models

Generated Concept Images

(e.g. striped textures)

Class Images

Removing Concepts from Class Images with I2I Models

(e.g. zebra images)

Concept-Removed Class Images (e.g. zebras with no stripes)

iii Evaluating the Synthetic Concepts with XAI Tools

Figure 1. High-level schematic representation of the proposed i synthesis of concept images with framework. It encompasses ⃝ ii generation of zero-shot T2I models and a predefined prompt, ⃝ counterfactual class images (i.e., without the concepts) with an I2I iii evaluation of the synthetic model using a fixed prompt, and ⃝ concepts against real ones by comparing both original and ablated class images with XAI tools.

hard to understand. Explainable AI (XAI) aims to reveal this hidden reasoning, enabling model debugging, bias detection, and trustworthy decisions in critical applications [1]. For image classification tasks, many XAI methods link internal activations to human-readable concepts, measuring, for instance, the contribution of a concept to a class’s score [4]. Testing with Concept Activation Vectors (TCAV) [9] first introduced this paradigm by representing each concept as a direction in the latent space, known as a Concept Activation Vector (CAV), and by measuring its influence on model predictions. Building on this, Visual-TCAV [17] improves concept magnitude estimation and localizes each

1. Introduction Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) excel at extracting complex patterns from data, but their decision-making processes are opaque and * These authors contributed equally. Correspondence to: riccardo.campi@polimi.it.

concept in an image via a concept map. These approaches require collecting images that illustrate user-defined semantic concepts, a process that is labor-intensive and often impractical, limiting both scalability and the variety of concepts that can be studied. To address these limitations, researchers are increasingly relying on synthetic concepts, which enable the (semi-)automated and scalable creation of training data for a much wider range of concepts [8]. However, when Text-to-Image (T2I) models are involved in the synthesis process, the resulting concepts may contain visual artifacts and often lack the richness and variability of real-world data. This can introduce bias, lower the quality of explanations, and amplify spurious model correlations, undermining user trust [2]. In this work, we present a concept generation framework that investigates how faithfully synthetic concepts match their real counterparts under different perspectives (Figure 1). In particular, this work uses zero-shot T2I generative models based on predefined prompts to produce concept images to address four complementary research questions:

sensitivity to user-defined concepts during image classification. To produce a CAV, an analyst collects example images representing a concept along with negative examples (e.g., random images). A linear classifier is trained to distinguish its activations within a specific layer, and the CAV is defined as the vector orthogonal to the classifier’s decision boundary. TCAV then compares this vector with the gradient of a target class in a set of test images to determine whether the concept positively or negatively influences the output. Later work showed that CAVs can also be computed directly via the Difference of Means (DoM) method, improving efficiency and stability to small perturbations [12, 15]. Visual-TCAV further improved the approach by replacing the flattening operation with a Global Average Pooling (GAP) during centroid computation, producing pooledCAVs that better separate spatial features from conceptual representations [17]. Additionally, its concept attribution score measures the magnitude of a concept’s influence on the model’s prediction, not just its sensitivity to that concept. It is computed by aggregating normalized integrated gradient attributions across feature maps using the normalized pooled-CAV weights, producing a spatial attribution map, which is subsequently masked by the normalized concept map and summed over spatial locations to obtain a scalar attribution score for the target class. CAVs built from user-defined concepts require manually curated image datasets for accurate detection, which can be time-consuming. Text2Concept [13] addresses this problem by utilizing CLIP [16] to generate CAVs from text descriptions, and training a linear layer to map CLIP features to a CNN layer. Similarly, Language-Guided CAV (LG-CAV) builds on this idea, proposing the use of CLIP, along with additional modules, to enhance concept quality and train CAVs from textual descriptions. [8]. On the other hand, an emerging line of work explored the alignment between real and synthetic CAVs generated via zero-shot and fine-tuned T2I models, enabling the refinement of generated images using a small set of real examples [2]. However, it revealed limitations, such as drops in semantic alignment with complex concepts (e.g., 3D objects) or the presence of artifacts. Building on prior efforts, this work introduces a conceptgeneration and validation pipeline that leverages zero-shot T2I models and predefined generation prompts. The goal is to assess how faithfully synthetic concepts replicate their real-image counterparts, both in terms of activation space representations and their influence on downstream explanation tasks.

RQ1 Alignment of Concept Representations: To what extent do the concept representations (i.e., CAVs) derived from synthetic images align with the ones derived from real images? RQ2 Concept Representation Intra-Similarity: How does, for each concept, increasing the number of concept representation training images affect their internal pairwise similarity, and how does this effect differ between real-image and synthetic-image sets? RQ3 Impact on Downstream Explanation Task: How does substituting real concept images with synthetic ones affect explanations (e.g., concept attribution scores) produced for target classes? RQ4 Counterfactual Testing on Downstream Task: If a concept is removed from class images using a prompt-driven Image-to-Image (I2I) model, how do explanations change for both real and generated concept images?

2. Related Work XAI has gained significant attention for its goal of making machine learning models more interpretable and understandable to humans [1]. This is especially important for neural networks, whose internal operations are often hard to interpret. In computer vision, to understand which high-level features influence model decisions, researchers have explored concept-based explanation methods, which assess the impact of human-interpretable concepts, ranging from shapes and colors to textures, object parts, and abstract traits like happiness or sadness, on a model’s predictions [4, 14]. A widely adopted technique, mostly used on convolutional architectures, is TCAV [9], which measures a model’s

3. Method The idea behind this work is to evaluate the faithfulness of synthetic concepts to real ones when integrating zero-shot T2I generative models into a concept-generation workflow for user-defined explainability methods. 2

Concept c + Relevant Class kc

ages Icgen for each concept c. For comparison, an equivalent, per-concept set of mc real images Icreal is collected.

Class Images

(e.g. zebra images)

(e.g. striped + zebra)

3.2. Concept Representation Alignment Concept Generation Prompt τgen (e.g. "a striped texture")

(e.g. "remove the stripes")

T2I Model g

I2I Model h

(e.g. Flux 1.1)

Generated Concept Images

The purpose of evaluating the alignment of concept representations between synthetic and generated data is to obtain an initial quantitative indication of their similarity. We extract latent representations from concept images using an image classification model f (e.g., a pre-trained CNN), which maps an image to its corresponding latent vector of activations up to a specific layer/block. Given a fixed concept c, generator g, and model f , we name Agen c the resulting synthetic latent activations. Analogously, Areal c are the latent activations derived from real concept images. A set of negative examples I neg yielding to Aneg is also required. Subsequently, a user-defined, concept-based XAI method for vision, Ψ, is used to compute CAVs via the function ψ cav . Specifically, given the concept c and the acneg real in the feature space of f , it tivations Agen c , Ac , and A produces their CAVs as

Concept Ablation Prompt τrm

(e.g. GPT-Image-1)

(e.g. striped textures)

Concept-Removed Class Images (e.g. zebras with no stripes)

f c

Real Concept Images

(e.g. striped textures)

Image Classification Model f

XAI Method Ψ

neg vcgen = ψ cav (Agen c , A ),

To quantify alignment, we compute the pairwise similarity ρc using the cosine similarity function S between the concept representations, which is suited for highlighting perceptually meaningful variations [21]:

g RQ1: Concept Representation Alignment

RQ2: Concept Representation Intra-Similarity

RQ3: Impact on Downstream Explanation Task

neg vcreal = ψ cav (Areal c , A ).

RQ4: Counterfactual Testing on Downstream Task

ρc = S(vcgen , vcreal ) .

Figure 2. Pipeline for generating synthetic concepts using zeroshot T2I models and predefined prompts, and evaluating them for a A contheir use in concept-based, user-defined XAI methods. ⃝ cept c and its appropriate class kc are selected, then ⃝ b synthetic concept images are generated from prompt templates τ gen using c real concept images are collected from a T2I model g, while ⃝ d Class kc images containing the concept are gathered and data. ⃝ e edited using I2I models h and prompt templates τ rm to rethen ⃝ move c, resulting in concept-removed class images. ⃝ f An image classification model f and an XAI method Ψ are used on both syng Four research questhetic and real images to compute CAVs. ⃝ tions are addressed: RQ1 the alignment between synthetic and real representations, RQ2 their intra-similarity as the number of images grows, RQ3 the impact on downstream explanation tasks, and RQ4 the consistency of explanations after concept removal.

Higher ρc indicates stronger alignment between synthetic and real CAVs, while lower values suggest limited semantic overlap.

3.3. Concept Representation Intra-Similarity The concept representation intra-similarity measures how consistent each concept’s CAV is as we increase the number of images used to compute it. Each comparison is performed between two randomly sampled, equal-sized subsets drawn from the same concept. This will enable us to quantify each concept’s representational stability as a function of its image set size, both for synthetic and real images. By examining how these curves differ, we can assess whether synthetic images provide a faithful approximation of real concept variability. To do so, within each concept c, we compute ρgen c (u) as the similarity between pairs of CAVs obtained from equally-sized random subsets of u synthetic images, with u ≤ nc /2. The factor of 1/2 is a constraint that ensures that the two subsets do not overlap. The same is done for

3.1. Synthetic Image T2I Generation The proposed framework (Fig. 2) begins by taking in a set of concepts, where each concept c is associated with a type tc (e.g., texture, object) and a predefined relevant class kc . Each unique type tc is associated with a dedicated prompt template τtgen (c), which takes the concept as inc put and produces as output its corresponding generation prompt. These are then passed to a randomly seeded T2I model g, which generates a collection of nc synthetic im-

ρreal c (u) , with u ≤ mc /2. real Although ρgen c (u) and ρc (u) are expected to grow toward 1 with larger u [2], discrepancies at the same subset

If synthetic concepts exhibit importance variations similar to those of their real counterparts, this would suggest that they encode comparable semantic information and can be reliably employed in explanation pipelines. Conversely, it indicates that the synthetic concepts fail to capture the same underlying semantics.

size indicate that synthetic images may fall short of representing the complexity of real concepts.

3.4. Impact on Downstream Explanation Task To evaluate the impact of synthetic concepts on a downstream explanation task, we measure the extent to which these contribute to classifier f ’s decisions, providing a basis for comparison with real ones. For each concept c and its relevant target class kc , we evaluate the effectiveness of vcgen by quantifying the influence of c on predicting kc . We rely on ψ score , which is the importance function defined within Ψ. The evaluation is grounded in activations Akc derived from class-specific images Ikc : score sgen (vcgen , kc , Akc ), c =ψ

3.5.2. Concept Removal Validation via Accuracy Drop To evaluate the I2I removal process itself, we compute the classification probability of the neural classifier f for class kc , independently of the selected layer, on both the original class images Ikc and on the images after removal Ikrmc . We then measure the classification probability drop on kc as ∆Pkc = fkc (Ikc ) − fkc (Ikrmc ) .

score sreal (vcreal , kc , Akc ). c =ψ

If changes in ∆Pkc and ∆sreal,rm (i.e., the importance c variation after concept removal for real concept images) are positively correlated, then removing the concept from the class-specific images using the I2I model would be considered successful, and the results of counterfactual testing on the downstream task with synthetic images would be stronger.

We then measure the difference between these scores to evaluate how closely synthetic concepts match the explanatory power of real ones: real ∆sc = |sgen c − sc | .

Large values of ∆sc indicate that the synthetic concept does not reproduce the importance pattern of its real counterpart, signaling a potential mismatch in semantic fidelity.

4. Experiments The experimental design investigates how diverse generative models, visual domains, and model architectures impact the faithfulness of synthetic concept images to their real counterparts, introducing several experimental choices that represent unique firsts in this research field. Our code can be found at https : / / github . com / DataSciencePolimi/ZeroShot-T2I-Concepts.

3.5. Counterfactual Testing on Downstream Task To further test the influence of a synthetic concept c for explanations, we remove it from class images Ikc and observe the impact on both real and generated concept images in the downstream task before and after concept removal. The removal process is carried out by an I2I generative model h, which is guided by a predefined prompt template τ rm (c) to edit each image. This produces a new set of class images Ikrmc where the concept is absent.

4.1. Concepts and Datasets We select multiple concept datasets, spanning different levels of visual abstraction, from low-level textures and materials to high-level object semantics. We include 13 concepts from the Describable Textures Dataset (DTD) [3], which provides perceptual texture concepts, such as cracked or honeycombed, to test whether synthetic ones capture low-level visual patterns. Then, we incorporate as concepts a subset of 10 ImageNet [5] classes, each assigned the type tc = object, requiring the network to combine multiple conceptual cues (e.g., tattoo, feline). We also utilize 10 concepts from the Flickr Material Database (FMD) [18], each with tc = texture and comprising both natural and artificial materials, such as foliage and metal. Unlike DTD, it offers greater variability and context, as materials appear under diverse lighting and scene conditions. Finally, in line with prior studies, we incorporate 8 concepts from search engines (e.g., leopard print, sphere) by manually collecting example images from publicly available sources, ensuring a diverse range of visual samples, whose type can be both object or texture.

3.5.1. Concept Importance Variation After Removal Comparing the variation in importance scores across explanations for real and generated images after removal reveals how closely synthetic concepts mirror the explanatory role of real concepts in downstream tasks, as seen through a counterfactual lens. First, we compute Arm kc as the activations obtained after removing c from class images. Then, we employ them to obtain the synthetic importance score sgen,rm = ψ score (vcgen , kc , Arm c kc ). The same process is applied to real images, yielding sreal,rm . c The importance variations after removal for synthetic and real concept images are then defined as gen,rm , ∆sgen,rm = sgen c c − sc real,rm ∆sreal,rm = sreal . c c − sc

Our concept set contains 41 concepts and 3,860 images, with the number of images per concept mc varying across datasets: 120 for DTD, 50 for ImageNet, 100 for FMD, and 100 for the search engine’s concepts. For each concept, we associate a corresponding ImageNet class (e.g., dotted→dalmatian, chequered→crossword puzzle), with each class containing 50 images, resulting in a total of 2,100 class images. A full listing of all the concepts used can be found in Section A.

(a) Real

(b) Flux 1.1

(d) GPT-Image-1

Figure 3. Example of striped concept images: comparison between a real one and synthetic renditions generated by Flux 1.1 (3b), SD 3.5 (3c), and GPT-Image-1 (3d).

4.2. Experimental Setup CAVs. All images are preprocessed to each model’s input dimensions, without extra augmentations. Figure 3 shows an example of a striped concept, while Section F shows a representative subset of the generated concepts. Concept generation prompts are designed to align with each unique semantic type tc , whether a texture or an object. In particular, each concept type is linked to its prompt template τtgen (c), into which the concept name c is dynamic cally inserted as a parameter:

T2I and I2I Generative Models The T2I models, denoted by g in out pipeline, are Flux 1.1 Schnell [10], Stable Diffusion 3.5 Medium (SD 3.5) [6], and GPT-Image1 [6, 10]. These represent distinct paradigms of visual synthesis. Flux employs a transformer-based rectified flow architecture capable of fine-grained spatial reasoning, producing highly detailed images with strong compositional structure. SD 3.5 performs generation in a latent space via a variational autoencoder, achieving high perceptual fidelity and efficiency. GPT-Image-1, by contrast, operates as a multimodal language model that integrates linguistic and visual reasoning, allowing precise adherence to textual instructions but often producing concept images with recurring styles. GPT-Image-1 is also used as an I2I model in our pipeline, h, specifically for concept removal from class images via prompt-guided editing.

T2I Object Generation Prompt Template A realistic photo of a [CONCEPT] shown clearly in action and in its natural setting. The image should focus on the object’s shape, material, and details, captured with natural lighting and minimal background distractions. Photographed from different angles or in natural settings, but always with the object clearly visible and realistically rendered.

Image Classification Models Models f with heterogeneous internal architectures are essential to ensure the validity of our findings does not depend on a single network design. For this reason, we use the last two convolutional layers from four selected architectures as the analytical backbone of our experiments. VGG-16 (block5 conv2 and block5 conv3) [19], ResNet-50-V2 (conv5 block2 out and conv5 block3 out) [7], and Inception-V3 (mixed9 and mixed10) [20] serve as canonical CNNs, while ConvNeXt (add 34 and add 35) [11] has a new-generation convolutional design informed by transformer architecture principles. All networks are pretrained on ImageNet [5], which makes them well-suited as feature extractors for generaldomain concepts, as in our case.

T2I Texture Generation Prompt Template A realistic close-up of the concept [CONCEPT] as the main subject, shown in high detail and natural lighting. The concept should fill the frame or dominate the image, with clear texture and form. Multiple perspectives and realistic, rich, colorful surfaces are encouraged. In contrast, the template τ rm (c) used to remove a concept c from its class images does not depend on its type: I2I Concept Removal Prompt Template

XAI Methods In this work, we use TCAV [9] with DoM and Visual-TCAV [17] as the main XAI methods Ψ to compute CAVs and importance scores. Additionally, we use CLIP [16] as an external validation for the alignment and intra-similarity of concept representations, providing further validation of the results.

Completely remove the concept of [CONCEPT] from the image, keeping it as visually similar as possible without [CONCEPT] being present at all. The resulting prompt and an I2I model h along with Ikc produce a dataset of ablated class images Ikrmc of the same size as the original ones. An example showing a golf ball image after concept removal is shown in Fig. 4, while other ablation examples are reported in Sec. G.

4.3. Concept Generation and Removal Each synthetic concept c consists of nc = 200 images per generative model g, which is sufficient to robustly estimate 5

(a) Original

(b) No grass

(d) No plastic

Figure 4. Example of concept removal from a golf ball class image, eliminating grass (4b), sphericity (4c), and plastic (4d).

Computations involving the generated concepts are carried out with 5 bootstrap replicates, where resampling is performed with replacement from their associated concept image sets at a unitary sampling ratio. At each iteration, a resampled dataset of equal size is constructed and used to recompute both vcgen and the importance scores sgen c .

Figure 5. Boxplot showing the distribution of similarities ρ between generated and real concept vectors (vcgen and vcreal ) across different T2I generative models computed using Visual-TCAV. The average similarity scores for Flux 1.1 (0.541 ± 0.204), GPTImage-1 (0.539±0.199), and Stable Diffusion 3.5 (0.588±0.190) indicate partial alignment between synthetic and real concepts, with enough divergence to suggest semantic differences.

4.4. Hardware and Execution Time All local computations took place on a 10 GB NVIDIA RTX 3080 GPU, a 16-core AMD 5800X processor, and 64 GB of system memory. Each concept dataset required 5–10 minutes to synthesize on our hardware, resulting in an average total generation time of 12 hours across both Flux and SD models. The remaining concept images, together with those used for concept removal, were generated using GPT-Image-1 via the OpenAI API and completed within 8 hours, at a cost of approximately $150. Subsequent stages, such as feature extraction, CAV computation, and concept importances, were executed locally in 4-6 hours for all architectures and datasets.

Figure 6. Concept representation intra-similarity scores ρ(u), computed both for the real concept images and the synthetic ones using Visual-TCAV. Increasing the number of concept images u results in greater intra-similarity, a consistent trend across both real and synthetic concepts. Synthetic ones exhibit higher intrasimilarity even with a few concept images, suggesting that zeroshot T2I models in this setting may have limited capacity to capture real-world concept diversity.

5. Results and Discussion Below, we present the results, mirroring the structure of Sec. 3, and discuss each step in detail to address the RQs. All results are computed using Visual-TCAV and presented aggregated by layer and by convolutional architecture, with more details in Sections B and D. Sections C and E contain instead TCAV-specific results.

To further validate our results, we compute ρCLIP by c comparing the CLIP embeddings of Icgen and Icreal , yielding on average 0.673 ± 0.097 for Flux 1.1, 0.706 ± 0.094 for GPT-I-1, and 0.708 ± 0.091 for SD 3.5, The pattern agrees with the one presented above, with sufficient divergence between real and synthetic concepts.

5.1. Concept Representation Alignment After computing vcgen and vcreal for each concept c, we compute their cosine similarity ρc to evaluate their alignment. Figure 5 compares the results from the selected T2I models. Quantitatively, we obtain an average similarity ρ̄ over all concepts of 0.541 ± 0.204 for Flux 1.1, 0.539 ± 0.199 for GPT-I-1, and 0.588 ± 0.19 for SD 3.5. More details are presented in Sections B and C. These results show that real and synthetic CAVs share some similarity but are also sufficiently distinct. Specifically, similarities remain relatively low, and their standard deviations are significant. Therefore, with respect to RQ1, synthetic images can only provide partial alignment with real ones, yielding concepts that may differ.

5.2. Concept Representation Intra-Similarity We conduct this experiment to compare the behavior of vcgen and vcreal as their concept image subset sizes u change, using the intra-similarity as a measure of convergence. For synthetic concepts, a significant difference indicates misalignment, while excessively high similarity suggests that synthetic images fail to capture the concepts’ complexity. 6

Figure 7. Boxplot showing the distributions of variations in importance scores (∆sc ) across different T2I generative models, computed using Visual-TCAV. It highlights low average differences and high variability between real and synthetic concepts (0.039 ± 0.055 for Flux, 0.043 ± 0.052 for GPT-I-1, and 0.041 ± 0.061 for SD 3.5). The accompanying KS test confirms that these importance distributions are statistically distinct (p < 10−6 ), underscoring the semantic divergence introduced by synthetic generation.

Figure 8. Violin plots illustrating the distribution of importance variations (∆srm c ) computed using Visual-TCAV after removing concepts from input class images using an I2I model. Most values cluster near zero for both real and generated concepts, but the latter have a narrower distribution (averages of 0.016 ± 0.037 for Flux, 0.013 ± 0.032 for GPT-I-1, and 0.016 ± 0.04 for SD 3.5) compared to the former (0.031 ± 0.065). This highlights that concept ablation has a lower impact on synthetic importances.

As illustrated in Figure 6, increasing the number of concept images reduces variability, a trend observed for both real and synthetic CAVs. However, even when the number of concept images is limited, synthetic CAVs are consisreal tently higher than real ones (ρgen c (u) > ρc (u) ∀u). Considering RQ2, this behavior highlights the limitations of the tested zero-shot T2I models in capturing the intrinsic diversity of concepts, underscoring their reduced ability to accurately reproduce real-world scenarios. and ρCLIP,gen using CLIP, We also compute ρCLIP,real c c which are independent from u, whose results on average are 0.743 ± 0.093 for real images, and 0.858 ± 0.079 for Flux 1.1, 0.911 ± 0.079 for GPT-I-1, and 0.876 ± 0.076 for SD 3.5, confirming the pattern presented above.

More details are presented in Sections D and E. Considering RQ3, even though synthetic data can approximate the visual characteristics of concept images, the observed deviations indicate that they fail to replicate the explanatory behavior of real images.

5.4. Counterfactual Testing on Downstream Task This part evaluates both the impact of synthetic concepts on ablated class images (RQ4) and the effectiveness of the I2I removal process itself. 5.4.1. Concept Importance Variation After Removal Using the ablated class images, we compute the importance ) and synthetic concepts variations for both real (∆sreal,rm c (∆sgen,rm ). We compare these scores in Figure 8, which c presents the violin plots of their distributions. From the figure, we can observe a difference in the impact of the ablation procedure when using real or synthetic concepts. Specifically, removing a concept results in a lower change in the importance score when considering generated concept images (averages of 0.016 ± 0.037 for Flux, 0.013 ± 0.032 for GPT-I-1, and 0.016 ± 0.04 for SD 3.5) compared to the baseline of real concept images (0.031 ± 0.065). Quantitatively, a two-sided KS test confirms that the distributions differ significantly (p < 10−6 ).

5.3. Impact on Downstream Explanation Task We analyze the importance differences to assess how closely the synthetic representations reproduce the explanatory behavior of their real counterparts. To this end, for each c, we extract the importances of all images with respect to their target class kc and compute the absolute value in the change in the importance score ∆sc . Figure 7 presents the distributions of this metric for the three T2I models. Overall, the absolute importance variations have a low average with high variability (0.039 ± 0.055 for Flux, 0.043 ± 0.052 for GPT-I-1, and 0.041 ± 0.061 for SD 3.5), indicating differences between real and synthetic concepts. To further validate this observation, we perform a two-sided Kolmogorov-Smirnov (KS) test to assess whether the distributions of real and synthetic importances differ. For all three T2I models, we observe a significant variation with high confidence (KS statistics of 0.196, 0.316, and 0.245, all with p < 10−6 ), indicating that the importances differ.

5.4.2. Concept Removal Validation via Accuracy Drop Finally, to evaluate the effectiveness of the concept removal procedure, we analyze whether a monotonic relationship exists between the importance variation ∆sreal,rm and the c corresponding drop in classification probability ∆Pkc . To this end, we compute these quantities across all benchmarked concepts and image classifiers. In this scenario, we observe a monotonic trend where higher variations in im7

ones for the selected relevant classes, while RQ4 found that concept removal causes a smaller shift in attribution scores for generated images than for real ones. This emphasizes the importance of carefully evaluating the use of synthetic concepts in XAI pipelines. Future work will include coupling our T2I generation pipeline with fine-tuning techniques or I2I generative models, such as few-shot learning. Additionally, the discussed methodology could be applied to domain-specific concepts to evaluate generative models’ capabilities on more specialized tasks.

References [1] Alejandro Barredo Arrieta, Natalia Dı́az-Rodrı́guez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion, 58:82–115, 2020. 1, 2 [2] Riccardo Campi, Santiago Borrego, Antonio De Santis, Matteo Bianchi, Andrea Tocchetti, and Marco Brambilla. Towards synthetic concept activation vectors via generative models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2711– 2719, 2025. 2, 3 [3] M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, , and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2014. 4 [4] Antonio De Santis, Riccardo Campi, Matteo Bianchi, Andrea Tocchetti, and Marco Brambilla. 2 - foundational approaches to post-hoc explainability for image classification. In Bi-directionality in Human-AI Collaborative Systems, pages 23–54. Academic Press, 2025. 1, 2 [5] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. 4, 5 [6] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In Proceedings of the 41st International Conference on Machine Learning. JMLR.org, 2024. 5 [7] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Computer Vision – ECCV 2016, pages 630–645, Cham, 2016. Springer International Publishing. 5 [8] Qihan Huang, Jie Song, Mengqi Xue, Haofei Zhang, Bingde Hu, Huiqiong Wang, Hao Jiang, Xingen Wang, and Mingli Song. Lg-cav: Train any concept activation vector with language guidance. In Advances in Neural Information Processing Systems, pages 39522–39551. Curran Associates, Inc., 2024. 2

Figure 9. Scatter plot illustrating the relationship between the importance variation (∆sreal,rm ), computed with Visual-TCAV usc ing real concept images on ablated class images, and the corresponding change in the model f ’s classification probability (∆Pkc ). Spearman’s rank correlation indicates a positive association (ρ = 0.656, p < 10−6 ). A fitted logistic curve is overlaid (RM SE = 0.240), revealing an increasing monotonic trend.

portance scores are associated with higher drops in the predicted probability (Spearman Rank Correlation ρ = 0.656, p < 10−6 ). This behavior is shown in Figure 9.

6. Conclusion This work presents a concept-generation framework that integrates zero-shot T2I generative models with predefined prompts into user-defined, concept-based XAI methods, aiming to assess whether automatically generated concepts can substitute for real ones while preserving their explanatory behavior. To this end, we evaluate (RQ1) the alignment between concept representations derived from synthetic and real images for the same semantic concept, (RQ2) how increasing the number of concept images affects concept representation intra-similarity across real and synthetic sets, (RQ3) the impact of replacing real concept images with synthetic ones on downstream explanation tasks, and (RQ4) how concept removal via prompt-driven image editing alters importance scores in counterfactual explanations for both real and generated images. The results obtained provide evidence that, for the considered T2I models, synthetic concepts generated using a zero-shot approach may have a limited ability to reproduce key aspects of real concepts. Specifically, RQ1 confirmed that generators achieve systematic but partial alignment of latent representations, RQ2 demonstrated that synthetic concepts may struggle in capturing the intrinsic diversity of real-world concepts, RQ3 showed that a significant variation exists between synthetic attribution scores and real 8

on Computer Vision and Pattern Recognition (CVPR), pages 2818–2826, 2016. 5 [21] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 586–595, 2018. 3

[9] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Proceedings of the 35th International Conference on Machine Learning, pages 2668–2677. PMLR, 2018. 1, 2, 5 [10] Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context image generation and editing in latent space, 2025. 5 [11] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11966–11976, 2022. 5 [12] Tyler Martin and Adrian Weller. Interpretable Machine Learning. M.Phil. diss., Dept. of Engineering, University of Cambridge, 2019. 2 [13] Mazda Moayeri, Keivan Rezaei, Maziar Sanjabi, and Soheil Feizi. Text2concept: Concept activation vectors directly from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 3744–3749, 2023. 2 [14] Christoph Molnar. Interpretable Machine Learning: A Guide For Making Black Box Models Explainable. Independently published, 2022. 2 [15] Frederik Pahde, Maximilian Dreyer, Moritz Weckbecker, Leander Weber, Christopher J. Anders, Thomas Wiegand, Wojciech Samek, and Sebastian Lapuschkin. Navigating neural space: Revisiting concept activation vectors to overcome directional divergence. In The Thirteenth International Conference on Learning Representations, 2025. 2 [16] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, pages 8748–8763. PMLR, 2021. 2, 5 [17] Antonio De Santis, Riccardo Campi, Matteo Bianchi, and Marco Brambilla. Visual-TCAV: Concept-based Attribution and Saliency Maps for Post-hoc Explainability in Image Classification, 2025. arXiv:2411.05698 [cs]. 1, 2, 5 [18] Lavanya Sharan, Ruth Rosenholtz, and Edward H. Adelson. Accuracy and speed of material categorization in real-world images. Journal of Vision, 14(10), 2014. 4 [19] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015. 5 [20] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference

A Framework for Evaluating Zero-Shot Image Generation in Concept-based Explainability Supplementary Material

A. Complete List of Concepts Table 1 contains a detailed overview of the concept-class dataset used in our study. The dataset comprises 41 concepts, each mapped to a unique class label derived from the ImageNet dataset. Real concept images are drawn from four distinct datasets: DTD, ImageNet, FMD, and search engine results. Type

Dataset

Concept

Relevant Class

Type

Dataset

Concept

Relevant Class

Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture Texture

DTD DTD DTD DTD DTD DTD DTD DTD DTD DTD DTD DTD DTD FMD FMD FMD FMD FMD FMD FMD FMD FMD FMD

Bubbly Chequered Cracked Crystalline Dotted Frilly Honeycombed Meshed Perforated Striped Waffled Woven Wrinkled Fabric Foliage Glass Leather Metal Paper Plastic Stone Water Wood

Beer glass Crossword puzzle Stone wall Honeycomb Dalmatian Overskirt Honeycomb Honeycomb Honeycomb Zebra Waffle iron Overskirt Overskirt Trench coat Spider web Beer glass Trench coat Waffle iron Crossword puzzle Golf ball Stone wall Beer glass Wooden spoon

Object Object Object Object Object Object Object Object Object Object Object Object Object Object Object Object Texture Texture

ImageNet ImageNet ImageNet ImageNet ImageNet ImageNet ImageNet ImageNet ImageNet ImageNet Search Search Search Search Search Search Search Search

Asparagus Cloister Feline Grandfather Guitarist Kitchen Lichen Rodent Steeple Tattoo Cast iron Fin Grass Newspaper Pen Sphere Leopard print Spotted

Stone wall Church Leopard Tench Electric guitar Waffle iron Stone wall Leopard Church Zebra Waffle iron Tench Golf ball Crossword puzzle Fountain pen Golf ball Leopard Dalmatian

Table 1. Summary of the concept-class dataset used in our experiments. Each concept is associated with a specific class (e.g., dotted → dalmatian). Concepts are drawn from four distinct datasets: Describable Textures Dataset (DTD), ImageNet, Flickr Material Database (FMD), and search engine results.

B. Average Cosine Similarity per Concept using Visual-TCAV

Class

beer glass

church

crossword puzzle

Concept

bubbly

glass

water

cloister

steeple

chequered

Flux GPT-Image-1 SD 3.5

0.666 ± 0.14 0.659 ± 0.038 0.689 ± 0.1

0.453 ± 0.139 0.389 ± 0.162 0.395 ± 0.103

0.59 ± 0.069 0.509 ± 0.14 0.523 ± 0.099

0.848 ± 0.017 0.389 ± 0.081 0.894 ± 0.026

0.726 ± 0.07 0.842 ± 0.042 0.764 ± 0.057

0.217 ± 0.141 0.586 ± 0.096 0.664 ± 0.102

electric guitar

fountain pen

Concept

Class

newspaper

crossword puzzle paper

dotted

spotted

guitarist

pen

Flux GPT-Image-1 SD 3.5

0.589 ± 0.052 0.702 ± 0.042 0.69 ± 0.125

0.589 ± 0.067 0.475 ± 0.061 0.533 ± 0.074

0.376 ± 0.126 0.654 ± 0.155 0.525 ± 0.098

0.375 ± 0.155 0.554 ± 0.144 0.346 ± 0.138

0.815 ± 0.04 0.699 ± 0.065 0.818 ± 0.036

0.595 ± 0.206 0.305 ± 0.174 0.482 ± 0.226

Class

dalmatian

golf ball

honeycomb

Concept

grass

plastic

sphere

crystalline

honeycombed

meshed

Flux GPT-Image-1 SD 3.5

0.411 ± 0.168 0.328 ± 0.194 0.485 ± 0.131

0.504 ± 0.092 0.247 ± 0.085 0.435 ± 0.096

0.483 ± 0.111 0.487 ± 0.094 0.561 ± 0.103

0.486 ± 0.083 0.482 ± 0.094 0.498 ± 0.153

0.707 ± 0.188 0.787 ± 0.07 0.866 ± 0.045

0.588 ± 0.123 0.411 ± 0.057 0.571 ± 0.145

Class

honeycomb

leopard

overskirt

Concept

perforated

feline

leopard print

rodent

frilly

woven

Flux GPT-Image-1 SD 3.5

0.336 ± 0.123 0.658 ± 0.064 0.641 ± 0.045

0.537 ± 0.132 0.732 ± 0.062 0.612 ± 0.114

0.6 ± 0.127 0.828 ± 0.13 0.682 ± 0.1

0.607 ± 0.072 0.685 ± 0.117 0.726 ± 0.095

0.473 ± 0.142 0.512 ± 0.128 0.432 ± 0.151

0.456 ± 0.182 0.378 ± 0.082 0.691 ± 0.066

Class

overskirt

spider web

stone wall

Concept

wrinkled

foliage

asparagus

cracked

lichen

stone

Flux GPT-Image-1 SD 3.5

0.535 ± 0.133 0.419 ± 0.106 0.559 ± 0.14

0.568 ± 0.192 0.628 ± 0.17 0.508 ± 0.217

0.622 ± 0.124 0.279 ± 0.223 0.693 ± 0.078

0.657 ± 0.041 0.696 ± 0.057 0.678 ± 0.116

0.382 ± 0.312 0.505 ± 0.115 0.569 ± 0.027

0.542 ± 0.099 0.559 ± 0.176 0.517 ± 0.144

Class

tench

trench coat

waffle iron

Concept

fin

grandfather

fabric

leather

cast iron

kitchen

Flux GPT-Image-1 SD 3.5

0.151 ± 0.084 0.282 ± 0.097 0.317 ± 0.118

0.666 ± 0.085 0.658 ± 0.155 0.681 ± 0.099

0.572 ± 0.111 0.508 ± 0.114 0.55 ± 0.102

0.602 ± 0.124 0.572 ± 0.129 0.567 ± 0.111

0.164 ± 0.114 0.31 ± 0.104 0.282 ± 0.144

0.773 ± 0.117 0.445 ± 0.086 0.715 ± 0.069

Class

waffle iron

wooden spoon

zebra

Concept

metal

waffled

wood

striped

tattoo

Flux GPT-Image-1 SD 3.5

0.458 ± 0.134 0.502 ± 0.108 0.399 ± 0.206

0.81 ± 0.047 0.826 ± 0.029 0.869 ± 0.037

0.542 ± 0.129 0.656 ± 0.131 0.572 ± 0.096

0.583 ± 0.236 0.624 ± 0.149 0.335 ± 0.151

0.525 ± 0.058 0.345 ± 0.088 0.755 ± 0.045

Table 2. Average cosine similarities between real and generated concepts whose CAVs are computed using Visual-TCAV. Results are aggregated by averaging across four architectures and two layers each (i.e., the outputs of the final two blocks for ResNet-50-V2, InceptionV3, and ConvNeXt, and the outputs of the last two convolutional layers of VGG-16).

C. Average Cosine Similarity per Concept using TCAV

Class

beer glass

church

crossword puzzle

Concept

bubbly

glass

water

cloister

steeple

chequered

Flux GPT-Image-1 SD 3.5

0.416 ± 0.137 0.563 ± 0.056 0.526 ± 0.1

0.275 ± 0.06 0.249 ± 0.046 0.258 ± 0.086

0.438 ± 0.049 0.517 ± 0.082 0.402 ± 0.056

0.689 ± 0.065 0.11 ± 0.032 0.74 ± 0.078

0.515 ± 0.108 0.779 ± 0.047 0.742 ± 0.056

0.151 ± 0.098 0.542 ± 0.051 0.746 ± 0.025

electric guitar

fountain pen

Concept

Class

newspaper

crossword puzzle paper

dotted

spotted

guitarist

pen

Flux GPT-Image-1 SD 3.5

0.535 ± 0.055 0.541 ± 0.04 0.759 ± 0.041

0.4 ± 0.068 0.376 ± 0.056 0.426 ± 0.037

0.448 ± 0.06 0.597 ± 0.099 0.568 ± 0.069

0.259 ± 0.119 0.443 ± 0.12 0.395 ± 0.133

0.693 ± 0.098 0.564 ± 0.072 0.724 ± 0.094

0.655 ± 0.11 0.237 ± 0.171 0.539 ± 0.072

Class

dalmatian

golf ball

honeycomb

Concept

grass

plastic

sphere

crystalline

honeycombed

meshed

Flux GPT-Image-1 SD 3.5

0.401 ± 0.159 0.285 ± 0.181 0.444 ± 0.131

0.428 ± 0.049 0.166 ± 0.134 0.362 ± 0.051

0.282 ± 0.036 0.263 ± 0.071 0.413 ± 0.049

0.413 ± 0.052 0.399 ± 0.05 0.376 ± 0.043

0.802 ± 0.05 0.77 ± 0.096 0.853 ± 0.037

0.584 ± 0.061 0.271 ± 0.071 0.659 ± 0.063

Class

honeycomb

leopard

overskirt

Concept

perforated

feline

leopard print

rodent

frilly

woven

Flux GPT-Image-1 SD 3.5

0.372 ± 0.053 0.61 ± 0.071 0.572 ± 0.065

0.387 ± 0.104 0.644 ± 0.053 0.485 ± 0.047

0.383 ± 0.256 0.878 ± 0.016 0.57 ± 0.202

0.296 ± 0.231 0.477 ± 0.086 0.575 ± 0.107

0.431 ± 0.09 0.454 ± 0.106 0.413 ± 0.102

0.405 ± 0.14 0.236 ± 0.102 0.706 ± 0.038

Class

overskirt

spider web

stone wall

Concept

wrinkled

foliage

asparagus

cracked

lichen

stone

Flux GPT-Image-1 SD 3.5

0.31 ± 0.061 0.313 ± 0.058 0.488 ± 0.06

0.366 ± 0.12 0.676 ± 0.048 0.368 ± 0.101

0.545 ± 0.071 0.107 ± 0.105 0.631 ± 0.093

0.585 ± 0.071 0.496 ± 0.257 0.666 ± 0.073

0.255 ± 0.133 0.523 ± 0.038 0.666 ± 0.055

0.309 ± 0.057 0.549 ± 0.045 0.318 ± 0.125

Class

tench

trench coat

waffle iron

Concept

fin

grandfather

fabric

leather

cast iron

kitchen

Flux GPT-Image-1 SD 3.5

0.08 ± 0.046 0.116 ± 0.071 0.099 ± 0.037

0.595 ± 0.059 0.541 ± 0.095 0.515 ± 0.062

0.565 ± 0.081 0.523 ± 0.051 0.562 ± 0.041

0.542 ± 0.11 0.528 ± 0.041 0.537 ± 0.095

0.162 ± 0.065 0.161 ± 0.078 0.245 ± 0.166

0.797 ± 0.016 0.25 ± 0.053 0.653 ± 0.03

Class

waffle iron

wooden spoon

zebra

Concept

metal

waffled

wood

striped

tattoo

Flux GPT-Image-1 SD 3.5

0.282 ± 0.022 0.411 ± 0.042 0.398 ± 0.036

0.759 ± 0.059 0.818 ± 0.075 0.835 ± 0.043

0.333 ± 0.103 0.582 ± 0.053 0.395 ± 0.075

0.462 ± 0.092 0.628 ± 0.089 0.376 ± 0.132

0.444 ± 0.033 0.193 ± 0.088 0.662 ± 0.031

Table 3. Average cosine similarities between real and generated concepts whose CAVs are computed using TCAV. Results are aggregated by averaging across four architectures and two layers each (i.e., the outputs of the final two blocks for ResNet-50-V2, Inception-V3, and ConvNeXt, and the outputs of the last two convolutional layers of VGG-16).

D. Average Importances per Concept using VisualTCAV

Class

beer glass

church

crossword puzzle

Concept

bubbly

glass

water

cloister

steeple

chequered

Real Flux GPT-Image-1 SD 3.5

0.027 ± 0.018 0.011 ± 0.011 0.006 ± 0.007 0.012 ± 0.01

0.104 ± 0.04 0.081 ± 0.058 0.085 ± 0.038 0.069 ± 0.058

0.042 ± 0.039 0.03 ± 0.02 0.001 ± 0.001 0.003 ± 0.003

0.07 ± 0.052 0.075 ± 0.055 0.015 ± 0.012 0.054 ± 0.036

0.111 ± 0.056 0.112 ± 0.052 0.082 ± 0.041 0.067 ± 0.034

0.086 ± 0.077 0.02 ± 0.021 0.022 ± 0.015 0.076 ± 0.092

electric guitar

fountain pen

Class

crossword puzzle

dalmatian

Concept

newspaper

paper

dotted

spotted

guitarist

pen

Real Flux GPT-Image-1 SD 3.5

0.123 ± 0.082 0.049 ± 0.052 0.093 ± 0.069 0.089 ± 0.089

0.039 ± 0.032 0.058 ± 0.045 0.013 ± 0.013 0.035 ± 0.046

0.012 ± 0.014 0.004 ± 0.009 0.005 ± 0.006 0.0 ± 0.0

0.171 ± 0.146 0.032 ± 0.049 0.027 ± 0.033 0.018 ± 0.028

0.149 ± 0.113 0.134 ± 0.085 0.089 ± 0.09 0.167 ± 0.079

0.04 ± 0.014 0.057 ± 0.027 0.032 ± 0.011 0.055 ± 0.043

Class

golf ball

honeycomb

Concept

grass

plastic

sphere

crystalline

honeycombed

meshed

Real Flux GPT-Image-1 SD 3.5

0.008 ± 0.01 0.004 ± 0.008 0.014 ± 0.011 0.003 ± 0.002

0.049 ± 0.028 0.014 ± 0.013 0.008 ± 0.01 0.013 ± 0.016

0.148 ± 0.107 0.064 ± 0.038 0.065 ± 0.025 0.051 ± 0.031

0.03 ± 0.026 0.058 ± 0.04 0.018 ± 0.019 0.012 ± 0.006

0.193 ± 0.136 0.156 ± 0.073 0.11 ± 0.074 0.193 ± 0.121

0.13 ± 0.124 0.106 ± 0.117 0.025 ± 0.035 0.028 ± 0.019

Class

honeycomb

Concept

perforated

feline

leopard print

rodent

frilly

woven

Real Flux GPT-Image-1 SD 3.5

0.095 ± 0.07 0.049 ± 0.034 0.064 ± 0.05 0.04 ± 0.024

0.125 ± 0.074 0.049 ± 0.04 0.056 ± 0.053 0.038 ± 0.032

0.128 ± 0.063 0.063 ± 0.083 0.067 ± 0.056 0.088 ± 0.076

0.027 ± 0.029 0.012 ± 0.011 0.012 ± 0.016 0.012 ± 0.015

0.025 ± 0.022 0.034 ± 0.031 0.006 ± 0.009 0.057 ± 0.035

0.003 ± 0.004 0.008 ± 0.011 0.003 ± 0.003 0.005 ± 0.008

Class

overskirt

spider web

Concept

wrinkled

foliage

asparagus

cracked

lichen

stone

Real Flux GPT-Image-1 SD 3.5

0.02 ± 0.024 0.03 ± 0.03 0.003 ± 0.005 0.013 ± 0.011

0.015 ± 0.014 0.005 ± 0.006 0.003 ± 0.006 0.008 ± 0.009

0.02 ± 0.019 0.004 ± 0.004 0.002 ± 0.001 0.003 ± 0.003

0.074 ± 0.061 0.049 ± 0.032 0.034 ± 0.033 0.03 ± 0.025

0.058 ± 0.071 0.006 ± 0.005 0.006 ± 0.004 0.013 ± 0.01

0.1 ± 0.08 0.067 ± 0.043 0.027 ± 0.02 0.063 ± 0.019

Class

leopard

overskirt

stone wall

tench

trench coat

waffle iron

Concept

fin

grandfather

fabric

leather

cast iron

kitchen

Real Flux GPT-Image-1 SD 3.5

0.055 ± 0.034 0.002 ± 0.002 0.002 ± 0.002 0.006 ± 0.011

0.036 ± 0.036 0.013 ± 0.01 0.023 ± 0.019 0.017 ± 0.011

0.028 ± 0.023 0.018 ± 0.016 0.006 ± 0.007 0.005 ± 0.004

0.015 ± 0.012 0.007 ± 0.005 0.004 ± 0.004 0.001 ± 0.0

0.13 ± 0.081 0.041 ± 0.029 0.028 ± 0.023 0.027 ± 0.022

0.019 ± 0.013 0.018 ± 0.011 0.048 ± 0.041 0.024 ± 0.013

Class

waffle iron

wooden spoon

zebra

Concept

metal

waffled

wood

striped

tattoo

Real Flux GPT-Image-1 SD 3.5

0.052 ± 0.034 0.027 ± 0.02 0.011 ± 0.011 0.024 ± 0.022

0.098 ± 0.059 0.06 ± 0.044 0.061 ± 0.042 0.095 ± 0.041

0.033 ± 0.032 0.025 ± 0.02 0.025 ± 0.026 0.014 ± 0.011

0.195 ± 0.148 0.111 ± 0.076 0.089 ± 0.08 0.033 ± 0.035

0.028 ± 0.026 0.012 ± 0.009 0.009 ± 0.009 0.04 ± 0.043

Table 4. Average importance scores for each concept computed with Visual-TCAV. Results are aggregated by averaging across four architectures and two layers each (i.e., the outputs of the final two blocks for ResNet-50-v2, Inception-V3, and ConvNeXt, and the outputs of the last two convolutional layers of VGG-16).

E. Average Importances per Concept using TCAV Class

beer glass

church

crossword puzzle

Concept

bubbly

glass

water

cloister

steeple

chequered

Real Flux GPT-Image-1 SD 3.5

0.72 ± 0.301 0.635 ± 0.486 0.789 ± 0.323 0.721 ± 0.419

0.872 ± 0.185 0.858 ± 0.293 0.978 ± 0.044 0.654 ± 0.407

0.529 ± 0.452 0.676 ± 0.429 0.122 ± 0.144 0.397 ± 0.315

0.789 ± 0.301 0.929 ± 0.137 0.71 ± 0.347 0.84 ± 0.22

0.995 ± 0.011 0.982 ± 0.036 0.979 ± 0.026 0.985 ± 0.039

0.638 ± 0.401 0.81 ± 0.22 0.65 ± 0.413 0.678 ± 0.419

electric guitar

fountain pen

Class

crossword puzzle

dalmatian

Concept

newspaper

paper

dotted

spotted

guitarist

pen

Real Flux GPT-Image-1 SD 3.5

0.863 ± 0.223 0.715 ± 0.409 0.764 ± 0.424 0.716 ± 0.433

0.638 ± 0.416 0.678 ± 0.409 0.702 ± 0.301 0.922 ± 0.174

0.166 ± 0.197 0.145 ± 0.164 0.282 ± 0.363 0.007 ± 0.02

0.939 ± 0.087 0.921 ± 0.091 0.42 ± 0.387 0.387 ± 0.401

0.99 ± 0.015 0.983 ± 0.035 0.948 ± 0.089 0.99 ± 0.021

0.825 ± 0.209 0.89 ± 0.148 0.988 ± 0.028 0.828 ± 0.111

Class

golf ball

honeycomb

Concept

grass

plastic

sphere

crystalline

honeycombed

meshed

Real Flux GPT-Image-1 SD 3.5

0.398 ± 0.422 0.368 ± 0.313 0.881 ± 0.166 0.428 ± 0.328

0.311 ± 0.36 0.45 ± 0.414 0.508 ± 0.379 0.636 ± 0.339

0.714 ± 0.26 0.735 ± 0.454 0.706 ± 0.442 0.833 ± 0.265

0.61 ± 0.407 0.9 ± 0.124 0.71 ± 0.285 0.645 ± 0.45

0.972 ± 0.037 0.968 ± 0.041 0.972 ± 0.037 0.983 ± 0.027

0.867 ± 0.096 0.67 ± 0.396 0.72 ± 0.387 0.802 ± 0.205

Class

honeycomb

Concept

perforated

feline

leopard print

rodent

frilly

woven

Real Flux GPT-Image-1 SD 3.5

0.698 ± 0.357 0.73 ± 0.44 0.727 ± 0.337 0.695 ± 0.422

0.968 ± 0.084 0.948 ± 0.104 0.935 ± 0.184 0.942 ± 0.147

0.977 ± 0.023 0.5 ± 0.367 0.97 ± 0.039 0.668 ± 0.421

0.78 ± 0.379 0.562 ± 0.476 0.878 ± 0.293 0.893 ± 0.252

0.474 ± 0.18 0.959 ± 0.078 0.513 ± 0.494 0.98 ± 0.05

0.048 ± 0.061 0.628 ± 0.488 0.61 ± 0.506 0.054 ± 0.058

leopard

overskirt

Class

overskirt

spider web

Concept

wrinkled

foliage

asparagus

cracked

lichen

stone

Real Flux GPT-Image-1 SD 3.5

0.27 ± 0.255 0.645 ± 0.418 0.117 ± 0.182 0.439 ± 0.374

0.455 ± 0.421 0.578 ± 0.397 0.25 ± 0.337 0.46 ± 0.383

0.468 ± 0.408 0.47 ± 0.46 0.537 ± 0.395 0.395 ± 0.402

0.683 ± 0.429 0.68 ± 0.435 0.852 ± 0.217 0.635 ± 0.426

0.707 ± 0.426 0.583 ± 0.453 0.815 ± 0.299 0.708 ± 0.442

0.675 ± 0.411 0.748 ± 0.461 0.625 ± 0.409 0.985 ± 0.035

Class

stone wall

tench

trench coat

waffle iron

Concept

fin

grandfather

fabric

leather

cast iron

kitchen

Real Flux GPT-Image-1 SD 3.5

0.572 ± 0.406 0.467 ± 0.239 0.575 ± 0.332 0.7 ± 0.331

0.502 ± 0.382 0.535 ± 0.414 0.565 ± 0.356 0.553 ± 0.381

0.4 ± 0.252 0.518 ± 0.405 0.552 ± 0.344 0.32 ± 0.245

0.375 ± 0.12 0.17 ± 0.064 0.222 ± 0.332 0.04 ± 0.037

0.74 ± 0.231 0.938 ± 0.073 0.84 ± 0.306 0.68 ± 0.433

0.805 ± 0.176 0.867 ± 0.133 0.942 ± 0.074 0.93 ± 0.109

Class

waffle iron

wooden spoon

zebra

Concept

metal

waffled

wood

striped

tattoo

Real Flux GPT-Image-1 SD 3.5

0.69 ± 0.3 0.685 ± 0.435 0.49 ± 0.392 0.513 ± 0.356

0.705 ± 0.269 0.832 ± 0.224 0.785 ± 0.23 0.84 ± 0.204

0.368 ± 0.271 0.718 ± 0.424 0.5 ± 0.382 0.55 ± 0.41

0.925 ± 0.101 0.993 ± 0.01 0.9 ± 0.115 0.617 ± 0.396

0.347 ± 0.374 0.527 ± 0.4 0.465 ± 0.344 0.415 ± 0.398

Table 5. Average importance scores for each concept computed with TCAV. Results are aggregated by averaging across four architectures and two layers each (i.e., the outputs of the final two blocks for ResNet-50-v2, Inception-V3, and ConvNeXt, and the outputs of the last two convolutional layers of VGG-16).

F. Generated Concepts Below is a randomly selected subset of the synthetic concepts generated in this study.

F.1. Glass Concept

(a) Real glass concept images from FMD.

(b) Glass concept images generated with Flux 1.1.

(d) Glass concept images generated with Stable Diffusion 3.5 Medium.

Figure 10. Comparison of real glass images and generated concept images across various models.

F.2. Water Concept

(a) Real water concept images from FMD.

(b) Water concept images generated with Flux 1.1.

(d) Water concept images generated with Stable Diffusion 3.5 Medium.

Figure 11. A comparison of real images (top row) with the corresponding concept images produced by the different generative models: Flux 1.1, GPT-Image-1, Stable Diffusion 3.5 Medium.

F.3. Bubbly Concept

(a) Real bubbly concept images from DTD.

(b) Bubbly concept images generated with Flux 1.1.

(d) Bubbly concept images generated with Stable Diffusion 3.5 Medium.

Figure 12. Real bubbly images from the DTD dataset (top row) compared with concept images generated by Flux 1.1, GPT-Image-1, and Stable Diffusion 3.5 Medium (subsequent rows).

F.4. Cloister Concept

(a) Real cloister concept images from ImageNet.

(b) Cloister concept images generated with Flux 1.1.

(d) Cloister concept images generated with Stable Diffusion 3.5 Medium.

Figure 13. A comparative example for the cloister concept. The first subfigure shows five authentic cloister images from the ImageNet dataset. Subsequent ones display five synthetic renditions of the same concept, generated respectively by Flux 1.1, GPT-Image-1, and Stable Diffusion 3.5 Medium.

F.5. Steeple Concept

(a) Real steeple concept images from ImageNet.

(b) Steeple concept images generated with Flux 1.1.

(d) Steeple concept images generated with Stable Diffusion 3.5 Medium.

Figure 14. Real steeple images from ImageNet compared with synthetic steeple images generated by Flux 1.1, GPT-Image-1, and Stable Diffusion 3.5 Medium. Each row contains five representative samples of the same concept.

F.6. Chequered Concept

(a) Real chequered concept images from DTD.

(b) Chequered concept images generated with Flux 1.1.

(d) Chequered concept images generated with Stable Diffusion 3.5 Medium.

Figure 15. Chequered concept images. Real DTD examples; images generated by Flux 1.1; images generated by GPT-Image-1; images generated by Stable Diffusion 3.5 Medium.

F.7. Newspaper Concept

(a) Real newspaper concept images from search engine results.

(b) Newspaper concept images generated with Flux 1.1.

(d) Newspaper concept images generated with Stable Diffusion 3.5 Medium.

Figure 16. Newspaper concept images. Row 1 shows five real samples; rows 2–4 display five synthetic images produced by Flux 1.1, GPT-Image-1, and Stable Diffusion 3.5 Medium, respectively.

F.8. Paper Concept

(a) Real paper concept images from FMD.

(b) Paper concept images generated with Flux 1.1.

(d) Paper concept images generated with Stable Diffusion 3.5 Medium.

Figure 17. Comparison of paper concepts: real FMD examples, Flux 1.1 generated images, GPT-Image-1 generated images, and Stable Diffusion 3.5 Medium generated images.

F.9. Dotted Concept

(a) Real dotted concept images from DTD.

(b) Dotted concept images generated with Flux 1.1.

(d) Dotted concept images generated with Stable Diffusion 3.5 Medium.

Figure 18. Real dotted concept images from the DTD dataset; images produced with Flux 1.1; images produced with GPT-Image-1; images produced with Stable Diffusion 3.5 Medium.

F.10. Spotted Concept

(a) Real spotted concept images from search engine results.

(b) Spotted concept images generated with Flux 1.1.

(d) Spotted concept images generated with Stable Diffusion 3.5 Medium.

Figure 19. Real and synthetic spotted-pattern concept images.

F.11. Grass Concept

(a) Real grass concept images from search engine results.

(b) Grass concept images generated with Flux 1.1.

(d) Grass concept images generated with Stable Diffusion 3.5 Medium.

Figure 20. Grass concept images: real photographs, Flux 1.1 outputs, GPT-Image-1 outputs, and Stable Diffusion 3.5 Medium outputs.

F.12. Plastic Concept

(a) Real plastic concept images from FMD.

(b) Plastic concept images generated with Flux 1.1.

(d) Plastic concept images generated with Stable Diffusion 3.5 Medium.

Figure 21. Plastic concept images from real photographs, Flux 1.1, GPT-Image-1, and Stable Diffusion 3.5 Medium.

F.13. Sphere Concept

(a) Real sphere concept images from search engine results.

(b) Sphere concept images generated with Flux 1.1.

(d) Sphere concept images generated with Stable Diffusion 3.5 Medium.

Figure 22

G. Ablated Class Images Below is a randomly selected subset of the ablated class images used in this study.

G.1. Beer Glass Class

(a) Original

(b) No bubbly

(d) No beer

Figure 23. Example of concept removal from a beer glass class image, eliminating bubbly, glass, and beer concepts.

G.2. Church Class

(a) Original

(b) No steeple

Figure 24. Example of concept removal from a church class image, eliminating the steeple concept and the cloister concept.

G.3. Crossword Puzzle Class

(a) Original

(b) No chequered

(d) No paper

Figure 25. Example of concept removal from a crossword puzzle class image, eliminating chequered, newspaper, and paper concepts.

G.4. Dalmatian Class

(a) Original

(b) No dotted

Figure 26. Example of concept removal from a dalmatian class image, eliminating dotted and spotted concepts.

A Framework for Evaluating Zero-Shot Image Generation in Concept-based Explainability

Analyze & explore related research in Conceptio

Related documents