On the Faithfulness of Post-Hoc Concept Bottleneck Models Laines Schmalwasser1,2 , Jan Blunk2,3⋆ , Niklas Penzel2 , Julia Niebling1 , and Joachim Denzler2 Institute of Data Science, German Aerospace Center, Jena, Germany Computer Vision Group Jena, Friedrich Schiller University Jena, Germany 3 GEOMAR Helmholtz Centre for Ocean Research Kiel, Germany Correspondence to [email protected] 1
arXiv:2606.30498v1 [cs.CV] 29 Jun 2026
2
Abstract. Human decision-making interprets the world through highlevel concepts, such as recognizing a bird by its belly color. To bridge the gap between opaque deep learning representations and human understanding, Post-Hoc Concept Bottleneck Models (post-hoc CBMs) project latent features onto interpretable concept spaces using auxiliary datasets or vision-language models. However, relying on target task accuracy as the primary measure of post-hoc CBM success obscures whether the learned concepts are semantically meaningful or merely predictive artifacts. For example, random concept projections can achieve competitive accuracy despite being semantically meaningless. In this work, we analyze the learned projections directly and identify two failure cases: First, for concept projections learned from auxiliary data, covariate shifts can lead to unfaithful concept representations for the target task. In particular, we provide an upper bound on the error introduced by this shift. Second, systematic label noise in surrogate concept labels generated by visionlanguage models leads to unfaithful projections. After formalizing these failure modes, we introduce novel metrics that decouple concept faithfulness from predictive accuracy. Our empirical results across real-world and synthetic benchmarks confirm that these metrics identify unfaithful behaviors that standard accuracy-based evaluation fails to detect4 . Keywords: Concept Bottleneck Models · Faithfulness · Interpretability
1
Introduction
Human understanding is fundamentally built on concepts [6, 18, 47]. Rather than analyzing raw sensory data, we identify objects through high-level attributes, such as recognizing a specific bird species by its “yellow belly” or “black crown.” In contrast, deep neural networks learn opaque, high-dimensional representations that often lack semantic clarity [7,32,37]. To bridge this gap and enforce interpretability, Concept Bottleneck Models (CBMs) were introduced [1, 2, 11, 17, 28, 32, 46, 67, 68]. ⋆ 4
Work done while the author was at Computer Vision Group Jena. Project page: https://posthoc-cbm-faithfulness.github.io/
2
L. Schmalwasser et al. 1 Train πθ on Auxiliary Dataset Xaux ○
Auxiliary Concept Set Xaux
Covariate Shift Ptask ̸= Paux
Frozen Backbone
Activation Space A
Problem: Missing Labels to Train π θ Concept Space C
a2
Task Dataset Xtask
a1
f
1.0 green 0.1 .. .. . .
πθ
a3
triangle
0.9
Class Space Y 0.9 . ... .. .. .. . .
red-striped-triangle
red
h
blue-solid-plus
0.0
2 Generate Surrogate Labels With VLM to Train πθ ○
Fig. 1: In the post-hoc CBM setup [49, 63, 71], inputs are mapped to activations using some frozen backbone model f . Then, a learned concept projection πθ extracts concepts before a final classifier h predicts a class based on these concepts. Unfortunately, concept 1 auxiliary concept labels in the downstream task domain are often unavailable. Thus, ○ 2 surrogate VLM labels are used [49, 63, 71]. We study these sets (e.g., Broden [7]) or ○ 1 a covariate approaches and identify two reasons for potential unfaithfulness of πθ : ○ 2 systematic surrogate label errors. shift of the concept set, and ○
CBMs constrain the model to first predict a set of human-understandable concepts from the input, and then base the final prediction solely on them, thereby promising transparent decision-making [25, 41, 43]. However, standard CBMs require dense concept annotations for the specific downstream task, which are rarely available in practice [15, 32, 49]. Post-Hoc Concept Bottleneck Models (post-hoc CBMs) circumvent this by freezing a pretrained feature extractor and learning only a lightweight projection from its internal representations to a concept space and final classifier [49, 63, 71] (we use post-hoc CBM to denote this general model class and PCBM for the concrete implementation in [71]). Crucially, post-hoc CBMs tackle the need for in-domain concept labels via two strategies: (1) Auxiliary Concept Datasets, where the projection is learned on a separate, richly annotated dataset (e.g., Broden [7]) before application to the target task [71]; or (2) Surrogate Labels from VLMs, where a Vision-Language Model (e.g., CLIP [55]) generates surrogate concept labels [49,50,57,63]. Both drastically reduce the annotation burden while maintaining competitive downstream classification accuracy [49, 71]. Unfortunately, evaluating downstream accuracy provides limited insight into the quality of the learned concept projection. Prior work empirically shows that random concepts achieve high performance [45]. In [42], this phenomenon is connected to the Johnson-Lindenstrauss (JL) lemma [26], with the expected error approaching zero as random projections increase [63]. We expand these insights via the smooth manifold JL lemma [5], based on previous findings that neural activations tend to lie on low-dimensional manifolds [3, 54]. We show that under certain assumptions, the original activations can be reconstructed from a modest number of random projections, resulting in high downstream performance. Consequently, we must investigate the concept projection itself to evaluate whether it faithfully extracts the intended concepts on the target distribution. Here, we analyze the two primary post-hoc CBM training mechanisms (Figure 1).
On the Faithfulness of Post-Hoc Concept Bottleneck Models
3
First, for auxiliary concept datasets, we identify covariate shift as the primary source of unfaithfulness. We demonstrate that even if a concept’s semantic definition is identical across domains, a geometric shift in the feature space can invalidate the learned projection on the downstream task. We formalize this by providing an upper bound for the error introduced by this shift based on [8] and propose an approximation of it as a practical measure of faithfulness. Second, for methods using VLM-based surrogate labels, the primary problem is label noise. However, our analysis shows that unfaithfulness is not solely caused by noise magnitude, but by systematic errors. In other words, random mistakes can cancel each other out, but the surrogate labeling function leads to unfaithfulness when mistakes are consistent, e.g., always predicting “sky” and “clouds” together. To detect this, we measure the surrogate-label error and propose a metric that explicitly quantifies the corresponding systematicity. We empirically validate our theoretical findings on standard benchmarks (CUB-200 [70] and CIFAR-10/100 [33]) and the synthetic Elements [48] dataset, which provides known ground-truth concepts. Our evaluation includes a range of post-hoc CBMs across standard backbones representing the two main training paradigms: those using auxiliary datasets [71] (e.g., Broden [7]) and those using surrogate labels generated by VLMs (LFCBM [49] and VLG-CBM [63] based on CLIP [55], DINOv3 [61] and Grounding DINO [39]). We first show that random concept projections can achieve competitive accuracy, confirming that downstream accuracy is an insufficient metric for meaningful representations. Then, we directly evaluate the learned concept projections using available ground truth [48], apply our proposed faithfulness metrics, and demonstrate that they successfully identify failures caused by covariate shifts and systematic errors, offering a practical tool for evaluating post-hoc CBMs beyond predictive performance. Our contributions can be summarized as follows: 1. We show that under certain assumptions, random concept projections can achieve competitive performance. 2. We identify and formalize two critical sources of unfaithfulness in the concept projections of post-hoc CBMs: covariate shifts when learning with auxiliary concept sets and systematic label noise in surrogate supervision. 3. We propose a set of novel faithfulness metrics that evaluate concept alignment independently of the predictive performance, revealing specific failure cases.
2
Related Work
post-hoc CBM evaluations are complicated by information leakage [20, 41, 43, 59, 65], where models bypass semantic concepts to base predictions on raw backbone activations. Notably, [59] links this phenomenon to the Johnson-Lindenstrauss lemma [26], showing that sufficient random projections preserve pairwise distances for finite sets. We note that the manifold Johnson-Lindenstrauss lemma [5] implies this holds for the complete activation manifold under the manifold hypothesis [3, 54]. Related to this, [63] studies the expected error of random projections for final predictions. Based on these findings, we emphasize the need
4
L. Schmalwasser et al.
to evaluate learned concept projections directly. Crucially, the faithfulness issues we identify persist even in the absence of information leakage. Other works similarly study CBM and post-hoc CBM limitations [15, 24, 40, 56] and concept faithfulness [34, 35]. While [34] analyzes the faithfulness of unsupervised concept explanations, it explicitly excludes post-hoc CBMs (our focus). For surrogate labels, [35] highlights flaws in GPT-3-derived concept sets [10] for label-free CBMs [49]. We generalize this by providing an alternative mechanism to analyze surrogate-label errors in VLM-based post-hoc CBMs. In concept embedding models [15], the concept bottleneck is constructed from two learned embeddings per concept to address the accuracy-interpretability trade-off. In contrast to our evaluation metrics, [24] studies CBM faithfulness by matching input regions to concepts. Finally, [56] empirically observes issues caused by distribution shifts in concept datasets. We provide an upper bound on the risk of post-hoc CBMs trained on auxiliary data, approximating the resulting generalization error. In contrast, [40] bounds the CBM risk on downstream tasks using the underlying backbone.
3
Method
We consider a standard deep learning setting where a fixed backbone f : X → A ⊆ Rd maps inputs to a high-dimensional latent activation space. While a classifier g : A → Y trained on top of f may achieve high performance, its decision-making process within the high-dimensional space A remains opaque. Post-Hoc Concept Bottleneck Models (post-hoc CBMs) [49, 63, 71] aim to make this process interpretable by introducing an intermediate concept bottleneck while holding f fixed. Specifically, they learn a concept projection πθ : A → C, parameterized by θ, that maps activations to a K-dimensional space C ⊆ RK of human-understandable concepts (typically K ≪ d). A subsequent classifier h : C → Y (e.g., a sparse linear layer) predicts target classes based on these concept scores, yielding the final model h ◦ πθ ◦ f . To construct the bottleneck layer, we initially assume access to a training set Xtask = {xi , ci }N i=1 sampled i.i.d. from a task distribution Ptask (X , C) over inputs and concept vectors, allowing us to optimize θ to recover these task-relevant concepts. Definition 1. We define a concept projection πθ as faithful if it minimizes the expected risk over the true task distribution Ptask (x, c): \label {eq:faithful} \argmin _\theta \mathbb {E}_{(x,c) \sim P_\mathrm {task}} \left [ \mathcal {L}(\pi _\theta (f(x)), c)\right ],
(1)
for some loss function L measuring the alignment of the predicted concepts πθ (f (x)) and the ground truth concepts c for an input x. In this work, we study the post-hoc concept bottleneck framework from this perspective of concept faithfulness, i.e., whether πθ extracts meaningful concepts after Definition 1. A faithful projection ensures πθ (f (x)) = EPtask [c | x],
On the Faithfulness of Post-Hoc Concept Bottleneck Models
5
recovering the intended concepts in expectation. Note that our formalization of faithfulness differs from similar notions in information leakage, e.g., [42, 43, 59], and aligns more closely with the informal version in [15]. First, we confirm that evaluating the predictive performance of h is insufficient to assess the faithfulness of the post-hoc CBM in Section 3.1. After establishing that downstream accuracy does not imply a faithful concept projection, we formalize the learning problem of πθ in Section 3.2 to derive direct faithfulness measures. We identify two primary sources of unfaithfulness: (1) covariate shifts from auxiliary training datasets, which we quantify via an upper bound on the generalization error; and (2) systematic label noise from VLM-generated surrogate concept labels. For surrogate labels, we show that unfaithfulness requires nonrandom error structures rather than magnitude alone, and derive a metric to explicitly quantify this systematicity. 3.1
Downstream Performance and Faithfulness
To demonstrate that high post-hoc CBM performance does not imply semantic alignment, we study random projections πθ without semantic meaning, e.g., independently sampled standard-normal coefficients in a linear layer. Previous work empirically shows that PCBMs [45] using such πθ can achieve high downstream performance. In [42], the authors note that such projections are geometrypreserving with high probability for finite sets of samples under the JohnsonLindenstrauss (JL) lemma [26] if the embedding dimension K is large enough. In other words, the complete backbone information is encoded into the concept activation, which is related to the area of information leakage [20, 42, 43, 59]. Here, we first link this to the manifold hypothesis, then derive a concrete mechanism to achieve high downstream performance under additional assumptions. In particular, consider the manifold hypothesis of deep learning [3, 54], i.e., that the activations in neural networks lie on or close to a manifold with an intrinsic dimension of m ≪ d. If this hypothesis holds, then the manifold version of the JL lemma (Theorem 3.1 in [5]) governs the behavior under random projections. Crucially, it establishes a connection between certain characteristics of this activation manifold and the number of random projections needed to preserve pairwise distances. We make this explicit in Appendix A.1. Nevertheless, if geometry is preserved, the original activations can be recovered [5]. Thus, for a sufficiently complex classifier h (e.g., an MLP [23]) it is possible to learn g ◦ πθ−1 . In such a case, the meaningless concepts by the random πθ would be used to recover the original activations, leading to high downstream performance. To make this argument more explicit, we next derive a concrete solution for h under an additional linearity assumption. Constructive Proof for a Linear Activation Subspace. While the discussion under the manifold hypothesis posits a possible solution for h, we now derive a constructive solution under the assumption that the activation manifold is a linear subspace of Rd . Our concrete assumptions are:
6
L. Schmalwasser et al.
(1) Linear Subspace: The activations lie on an m-dimensional subspace S ⊂ Rd with orthonormal basis U , such that f (x) = a = U z for some latent z ∈ Rm . (2) Linear Backbone: The backbone classifier g is linear prior to a softmax, i.e., g(a) = softmax(Wg a). This reflects the standard post-hoc CBM setup of inserting the concept bottleneck at the penultimate layer, e.g., [32, 56, 71]. (3) Random Projection: The projection is defined as πθ (a) = σ(P a), where P ∈ RK×d is a fixed Gaussian matrix (K ≥ m) and σ is bijective (e.g., tanh). Under these assumptions, we can construct a classifier h that exactly recovers the original prediction g(f (x)): h(c) := g(a_{\mathrm {rec}}(c)) = \mathrm {softmax}(W_g \cdot [\underbrace {U (P U)^\dagger \sigma ^{-1}(c)}_{\text {Reconstructed } a \in \mathcal {A}}]),
(2)
where arec : C → S reconstructs the activations a = arec (πθ (a)) and † denotes the Moore-Penrose pseudoinverse [52]. We provide the full derivation in Appendix A.2. Consequences. Under the discussed assumptions, h does not need to learn a decision based on semantic concepts. Instead, a degenerate solution is to reconstruct the original activations a before applying g. This aligns with [63], who demonstrated that random concept projections achieve zero expected error as the concept dimension K approaches the activation dimension d. Our analysis adds to this result in two aspects: First, under the manifold hypothesis (m ≪ d) [3, 54], the manifold JL lemma [5] implies that K need not approach d to enable recovery. Second, beyond bounding the expected error [63], our constructive derivation under additional linearity assumption using [52] demonstrates that exact, sample-wise reconstruction may be possible. Consequently, if h is optimized rather than fixed, downstream accuracy becomes a poor proxy for concept faithfulness after Definition 1. This necessitates a direct analysis of the learned projection πθ , which we formalize next. 3.2
Faithfulness of the Learned Concept Projection
Assuming access to a training set Xtask = {xi , ci }N i=1 for πθ sampled i.i.d. from the task distribution Ptask , we minimize the expected risk Jtask (θ; L): \label {eq:exp-risk} J_\mathrm {task}(\theta ;\mathcal {L}) &= \mathbb {E}_{(x,c) \sim P_{\mathrm {task}}} \left [ \mathcal {L}(\pi _\theta (f(x)), \, c) \right ] \\ &\approx \frac {1}{N} \sum _{i=1}^N \mathcal {L}(\pi _\theta (f(x_i)), \, c_i). (4) To interpret this optimization as a Maximum Likelihood Estimation (MLE), we assume that the projection πθ defines the parameters of a conditional distribution pθ (· | f (x)) over the concept space. The empirical risk minimization becomes equivalent to MLE if the loss function L matches the negative log-likelihood of observing the true concept c under this model distribution [9, 19]: \mathcal {L}(\pi _\theta (f(x)), \, c) = - \log p_{\theta }(c \mid f(x)) + \text {const.}
(5)
On the Faithfulness of Post-Hoc Concept Bottleneck Models
7
Consequently, the learned projection πθ represents the optimal estimator for the concept scores distributed according to Ptask (x, c) = Ptask (x)Ptask (c|x). It minimizes Equation (3) and is, therefore, faithful according to Definition 1. In practice, we identify two key problems that violate this theoretically optimal scenario: First, the dataset used to learn the concept projection may differ from the task we are interested in solving, e.g., [30, 58, 71]. Intuitively, using an auxiliary concept dataset rather than a task-relevant Xtask leads to covariate shift, i.e., Paux (x) ̸= Ptask (x). Second, to address this, prior work aims to assign task-relevant labels to in-domain training data, leading to surrogate labels c̃ with Ptask (c̃|x) ̸= Ptask (c|x), e.g., [49,63]. Both scenarios are distributional mismatches (infracting Definition 1) while fitting πθ , which we will detail next. Covariate Shift of Concept Datasets. Training the projection πθ requires access to a training set annotated with ground-truth concepts. Because these are rarely available for the downstream task data Xtask , we typically train πθ by minimizing the empirical risk on a smaller auxiliary dataset Xaux (e.g., Broden [7]) sampled from a distribution Paux [30, 71]. However, this introduces the risk of distribution shift (Ptask ̸= Paux ). Intuitively, images of a concept may appear visually different in Xaux than in Xtask , degrading the faithfulness of the learned concept projection after Definition 1. In particular, the optimization objective in Equation (3) changes to minimizing a corresponding Jaux . To quantify this effect, we utilize an upper bound for the target generalization error from domain adaptation theory (Theorem 2 in [8]): \label {eq:cov-shift-error} J_{\mathrm {task}}(\theta ;L_1) \leq J_{\mathrm {aux}}(\theta ;L_1) + \frac {1}{2} \hat {d}_{\mathcal {H}\Delta \mathcal {H}}\!(\mathbb {X}_{\mathrm {task}},\!\mathbb {X}_{\mathrm {aux}}) + \Omega (\mathrm {dim}_{\mathrm {VC}},\!N,\!\delta ) + \lambda _{\mathrm {ideal}}. (6) Here, the expected L1 (absolute) task risk Jtask (θ; L1 ) is bounded by the expected auxiliary risk Jaux (θ; L1 ) plus the scaled empirical H∆H-divergence dˆH∆H [8, 29] between the marginal distributions. The term Ω accounts for finite sample estimation and depends on the VC dimension dimVC , sample size N (assuming |Xaux | ≈ |Xtask |), and the desired confidence level δ. λideal represents the task’s adaptability (the combined error of the optimal hypothesis on both distributions). In concept probing, we assume this shift is primarily a Covariate Shift [60,64], meaning the semantic definition of the concept Paux (c|x) = Ptask (c|x) remains constant while the input marginals Paux (x) ̸= Ptask (x) change. Under this assumption, there exists a single optimal decision rule shared across distributions, rendering λideal negligible (full formal definitions and derivations in Appendix A.3). Consequences. When training on in-domain data is infeasible, we can estimate the faithfulness of πθ after Definition 1 on a given auxiliary dataset via this upper bound (Equation (6)). Assuming a fixed πθ architecture and equal dataset sizes (rendering Ω constant), and with λideal negligible, the empirical divergence dˆH∆H becomes the decisive metric for unfaithfulness. Following [8], we approximate dˆH∆H by training a domain discriminator to separate Xaux and Xtask . Intuitively, if a simple classifier can distinguish the auxiliary and task data, we expect a larger absolute task error Jtask (θ; L1 ), violating Definition 1.
8
L. Schmalwasser et al.
Faithfulness of Surrogate Label Functions. To avoid potential covariate shifts and the resulting generalization penalty created by training the projection πθ on a separate auxiliary dataset, it would be ideal to train πθ directly on the in-domain data Xtask sampled from Ptask using an oracle concept function c∗ : X → C with ∀(x, c) ∈ Xtask : c∗ (x) = c. However, because these ground-truth annotations are commonly unavailable in post-hoc settings, we must rely on a surrogate labeling function c̃ : X → C (e.g., derived from VLMs [49, 57, 63]; see Appendix A.4 for examples) to identify the concepts. Consequently, the optimization objective shifts to minimizing the empirical risk with respect to the surrogate labels: \tilde {J}_\mathrm {task}(\theta ;\mathcal {L}) = \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \mathcal {L}\left (\pi _\theta (f(x)), \, \tilde {c}(x) \right ) \right ].
(7)
The bottleneck’s faithfulness thus depends on whether the parameters θ̃ optimized for the surrogate c̃ also minimize the true risk with respect to c∗ . Theoretical Analysis. We analyze this condition by formulating πθ as a Multivariate Generalized Linear Model (GLM) [16, 44]. We assume the conditional distribution of the concepts belongs to the Multivariate Exponential Dispersion Family (EDF) [27, 44], which encompasses standard loss settings such as Mean Squared Error (Gaussian) and Cross-Entropy (Bernoulli/Multinomial). In ∗ Appendix A.5, we derive the gradient of the true objective Jtask (θ; L) at the surrogate optimum θ̃. For canonical link functions, this gradient is proportional to the outer product (⊗) of the surrogate error and backbone activations f (x): \begin {split} \label {eq:true_grad_surrogate} \nabla \! J^*_\mathrm {task}(\tilde {\theta };\mathcal {L}) \! &= \! \mathbb {E}_{P_{\mathrm {task}}} \! \big [ \nabla \! \mathcal {L}(\pi _{\tilde {\theta }}(f(x)), c^*(x)) \big ] \! \propto \! \mathbb {E}_{P_{\mathrm {task}}} \! \big [\! \underbrace { ( \tilde {c}(x) \! - \! c^*(x) ) }_{\text {\tiny Surrogate Error $\delta (x)$}} \! \otimes \! f(x) \big ]. \end {split}
(8)
For a faithful post-hoc CBM (i.e., θ̃ is a global minimum of the objective, see Definition 1), this gradient must be zero. This reveals two distinct mechanisms for faithfulness of πθ trained to emulate the surrogate label function c̃: (1) Surrogate Accuracy: The surrogate labels have to be accurate, i.e., c̃(x) ≈ c∗ (x), causing the surrogate error term δ(x) = c̃(x) − c∗ (x) to vanish. (2) Error Orthogonality: Further, Equation (8) implies that a model can remain faithful even with noisy surrogate labels if the surrogate error is orthogonal to the activation space in expectation. This requires the errors in the surrogate labels to be uncorrelated with the activations f (x). Geometric Interpretation. Figure 2 visualizes the orthogonality condition. We consider N training samples yielding activation vectors αj = [fj (x1 ), . . . ,fj (xN )]⊤ for backbone neuron j, and surrogate discrepancy vectors ∆k = [δk (x1 ), . . . ,δk (xN )]⊤ for concept k. Equation (8) shows that faithfulness does not strictly require perfect labels (∆k = 0). Instead, it is sufficient if the errors ∆k are orthogonal to the activations αj (⟨∆k , αj ⟩ ≈ 0). In other words, random, unsystematic annotation
On the Faithfulness of Post-Hoc Concept Bottleneck Models Value for Sample s
Value for Sample s ∆k
9
∆k
αj αj Projection ̸= 0 Projection = 0
Value for Sample r
(a) Faithful Model (Orthogonal Case)
Value for Sample r
(b) Unfaithful Model (Non-Orthogonal Case)
Fig. 2: Visualization of the error orthogonality condition in a 2D sample space. The axes represent two samples r and s from the training set of πθ . (a) A faithful model is learned if the surrogate label errors of these samples (∆k ) are orthogonal to their activations (αj ), resulting in a zero projection. (b) Non-orthogonal errors have a non-zero projection onto the feature vector, resulting in a non-zero gradient for the true objective.
noise in the surrogate labels can cancel out during training. However, systematic errors (such as a VLM consistently predicting the concept “Boat” whenever it detects a “Water” texture, even if no boat is present), create non-orthogonal error components. This results in a non-zero gradient for the true objective, causing the learned πθ̃ to diverge from the ground-truth semantics, violating Definition 1. Consequences. We therefore propose two metrics to evaluate the faithfulness of surrogate-based post-hoc CBMs. First, Surrogate Accuracy directly measures the raw discrepancy δ(x). Second, we evaluate the Alignment of Surrogate Errors and Activations using the absolute Pearson correlation [51] ρk,j = |corr(∆k , αj )|. A correlation of ρ ≈ 0 indicates orthogonal noise. Conversely, if |ρ| is large and significant (e.g., p < 0.05), the surrogate noise is systematically tied to the backbone features. Thus, a post-hoc CBM is unfaithful only if surrogate labels are inaccurate and those errors systematically correlate with backbone features.
4
Experiments
To validate our theoretical findings, we evaluate the faithfulness of concept projections πθ trained using various post-hoc CBM methods. First, we demonstrate that downstream performance is insufficient to assess concept bottleneck layers by studying random concept projections. Next, we analyze the learned πθ directly, leveraging a synthetic dataset with ground-truth concept information. This allows us to measure the concept accuracy and the geometric alignment of the concept bottleneck approaches. Using an optimal ground-truth classifier for the downstream task, we can align these results with downstream performance. Finally, we evaluate our two theoretically identified sources of reduced faithfulness: concept-set covariate shift and systematic surrogate labeling errors. 4.1
Experiment Setup
Datasets and Backbone Models. We validate our theoretical findings on standard real-world benchmarks: CUB-200 [70] using a ResNet18 backbone [21, 66] and
10
L. Schmalwasser et al.
(a) Example of Ptask
∀ (b) Example of Pnear
∃ (c) Example of Pnear
(d) Example of POOD
Fig. 3: Samples of the concept “striped” for each visual set used. In Figure 3a, there is exactly one object with the concept. Figure 3b shows a sample from the backbone domain where all objects share the concept. Figure 3c shows a sample from the training domain and Figure 3d shows a complete out-of-domain example.
CIFAR-10/100 [33] using CLIP ResNet-50 (RN50) [13] as backbone. Additionally, we consider the synthetic Elements dataset [48], which includes ground-truth concept annotations. Elements samples consist of objects defined by three latent attributes: shape, color, and texture (e.g., “red striped square”). To create a realistic shift between backbone pretraining and downstream application, we train the backbone f to identify these objects in a multi-object classification task following [48], whereas the downstream post-hoc CBM task involves classifying images containing only a single object. In all experiments, the backbone f remains frozen. Probing Datasets for Concept Learning. For standard PCBMs [71], we train the concept projection πθ on different auxiliary distributions, while keeping the semantic concept definitions fixed (visualized in Figure 3): – In-Domain: We train πθ using samples from the exact downstream distribution Ptask . For Elements, this corresponds to single-object images exhibiting the specific concept (e.g., all images containing “red”). – Near-Domain: Concepts are sampled from a distribution that is distinct ∀ from, but visually similar to Ptask . For Elements, we consider Pnear with multiple objects per image that all share concepts including the target, and ∃ Pnear , where at least one object exhibits the target concept. – Out-of-Distribution: Concepts are learned from a visually distinct distribution POOD (e.g., abstract patterns) that share the concept label but differ structurally from the downstream task. This represents the domain gap common in post-hoc analysis (e.g., datasets like Broden [7]). Post-Hoc Concept Bottleneck Models (post-hoc CBMs). In our experiments, we train post-hoc CBMs with two strategies: training πθ on an auxiliary concept dataset and using a VLM to generate surrogate concept labels. Specifically, we utilize a VLM (CLIP [55] and DINOv3 [61], each with a ViT-B/16 [14]) as the backbone f and learn πθ using the text embedding for each concept [71]. Additionally, we consider Label-Free CBMs (LFCBMs) [49], where surrogate
CIFAR-100
100
0.8
Accuracy (%)
80
0.6
60
0.4
40
0.2
20 0
Activation Reconstruction MSE (10 3)
On the Faithfulness of Post-Hoc Concept Bottleneck Models
0
200
400 600 Number of Concepts K
800
1000
0.0
11
Random Concept Projection Activation Reconstruction MSE Base Classifier Methods: VLG-CBM (Grounding DINO) LFCBM (CLIP) LFCBM (DINOv3) PCBM (Image Set) Concept Sources: GPT-3.5 ConceptNet 5.5 Broden
Fig. 4: High downstream accuracy does not imply faithfulness. The dashed line shows the classifier’s performance without a bottleneck layer. The downstream classification accuracy of a post-hoc CBM increases with the number of random concept vectors. Additionally, various post-hoc CBMs that use meaningful concept sources only marginally outperform random concepts. Hence, downstream performance does not indicate faithful concept representations, which aligns with our theoretical findings in Section 3.1.
concept labels for Ptask are generated by computing the similarity between the CLIP-representation [55] of the input image and the text embeddings for the considered concepts. Further, we evaluate Vision-Language-Guided CBMs (VLGCBMs) [63], which use the open-vocabulary object detector Grounding DINO [39] to generate binary surrogate concept labels for Ptask for each concept based on the predicted presence of concepts in the image [63]. For all post-hoc CBM variants, the bottleneck size K is determined by the number of concepts derived from the concept source (we consider Broden [7], ConceptNet 5.5 [62], and GPT-3 [10]). 4.2
The Illusion of Classifier Accuracy
Setup. To validate that downstream accuracy is an insufficient metric for concept bottleneck quality (Section 3.1), we follow a similar setup to [45] and build posthoc CBMs using frozen, standard-normal random projections πθ of increasing dimension K. For comparison, we train post-hoc CBMs using dominant paradigms from the literature: (1) Auxiliary concept datasets [71], e.g., Broden [7], and (2) VLM surrogate labeling [49,63] using either the same concepts as in (1) or external sources (ConceptNet 5.5 [62] or GPT 3.5 [10]). In all cases, the downstream classifier h is trained after freezing πθ (see Appendix B.1 for additional details). Results. Figure 4 demonstrates that for CIFAR-100 [33] (other datasets in Appendix B.2), increasing the bottleneck dimension K of random projections decreases activation reconstruction error and closes the downstream accuracy gap to a fully supervised, bottleneck-free backbone. Notably, the post-hoc CBMs [49, 63, 71] trained on common concept sources [7, 10, 62] achieve similar performance to random projections of equal size K. This confirms that high downstream
12
L. Schmalwasser et al.
Table 1: Faithfulness vs. Performance. Comparison of post-hoc CBM variants with respect to the ground truth concept labels and direction on the Elements dataset [48]. OOD data degrades faithfulness significantly more than accuracy suggests. Each metric is averaged over the K concepts. To demonstrate faithfulness under error orthogonality, we also train πθ on ground-truth concept labels while adding 25% random noise.
Method
Dataset/ Label
Concept Projection πθ sim∢ [↑]
Downstream Classifier
Acc. Paux [↑] Acc. Ptask [↑] Acc. h [↑] Acc. h∗ [↑]
(1) Training on Auxiliary Distributions (Oracle Labels) PCBM [71]
Ptask ∀ Pnear ∃ Pnear POOD
0.96±0.01 0.97±0.01 0.57±0.02 0.47±0.01
1.00±0.00 1.00±0.00 0.82±0.02 0.98±0.01
1.00±0.00 0.70±0.02 0.89±0.01 0.77±0.01
1.00±0.00 1.00±0.00 1.00±0.00 1.00±0.00
1.00±0.00 0.99±0.01 0.85±0.04 0.21±0.05
0.98±0.02 0.56±0.02 0.58±0.02
1.00±0.00 0.99±0.00 1.00±0.00
0.98±0.03 0.54±0.04 0.78±0.05
(2) Surrogate Concept Labels (Training Dist.: Ptask ) LFCBM [49]
25% Label Noise 0.02±0.03 CLIP [55] 0.05±0.02 DINOv3 [61] 0.05±0.01
VLG-CBM [63] G–DINO [39]
0.02±0.02
0.75±0.01 0.79±0.01 0.85±0.01 0.74±0.01
0.85±0.01
1.00±0.01
0.04±0.01
accuracy is insufficient evidence for a meaningful bottleneck, requiring a direct concept projection analysis instead. 4.3
Quantifying Misalignment and Concept Projection Performance
Setup. Having shown that downstream accuracy is deceptive, we now directly measure the faithfulness of πθ . We evaluate post-hoc CBMs trained on four ∀ ∃ auxiliary distributions (Ptask , Pnear , Pnear , POOD ; Figure 3), alongside VLMsurrogate models: LFCBM [49] (using CLIP [55] and DINOv3 [61]) and VLGCBM [63] (using Grounding DINO [39]). We calculate the concept accuracy of πθ on both its training (Paux ) and downstream (Ptask ) distributions, and geometric faithfulness via the cosine similarity between learned concept vectors vk and target ground-truth directions vk∗ : sim∢ = (vk · vk∗ )/(∥vk ∥∥vk∗ ∥) (Appendix B.3). To further determine if h masks identified unfaithfulness, we compare its accuracy against an oracle classifier h∗ that uses the optimal rule for mapping ground-truth concepts to classes (cf. independent bottleneck strategy [32]). Though related to information leakage (see Section 2), faithfulness is complementary. Unfaithfulness can occur without leakage and vice versa (analyzed in Appendix B.4). Results. Table 1 summarizes the results for the Elements dataset [48], where we have access to ground-truth concept labels. We split our analysis into the two primary post-hoc CBM setups: (1) training on auxiliary datasets [71] and (2) using in-domain surrogate concept labels [49, 63]. For (1), all models achieve high accuracy on Paux , but POOD in particular yields unfaithful target representations, as indicated by the concept accuracy and the geometric alignment sim∢ . Crucially, the learned h masks these failures with high downstream accuracy. In contrast, the oracle h∗ demonstrates the
On the Faithfulness of Post-Hoc Concept Bottleneck Models
13 0.5
0.8
0.84
Ptask
Source Domain
0.7 0.6 ∀ Pnear
0.71
-0.02
0.47
0.86 0.5 0.4
∃ Pnear
0.62
0.62
-0.01
0.75
POOD
0.71
0.79
0.57
-0.03
Ptask
∀ ∃ Pnear Pnear Target Domain
POOD
0.3 0.2 0.1
0.00
0.50
0.32
0.49 0.4
∀ Pnear
0.45
0.00
0.26
0.49
∃ Pnear
0.39
0.49
0.00
0.47
POOD
0.49
0.50
0.40
0.00
Ptask
∀ ∃ Pnear Pnear Target Domain
POOD
0.2
0.1
0.0
(a) Average Pairwise H∆H-Divergence Matrix
0.3
Mean Generalization Error
0.46
Source Domain
0.76
Mean H∆H-Divergence
Ptask
-0.03
0.0
(b) Pairwise Generalization Errors
Fig. 5: Visualization of the estimated H∆H-divergence as a proxy for the upper bound of the task domain error. We pairwise compare the four probing datasets for the synthetic Elements downstream task [48], highlighting the actual downstream task in our setup. Note that the colormaps in (a) and (b) use different scales. We find that stronger covariate shifts lead to increased unfaithfulness, whereas lower divergence (i.e., in-domain concept sets) yields improved πθ .
unfaithfulness. In particular, the model trained on POOD drops to 21% accuracy, consistent with the expected impact of a severe shift in concept representation. For (2), LFCBMs [49] with noisy ground-truth labels (25% random noise) predict target concepts accurately. This matches our expectation from Section 3.2 that unsystematic label errors average out during the training of πθ . In contrast, LFCBMs [49] trained with CLIP [55] and DINOv3 [61] surrogate labels, achieve poor performance in the target distribution, though DINOv3 improves marginally upon CLIP. These observations are confirmed by the learned classifiers h, which show near-perfect accuracy for all settings. However, the oracle h∗ can again confirm the underlying unfaithfulness. VLG-CBMs [63] using Grounding DINO [39] exacerbate this: h perfectly solves the task, but h∗ beats random guessing (1/36 ≈ 0.028) only marginally. To explain the representation failures demonstrated in this experiment, we next investigate our theoretically derived sources of unfaithful concept projections πθ (Section 3.2).
4.4
Measuring the Impact of Covariate Shift
Setup. To evaluate the impact of covariate shifts when training πθ on an auxiliary dataset, we estimate the dˆH∆H -divergence [8, 29] between source and target domains, which serves as an upper bound to the generalization error (Section 3.2). Following [8], we approximate it by training a modified Huber loss discriminator [72] on the backbone activations to separate the distributions, computing dˆH∆H ≈ 2(1 − 2ϵ) based on the discriminator’s classification error ϵ. We then compare these estimates against the actual generalization errors of πθ .
L. Schmalwasser et al. Label Flip Probability 25%
|ρ|k |∆|k Sig.
|ρ|k |∆|k Sig.
0.10 0.03
X
Circle
0.04 0.30
×
Square
0.17 0.12
X
Square
0.07 0.23
×
Triangle
0.06 0.23
×
0.6
Plus
0.05 0.22
×
0.5
Red
0.07 0.26
×
Green
0.05 0.27
×
Blue
0.06 0.20
×
Solid
0.07 0.28
×
0.2 0.1
Triangle
0.00
×
Plus
0.24 0.20
†
X
Red
0.12 0.07
X
Green
0.16 0.09
X
Blue
†
Surrogate Errors ∆k
Surrogate Errors ∆k
VLG-CBM Circle
0.00
×
Solid
0.14 0.10
X
Striped
0.14 0.09
X
Striped
0.06 0.22
×
Dotted
0.18 0.17
X
Dotted
0.06 0.28
×
0
5 10 15 20 25 30 35 40 45 50 55 60
Activations αj
(a) Systematic Surrogate Error Introduced by Grounding DINO
0
0.7
0.4 0.3
Pearson |ρ|
14
0.0
5 10 15 20 25 30 35 40 45 50 55 60
Activations αj
(b) Unsystematic (Random) Surrogate Error
Fig. 6: Systematic vs. Unsystematic Surrogate Errors. Unfaithful projections occur when surrogate errors are both large (|∆|k ) and highly correlated (|ρ|k ) with activations αj . (a) Surrogates from Grounding DINO [39] exhibit systematic, correlated errors, causing unfaithful projections. (b) Random unsystematic label noise produces no correlation. If the VLM perfectly predicts a concept (|∆|k = 0), correlation is undefined (†).
Results. As shown in Figure 5 (with additional details and results in Appendix B.5), empirical divergences align closely with actual πθ generalization errors. The out-of-distribution domain POOD consistently yields the highest divergences and generalization errors. Conversely, matched source and target domains (on the diagonal) yield near-zero empirical divergences, confirming the high PCBM [71] performance observed in Table 1 (slightly negative values occur if the discriminator’s accuracy falls marginally below 0.5). Among mismatched ∃ ∀ domains, Pnear and Pnear show the lowest divergence, reflecting the latter being a subdistribution of the former. Overall, we find a strong correlation (> 0.9) between the estimated dˆH∆H and actual generalization errors, demonstrating that dˆH∆H is an effective proxy for the unfaithfulness introduced by a covariate shift of the auxiliary concept set. This is highly practical because it can be estimated entirely without ground-truth concept labels for the downstream task. 4.5
Measuring Systematic Surrogate Label Errors
Setup. Following Section 3.2, we analyze the faithfulness of post-hoc CBMs trained with surrogate labels based on the surrogate error ∆k for concept k using two metrics: (1) error magnitude (mean absolute error), and (2) error orthogonality (absolute Pearson correlation [51] |ρk,j | between surrogate label errors and backbone activations αj ). Because calculating ∆k requires ground truth concepts, we evaluate on the Elements dataset [48]. We compare surrogate labels generated with Grounding DINO [39, 63] against a baseline with unsystematic label noise obtained by randomly flipping a portion of the ground-truth concept labels (see Appendix B.6 for CLIP [55], DINOv3 [61], and CUB-200 [70]).
On the Faithfulness of Post-Hoc Concept Bottleneck Models
15
Results. Figure 6a shows these correlations alongside per-concept summary statistics |ρ|k and |∆|k (calculated as the absolute average over per-activation scores, using a Holm-Bonferroni correction [22] at p = 0.05 to determine significance). As established in Equation (8), optimizing πθ yields unfaithful representations when both values are large. Unlike the random-noise baseline, where the correlations effectively vanish, VLM surrogate errors are clearly systematic and correlated with specific activations. Qualitatively, we find that Grounding DINO [39] struggles to positively detect these concepts, which we attribute to the domain shift between its natural-image pretraining and our synthetic setup (geometric shapes on white backgrounds). Because the VLM label noise is non-random, training on it can cause πθ to learn unfaithful projections (see (2) in Table 1). While faithfulness is preserved for concepts that are labeled perfectly by the VLM (e.g., “Triangle” and “Blue” for Grounding DINO [39]), achieving purely random, uncorrelated surrogate errors may be difficult in many realworld scenarios. Since we identify systematic surrogate label errors as a source of unfaithfulness, correlation can be used to rank imperfect surrogate labeling functions by the severity of this failure mode. For Ptask , the average ρ across concepts is lowest for Grounding DINO [39] (0.142), followed by CLIP [55] (0.152) and DINOv3 [61] (0.155), making Grounding DINO the preferred surrogate labeling function under this criterion.
5
Conclusions
While post-hoc CBMs promise transparent decision-making, current evaluations often conflate predictive performance with conceptual alignment. As our theoretical analysis based on [5] shows, even non-semantic random projections achieve competitive accuracy. Consequently, downstream task performance is an uninformative metric for concept bottleneck quality, requiring the direct inspection of the concept projection πθ . Doing so reveals two primary sources of unfaithfulness in standard post-hoc CBM training. First, relying on auxiliary datasets may introduce a covariate shift that can invalidate learned concepts in the target domain. Based on [8], we provide an upper bound on the generalization error and a practical metric to measure it. Second, we demonstrate that systematic label errors introduced by VLM surrogate supervision lead to unfaithfulness. By formalizing these issues and validating them across real-world and synthetic benchmarks, we establish an evaluation framework for post-hoc CBM unfaithfulness. These insights open several avenues for future research. Our formalization of distribution shifts suggests the need for explicit domain adaptation techniques within the post-hoc CBM framework to align feature spaces, rather than just labels. Furthermore, the presence of systematic VLM label noise requires the development of de-biasing mechanisms that disentangle correlated errors during surrogate supervision. Ultimately, we provide the theoretical and practical tools necessary to investigate learned concept projections.
16
L. Schmalwasser et al.
References 1. Almudévar, A., Hernández-Lobato, J.M., Ortega, A.: There Was Never a Bottleneck in Concept Bottleneck Models. In: The Fourteenth International Conference on Learning Representations (2026) 1 2. Alukaev, D., Kiselev, S., Pershin, I., Ibragimov, B., Ivanov, V., Kornaev, A., Titov, I.: Cross-Modal Conceptualization in Bottleneck Models. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. pp. 5241–5253 (2023) 1 3. Ansuini, A., Laio, A., Macke, J.H., Zoccolan, D.: Intrinsic dimension of data representations in deep neural networks. Advances in Neural Information Processing Systems 32 (2019) 2, 3, 5, 6, 21 4. Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, J., Tu, J., Wan, J., Wang, P., Wang, P., Wang, Q., Wang, Y., Xie, T., Xu, Y., Xu, H., Xu, J., Yang, Z., Yang, M., Yang, J., Yang, A., Yu, B., Zhang, F., Zhang, H., Zhang, X., Zheng, B., Zhong, H., Zhou, J., Zhou, F., Zhou, J., Zhu, Y., Zhu, K.: Qwen3-VL Technical Report. arXiv preprint arXiv:2511.21631 (2025) 24 5. Baraniuk, R.G., Wakin, M.B.: Random Projections of Smooth Manifolds. Found. Comput. Math. 9(1), 51–77 (2009) 2, 3, 5, 6, 15, 21 6. Barsalou, L.W.: Perceptual symbol systems. Behav. Brain. Sci. 22(4), 577–660 (1999) 1 7. Bau, D., Zhou, B., Khosla, A., Oliva, A., Torralba, A.: Network Dissection: Quantifying Interpretability of Deep Visual Representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6541–6549 (2017) 1, 2, 3, 7, 10, 11, 32, 36, 37 8. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., Vaughan, J.W.: A theory of learning from different domains. Mach. Learn. 79(1-2), 151–175 (2010) 3, 7, 13, 15, 22, 35, 36, 37 9. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006) 6 10. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems 33, 1877–1901 (2020) 4, 11, 32 11. Chauhan, K., Tiwari, R., Freyberg, J., Shenoy, P., Dvijotham, K.: Interactive concept bottleneck models. In: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence. AAAI’23/IAAI’23/EAAI’23, AAAI Press (2023) 1 12. Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., et al.: Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities. arXiv preprint arXiv:2507.06261 (2025) 24
On the Faithfulness of Post-Hoc Concept Bottleneck Models
17
13. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255 (2009) 10, 31 14. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In: International Conference on Learning Representations (2021) 10 15. Espinosa Zarlenga, M., Barbiero, P., Ciravegna, G., Marra, G., Giannini, F., Diligenti, M., Shams, Z., Precioso, F., Melacci, S., Weller, A., et al.: Concept Embedding Models: Beyond the Accuracy-Explainability Trade-Off. Advances in Neural Information Processing Systems 35, 21400–21413 (2022) 2, 4, 5, 21, 25 16. Fahrmeir, L., Lang, S.: Bayesian Inference for Generalized Additive Mixed Models Based on Markov Random Field Priors. J. R. Stat. Soc. Ser. C 50(2), 201–220 (2001) 8, 25, 26, 34 17. Galliamov, K., Kazmi, S.M.A., Khan, A., Rivera, A.R.: Concepts’ Information Bottleneck Models. In: The Fourteenth International Conference on Learning Representations (2026) 1 18. Gardenfors, P.: Conceptual spaces: The geometry of thought. MIT press (2004) 1 19. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016) 6 20. Havasi, M., Parbhoo, S., Doshi-Velez, F.: Addressing Leakage in Concept Bottleneck Models. Advances in Neural Information Processing Systems 35, 23386–23397 (2022) 3, 5 21. He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) 9, 31 22. Holm, S.: A Simple Sequentially Rejective Multiple Test Procedure. Scand. J. Stat. 6(2), 65–70 (1979) 15, 39, 41 23. Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989) 5, 21 24. Huang, Q., Song, J., Hu, J., Zhang, H., Wang, Y., Song, M.: On the concept trustworthiness in concept bottleneck models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 21161–21168 (2024) 4 25. Jacovi, A., Goldberg, Y.: Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness? In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 4198–4205 (2020) 2 26. Johnson, W.B., Lindenstrauss, J., et al.: Extensions of Lipschitz mappings into a Hilbert space. Contemp. Math. 26(189-206), 1 (1984) 2, 3, 5, 21 27. Jørgensen, B.: Exponential Dispersion Models. J. R. Stat. Soc. Ser. B 49(2), 127–162 (1987) 8, 26 28. Kazmierczak, R., Berthier, E., Frehse, G., Franchi, G.: CLIP-QDA: An Explainable Concept Bottleneck Model. Transact. Mach. Learn. Res. (2024) 1 29. Kifer, D., Ben-David, S., Gehrke, J.: Detecting Change in Data Streams. In: Proceedings of the Thirtieth International Conference on Very Large Data Bases Volume 30. p. 180–191. VLDB ’04, VLDB Endowment (2004) 7, 13 30. Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., Sayres, R.: Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). In: Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 2668–2677. PMLR (2018) 7, 31, 32 31. Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization. In: The Third International Conference on Learning Representations (2015) 31
18
L. Schmalwasser et al.
32. Koh, P.W., Nguyen, T., Tang, Y.S., Mussmann, S., Pierson, E., Kim, B., Liang, P.: Concept Bottleneck Models. In: Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 119, pp. 5338–5348. PMLR (2020) 1, 2, 6, 12 33. Krizhevsky, A.: Learning Multiple Layers of Features from Tiny Images. Tech. rep. (2009) 3, 10, 11, 31, 32, 33 34. Kumar, S., Ahuja, N.: Measuring the (Un) Faithfulness of Concept-Based Explanations. arXiv preprint arXiv:2504.10833 (2025) 4 35. Lai, S., Hu, L., Wang, J., Berti-Equille, L., Wang, D.: Faithful Vision-Language Interpretation via Concept Bottleneck Models. In: The Twelfth International Conference on Learning Representations (2024) 4 36. Li, J., Li, D., Xiong, C., Hoi, S.: BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. In: Proceedings of the 39th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 162, pp. 12888–12900 (2022) 24 37. Lipton, Z.C.: The Mythos of Model Interpretability. Commun. ACM 61(10), 36–43 (2018) 1 38. Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., Lee, Y.J.: LLaVA-NeXT: Improved reasoning, OCR, and world knowledge (2024) 24 39. Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., Zhu, J., Zhang, L.: Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection. In: Computer Vision – ECCV 2024. p. 38–55. Lecture Notes in Computer Science (2024) 3, 11, 12, 13, 14, 15, 24, 36, 38, 39 40. Luyten, M.R., van der Schaar, M.: A theoretical design of concept sets: improving the predictability of concept bottleneck models. In: The Thirty-eighth Annual Conference on Neural Information Processing Systems (2024) 4 41. Mahinpei, A., Clark, J., Lage, I., Doshi-Velez, F., Pan, W.: Promises and Pitfalls of Black-Box Concept Learning Models. arXiv preprint arXiv:2106.13314 (2021) 2, 3 42. Makonnen, M., Vandenhirtz, M., Laguna, S., Vogt, J.E.: Measuring Leakage in Concept-Based Methods: An Information Theoretic Approach. In: ICLR 2025 Workshop: XAI4Science: From Understanding Model Behavior to Discovering New Scientific Knowledge (2025) 2, 5, 21, 34 43. Margeloiu, A., Ashman, M., Bhatt, U., Chen, Y., Jamnik, M., Weller, A.: Do Concept Bottleneck Models Learn as Intended? ICLR 2021 Workshop on Responsible AI (2021) 2, 3, 5 44. McCullagh, P., Nelder, J.A.: Generalized Linear Models. Monographs on statistics and applied probability, 2 edn. (1989) 8, 25, 26, 27, 28, 34 45. Midavaine, N., Go, G.H.T., Canez, D., Simion, I., Chatterji, S.: [Re] On the Reproducibility of Post-Hoc Concept Bottleneck Models. Trans. Mach. Learn. Res. (2024) 2, 5, 11, 30 46. Moayeri, M., Rezaei, K., Sanjabi, M., Feizi, S.: Text-To-Concept (and Back) via Cross-Model Alignment. In: Proceedings of the 40th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 202, pp. 25037–25060. PMLR (2023) 1 47. Murphy, G.: The Big Book of Concepts. MIT Press (2002) 1 48. Nicolson, A., Schut, L., Noble, A., Gal, Y.: Explaining Explainability: Recommendations for Effective Use of Concept Activation Vectors. Trans. Mach. Learn. Res. (2025) 3, 10, 12, 13, 14, 31, 32, 33, 34, 36, 38, 39, 40, 41 49. Oikarinen, T., Das, S., Nguyen, L.M., Weng, T.W.: Label-free Concept Bottleneck Models (2023) 2, 3, 4, 7, 8, 10, 11, 12, 13, 23, 31, 32, 34
On the Faithfulness of Post-Hoc Concept Bottleneck Models
19
50. Oikarinen, T., Weng, T.W.: Clip-dissect: Automatic description of neuron representations in deep vision networks. In: The Eleventh International Conference on Learning Representations (2023) 2, 23 51. Pearson, K.: VII. Mathematical Contributions to the Theory of Evolution. III. Regression, Heredity, and Panmixia. Philos. Trans. R. Soc. A. (187), 253–318 (12 1896) 9, 14, 36, 38, 39, 40, 41 52. Penrose, R.: On best approximate solutions of linear matrix equations. In: Math. Proc. Camb. Philos. Soc. vol. 52, pp. 17–19. Cambridge University Press (1956) 6, 22 53. Penzel, N., Denzler, J.: Locally explaining prediction behavior via gradual interventions and measuring property gradients. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 7398– 7408 (March 2026). https://doi.org/10.48550/arXiv.2503.05424, https: //propgrad.github.io 25 54. Pope, P., Zhu, C., Abdelkader, A., Goldblum, M., Goldstein, T.: The Intrinsic Dimension of Images and Its Impact on Learning. In: International Conference on Learning Representations (2021) 2, 3, 5, 6, 21 55. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning Transferable Visual Models From Natural Language Supervision. In: Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 8748–8763. PMLR (2021) 2, 3, 10, 11, 12, 13, 14, 15, 23, 24, 31, 36, 38, 39 56. Ramaswamy, V.V., Kim, S.S., Fong, R., Russakovsky, O.: Overlooked Factors in Concept-based Explanations: Dataset Choice, Concept Learnability, and Human Capability. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10932–10941 (2023) 4, 6 57. Sampat, S.K., Patel, M., Yang, Y., Baral, C.: Help Me Identify: Is an LLM+ VQA System All We Need to Identify Visual Concepts? arXiv preprint arXiv:2410.13651 (2024) 2, 8, 23, 24 58. Schmalwasser, L., Penzel, N., Denzler, J., Niebling, J.: Fastcav: Efficient computation of concept activation vectors for explaining deep neural networks. In: International Conference on Machine Learning (ICML) (2025), https://fastcav.github.io/ 7 59. Schoen, R., Abeloos, B., Herbin, S.: Measuring and Addressing Information Leakage in Concept Bottleneck Models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 624–632 (2025) 3, 5, 34 60. Shimodaira, H.: Improving predictive inference under covariate shift by weighting the log-likelihood function. J. Stat. Plan. Inference 90(2), 227–244 (2000) 7, 23 61. Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., Massa, F., Haziza, D., Wehrstedt, L., Wang, J., Darcet, T., Moutakanni, T., Sentana, L., Roberts, C., Vedaldi, A., Tolan, J., Brandt, J., Couprie, C., Mairal, J., Jégou, H., Labatut, P., Bojanowski, P.: DINOv3. arXiv preprint arXiv:2508.10104 (2025) 3, 10, 12, 13, 14, 15, 24, 36, 38, 39 62. Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5: an open multilingual graph of general knowledge. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. p. 4444–4451 (2017) 11, 32 63. Srivastava, D., Yan, G., Weng, T.W.: VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidance (2024) 2, 3, 4, 6, 7, 8, 11, 12, 13, 14, 21, 23, 24, 32, 34
20
L. Schmalwasser et al.
64. Sugiyama, M., Kawanabe, M.: Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation. The MIT Press (2012) 7, 23 65. Sun, A., Yuan, Y., Ma, P., Wang, S.: Eliminating information leakage in hard concept bottleneck models with supervised, hierarchical concept learning. arXiv preprint arXiv:2402.05945 (2024) 3 66. Sémery, O.: imgclsmob: Deep learning networks. https://github.com/osmr/ imgclsmob (2024), GitHub repository, accessed February 2026 9 67. Tan, A., Zhou, F., Chen, H.: Explain via Any Concept: Concept Bottleneck Model with Open Vocabulary Concepts. In: Computer Vision – ECCV 2024. p. 123–138. Lecture Notes in Computer Science, Springer-Verlag (2024) 1 68. Vandenhirtz, M., Laguna, S., Marcinkevičs, R., Vogt, J.E.: Stochastic Concept Bottleneck Models (2024) 1 69. Vershynin, R.: High-Dimensional Probability: An Introduction with Applications in Data Science, vol. 47 (2018) 22 70. Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., Perona, P.: Caltech-UCSD Birds 200 (2010) 3, 9, 14, 31, 32, 33, 34, 36, 37, 40, 41 71. Yuksekgonul, M., Wang, M., Zou, J.: Post-hoc concept bottleneck models. In: The Eleventh International Conference on Learning Representations (2023) 2, 3, 4, 6, 7, 10, 11, 12, 14, 21, 25, 30, 31, 32, 34 72. Zhang, T.: Solving large scale linear prediction problems using stochastic gradient descent algorithms. In: Proceedings of the Twenty-First International Conference on Machine Learning. Association for Computing Machinery (2004) 13, 36 73. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B 67(2), 301–320 (03 2005) 31
On the Faithfulness of Post-Hoc Concept Bottleneck Models
A
Additional Theoretical Details
A.1
Johnson-Lindenstrauss for Smooth Manifolds
21
While neural activation spaces are high-dimensional (A ⊂ Rd ), valid activations empirically concentrate on a manifold M of much lower intrinsic dimension m ≪ d [3, 54]. If this manifold hypothesis of deep learning holds, then the Johnson-Lindenstrauss (JL) lemma for smooth manifolds [5] governs the behavior under random projections πθ : A → C ⊂ RK . Additionally, while [42] discusses this for finite datasets (the standard JL lemma [26]), we note that [5] implies that it holds for the complete activation manifold. Next, we follow [5] to make the manifold JL lemma explicit for our use case of concept embeddings. First, for the standard JL lemma [26], the number of random projections needed mainly depends on the number of points and the original dimension (d in our case). For the manifold version, it depends on the manifold’s properties [5]. Let M ⊂ Rd be a compact m-dimensional Riemannian submanifold with a volume V , a condition number 1/τ , and the geodesic covering regularity R after [5]. According to the Manifold Johnson-Lindenstrauss lemma (Thm. 3.1 in [5]), a random projection into K dimensions preserves pairwise Euclidean distances on M with distortion 0 < ϵ < 1 with probability 1 − p if \label {eq:mjl} K = \mathcal {O}\left (\frac {m \log (dVR\tau ^{-1}\epsilon ^{-1})\log (p^{-1})}{\epsilon ^2}\right ).
(9)
Crucially, K scales linearly with the intrinsic dimension m but logarithmically with the activation space dimension d. Thus, if K ≪ d sufficiently exceeds m, the geometry of A is preserved in C. There is also a logarithmic dependence on the volume V , the geodesic covering regularity R, and the condition number 1/τ . Specifically, the condition number bounds the curvature of the manifold [5]. Thus, highly curved manifolds (large 1/τ ) need a larger number of random projections. We found this also in our concrete derivation for a linear subspace Appendix A.2, where the intrinsic dimension m is enough. In any case, a sufficiently expressive nonlinear classifier h : C → Y, e.g., an MLP [23], can approximate g ◦ πθ−1 and solve the downstream task with high performance. This insight aligns with performance gains observed using non-linear classifiers on concept bottlenecks [15, 71]. Crucially, our discussion here is based on the manifold hypothesis. While there is empirical evidence that this applies in common cases, e.g., for realistic datasets [3, 54], we want to stress that settings where it does not hold may be possible. Note that in such cases [63] proves that the expected error under random projections goes towards zero for the embedding dimension K approaching the ambient dimensions d.
22
L. Schmalwasser et al.
A.2
Derivation of a Sufficient Classifier Under Linear Subspace Assumption
As mentioned in our main paper, for sufficiently expressive h, it is possible to achieve high performance even for random concept projections πθ . Our argument is based on inverting the original activations a = f (x). Following the assumptions stated in our main paper, we derive h as follows: Let σ −1 (c) = P a be the pre-activation concepts before the application of the bijective activation function σ. Following Assumption (1), we have σ −1 (c) = P U z. Let M = P U ∈ RK×m be the matrix product of the random projection matrix P and the orthonormal basis U of the linear activation subspace S. Since U has full rank and P is a standard Gaussian matrix, M is full column rank with probability 1 (almost surely) if K ≥ m (Assumption (2) and Assumption(3)). This is implied by the two-sided bounds of the singular values (Thm. 4.6.1 in [69]) for such matrices. Consequently, M is left-invertible. The latent vector z can be recovered exactly using the Moore-Penrose pseudoinverse M † = (M ⊤ M )−1 M ⊤ [52]: z = M^\dagger \sigma ^{-1}(c) = (PU)^\dagger \sigma ^{-1}(c).
(10)
That σ −1 exists follows from the bijectivity (Assumption (3)). The original activations a are reconstructed as \hat {a} = U z = U (PU)^\dagger \sigma ^{-1}(c)
(11)
and we define arec : C → S with arec (c) := U (P U )† σ −1 (c) as the function reconstructing the original activations for a given c ∈ C. Finally, we construct the sufficient classifier h to mimic the original g \nonumber h(c) &:= g(a_{\mathrm {rec}}(c))\\ &= \mathrm {softmax}\left ( W_g \cdot \left [ U (P U)^\dagger \sigma ^{-1}(c) \right ] \right ). (12) A.3
Formalization of the Covariate Shift Assumption
In Section 3.2, we utilize the domain adaptation bound from [8] to bound the expected L1 (absolute) task risk after training on auxiliary concept sets. Here, we provide formal definitions for the task adaptability term and the covariate shift assumption. The final term of the bound, λideal , represents the combined error of the optimal hypothesis that performs best on both the auxiliary and task distributions simultaneously: \lambda _{\mathrm {ideal}} = \min _{\theta } \Big ( \underbrace {\mathbb {E}_{x \sim P_{\mathrm {aux}}} [|\pi _{\theta }(f(x)) - c^*_{\mathrm {aux}}(x)|]}_{\text {Error vs Probing Set Rule}} + \underbrace {\mathbb {E}_{x \sim P_{\mathrm {task}}} [|\pi _{\theta }(f(x)) - c^*_{\mathrm {task}}(x)|]}_{\text {Error vs Downstream Task Rule}} \Big ), (13) where c∗aux and c∗task denote the underlying ground-truth concept labeling functions for the respective domains, Paux and Ptask .
On the Faithfulness of Post-Hoc Concept Bottleneck Models
23
In the context of concept probing, we assume the distribution shift is primarily a Covariate Shift [60, 64]. This implies that while the probability of observing specific input samples differs between the domains, the semantic definition of the concepts remains stable (i.e., the conditional probability of the concept given the input is constant): &\forall x, c \in \mathcal {X} \times \mathcal {C}: P_{\mathrm {task}} (c|x) = P_{\mathrm {aux}}(c|x), \text {~and~} \exists x \in \mathcal {X}: P_{\mathrm {task}} (x) \neq P_{\mathrm {aux}}(x). (14) Under this assumption, there exists a single optimal decision rule c∗ shared across distributions. Consequently, λideal simplifies to the combined error of the best shared projection: \lambda _{\mathrm {ideal}} = \min _{\theta } \Big ( \underbrace {\mathbb {E}_{x \sim P_{\mathrm {aux}}} [|\pi _{\theta }(f(x)) - c^*(x)|]}_{\text {Error vs Shared Rule}} + \underbrace {\mathbb {E}_{x \sim P_{\mathrm {task}}} [|\pi _{\theta }(f(x)) - c^*(x)|]}_{\text {Error vs Shared Rule}} \Big ). (15) As argued in the main text, this allows us to treat λideal as negligible and isolate the empirical H∆H-divergence as our proxy for the unfaithfulness of πθ trained on the auxiliary concept dataset Xaux . A.4
Vision-Language-Based Surrogate Labeling Functions
As noted in our main paper, to minimize domain shift when learning the concept projection πθ , it is advisable to directly train on in-domain data with respect to the downstream task. However, often concept annotations in sufficient quantity are unavailable directly. Hence, surrogate labeling functions based on vision language models (VLMs) are used in practice, e.g., [49, 57, 63], to approximate the true labeling function c∗ . Note that in any case, sets of concepts as text descriptions may be derived from Large Language Models (LLMs) specifically for the downstream classes [49, 57, 63]. Here, we describe two current frameworks for annotating such concepts for in-domain data. Surrogate Labels From Vision-Language Embeddings [49]. A promising strategy for generating surrogate concept labels is to leverage the zero-shot capabilities of vision-language encoders (VLEs) [49, 50]. Let eV : X → Rd and eT : T → Rd denote the vision and text encoders of a pre-trained VLM (e.g., CLIP [55]), sharing a joint embedding space. Given a set of concept prompts T = {tk }K k=1 , we define the surrogate labeling function c̃VLE : X → C using a similarity metric sim : Rd × Rd → R. For an input x, the K concept scores are computed as: \tilde {c}_{\mathrm {VLE}}(x) \;=\; \big (\mathrm {sim}(e_V(x), e_T(t_k))\big )_{k=1}^K \in \mathbb {R}^K.
(16)
Common choices for the similarity metric include standard cosine similarity, ⟨u,v⟩ scos (u, v) = ∥u∥∥v∥ , or sharpened variants designed to suppress noise from lowsimilarity such as the cubed cosine similarity scos3 (u, v) = (scos (ū3 , v̄ 3 )), where ū and v̄ are standardized to mean 0 and variance 1 [49]. Consequently, a post-hoc CBM with VLE-based surrogate concept label generation optimizes: \tilde {J}_\mathrm {task}(\theta ;\mathcal {L}) = \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \mathcal {L}\left (\pi _\theta (f(x)), \, \tilde {c}_{\mathrm {VLE}}(x)\right ) \right ],
(17)
24
L. Schmalwasser et al.
thereby aligning model activations with VLM-derived surrogate concepts. In practice, we use CLIP [55] and DINOv3 [61] encoders in our experiments. Surrogate Labels From Grounded Object Detection [63]. Alternatively, surrogate concepts can be generated by spatially grounding language concepts using openvocabulary object detectors (e.g., Grounding-DINO [39]). Following the VLGCBM [63] approach, an LLM is employed to generate a set of candidate concept prompts T = {tk }K k=1 relevant to the classes in Y. To obtain the surrogate labels, a grounded object detector Dobj takes an image x ∈ X and the prompt set T as input, and returns a set of detected bounding boxes with associated confidence scores. Let the output detections be B(x) = {(bm , σm , km )}M m=1 , where bm represents the box coordinates, σm ∈ [0, 1] is the confidence score, and km ∈ {1, . . . , K} is the index of the matched concept tkm . Then, the grounded surrogate labeling function c̃VLG : X → {0, 1}K is based on thresholding the detection scores: \tilde {c}_{\mathrm {VLG}}(x) \;=\; (\mathds {1}\left [ \exists m : k_m = k \land \sigma _m > \tau \right ])_{k=1}^K,
(18)
where τ is a pre-defined confidence threshold and 1[·] is the indicator function. Unlike the continuous scores discussed above for c̃VLE , this method yields a binary concept vector. The post-hoc CBM is then trained using these binary labels, typically treating the concept alignment as a multi-label classification task, optimizing: \tilde {J}_\mathrm {task}(\theta ;\mathcal {L}) = \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \mathcal {L}\left (\pi _\theta (f(x)), \, \tilde {c}_{\mathrm {VLG}}(x)\right ) \right ].
(19)
Surrogate Labels From Visual Question Answering [57]. A direct alternative to bounding-box detection is to prompt advanced vision-language models, e.g., [4, 12, 36, 38], and phrasing surrogate labeling as a Visual Question Answering (VQA) problem [57]. Unlike embedding-based methods that rely on latent space similarity, this strategy treats concept extraction as a Boolean verification task in language space. Let Φ be a generative VLM that maps an image x ∈ X and a text prompt p to a probability distribution over a vocabulary V. For each concept tk ∈ T, a concept query prompt pk is constructed, such as “Is the concept ’tk ’ present in this image? Answer Yes or No.”. The surrogate labeling function c̃VQA : X → [0, 1]K is derived by computing the probability of the affirmative token tyes (e.g., “Yes”) given the image and prompt: \tilde {c}_{\mathrm {VQA}}(x) \;=\; (P_\Phi (t_{\mathrm {yes}} \mid x, p_k))_{k=1}^K.
(20)
Depending on the implementation, this score may be used directly as a soft concept probability or is thresholded into a binary concept vector to optimize πθ : \tilde {J}_\mathrm {task}(\theta ;\mathcal {L}) = \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \mathcal {L}\left (\pi _\theta (f(x)), \, \tilde {c}_{\mathrm {VQA}}(x)\right ) \right ].
(21)
On the Faithfulness of Post-Hoc Concept Bottleneck Models True 0, Surr. 0 (Correct) True 1, Surr. 1 (Correct) Learned Boundary πΘ f2 (x)
25
True 1, Surr. 0 (Error) True 0, Surr. 1 (Error) Ideal Boundary f2 (x)
f2 (x)
Shift
f1 (x)
(a) True Concept Labels
f1 (x)
f1 (x)
(b) Systematic Concept Errors
(c) Random Concept Errors
Fig. 7: Geometric intuition for surrogate faithfulness based on a concept encoded as binary classification (shapes denote true concept labels according to the ground-truth concept labeling function c∗ ; colored borders denote surrogate labels generated with c̃). (a) The ideal boundary learned from true labels. (b) If the surrogate labeling function generates errors that are systematically correlated with specific features, the learned boundary differs from the ideal boundary, leading to unfaithfulness. (c) If errors in the surrogate labels are random (specifically, if E[δ(x) ⊗ f (x)] ≈ 0, meaning they are orthogonal to the backbone features in expectation), the ideal boundary is recovered. This allows us to reach a faithful πΘ despite imperfect surrogate labels.
A.5
Conditions for Surrogate Label Faithfulness
In the post-hoc CBM framework, the concept projection πθ : A → C maps the d-dimensional backbone activations f (x) ∈ A ⊆ Rd to a K-dimensional concept space C ⊆ RK (Section 3). Because a ground-truth labeling function c∗ is typically unavailable in post-hoc settings, this projection is often trained using a surrogate labeling function c̃ (Section 3.2), commonly derived from the zero-shot capabilities of VLMs (Appendix A.4). Consequently, the post-hoc CBM’s faithfulness depends on whether training πθ on the surrogate labels from c̃ results in a valid solution with respect to the true labels from c∗ . This is important to ensure the practical benefits of post-hoc CBMs, e.g., facilitating expert interventions [71]. If the concept predictions are not aligned with the human semantics encoded in c∗ , interventions can be problematic [15], which is difficult to verify in latent concept bottle layers [53]. In the following, we show under which conditions we can learn a faithful concept projection πθ based on c̃ (Figure 7), i.e., minimizing Jtask (θ; L), Definition 1. We begin by formulating πθ within the framework of Multivariate Generalized Linear Models (GLM) [16, 44]. The GLM assumption captures the standard post-hoc CBM architecture, in which the concept projection πθ is commonly represented by a linear layer. This restricts the parameters θ to a learnable matrix Θ ∈ RK×d (the linear probe weights). Consequently, the concept projection takes
26
L. Schmalwasser et al.
the form πΘ (f (x)). Our analysis proceeds in five steps: First, we present the distributional (1) and structural (2) assumptions under which we model πθ as a GLM. Then, we derive the gradients for the log-likelihood for the true concept labels (3) and the surrogate objective (4), before finally using the gradient of the true objective (5) to derive the exact conditions for surrogate faithfulness after Definition 1. 1. Distributional Assumption. Because concepts in post-hoc CBMs can be modeled in various ways, such as binary indicators (e.g., presence of “wings”) or continuous similarity scores (e.g., similarity with a VLM prompt), we need a general framework to model the concept distribution. The Multivariate Exponential Dispersion Family (EDF) [27,44] provides this generalization: It is a broad family of probability distributions that generalizes almost all relevant distributions used in practice (including Normal, Binomial, Poisson, and Gamma distributions), offering a unified mathematical treatment for the conditional expectation µ. We assume the conditional distribution of the concept vector c ∈ C given an input f (x) ∈ A follows an EDF defined by a natural parameter vector η ∈ RK [27, 44]: p(c \mid \eta , \phi ) = \exp \left ( \frac {\langle c, \eta \rangle - A(\eta )}{s(\phi )} + k(c, \phi ) \right ),
(22)
where: – ⟨·, ·⟩ denotes the standard dot product in RK . – c ∈ C is the observed concept vector. – η ∈ RK is the natural parameter vector determining the distribution’s mean. – A : RK → R is the strictly convex cumulant function (log-partition function). – ϕ > 0 is the scalar dispersion parameter scaling the variance. – s(ϕ) is the dispersion function, a strictly positive scaling function (typically s(ϕ) = ϕ). – k(c, ϕ) ensures that the resulting distribution is normalized. Since it is independent of η, it vanishes upon differentiation. Here, only η and ϕ are determined by πΘ , while all remaining parameters are fixed by the general distribution. This framework covers discrete and continuous concept label distributions, such as binary concepts following a Bernoulli distribution (ϕ = 1) and continuous concepts following a Gaussian distribution (ϕ = σ 2 for variance σ 2 ). A fundamental property of the EDF that we will leverage is that the expected concept vector µ = E[c | η, ϕ] is directly given by the gradient of the cumulant function [44]: \label {eq:edf_mean} \mu = \nabla _{\eta } A(\eta ). (23) 2. Structural Assumption. To formally model the concept projection, we follow the standard GLM framework [16,44], which connects the backbone activations to the expected concept values µ through a linear predictor ζ and an invertible link function ψ. In the context of post-hoc CBMs, the linear predictor corresponds to
On the Faithfulness of Post-Hoc Concept Bottleneck Models
27
the learnable linear layer applied to the backbone features. We define this predictor as a function ζ : A → RK , parameterized by the weight matrix Θ ∈ RK×d : \zeta (f(x)) = \Theta f(x).
(24)
Because the linear output is unbounded while concept expectations may be constrained (e.g., µ ∈ [0, 1]K ), the link function ψ : C → RK is used to enforce !
the relationship ψ(µ) = ζ(f (x)). In neural network terms, the inverse link function ψ −1 : RK → C corresponds to the activation function (e.g., sigmoid or softmax) that transforms the linear logits into the final prediction: \psi (\mu ) = \zeta (f(x)) \Longleftrightarrow \mu = \psi ^{-1}(\zeta (f(x))) = \psi ^{-1}(\Theta f(x)).
(25)
Specifically, we assume that ψ is the canonical link function, which equates the mean µ to the natural parameter η such that ψ(µ) := η [44]: \eta = \psi (\mu ) = \zeta (f(x)) = \Theta f(x).
(26)
This assumption covers standard choices for the combinations of distributional assumptions and activation functions (such as pairing a Bernoulli distribution with logit link) and simplifies the gradient derivations. By combining this structural assumption with the EDF property from Equation (23), the model’s concept prediction ĉΘ (x) is given by the gradient of the cumulant function at the linear predictor: \hat {c}_\Theta (x) = \mu = \nabla _{\eta } A(\eta ) = \nabla _{\Theta f(x)} A(\Theta f(x)).
(27)
3. Gradient of the Log-Likelihood. Our optimization objective is to minimize the Negative Log-Likelihood (NLL) on the training data. For a single observation (f (x), c) ∈ A × C, the loss function L is derived from the EDF density: \mathcal {L}(\pi _\Theta (f(x)), c) &= - \log p(c \mid \eta , \phi ) \\ &= - \log \exp \left ( \frac {\langle c, \eta \rangle - A(\eta )}{s(\phi )} + k(c, \phi ) \right ) \\ &= - \left ( \frac {\langle c, \eta \rangle - A(\eta )}{s(\phi )} + k(c, \phi ) \right ) \\ &\;\propto \; - \frac {1}{s(\phi )} \left ( \langle c, \eta \rangle - A(\eta ) \right ) ,
(31) where we have omitted terms independent of Θ. Substituting the canonical link η = Θf (x), we differentiate with respect to the weight matrix Θ. Based on Equation (23), we define the concept prediction as ĉΘ (x) = µ = ∇η A(η). Applying the chain rule for matrix calculus and leveraging the identity ∇Θ ⟨u, Θv⟩ = uv ⊤ ,
28
L. Schmalwasser et al.
we obtain: \nabla _\Theta \mathcal {L}(\pi _\Theta (f(x)), c) &= - \frac {1}{s(\phi )} \nabla _\Theta \left ( \langle c, \eta \rangle - A(\eta ) \right ) \\ &= - \frac {1}{s(\phi )} \left ( \nabla _\Theta \left ( c^\top \Theta f(x) \right ) - \nabla _\Theta A(\Theta f(x)) \right ) \\ &= - \frac {1}{s(\phi )} ( \nabla _\Theta \left ( c^\top \Theta f(x) \right ) - \underbrace {\nabla _{\eta } A(\eta )}_{\hat {c}_\Theta (x)} f(x)^\top ) \\ &= - \frac {1}{s(\phi )} \left ( c f(x)^\top - \hat {c}_\Theta (x) f(x)^\top \right ).
(35) Rearranging the terms yields the final gradient form: \label {eq:gradient_mnef} \nabla _\Theta \mathcal {L}(\pi _\Theta (f(x)), c) = \frac {1}{s(\phi )} \underbrace {\left ( \hat {c}_\Theta (x) - c \right )}_{\text {Residual}} \otimes \underbrace {f(x)}_{\text {Input}},
(36)
where ⊗ denotes the outer product. That is, for a distribution in the Multivariate EDF with a canonical link, the gradient is the outer product of the concept residual and the backbone activations, scaled by the inverse dispersion. 4. Surrogate Optimality Condition. We assume the projection parameters Θ are learned using a surrogate labeling function c̃ : X → C (see Appendix A.4 for examples) rather than the inaccessible ground truth. Thus, the surrogate objective J˜task (Θ; L) is the expected risk over the task distribution Ptask , calculated using these surrogate labels: \label {eq:surrogate-cost-function} \tilde {J}_\mathrm {task}(\Theta ;\mathcal {L}) = \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \mathcal {L}(\pi _\Theta (f(x)), \tilde {c}(x)) \right ].
(37)
Because we consider a GLM with a canonical link function, the negative loglikelihood is convex with respect to the linear parameters Θ (provided the cumulant function A is convex, which holds by definition for our EDF) [44]. Consequently, the optimization problem is convex, and any stationary point corresponds to a global minimum. We assume the optimization procedure reaches such a stationary point (e.g., via unregularized empirical risk minimization), yielding learned parameters Θ̃ ∈ RK×d that satisfy the first-order condition ∇J˜task (Θ̃; L) = 0. To evaluate this gradient, we also have to specify the distributional assumptions for the surrogate labels generated with c̃. We assume that the conditional distribution of the surrogate labels belongs to the same EDF as the ground-truth concepts, but can be parameterized by its own distinct surrogate natural parameter η̃ and dispersion parameter ϕ̃. For example, if concepts are modeled as Gaussian variables, we would assume that both ground-truth and surrogate concepts are normally distributed but may exhibit a different mean and standard deviation σ̃ (where ϕ̃ = σ̃ 2 ) compared to the underlying ground truth. Substituting the corresponding gradient from Equation (36) into Equation (37), we obtain:
On the Faithfulness of Post-Hoc Concept Bottleneck Models
29
\nabla \tilde {J}_\mathrm {task}(\tilde {\Theta };\mathcal {L}) &= \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \nabla _\Theta \mathcal {L}(\pi _{\tilde {\Theta }}(f(x)), \tilde {c}(x)) \right ]\\ &= \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \frac {1}{s(\tilde {\phi })} \left ( \hat {c}_{\tilde {\Theta }}(x) - \tilde {c}(x) \right ) \otimes f(x) \right ] = 0_{K \times d}. (39) Since the dispersion scaling factor s(ϕ̃) is strictly positive and constant with respect to the expectation, it factors out. This implies an orthogonality condition between the surrogate residuals and the backbone activations: \label {eq:surrogate_orthogonality} \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \left ( \hat {c}_{\tilde {\Theta }}(x) - \tilde {c}(x) \right ) \otimes f(x) \right ] = 0_{K \times d}.
(40)
Note that while we assume that ∇J˜task (Θ̃; L) = 0K×d , this does not imply J˜task (Θ̃; L) = 0 (which is congruent with our empirical results in Table 1). ∗ 5. Gradient of the True Objective. We define the true objective Jtask (Θ; L) as the expected risk with respect to the ground-truth concept function c∗ (x) and the true dispersion parameter ϕ∗ :
J^*_\mathrm {task}(\Theta ;\mathcal {L}) = \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \mathcal {L}(\pi _\Theta (f(x)), c^*(x)) \right ].
(41)
We evaluate the gradient of this objective at the parameters Θ̃ learned via the surrogate optimization. Substituting the gradient form from Equation (36): \nabla J^*_\mathrm {task}(\tilde {\Theta };\mathcal {L}) &= \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \frac {1}{s(\phi ^*)} \left ( \hat {c}_{\tilde {\Theta }}(x) - c^*(x) \right ) \otimes f(x) \right ] \\ &= \frac {1}{s(\phi ^*)} \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \left ( \hat {c}_{\tilde {\Theta }}(x) - c^*(x) \right ) \otimes f(x) \right ] . (43) To analyze the faithfulness, we expand the residual term by adding and subtracting the surrogate label vector c̃(x). Using the linearity of the outer product and the expectation, we decompose the gradient into two components: \nabla J^*_\mathrm {task}(\tilde {\Theta };\mathcal {L}) &= \frac {1}{s(\phi ^*)} \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \left ( \hat {c}_{\tilde {\Theta }}(x) - \tilde {c}(x) + \tilde {c}(x) - c^*(x) \right ) \otimes f(x) \right ] \\ &= \frac {1}{s(\phi ^*)} \Bigg ( \underbrace {\mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ (\hat {c}_{\tilde {\Theta }}(x) - \tilde {c}(x)) \otimes f(x) \right ]}_{0 \text { via surrogate optimality (\Cref {eq:surrogate_orthogonality})}} \nonumber \\ &\quad \quad \quad \quad + \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ (\tilde {c}(x) - c^*(x)) \otimes f(x) \right ] \Bigg ).
(45) The first term vanishes because Θ̃ is a stationary point of the surrogate objective. Consequently, the gradient of the true objective depends only on the second term:
\label {eq:final_alignment} \nabla J^*_\mathrm {task}(\tilde {\Theta };\mathcal {L}) = \frac {1}{s(\phi ^*)} \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \underbrace {\left ( \tilde {c}(x) - c^*(x) \right )}_{\text {Label Discrepancy}} \otimes f(x) \right ].
(46)
30
L. Schmalwasser et al.
∗ Now, Jtask (Θ̃; L) depends only on the discrepancy between the true concept labels from c∗ and the surrogate concept labels generated with c̃, not on the specific concept projection πΘ̃ . This result provides a mathematical condition for the surrogate faithfulness. The surrogate parameters Θ̃ are optimal for the true objective if and only if this gradient (Equation (46)) is the zero matrix. This condition can be satisfied in two distinct ways. First, and most intuitively, if the surrogate labels are perfectly accurate (c̃(x) = c∗ (x) for all x), the discrepancy term vanishes, and the gradient is zero. Second, and more subtly, the model remains faithful even with imperfect labels if the label discrepancy is orthogonal to the activation space in expectation. For the gradient matrix to be zero, each of its elements must be zero. Let δ(x) = c̃(x) − c∗ (x) be the label discrepancy vector. The condition for the (k, j)-th element is:
\label {eq:orthogonality_elementwise} \mathbb {E}_{x \sim P_{\mathrm {task}}} \left [ \delta _k(x) \cdot f_j(x) \right ] = 0, \quad \forall k \in \{1,\dots ,K\}, j \in \{1,\dots ,d\}.
(47)
To build intuition for this condition, we can consider its empirical approximation over the N samples of the training set, as shown in Figure 2. Here, it requires the dot product of every two vectors in the N -dimensional sample space to be zero: the vector of label discrepancies for concept k, ∆k = [δk (x1 ), . . . , δk (xN )]⊤ , and the vector of feature values for dimension j, αj = [fj (x1 ), . . . , fj (xN )]⊤ . Therefore, faithfulness does not strictly require accurate surrogate labels; it only requires that the errors in these labels are not systematically correlated with the features the backbone model has learned.
B
Additional Experimental Details
B.1
Experimental Setup: Random Concept Projections
As motivated in Section 4.2, we empirically investigate whether downstream predictive accuracy is a sufficient metric for evaluating the quality of a Post-Hoc Concept Bottleneck Model (PCBM) [71]. Our theoretical analysis in Section 3.1 suggests that even using semantically uninformative random projections for the concept projection πθ can preserve enough information to enable high task performance. To empirically validate our theoretical results, we examine the performance of a PCBM equipped with a randomly generated concept projection, following a setup similar to that of [45]. If a meaningless bottleneck can yield high predictive accuracy, accuracy alone is an unreliable proxy for concept faithfulness. Concept Projection. Generally, PCBMs implement the concept projection πθ : A → C mapping from d-dimensional backbone activations a ∈ A ⊆ Rd to a K-dimensional concept space C ⊆ RK as a linear layer with a potentially nonlinear activation function (Appendix A.5). In this experiment, we follow the standard PCBM procedure [71] and implement πθ as a linear transformation of the backbone activations (i.e., selecting identity transformation as activation function). We further follow [71] and set the bias vector b ∈ RK to 0. Consequently,
On the Faithfulness of Post-Hoc Concept Bottleneck Models
31
πθ is parameterized only by a weight matrix Θ ∈ RK×d and implements the following mapping for backbone activations a ∈ A: \pi _{\Theta }(a) = \Theta a.
(48)
Construction of Random Concept Projections. To construct a semantically meaningless concept projection, we generate a random weight matrix Θrand ∈ RK×d in a two-step process following our theoretical results presented in Section 3.1. ′ First, we sample an initial random matrix Θrand where each element follows a standard normal distribution: \Theta '_{\mathrm {rand}} \sim \mathcal {N}(0, 1)^{K \times d}.
(49)
′ Afterward, we perform L2 -normalization of each row vector of Θrand to obtain the final weight matrix Θrand (similarly to CAVs [30]). Since the transformation matrix Θrand consists of unit-length basis vectors with random direction, the resulting random concept projection πΘrand (a) = Θrand a maps the backbone activations a ∈ A into a semantically meaningless concept space.
Model Architecture and Training. While in the standard PCBM framework [71] only the backbone model f : X → A is frozen, we now also freeze the concept projection, implementing it as the random concept projection πΘrand defined above. Thus, the downstream classifier h : C → Y is the only trainable component, optimized on the outputs of πΘrand . Following [49], we implement h as a single linear layer trained with a cross-entropy loss and elastic-net regularization [73]. This regularization encourages sparsity in the concept-to-class weights, which is commonly desired for the interpretability of the classifier head. The backbone architectures and bottleneck insertion points are adapted for each dataset: – CUB [70]: We follow the standard train-test split [70] and use a ResNet18 [21] pretrained on ImageNet [13] for the backbone f . The concept bottleneck is inserted after the final global average pooling layer (features.final_pool). The classifier head h is trained with the Adam optimizer [31] at a learning rate of 2 × 10−4 . – Elements [48]: Following [48], we select a simple 5-layer convolutional neural network (ConvNet) as backbone, which uses 64 channels per block. The bottleneck is inserted after the third convolutional layer (layer3). The classifier h is trained with Adam [31] at a learning rate of 1 × 10−5 . – CIFAR-10 & CIFAR-100 [33]: We follow the default train-test split [33]. The backbone is a ResNet-50 [21] pretrained via CLIP [55], where we insert the concept bottleneck after the final residual block (layer4). The classifier h is trained with Adam [31] at a learning rate of 7 × 10−4 . Reconstruction of Backbone Activations. To directly quantify the information preserved by the random concept projection πΘrand , we train a decoder to reconstruct the original backbone activations. We define a linear decoder D : C → A, parameterized by a weight matrix ΘD ∈ Rd×K , which is trained to minimize
32
L. Schmalwasser et al.
the Mean Squared Error (MSE) between the original activations a ∈ A and the reconstructed activations â = D(πΘrand (a)). A low reconstruction MSE indicates that the concept space, despite being generated randomly, retains sufficient information to recover the original activations. This validates our theoretical argument that a sufficiently expressive downstream classifier h can learn to approximate this inverse mapping, thereby bypassing any semantic interpretation of the concepts to achieve high task accuracy. B.2
Additional Results: Random Concept Projections
In this section, we provide extended results for the experiments on random concept projections introduced in Section 4.2, with setup details available in Appendix B.1. Figure 8 visualizes the task classification accuracy of PCBMs [71] with concept projections trained on the CUB [70], CIFAR-10 [33], and Elements [48] datasets and compares them with PCBMs utilizing a semantically meaningless concept projection πΘrand for the respective dataset (Appendix B.1). For the concept projection πΘrand , we scale the bottleneck dimension K of the weight matrix Θrand ∈ RK×d to align with the number of concepts present in the trained PCBMs. The resulting baseline curve extends slightly past the maximum number of concepts used by any PCBM for the task. For the trained concept projections πθ , we use auxiliary data (Broden [7]) and VLM-based surrogate methods (LFCBM [49] and VLG-CBM [63], see Appendix A.4). The concepts for the VLM-based surrogate methods are sourced from ground-truth concept labels, ConceptNet 5.5 [62], and GPT-3.5 [10]. Consistent with the findings presented in the main text (Section 4.2), we observe that as the dimensionality K of the random concept projection πΘrand increases, its downstream test accuracy asymptotically approaches that of an unconstrained base classifier without a bottleneck layer. Additionally, the PCBMs with learned concept projections only marginally outperform the random concept projections of equivalent dimensionality K, indicating that task accuracy is an inadequate metric to measure the semantic meaningfulness of a PCBM. To investigate why random concept projections can achieve this classification performance, we additionally report the mean squared error (MSE) for reconstructing the d-dimensional backbone activations from the K-dimensional outputs of the random projection (Appendix B.1). We observe that as K increases, the reconstruction MSE monotonically decreases, suggesting that higher-dimensional random projections retain sufficient information about the backbone activations for the downstream classifier h to solve the task directly (c.f., Section 3.1). Consequently, these results highlight that downstream task accuracy is an insufficient metric to assess whether a PCBM relies on semantically meaningful concepts. B.3
Measuring Alignment of Concept Activation Vectors
To quantify the geometric faithfulness of the concept transformation πθ , we compare the learned concept directions with the ground truth. For CAV-based concept transformations [30], this is straightforward. Here, we measure the
Activation Reconstruction MSE (10 3)
Accuracy (%)
Random Concept Projection rand Activation Reconstruction MSE Base Classifier
Random Concept Projection rand Activation Reconstruction MSE Base Classifier
120 100
80
80
60
60
40
40
20 0
Random Concept Projection rand Activation Reconstruction MSE Base Classifier
Activation Reconstruction MSE (10 3)
CUB
100
20 0
200
400 600 Number of Concepts K
800
1000
33
Activation Reconstruction MSE (10 3)
On the Faithfulness of Post-Hoc Concept Bottleneck Models
0
Methods: VLG-CBM (Grounding DINO) LFCBM (DINOv3) LFCBM (CLIP) PCBM (Image Set) Concept Sources: GPT-3.5 Broden GT
(a) CUB
CIFAR-10
100
Accuracy (%)
80 60 40 20 0
0
25
50
75 100 125 150 Number of Concepts K
175
200
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
Methods: PCBM (Image Set) LFCBM (DINOv3) LFCBM (CLIP) VLG-CBM (Grounding DINO) Concept Sources: Broden GPT-3.5 ConceptNet 5.5
(b) CIFAR10
Accuracy (%)
Elements 100
500
80
400
60
300
40
200
20
100
0
0
50
100 150 Number of Concepts K
200
250
0
Methods: VLG-CBM (Grounding Dino) LFCBM (DINO) LFCBM (CLIP) PCBM (Image Set) Concept Sources: GPT-3.5 ConceptNet 5.5 GT
(c) Elements
Fig. 8: Extended results for the random concept projection experiments in Figure 4 on (a) CUB [70], (b) CIFAR-10 [33], and (c) Elements [48]. As the dimensionality K of πΘrand increases, the downstream task accuracy (blue line) approaches that of an unconstrained base classifier without a bottleneck layer (dashed line). This is consistent with a corresponding drop in activation reconstruction MSE (orange line), indicating that higher-dimensional random projections retain sufficient information to reconstruct the backbone activations. The PCBMs with random concept projections also achieve a performance similar to that of standard PCBMs of similar dimensionality, where πθ is trained on an auxiliary concept set or VLM-based surrogate labels (scatter points).
34
L. Schmalwasser et al.
Table 2: Information leakage following [42, 59]. We highlight the generalization error on Ptask (Equation (1)) in bold. Note that NICC [59] for the optimal h∗ is zero by definition (marked by † ). Method
PCBM [71]
Paux
Ptask ∀ Pnear ∃ Pnear POOD
P task Error [↓] 0.0000±0.0000 0.5000±0.0076 0.3225±0.0030 0.4852±0.0019
NICC [↓] [59]
I(y; ĉ, c) [↓] [42] ∗
h
h
0.5563±0.0038 0.4335±0.0065 0.4909±0.0144 0.4105±0.0130
0.0† 0.0† 0.0† 0.0†
0.0040±0.0002 0.0040±0.0002 0.0047±0.0003 0.0048±0.0002
geometric faithfulness by extracting the concept activation vectors vk (for concept k) and computing their alignment with the ground-truth directions vk∗ (obtained by training on the full training dataset Xtask with known ground-truth concept labels) via cosine similarity: \mathrm {sim}_\sphericalangle (v_k, v_k^*) = \frac {\langle v_k, v_k^* \rangle }{\| v_k \| \| v_k^* \|}.
(50)
Since both LFCBMs [49] and VLG-CBMs [63] implement πθ as a Generalized Linear Model [16, 44] (Appendix A.5), they follow the form: \pi _\Theta (f(x)) = \psi ^{-1}(\Theta f(x)),
(51)
where ψ denotes the link function of the GLM and Θ ∈ RK×d is a learned weight matrix. Thus, we consider the normalized row vectors of Θ as our concept activation vectors vk and calculate sim∢ analogously to the CAV-based PCBMs. B.4
Additional Results: Alternative Faithfulness Metrics
Unlike information leakage approaches (e.g., NICC [59] and [42]) that assess the interplay between πθ and the downstream classifier h, our definition of faithfulness strictly isolates πθ . These perspectives are complementary: unfaithfulness can occur without leakage, and conversely, for a faithful πθ , an optimized h may still exploit leaked information. This is demonstrated in Table 2, which evaluates the information leakage metrics presented by [42, 59] and reveals differences with respect to our approach. Row 1 shows πθ trained on Ptask minimizing the associated error, yet h exploits leaked information (NICC > 0). In contrast, for the specified optimal h∗ , NICC reports zero leakage even if Paux ̸= Ptask , leading to unfaithful πθ . B.5
Additional Results: Covariate Shift
We extend the covariate shift experiments conducted on the synthetic Elements dataset [48] (Section 4.4) to the real-world CUB dataset [70]. Specifically, we compare the impact of training the concept projection πθ on auxiliary concept
On the Faithfulness of Post-Hoc Concept Bottleneck Models
35
Algorithm 1 Estimation of H∆H-Divergence and Generalization Error Require: Set of M distributions D = {P1 , . . . , PM }, set of concepts K = {1, . . . , K}, backbone network f : X → A, and oracle concept labeling function c∗ : X → [0, 1]K . 1: Initialize results matrices H, E ∈ RM ×M 2: for s = 1 to M and t = 1 to M do 3: Set Psource ← Ps , Ptarget ← Pt 4: Initialize cumulative divergence D ← 0, cumulative error E ← 0 5: for k = 1 to K do 6: 7: 8: 9: 10: 11:
1. Data Sampling and Preparation X+ ∼ Psource (· | ck = 1) X− ∼ Ptarget (· | ck = 0) Xtask ∼ Ptarget Xaux ← {(x, 1) | x ∈ X+ } ∪ {(x, 0) | x ∈ X− } test Xtrain aux , Xaux ← Split(Xaux ) train Xtask , Xtest task ← Split(Xtask )
12: 13:
2. Concept Classifier Optimization train Atrain aux ← {(f (x), y) | (x, y) ∈ Xaux } (k) train πθ ← TrainCAV(Aaux )
14: 15: 16: 17: 18: 19:
3. Domain Discriminator Optimization and Evaluation train train Atrain disc ← {(f (x), 0) | (x, y) ∈ Xaux } ∪ {(f (x), 1) | x ∈ Xtask } hdisc ← TrainSGDClassifier(Atrain ) disc test test Atest disc ← {(f (x), 0) | (x, y) ∈ Xaux } ∪ {(f (x), 1) | x ∈ Xtask } (k) test ϵdisc ← Error(hdisc , Adisc ) (k) (k) dˆH∆H ← 2(1 − 2ϵdisc ) \triangleright Approximation following [8] (k) D ← D + dˆ
20: 21:
4. Generalization Error Calculation ∗ test Atest task ← {(f (x), ck (x)) | x ∈ Xtask } test Atest ← {(f (x), y) | (x, y) ∈ X aux aux }
22:
test ϵgen ← Error(πθ , Atest task ) − Error(πθ , Aaux )
H∆H
(k)
(k)
\triangleright Add oracle labels
(k)
(k)
23: E ← E + ϵgen 24: end for 25: Record averages: Hs,t ← D/K, 26: end for
Es,t ← E/K
27: return H (Pairwise Divergences), E (Pairwise Generalization Errors)
36
L. Schmalwasser et al.
sets from Broden [7] to training it on in-domain CUB concept sets. After mapping the aligned concepts between the two datasets (e.g., matching “Bill” in CUB to “Beak” in Broden, and “Upperparts” to “Body”), we can leverage CUB’s ground-truth annotations to compute the true generalization error of the πθ and use it to validate our proposed label-free faithfulness metric: the estimated H∆H-divergence [8], which bounds the generalization error (Section 3.2). To reflect realistic post-hoc interpretability scenarios in which concept annotations for the target domain are missing, we draw only negative concept examples from the target domain and restrict the sampling of positive examples to the auxiliary domain. As detailed in Algorithm 1, we construct an auxiliary training set Xaux for each concept k by combining positive samples from the source distribution with negative samples from the target distribution. The concept projection πθ is then trained per-concept on Xaux . To prevent data leakage caused by the limited sample sizes, we strictly enforce that no input sample appears in both the training and test sets. Next, we calculate our faithfulness metric for distribution shift by training a linear domain discriminator on the backbone features to separate Xaux from the target distribution (Section 3.2). Following [8], we optimize it using a Huber loss [72], and approximate the H∆H-divergence as dˆH∆H ≈ 2(1 − 2ϵ) based on the discriminator’s classification error ϵ. We execute this procedure for all pairwise combinations of CUB and Broden acting as source and target domains. The resulting divergence estimates and corresponding true generalization errors are presented in Figure 9. Consistent with our findings on the Elements dataset, we observe a strong positive Pearson correlation [51] (r > 0.85) between our divergence metric and the actual generalization error. This confirms that relying on visually distinct auxiliary datasets, such as Broden, introduces a measurable covariate shift that explicitly degrades the faithfulness of the learned concept representations. Furthermore, it demonstrates that approximating the generalization error by the empirical H∆H-divergence metric successfully identifies this degradation without requiring target domain concept annotations. B.6
Additional Results: Systematic Surrogate Label Errors
In this section, we expand the analysis of systematic surrogate concept label errors by evaluating additional Vision-Language Models (VLMs; Appendix A.4). Following Section 3.2, we compute the Pearson correlation [51] between the backbone activations and the surrogate errors produced by CLIP [55], DINOv3 [61], and Grounding DINO [39]. We compare these results against a synthetic baseline of unsystematic label noise, implemented by randomly flipping the binary concept labels with a 40% probability. Figure 10 presents the Pearson correlation coefficients and average absolute surrogate errors for the Elements dataset [48]. To highlight the systematic nature of the VLM-induced errors, Figure 11 masks the correlation matrix, displaying only statistically significant entries (p < 0.05). We repeat the same experiment for the real-world CUB dataset [70], presenting the unmasked correlations in Figure 12 and the masked significant correlations in Figure 13. To compute
On the Faithfulness of Post-Hoc Concept Bottleneck Models
37
0.8
0.5 0.4 0.3
Broden
0.81
0.01
0.2
CUB
0.00
0.19
Source Domain
0.6
Mean H∆H-Divergence
0.76
Source Domain
0.00
0.20
0.15
0.10
Broden
0.33
0.00 0.05
0.1 0.0
CUB
Broden Target Domain
(a) Average Pairwise H∆H-Divergence Matrix
0.25
Mean Generalization Error
0.30
0.7
CUB
0.00
CUB
Broden Target Domain
(b) Pairwise Generalization Errors
Fig. 9: Visualization of the estimated H∆H-divergence [8] as a proxy for the upper bound of the task generalization error for the CUB dataset [70], when comparing training πθ based on the ground-truth concept labels to training it on the Broden dataset [7]. As in the main text, stronger distribution shifts between the concept-probing datasets and the downstream domain, as indicated by our divergence metric, indicate greater unfaithfulness, as observed from the true generalization error. By estimating the H∆Hdivergence, we can identify this degradation in faithfulness without requiring target domain concept annotations.
the surrogate label error for CUB, we use the human-annotated ground-truth attributes and analyze a subset of the first 50 concepts. Across all evaluated VLMs and datasets, the surrogate label errors exhibit statistically significant correlations with the backbone activations. On the other hand, the 40% randomnoise baseline demonstrates no such systematic structure. These results further validate that the current VLM-based generation of surrogate labels can violate the error orthogonality requirement, introducing systematic errors into the concept bottleneck.
L. Schmalwasser et al.
|ρ|k |∆|k Sig.
Circle
0.20 0.19
X
Square
0.20 0.21
X
Square
0.21 0.25
X
Triangle
0.10 0.05
X
Triangle
0.18 0.17
X
Plus
0.15 0.20
X
Plus
0.26 0.26
X
Red
0.18 0.35
X
Red
0.06 0.03
×
Green
0.12 0.18
X
Blue
0.18 0.28
X
Solid
0.21 0.30
X
Striped
0.12 0.21
X
0.17 0.22
X
0
† 0.00 × 0.05) Label (p ≤ Green Flip Probability 40% – sig.
Square
Triangle
Blue
0.08 0.01
X
Solid
0.18 0.25
X
Striped
0.20 0.35
X
0.18 0.40
X
Dotted
5 10 15 20 25 30 35 40 45 50 55 60
0
Plus
Activations αj
(a) CLIP
Red
(b) DINOv3
Green
Label Flip Probability 40%
|ρ|k |∆|k Sig.
Circle
0.10 0.03
X
Square
0.17 0.12
X
Triangle
0.00
×
Plus
0.24 0.20
†
X
Red
0.12 0.07
X
Green
0.16 0.09
X
Blue
0.00
×
Solid
0.14 0.10
†
X
Striped
0.14 0.09
Dotted
0.18 0.17
5 10 15 20 25 30 35 40 45 50 55 60
Blue Solid Striped Dotted
0.4 |ρ|k |∆|k Sig.
Circle
0.05 0.41
×
Square
0.09 0.46
X
Triangle
0.07 0.37
×
Plus
0.05 0.34
×
Red
0.05 0.40
×
Green
0.06 0.39
X
Blue 5 10
0.04 0.41
50
×
0.05 0.40
×
X
Striped
0.05 0.40
×
X
Dotted
0.06 0.38
×
0
0.6 0.5
Solid
0
0.7
5 10 15 20 25 30 35 40 45 50 55 60
Activations αj
VLG-CBM
0
Circle
Surrogate Errors ∆k
X
Dotted
Surrogate Errors ∆k
DINOv3
|ρ|k |∆|k Sig. 0.09 0.07
Surrogate Errors ∆k
Surrogate Errors ∆k
CLIP Circle
15
20
25
30
35
40
45
55
0.3
Pearson |ρ|
38
0.2 0.1 0.0 60
5 10 15 20 25 30 35 40 45 50 55 60
Activations αj
Activations αj
(c) Grounding DINO
(d) Random Label Noise
Fig. 10: Systematic vs. unsystematic surrogate errors for various VLMs (Appendix A.4), extending the results from Figure 6. We visualize the Pearson correlation [51] between backbone activations and surrogate label errors on the Elements dataset [48]. Concept labels generated by (a) CLIP [55], (b) DINOv3 [61], and (c) Grounding DINO [39] all exhibit systematic, correlated errors (visible as dense vertical bands). In contrast, randomly flipping 40% of the concept labels (d) yields an unsystematic error profile, satisfying the error orthogonality condition (Section 3.2). If a concept is predicted without error (|∆|k = 0), the correlation coefficient is undefined (†).
On the Faithfulness of Post-Hoc Concept Bottleneck Models
CLIP – sig. (p ≤ 0.05)
Circle
DINOv3 – sig. (p ≤ 0.05)
Circle
Square
Square
Triangle
Triangle
Plus
Plus
Red
39
Red
Label Flip Probability 40% – sig. (p ≤ 0.05) Green
Blue
Circle
Solid
Square
Striped
Triangle
Dotted 0
5
10
15
20
25
30
35
40
45
50
55
60
0.7
Striped
0.6
Dotted
Plus
0
Red
(a) CLIP VLG-CBM – sig. (p ≤ 0.05)
Circle
Blue Solid
Green
Square
Blue
Triangle
Circle
10
60
0.5
(b) DINOv3 Label Flip Probability 40% – sig. (p ≤ 0.05)
15
20
25
30
35
40
45
50
55
0.4
Square
0.3
Triangle
Plus
Solid
Plus
Red
Striped
Red
Green
5
0.2 0.1
Green
Dotted
Blue
Blue
Solid
0 Solid 5
Striped
Striped
Dotted
Pearson |ρ|
Green
0.0 10
15
5
10
20
25
30
25
30
35
40
45
50
45
50
55
55
60
Dotted 0
5
10
15
20
25
30
35
40
45
(c) Grounding DINO
50
55
60
0
15
20
35
40
60
(d) Random Label Noise
Fig. 11: Statistically significant Pearson correlation coefficients [51] from Figure 10 (surrogate errors on the Elements dataset [48]). We display only correlation coefficients that remain significant after a Holm-Bonferroni correction [22] (p < 0.05). Surrogate labels generated by (a) CLIP [55], (b) DINOv3 [61], and (c) Grounding DINO [39] exhibit dense blocks of statistically significant correlations, confirming that their labeling errors are systematic. Conversely, the random noise baseline (d) shows no significant correlations beyond the expected false positive rate.
L. Schmalwasser et al.
Curved bill
|ρ|k |∆|k Sig. 0.02 0.02 X
0.05 0.14
X
Dagger bill
0.05 0.14
X
Hooked bill
0.02 0.02
X
Hooked bill
0.02 0.02
X
Needle bill
0.04 0.03
X
Needle bill
0.04 0.03
X
Hooked seabird bill
0.07 0.06
X
Hooked seabird bill
0.07 0.07
X
Spatulate bill
0.05 0.04
X
Spatulate bill
0.05 0.04
X
All-purpose bill
0.06 0.40
X
All-purpose bill
0.06 0.40
X
Cone bill
0.06 0.25
X
Cone bill
0.06 0.25
X
Specialized bill
0.03 0.02
X
Specialized bill
0.03 0.02
X
Blue wing
0.04 0.05
X
Blue wing
0.04 0.05
X
Brown wing
0.05 0.26
X
Brown wing
0.05 0.25
X
Iridescent wing
0.02 0.02
X
Iridescent wing
0.02 0.02
X
X
Grey wing
0.04 0.30
X
Yellow wing
0.07 0.09
X
Yellow wing
0.07 0.09
X
Olive wing
0.04 0.04
X
Olive wing
0.04 0.04
X
Green wing
0.03 0.02
X
Green wing
0.03 0.02
X
Pink wing
0.01 0.00
×
Pink wing
0.01 0.00
X
Orange wing
0.03 0.03
X
Orange wing
0.03 0.03
X
Black wing
0.04 0.39
X
Black wing
0.03 0.37
X
White wing
0.05 0.26
X
White wing
0.05 0.25
X
Red wing
0.03 0.02
X
Red wing
0.03 0.02
X
Buff wing
0.06 0.18
X
Buff wing
0.06 0.18
X
Blue upperparts
0.04 0.05
X
Blue upperparts
0.04 0.05
X
Brown upperparts
0.05 0.23
X
Brown upperparts
0.05 0.23
X
Iridescent upperparts
0.02 0.03
X
Iridescent upperparts
0.02 0.03
X
Purple upperparts
0.01 0.01
X
Purple upperparts
0.01 0.01
X
Rufous upperparts
0.02 0.01
X
Rufous upperparts
0.02 0.01
X
Grey upperparts
0.04 0.30
X
Grey upperparts
0.04 0.29
X
Yellow upperparts
0.07 0.09
X
Yellow upperparts
0.07 0.09
X
Olive upperparts
0.04 0.04
X
Olive upperparts
0.04 0.04
X
Green upperparts
0.03 0.02
X
Green upperparts
0.03 0.02
X
Pink upperparts
0.01 0.01
X
Pink upperparts
0.01 0.00
X
Orange upperparts
0.03 0.03
X
Orange upperparts
0.03 0.03
X
Black upperparts
0.03 0.38
X
Black upperparts
0.03 0.34
X
White upperparts
0.05 0.23
X
White upperparts
0.05 0.23
X
Red upperparts
0.04 0.02
X
Red upperparts
0.04 0.02
X
Buff upperparts
0.06 0.17
X
Buff upperparts
0.06 0.17
X
Blue underparts
0.04 0.03
X
Blue underparts
0.04 0.03
X
Surrogate Errors ∆k
0.04 0.30
VLG-CBM
|ρ|k |∆|k Sig. 0.02 0.02 X
Dagger bill
0.05 0.14
X
Hooked bill
0.02 0.02
X
Needle bill
0.04 0.03
X
Hooked seabird bill
0.07 0.06
X
Spatulate bill
0.05 0.04
X
All-purpose bill
0.06 0.40
X
Cone bill
0.06 0.25
X
Specialized bill
0.03 0.02
X
Blue wing
0.05 0.05
X
Brown wing
0.06 0.27
X
Iridescent wing
0.02 0.02
X
Purple wing
0.01 0.00
X
Rufous wing
0.03 0.01
X
Grey wing
0.04 0.31
X
Yellow wing
0.07 0.09
X
Olive wing
0.04 0.04
X
Green wing
0.03 0.02
X
Pink wing
0.01 0.00
×
Orange wing
0.03 0.02
X
Black wing
0.05 0.42
X
White wing
0.05 0.26
X
Red wing
0.03 0.02
X
Buff wing
0.06 0.18
X
44 8
38 4
0.5
(b) DINOv3
0.4
Label Flip Probability 40%
Green Blue Solid
Curved bill
|ρ|k |∆|k Sig. 0.01 0.40 ×
Dagger bill
0.01 0.40
×
Hooked bill
0.01 0.40
×
Needle bill
0.01 0.40
×
Hooked seabird bill
0.01 0.40
×
Spatulate bill
0.01 0.40
×
All-purpose bill
0.01 0.40
×
Striped
Cone bill
0.01 0.41
×
Specialized bill
0.01 0.41
×
Blue wing
0.01 0.40
×
Dotted
Brown wing
0.01 0.40
×
Iridescent wing
0.01 0.40
×
0.01 0.40
×
0.01 0.40
Purple wing
0
5
10Rufous 15wing 20
45
×
0.01 0.41
×
Yellow wing
0.01 0.41
×
Grey wing
25
30
35
40
50
Olive wing
0.01 0.40
×
Green wing
0.01 0.40
X
Pink wing
0.01 0.41
Orange wing
0.01 0.40
×
0.01 0.41
X
Red wing
0.01 0.41
X
Buff wing
0.01 0.40
×
0.05 0.05
X
Blue upperparts
0.01 0.40
×
0.06 0.24
X
Brown upperparts
0.01 0.41
×
Iridescent upperparts
0.02 0.03
X
Iridescent upperparts
0.01 0.40
×
Purple upperparts
0.01 0.01
X
Purple upperparts
0.01 0.40
×
Rufous upperparts
0.02 0.01
X
Rufous upperparts
0.01 0.40
×
Grey upperparts
0.04 0.30
X
Grey upperparts
0.01 0.40
×
Yellow upperparts
0.07 0.09
X
Yellow upperparts
0.01 0.41
×
Olive upperparts
0.04 0.04
X
Olive upperparts
0.01 0.41
×
Green upperparts
0.03 0.02
X
Green upperparts
0.01 0.39
×
Pink upperparts
0.01 0.00
X
Pink upperparts
0.01 0.40
×
Orange upperparts
0.03 0.03
X
Orange upperparts
0.01 0.41
×
Black upperparts
0.05 0.39
X
Black upperparts
0.01 0.40
×
White upperparts
0.05 0.23
X
White upperparts
0.01 0.40
×
2
6
0
4
8
19
25
32
38
44
0
8
64
8
12
4
×
44
0.01 0.39
38
Blue underparts
0
X
6
0.04 0.03
2
×
Blue underparts 32
×
0.01 0.41
25
0.01 0.39
Buff upperparts
19
Red upperparts
X
0
X
0.06 0.17
8
0.04 0.02
64
Red upperparts Buff upperparts
12
0.2 0.1 0.0 55
60
×
0.01 0.40
Blue upperparts
(c) Grounding DINO
0.3
×
Black wing White wing
Brown upperparts
Activations αj
0.6
Activations αj
Red
Curved bill
0.7
Pearson |ρ|
Activations αj
32 0
Plus
25 6
0
Triangle
19 2
Square
64
Circle
Label Flip Probability 40% – sig. (p ≤ 0.05)
12 8
44 8
X
Grey wing
38 4
X
0.03 0.01
32 0
0.01 0.00
Rufous wing
25 6
Purple wing
X
19 2
X
0.03 0.01
0
0.01 0.01
64
Purple wing Rufous wing
(a) CLIP
Surrogate Errors ∆k
DINOv3
|ρ|k |∆|k Sig. 0.02 0.02 X
Dagger bill
12 8
Surrogate Errors ∆k
CLIP Curved bill
Surrogate Errors ∆k
40
Activations αj
(d) Random Label Noise
Fig. 12: Systematic vs. unsystematic surrogate errors for various VLMs (Appendix A.4) evaluated on the CUB dataset [70]. Following the methodology of Figure 10, we visualize the Pearson correlation [51] between backbone activations and surrogate concept label errors. Due to space constraints, we only report results for the first 50 concepts of the CUB dataset. Consistent with our findings for the Elements dataset [48], the VLM-based surrogate concept labels (a–c) exhibit highly correlated errors, violating the orthogonality condition for faithfulness (Section 3.2). In contrast, the 40% random noise baseline (d) produces uncorrelated, orthogonal errors. If a concept is predicted without error (|∆|k = 0), the correlation coefficient is undefined (†).
On the Faithfulness of Post-Hoc Concept Bottleneck Models
CLIP – sig. (p ≤ 0.05)
Curved bill
Dagger bill Hooked bill
Needle bill
Needle bill
Hooked seabird bill
Hooked seabird bill
Spatulate bill
Spatulate bill
All-purpose bill
All-purpose bill
Cone bill
Cone bill
Specialized bill
Specialized bill
Blue wing
Blue wing
Brown wing
Brown wing
Iridescent wing
Iridescent wing
Purple wing
Purple wing
Rufous wing
Rufous wing
Grey wing
Grey wing
Yellow wing
Yellow wing
Olive wing
Olive wing
Green wing
Green wing
Pink wing
Pink wing
Orange wing
Orange wing
Black wing
Black wing
White wing
White wing
Red wing
Red wing
Buff wing
Buff wing
Blue upperparts
Blue upperparts
Brown upperparts
Brown upperparts
Iridescent upperparts
Iridescent upperparts
Purple upperparts
Purple upperparts
Rufous upperparts
Rufous upperparts
Grey upperparts
Grey upperparts
Yellow upperparts Olive upperparts
LabelYellow Flip Probability 40% – sig. (p ≤ 0.05) upperparts Olive upperparts
Circle
Green upperparts Pink upperparts Orange upperparts
Green upperparts Pink upperparts Black upperparts
White upperparts
White upperparts
Red upperparts
Buff upperparts
44 8
38 4
32 0
25 6
19 2
0.5
(b) DINOv3 Label Flip Probability 40% – sig. (p ≤ 0.05)
0.4
Dagger bill
Hooked bill
Hooked bill
Needle bill
0.3
Needle bill
Blue
Hooked seabird bill
Pearson |ρ|
Curved bill
Green
Dagger bill
12 8
0
Plus Red
(a)VLG-CBM CLIP– sig. (p ≤ 0.05)
64
44 8
38 4
32 0
25 6
19 2
12 8
Blue underparts 64
Hooked seabird bill
Spatulate bill
Spatulate bill
Solid
All-purpose bill
0.2
All-purpose bill
Cone bill
Cone bill
Specialized bill
Specialized bill
Striped
Blue wing
Blue wing
Brown wing
0.1
Brown wing
Dotted
Iridescent wing Purple wing
Iridescent wing Purple wing
Rufous wing
0.0
Rufous wing
0
Grey wing Yellow wing
5
10 Grey 15wing20
25
30
35
40
45
50
55
60
Yellow wing
Olive wing
Olive wing
Green wing
Green wing
Pink wing
Pink wing
Orange wing
Orange wing
Black wing
Black wing
White wing
White wing
Red wing
Red wing
Buff wing
Buff wing
Blue upperparts
Blue upperparts
Brown upperparts
Brown upperparts
Iridescent upperparts
Iridescent upperparts
Purple upperparts
Purple upperparts
Rufous upperparts
Rufous upperparts
Grey upperparts
Grey upperparts
Yellow upperparts
Yellow upperparts
Olive upperparts
Olive upperparts
Green upperparts
Green upperparts
Pink upperparts
Pink upperparts
Orange upperparts
Orange upperparts
Black upperparts
Black upperparts
White upperparts
White upperparts
0
4
8
32
38
44
8 44
6
4 38
25
0 32
2
6 25
8
2 19
0
8 12
64
(c) Grounding DINO
19
Red upperparts Buff upperparts Blue underparts 12
Red upperparts Buff upperparts Blue underparts
64
0
0.6
Red upperparts
Triangle
Buff upperparts Blue underparts
0
0.7
Orange upperparts
Square
Black upperparts
Curved bill
DINOv3 – sig. (p ≤ 0.05)
Curved bill
Dagger bill Hooked bill
41
(d) Random Label Noise
Fig. 13: Statistically significant Pearson correlation coefficients [51] from Figure 12 (surrogate errors on the CUB dataset [70]). We display only correlation coefficients that remain significant after a Holm-Bonferroni correction [22] (p < 0.05). The results mirror those of the Elements dataset [48]: the surrogate labels generated by VLM-based methods (a, b, c) produce a dense matrix of statistically significant, correlated errors. For the random baseline, we observe a sparse matrix of significant correlations, which matches the expected false-positive rate of repeated testing.