Conceptualizing Embeddings: Sparse Disentanglement for Vision–Language Models
arXiv:2605.22679v1 [cs.CV] 21 May 2026
Piotr Kubaty∗ 1,2 Patryk Marszałek∗ 1,2 Łukasz Struski 1 Adam Wróbel 1,2 Jacek Tabor 1,3 Marek Śmieja 1 1 2
Faculty of Mathematics and Computer Science, Jagiellonian University, Kraków, Poland Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland 3 Centre for Credible AI, Warsaw University of Technology, Warsaw, Poland
Abstract Vision-language models learn powerful multimodal embeddings, yet their internal semantics remain opaque. While sparse autoencoders (SAEs) can extract interpretable features, they rely on expanding the representation dimension, which compromises the original geometry and introduces redundancy. We introduce CEDAR (Conceptual Embedding Disentanglement via Adaptive Rotation), a posthoc method that reveals the compositional structure of pretrained embeddings without increasing dimensionality. By learning an invertible transformation with a top-k sparsity bottleneck, CEDAR concentrates semantic information into axisaligned disentangled coordinates. In CLIP-like architecture, individual coordinates can be interpreted with textual concepts, while for generative models such as BLIP, they can be decoded into natural language descriptions. Experiments demonstrate that CEDAR achieves a competitive reconstruction-sparsity trade-off while producing explanations that are more interpretable and better aligned with human perception. Our results suggest that the apparent entanglement in vision-language representations can be resolved through a suitable change of basis, eliminating the need for overcomplete expansions.
1
Introduction
Vision–language models such as CLIP [1], BLIP [2], and CoCa [3] are a cornerstone of multimodal learning, achieving strong performance by learning high-dimensional shared embeddings. Despite this success, these representations remain largely opaque: it is unclear how semantic factors are encoded, how they interact, and which components are most relevant for a given input. Existing interpretability methods capture only a partial view of this structure. Many focus on local explanations, such as highlighting image regions or attributing predictions to features [4, 5, 6, 7, 8], without revealing the global organization of the embedding space. Others aim to align embeddings with human-interpretable concepts [9, 10, 11], but typically rely on predefined directions, supervision, or assumptions about the original embedding geometry, limiting their flexibility and scalability. A prominent line of work employs sparse autoencoders (SAEs) [12, 13, 14, 15, 16] to learn overcomplete representations where neurons correspond to interpretable features. These methods demonstrate that pretrained embeddings contain extractable semantic structure. However, they typically expand the representation into a higher-dimensional space, which alters the original geometry and produces lossy, not directly invertible representations. This makes it difficult to faithfully relate the explanations back to the underlying model. Preprint.
Top-K concepts Explanation
Representation
lizard juvenile purple blue
Summarisation: “a lizard on a rock”
red
Disentangled activations
Figure 1: Overview of the CEDAR pipeline. We disentangle representations in vision–language models into sparse components. Depending on the model architecture, we then extract semantic concepts associated with individual neurons or generate sparse textual descriptions.
In this work, we take a different approach. Instead of expanding the representation, we ask: Can we reveal the semantic structure of pretrained embeddings by reorganizing them within their original space? To address this, we propose CEDAR (Conceptual Embedding Disentanglement via Adaptive Rotation), a post-hoc method that learns an invertible reparameterization of the embedding space. The key idea is to identify a global coordinate system in which semantic information becomes concentrated in a small number of dimensions. This is achieved by applying a learned transformation and enforcing a top-k sparsity bottleneck in the transformed space, while reconstructing the original embedding. This formulation leads to a representation with three desirable properties: • Sparsity: each input is described by a small number of active coordinates, • Disentanglement: coordinates correspond to approximately independent semantic factors, • Axis alignment: each dimension captures a coherent and reusable concept. Crucially, because the transformation is invertible, CEDAR preserves the structure of the original embedding space and enables faithful reconstruction, in contrast to expansion-based methods such as SAEs. Our approach is model-agnostic and supports different modes of interpretation depending on the vision–language model. In CLIP-like architectures, individual coordinates can be aligned with textual concepts, yielding compact explanations analogous to SAE methods—but without representation expansion. For generative models such as BLIP, these same sparse coordinates can be decoded into natural language descriptions, providing sentence-level explanations. This flexibility enables CEDAR to unify concept-based and free-form interpretability within a unified framework. We evaluate CEDAR against several sparse autoencoder variants on standard benchmarks. Our method achieves a competitive reconstruction–sparsity trade-off, particularly in low-dimensional regimes, while yielding more interpretable explanations as demonstrated in a user study. These findings suggest that the semantic structure of vision–language embeddings can be effectively uncovered through a suitable change of basis, without increasing dimensionality or re-training model weights.
2
Related works
Vision-language models such as CLIP [1] learn powerful multimodal embeddings, but their internal structure remains difficult to interpret. Most existing approaches focus on local explanations, such as attribution maps or saliency methods [4, 5, 6, 7, 8], which identify input regions relevant to a specific prediction. While useful, these techniques provide limited insight into how semantic information is globally organized within the embedding space. Concept-based methods [9, 10, 11] aim to connect model representations with human-understandable concepts. Approaches such as Testing with Concept Activation Vectors (TCAV) [11] and linear probing identify directions in representation space corresponding to predefined concepts, often leveraging text embeddings in vision-language models. More recent work explores automated concept discovery and alignment without manual supervision. However, these approaches operate within the original embedding geometry and implicitly assume that meaningful concepts are already linearly separable. As a result, they do not attempt to restructure the representation to produce compact, sparse, or disentangled concept representations. 2
A prominent line of work in mechanistic interpretability uses sparse autoencoders (SAEs) [12, 13, 14, 15, 16] to uncover interpretable features in pretrained representations. These methods learn overcomplete dictionaries in which individual units correspond to approximately monosemantic features, addressing the problem of feature superposition in neural networks. By expanding the dimensionality and enforcing sparsity, SAEs enable the recovery of interpretable structure across a range of domains, including language and vision models. However, this approach relies on the assumption that disentangled features cannot be expressed in the original basis and require higherdimensional representations. Consequently, the learned representations are not unique, and the expansion alters the geometry of the original embedding space. Recent work has explored improving interpretability by reorganizing pretrained embeddings without modifying the underlying model. Methods such as EPIC [17, 18] and PluGeN [19, 20, 21] decompose representations into structured components or align them with semantic attributes. While these approaches demonstrate that reparameterization can improve interpretability, they do not explicitly enforce sparsity, axis alignment, or invertibility, and therefore do not yield a global coordinate system in which semantic factors are directly readable. Producing compact and information-preserving explanations remains a central challenge. Bottleneckbased approaches restrict representations to a small number of variables, often trading off interpretability against reconstruction fidelity [22, 23, 24]. In contrast, we maintain the original dimensionality of the embedding space and instead enforce sparsity in a learned basis. This allows us to study highly compressed, low-activation regimes while preserving information through invertibility. In contrast to prior work, we propose to learn a global, invertible reparameterization of the embedding space that induces sparsity and axis alignment directly in the original dimensionality. This formulation tests the hypothesis that much of the apparent entanglement in modern embeddings arises from basis misalignment rather than insufficient dimensionality. By combining sparsity, invertibility, and alignment with language, our method yields compact, globally consistent, and human-interpretable representations without expanding the embedding space.
3
Disentangled sparse reparameterization of embedding space
In this section, we introduce CEDAR (Conceptual Embedding Disentanglement via Adaptive Rotation), a post-hoc method for extracting interpretable structures from frozen vision-language embeddings. Unlike standard fine-tuning, CEDAR learns an invertible reparameterization of the embedding space. This approach, inspired by EPIC [17] and PluGeN [19], reorganizes existing features into latent semantic axes without altering the underlying encoder. This way, we effectively reorganize existing representations rather than learning new ones. First, we present the motivation for the problem under consideration. Next, we explain our method and describe its training procedure. Finally, we cover interpretability mechanism for multi-modal architectures. Motivation and problem statement. Our core hypothesis is that pretrained embeddings already contain disentangled semantic factors, but these factors are distributed across many coordinates and are therefore difficult to interpret. We seek a coordinate transformation U that reorganizes the embedding space such that semantic information becomes concentrated in a small number of dimensions. More specifically, we aim to obtain a representation with the following properties: • Energy concentration – most of the signal mass is captured by the top-k coordinates, allowing each sample to be represented by only a small set of active factors, • Axis alignment – individual coordinates correspond to coherent and reusable semantic factors that remain consistent across the data distribution, • Competition – dimensions compete for activation, discouraging redundant features and promoting specialization. Let z = E(x) ∈ RD be a fixed embedding obtained from a pretrained image encoder E. We assume the existence of a representation basis in which semantic factors are sparse and approximately axisaligned. Therefore, we look for k scalars αi such that the transformed embedding z̃ = U(z) is expressed as a sparse decomposition: 3
Figure 2: Overview of CEDAR architecture and training procedure. Given embeddings from a frozen backbone encoder, we apply an invertible, dimension-preserving transformation U to obtain a proxy representation in which most of the embedding norm is concentrated in a small subset of coordinates. A sparse, compressed representation is then obtained via a top-k selection operator. The inverse transformation U −1 maps the representation back to the original space. The model is trained in a reconstruction manner.
z̃ ≈
X
αi ei ,
|S| = k,
S ⊂ {1, . . . , D},
i∈S
where S denotes the indices of the top-k coordinates and ei is the canonical basis. Intuitively, S represents a small set of disentangled concepts sufficient to reconstruct the original embedding. Sparse representations are easier to interpret as individual coordinates can be associated with distinct concepts. At the same time, the transformation should remain invertible, enabling faithful reconstruction while preserving the geometric structure of the pretrained space, which is essential for CLIP-like models where text and image embeddings must remain aligned. This allows us to impose sparsity and disentanglement without altering the underlying representation space. This leads us to the following optimization problem: Problem formulation. Let z = E(x) ∈ RD . We seek a representation ẑ with bounded sparsity, such that reconstruction error is minimized: ẑ = arg min ∥z − z′ ∥1 ′ z
s.t.
∥z′ ∥0 ≤ k.
where k ≪ D and ∥.∥0 denotes l0 norm CEDAR. Rather than solving this sparse approximation problem directly in the original embedding space, where semantic factors are highly entangled, we transform the representation into a space where information becomes more concentrated and axis-aligned. To this end, we first introduce an 1-1 invertible transformation U : RD −−→ RD given by: onto
z̃ = U(z) = U (z − b), where b denotes the mean embedding computed over the training set, and U is described at the end of this secition. We then perform sparsification in the transformed space using a top-k operator Πk , which retains the k coordinates with the largest absolute magnitude and sets all remaining coordinates to zero. The resulting sparse representation is mapped back to the original space using the inverse transformation. Formally, we optimize the following reconstruction objective: min Ez U
z − U −1 Πk (U (z − b)) + b 1 .
(1)
Because U is invertible, the original embedding can be faithfully reconstructed while preserving the geometric structure of the pretrained representation space. 4
We parameterize U as an orthogonal linear operator: U = exp(A − A⊤ ), which guarantees invertibility U ⊤ U = I and preserves the geometry of the embedding space ∥U z∥2 = ∥z∥2 . The orthogonality constraint prevents the model from trivially concentrating information through simple coordinate scaling or dimensional collapse. Instead, the only way to minimize reconstruction error under top-k sparsification is to redistribute signal energy across coordinates such that most of the information becomes concentrated within a small subset of dimensions. D Training procedure. Let P = {pn }N n=1 ⊂ R denote the set of visual embeddings extracted from a frozen pretrained backbone. Depending on the architecture, P may correspond either to a single global representation (N = 1), as in CLIP-like models, or to a sequence of visual tokens, as in vision encoder–text decoder architectures such as BLIP or CoCa.
Each embedding vector is processed independently, p̂n = U −1 Πk (U (pn − b)) + b using the proposed orthogonal transformation U and top-k operator. We optimize the average reconstruction error according to Eq. 1 over all tokens. A key difficulty arises from the non-smooth transition between dense and sparse representations induced by the Πk operator. To address this, we employ a curriculum over the sparsity level k, starting from relatively dense reconstructions and gradually transitioning to the target sparse regime. After reaching the target sparsity, k is randomly sampled during training, which improves robustness across sparsity levels and mitigates overfitting to a fixed k value. Full formulation is provided in Appendix A. Sparse semantic explanations in VLMs. Once U has been trained, each coordinate in the trans−1 formed space corresponds to a direction in the original embedding space, given by cd = U[d,:] . These directions define a global set of semantic axes shared across all samples. Concept-based explanations. To associate individual axes with human-interpretable concepts, we leverage the CLIP [1] text encoder. Given a predefined vocabulary of text embeddings {tj }, each axis is matched to its nearest semantic concept using cosine similarity: j ∗ (d) = arg max j
⟨cd , tj ⟩ . ∥cd ∥∥tj ∥
For a given input, the sparse support set S identifies the subsetP of active concepts, while the corresponding coefficients determine their relative contribution: z̃ ≈ i∈S αi ei . This yields a sparse and globally consistent explanation mechanism, where interpretations are expressed directly in terms of reusable semantic coordinates rather than sample-specific attribution maps. Free-form explanations. In generative vision–language models, the same sparse representation can additionally be interpreted through the text decoder. After sparsification and reconstruction, the resulting embeddings are passed to the decoder without modifying the original generation pipeline. Importantly, all visual tokens are reconstructed using the same sparse coordinate system and the same global set of semantic axes. The generated description therefore depends only on a small subset of active coordinates determined by the sparsity level k. By decreasing the sampling temperature and generating multiple captions from the same sparse representation, one can further identify recurring words and semantic patterns. This provides a complementary form of interpretation, where semantic information is expressed not only through aligned coordinates, but also through natural language descriptions induced by the sparse embedding representation.
4
Interpreting image embeddings with text concepts
In this section, we first showcase qualitative visual examples of our method, followed by the results of three tasks from the user study we performed. The experimental setup is the same as in Section 5. Illustrative examples. We present a visual analysis of how CEDAR interprets complex scenes. This includes captions derived from a sparse BLIP [2] representation restricted to the top-10 most 5
CEDAR
MSAE
Concepts in disentangled CLIP space
Concepts in disentangled CLIP space
marley
dog husky
dogs
wolf
petrol wolf outdoors
Caption via sparse BLIP embedding: a dog standing in the grass Concepts in disentangled CLIP space
Concepts in disentangled CLIP space
apartment
apartment
wooden
wooden
victorian
dining
closeouts
goodwill
chairs
hertfordshire
Caption via sparse BLIP embedding: the table is made of wood
Figure 3: Comparison of CEDAR and MSAE interpretations. CEDAR associates concept keywords with sparse BLIP captions, whereas MSAE is restricted to keyword-based extraction. Bar lengths indicate activation score magnitudes. Notably, in the first example, CEDAR requires fewer active concepts for reconstruction than MSAE.
Activation: 15.84
Activation: 15.73
Activation: 15.22
Activation: 14.72
Figure 4: Images with highest activation for neuron 347 in the disentangled space, associated with the concept “lure”.
activating channels as well as disentangled CLIP [1] concepts. We compare these results with those obtained using the Matryoshka Sparse Autoencoder (MSAE) [16], a recent state-of-the-art SAE model. Figure 3 illustrates two representative examples where our model bridges the gap between high-level semantics and granular visual attributes. Additional examples showcasing the model’s interpretability across various domains are available in Appendix C. In the first example, while MSAE retrieves lower-level or noisier concepts such as “marley” and “petrol,” CEDAR more accurately identifies the presence of a “dog” alongside finer-grained taxonomic distinctions, including “husky” and “wolf.” Furthermore, CEDAR generates a sparse BLIP caption (“a dog standing on the grass”), demonstrating that semantic fidelity is preserved even under sparse representations, whereas the default MSAE formulation is not capable of producing natural language descriptions. In the second case, CEDAR further demonstrates a holistic interpretation by associating stylistic attributes such as “wooden” and “victorian” with a coherent descriptive caption. Together, these results suggest that CEDAR effectively disentangles distinct semantic factors, providing a more interpretable structure of the latent space. Following the interpretation of individual scenes, we analyze the semantic consistency of the disentangled space by examining top activations for specific neurons. Figure 4 displays the images that most strongly activate Neuron 347, which the model associates with the concept "lure." Across varying contexts, ranging from close shots of caught fish to images of anglers with equipment, the neuron consistently fires for visual features related to fishing. This uniformity indicates that the disentangled dimensions correspond to stable, human interpretable visual concepts rather than arbitrary features. 6
User study. We conducted three user studies to evaluate the quality of concept-based explanations and textual descriptions, including comparisons with MSAE and the dense (non-compressed) model. Additional details of the study protocols and example tasks are provided in Appendix D.1 and Appendix D.2. In Studies 1 and 2, we compare CEDAR with MSAE using embeddings obtained from the CoCa backbone [3]. In Study 3, we evaluate an encoder–decoder architecture consisting of a SWIN vision encoder [25] and a GPT-2 text decoder [26]. Study 1: Method preference. We evaluate pairwise preference between the sets of concept-based explanations using a 5-point scale. As shown in Figure 5, CEDAR is preferred over MSAE at both intensity levels, indicating a consistent shift towards CEDAR rather than isolated strong preferences. This suggests that CEDAR produces more representative and useful concept sets for describing image content.
User Preference of Concept Explanations
Strongly prefer MSAE Slightly prefer MSAE Neutral Slightly prefer CEDAR Strongly prefer CEDAR
60
40
20
0
20
40
60%
Figure 5: Participants consistently favor CEDAR across preference levels, with a significant share expressing a strong preference for our method. Study 2: Concept selection. We assess how individual concepts align with human judgments by asking participants to select all relevant concepts from a mixed set comprising 3 elements per method. Analysis of the selection distribution, shown in Figure 6, reveals a clear advantage of CEDAR over MSAE. Specifically, CEDAR exhibits a significantly higher density in the top-tier categories, where a majority (2 or 3 out of 3) of its suggested concepts were selected. Conversely, MSAE’s distribution is skewed toward the lower-end, with most cases 0 or 1 concepts being selected. This shift indicates that CEDAR consistently provides a more relevant set of descriptors, whereas MSAE’s concepts are frequently rejected by human evaluators. 0/3 1/3
2/3 3/3
MSAE 0
20
40
CEDAR 60
80
100%
0
20
40
60
80
100%
Figure 6: Distribution of the number of concepts generated by each method being selected. Concepts from CEDAR are more frequently identified as relevant compared to MSAE. Study 3: Description quality. We evaluate the quality of full textual descriptions obtained with text decoder based on the representations created by CEDAR, a sparsified representation w/o disentanglement,s and full dense representation (no sparsification) using a 3-point scale: does not describe, partially describes, and describes the image. Figure 7 shows the distribution of ratings across methods. The variant without disentanglement, i.e., sparsification directly in the original embedding space, is dominated by low ratings, indicating that its captions rarely describe the image. In contrast, CEDAR achieves a substantially higher proportion of top ratings and closely matches the dense baseline using full embeddings without sparsification. This indicates that learning the transformation U is essential for preserving semantic information under sparse compression.
Sparse Model w/o Disentanglement 100 80
60
40
Does not describe
20
0
20
40
Partially describes
60
CEDAR Sparse Model 80 100%
100 80
60
40
20
0
20
40
60
80 100%
40
60
80 100%
Dense Model
Describes
100 80
60
40
20
0
20
Figure 7: CEDAR achieves ratings close to the dense model without sparsification and substantially outperforms the non-disentangled sparse approach.
7
5
Evaluating the compressed embeddings
In this section, we evaluate the quality of compressed embeddings without referring to their text interpretations. Experimental setup. The experimental protocol closely follows the Matryoshka Sparse Autoencoder (MSAE) [16] framework to ensure a fair and standardized comparison. CLIP [1] with a ViT-L/14 [27] image encoder is used as the backbone, and all methods operate on the class token representation. All methods (CEDAR and 4 variants of SAE) are trained on ImageNet-1K [28] training set and evaluated on its validation split. Details of the experimental setup are included in Appendix B. To ensure a fair comparison between methods that differ in the dimensionality of the representation, we fix the average reconstruction error, FVU [29], and evaluate the remaining metrics: the number of nonzero activations (K), information capacity of the representation (IC), cosine similarity between original and reconstructed representations (CS), cross-entropy of the linear probing on ImageNet validation labels (LP), and Centered Kernel Nearest Neighbor Alignment (CKNNA) [30], see Appendix B.
Table 1: Comparison of the compressed embeddings under fixed reconstruction error levels. FV U
Model
K↓
IC ↓
CS ↑
LP ↓
CKN N A ↑
0.25
MSAE TopK ReLu BatchTopK CEDAR
8.090 7.001 27.305 4.207 11.549
59.488 52.206 171.395 32.939 57.381
0.866 0.866 0.870 0.866 0.865
2.062 2.068 2.456 1.946 2.087
0.405 0.444 0.339 0.416 0.425
0.3
MSAE TopK ReLu BatchTopK CEDAR
5.322 4.523 13.728 2.633 5.411
40.798 35.145 94.694 21.366 30.206
0.837 0.836 0.840 0.836 0.836
2.690 2.652 3.139 2.546 2.592
0.354 0.403 0.299 0.370 0.341
0.35
MSAE TopK ReLu BatchTopK CEDAR
3.406 2.786 6.828 1.591 2.690
27.109 22.461 50.911 13.286 16.272
0.807 0.805 0.810 0.804 0.806
3.491 3.413 3.978 3.327 3.351
0.296 0.352 0.247 0.323 0.247
How much information is needed for reconstruction? Table 1 analyzes the performance of disentanglement methods at three matched reconstruction error levels (FVU = 0.25, 0.3, 0.35), allowing us to study how much information is required to achieve a given reconstruction fidelity. By controlling FVU, we can directly compare methods in terms of the number of active features (K) and the corresponding information capacity (IC). Across all methods, lower reconstruction error consistently requires more active components and induces higher information capacity, indicating a clear trade-off between sparsity and reconstruction fidelity. Importantly, cosine similarity (CS) mirrors the behavior of FVU across settings, suggesting that it also reliably reflects reconstruction quality in this regime. At the method level, BatchTopK [31] shows a generally favorable trade-off between sparsity and reconstruction quality, often requiring relatively few active components while maintaining competitive similarity scores. TopK [29] performs comparably, frequently matching BatchTopK [31] in reconstruction quality, though sometimes using slightly more features. In contrast, ReLU [32] operates in a less compressed regime overall, relying on substantially more active features and higher information capacity. Despite these differences, CEDAR remains competitive across both sparsity and semantic preservation metrics, performing on par with the strongest baselines in most settings. Notably, in terms of information capacity (IC), CEDAR ranks among the top-2 methods at higher sparsity regimes, indicating that it achieves comparable reconstruction fidelity while using less encoded information. An additional advantage of our approach is that it is inherently lossless in its default form, as it is based on an invertible transformation. 8
BatchTopK CEDAR
MSAE ReLu
TopK
BatchTopK CEDAR
TopK
2.5
0.50 0.45
CKNNA
LP (CE)
2.0
0.40
1.5
0.35 0.30
MSAE ReLu
0.10
0.15
0.20
FVU
0.25
1.0
0.30
0.10
0.15
0.20
FVU
0.25
0.30
Figure 8: Semantic quality of reconstructed embeddings measured by CKNNA [30] (left) and linear probing cross-entropy (LP) (right) under varying FVU levels. Are the reduced embeddings semantically meaningful? Table 1 also reports semantic quality metrics, including CKNNA, which captures structural alignment, and linear probing cross-entropy (LP), which reflects downstream task performance. These metrics allow us to evaluate how well the reduced embeddings preserve meaningful structure and class-relevant information. From the table, we observe noticeable variation across methods even at matched reconstruction levels, suggesting that similar reconstruction fidelity does not guarantee comparable preservation of semantic structure. Figure 8 further illustrates how these metrics evolve as reconstruction error (FVU) decreases. As expected, increasing FVU leads to a gradual degradation in both metrics, indicating that reduced embeddings lose semantic information as reconstruction quality decreases. However, the rate of this degradation differs across methods. TopK and BatchTopK consistently maintain strong semantic alignment, with TopK achieving the highest CKNNA across most regimes, suggesting better preservation of neighborhood structure. In terms of downstream performance, BatchTopK and MSAE remain competitive, achieving lower cross-entropy compared to other methods at higher FVU levels. ReLU-SAE shows the weakest performance in both metrics, with significantly worse CKNNA and higher classification error, confirming that its denser representations do not translate into better semantic preservation. Our method follows trends similar to the strongest baselines, maintaining competitive performance in both CKNNA and linear probing across all FVU levels. While not always achieving the best scores, it preserves semantic structure and downstream performance at a comparable level, indicating that the reduced embeddings remain meaningfully aligned with the original representation space.
6
Conclusions
We introduced CEDAR, a post-hoc method for interpreting vision-language embeddings through sparse, disentangled, and text-aligned representations within the original embedding space. In contrast to sparse autoencoder-based approaches, which rely on overcomplete expansions, our method learns an invertible reparameterization that induces sparsity without increasing dimensionality. CEDAR achieves a reconstruction–sparsity trade-off comparable to existing methods while preserving semantic alignment and downstream performance. Its competitive information efficiency suggests that semantic structure in pretrained embeddings can be recovered without expanding the representation. More broadly, these findings support the hypothesis that much of the apparent entanglement in modern embeddings results from basis misalignment rather than insufficient dimensionality. Beyond quantitative performance, CEDAR provides a flexible framework for generating interpretable representations that can be aligned with language or used for downstream analysis. We believe this perspective opens new directions for understanding and manipulating learned representations through structured reparameterizations. Limitations. While promising, our current approach relies on orthogonal transformations, which limits its expressiveness. To address this limitation, future work could explore more flexible invertible 9
linear mappings or nonlinear architectures based on normalizing flows. Furthermore, CEDAR currently performs disentanglement in the embedding space before mapping the resulting factors to textual concepts. Incorporating text-based supervision directly into the training objective could further improve the semantic alignment of the learned factors.
Acknowlegdements The work of Jacek Tabor and Łukasz Struski was supported by the National Science Centre, Poland, grants no. 2023/49/B/ST6/01137. The research of Patryk Marszałek and Marek Śmieja was supported by the National Science Centre (Poland), grant no. 2023/50/E/ST6/00169. Some experiments were performed on servers purchased with funds from the flagship project entitled “Artificial Intelligence Computing Center Core Facility” from the DigiWorld Priority Research Area within the Excellence Initiative – Research University program at Jagiellonian University in Kraków. The contribution of Piotr Kubaty, Patryk Marszałek, and Adam Wróbel to this research was conducted at the Faculty of Mathematics and Computer Science, and the Doctoral School of Exact and Natural Sciences of the Jagiellonian University.
References [1] Alec Radford, Jong Wook Kim, Chris Hallacy, et al. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020, 2021. [2] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping languageimage pre-training for unified vision-language understanding and generation. In International conference on machine learning, pages 12888–12900. PMLR, 2022. [3] Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models, 2022. [4] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning (ICML), 2017. [5] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE, 10(7):e0130140, 2015. [6] Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), 2020. [7] Hila Chefer, Shir Gur, and Lior Wolf. Transformer interpretability beyond attention visualization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. [8] Adam Wróbel, Siddhartha Gairola, Jacek Tabor, Bernt Schiele, Bartosz Zieliński, and Dawid Rymarczyk. Dave: Distribution-aware attribution via vit gradient decomposition. arXiv preprint arXiv:2602.06613, 2026. [9] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, et al. Concept bottleneck models. In International Conference on Machine Learning (ICML), 2020. [10] Amirata Ghorbani, James Wexler, James Zou, and Been Kim. Towards automatic concept-based explanations. In Advances in Neural Information Processing Systems (NeurIPS), 2019. [11] Been Kim, Martin Wattenberg, Justin Gilmer, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International Conference on Machine Learning (ICML), 2018. [12] Trenton Bricken, Adly Templeton, et al. Towards monosemanticity: Decomposing language models with dictionary learning. arXiv preprint arXiv:2301.05217, 2023. 10
[13] Sam Cunningham et al. Scaling monosemanticity: Extracting interpretable features from claude models. arXiv preprint arXiv:2406.04093, 2024. [14] Leo Gao et al. Top-k sparse autoencoders. arXiv preprint arXiv:2501.XXXXX, 2025. [15] Senthooran Rajamanoharan et al. Gated sparse autoencoders. arXiv preprint arXiv:2404.16014, 2024. [16] Vladimir Zaigrajew, Hubert Baniecki, and Przemyslaw Biecek. Interpreting clip with hierarchical sparse autoencoders. arXiv preprint arXiv:2502.20578, 2025. [17] Piotr Borycki, Magdalena Tr˛edowicz, Szymon Janusz, Jacek Tabor, Przemysław Spurek, Arkadiusz Lewicki, and Łukasz Struski. Epic: Explanation of pretrained image classification networks via prototypes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 17366–17373, 2026. [18] Łukasz Struski, Dawid Rymarczyk, and Jacek Tabor. Infodisent: Explainability of image classification models by information disentanglement. arXiv preprint arXiv:2409.10329, 2024. [19] Maciej Wołczyk, Magdalena Proszewska, Łukasz Maziarka, Maciej Zieba, Patryk Wielopolski, Rafał Kurczab, and Marek Smieja. Plugen: Multi-label conditional generation from pre-trained models. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 8647–8656, 2022. [20] Magdalena Proszewska, Maciej Wołczyk, Maciej Zieba, Patryk Wielopolski, Łukasz Maziarka, and Marek Śmieja. Multi-label conditional generation from pre-trained models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(9):6185–6198, 2024. [21] Adrian Suwała, Bartosz Wójcik, Magdalena Proszewska, Jacek Tabor, Przemysław Spurek, and Marek Śmieja. Face identity-aware disentanglement in stylegan. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5222–5231, 2024. [22] Divyansh Srivastava, Ge Yan, and Tsui-Wei Weng. Vlg-cbm: Training concept bottleneck models with vision-language guidance. Advances in Neural Information Processing Systems, 37:79057–79094, 2024. [23] Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19187–19197, 2023. [24] Sukrut Rao, Sweta Mahajan, Moritz Böhle, and Bernt Schiele. Discover-then-name: Taskagnostic concept bottlenecks via automated concept discovery. In European Conference on Computer Vision, pages 444–461. Springer, 2024. [25] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows, 2021. [26] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. [27] Alexey Dosovitskiy et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021. [28] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009. [29] Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093, 2024. [30] Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. Position: The platonic representation hypothesis. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 20617–20642, 2024. 11
[31] Bart Bussmann, Patrick Leask, and Neel Nanda. Batchtopk sparse autoencoders. arXiv preprint arXiv:2412.06410, 2024. [32] Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2(5):6, 2023.
12
A
Formulation of the Training Curriculum
We consider a homotopy-style curriculum: k(t) = D +
t (ktarget − D), τ
t ≤ τ,
which gradually alters the objective from identity reconstruction (t = 0, k(t) = D) to a sparse projection (t = τ , k(t) = ktarget ). This can be interpreted as solving a sequence of optimization problems: h i min E ∥z − ẑ(k) ∥1 , k = D → ktarget , U
where each stage initializes the next. Once the target sparsity regime is reached, we introduce stochasticity by sampling: k ∼ U({1, . . . , kmax }), which prevents overfitting to a fixed support size and encourages robustness of the learned basis across different sparsity levels. We set ktarget to 10, as bigger values would harm interpretability of the sparse model, and kmax to 2ktarget − 1 to maintain expected value of k at ktarget . In practice, this leads to more stable support sets S and reduces sensitivity to the exact choice of k at inference time.
B
Details of the experimental setup
Datasets and baselines. We compare against MSAE [16] (reverse weighting) [16], TopK-SAE (k = 64) [29], ReLU-SAE [32] (λ = 0.01) [32], and BatchTopK-SAE (k = 32) [31], all trained on ImageNet-1K training set and evaluated on its validation split. Similarly to CEDAR, SAE models √ are standardized using the mean of CLIP [1] representations computed on the training set, with n used as a scaling factor. Our method uses the default 768-dimensional CLIP [1] embedding, while SAE baselines employ an 8× expansion to 6144 dimensions. All baselines are implemented using the official MSAE [16] codebase with default hyperparameters, except for the specific settings explicitly stated above. Metrics. In the experiment sections, we evaluate models using a range of complementary metrics. Fraction of Variance Unexplained (FVU), also known as normalized MSE, measures reconstruction quality by relating the reconstruction error to the variance of mean-centered inputs. We denote by K the number of non-zero activations in the sparse representation. Information capacity (IC) quantifies the amount of information that can be encoded using the k of n coordinates, which is defined as log nk , where n is the representation dimensionality (768 for CEDAR, 6144 for SAE baselines) and k is the number of non-zero neurons activations. The metric is computed per image sample and then averaged over the dataset. Alignment between original and reconstructed representations is captured using cosine similarity (CS). Semantic preservation is further evaluated via linear probing. Following the MSAE [16] pipeline, a linear classifier is trained on original representations with AdamW optimizer and a ReduceLROnPlateau scheduler. Performance is then measured on reconstructed embeddings using cross-entropy on ImageNet validation labels. Finally, Centered Kernel Nearest Neighbor Alignment (CKNNA) [30] measures structural consistency by comparing mutual nearest neighbor relationships in the original and reconstructed feature spaces. To ensure a fair comparison between methods that differ in the dimensionality of the sparse representation, we fix the average reconstruction error (FVU) and evaluate the remaining metrics. For each method, this target is matched by controlling the activation level in the sparse space, where activations below a threshold are set to zero. The threshold is selected independently for each method using binary search to achieve the desired FVU level.
C
Additional Qualitative Results of CEDAR
We first present more visual examples of CEDAR interpretations, extending the analysis from Section 4. Figure 9 provides further evidence of the model’s ability to maintain semantic consistency across diverse image categories while enforcing sparsity in the BLIP [2] embedding space and ensuring clear concept alignment in the disentangled CLIP [1] space. 13
Furthermore, we visualize the semantic purity of specific neurons in the disentangled space. Figure 10 illustrates top activations for the "printer" concept, while Figure 11 and Figure 12 demonstrate the model’s capacity to isolate complex social roles and specific technological styles, respectively. Collectively, these results reinforce that the disentangled dimensions represent stable and humaninterpretable visual primitives. Concepts in disentangled CLIP space
Concepts in disentangled CLIP space
lizard
dog
juvenile
kittens
purple
terrier
blue
white
red 0
2
4
6
8
10
12
0
Caption via sparse BLIP embedding: a lizard on a rock
2
4
6
8
10
12
14
Caption via sparse BLIP embedding: a dog sitting on top of a pile of leaves
Concepts in disentangled CLIP space
Concepts in disentangled CLIP space
dinner
sailing
soups
underwater
pulp
missile military 0
2
4
6
8
10
12
14
16
0
Caption via sparse BLIP embedding: a bowl of soup with a spoon full of soup
2
4
6
8
10
12
Caption via sparse BLIP embedding: the water is calm Concepts in disentangled CLIP space
Concepts in disentangled CLIP space
desserts
shenzhen
dinner
kyoto
red
kid fires downtown 0
1
2
3
4
5
6
0
7
2
4
6
8
Caption via sparse BLIP embedding: a close up of a red and white dessert
Caption via sparse BLIP embedding: a young boy is standing in front of a group of people
Figure 9: Further qualitative results illustrating CEDAR: human-readable concepts derived from the disentangled CLIP [1] space alongside captions produced from sparse BLIP [2] embeddings.
14
Neuron 12 ( printer ) Activation: 12.49
Activation: 12.41
Activation: 12.39
Activation: 12.38
Activation: 12.17
Activation: 11.99
Activation: 11.81
Activation: 11.80
Figure 10: Top-activating samples for neuron 12 in the disentangled representation, aligned with the concept “printer”.
Neuron 611 ( grandfather ) Activation: 12.49
Activation: 12.47
Activation: 11.98
Activation: 11.95
Activation: 11.79
Activation: 11.70
Activation: 11.41
Activation: 11.31
Figure 11: Examples eliciting the strongest responses from neuron 611 in the disentangled space, corresponding to the concept “grandfather”.
15
Neuron 203 ( analog ) Activation: 13.44
Activation: 12.47
Activation: 12.17
Activation: 11.84
Activation: 11.80
Activation: 11.56
Activation: 12.03
Activation: 11.51
Figure 12: Samples that maximally activate neuron 203 within the disentangled embedding, reflecting the concept “analog”.
D
User Study
D.1
Detailed Protocol
User studies were conducted using Google Forms. Studies 1, 2, and 3 involved 23, 22, and 21 participants, respectively. Study 1: Method Preference The study consisted of 8 independent comparison tasks. In each task, participants were shown a single image along with two concept-based explanations, labeled A and B. Each explanation was represented as a list of five concepts. The assignment of methods (CEDAR and MSAE [16]) to labels A and B was independently randomized for each task to mitigate ordering bias. Participants were instructed to rate their preference between the two explanations using a 5-point scale: Strongly prefer A, Slightly prefer A, Equally prefer A and B, Slightly prefer B, and Strongly prefer B. Study 2: Concept Selection The study consisted of 8 independent tasks. In each task, participants were presented with a single image and a set of six concepts. These concepts were drawn from two methods: three from CEDAR and three from MSAE [16]. All six concepts were displayed simultaneously and their order was randomized for each task to avoid method-specific grouping effects. Participants were instructed to select all concepts they considered relevant to the image, with no restriction on the number of selections. Study 3: Description Quality The study consisted of 10 independent tasks. In each task, participants were shown a single image along with three textual descriptions generated by different models: the dense model, our method, and a sparse model without a disentanglement mechanism. The order of the descriptions was independently randomized for each task. Participants were asked to evaluate how well each description reflects the image content, focusing on the overall impression rather than fine-grained details. Each description was independently rated using a 3-point ordinal scale: 1 – Does not describe the image, 2 – Partially describes the image, and 3 – Describes the image. Participants were allowed to assign the same rating to multiple descriptions. D.2
Instructions and Questionnaire
This section provides examples of the instructions, task descriptions, and survey questions used in each study. 16
Figure 13: User interface screenshots for two example tasks in Study 1. In the screenshot on the left, CEDAR explanation is shown as Concept list B, wheras in the one on the right, CEDAR explanation is shown as Concept list A
Figure 14: User interface screenshots for two example tasks in Study 2. In the screenshot on the left, CEDAR concepts are: trees, water, boat, and in the one on the right, CEDAR concepts are: dessert, bowl, table
17
Figure 15: User interface screenshots for two example tasks in Study 3. Descriptions generated by CEDAR are "A black bird standing on top of a lush green field." and "A view of a beach with a large body of water."
E
Broader Impacts
CEDAR is designed to improve the interpretability of pretrained vision-language representations by learning sparse and semantically structured reparameterizations of embedding spaces. By exposing which latent coordinates are active for a given input and aligning these coordinates with textual concepts, the method provides a more transparent view into the internal organization of large pretrained models. Since CEDAR operates post-hoc on frozen representations, it can also be applied without modifying the original model or requiring costly retraining. At the same time, the semantic concepts associated with individual coordinates are obtained through similarity to text embeddings and therefore do not constitute causal or guaranteed explanations of model behavior. Sparse decompositions may also inherit biases and spurious correlations present in the underlying pretrained representations. Overall, we believe that methods such as CEDAR can contribute toward more transparent and analyzable multimodal systems, while emphasizing that interpretability outputs require careful human interpretation and validation.
18