ConceptioArchivearXiv CS
arXiv CSopen access

CENDRe: Concept Extraction with Natural Domain Representations

Unknown · 2026 · arxiv_cs
arXiv CS · Papers · License: Open Access · 2026
Open Source ↗Direct PDF ↓
machine learning, deep learning, neural networks

CENDRe: Concept Extraction with Natural Domain Representations

Antonia Holzapfel∗ Andres Felipe Posada Moreno Sebastian Trimpe Institute for Data Science in Mechanical Engineering (DSME) RWTH Aachen University Theaterstraße 35-39, 52062 Aachen, Germany

arXiv:2607.29621v1 [cs.LG] 31 Jul 2026

Abstract Convolutional neural networks (CNNs) are widely used for time-series classification, but their deployment in critical domains requires understanding the temporal and spectral patterns that drive their predictions. Concept extraction (CE) methods identify such patterns by analyzing representations within the models’ latent space. However, existing time-series CE methods have three limitations: they operate only in the time domain and overlook frequency features, predefine the number of concepts, and produce localizations misaligned with the regions the model uses. We address these limitations by proposing CENDRe, a concept extraction method for CNNs. It first discovers concepts by clustering per-timestep latent representations in two stages, where silhouette-guided aggregation selects the number of concepts automatically. Then, it localizes each concept through gradients of a presence score that contrasts the latent representations with their prototypes, producing masks that concentrate on the regions driving the concept. These gradients, propagated through a differentiable invertible mapping of the input such as a Fourier transform, yield localizations for the same concepts in the frequency domain. Finally, each concept receives a relevance score that quantifies its contribution to each class. On synthetic benchmarks, CENDRe achieves representation correctness comparable to state-of-the-art CE methods and significantly higher importance correctness. On real bearing-fault data, CENDRe extracts the frequency bands driving the model’s predictions, located in regions commonly inspected for fault diagnosis, producing evidence to assess the model that time-domain CE methods cannot.

1

Introduction

Deep learning, particularly convolutional neural networks (CNNs), has become a widespread choice for time-series classification and regression. Applications range from condition monitoring and predictive maintenance on industrial equipment [25, 47] to early-warning and diagnostic systems in healthcare [26, 40, 14], settings where a wrong prediction can translate into costly misbehaviours. In such domains, stakeholders cannot simply take a prediction at face value: they need explanations to verify that it is based on plausible signal features and not on spurious correlations. For such inspection, explanations must be both faithful to the model’s behavior and expressed in an interpretable domain. Time-domain explanations cover at most half the story. Other domains, particularly the frequency domain, are often better suited to interpreting signals. Since models trained on time signals, especially CNNs, can learn features in either the time or frequency domain, explanation methods should also operate on both domains to make such features intelligible. Among XAI methods, concept methods [36] are post-hoc and grounded in the model’s own latent space, capturing its global behavior. Specifically, Concept Extraction (CE) discovers recurring latent ∗ Corresponding author: [email protected]

Preprint.

patterns that the model learned to distinguish across a dataset [24]. While original work in CE has focused on explaining images [15, 22, 37, 38], recent works have transferred CE to time series [19, 52]. However, these existing methods have three main limitations: First, their explanations live exclusively in the time domain, leaving any frequency-domain features the model uses unexplained. Second, the number of concepts K must be fixed before extraction, forcing a manual sweep that biases the resulting explanations. Yet K reflects how the model decides and should be determined by the data, not the user. Third, their localization methods produce masks that may either miss regions the model uses [38, 19] or cover regions it does not [52, 51]. These limitations restrict the practical usefulness of existing CE time-series methods, as highly relevant frequency-domain concepts are not covered, and strong priors on concept number and localization are included. We address these three limitations with CENDRe (Concept Extraction with Natural Domain Representations), a CE method for time-series CNNs that extracts concepts in any natural domain, that is, any invertible transform of the input in which signal content is directly interpretable, such as time or frequency. It does so by clustering timestep-wise activations in two stages, with silhouette-guided aggregation selecting the number of concepts automatically, rather than a user. Each concept is then localized through gradients of a contrastive presence score that compares the latent representations and concept prototypes, with respect to the input. These gradients produce masks that concentrate attribution on the regions influencing the concept. The same gradients, propagated through virtual inspection layers [48], yield concept localizations in other natural domains (for example, the frequency domain). We evaluate CENDRe on synthetic time-series with ground-truth concepts in both domains and on two natural bearing-fault datasets [28, 45]. Across all settings, CENDRe extracts concepts in both the time and frequency domains, while the baselines are confined to the time domain. On time-domain concepts, CENDRe outperforms the baselines on importance correctness while recovering the same ground-truth concepts. On frequency-domain concepts, CENDRe recovers the ground-truth bands on synthetic data and extracts bands that include the characteristic fault frequencies experts use to identify each fault type on real bearing-fault data [28, 45]. To summarize, the main contributions in this work are: 1. CENDRe. We introduce a CE method for time-series CNNs that, to our knowledge, is the first to extract and visualize concepts in both the time and frequency domains. 2. Concept localization mechanism that transfers across domains. We backpropagate gradients of a contrastive presence score through differentiable invertible transforms of the input, yielding concept masks in any natural domain. 3. Automatic concept number selection. We develop a procedure that automatically identifies the number of relevant concepts K from the latent structure of the data, providing insight into the model. This replaces the manual choice of K with the micro-cluster count J, to which the extraction is substantially less sensitive. 4. Evaluation on synthetic and real datasets. On synthetic time-series, CENDRe substantially outperforms prior CE baselines on importance correctness and uniquely recovers groundtruth concepts in the frequency domain alongside the time domain. On real bearing-fault datasets, it identifies the characteristic fault frequencies experts use for diagnosis.

2

Related Work

Concept-based explanations describe a model through human-interpretable patterns that correspond to directions or regions in its latent space [36]. Concept Extraction (CE) discovers such patterns automatically from data, without requiring a predefined example set. In the image domain, established CE methods include ACE [15], ECLAD [38], ConceptSHAP [51], and PACE [22]. In time-series, prior concept-based work has applied concept testing with TCAV [24] and manually curated concept sets [30, 12]. Only two recent works perform concept extraction: ECLAD-ts [19] clusters pertimestep local aggregated descriptors at a probe layer, and MultiVISION [52] clusters segments defined by the effective receptive field (ERF) of highly activated neurons. CENDRe is a CE method for time-series CNNs that extracts concepts across natural domains, including the frequency domain. Despite the central role of frequency domain analysis in signal processing and related areas [35, 3], no CE method operates in this domain. Spectral attribution methods, on the other hand, produce local explanations directly in the frequency domain either via perturbation or gradients. Perturbation 2

methods in the frequency or time-frequency domain [9, 5] probe the model through targeted alterations of the spectrum, and virtual inspection layers [48] insert a differentiable, invertible transform into the forward graph so that attribution can be propagated to any chosen signal representation. These methods are local, explaining individual predictions rather than global model behavior. CENDRe adopts the virtual inspection layer mechanism to bring global concept explanations into the frequency domain, which, to our knowledge, has not been done before. In CE the number of concepts K is typically fixed manually through sweeps or visual inspection [15, 22, 52, 38, 19]. The same pattern holds in prototype-based XAI, where the per-class prototype count is fixed in advance [8, 13], and silhouette scores appear only post hoc to evaluate prototype quality [10, 31]. In general clustering, by contrast, silhouette analysis has long been proposed as a criterion for selecting K [41], alongside alternatives such as density-based methods [6], which we include as variants. To our knowledge, CENDRe is the first CE method to determine the number of concepts automatically, guided by silhouette scores, and thus avoiding the inductive bias introduced by manual concept number selection. To localize concepts in the input, existing time-series CE methods typically rely on two heuristics: either upsampling low-resolution activation maps to the input size [38, 19], or taking the full ERF of the probe layer as a concept patch [52]. Upsampling attributes importance to the aggregation position even though its activation is driven by the full ERF, while the full-ERF patch can include regions the model did not meaningfully rely on. CENDRe avoids both caveats by taking the gradients with respect to the latent representation of the concept, propagated through virtual inspection layers, so that the attribution is concentrated on the regions that influence the concept, yielding compact localizations across domains.

3

Concept Extraction with Natural Domain Representations (CENDRe)

CENDRe is a CE method for time-series CNNs that produces explanations across natural domains with a data-driven number of concepts. Realizing this raises three core technical challenges: (i) discovering the number of concepts K from the data rather than imposing it, (ii) localizing each concept on the regions the model actually uses, and (iii) extending these localizations to natural domains beyond the input. The paragraphs below introduce our per-timestep latent representation, address each of these challenges, and conclude with a class-wise concept-importance score. Latent representation via LADs. For concept discovery, we need a post-hoc representation of the model’s latent space that captures patterns at multiple levels of abstraction and scales for clustering. We adapt LADs [38, 19]: timestep-wise aggregations of activations across layers that leverage CNNs’ approximate translation equivariance, so similar local patterns yield similar LADs regardless of position. To our knowledge, no other CE representation combines these properties. As a first step, we build a per-timestep descriptor that summarizes the model’s local encoding at multiple levels of abstraction. For this, we consider a trained 1D CNN f : RT ×D → Rnout for supervised classification or regression on multivariate time-series of length T with D channels and nout targets. We select a subset L of its internal layers as probe layers. Concept discovery operates on a dataset D ⊂ RT ×D . For each input sample x ∈ D, aℓ (x) ∈ RTℓ ×Dℓ denotes the activation map at probe layer ℓ ∈ L. We linearly upsample each activation map to the longest probe length T ∗ via fU . For each x ∈ D and timestep t ∈ {0, . . . , T ∗ − 1}, the LAD zt (x) is the per-timestep slice of the concatenated upsampled maps,   P ∗ zt (x) = fU (aℓ1 (x))[t] . . . fU (aℓ|L| (x))[t] ∈ RD , D∗ := ℓ∈L Dℓ . (1) Thus, similar LADs correspond to similar local evidence across the selected layers, and the collection of LADs over the dataset is the input to the first clustering stage. Concept discovery via clustering. CENDRe discovers latent concepts by clustering LADs zt (x) with agglomerative or density-based methods, which select the number of concepts K automatically and can capture non-spherical clusters expected in high-dimensional latent spaces. Applying such methods directly to all LADs is infeasible at scale, so we first compress LADs to micro-centroids, and then cluster those micro-centroids into concepts. 3

In the first stage, mini-batch k-means on LADs produces a set of J ≫ K micro-centroids Γ = D∗ {µj }J−1 , and we assign each LAD to its nearest micro-centroid, j=0 with µj ∈ R π(x, t) :=

arg min ∥zt (x) − µj ∥22 .

(2)

j∈{0,...,J−1}

In the second stage, we cluster the J micro-centroids into K concepts via a mapping κ : {0, . . . , J − 1} → {0, . . . , K − 1}, so each concept Ck = {µj | κ(j) = k} is a group of micro-centroids capturing similarly encoded patterns. By default, we use silhouette-guided hierarchical aggregation under cosine distance, taking K as the value that maximizes the average silhouette score across micro-centroids (CENDResilhouette ) [33, 41]. We additionally evaluate two variants in Sec. 5: k-means [43] directly on LADs with predefined K (CENDRekMeans ), and HDBSCAN [6] on the micro-centroids (CENDReHDBSCAN ). Hyperparameters are listed in App. C.3. Concept masks. We localize each concept on each input through gradients with respect to an interpretable domain. Gradients concentrate the attribution on regions that relate to the concept, rather than covering the full receptive field or shrinking past the relevant evidence within it. Pulling the same gradient through any invertible differentiable transform allows the visualization of the concept in the corresponding domain (e.g. the input domain or the frequency domain). The presence score, defined below, captures what makes a LAD belong to concept Ck by contrasting its similarity to the assigned micro-centroid with the mean similarity to micro-centroids of the other concepts. To express this contrast, we partition the micro-centroid indices into the in-concept set Jk := κ−1 (k) ⊆ {0, . . . , J − 1} and its complement J¯k := {0, . . . , J − 1} \ Jk , and we collect the timesteps where the LAD of x is in Ck in Tk (x) := {t ∈ {0, . . . , T ∗ − 1} | π(x, t) ∈ Jk }. To assess how similar a LAD is to a micro-centroid, we use the inverse-decay similarity d(z, µ) := (1 + ∥z − µ∥2 )−1 ∈ (0, 1] between a LAD z and a micro-centroid µ. For every t ∈ Tk (x), the presence score is then   1 X ρk (x, t) := d zt (x), µπ(x,t) − ¯ d zt (x), µj . (3) |Jk | j∈J¯k

A high presence score indicates strong concept membership: the LAD is close to its assigned microcentroid and far from every micro-centroid of the other concepts. We aggregate it over Tk (x) to P obtain a sample-level concept presence ρ̄k (x) := |Tk (x)|−1 t∈Tk (x) ρk (x, t), with ρ̄k (x) = 0 when Tk (x) = ∅. To obtain a mask in an arbitrary domain we follow virtual inspection layers [48]: a differentiable invertible transform ϕ is inserted into the computational graph as ϕ−1 ◦ ϕ for attribution only, leaving the model’s forward mapping unchanged. The transform is fixed and has no learnable parameters, so no training is involved. We instantiate ϕ in two ways: the input domain (ϕ = id) and the frequency domain (ϕ = F), where F : RT ×D → CF ×D is the real-input fast Fourier transform (rFFT) with F := ⌊T /2⌋ + 1 and xf := F(x). App. D.1.2 instantiates two further choices, the short-time Fourier and discrete wavelet transforms, which resolve concepts jointly in time and frequency. Through the inserted inverse transform, ρ̄k becomes a function of xf , and ∇xf ρ̄k denotes its gradient at xf = ϕ(x). The corresponding masks are the sensitivities of ρ̄k to the input and to the log-power-spectrum, respectively (derivation in App. A),   mtime (x) := ReLU ∇x ρ̄k (x) , mfreq (4) k k (x) := ReLU ℜ xf ⊙ ∇xf ρ̄k (x) . The ReLU keeps the components where amplifying the signal, or its spectral power, increases the concept’s presence. Negative components mark evidence against the concept and are discarded for localization. Although LADs are indexed on the upsampled grid t ∈ {0, . . . , T ∗ − 1}, masks are obtained by backpropagation to the original input x ∈ RT ×D or its rFFT image xf ∈ CF ×D . For visualization, a noise tunnel is applied on the masks [44] and they are smoothed by a moving average along the domain axis. Concept importance scores. For each class c, concept Ck , and input channel ch, we ask whether the presence of Ck on channel ch pushes the model toward or away from predicting c, and by how much. The importance score Ic,k,ch ∈ [−1, 1] encodes this: magnitude reports the strength of the 4

effect. The sign on its own carries no meaning, but opposite signs across classes indicate that the concept pushes those classes apart. We compute Ic,k,ch from the gradient of the per-class score yc with respect to the input, weighted by the concept mask of Ck on channel ch. For classification, yc (x) is the logit f (x)c . For regression, yc (x) is the c-th regression output. We measure the instance-level relevance as the overlap between the regions where the concept lives dom and the regions to which yc is sensitive. Let mdom k,ch (x) and sc,ch (x) denote the channel slices of the mask and the sensitivity of yc in domain dom ∈ {time, freq}. We write xch ∈ RT and xf,ch ∈ CF for the time- and frequency-domain channels of x. The relevance is then their inner product over the dom corresponding domain axis, rc,k,ch (x) := mdom k,ch (x), sc,ch (x) . time The sensitivity sdom c,ch (x) is obtained by backpropagation in the chosen domain: sc,ch (x) =  freq dom ∂yc (x)/∂xch , and sc,ch (x) = ℜ xf,ch ⊙ ∇xf,ch yc (x) (App. A). We smooth sc,ch (x) with a noise tunnel to damp single-point gradient noise. Then, we average rc,k,ch (x) over Dk,ch := { x ∈ D | mdom k,ch (x) ̸= 0 }, the samples on which Ck leaves a non-zero mask on channel ch, to obtain a datasetlevel estimate of the concept’s effect. If Dk,ch = ∅, we set the expectation to 0. We divide the result by the largest such average across all (c′ , k ′ , ch′ ) to make entries comparable on [−1, 1]:

Ic,k,ch =

Ex∼Dk,ch [ rc,k,ch (x) ] , maxc′ ,k′ ,ch′ Ex∼Dk′ ,ch′ [ rc′ ,k′ ,ch′ (x) ] + ε

for a small ε > 0. To obtain a single, class-agnostic importance value per concept–channel pair (as required by downstream metrics), we contrast the most-impacted class against the mean of the rest, Ik,ch = 0.5 Ic⋆ (k,ch), k,ch −

1

X

nout − 1

c̸=c⋆ (k,ch)

Ic,k,ch ∈ [0, 1],

(5)

with c⋆ (k, ch) := arg maxc |Ic,k,ch |. For nout = 1, we set Ik,ch := |I0,k,ch |. Values close to 1 identify concepts that selectively drive the prediction for at least one class, whereas values close to 0 mark concepts whose contribution is either negligible or uniform across classes.

4

Experimental design

A CE method should identify the recurring patterns a model learns, how they relate to the input, and how they influence the predictions. Yet, there is no direct ground truth on what a CNN relies on. Thus, we evaluate CE methods on synthetic datasets where each input is annotated with the masks of its class-discriminative primitives, assuming a model trained on these datasets relies on them. A faithful CE method should produce concepts tied to those primitives (representation) and assign higher importance to concepts on discriminative primitives than to unrelated ones (importance). We compare CE methods by running each through multiple trained models and measuring representation and importance on the resulting concepts. In an experimental run, we train a CNN on a synthetic dataset, run the CE method on the same data to obtain concepts with time- and frequency-domain masks and per-class importance scores, and score these concepts on the validation data against the primitive masks. We repeat across datasets and seeds, then compare CE methods on representation and importance correctness on the synthetic data, and qualitatively on the real bearing-fault datasets CWRU and BearingPD by inspecting how concepts relate to characteristic fault frequencies. 4.1

Evaluation metrics

To evaluate a CE method, we assess how much the resulting concepts align with the primitives through two correctness scores: representation, measuring how well concept masks align with primitive masks, and importance, measuring whether each concept’s importance score reflects its primitive’s function in the dataset. To this end, we propose two modifications over the evaluation procedure of [38, 19]: We score concept alignment with the Relevance Mass Accuracy rather than the EDT score, and we compute the correctness metrics using soft variants (sRC, sIC) that avoid the sensitivity to an alignment threshold of the original Representation and Importance Correctness scores. 5

Alignment. For the alignment comparison between concepts and primitives to make sense, all masks must be on the same scale. CENDRe masks mdom (x) can take any real value, so we normalize k them per instance to m̃dom (x) ∈ [0, 1] by clamping negatives to zero and dividing by the maximum, k while baseline masks are already bounded or binary. On the ground-truth side, gpdom (x) ∈ {0, 1} is the binary mask of primitive p, with np primitives and K extracted concepts in total. We score the alignment with the Relevance Mass Accuracy (RMA) [2], which for a continuous mask u ∈ R|dom|×D and binary reference v ∈ {0, 1}|dom|×D (with |time| = T and |freq| = F ) measures the fraction of u’s positive energy that falls inside v, P t,ch ReLU(ut,ch ) · 1[vt,ch > 0] P ∈ [0, 1], (6) RMA(u, v) = t,ch ReLU(ut,ch ) + ε where ε > 0 penalizes empty masks. We choose RMA over the EDT score because it accepts continuous masks directly and tolerates models that use only part of a primitive’s support. Representation and Importance metrics. With the concept and primitive alignments computed per instance, we aggregate them into one score per property. Following [19], we treat each pair (k, ch) of concept and channel as its own evaluation unit, and define its alignment to a pair (p, ch) of primitive dom and channel as the average RMA over the dataset, Ap,k,ch := Ex∼D [RMA(m̃dom k,ch (x), gp,ch (x))] ∈ [0, 1]. For each primitive, Soft Representation Correctness (sRC) picks the concept whose alignment summed across channels is highest, and averages this over primitives and channels. Soft Importance Correctness (sIC) instead weights each concept’s class-agnostic importance Ik,ch from Eq. (5) by its best alignment to any primitive,  P X 1 X k,ch maxp Ap,k,ch Ik,ch P sRC := max Ap,k,ch , sIC := ∈ [0, 1]. (7) np D p k k,ch |Ik,ch | ch

4.2

Experimental setup

Models and training. We train three 1D CNN architectures: InceptionTime10 [21], ResNet1D18 [16], and DenseNet1D-121 [20]. Each model takes the standard hyperparameters of its reference implementation, with full configurations listed in App. C.1. We optimize the negative log-likelihood using AdamW [29] (learning rate 10−4 , weight decay 10−7 ) with early stopping on the validation loss (patience 20, up to 500 epochs). Each dataset is split 80/20 with balanced sampling, and every training configuration is repeated over 11 seeds (0–10). Datasets. We use two synthetic binary-classification families with known ground-truth primitive masks: SYNTHETIC L OCAL, which injects localized temporal shapes (square, circle, triangle) into a noisy sinusoidal baseline, and SYNTHETIC F REQUENCY, which places class-discriminative peaks in specific frequency bands (mid, mid-high, high). For real-world signals we use CWRU Bearing [45], a rolling-element bearing fault dataset with four classes (healthy plus three fault locations) and two accelerometer channels (fan end and drive end). Both channels are sampled at fs = 12 kHz and segmented into windows of T = 2048 samples with stride 1024 (50% overlap). The full dataset list is in Table 1 of App. B, with additional synthetic variants and a second bearing-fault benchmark, which are evaluated in App. D. Concept extraction hyperparameters. All CE methods are implemented in PyTorch [1] and require probe layers per model, listed in Table 4 of App. C. Each method runs on 256 samples with the same random seed as the corresponding model training, is fit on the model’s training partition and evaluated on its validation partition. The methods differ in how the number of concepts K is set. For fixed-K methods (ECLAD-ts, MultiVISION, CENDRekMeans ) we sweep over K ∈ {2, 4, 6, 8, 10} and report the configuration with the highest sRC. For methods that determine K automatically (CENDResilhouette , CENDReHDBSCAN ) we take 50 micro-clusters, a choice whose effect the sensitivity analysis in App. E quantifies. CENDRe’s noise tunnel uses 5 samples with Gaussian perturbations of standard deviation 0.01 for instance masks, 0.1 for logit sensitivities, and a moving average of 5 for visualizations. 6

5

Empirical results

We report results on the SYNTHETIC L OCAL and SYNTHETIC F REQUENCY dataset families, and on the natural bearing-fault dataset CWRU. The experimental setup follows Section 4, with further evaluations in App. D. The main findings of our experiments are: (i) On SYNTHETIC L OCAL, CENDRe extracts concept regions that match the ground truth qualitatively, with representation correctness similar to ECLAD-ts but higher importance correctness. (ii) Frequency-domain concepts on SYNTHETIC F REQUENCY localize in the ground-truth bands, with high representation and importance correctness. (iii) Concepts extracted from CWRU separate cleanly by class and align with characteristic fault frequencies, evidencing that CENDRe transfers from controlled synthetic settings to real signals.

C0 I0 = −1.0 I1 = 1.0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 1

ŷ: 1

1 0 0.01

0.00

C1 I0 = 0.6 I1 = −0.6

CENDResilhouette

Importance Score

1 0

0.005 0.000 0

Importance Score

100

200

y: 0

300

400

0

ŷ: 0

100

200

y: 0

300

400

0

ŷ: 0

100

200

y: 0

300

400

0

ŷ: 0

100

200

y: 1

300

400

0

ŷ: 1

100

200

y: 1

300

400

0

ŷ: 1

100

200

y: 1

300

400

ŷ: 1

1.5

ECLAD-ts

C0 I = −0.6

1.0 0.5 0.0 −0.5

1.5

C1 I = −1.0

1.0 0.5 0.0 −0.5 0

Importance Score

200

y: 0

400

0

ŷ: 0

200

y: 0

400

0

ŷ: 0

200

y: 0

400

0

ŷ: 0

200

y: 1

400

0

ŷ: 1

200

y: 1

400

0

ŷ: 1

200

y: 1

400

ŷ: 1

1.5

C0 I = 1.0

0.5 0.0 −0.5

1.5 1.0

C1 I = 0.7

MultiVISION

1.0

0.5 0.0 −0.5 0

200

400

0

200

400

0

200

400

0

square

Primitives

y: 0

y: 0

200

400

0

200

400

0

200

400

triangle

y: 0

y: 1

y: 1

y: 1

1

0

0.0

0.5

1.0 0.0

0.5

1.0 0.0

0.5

1.0 0.0

0.5

1.0 0.0

0.5

1.0 0.0

0.5

1.0

Time [s]

Figure 1: Concepts extracted from a ResNet1D-18 trained on a SYNTHETIC L OCAL square-triangle dataset. Within each grid, rows are concepts, columns are samples, and row labels show importance scores. The bottom row shows the ground-truth primitive locations for the same samples. CENDRe and ECLAD-ts extract the same input features, while MultiVISION shows no coherent pattern. Synthetic datasets. The SYNTHETIC L OCAL family tests whether the compared CE methods extract the local cues time-series CNNs rely on for classification. Figure 1 shows CE on a representative square-triangle variant from this family. ECLAD-ts highlights the complete square defined by the ground-truth primitive. CENDRe picks up the same regions with importance scores of similar magnitude, but weights the corners more strongly, where the signal differs most recognizably from the baseline. MultiVISION, by contrast, returns no coherent pattern. Overall, CENDRe extracts the local cues without tuning the number of concepts, and additionally highlights the most important 7

structures within each concept. Figure 7 of App. D.1.1 repeats this comparison on two further architectures and the two remaining CENDRe variants, with the same outcome. The quantitative metrics in Figure 2 (top row) are consistent with this qualitative picture. For the

Figure 2: Boxplots of sRC (left) and sIC (right) on SYNTHETIC L OCAL (top) and SYNTHETIC F RE QUENCY (bottom); higher is better. On SYNTHETIC L OCAL , CENDRe achieves sRC close to ECLAD-ts and higher sIC, while MultiVISION is weak on both metrics. On SYNTHETIC F RE QUENCY , CENDRe variants reach sRC and sIC values close to those on SYNTHETIC L OCAL . CENDResilhouette/HDBSCAN variants, sRC is slightly lower than ECLAD-ts’s on SYNTHETIC L OCAL (k-means is essentially on par). Two factors plausibly contribute. First, the model exploits only a portion of the ground-truth primitive (a shortcut-learning pattern), and CENDResilhouette/HDBSCAN stays faithful to that partial use, while ECLAD-ts’s broader masks cover the full primitive. Second, k-means and ECLAD-ts are reported at the K with best sRC, which biases them toward humanrecognizable concepts, whereas silhouette and HDBSCAN choose K from the latent structure itself. The latter is also visible in sIC, which is higher for the CENDResilhouette/HDBSCAN variants, meaning their masks better match the model’s per-timestep importance signal. MultiVISION underperforms on both metrics, in line with the lack of coherent concept structure observed qualitatively. The same ranking holds on the multichannel and confounded variants of the family (Figure 10 of App. D.1.5). Per-dataset values and pairwise significance tests are in App. D.1.3 and App. D.1.4. Across CENDRe variants the differences are small qualitatively (the three variants point to the same regions). Quantitatively CENDResilhouette and CENDReHDBSCAN perform similarly, while CENDRekMeans trades slightly higher sRC for slightly lower sIC, consistent with the K-selection effect noted above. We favor the silhouette variant because it reaches comparable quality in a single run without a K sweep, and picks a usable K across datasets out of the box. It is also the more robust of the two to its remaining hyperparameter, the number of micro-clusters J. Over three orders of magnitude, J explains 0.02 of the sRC variance under CENDResilhouette and 0.14 under CENDReHDBSCAN , against 0.70 for the random seed (App. E). The SYNTHETIC F REQUENCY family tests whether CE methods extract spectral cues exploited by the model. Among the compared methods, only CENDRe extends to the frequency domain, through its virtual inspection layers, so the baselines are omitted from this analysis. Figure 3 shows that the time-domain masks have no interpretable pattern, whereas the frequency-domain masks concentrate concept mass at specific bands, both per-sample and in the global concept correspondence. In the midhighFreq-highFreq variant shown, C0 aligns with a single high-frequency band, while C1 mixes mid-high and near-zero content, reflecting how the model composes information from several frequency components. When a model relies on frequency-related features, CENDRe captures them at both instance and global scale, turning uninterpretable time-domain masks into interpretable frequency explanations. Quantitatively, Figure 2 (bottom row) shows the same CENDRe behavior as the local case. CENDResilhouette/HDBSCAN variants reach slightly lower sRC than k-means but higher 8

Importance Score

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 1

ŷ: 1

C0 I0 = −1.0 I1 = 1.0

Channel 0

0.00

−0.05

0.15

0.05 0.00

C0

0.05

C1 I0 = −0.6 I1 = 0.6

Time domain

0.05

0.10

0.00

0.05 −0.05

0.025

0.00

0.000 0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Time [s]

ŷ: 0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 1

ŷ: 1 0.15

C1

C0 I0 = −1.0 I1 = 1.0

y: 0 10 5

0.10

0.05

0.0005 0.0000

C1 I0 = −0.6 I1 = 0.6

Frequency domain

Importance Score

10

0.00

5

0

50

100

150

200

Frequency [Hz]

0.0005 0.0000 0

50

100

150

200

0

50

100

150

200

0

50

100

150

200

0

50

100

150

200

0

50

100

150

200

0

50

100

150

200

Frequency [Hz]

Figure 3: Concepts extracted from a ResNet1D-18 trained on a SYNTHETIC F REQUENCY midhighFreq-highFreq dataset. The two left grids show time-domain (top) and frequency-domain (bottom) masks; within each, rows are concepts, columns are samples, and row labels show importance scores. The right panel shows the global concept-frequency correspondence, obtained by aggregating the frequency masks across samples. Time-domain masks show no interpretable pattern, whereas frequency-domain masks localize in specific bands, as confirmed by the correspondence plot.

sIC. Mean sRC is slightly below SYNTHETIC L OCAL’s, while sIC is comparable. CENDRe thus extracts concepts in the frequency domain at quality close to the time domain. Natural datasets. On CWRU (Figure 4), concepts separate mostly by class. C0 aligns with a sharp, dominant peak at ∼2000 Hz on the drive-end channel, and shows weaker activity at ∼250 and ∼1000 Hz on the fan-end channel. Concepts C2 and C3 correspond more weakly to bands around 800–1000 Hz on the drive-end channel and on the fan-end channel, respectively. These specific bands give domain experts the information needed to assess whether the model is making predictions for the right frequency reasons. Analogous results on BearingPD (App. D.2.2) and on five datasets of the UCR archive [11] (App. D.2.1) support that CENDRe transfers from controlled synthetic settings to real signals across domains.

6

Conclusion

We have presented CENDRe, a concept extraction method for time-series CNNs that addresses the three limitations of prior time-series CE methods. First, by propagating concept masks through virtual inspection layers, CENDRe extracts concepts in both time and frequency domains. The frequency masks recover the ground-truth bands on synthetic data and locate concept-specific bands on the natural bearing-fault datasets, where they support verification by domain experts. Second, silhouette-guided aggregation removes the need to fix the number of concepts by hand and delivers results comparable to the best configuration of a manual K sweep in a single run. This replaces the hyperparameter K with the much less sensitive micro-cluster count J. Third, the gradient-based localization concentrates on the regions that drive the concept, yielding representation correctness comparable to prior methods and higher importance correctness on the synthetic benchmarks. Being the first CE method to combine time and frequency domain concepts, CENDRe opens new avenues for future research. The same pipeline can be applied to other invertible representations such as time-frequency distributions. Architectures, by contrast, remain a current limitation: CENDRe’s per-timestep LADs rely on the approximate translation equivariance of 1D CNNs. Extending the latent representation to architectures without this property, such as transformers or state-space models, is a natural next step. Furthermore, the discovered concepts could feed back into the development cycle, for instance to guide model debugging or to flag suspicious reliance on shortcut cues. Domainspecific user studies, assessing whether CENDRe’s masks help experts within their own field, are also a natural direction for future work. 9

Frequency domain

Time domain Importance Score

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 2

ŷ: 2

y: 3

Importance Score

ŷ: 3

0

ŷ: 0

2000

4000

y: 1

ŷ: 1

y: 2

ŷ: 2

2000

4000

y: 3

ŷ: 3

2000

4000

200000

−5

0

0.025

0.05

0.000

0.00

10000

0.5

C0 — fe I2 = 1.0 I3 = −0.7

C0 — fe I2 = 1.0 I3 = −0.7

y: 0

400000

C0 — de I0 = −0.6 I2 = 0.9

C0 — de I0 = −0.6 I2 = 0.9

5

0.0

5000

−0.5 0.01

0.01

0.00

10000

C1 — fe I2 = 1.0 I3 = −0.8

C1 — fe I2 = 1.0 I3 = −0.8

0.00

0.5 0.0

5000

−0.5 0.01

0.01 0.00

10000

C2 — fe I2 = 0.2

C2 — fe I2 = 0.2

0.00

0.5 0.0

5000

−0.5 0.005 0.01

0.000

0.00

400000

C3 — de I2 = 0.2

C3 — de I2 = 0.2

5

0

200000

−5

0

0.005

0.01

0.000 0.00

0.05

0.10

0.15

0.00

0.05

0.10

0.15

0.00

0.05

0.10

0.15

0.00

0.05

0.10

0.00

0.15

0

6000

0

2000

4000

Time [s]

6000

0

6000

0

6000

Frequency [Hz]

C0

C1

C2

C3

de

0.2

0.1

0.0

0.3

0.2

fe

Concept correspondence

0.3

0.1

0.0 0

2000

4000

6000

0

2000

4000

6000

0

2000

4000

6000

0

2000

4000

6000

Frequency [Hz]

Figure 4: Concepts extracted from an InceptionTime10 trained on CWRU. The top grids show time-domain (left) and frequency-domain (right) masks; rows are concepts on channels with high importance and columns are samples. The bottom panel shows the concept-frequency correspondence per channel, with a fan end (fe) and drive end (de) channel. Each concept localizes almost exclusively on one class, with the frequency masks exposing characteristic fault frequencies.

Acknowledgments and Disclosure of Funding We thank Franziska Hoffmann for her valuable help in designing the synthetic datasets. In addition, model training and experiments were partially performed with computing resources granted by RWTH Aachen University, under projects p0021919 and p0027430. This work is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 520460745. Funded by the European Union. This work has received funding from the European High Performance Computing Joint Undertaking (JU) and from the German Federal Ministry of Research, Technology and Space (BMFTR), the Ministry of Culture and Science of North Rhine-Westphalia (MKW NRW) and the Hessian Ministry of Science and Research, Arts and Culture (HMWK) under grant agreement No 101250682.

References [1] Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang, Kshiteej Kalambarkar, Laurent Kirsch, Michael Lazos, Mario Lezcano, Yanbo Liang, Jason Liang, Yinghai Lu, C. K. Luk, Bert Maher, Yunjie Pan, Christian Puhrsch, Matthias Reso, Mark Saroufim, Marcos Yukio Siraichi, Helen Suk, Shunting Zhang, Michael Suo, Phil Tillet, Xu Zhao, Eikan Wang, Keren Zhou, Richard Zou, Xiaodong Wang, Ajit Mathews, William Wen, Gregory Chanan, Peng Wu, and Soumith Chintala. Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compi10

lation. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’24, page 929–947. Association for Computing Machinery, 2024. doi: 10.1145/3620665.3640366. [2] Leila Arras, Ahmed Osman, and Wojciech Samek. Clevr-xai: A benchmark dataset for the ground truth evaluation of neural network explanations. Information Fusion, 81:14–40, 2022. [3] Karl Johan Åström and Richard Murray. Feedback systems: an introduction for scientists and engineers. Princeton university press, 2021. [4] James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Journal of machine learning research, 13(2), 2012. [5] Thea Brüsch, Kristoffer K Wickstrøm, Mikkel N Schmidt, Tommy S Alstrøm, and Robert Jenssen. Explaining time series models using frequency masking. arXiv preprint arXiv:2406.13584, 2024. doi: 10.48550/arXiv.2406.13584. [6] Ricardo JGB Campello, Davoud Moulavi, and Jörg Sander. Density-based clustering based on hierarchical density estimates. In Pacific-Asia conference on knowledge discovery and data mining, pages 160–172. Springer, 2013. [7] Francesca Campolongo, Jessica Cariboni, and Andrea Saltelli. An effective screening design for sensitivity analysis of large models. Environmental modelling & software, 22(10):1509–1518, 2007. [8] Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems, 32, 2019. [9] Hyunseung Chung, Sumin Jo, Yeonsu Kwon, and Edward Choi. Time is not enough: Timefrequency based explanation for time-series black-box models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 394–403, 2024. doi: 10.1145/3627673.3679844. [10] Enyan Dai and Suhang Wang. Towards prototype-based self-explainable graph neural network. ACM Transactions on Knowledge Discovery from Data, 19(2):1–20, 2025. [11] Hoang Anh Dau, Eamonn Keogh, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, Yanping, Bing Hu, Nurjahan Begum, Anthony Bagnall, Abdullah Mueen, and Hexagon-ML Batista, Gustavo. The ucr time series classification archive, October 2018. [12] Thomas Decker, Michael Lebacher, and Volker Tresp. Explaining Deep Neural Networks for Bearing Fault Detection with Vibration Concepts. In INDIN, pages 1–6, 2023. doi: 10.1109/INDIN51400.2023.10218170. [13] Jon Donnelly, Alina Jade Barnett, and Chaofan Chen. Deformable protopnet: An interpretable image classifier using deformable prototypes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10265–10275, 2022. [14] Riccardo Fratti, Niccolò Marini, Manfredo Atzori, Henning Müller, Cesare Tiengo, and Franco Bassetto. A multi-scale cnn for transfer learning in semg-based hand gesture recognition for prosthetic devices. Sensors, 24(22):7147, 2024. [15] Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards Automatic Conceptbased Explanations. In NeurIPS, volume 32. Curran Associates, Inc., 2019. [16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016. [17] Jon Hills, Jason Lines, Edgaras Baranauskas, James Mapp, and Anthony Bagnall. Classification of time series by shapelet transformation. Data mining and knowledge discovery, 2014. 11

[18] Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics, pages 65–70, 1979. [19] Antonia Holzapfel, Andres Felipe Posada Moreno, and Sebastian Trimpe. Concept extraction for time series with eclad-ts. In World Conference on Explainable Artificial Intelligence, pages 90–112. Springer, 2025. [20] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, pages 4700–4708, 2017. [21] Hassan Ismail Fawaz, Benjamin Lucas, Germain Forestier, Charlotte Pelletier, Daniel F Schmidt, Jonathan Weber, Geoffrey I Webb, Lhassane Idoumghar, Pierre-Alain Muller, and François Petitjean. Inceptiontime: Finding alexnet for time series classification. Data Mining and Knowledge Discovery, 34(6):1936–1962, 2020. [22] Vidhya Kamakshi, Uday Gupta, and Narayanan C Krishnan. PACE: Posthoc ArchitectureAgnostic Concept Extractor for Explaining CNNs. In IJCNN, pages 1–8, 2021. doi: 10.1109/ IJCNN52387.2021.9534369. [23] Dave S Kerby. The simple difference formula: An approach to teaching nonparametric correlation. Comprehensive Psychology, 3:11–IT, 2014. [24] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). In ICML, pages 2668–2677. PMLR, 2018. [25] Yejin Kim and Young-Keun Kim. Time-frequency multi-domain 1d convolutional neural network with channel-spatial attention for noise-robust bearing fault diagnosis. Sensors, 23(23): 9311, 2023. [26] Dimitri Kraft and Peter Rumm. Atrial fibrillation and atrial flutter detection using deep learning. Sensors, 25(13):4109, 2025. [27] Gregory Lee, Ralf Gommers, Filip Waselewski, Kai Wohlfahrt, and Aaron O’Leary. Pywavelets: A python package for wavelet analysis. Journal of Open Source Software, 4(36):1237, 2019. [28] Christian Lessmeier, James Kuria Kimotho, Detmar Zimmer, and Walter Sextro. Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: A benchmark data set for data-driven classification. In PHM society European conference, volume 3, 2016. [29] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. [30] Anders Gjølbye Madsen, William Theodor Lehn-Schiøler, Áshildur Jónsdóttir, Bergdís Arnardóttir, and Lars Kai Hansen. Concept-based explainability for an eeg transformer model. In MLSP, pages 1–6. IEEE, 2023. [31] Helena Monke, Benjamin Sae-Chew, Benjamin Fresz, and Marco F Huber. From confusion to clarity: Protoscore-a framework for evaluating prototype-based xai. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, pages 2215–2231, 2025. [32] Max D Morris. Factorial sampling plans for preliminary computational experiments. Quality control and applied statistics, 37(6):307–310, 1992. [33] Daniel Müllner. Modern hierarchical, agglomerative clustering algorithms. arXiv preprint arXiv:1109.2378, 2011. [34] Stephen Olejnik and James Algina. Generalized eta and omega squared statistics: measures of effect size for some common research designs. Psychological methods, 8(4):434, 2003. [35] Alan V Oppenheim. Discrete-time signal processing. Pearson Education India, 1999. [36] Eleonora Poeta, Gabriele Ciravegna, Eliana Pastor, Tania Cerquitelli, and Elena Baralis. Conceptbased explainable artificial intelligence: A survey. ACM Computing Surveys, 2023. 12

[37] Andrés Felipe Posada-Moreno, Lukas Kreisköther, Tassilo Glander, and Sebastian Trimpe. Scale-preserving automatic concept extraction (SPACE). Machine Learning, 112(11):4495– 4525, 2023. doi: 10.1007/s10994-023-06373-2. [38] Andrés Felipe Posada-Moreno, Nikita Surya, and Sebastian Trimpe. ECLAD: Extracting Concepts with Local Aggregated Descriptors. Pattern Recognition, 147:110146, 2023. [39] Robert B Randall and Jerome Antoni. Rolling element bearing diagnostics—a tutorial. Mechanical systems and signal processing, 25(2):485–520, 2011. [40] Swathy Ravi and Ashalatha Radhakrishnan. A hybrid 1d cnn-bilstm model for epileptic seizure detection using multichannel eeg feature fusion. Biomedical physics & engineering express, 10 (3):035040, 2024. [41] Peter J Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53–65, 1987. [42] Andrea Saltelli, Marco Ratto, Terry Andres, Francesca Campolongo, Jessica Cariboni, Debora Gatelli, Michaela Saisana, and Stefano Tarantola. Global sensitivity analysis: the primer. John Wiley & Sons, 2008. [43] David Sculley. Web-scale k-means clustering. In Proceedings of the 19th international conference on World wide web, pages 1177–1178, 2010. [44] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825, 2017. [45] Wade A. Smith and Robert B. Randall. Rolling element bearing diagnostics using the Case Western Reserve University data: A benchmark study. Mechanical Systems and Signal Processing, 64–65:100–131, 2015. [46] Ilya M Sobol. Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates. Mathematics and computers in simulation, 55(1-3):271–280, 2001. [47] Emmanuel Stathatos, Evangelos Tzimas, Panorios Benardos, and George-Christopher Vosniakos. Convolutional neural networks for raw signal classification in cnc turning process monitoring. Sensors, 24(5):1390, 2024. [48] Johanna Vielhaben, Sebastian Lapuschkin, Grégoire Montavon, and Wojciech Samek. Explainable ai for time series via virtual inspection layers. Pattern Recognition, 150:110309, 2024. [49] Frank Wilcoxon. Individual comparisons by ranking methods. In Breakthroughs in statistics: Methodology and distribution, pages 196–202. Springer, 1992. [50] Moritz Wolter, Felix Blanke, Jochen Garcke, and Charles Tapley Hoyt. Ptwt-the pytorch wavelet toolbox. Journal of Machine Learning Research, 25(80):1–7, 2024. [51] Chih-Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. On completeness-aware concept-based explanations in deep neural networks. In NeurIPS, volume 33, pages 20554–20565, 2020. [52] Raneen Younis, Sergej Zerr, and Zahra Ahmadi. Multivariate time series analysis: An interpretable cnn-based model. In International Conference on Data Science and Advanced Analytics (DSAA), pages 1–10. IEEE, 2022.

A

Derivation of the spectral concept mask

This appendix derives the frequency-domain CENDRe formulas of Sec. 3 as the sensitivity of a real scalar functional to the log-power spectrum. The same derivation applies both to the spectral concept mask (setting E = ρ̄k ) and to the frequency-domain logit sensitivity (setting E = yc (·)). 13

Setup. Let xf = F(x) ∈ CF ×D be the rFFT of the time-domain signal x, and let E : CF ×D → R be any real-valued, differentiable scalar functional. Writing xf = u + iv with u = ℜ(xf ) and ×D v = ℑ(xf ), the power spectrum is P = |xf |2 = u2 + v 2 ∈ RF ≥0 . All operations below are element-wise over the F × D grid. Wirtinger preliminaries. Treating xf and xf as formally independent, the Wirtinger derivatives are   ∂xf E := 12 ∂u + i ∂v E. ∂xf E := 12 ∂u − i ∂v E,

For real-valued E we have ∂xf E = ∂xf E. Following the convention adopted by standard automaticdifferentiation libraries (e.g. PyTorch), we set ∇xf E := ∂xf E, so that the direction of real steepest ascent in the (u, v) parametrization is 2∇xf E. Phase-fixed power perturbation. Writing xf = reiθ with r = eiθ dxf xf = √ = , dP 2P 2 P Chain rule.

P and holding the phase θ fixed,

dxf xf = . dP 2P

The complex chain rule gives

 dE dxf dxf 1  = ∂ xf E ⊙ + ∂xf E ⊙ = ∂xf E ⊙ xf + ∂xf E ⊙ xf , dP dP dP 2P where the second equality uses ∂xf E = ∂xf E. Since the two terms in the bracket are complex conjugates of each other,  1 dE = ℜ xf ⊙ ∇xf E . dP P Log-power form. Multiplying through by P yields the sensitivity with respect to log P , which avoids the 1/P singularity at empty bins and gives a dimensionless quantity:  dE = ℜ xf ⊙ ∇ xf E . (8) d log P CENDRe spectral mask and logit sensitivity. Substituting E = ρ̄k (xf ) into (8) and keeping only the positive contributions recovers the spectral concept mask of Sec. 3,  mfreq k (x) = ReLU ℜ(xf ⊙ ∇xf ρ̄k (xf )) . Similarly, setting E = yc (x) yields the frequency-domain logit sensitivity sfreq c,ch (x) = ℜ(xf,ch ⊙ ∇xf,ch yc (x)) used in the relevance score.

B

Synthetic Dataset Details

We construct four synthetic binary-classification time-series datasets with known ground-truth concept masks (primitives), summarized in Table 1. SYNTHETIC L OCAL and SYNTHETIC F REQUENCY are reported in the main paper, while SYNTHETIC L MC and SYNTHETIC L CONF are included only here, as additional tests for per-channel attribution and confounder robustness. Table 1: The four synthetic datasets. Each is instantiated as several variants pairing primitives or sweeping a confounder ratio. Name

D

Variants

Role

SYNTHETIC L OCAL SYNTHETIC F REQUENCY SYNTHETIC L MC SYNTHETIC L CONF

1 1 2 1

9 6 6 6

Localized shapes Frequency bands Per-channel attribution Confounder robustness

Each variant contains 1,000 samples of length T = 400. A base signal is generated by superimposing low-frequency sinusoidal components with additive Gaussian noise, then class-discriminative patterns 14

are injected at random, non-overlapping positions. Binary ground-truth masks gpdom ∈ {0, 1}|dom|×D (with |time| = T for time-domain patterns and |freq| = F = ⌊T /2⌋ + 1 for frequency-domain patterns) record the exact locations of each injected pattern per channel, plus the complementary “absence” masks described below. Shared generation parameters are summarized in Table 2. Negatives as primitives. A model can learn “absence of square” or “absence of the frequency band” as a concept just as readily as their presence. We therefore treat the absence of every involved primitive as a primitive of its own, with a ground-truth mask covering the regions where the primitive is absent. This ensures that the correctness scores reward concepts that capture absence as well as presence. Table 2: Shared generation parameters for all synthetic datasets. Parameter

Value

Sample length T Samples per dataset Class balance Overlap allowed

400 1 000 50/50 No

Shape datasets (SYNTHETIC L OCAL / SYNTHETIC L MC / SYNTHETIC L CONF) Base signal Noisy sinusoidal (fs =1000 Hz) Base noise level 0.05 Base components 2 sinusoids, [0.5, 2] Hz Shape length 40 time steps Shape amplitude 1.0 Frequency datasets (SYNTHETIC F REQUENCY) Base signal Frequency-domain generator (fs =400 Hz) Base noise level 0.02 Base components 8 sinusoids, [1, 5] Hz Peaks per band 5 Peak amplitude U[1.5, 2.0] Random phase Yes

B.1

SYNTHETIC L OCAL — Localized Shape Patterns

SYNTHETIC L OCAL uses three primitive shapes of length 40:

• Square: a constant-amplitude block. • Triangle: a triangular envelope that peaks at the center and decays linearly to zero at both ends. • Circle: a semi-elliptical (half-circle) envelope, producing a smooth bump. Each shape is injected at a uniformly random position, avoiding overlap with previously placed patterns. Nine variants (D=1) span three regimes: • Paired (3 variants). Each class is characterized by a different shape. Purpose: discriminate two class-specific shapes in a clean setting. Variants: square-circle, square-triangle, triangle-circle. • Single-shape vs nothing (3 variants). One class contains a single shape, the other contains only the noisy base signal. Purpose: probe presence detection of a single primitive against a background. Variants: square-nothing, triangle-nothing, circle-nothing. • Disjunctive vs nothing (3 variants). One class contains either of two shapes (chosen at random per sample), the other contains only the noisy base signal. Purpose: test whether the method recovers a concept defined as a disjunction of primitives. Variants: squareorcircle-nothing, squareortriangle-nothing, triangleorcircle-nothing. Figure 5 shows an example from the square-circle variant. 15

Amplitude

0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00

Class 0

Class 1 square

0

50

100

150

200

Time step

250

300

350

400

circle

0

50

100

150

200

Time step

250

300

350

400

Figure 5: Example samples from the SYNTHETIC L OCAL square-circle variant. Class 0 contains a square pattern (blue region) and class 1 a circle pattern (red region), each injected at a random position into a noisy sinusoidal base signal. B.2

SYNTHETIC F REQUENCY — Frequency Band Patterns

SYNTHETIC F REQUENCY uses frequency-domain patterns drawn from three bands:

• midFreq: [70, 75] Hz • midhighFreq: [125, 130] Hz • highFreq: [170, 175] Hz Each band is realized by injecting 5 sinusoidal peaks with amplitudes drawn from U[1.5, 2.0] and random phases into the discrete Fourier transform of the base signal, then inverting back to the time domain. Ground-truth masks are defined in the frequency domain, marking the bins within each band. Six variants (D=1) span two regimes: • Pure paired (3 variants). Each class is characterized by a different band, with no other peaks. Purpose: clean spectral discrimination, the frequency analogue of the paired regime in SYNTHETIC L OCAL. Variants: midFreq-highFreq, midFreq-midhighFreq, midhighFreq-highFreq. • Paired + confounder (3 variants). Each pure-paired base is augmented with the third (unused) band injected at a 50/50 ratio, equally likely in both classes. Purpose: test whether the method ignores a spectrally prominent but uninformative band and focuses on the causal pair. Variants: midFreq-highFreq_conf-midhighFreq-50-50, midFreq-midhighFreq_conf-highFreq-50-50, midhighFreq-highFreq_conf-midFreq-50-50. Figure 6 shows an example from the midFreq-highFreq variant. We deliberately keep SYNTHETIC F REQUENCY single-channel. With discriminative frequencies present in some channels and absent in others, the cross-channel structure is dominated by the low-frequency base signal, and a CNN trained on such a setup is too likely to learn local timedomain shapes induced by these residuals rather than the discriminative bands themselves. The synthetic benchmarks are only informative when model behavior is strongly predictable from the dataset construction, a property that is already strained on simpler datasets, which the multi-channel frequency setting would defeat. B.3

SYNTHETIC L MC — Multi-Channel concept localization

SYNTHETIC L MC reuses the three shape primitives of SYNTHETIC L OCAL but signals are two-channel (D=2). Six variants in the following format exist:

• Random-channel single-shape (3 variants). One class contains a single shape injected into a uniformly chosen channel, the other contains only the noisy base signal in both channels. Purpose: test whether the method recovers a channel-agnostic concept, defined by shape rather than channel index. Variants: square-nothing_ch2_random, triangle-nothing_ch2_random, circle-nothing_ch2_random. • Specific-channel paired (3 variants). Each of two class-discriminative shapes is assigned to a designated channel, one shape per channel. Purpose: test per-channel 16

Class 0

time domain

Class 1

0.01

Amplitude

Amplitude

0.02 0.00 0.01 0.02 0.03 0

50

100

150

200

250

300

Time step Class 0 frequency domain

350

0

50

2.5

midFreq

1.5

100

150

time domain

200

250

300

Time step Class 1 frequency domain

1.0 0.5

350

400

highFreq

2.0

|FFT|

|FFT|

400

0.03 0.02 0.01 0.00 0.01 0.02 0.03

1.5 1.0 0.5

0.0

0.0 0

25

50

75

100

125

Frequency (Hz)

150

175

200

0

25

50

75

100

125

Frequency (Hz)

150

175

200

Figure 6: Example samples from the SYNTHETIC F REQUENCY midFreq-highFreq variant. Top: time-domain signals. Bottom: FFT magnitudes with ground-truth frequency bands highlighted (midFreq in blue, highFreq in red). Class 0 contains the midFreq pattern, class 1 the highFreq pattern. attribution, whether the method assigns each concept to the correct channel rather than mixing evidence across them. Variants: square-circle_ch2_specificChannels, square-triangle_ch2_specificChannels, triangle-circle_ch2_specificChannels. B.4

SYNTHETIC L CONF — Confounder Robustness

SYNTHETIC L CONF reuses the three shape primitives of SYNTHETIC L OCAL (D=1) and adds a non-discriminative confounder shape on top of each paired base, with controlled class–confounder correlation. Six variants combine two paired bases with three confounder ratios:

• Bases. square-circle with triangle as confounder, and triangle-circle with square as confounder. • Ratios. 40/60, 50/50, and 60/40, where the ratio gives the proportion of confounder presence in class 0 versus class 1. Purpose: test robustness to spurious correlations. At 50/50 the confounder carries no label information, while at 40/60 and 60/40 it does, so a method that latches onto the prominent shape rather than the causal pair is penalized more strongly at the off-balance ratios.

C

Implementation details

This appendix collects the implementation details needed to reproduce CENDRe and the baselines: backbone architectures, the probe layers used for concept extraction, and the CENDRe clustering hyperparameters. C.1

Model architectures

Table 3 lists the exact hyperparameters of the three 1D CNN architectures used throughout the experiments (§4.2). All three are direct 1D adaptations of the corresponding standard image-classification architectures, with 2D convolutions replaced by 1D convolutions and 2D pooling by 1D pooling. C.2

Probe layers

The choice of which layers to analyze constitutes the most critical hyperparameter for CE. Table 4 reports the specific layers selected for each network. In ECLAD-ts, probe layers are sampled at 17

Table 3: Architecture hyperparameters used for the three 1D CNN backbones. Defaults follow each architecture’s reference implementation. Model

Configuration

InceptionTime10 [21]

10 inception modules, 32 filters per branch, base kernel size 40, with bottleneck layer and residual connections every three modules.

ResNet1D-18 [16]

Four stages with [2, 2, 2, 2] residual blocks, expansion factor 1.

DenseNet1D-121 [20]

Block configuration [6, 12, 24, 16], growth rate 32, compression factor 0.5, 64 initial features.

uniform intervals throughout the architecture. MultiVISION instead relies on a single bottleneck layer, which prevents the analysis of blocks that residual connections might bypass. CENDRe adopts the same probe layers as ECLAD-ts. For MultiVISION, the neuron activation extraction threshold is set to the 0.99 quantile of the layer under examination, and activations are grouped via k-means clustering. Table 4: Layers used for CE with each method and model. Model

ECLAD-ts and CENDRe

Model model.inception_block.inception_layers.nb InceptionTime10 .bottleneck, nb ∈ {6, 7, 8, 9} ResNet1D-18 model.layers.nb .1.relu nb ∈ {0, 1, 2, 3} ResNet1D-18 DenseNet1D-121 model.features.transitionnb .conv, nb ∈ {1, 2, 3}; DenseNet1D-121 model.features.denseblock4.block.15.conv2 InceptionTime10

C.3

MultiVISION model.inception_block. inception_layers.9.bottleneck model.layers.2.1.relu model.features.transition3.conv

CENDRe clustering hyperparameters

The second stage of CENDResilhouette and CENDReHDBSCAN runs on the J = 50 micro-centroids returned by mini-batch k-means. CENDResilhouette uses average linkage and selects K ∈ {2, . . . , 10} by the silhouette score, both evaluated under cosine distance. CENDReHDBSCAN uses min_cluster_size = 5 under cosine distance. Micro-centroids that HDBSCAN labels as noise are excluded from J¯k when computing ρk .

D

Additional results

This appendix extends Section 5 in two directions. §D.1 broadens the synthetic evaluation: it compares all three CENDRe variants across three architectures, visualizes concepts under two further invertible transforms, breaks the quantitative comparison down per dataset with significance tests, and extends it to two dataset families that test multichannel localization and confounder robustness. §D.2 extends the qualitative evaluation to five real-world datasets from the UCR archive and to a second bearing-fault dataset, BearingPD, on two architectures. D.1

Synthetic datasets

We extend the synthetic evaluation of Section 5 along four lines. We first repeat the qualitative comparison of Figure 1 across all three CENDRe variants and three architectures. We then visualize concepts under two further invertible transforms, the short-time Fourier and the wavelet transform (§D.1.2). Next, we break the family-level quantitative comparison of Figure 2 down per dataset and test the resulting ordering for statistical significance (§D.1.3, §D.1.4). Finally, we extend the quantitative comparison to two appendix-only dataset families, SYNTHETIC L MC and SYNTHETIC L CONF, testing multichannel localization and confounder robustness. D.1.1

Qualitative comparison across CENDRe variants and architectures

Section 5 reports the silhouette-guided aggregation variant (CENDResilhouette ) as the default, extracted from a single ResNet1D-18. Figure 7 extends that view in two directions, adding the two remaining CENDRe variants and repeating the extraction on two further architectures. It shows 18

InceptionTime10

ResNet1D-18 Importance Score

0

0

y: 0

ŷ: 0

y: 1

Importance Score

ŷ: 1

0.00 0.2

0.4

0.6

0.8

1

1.0

0.0

0.2

0.4

0.6

0.8

0.05

y: 0 0.0

C0 I0 = −1.0 I1 = 1.0

0

ŷ: 0

0.2

0.4

0.6

0.0

0.2

Importance Score

ŷ: 1

0.4

0.6

0.8

0.01

0

0.0

0.00

1

1

ŷ: 0

y: 1

ŷ: 1

0.00 0.2

0.4

0.6

0.8

1

1.0

0.0

0.2

0.4

0.6

0.8

0.05

y: 0

0

ŷ: 0

y: 1

ŷ: 1

0.00

1.0

Time [s]

Importance Score

0.0

0.2

0.4

0.6

0.8

1

1.0

0.0

0.2

0.4

0.6

0.8

0

0.000

1

1

C1 I0 = 0.2

0.000

y: 0

0.0000 0.0

0.2

ŷ: 0

0.4

0.6

y: 1 0.8

1.0

0.0

0.2

ŷ: 1

0.4

0.6

0.8

1.0

0.005

y: 0

0.000

1.5

0.0

0.2

ŷ: 0

0.4

0.6

y: 1 0.8

1.0

0.0

0.2

Importance Score

ŷ: 1

0.4

0.6

0.8

1.0

0.5 0.0

1.0

C1 I = −0.2

1.5

1.0 0.5 0.0 −0.5 400

0

ŷ: 0

200

y: 1

400

ŷ: 1

200

y: 0

400

0

ŷ: 0

200

y: 1

400

ŷ: 1

0.0

C0 I = 1.0

1.0

C0 I = 1.0

1.0 0.5 0.0

1.5

1.5

1.0

1.0

1.0

−0.5

C1 I = 0.6

1.5

C1 I = 0.4

−0.5

0.0

0.5 0.0 −0.5

200

400

0.0

0.2

0.8

1.0

0.8

1.0

ŷ: 1

0.4

0.6

ŷ: 0

0.4

0.6

y: 1 0.8

1.0

0.0

0.2

ŷ: 1

0.4

0.6

200

400

0

ŷ: 0

200

y: 1

400

ŷ: 1

0.0

−0.5

0.5

1.0

0.5

−0.5

0

0.2

y: 0

1.0

400

0.0

0

Importance Score 1.5

0.5

y: 1 0.8

0.0

1.5

200

0.6

0.5

1.5

0

ŷ: 0

−0.5 0

Importance Score

1.0

0.0

1.5

200

0.8

0.5

1.0

y: 0

0.6

Time [s]

1.5

0

0.4

y: 0

0.00

−0.5

−0.5

ŷ: 1

0.4

1.0

−0.5

0.0

y: 1

0

−0.5

0.5

0.2

Time [s]

0.01

1.5

C0 I = 1.0

C0 I = 1.0

0.0

Importance Score

0.2

Time [s]

1.0

0.5

0.0

0

1

Importance Score

y: 0 0.0

1

0.00

0.0025

1.0

0.00

0.005

0

0.8

0

0.005

0

0.6

Time [s]

0.01

1.0

Time [s]

C0 I0 = −1.0 I1 = 1.0

y: 0

Importance Score

C1 I0 = −0.6 I1 = 0.6

0.00

1

0.01

ŷ: 0

0.4

0

0.00

0

0.2

1

0.02

0

y: 0

ŷ: 1

0.00

1.0

0.02

C1 I = −0.2

C0 I = −0.6 C1 I = −1.0

1.0

Time [s]

Time [s]

C0 I = 1.0

0.8

1

y: 1

0

0.02

1.0

C1 I = 0.7

y: 1

0.00

1.0

Time [s]

C0 I0 = −1.0 I1 = 1.0

0.01

C1 I0 = −0.6 I1 = 0.6

1

C1 I0 = −0.1

0.000

0

ŷ: 0

0

1

0

y: 0 1

0.000

C1 I0 = −0.9 I1 = 0.8

C0 I0 = −1.0 I1 = 1.0 C1 I0 = 1.0 I1 = −1.0

1

1

0.02

ECLAD-ts

Importance Score

ŷ: 1

0.00

1.5

MultiVISION

y: 1

0.025

0.0

Importance Score

DenseNet1D-121

ŷ: 0

0.025

C1 I0 = −0.2

C0 I0 = −1.0 I1 = 1.0 C1 I0 = 0.6 I1 = −0.6

Importance Score

y: 0

C0 I0 = −1.0 I1 = 1.0

ŷ: 1

1

0.0

CENDReHDBSCAN

y: 1

C0 I0 = −1.0 I1 = 1.0

Importance Score

CENDRekMeans

ŷ: 0

C0 I0 = −1.0 I1 = 1.0

C0 I0 = −1.0 I1 = 1.0

y: 0

0.02

C1 I0 = 0.6 I1 = −0.6

CENDResilhouette

Importance Score

0.5 0.0 −0.5

0

200

400

0

200

400

0

200

400

0

200

400

Figure 7: Concepts extracted by CENDResilhouette , CENDReHDBSCAN , CENDRekMeans , ECLADts, and MultiVISION (rows) from a ResNet1D-18, an InceptionTime10, and a DenseNet1D-121 (columns, in that order) trained on the same SYNTHETIC L OCAL square-triangle dataset as Figure 1. For space, each panel shows the two most important concepts; within a panel, rows are concepts, columns are one class-0 and one class-1 sample, and row labels report the importance scores. Across architectures, ECLAD-ts isolates the square as a solid block, CENDRe recovers the same regions as ECLAD-ts but as graded masks whose largest values partially sit at the edges, and MultiVISION shows no correspondence to either primitive. On DenseNet1D-121 the CENDRe variants extract additional low-importance concepts (not shown) that further partition the background.

the concepts extracted by the five CE methods (rows) from a ResNet1D-18, an InceptionTime10, and a DenseNet1D-121 (columns) trained on the same SYNTHETIC L OCAL square-triangle dataset as Figure 1. CENDResilhouette and CENDReHDBSCAN set the number of concepts automatically; for CENDRekMeans , ECLAD-ts, and MultiVISION we report the run whose number of concepts maximizes the RC score. For space, each panel shows the two most important concepts. The three CENDRe variants stay close to each other within every column, pointing to similar regions with comparable importance scores. Reporting CENDResilhouette alone in the main text therefore loses 19

little. Against the baselines, the comparison reproduces the one in Section 5 on all three architectures. ECLAD-ts marks the square as a uniform block and its complement as a second concept, while CENDRe spreads graded mass over similar regions, with the largest values at the edges of the shapes. MultiVISION splits the signal into broad contiguous segments whose boundaries follow neither primitive. All three architectures yield concepts that map onto the primitives, with slight differences between them. On the ResNet1D-18 and the InceptionTime10, every method separates the discriminative shapes from the background. The DenseNet1D-121 follows the same pattern, with the CENDRe variants splitting the background further into additional low-importance concepts, left out of the figure for space. D.1.2

Visualizations in other natural domains

In Section 3, we instantiate the transform ϕ as a Fourier transform to visualize the concepts in the frequency domain, but mention how other invertible transforms could be used instead. While we don’t test them quantitatively, such transforms are straightforward to implement and can be used to visualize the same concepts in other domains, e.g., wavelet transforms or short-time Fourier transforms for time-frequency analysis. We have implemented such transforms by creating virtual inspection layers for short-time Fourier transform (STFT) and discrete wavelet transform (DWT). The STFT builds on the implementation of PyTorch [1], and the DWT on the differentiable filter banks of the PyTorch Wavelet Toolbox [50], which in turn builds on PyWavelets [27]. The STFT S : RT ×D → CF ×N ×D applies the rFFT to overlapping, windowed segments of length W with hop size H, producing F := ⌊W/2⌋ + 1 frequency bins across N time frames. Because S is complex-valued like F, the sensitivity in Eq. 4 carries over unchanged after substituting xs := S(x) for xf . Its derivation only requires a complex transform output, not a specific transform (App. A). We invert S exactly by choosing an analysis window that satisfies the nonzero-overlap-add condition. The DWT W : RT ×D → RT ×D decomposes x into approximation and detail coefficients through an orthogonal wavelet filter bank. This filter bank admits an exact inverse by construction. Since W is real-valued, unlike F and S, its mask reduces to the plain ReLU gradient, following the same pattern as mtime in Eq. 4. We selected STFT and DWT because both are exactly invertible, matching k the requirement virtual inspection layers place on ϕ. Other time-frequency transforms, such as the continuous wavelet transform, lack an exact inverse and are left for future work. In Figure 8 we show qualitative results of reports in the time-frequency domain using such transforms. Across the four views, both concepts land where the ground truth places them, but the choice of ϕ decides how granular that visualization is. The FFT already isolates C0 in the midFreq band, while its C1 mask spreads over a wide range with no dominant peak. The STFT resolves C1 into the highFreq band and shows it active at isolated time frames rather than across the whole record. Comparing C0 in the FFT and STFT views shows that the lower activation magnitude of the concept corresponds with it being active in a shorter timespan. The DWT recovers the same time structure, but its highest band spans 100 to 200 Hz, so the highFreq band cannot be separated within it. D.1.3

Quantitative evaluation: summary per dataset

In Figure 2 of Section 5, each box summarizes the sRC and sIC values across all datasets of a given family. Tables 5 and 6 report the same runs resolved per dataset, as mean ± standard deviation with a 95% confidence interval over the 33 runs per cell (3 architectures × 11 seeds), with the best method per dataset in bold. √ The interval is the standard 95% Student-t confidence interval for the mean, ȳ ± t0.975, n−1 s/ n. The family-level ranking of Figure 2 holds dataset by dataset and is not an artifact of pooling. On SYNTHETIC L OCAL, ECLAD-ts attains the highest sRC on all 9 datasets, with a family mean of 0.741 [0.731, 0.751] separated from CENDRekMeans ’s 0.704 [0.695, 0.714], the next-best method. CENDResilhouette and CENDReHDBSCAN trail ECLAD-ts by 0.09 and 0.11 sRC respectively, on every dataset and with comparable spread, while MultiVISION is last on sRC everywhere. The ordering reverses in sIC, where CENDReHDBSCAN is best on all nine datasets (family mean 0.858), ahead of CENDResilhouette (0.792) and MultiVISION (0.722), with CENDRekMeans at 0.658 and ECLAD-ts at 0.556. ECLAD-ts is also the least stable method under sIC, with per-dataset standard deviations up to 0.305 against 0.140 for the two automatic CENDRe variants. Its sRC lead therefore does not 20

Importance Score

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

Importance Score

ŷ: 1

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

C0 I0 = −0.8 I1 = 1.0

C0 I0 = −0.8 I1 = 1.0

0.05

0.00

20 10 0

0.1

0.05 0.0

0.00

C1 I1 = 0.4

C1 I1 = 0.4

0.05

0.00

20 10 0

0.01 0.05 0.00

0.00 0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0

50

100

150

200

0

50

100

150

Time [s]

(a) Time domain y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

0

50

100

150

200

0

50

100

150

200

(b) Frequency domain (rFFT) ŷ: 1

y: 1

Importance Score

ŷ: 1

200

150

100

Freq [Hz] C0 I0 = −0.8 I1 = 1.0

200

100

25

0 200

0 200

150

100

100

y: 0

ŷ: 0

0.4

0.6

y: 0

ŷ: 0

0.4

0.6

y: 1

ŷ: 1

0.4

0.6

y: 1

ŷ: 1

0.4

0.6

50

50

Freq [Hz] C1 I1 = 0.4

Freq [Hz] C1 I1 = 0.4

Freq [Hz] C0 I0 = −0.8 I1 = 1.0

Importance Score

200

Frequency [Hz]

50

25

50

0

0 0.0

0.2

0.4

0.6

0.8

1.00.0

0.2

0.4

0.6

0.8

1.00.0

0.2

0.4

0.6

0.8

1.00.0

0.2

0.4

0.6

0.8

0.0

1.0

0.2

0.8

0.0

0.2

0.8

0.0

0.2

0.8

0.0

0.2

0.8

Time [s]

Time [s]

(c) Time-frequency domain (STFT)

(d) Time-frequency domain (wavelet)

Figure 8: The same two concepts, extracted with CENDResilhouette from an InceptionTime10 trained on the SYNTHETIC F REQUENCY midFreq-highFreq variant, visualized under four choices of the transform ϕ. The time-domain masks (a) carry no interpretable pattern, while all three spectral views place C0 in the ground-truth midFreq band at [70, 75] Hz. The rFFT (b) resolves that band as a sharp peak, but leaves C1 ambiguous, spreading low mask values over a wide frequency range. The STFT (c) removes the ambiguity: C1 concentrates in the highFreq band at [170, 175] Hz and is active only at isolated time frames, whereas C0 persists across the full record. The DWT (d) reproduces the same time structure at coarser frequency resolution, since its highest dyadic band spans 100 to 200 Hz and cannot separate the highFreq band inside it. Table 5: Distributional summary (mean ± std, 95% CI) of sRC across CE methods and datasets. CENDResilhouette

CENDReHDBSCAN

CENDRekMeans

ECLAD-ts

MultiVISION

mean ± std

95% CI

mean ± std

95% CI

mean ± std

95% CI

mean ± std

95% CI

mean ± std

95% CI

SYNTHETIC L OCAL circle-nothing square-circle triangle-circle square-nothing square-triangle triangle-nothing squareorcircle-nothing triangleorcircle-nothing squareortriangle-nothing

0.731 ± 0.035 0.599 ± 0.053 0.622 ± 0.073 0.710 ± 0.028 0.597 ± 0.066 0.742 ± 0.054 0.617 ± 0.054 0.623 ± 0.063 0.618 ± 0.060

[0.718, 0.743] [0.581, 0.618] [0.596, 0.648] [0.701, 0.720] [0.573, 0.620] [0.723, 0.762] [0.598, 0.637] [0.601, 0.646] [0.597, 0.640]

0.716 ± 0.030 0.561 ± 0.035 0.575 ± 0.061 0.711 ± 0.034 0.563 ± 0.049 0.729 ± 0.053 0.620 ± 0.035 0.618 ± 0.069 0.600 ± 0.073

[0.705, 0.726] [0.549, 0.574] [0.553, 0.597] [0.699, 0.724] [0.545, 0.580] [0.710, 0.747] [0.607, 0.633] [0.592, 0.643] [0.573, 0.627]

0.774 ± 0.049 0.662 ± 0.066 0.672 ± 0.087 0.757 ± 0.046 0.639 ± 0.062 0.804 ± 0.059 0.669 ± 0.064 0.677 ± 0.067 0.684 ± 0.073

[0.756, 0.791] [0.639, 0.686] [0.642, 0.703] [0.741, 0.774] [0.617, 0.661] [0.783, 0.825] [0.646, 0.691] [0.653, 0.700] [0.658, 0.710]

0.824 ± 0.060 0.723 ± 0.094 0.689 ± 0.099 0.813 ± 0.057 0.674 ± 0.087 0.838 ± 0.061 0.705 ± 0.046 0.701 ± 0.040 0.703 ± 0.040

[0.803, 0.846] [0.690, 0.756] [0.654, 0.724] [0.792, 0.833] [0.643, 0.704] [0.817, 0.860] [0.688, 0.721] [0.687, 0.715] [0.689, 0.717]

0.584 ± 0.070 0.450 ± 0.052 0.443 ± 0.057 0.600 ± 0.045 0.453 ± 0.039 0.609 ± 0.031 0.526 ± 0.042 0.525 ± 0.040 0.533 ± 0.035

[0.559, 0.609] [0.431, 0.468] [0.423, 0.463] [0.584, 0.616] [0.439, 0.467] [0.598, 0.620] [0.511, 0.541] [0.511, 0.540] [0.521, 0.546]

Mean

0.651 ± 0.078

[0.642, 0.660]

0.633 ± 0.082

[0.623, 0.642]

0.704 ± 0.084

[0.695, 0.714]

0.741 ± 0.091

[0.731, 0.751]

0.525 ± 0.077

[0.516, 0.534]

0.503 ± 0.042 0.490 ± 0.050

[0.488, 0.518] [0.472, 0.508]

0.504 ± 0.042 0.490 ± 0.049

[0.489, 0.519] [0.473, 0.508]

0.593 ± 0.058 0.554 ± 0.054

[0.573, 0.614] [0.535, 0.573]

– –

– –

– –

– –

0.532 ± 0.034 0.503 ± 0.036

[0.520, 0.544] [0.490, 0.516]

0.532 ± 0.034 0.502 ± 0.038

[0.520, 0.544] [0.489, 0.516]

0.585 ± 0.052 0.570 ± 0.047

[0.567, 0.603] [0.553, 0.586]

– –

– –

– –

– –

0.546 ± 0.037 0.488 ± 0.031

[0.533, 0.559] [0.477, 0.499]

0.546 ± 0.036 0.489 ± 0.030

[0.533, 0.559] [0.478, 0.500]

0.583 ± 0.047 0.540 ± 0.049

[0.566, 0.599] [0.523, 0.558]

– –

– –

– –

– –

0.510 ± 0.044

[0.504, 0.517]

0.511 ± 0.044

[0.504, 0.517]

0.571 ± 0.054

[0.563, 0.578]

SYNTHETIC F REQUENCY midFreq-highFreq midFreq-highFreq _conf-midhighFreq-50-50 midFreq-midhighFreq midFreq-midhighFreq _conf-highFreq-50-50 midhighFreq-highFreq midhighFreq-highFreq _conf-midFreq-50-50

Mean

Table 6: Distributional summary (mean ± std, 95% CI) of sIC across CE methods and datasets. CENDResilhouette

CENDReHDBSCAN

CENDRekMeans

ECLAD-ts

MultiVISION

mean ± std

95% CI

mean ± std

95% CI

mean ± std

95% CI

mean ± std

95% CI

mean ± std

95% CI

SYNTHETIC L OCAL circle-nothing square-circle triangle-circle square-nothing square-triangle triangle-nothing squareorcircle-nothing triangleorcircle-nothing squareortriangle-nothing

0.806 ± 0.091 0.721 ± 0.127 0.794 ± 0.123 0.799 ± 0.103 0.766 ± 0.097 0.784 ± 0.125 0.819 ± 0.098 0.812 ± 0.098 0.827 ± 0.097

[0.774, 0.839] [0.676, 0.766] [0.751, 0.838] [0.763, 0.836] [0.731, 0.800] [0.740, 0.828] [0.784, 0.854] [0.778, 0.847] [0.792, 0.861]

0.850 ± 0.100 0.816 ± 0.140 0.849 ± 0.121 0.873 ± 0.090 0.842 ± 0.111 0.873 ± 0.115 0.870 ± 0.094 0.884 ± 0.084 0.869 ± 0.095

[0.814, 0.886] [0.767, 0.866] [0.806, 0.893] [0.840, 0.905] [0.803, 0.881] [0.832, 0.914] [0.835, 0.905] [0.853, 0.915] [0.834, 0.904]

0.666 ± 0.140 0.605 ± 0.098 0.653 ± 0.145 0.668 ± 0.135 0.626 ± 0.110 0.661 ± 0.174 0.712 ± 0.128 0.676 ± 0.123 0.651 ± 0.128

[0.616, 0.716] [0.570, 0.639] [0.602, 0.705] [0.620, 0.716] [0.587, 0.665] [0.599, 0.723] [0.667, 0.758] [0.633, 0.720] [0.606, 0.697]

0.684 ± 0.283 0.550 ± 0.250 0.474 ± 0.298 0.466 ± 0.234 0.596 ± 0.228 0.599 ± 0.287 0.551 ± 0.193 0.558 ± 0.305 0.523 ± 0.217

[0.584, 0.784] [0.461, 0.638] [0.369, 0.580] [0.383, 0.549] [0.515, 0.677] [0.497, 0.701] [0.483, 0.619] [0.450, 0.666] [0.446, 0.600]

0.686 ± 0.137 0.735 ± 0.120 0.743 ± 0.135 0.711 ± 0.078 0.731 ± 0.104 0.741 ± 0.083 0.707 ± 0.100 0.739 ± 0.065 0.706 ± 0.092

[0.637, 0.734] [0.692, 0.777] [0.695, 0.791] [0.683, 0.739] [0.694, 0.768] [0.712, 0.771] [0.672, 0.743] [0.717, 0.762] [0.673, 0.738]

Mean

0.792 ± 0.110

[0.779, 0.805]

0.858 ± 0.107

[0.846, 0.871]

0.658 ± 0.134

[0.642, 0.673]

0.556 ± 0.262

[0.526, 0.586]

0.722 ± 0.105

[0.710, 0.734]

0.802 ± 0.096 0.776 ± 0.121

[0.768, 0.836] [0.733, 0.819]

0.804 ± 0.096 0.775 ± 0.123

[0.770, 0.837] [0.731, 0.818]

0.694 ± 0.127 0.690 ± 0.114

[0.649, 0.739] [0.650, 0.731]

– –

– –

– –

– –

0.820 ± 0.111 0.792 ± 0.109

[0.781, 0.860] [0.754, 0.831]

0.819 ± 0.111 0.790 ± 0.111

[0.780, 0.859] [0.751, 0.830]

0.744 ± 0.161 0.690 ± 0.108

[0.687, 0.801] [0.651, 0.728]

– –

– –

– –

– –

0.791 ± 0.103 0.770 ± 0.128

[0.754, 0.828] [0.724, 0.815]

0.789 ± 0.103 0.772 ± 0.127

[0.752, 0.825] [0.726, 0.817]

0.709 ± 0.144 0.679 ± 0.116

[0.658, 0.761] [0.638, 0.720]

– –

– –

– –

– –

0.792 ± 0.112

[0.776, 0.807]

0.791 ± 0.112

[0.776, 0.807]

0.701 ± 0.130

[0.683, 0.719]

SYNTHETIC F REQUENCY midFreq-highFreq midFreq-highFreq _conf-midhighFreq-50-50 midFreq-midhighFreq midFreq-midhighFreq _conf-highFreq-50-50 midhighFreq-highFreq midhighFreq-highFreq _conf-midFreq-50-50

Mean

transfer to sIC: broader masks that cover the ground-truth primitive, selected at the K with the best sRC, do not track the model’s per-timestep importance signal. 21

On SYNTHETIC F REQUENCY, only the CENDRe variants are applicable, since the baselines have no frequency-domain counterpart, which the tables mark as “–”. Both automatic variants lose about 0.06 sRC to CENDRekMeans and gain about 0.09 sIC, on every dataset of the family, the same trade observed on SYNTHETIC L OCAL. CENDResilhouette and CENDReHDBSCAN behave similarly on SYNTHETIC F REQUENCY (family means within 0.001 sRC and sIC of one another) though with 11 seeds their per-run scores no longer coincide everywhere, matching exactly in 42% of the (dataset, architecture, seed) combinations. Compared with SYNTHETIC L OCAL, mean sRC is lower (0.510 against 0.651 for CENDResilhouette ) while mean sIC is unchanged (0.792 in both cases). Frequencydomain concepts thus track the model’s importance signal as well as time-domain ones, while matching the ground-truth band less tightly. D.1.4

Quantitative evaluation: statistical significance

Section 5 compares the methods by reading the boxplots of Figure 2, where an ordering is visible but not tested. We test that ordering here, with no additional experiments. Each run is one combination of dataset, architecture, and seed, and every applicable method is evaluated on all of them, so the scores form matched pairs. We test each architecture separately over 11 seeds, giving 99 matched pairs per method pair on SYNTHETIC L OCAL (9 datasets × 11 seeds) and 66 on SYNTHETIC F REQUENCY (6 datasets × 11 seeds). Runs in which CENDReHDBSCAN returned no concepts are dropped from the pairs involving it. We compare each pair with a two-sided Wilcoxon signed-rank test [49], and Holm-correct the p-values over all pairs within a family, metric, and architecture [18]. We accompany each test with the matched-pairs rank-biserial correlation r [23], the share of pairs favoring the row method minus the share favoring the column method. Figure 9 reports both, with cell color giving r (red for row above column, blue for row below) and cell text the Holm-adjusted p-value. On SYNTHETIC F REQUENCY, only the three CENDRe variants enter the comparison, since the baselines have no frequency-domain counterpart. Of the 78 tests in Figure 9 (10 method pairs on SYNTHETIC L OCAL and 3 on SYNTHETIC F REQUENCY, per architecture and metric), 65 reach p < 0.01, and the sign of r follows the ordering of §D.1.3 throughout. The tests therefore confirm the reading of Figure 2. The 13 pairs that do not separate concentrate in three places, with two isolated exceptions. The first is CENDResilhouette against CENDReHDBSCAN on SYNTHETIC F REQUENCY, which separates on neither metric nor any architecture (p ≥ 0.014 across all six tests). This matches their family means in §D.1.3, which differ by less than 0.001 in both metrics. On SYNTHETIC L OCAL the two criteria separate everywhere except in sRC on the InceptionTime10 (p = 0.25). Both methods select the number of centroids automatically and visualize concepts by the same means, so similar behaviour on some datasets is expected. The second is CENDRekMeans against ECLAD-ts in sRC on SYNTHETIC L OCAL, indistinguishable on the DenseNet1D-121 (p = 0.71) and the ResNet1D-18 (p = 0.10) but separated on the InceptionTime10. The gap of 0.035 sRC between their family means is therefore carried by a single architecture. In sIC the same pair separates on all three, with ECLAD-ts behind. The two methods share the same clustering step, so their scores differ mainly through the mode of concept visualization. The visualizations converge when the CENDRe masks are dense, which can occur on individual architecture-dataset combinations. The third is the InceptionTime10 in sIC on SYNTHETIC F REQUENCY, where no pair of CENDRe variants reaches the 1% level (p between 0.011 and 0.015). The other two architectures separate CENDRekMeans from both automatic variants there, so this reflects the backbone rather than the metric. The two remaining cells both involve MultiVISION in sIC on SYNTHETIC L OCAL: against CENDResilhouette on the DenseNet1D-121 (p = 0.055) and against CENDRekMeans on the InceptionTime10 (p = 0.18). Both are consistent with MultiVISION ranking mid-field on sIC rather than last, and neither affects the ordering among the CENDRe variants or against ECLAD-ts. D.1.5

Multichannel localization and confounder robustness

We extend the quantitative evaluation of Figure 2 to two synthetic datasets: SYNTHETIC L MC (D=2, per-channel attribution) and SYNTHETIC L CONF (D=1, paired with a third shape as confounder at 40/60, 50/50, and 60/40 ratios). Figure 10 reports sRC and sIC across all variants and the three 22

syntheticlocal DenseNet1D121

CENDREACsilhouette (A)

3.82e-11*

CENDREHDBSCAN (B)

InceptionTime10

8.35e-10*

1.83e-04*

5.70e-17*

7.27e-15*

8.89e-13*

5.70e-17*

7.11e-01

5.70e-17*

2.49e-01

ResNet1D18

2.09e-14*

5.70e-17*

6.11e-17*

1.61e-12*

3.47e-15*

2.11e-14*

3.88e-14*

5.70e-17*

9.67e-12*

2.95e-16*

5.70e-17*

5.70e-17*

8.62e-17*

5.70e-17*

5.70e-17*

1.01e-01

5.70e-17*

1.00 0.75 0.50 ECLAD-ts (D)

5.70e-17*

5.70e-17*

5.70e-17*

0.25 0.00

MultiVISION (E) (A)

(B)

syntheticfrequency

(C)

(D)

(E)

(A)

(B)

DenseNet1D121

CENDREACsilhouette (A)

(C)

(D)

(E)

(A)

(B)

InceptionTime10

5.87e-01

4.92e-12*

CENDREHDBSCAN (B)

3.25e-01

2.36e-05*

4.92e-12*

(C)

(D)

(E)

ResNet1D18

−0.25 −0.50

3.82e-01

5.15e-12*

2.36e-05*

5.15e-12*

rank-biserial r (row > col)

CENDREkMeans (C)

−0.75 −1.00

CENDREkMeans (C) (A)

(B)

(C)

(A)

(B)

(C)

(A)

(B)

(C)

(a) sRC syntheticlocal DenseNet1D121

CENDREACsilhouette (A)

1.08e-11*

CENDREHDBSCAN (B)

InceptionTime10

5.52e-07*

4.17e-07*

5.53e-02

6.66e-15*

8.76e-12*

1.08e-11*

9.65e-04*

1.37e-04*

7.50e-05*

ResNet1D18

6.73e-06*

7.54e-09*

2.20e-04*

2.19e-10*

3.63e-11*

1.42e-09*

6.59e-03*

1.84e-01

1.06e-10*

1.02e-15*

1.27e-13*

1.04e-11*

2.29e-16*

5.19e-14*

5.19e-14*

3.61e-04*

9.09e-09*

1.00 0.75 0.50 ECLAD-ts (D)

9.41e-07*

6.95e-05*

2.35e-08*

0.25 0.00

MultiVISION (E) (A)

(B)

syntheticfrequency

(C)

(D)

(E)

(A)

(B)

DenseNet1D121

CENDREACsilhouette (A)

9.62e-01

CENDREHDBSCAN (B)

(C)

(D)

(E)

(A)

InceptionTime10 3.58e-11*

1.45e-02

3.58e-11*

(B)

(C)

(D)

(E)

ResNet1D18 1.11e-02

7.47e-01

1.45e-02

−0.25 −0.50

1.09e-09*

8.12e-10*

rank-biserial r (row > col)

CENDREkMeans (C)

−0.75 −1.00

CENDREkMeans (C) (A)

(B)

(C)

(A)

(B)

(C)

(A)

(B)

(C)

(b) sIC

Figure 9: Pairwise Holm-corrected Wilcoxon signed-rank tests between CE methods, per architecture, on the runs of Figure 2, for (a) sRC and (b) sIC. Within each panel, the upper block is SYNTHETIC L OCAL and the lower block SYNTHETIC F REQUENCY, and the three column groups are the DenseNet1D-121, the InceptionTime10, and the ResNet1D-18. Only the upper triangle is shown, since the test is symmetric. Cell color gives the matched-pairs rank-biserial correlation r (red for row above column, blue for row below), and cell text the Holm-adjusted p-value, with an asterisk marking p < 0.01. On SYNTHETIC F REQUENCY, only the three CENDRe variants enter the comparison, since the baselines have no frequency-domain counterpart.

architectures. These two families run over 3 seeds (0–2), rather than the 11 of the main families. The ranking is similar to the main paper. CENDResilhouette and CENDReHDBSCAN trade slightly lower sRC for noticeably higher sIC than ECLAD-ts, CENDRekMeans is similar to ECLAD-ts on both metrics, and MultiVISION underperforms on sRC. Across these settings, CENDReHDBSCAN reaches slightly higher sIC than CENDResilhouette , consistent with density-based clustering being better suited to the non-spherical structures one expects in latent spaces. We still report silhouette-guided aggregation as the default in the main text because HDBSCAN occasionally requires per-dataset tuning of its neighborhood parameters when its 23

Figure 10: Quantitative evaluation on SYNTHETIC L MC (top, per-channel attribution) and SYNTHETIC L CONF (bottom, paired with confounders). The ranking mirrors Figure 2: CENDResilhouette and CENDReHDBSCAN achieve sRC close to ECLAD-ts and higher sIC, CENDRekMeans tracks ECLAD-ts on both metrics, and MultiVISION is weak on sRC. single-pass run fails to find any dense region (see failure rates below); on more complex problems where this tuning effort is acceptable, HDBSCAN may be the stronger choice. CENDReHDBSCAN occasionally fails to converge, with HDBSCAN labeling all 50 micro-centroids as noise, on 11/297 SYNTHETIC L OCAL and 5/54 SYNTHETIC L CONF runs (all on InceptionTime10 or ResNet1D-18, concentrated on triangle-circle bases). No failures occur on SYNTHETIC F REQUENCY (0/198) or SYNTHETIC L MC (0/54). Failed runs are excluded from the boxplot statistics, so the boxplots are conditional on convergence. D.2

Natural datasets

We complement the CWRU results of Section 5 with two further natural-data evaluations. We extend the qualitative evaluation to five univariate datasets from the UCR archive, covering cardiac, automotive, motion, robotic, and semiconductor signals (§D.2.1). We then extract concepts from a second bearing-fault dataset, Paderborn Bearing (BearingPD), on two architectures, to test whether CENDRe recovers the same fault-relevant frequency bands under a different rig and sensor setup. D.2.1

UCR

We extend the qualitative evaluation to five univariate datasets of the UCR archive [11], covering cardiac, automotive, motion, robotic, and semiconductor signals. For each dataset we train the three architectures of Section 4.2 and extract concepts from one of them, using CENDResilhouette with the hyperparameters of Section 4.2. Each report shows the time-domain masks, the frequency-domain masks, and the global concept-frequency correspondence. The archive distributes most of its datasets without the sampling rate of the original recording. Where the rate is undocumented we plot the frequency axis in cycles per sample, under fs = 1, rather than assume a rate. Where it is documented we state it in the corresponding paragraph and label the time and frequency axes in seconds and Hz. The choice affects the labeling of the axes alone, since the masks are computed on the sampled series either way. ECG200 [11] traces the electrical activity recorded during a single heartbeat, on one electrocardiogram channel. The two classes are a normal heartbeat and a myocardial infarction. Series have T = 96 samples. CENDResilhouette extracts K = 4 concepts, which partition the beat into consecutive segments. C0 and C1 carry the largest and opposite-signed importance, with C0 activating on the plateau that follows the main negative deflection, mostly in the y = 1 series, and C1 on the deflection itself. C2 and C3 cover the late and the early part of the beat at lower importance. In frequency, all 24

four concepts keep their mass below 0.25 cycles per sample and peak near 0.02, so the model reads the morphology of the beat rather than high-frequency content.

C0 I0 = −1.0 I1 = 1.0

Importance Score

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 1

ŷ: 1

Channel 0

2.5 0.0

0.20 −2.5 0.005

0.15

C0

C1 I0 = −0.9 I1 = 0.9

2.5 0.0

0.10 0.05

−2.5 0.005

0.00

0.000

C2 I1 = −0.5

Time domain

0.000

2.5 0.0

0.20

−2.5

0.005

0.15

C1

C3 I0 = 0.0

0.000

2.5

0.10

0.0

0.05 −2.5

0.0025

0.00 0.0000 0

20

Importance Score

40

y: 0

60

80

0

20

ŷ: 0

40

y: 0

60

80

0

20

ŷ: 0

40

y: 0

60

80

0

20

ŷ: 0

40

y: 1

60

80

0

20

ŷ: 1

40

y: 1

60

80

0

20

ŷ: 1

40

y: 1

60

80

ŷ: 1

0.15

1000

C2

C0 I0 = −1.0 I1 = 1.0

0.20 2000

0.01

C1 I0 = −0.9 I1 = 0.9

0.10 0.05

2000

0.00 1000

0.01 0.00

2000

0.20

1000

0.15

C3

C2 I1 = −0.5

Frequency domain

0.00

0.005

0.10

C3 I0 = 0.0

0.000

0.05

2000

0.00

1000

0.0

0.002

0.1

0.2

0.3

0.4

0.5

0.000 0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Figure 11: Concept extraction from a DenseNet1D-121 trained on ECG200 (two classes: normal heartbeat, myocardial infarction), reaching 91.67% test accuracy. Top left: time-domain concept masks. Bottom left: frequency-domain concept masks. Right: global concept-frequency correspondence. The concepts split the beat into consecutive segments, and none of them reads content above 0.25 cycles per sample.

FordA [11] contains engine noise measurements from an automotive subsystem, with the task of deciding whether a given symptom is present. Series have T = 500 samples, recorded under typical operating conditions and with minimal contamination. CENDResilhouette extracts K = 2 concepts with opposite class importance. C0 concentrates on short high-amplitude bursts and C1 on the remaining oscillation, the two alternating along the window instead of splitting it into contiguous segments. Both confine their frequency mass to the band below 0.15 cycles per sample, which carries the dominant spectral peak of the signal, and both peak near 0.05. Within that band the two profiles differ in shape: C0 concentrates its mass around the 0.05 peak and decays quickly above it, while C1 spreads a broader shoulder that persists toward 0.15. GunPoint [11] tracks the horizontal position of the centroid of an actor’s right hand, extracted from video of one of two motions. In the gun class the actor draws a replica gun from a hip holster, points it at a target for about one second, and returns it. In the point class the actor performs the same motion with an extended index finger and no prop. Series have T = 150 samples from two actors. The motion was recorded at 30 Hz, so the axes are labeled in seconds and in Hz over [0, 15]. CENDResilhouette extracts K = 4 concepts, all of which localize on the transitions of the hand trajectory and none on the plateau where the hand is held on target. C0 and C1 cover the rising edge around 1–2 s and C2 the falling edge around 3–4 s in the y = 0 series, while C3 covers both edges in the y = 1 series. Every concept peaks between 0.3 and 0.6 Hz and carries no mass above 2 Hz, on the order of the one-second gesture. The extraction recovers the draw and the return as separate concepts, which is the distinction the two motions are defined by, as has been observed in literature [17]. SonyAIBORobotSurface1 [11] contains accelerometer readings from a Sony AIBO robot, taken while the robot walks on one of two surfaces. The two classes are cement and carpet. The robot 25

C0 I0 = −0.9 I1 = 1.0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 1

ŷ: 1

Channel 0

2 0 −2

0.04

0.001

0.03

C0

0.000

C1 I0 = 0.6 I1 = −0.6

Time domain

Importance Score

2

0.02

0

0.01

−2

0.00

0.0005 0.0000 0

100

400

500

0

100

ŷ: 0

200

y: 0

300

400

500

0

100

ŷ: 0

200

y: 0

300

400

500

0

100

ŷ: 0

200

y: 1

300

400

500

0

100

ŷ: 1

200

y: 1

300

400

500

0

100

ŷ: 1

200

y: 1

300

400

500

ŷ: 1

40000

0.04

20000

0.03

C1

C0 I0 = −0.9 I1 = 1.0

y: 0

300

0.005

0.02

0.000

C1 I0 = 0.6 I1 = −0.6

Frequency domain

Importance Score

200

40000

0.01

20000

0.00 0.0

0.0025

0.1

0.2

0.3

0.4

0.5

0.0000 0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Figure 12: Concept extraction from an InceptionTime10 trained on FordA (two classes: symptom present, symptom absent), reaching 92.48% test accuracy. Top left: time-domain concept masks. Bottom left: frequency-domain concept masks. Right: global concept-frequency correspondence. Both concepts peak near 0.05 cycles per sample; C0 concentrates its mass around that peak, while C1 spreads a broader shoulder across the band below 0.15. Importance Score

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 1

ŷ: 1

Channel 0

C0 I0 = 1.0 I1 = −0.8

2

0.8 0

0.6

C0

0.05 0.00

C1 I0 = 0.6

0.4 0.2

0

0.0 0.025 0.000

2

C2 I0 = 0.4

Time domain

2

0.8 0

0.6 0.02

C1

0.00

C3 I0 = 0.1

2

0.4 0.2

0

0.0

0.005 0.000 0

1

2

3

4

5

0

1

2

3

4

5

0

1

2

3

4

5

0

1

2

3

4

5

0

1

2

3

4

5

0

1

2

3

4

5

Time [s]

0.8 Importance Score

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 1

ŷ: 1

0.6

C2

C0 I0 = 1.0 I1 = −0.8

10000

5000

0.4 0.2

0.5

C1 I0 = 0.6

0.0

5000

0.2

0.8

0.0

10000

0.6 5000

C3

C2 I0 = 0.4

Frequency domain

0.0

10000

0.1

0.4 0.2

0.0

C3 I0 = 0.1

10000

0.0

5000

0 0.05 0.00 0

5

10

15

0

5

10

15

0

5

10

15

0

5

10

15

0

5

10

15

0

5

10

15

5

10

15

Frequency [Hz]

Frequency [Hz]

Figure 13: Concept extraction from an InceptionTime10 trained on GunPoint (two classes: gun draw, point), reaching 100% test accuracy. Top left: time-domain concept masks. Bottom left: frequencydomain concept masks. Right: global concept-frequency correspondence. All four concepts localize on the draw and the return of the hand, and none on the plateau in between.

carries roll, pitch, and yaw accelerometers, of which the archive distributes the x axis alone. Series have T = 70 samples. The accelerometer samples at 125 Hz, so the axes are labeled in seconds and in Hz over [0, 62.5]. CENDResilhouette extracts K = 2 concepts that separate by class rather than by band. C0 activates on the y = 1 series and C1 on the y = 0 series, each spread over the whole 0.55 s window instead of a contiguous segment. Both concentrate their mass between 5 and 25 Hz, peak near 8 Hz, and decay above 30 Hz, in the range of the step cadence of the walk. Within that band the two profiles differ in shape: C0 splits its mass into two sharp peaks near 8 and 14 Hz, while C1 pairs the 8 Hz peak with a lower but longer ridge that extends to about 30 Hz. 26

C0 I0 = −1.0 I1 = 1.0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 1

ŷ: 1

Channel 0

2 0

0.12

−2

0.10

0.0025

0.08

C0

0.0000

C1 I1 = −0.3

Time domain

Importance Score

2

0.06 0.04

0

0.02

−2 0.001

0.00

0.000 0.0

0.2

0.4

0.0

0.2

0.4

0.0

0.2

0.4

0.0

0.2

0.4

0.0

0.2

0.4

0.0

0.2

0.4

Time [s]

ŷ: 0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 1

0.12

ŷ: 1

0.10

1000

0.08

C1

C0 I0 = −1.0 I1 = 1.0

y: 0 1500

500

0.06 0.04

0.0025 0.0000

C1 I1 = −0.3

Frequency domain

Importance Score

1500

0.02

1000

0.00

500

0

0.002

20

40

60

Frequency [Hz]

0.000 0

20

40

60

0

20

40

60

0

20

40

60

0

20

40

60

0

20

40

60

0

20

40

60

Frequency [Hz]

Figure 14: Concept extraction from a DenseNet1D-121 trained on SonyAIBORobotSurface1 (two classes: cement, carpet), reaching 97.50% test accuracy. Top left: time-domain concept masks. Bottom left: frequency-domain concept masks. Right: global concept-frequency correspondence. Both concepts peak near 8 Hz within the gait band; C0 adds a second sharp peak near 14 Hz, while C1 spreads a lower ridge up to about 30 Hz.

Wafer [11] contains inline process control measurements from semiconductor fabrication, each series recorded by one sensor while one tool processes one silicon wafer. The two classes are normal and abnormal, with abnormal covering 10.7% of the training and 12.1% of the test series. Series have T = 152 samples. CENDResilhouette extracts K = 5 concepts. Four of them cover consecutive stages of the process trace, with C1 and C4 on the initial transient, C3 on the plateau, and C2 on the final ramp. The fifth, C0 , carries the largest importance magnitude with opposite sign across the two classes and spans the whole window of the y = 1 series. The frequency masks are broader than on the other four datasets: C2 , C3 , and C4 peak below 0.03 cycles per sample, while C0 and C1 cover the full band, C1 as a near-uniform comb. That comb is the signature of the step-like transients of the trace rather than of a narrowband feature, which is what a purely spectral summary of this model would misread. D.2.2

BearingPD

We complement the CWRU example in the main text with concept extraction on a second natural bearing-fault dataset. Paderborn Bearing (BearingPD) [28] contains accelerometer recordings from a motor test rig at Paderborn University, covering healthy bearings and bearings with artificially induced or naturally degraded inner and outer race damage; we use a three-class setup (healthy, inner-race fault, outer-race fault). Recordings are sampled at fs = 64 kHz on a single vibration channel and segmented into non-overlapping windows of T = 5000 samples. Training follows Section 4.2, with two additions: Inputs are standardized by the global mean and standard deviation of the training set, and Gaussian noise of standard deviation 0.01 is added to the standardized signal during training as augmentation. On BearingPD on the InceptionTime10 model (Figure 16a), the extracted concepts partly separate by class, though some concepts activate in more than one class with differing activation strengths. The frequency-domain masks and the global concept-correspondence concentrate concept mass on four distinguishable bands, with markedly different distributions across concepts: C0 spans all four, with its largest mass at 0.6 kHz and comparable mass in the 2 and 3.4 kHz bands; C1 and C2 concentrate on the low band, peaking at 1.3 kHz, and retain only weak activity at 2 and 3.4 kHz; all three carry a narrow peak near 10 kHz, and none shows a distinct peak above it. All four bands fall inside the structural-resonance region typically used for envelope analysis on this kind of bearing rig [39], and the per-concept differences within that region reflect that the model has partitioned the resonance content into class-specific sub-bands rather than treating it as a single feature. These specific bands give domain experts the information needed to assess whether the model is reasoning over fault-relevant spectral content, and the agreement with the literature regions supports that CENDRe transfers from controlled synthetic settings to real signals. 27

C0 I0 = 1.0 I1 = −1.0

Importance Score

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 1

ŷ: 1

Channel 0

5

0.15 0 0.01

0.10

C0

C1 I0 = 0.2

0.00

5

0.05

0.00

0.025

C2 I0 = −0.1

0.000

5

0.15

0 0.01

0.10

C1

0.00

C3 I0 = −0.1

Time domain

0

5

0.05 0

0.00

0.01

C4 I0 = −0.1

0.00

5

0

0.15

0.05 0.00 50

100

y: 0

ŷ: 0

150

0

50

100

y: 0

ŷ: 0

150

0

50

100

y: 0

ŷ: 0

150

0

50

100

y: 1

ŷ: 1

150

0

50

100

y: 1

ŷ: 1

150

0

50

100

y: 1

ŷ: 1

0.10

150

C2

0

Importance Score

0.05

C0 I0 = 1.0 I1 = −1.0

10000

0.00

5000

0.005 0.000

0.10

C3

C1 I0 = 0.2

0.15

5000

0.025 0.000

0.05

5000

0.00

C2 I0 = −0.1

10000

0.05 0.00

10000

0.15 5000

0.10

C4

C3 I0 = −0.1

Frequency domain

10000

0.025 0.000

0.05

C4 I0 = −0.1

10000

5000

0.00 0.0

0.1

0.2

0.3

0.4

0.5

0.005 0.000 0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Figure 15: Concept extraction from a ResNet1D-18 trained on Wafer (two classes: normal, abnormal), reaching 100% test accuracy. Top left: time-domain concept masks. Bottom left: frequency-domain concept masks. Right: global concept-frequency correspondence. Four concepts follow the stages of the process trace, while C0 spans the whole window and carries the strongest class-discriminative importance.

On BearingPD on the DenseNet1D-121 model (Figure 16b), CENDRe recovers the same four bands, distributed differently across the concepts. The concepts again separate only partly by class, with some activating in more than one class at differing strengths. C0 places its largest mass at 3.4 kHz, with the low pair close behind and a clear peak near 10 kHz; C1 is dominated by a sharp peak at 1.3 kHz, roughly twice the height of any peak in the other two concepts; C2 carries the 0.6 and 1.3 kHz peaks at comparable height, with weaker mass at 3.4 and 10 kHz. Both architectures resolve the low band into the same 0.6 and 1.3 kHz pair, and in both, one concept is dominated by a single member of the pair while the others carry both at comparable height, though the concept indices playing each role differ. Just as for InceptionTime10, these bands give domain experts the information needed to assess whether the model is reasoning over fault-relevant spectral content. Recovering them under a second architecture supports that CENDRe transfers from controlled synthetic settings to real signals independently of the backbone.

E

Hyperparameter sensitivity

The CENDResilhouette/HDBSCAN variants replace the number of concepts K with the number of micro-clusters J. We investigate the sensitivity of CENDRe to J by sweeping J with every other hyperparameter fixed at its main-experiment value. The grid is bounded by the method itself: the upper bound is the number of LADs in one mini-batch (Jmax = 3200 in our configuration), since k-means cannot form more clusters than it sees in one partial fit, and the lower bound Jmin = 11 follows from the agglomerative candidate search over up to 10 concepts. Within [Jmin , Jmax ] we sample 10 log-spaced values, snapped to include the main-experiment value J = 50, giving 28

Importance Score

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 2

ŷ: 2

y: 2

ŷ: 2

Channel 0

C0 I0 = −0.8 I1 = −0.5 I2 = 1.0

5 0

0.0125 0.0100

C0

0.0000

C1 I0 = 0.6

5 0

0.0075 0.0050 0.0025

0.002

0.0000

0.000

5

C2 I1 = 0.4

Time domain

0.0025

0

0.0125 0.0100

0.001 0.00

0.02

0.04

0.06

0.08 0.00

0.02

0.04

0.06

0.08 0.00

0.02

0.04

0.06

0.08 0.00

0.02

0.04

0.06

0.08 0.00

0.02

0.04

0.06

0.08 0.00

0.02

0.04

0.06

0.08

Time [s]

Importance Score

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 2

ŷ: 2

y: 2

0.0075 0.0050 0.0025

ŷ: 2

C0 I0 = −0.8 I1 = −0.5 I2 = 1.0

0.0000 200000

0 0.005

0.0125

0.000

C1 I0 = 0.6

0.0100

C2

200000

0

0.0075 0.0050

0.01

0.0025

0.00

0.0000

C2 I1 = 0.4

Frequency domain

y: 0

C1

0.000

200000

0

0

10000

20000

30000

Frequency [Hz]

0.01 0.00 0

10000

20000

30000

0

10000

20000

30000

0

10000

20000

30000

0

10000

20000

30000

0

10000

20000

30000

0

10000

20000

30000

Frequency [Hz]

(a) InceptionTime10, where C0 carries the 2 and 3.4 kHz bands alongside the low pair, while C1 and C2 concentrate on the low pair. Importance Score

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 2

ŷ: 2

y: 2

ŷ: 2

Channel 0

C0 I0 = −0.7 I2 = 1.0

0.010 0

0.008 0.0005

0.006

C0

0.0000

C1 I0 = −0.6 I1 = 0.8

5 0

0.000

0.0000

5

0.010

0

0.008

0.0005

0.0000 0.02

0.04

0.06

0.08 0.00

0.02

0.04

0.06

0.08 0.00

0.02

0.04

0.06

0.08 0.00

0.02

0.04

0.06

0.08 0.00

0.02

0.04

0.06

0.08 0.00

0.02

0.04

0.06

0.08

Time [s]

C0 I0 = −0.7 I2 = 1.0

Importance Score

y: 0

ŷ: 0

y: 0

ŷ: 0

y: 1

ŷ: 1

y: 1

ŷ: 1

y: 2

ŷ: 2

y: 2

0.006

C1

0.00

0.000 200000

0

0.010

0.0000

0.008 0.006

200000

C2

C1 I0 = −0.6 I1 = 0.8

0.004 0.002

ŷ: 2

0.0025

0

0.004

0.001

0.002

0.000

C2 I0 = 0.6

Frequency domain

0.004 0.002

0.0005

C2 I0 = 0.6

Time domain

y: 0 5

0.000 200000

0 0

10000

20000

30000

Frequency [Hz]

0.0025 0.0000 0

10000

20000

30000

0

10000

20000

30000

0

10000

20000

30000

0

10000

20000

30000

0

10000

20000

30000

0

10000

20000

30000

Frequency [Hz]

(b) DenseNet1D-121, where C0 peaks at 3.4 kHz, C1 is dominated by a sharp 1.3 kHz peak, and C2 carries the 0.6–1.3 kHz pair at comparable height.

Figure 16: Concept extraction on BearingPD (three classes: healthy, inner-race fault, outer-race fault) for two architectures, each trained with seed 0 to 100% validation accuracy. Top left of each panel: time-domain concept masks, which localize between high-amplitude transients and pick up patterns that align with the periodic impact structure of the signal. Bottom left: frequency-domain concept masks, whose mass concentrates on four bands at roughly 0.6–1.3, 2, 3.4, and 10 kHz. Right: global concept-frequency correspondence per channel, where both architectures show a peak near 10 kHz. The two architectures recover the same four bands, all inside the structural-resonance region commonly used for envelope-based bearing diagnostics [39], and differ only in how the bands distribute across concepts.

29

sRC

circle-nothing ch1 specificChannels

midFreq-highFreq ch1 specificChannels

midFreq-highFreq ch1 specificChannels conf-midhighFreq-50-50

0.7

0.7

0.7

0.7

0.6

0.6

0.6

0.6

0.5

0.5

0.5

0.5

101

sIC

squareorcircle-nothing ch1 specificChannels

102

103

101

102

103

101

102

103

101

1.0

1.0

1.0

1.0

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.2

0.2 101

102

103

0.2 101

102

103

102

103

CE Method ACsilhouette HDBSCAN

0.2 101

102

103

40

40

40

30

30

30

30

20

20

20

20

10

10

10

10

0

0

0

102

103

101

102 n clusters

103

K

40

101

101

102 n clusters

103

101

102 n clusters

103

0 101

102 n clusters

103

Figure 17: Response of sRC (top row) and sIC (bottom row) to the number of micro-clusters J, per dataset, for CENDResilhouette and CENDReHDBSCAN . Markers are the mean over 11 seeds and shading is one standard deviation, with all panels on a common y-range. The dashed vertical line is the value J = 50 used in the main experiments. J ∈ {11, 21, 50, 73, 137, 257, 483, 907, 1704, 3200}. The sweep covers two datasets from each of the main synthetic families and the 11 seeds of the main experiments, reused at every level of J so that seed acts as a block rather than an independent replicate. Runs in which the extractor returned no concepts are excluded, which affects CENDReHDBSCAN at small J only. Figure 17 shows the seed mean and standard deviation of sRC and sIC per dataset. CENDResilhouette is flat over three orders of magnitude of J on all four datasets, except at the two smallest values, where too few micro-clusters remain for the candidate search. CENDReHDBSCAN matches it on SYNTHETIC F REQUENCY , but on SYNTHETIC L OCAL its sIC collapses above a few hundred microclusters, with a widening seed spread. We quantify the sensitivity with two standard measures, both taken under a log-uniform distribution of J on [Jmin , Jmax ], which the log-spaced grid samples uniformly [4]. Writing the score as y(J, ε), where ε collects everything the seed controls, the first-order Sobol index [46, 42] S1J =

VarJ (Eε [y | J]) Var(y)

(9)

is the share of the total variance explained by the choice of J, and S1seed exchanges the roles of J and ε. We estimate Equation 9 from the paired design as a two-way variance decomposition over levels of J and seeds, in the bias-corrected ω 2 form [34], weighting the levels by their log spacing. A variance share carries no direction, so we add the elementary effects of Morris [32]. Taking the factor as log2 J, which the geometric grid samples at near-uniform steps, the effect of the step from Jj to Jj+1 under seed s is ys,j+1 − ys,j EEs,j = . (10) log2 Jj+1 − log2 Jj Their mean µEE reads as the average change in the score per doubling of J, in the units of that score. We report the signed mean rather than the absolute-value variant µ∗ [7], since with a single swept factor the sign carries the direction of the drift. Tables 7a, 7b, and 7c report the three measures. For CENDResilhouette , J explains on average 0.022 of the variance in sRC, 0.072 in sIC, and 0.010 in the selected K itself, against 0.702, 0.577, and 0.159 for the seed. The mean elementary effect on K is a negligible +0.02 concepts per doubling of J, consistent with the near-zero effect already observed on sRC and sIC. Choosing J is therefore a smaller decision than the run-to-run variation the method already tolerates, for the score and for K alike. CENDReHDBSCAN is more sensitive on average in sRC and sIC (S1J = 0.144 and 0.320). 30

Its selected K tracks J far more directly, with S1J = 0.416 on average and a mean elementary effect reaching +3.67 concepts per doubling of J on CIRCLE - NOTHING. This uncontrolled growth in K plausibly explains the sIC collapse noted above. Sensitivity to J, in the score and in the selected K, is therefore a property of the density-based cluster mode rather than of CENDRe itself. The one exception is sIC on squareorcircle-nothing, where the silhouette variant also reaches S1J = 0.23. Table 7: Sensitivity to the number of micro-clusters J, for sRC, sIC, and K. The columns S1J and S1seed are the first-order Sobol indices of Equation 9 for J and for the random seed, and µEE is the mean elementary effect of Equation 10, in units of the reported score per doubling of J. The final row of each panel averages each index over datasets. (a) Sensitivity of sRC to J. CENDResilhouette

CENDReHDBSCAN

Dataset

S1J

S1seed

µEE

S1J

S1seed

µEE

circle-nothing squareorcircle-nothing midFreq-highFreq midFreq-highFreq _conf-midhighFreq-50-50

0.009 0.008 0.069 0.000

0.666 0.696 0.591 0.854

-0.0021 +0.0005 +0.0009 +0.0003

0.081 0.448 0.036 0.012

0.383 0.045 0.746 0.910

+0.0078 +0.0174 +0.0032 +0.0022

Mean

0.022

0.702

-0.0001

0.144

0.521

+0.0076

(b) Sensitivity of sIC to J. CENDResilhouette

CENDReHDBSCAN

Dataset

S1J

S1seed

µEE

S1J

S1seed

µEE

circle-nothing squareorcircle-nothing midFreq-highFreq midFreq-highFreq _conf-midhighFreq-50-50

0.000 0.233 0.056 0.000

0.633 0.479 0.290 0.907

-0.0029 +0.0118 +0.0003 -0.0016

0.599 0.640 0.041 0.000

0.047 0.094 0.207 0.729

-0.1073 -0.0932 +0.0066 +0.0029

Mean

0.072

0.577

+0.0019

0.320

0.269

-0.0478

(c) Sensitivity of K to J. CENDResilhouette

CENDReHDBSCAN

Dataset

S1J

S1seed

µEE

S1J

S1seed

µEE

circle-nothing squareorcircle-nothing midFreq-highFreq midFreq-highFreq _conf-midhighFreq-50-50

0.041 0.000 0.000 0.000

0.635 0.000 0.000 0.000

+0.0880 -0.0108 +0.0000 +0.0000

0.589 0.470 0.356 0.248

0.041 0.001 0.026 0.010

+3.6664 +2.7356 +0.1462 +0.1250

Mean

0.010

0.159

+0.0193

0.416

0.020

+1.6683

F

Computational resources

The experiments span two systems, an institutional HPC cluster and a local workstation. Each HPC GPU node provides two Intel Xeon CPUs (24 physical cores each, 192 GB RAM), four NVIDIA A100 (40 GB) GPUs, and an InfiniBand interconnect. The workstation has dual Intel Xeon Gold 6330 CPUs (2×28 cores, 112 logical threads at 2.00 GHz, 1 TiB RAM) and two NVIDIA A100-PCIE40GB GPUs.Model training ran entirely on the cluster. The CE and synthetic-metric experiments ran on both systems, so their runtimes are reported as wall-clock measurements taken on the workstation under controlled conditions and aggregated over the full sweep. F.1

Model training

Model training was performed on 3 architectures over 27 dataset configurations, with 11 seeds on the 15 SYNTHETIC L OCAL and SYNTHETIC F REQUENCY datasets and 3 seeds on the 12 SYNTHETIC L MC and SYNTHETIC L CONF datasets, totaling 603 trained models. Training consumed a total of 40.64 GPU hours. 31

F.2

Concept extraction and synthetic metrics

Runtimes below are wall-clock measurements collected on the workstation from controlled batches on a single partition with 25 allocated CPUs and no concurrent GPU users. Both CE and synthetic-metric runtimes are reported per (architecture, method) as the mean over 3 seeds on the SYNTHETIC L OCAL square-triangle variant, summed over the K-sweep used in the comparison. For the AC-clustered CENDRe variants (silhouette, HDBSCAN), K is determined in a single pass, so a single run replaces the sweep, and the tables reflect this (they are 6–9× faster than the fixed-K baselines). The experiments on ECLAD-ts and MultiVISION were done on the SYNTHETIC L OCAL family (9 datasets, 11 seeds) and the SYNTHETIC L MC and SYNTHETIC L CONF families (12 datasets, 3 seeds), which correspond to 9 × 11 + 12 × 3 = 135 runs per architecture. The CENDRe variants additionally run on the SYNTHETIC F REQUENCY family (6 datasets, 11 seeds), making for a total of 135 + 66 = 201 runs per architecture.

Table 8: Mean CE runtime per (architecture, method), in seconds, over 3 seeds on a representative SYNTHETIC L OCAL variant. Fixed-K methods (ECLAD-ts, MultiVISION, CENDRekMeans ) sum over K ∈ {2, 4, 6, 8, 10}; AC variants run once. Architecture

InceptionTime10 ResNet1D-18 DenseNet1D-121

ECLAD-ts

MultiVISION

CENDRekMeans

CENDResilhouette

CENDReHDBSCAN

102 88 251

120 103 321

118 91 227

18 11 40

15 11 35

Concept extraction. Aggregated over the full sweep, CE consumed 68.54 GPU hours. A further 29.97 GPU hours were spent on exploratory and repeated runs during development, an amount that does not grow with the seed count. Synthetic metrics. The representation-correctness (RC) and importance-correctness (IC) metrics are computed jointly in a single pass, so the table reports one runtime per (architecture, method) covering both metrics together. Aggregated over the full sweep, synthetic-metric computation conTable 9: Mean synthetic-metric runtime (RC and IC computed jointly) per (architecture, method), in seconds, over 3 seeds on a representative SYNTHETIC L OCAL variant. Fixed-K methods sum over K ∈ {2, 4, 6, 8, 10}; AC variants run once. Architecture

InceptionTime10 ResNet1D-18 DenseNet1D-121

ECLAD-ts

MultiVISION

CENDRekMeans

CENDResilhouette

CENDReHDBSCAN

108 107 111

74 55 72

333 308 464

42 23 77

29 23 50

sumed 95.08 GPU hours. As for CE, an additional 39.58 GPU hours were spent during development, independent of the seed count. In total, across model training, CE and synthetic-metric experiments, the computational resources used to produce this paper amount to approximately 274 GPU hours.

32

Record · ID 422235 · SHA-256 1e3065aeb52f38e0
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.